Let's say you have a large database on a busy server, and queries that are eating into your resources. Would you like to be able to speed things up?
It usually turns out that there are certain queries which are very frequent and which return the same results each time (or for a period of time) which account for most of your traffic - although over time new queries will join the "popular" club and others will fade into the "was once popular" category. If you run a forum, or a blog, or an image resource site like we do, you'll probably recognise the metric.
Memcached provides a daemon which you can run on your server which grabs a specified amount of memory into which common query results (or other information) can be stored for re-use. It uses a hashing technique for very fast lookup, and it keeps tabs on the time / date that each item was last accessed so that it can drop the least popular items once it runs out of memory. It's really very clever because:
• Only small code changes are needed to your application, around the slow queries.
• It is self-adjusting in that it will identify the data it ought keep in memory and that will change as query popularity changes
• It works across multiple cpus, (i.e. you can have a network of servers) so that you make every bigger savings in a load balanced environment.
I've installed memcached on our server, and started the daemon. I've downloaded the PHP (Pecl) drivers and installed those too and put together a demonstration om memcached. You can run it
here and view the source code
here. I would (of course) be delighted to go into this in much more detail - and do some practical exercises - on our
PHP techniques workshop.
Installing Memcached, and the PHP drivers
To quote a website I found this morning when doing the installation ...
"Installation instructions are fairly straightforward which is why I'm not going to discuss those here. Once you've installed the extension you need to restart your Apache web server. You can check whether the extension is running by using phpinfo(). If all is ok you'll see a section about memcache.".
Hmm - it might have been easy for that author, but I didn't feel that it would be trivial for our customer base, so here are some pointers at least - a summary of what I did.
Memcached Installation
1. Pre-requisite - libevent.
from http://www.monkey.org/~provos/libevent/
a) Download and untar the file
b) cd into the directory created
c) ./configure
d) make
as root:
e) make install
2. Memcached itself
from http://www.danga.com/memcached/download.bml
a) Download and untar the file
b) cd into the directory created
c) ./configure --prefix=/usr/local/memcached
(I chose to make up a separate directory in case I want to (re)move later)
d) make
as root:
e) make install
PHP drivers for memcached
from http://pecl.php.net/package/memcache
a) Download and untar the file
b) cd into the directory created
c) phpize && ./configure --enable-memcache && make
as root
d) cp modules/memcache.so /usr/local/lib/php
e) vi /usr/local/lib/php.ini
change
; extension_dir = "./"
extension_dir = "/usr/local/lib/php"
add
extension=memcache.so
f) stop and restart the web server
g) Check that the memcache extension loaded (via a phpinfo page)
Starting the memcached daemon
As a regular user with minimal privileges, I ran
/usr/local/memcached/bin/memcached -d -l 127.0.0.1 -m 32 -p 11211
-d - start as a daemon
-l - listen on 127.0.0.1 (loopback, for local use)
-m - allow 32 Mbytes of memory
-p - port number on which to run
(11211 is the default anyway)
I got the following error:
/usr/local/memcached/bin/memcached: error while loading shared libraries: libevent-1.4.so.2: cannot open shared object file: No such file or directory
which I corrected as root as follows:
cd /usr/lib
ln -s /usr/local/lib/libevent-1.4.so.2 libevent-1.4.so.2
Starting on reboot too!
I have added the following onto the end of /etc/rc.local
/usr/local/memcached/bin/memcached -d -l 127.0.0.1 -m 32 -p 11211 -u wellho
(The scripts runs as root - the -u option switches in to user wellho!)
This technical note introduces a huge subject ... drivers are available for other languages too, a single cache can be used for multiple applications, caching can be done cross-server, records can be updated (and you need to take care with records that may be updated in the database to avoid serving expired stuff). This is an excellent way - if you have a computer with a lot of memory but a cpu that's running flat out - to trade in memory to regain cpu cycles, but beware on 32 bit systems of going over 2Gbytes (if you do, run multiple memcached instances!)
Note too - my server is behind a firewall, so remote access to port 11211 should not be possible!
(written 2008-08-02)
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
H307 - PHP - Web2 and caching [1633] Changing a screen saver from a web page (PHP, Perl, OSX) - (2008-05-06)
[1647] Exchange Rates - PHP with your prices in your users currency - (2008-05-19)
[1812] Starting Ajax - easy example of browser calling up server data - (2008-09-27)
[1813] Ajax - going Asyncronous and what it means - (2008-09-28)
[1814] Javascript/HTML example, dynamic server monitor - (2008-09-28)
[1926] Flash (client) to PHP (server) - example - (2008-12-06)
[1995] Automated server heartbeat and health check - (2009-01-16)
[2196] New Example - cacheing results in PHP for faster loading - (2009-05-24)
[2321] Uploading and Downloading files - changing names (Perl and PHP) - (2009-08-04)
[2545] Scraping content for your own page via PHP - (2009-12-21)
[3029] PHP data sources - other web servers, large data flows, and the client (browser) - (2010-11-04)
[3094] Setting your user_agent in PHP - telling back servers who you are - (2010-12-18)
[3186] How to add a customised twitter feed to your site - (2011-02-27)
[3458] On this day ... one PHP script with three uses - (2011-09-26)
[3955] Building up from a small PHP setup to an enterprise one - (2012-12-16)
[3999] Handling failures / absences of your backend server nicely - (2013-02-08)
[4055] Using web services to access you data - JSON and RESTful services - (2013-03-29)
[4075] Further recent PHP examples - (2013-04-28)
[4106] Web server efficiency - saving repetition through caches - (2013-05-30)
[4136] How do I post automatically from a PHP script to my Twitter account? - (2013-07-10)
[4627] Caching results in an object for efficiency - avoiding re-calculation - (2016-01-20)
A164 - Web Application Deployment - Services and Regular Jobs [544] Repeating tasks with crontab - (2005-12-27)
[907] Browser -> httpd -> Tomcat -> MySQL. Restarting. - (2006-10-28)
[1028] Linux / Unix - process priority and nice - (2007-01-10)
[1288] Linux run states, shell special commands, and directory structures - (2007-08-03)
[1553] Automatic startup and shutdown of Tomcat - (2008-02-24)
[1700] FTP server on Fedora Linux - (2008-07-06)
[1731] Apache httpd, MySQL, PHP - installation procedure - (2008-08-01)
[1765] Dialects of English and Unix - (2008-08-21)
[1903] daemons - what is running on my Linux server? - (2008-11-23)
[2145] Using the internet to remotely check for power failure at home (PHP) - (2009-04-29)
[2182] What Linux run level am I in? - (2009-05-15)
[3011] What are .pid files? - (2010-10-23)
[3143] On time - (2011-01-23)
[3791] The Kernel, Shells and Daemons. Greek Gods in computing - (2012-07-01)
[3792] Managing daemons from a terminal session - (2012-07-01)
[4487] Starting MySQL. ERROR! The server quit without updating PID file - how we fixed it. - (2015-05-06)
Some other Articles
Rules, suggestions, considerations for Lua variable namesCurrent visitors from around the world - PHPFinding words and work boundaries (MySQL, Perl, PHP)All around the world?memcached - overview, installation, example of use in PHPOld pictures and comparisonsPunting on the CamBack from the futureA short Perl example