The encyclopedia · Software & IT · Technical decision · 2003–2004
LiveJournal built memcached in 2003 and turned spare RAM into a site-wide cache
Strapped LiveJournal pooled spare RAM across its web farm into one distributed cache, so MySQL stopped taking every read and the site got blazing fast.
LiveJournal · Danga Interactive
The solution
By 2003 LiveJournal's blog network ran on about 70 machines, and every page view hammered its databases. Brad Fitzpatrick found his web nodes were 'unconditionally hitting our databases', and database caches of 4–8 GB were the ceiling — the conventional fix was buying 64-bit database servers with more memory.
He chose the opposite: cache at the web tier. mod_perl processes could not share memory, and System V shared memory worked only on one machine, so he built a 'global hash table' spanning all servers — memcached. Each daemon runs wherever spare RAM exists, and a consistent hash sends each key to the same node, so the whole farm behaves like one cache.
The result was immediate: memcached 'takes up no CPU and makes the site blazing fast'. LiveJournal went live with it in late May 2003, and the design — avoid disks, scale out with many little machines rather than up — spread to Slashdot and Wikipedia before becoming a standard web building block.
Why it worked
- Reads hit RAM on web nodes, not the database disk
- Consistent key-to-node mapping keeps one cache image across machines
- No replication or persistence keeps the cache simple and fast
- Cheap spare memory beat buying bigger database servers
What can be applied
When reads overwhelm a database, scale the cache, not the database: pool idle memory across machines you already own instead of buying a bigger server.
Aftermath
Memcached became one of the most widely used caching systems on the web, adopted by Wikipedia, Facebook, Twitter and thousands of other sites; Facebook later published how it scaled the same design to tens of thousands of servers. The project remains open source under Brad Fitzpatrick's original architecture.
Sources
- Distributed Caching with Memcached
- memcached takes up no CPU and makes the site blazing fast (brad's life)
spotted an error? The archive wants to know.