Entries (RSS)  |  Comments (RSS)

Memcached and MySQL – What good is it?

I posted this in response to a post on GigaOM, but it was such a long comment, I felt that it was worthy as a post on it’s own.

The workloads of social networking sites fall mostly into the ‘read lots, write once’ class (most of the web exists within this paradigm.) Regardless of the database company that’s responsible for the software, the main idea in scaling this read heavy workload is to remove the burden from the database and move it to distributed memory stores.

As an engineer, you want applications to pull from the same cache pool to reduce I/O pressure. To ensure that every machine isn’t replicating data in individual caches, you have to go distributed. That’s the win with memcached.

Putting a distributed cache between the application and the database increases performance and shares data across your application servers, something that the database cannot do on it’s own. The database has on-disk and in memory caching, but eventually you’ll run out of memory on a single host if your working set exceeds the host’s memory.

Memcached also covers up replication lag (MySQL is terrible at replication, Oracle not so much) in large environments by putting data into the distributed cache (Write-through caching) before the slave database has finished it’s writing. Data is available immediately to clients, before the replication has completed.

It will also provide a large amount of savings when you’re constantly executing that O(n x m) query to find out who is friends with whom on your social networking site.

This comes with a cost, though. Relational database functions, like joining across large data sets, and atomic operations, become very difficult to execute. Memcached becomes the central server, and there is always a fear that an important key will drop out of cache because of a random eviction.

It’s not without risk, either. Dependence on the cache can hurt you severely if lots of memcached servers fail (and they do fail), Leaving you in a ‘cold cache’ situation where it can take hours to repopulate your working set back into the cache pool.

Don’t question MySQL’s performance — relational databases are great, but they are not the only solution to storage problems. the two problems that are being solved here are, highly orthogonal.

I’d also like to state that the majority of alternate key-value store databases listed in Richard Jones’ article and in Lenoard Lin’s blog are really not ready for high production loads (with maybe the exception of Tokyo Cabinet, HDFS, and Cassandra). There is still a ton of ‘secret sauce’ the large sites are keeping quiet about in order to make these into effective data stores.

Lin states this in his review as well: “Your comfort-level running in prod may vary, but for most sane people, I doubt you’d want to.”

Tread lightly.

This entry was posted on Sunday, May 17th, 2009 at 12:56 pm and is filed under peformance, social networking, software engineering, systems administration. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

blog comments powered by Disqus