http://engineering.twitter.com/2010/03/unicorn-power.html

The workloads of social networking sites fall mostly into the ‘read lots, write once’ class (most of the web exists within this paradigm.) Regardless of the database company that’s responsible for the software, the main idea in scaling this read heavy workload is to remove the burden from the database and move it to distributed memory stores.

As an engineer, you want applications to pull from the same cache pool to reduce I/O pressure. To ensure that every machine isn’t replicating data in individual caches, you have to go distributed. That’s the win with memcached.

Putting a distributed cache between the application and the database increases performance and shares data across your application servers, something that the database cannot do on it’s own. The database has on-disk and in memory caching, but eventually you’ll run out of memory on a single host if your working set exceeds the host’s memory.

Memcached also covers up replication lag (MySQL is terrible at replication, Oracle not so much) in large environments by putting data into the distributed cache (Write-through caching) before the slave database has finished it’s writing. Data is available immediately to clients, before the replication has completed.

It will also provide a large amount of savings when you’re constantly executing that O(n x m) query to find out who is friends with whom on your social networking site.

This comes with a cost, though. Relational database functions, like joining across large data sets, and atomic operations, become very difficult to execute. Memcached becomes the central server, and there is always a fear that an important key will drop out of cache because of a random eviction.

It’s not without risk, either. Dependence on the cache can hurt you severely if lots of memcached servers fail (and they do fail), Leaving you in a ‘cold cache’ situation where it can take hours to repopulate your working set back into the cache pool.

Don’t question MySQL’s performance — relational databases are great, but they are not the only solution to storage problems. the two problems that are being solved here are, highly orthogonal.

I’d also like to state that the majority of alternate key-value store databases listed in Richard Jones’ article and in Lenoard Lin’s blog are really not ready for high production loads (with maybe the exception of Tokyo Cabinet, HDFS, and Cassandra). There is still a ton of ’secret sauce’ the large sites are keeping quiet about in order to make these into effective data stores.

Lin states this in his review as well: “Your comfort-level running in prod may vary, but for most sane people, I doubt you’d want to.”

Tread lightly.

For many years I’ve had a need for a module like this — A distributed blocking system which could operate across large web serving clusters and register hits in a central store. With rate limiting, incrementing counters on a single host is fairly useless when you have hundreds of servers behind a load balancer.

An attacker could hit many machines within the limit period before being detected, because there would be no central count. By keeping the counts in a memcache pool, all servers share the same data.

It won’t defend against attacks coming from random proxy addresses (say, Tor), and might unfairly count hundreds of users who live behind a single proxy (like corporate NAT), but it offers some protection against attacks coming from a single source IP.

The software is released under the Apache 2.0 Open Source License.

From the docs:

mod_memcache_block is an Apache module that allows you to block access to your servers using a block list stored in memcache. It also offers distributed rate limiting based on HTTP response code.

FEATURES

Distributed White and Black listing of IPs, ranges, and CIDR blocks
Configurable timeouts, memcache server listings
Support for continuous hasing using libmemcached’s Ketama
Windowded Rate limiting based on Response code (to block brute-force dictionary attacks against .htpasswd, for example)

REQUIREMENTS

libmemcached-0.25 or better
Memcached server
Apache 2.x (tested with 2.2.11)

Source code is available here:
http://github.com/netik/mod_memcache_block

If you would like to work on mod_memcache_block, contact me with your GitHub username and I’ll give you commit access on github.

Normally I don’t re-post Twitter articles here but this one on the New York magazine was wistful, fair, balanced, and gave a good representation of what it’s like to work here.

The reporter was in the office on the very day the US Airways flight crashed into the Hudson, and he recorded our (completely boring) reactions to the event.

Sure, the Twitter guys still have no idea how to make money off their fabulous invention. But for now they are living in a dreamworld of infinite possibilities, maybe the last one on Earth.

How Tweet it Is – New York Magazine

For a browser with a completely empty cache, the browser will load the requested library (and block all other requests during the load). For every site the user visits, the library is loaded again and again because the cache does not work across sites. Loading site1.com/jquery.js and site2.com/jquery.js takes the same amount of time.

Google’s come up with a clever solution to this problem. Instead of pointing to your local copy of whatever AJAX library you want, you point to Google’s copy using their AJAX Libraries API.

There’s many advantages to this. If the user has been to Gmail or any Google property recently, the library is already loaded in their browser, ready to execute. If the user goes to other sites which use the API, their cache is already primed and does not need to be loaded again. 

We’ve just put this in place over at Twitter, and it should improve page load times dramatically.

The software is able to replace faces within images from a database of candidate images drawn from the Internet and other sources, and does so automatically.

A movie of their efforts exists here…

It’s the only piece of software I know of that has ‘–thpppt’ as an option (Install it, and you’ll see.) It’s an optimized replacement for many of your favorite awk | grep | sed combinations, and includes syntax highlighting.

In other news, I’ve resolved a number of bugs and issued new code for running mod_telemetry on 64-bit Linux. Check out the SVN trunk for the latest branch. The data that this module has been providing to me has been invaluable for researching slow points in the back end.

It’s on May 5th, 2008, only three weeks away!

Some info, follows, courtesy of the Sun site:

Join experts and contributors from dozens of free and open source projects driving innovation today. Gain skills and find solutions to advance your infrastructure, develop next-generation applications, and create compelling web experiences. Choose from 70+ sessions led by experts from 30+ communities on OSes, databases, web and app servers, scripting, Web 2.0, and more

In my case, I was parsing an Apache configuration file and displaying VirtualHost entries, but you can use this for just about anything (of course, models work too if you’d like to go against the DB)

First, install the auto complete plugin. Rails 2.0 doesn’t have autocomplete in the core anymore, so you’ll have to create your project with the usual rails myproject syntax, and then enter that directory with cd myproject.

Type “script/plugin install auto_complete” to install the auto complete plugin.

Next, we create a simple class to hold our data. You could parse a file here (say, with File.open()), or pull data from other sources in the initialize function of this class. If you’ve got a model in the database, you can use that.

class Apacheconf

@names = []

def names
@names
end

def initialize
# replace this code later with parsing code

@names = Array.new
@names << “alpha”
@names << “beta”
@names << “gamma”
@names << “delta”
end

end

… then we add some code and CSS to our view for the affiliate/index method in app/views/affiliate/index.rhtml

<html>

<head>

<%= javascript_include_tag :defaults %>

<style>

div.auto_complete {
  width: 300px; 
  background: #fff; 
} 

div.auto_complete ul { 
  border: 1px solid #888; 
  margin: 0px; 
  padding: 0px; 
  width: 100%; 
  list-style-type: none; 
} 

div.auto_complete ul li { 
  margin: 0px; 
  padding: 3px; 
} 

div.auto_complete ul li.selected { 
  background-color: #ffb; 
} 

div.auto_complete ul strong.highlight { 
  color: #800; 
  margin: 0px; 
  padding: 0px; 
} 

</style>
</head>
<body>
<% form_tag :action => “edit” do %>
<p><%= text_field ‘affiliate’, ‘name’, :autocomplete => ‘off’ %></p>
<div class=”auto_complete” id=”affiliate_name_auto_complete”></div> 
<%= auto_complete_field :affiliate_name,
     :url => { :action=>’autocomplete_affiliate_name’ },
     :tokens => ‘,’ %>
    <%= submit_tag %>
    <% end %>
</body>

… and then we insert a very small amount of code into the controler (in app/controllers/affiliate_controler.rb)  to handle the Ajax callback:

class AffiliateController < ApplicationController
  skip_before_filter :verify_authenticity_token, nly => [:autocomplete_affiliate_name]  

def autocomplete_affiliate_name
     # this code is called over and over again by Ajax
  #
     # the next line does a search through the array for terms starting with the 
     # specified parameters.

     @x = Apacheconf.new.names
     @affiliates = @x.select { |v| v =~ /^#{params[:affiliate][:name]}/ }

     # don’t render the layout – we only want the partial to be rendered
     render :layout=>false 
  end

end

The beauty of this that with the plugin installed, it takes very little work to get autocomplete working.