http://engineering.twitter.com/2010/03/unicorn-power.html

There’s two things. First, on outgoing mail, we convert the address to a special VERP address. An email address of “user@example.com” becomes “twitter-welcome-user=example.com@postmaster.twitter.com”. We usually replace “welcome” with something topical so we can track the origin of the message.

Why VERP?

If the mail bounces, the mailer-daemon on the other end will bounce the message back to the postmaster machine, which is a transport method in our inbound Postfix server pool. If an MTA along the transit path destroys the original destination email address, we can recover it from the To: address on the bounce. DJB first devised this for use with qMail, but it works great here.

That server runs a Ruby (originally written in perl) script, that pushes data to an internal API that increments the email address with a bounce score . Get too many points in the bounce score table, and we disable sending to your account. If you change your email address in our database, or ask us to try again, we clear the bounce score. We also clear the bounce score after 30 days or by admin request.

We identify and score bounced emails through the use of a simple regexp based scoring mechanism and/or use the DSN if a Delivery Status Notice is available. A “5″ indicates “hard fail, “2″ = soft fail, and “1″ = we don’t know. Anyone scoring over five is disabled. If it’s the “welcome to twitter” message, we mark it as 10 immediately. Your first mail has to go through, period.

Disabling is accomplished through a reverse blacklist through Postfix’s smtp_recipient_restrictions configuration directive and MySQL via proxymap.

The directives to do this under Postfix 2.5 are pretty simple.

In main.cf, On the central outgoing server(or servers…), Add:

# bounce handling
bouncehandler_destination_recipient_limit = 1
transport_maps = btree:/etc/postfix/transport

smtpd_delay_reject = yes
smtpd_recipient_restrictions = \
  check_recipient_access mysql:/etc/postfix/reject-bouncing.cf, \
        permit_mynetworks, \
        reject_unauth_destination

/etc/postfix/reject-bouncing.cf:

user = your_db_username
password = your_db_password
dbname = your_db_name
hosts = your_db_host
query = SELECT 'DISCARD in_bouncers_table' FROM bouncers WHERE email='%s' AND score >= 5

In master.cf, on the incoming server, add:

bouncehandler  unix  -       n       n       -       -       pipe
  flags=DRhu user=nobody argv=/etc/postfix/bouncehandler.pl

I use the following function to calculate severity of a bounce. It’s not perfect:

sub get_severity {
    my ($es) = @_;
    my $body = $es->body;
    my $subject = $es->header("Subject");
    my $score = 1;

    # DSN Failures
    if ($body =~ /Action: failed/) {
	$score = 5;
    }

    # check through body of message and try to score the bounce.
    # temp failures.
    if ($body =~ /connection refused|host unreachable|host or domain not found/mi) {
	$score = 5;
    } 

    if ($body =~ /quota|mail(box|folder)* is full/mi) {
	# mailbox full, not as bad, we'll try up to 4 times.
	$score = 2;
    } 

    # wierd temporary failures
    if ($body =~ /(junk mail|spam) try again later/mi) {
	$score = 1;
    }

    # temp.
    return $score;
}

I leave it to you to write bouncehandler.pl, including the above code, and the database schema. It’s very simple. Parse the message, create rules, insert/update a record in your database. I used Email::Simple for parsing, and DBI/Mysql for database access.

The DB Schema needs to contain these columns (at a minimum): The email address, a score column, created_at, and updated_at.

My queries look something like this:

# prepare SQL statements for later use
# last times are based on our receipt, not theirs.
$findemail_sth = $dbh->prepare('SELECT id, score FROM bouncers WHERE email=?');
$insert_sth    = $dbh->prepare('INSERT INTO bouncers (email,score,updated_at,user_id) VALUES (?,?,now(),?)');
$update_sth    = $dbh->prepare('UPDATE bouncers SET score=?, updated_at=now(),user_id=? where id=?');

I’m in the lower left of this photo. It’s not the best, but it’s representative of our collective emotion at Twitter HQ on that historic night!

Image Courtesy of Scott Beale, Laughing Squid.

The rest of the photos and Scott’s article Lives here …