A bit of last-minute news, but I’ll be on a panel at SXSW Interactive: “Using GPS & Location to Enhance Social Networking”.
First there were social networks, and then there were location-based social networks, and now GPS and navigation-enhanced mobile social networks. This panel will explore how these emerging platforms integrate with existing social networks (facebook, twitter, etc), leverage GPS navigation functionality, and take location-aware social networking to the next level.
It’s at 5pm on Tuesday, the 17th of March.
More details at available on the panel’s schedule page.
Posted by John Adams on February 13th, 2009
Normally I don’t re-post Twitter articles here but this one on the New York magazine was wistful, fair, balanced, and gave a good representation of what it’s like to work here.
The reporter was in the office on the very day the US Airways flight crashed into the Hudson, and he recorded our (completely boring) reactions to the event.
Sure, the Twitter guys still have no idea how to make money off their fabulous invention. But for now they are living in a dreamworld of infinite possibilities, maybe the last one on Earth.
How Tweet it Is – New York Magazine
Posted by John Adams on February 9th, 2009
Since people started using virtual hosts by name with Apache HTTPD and other web servers, it has become very difficult to figure out which virtual hosts live on a single IP, if all you have is the IP address.
Have a look at the Robtex Internet Swiss Army Knife. It solves this problem, and far more, including AS# lookups, BGP dereferencing, and DNS checks. There’s a firefox search toolbar available for the site (very useful!) and RBL (blacklist) check tools right on the main page of the site.
Posted by John Adams on February 2nd, 2009
Often on our local network, someone will start using up all of our outbound Internet bandwidth, and this leads to the network administrator’s dilemma:
How do we find the user in question so we can thump them on the head to make them stop?
This is a basic exercise in information gathering. For the most part, we’ll have the user’s IP address, and we’re a mac shop with many users running iTunes. If the user is sharing their library, you can use iTunes as a covert means of determining a user’s name, as iTunes will use the local computer’s name as the library name.
Telnet to the machines DAAP port, and issue:
John-adamss-macbook-pro:~ jna$ telnet x.x.x.x 3689
Trying x.x.x.x...
Connected to x.x.x.x.
Escape character is '^]'.
GET /server-info HTTP/1.1
Host: x.x.x.x
Client-DAAP-Version: 3.7
User-Agent: iTunes/8.0.2 (Macintosh; N; Intel)
Accept-Language: en-us, en;q=0.50
HTTP/1.1 200 OK
Date: Tue, 13 Jan 2009 21:26:38 GMT
DAAP-Server: iTunes/8.0.2 (Mac OS X)
Content-Type: application/x-dmap-tagged
Content-Length: 280
msrvmstt?mproaproaeSVaeFPatedmsedmsmlmsmOk?[minmUSER NAME’s LibrarymslrmstmsalmsasmsupmspimsexmsbrmsqymsixmsrsmsdcmstcImmsto???
Other options for this include attempting to sign on to the server with Apple-K if AFP on TCP port 548 is active (which will reveal the computer’s name) and using nmap with service detection to glean information about the host.
Posted by John Adams on January 13th, 2009
On this page, Netgear releases Firmware version 1.0.30 for the WNR3500 router.
In my previous Apple Macbook Pro to Local network host (Mac Mini) testing, my top connection speed was around 2.4 Mbps. After the upgrade, it’s between 4.65Mbps and 7.5Mbps. Nothing near the promised speeds of 802.11N (300Mbit/sec), but I suspect that this is because of an incompatibility between Apple’s hardware and Netgear’s Hardware.
------------------------------------------------------------
Client connecting to 10.1.1.15, TCP port 5001
TCP window size: 129 KByte (default)
------------------------------------------------------------
[ 3] local 10.1.1.70 port 51617 connected with 10.1.1.15 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 8.59 MBytes 7.19 Mbits/sec
Update:
After disassociating and reassociating with the AP, speeds went way up:
retina:~ jna$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 4] local 10.1.1.15 port 5001 connected with 10.1.1.70 port 52865
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 34.8 MBytes 29.1 Mbits/sec
With other devices on the WLAN, speeds go down. My current theory is that 802.11g devices on the same wireless network (such as the older Macbooks that we have) drag 802.11n speeds down, but I’m yet to be able to prove that.
Posted by John Adams on January 10th, 2009
Macworld 2009 is this week in my home town of San Francisco, and while we won’t have Jobs’ famous Keynote speech nor the participation of Apple this year, we will have the usual barrage of Macworld Parties.
A great list of events and parties is right here.
My week looks a bit like this:
Posted by John Adams on January 5th, 2009
Today we’re going to teach you how to deal with having too many computers. Moving media around is a living hell because iPhoto and iTunes assume that you only ever possess one library. Sure, you can play music and movies purchased in the store on multiple machines, but what about your own library? How do you use that on multiple machines without moving things around?
At home I have a number of Macs, with one large machine (~1.5TB disk, 4 GB RAM) dedicated to digital photo editing. This machine houses a large volume of photos in it’s “Final Exports” folder. It’s not my main computer – my main computer is a MacBook Pro which travels with me nearly everywhere, and when I don’t have that, I have my iPhone.
I want my photos with me everywhere (or, at least, the last few hundred of them) so I can show people the last great event I went to, or that thing in the club that time. Here’s my solution.
1) Keep the photos on the large machine, where I edit photos in Adobe Lightroom and export them to the “Final Exports” folder.
2) Keep the laptop as the primary sync machine for the iPhone
3) Sync the iphone to the laptop, and retrieve the latest photos.
iPhoto 7 has a wonderful feature called Photocasting which will read lists of latest photos from the Internet (say, flickr, for example.) using a format that is very similar to RSS, but completely not compliant with current RSS standards.
The following Ruby script, and ERB template will turn a directory of directories into a pubsub feed for iphoto. You save your files in this form:
Final_Exports/dir1
Final_Exports/dir1/1.jpg
Final_Exports/dir1/2.jpg (and so on…)
Final_Exports/dir2
Final_Exports/dir…
Final_Exports/dirN (and so on…)
I use the scripts to generate RSS, and then put the RSS file somewhere on the Internet (the same directory with the photos works well, as my machines are internet accessible.) Running the script from cron once a day and syncing the phone, keeps you up to date.
Scripts:
makeiphotorss.rb
makeiphotorss.erb
Posted by John Adams on December 31st, 2008
At Twitter, we have many users which sign up for the service and mistype or enter invalid email addresses. Our product group doesn’t want us to use email verification, and for the most part, we cannot because we accept signups via mobile (through the 40404 SMS short-code.) If we bounce too many messages for any major provider with a clue (think: hotmail, gmail, yahoo…) they’ll turn us off. If a user signs up for the service with an invalid email service, there’s a fair chance that we could be sending email to a dead address. What do we do?!
There’s two things. First, on outgoing mail, we convert the address to a special VERP address. An email address of “user@example.com” becomes “twitter-welcome-user=example.com@postmaster.twitter.com”. We usually replace “welcome” with something topical so we can track the origin of the message.
Why VERP?
If the mail bounces, the mailer-daemon on the other end will bounce the message back to the postmaster machine, which is a transport method in our inbound Postfix server pool. If an MTA along the transit path destroys the original destination email address, we can recover it from the To: address on the bounce. DJB first devised this for use with qMail, but it works great here.
That server runs a Ruby (originally written in perl) script, that pushes data to an internal API that increments the email address with a bounce score . Get too many points in the bounce score table, and we disable sending to your account. If you change your email address in our database, or ask us to try again, we clear the bounce score. We also clear the bounce score after 30 days or by admin request.
We identify and score bounced emails through the use of a simple regexp based scoring mechanism and/or use the DSN if a Delivery Status Notice is available. A “5″ indicates “hard fail, “2″ = soft fail, and “1″ = we don’t know. Anyone scoring over five is disabled. If it’s the “welcome to twitter” message, we mark it as 10 immediately. Your first mail has to go through, period.
Disabling is accomplished through a reverse blacklist through Postfix’s smtp_recipient_restrictions configuration directive and MySQL via proxymap.
The directives to do this under Postfix 2.5 are pretty simple.
In main.cf, On the central outgoing server(or servers…), Add:
# bounce handling
bouncehandler_destination_recipient_limit = 1
transport_maps = btree:/etc/postfix/transport
smtpd_delay_reject = yes
smtpd_recipient_restrictions = \
check_recipient_access mysql:/etc/postfix/reject-bouncing.cf, \
permit_mynetworks, \
reject_unauth_destination
/etc/postfix/reject-bouncing.cf:
user = your_db_username
password = your_db_password
dbname = your_db_name
hosts = your_db_host
query = SELECT 'DISCARD in_bouncers_table' FROM bouncers WHERE email='%s' AND score >= 5
In master.cf, on the incoming server, add:
bouncehandler unix - n n - - pipe
flags=DRhu user=nobody argv=/etc/postfix/bouncehandler.pl
I use the following function to calculate severity of a bounce. It’s not perfect:
sub get_severity {
my ($es) = @_;
my $body = $es->body;
my $subject = $es->header("Subject");
my $score = 1;
# DSN Failures
if ($body =~ /Action: failed/) {
$score = 5;
}
# check through body of message and try to score the bounce.
# temp failures.
if ($body =~ /connection refused|host unreachable|host or domain not found/mi) {
$score = 5;
}
if ($body =~ /quota|mail(box|folder)* is full/mi) {
# mailbox full, not as bad, we'll try up to 4 times.
$score = 2;
}
# wierd temporary failures
if ($body =~ /(junk mail|spam) try again later/mi) {
$score = 1;
}
# temp.
return $score;
}
I leave it to you to write bouncehandler.pl, including the above code, and the database schema. It’s very simple. Parse the message, create rules, insert/update a record in your database. I used Email::Simple for parsing, and DBI/Mysql for database access.
The DB Schema needs to contain these columns (at a minimum): The email address, a score column, created_at, and updated_at.
My queries look something like this:
# prepare SQL statements for later use
# last times are based on our receipt, not theirs.
$findemail_sth = $dbh->prepare('SELECT id, score FROM bouncers WHERE email=?');
$insert_sth = $dbh->prepare('INSERT INTO bouncers (email,score,updated_at,user_id) VALUES (?,?,now(),?)');
$update_sth = $dbh->prepare('UPDATE bouncers SET score=?, updated_at=now(),user_id=? where id=?');
Posted by John Adams on December 12th, 2008
Viewdle, a video search engine, launched recently, and won the 2008 LeWeb Gold prize . It’s very similar to a technology that casinos have had for years. In previous times they’d look up your face in the five-volume Griffin GOLD book, a litany of cheats. Machine vision has surpassed the book, by far.
I’ve long been fascinated by casino security. Only the military and casinos have access to massive budgets used to track, identify, and secure locations. While other corporations may have similar budgets they lack the zeal of the casinos and their lust for security technologies.
This application is essentially video facial recognition, combined with crowd sourced tagging. The casinos did this, but shared data between themselves across the Internet. Viewdle opens up this possibility to the public.
A nice flowchart of their process is here…
They appear to have close ties with Reuters, and Reuters is using Viewdle for theironline people search.
Posted by John Adams on December 11th, 2008
Lately I’ve been examining the actual performance charastics of wireless networking devices.
I recently purchased a Netgear WNR3500, and testing with iperf tells me that I see a maximum of 35Mbit/s. Many commenters have written in to ask me to try forcing the 802.11 connection to 802.11g, so I did that by installing my older Apple Airport Extreme (snow) base station.
Performance on that device is even worse:
dhcp-102:iperf-2.0.4 jna$ src/iperf -w256k -c 10.1.1.15
------------------------------------------------------------
Client connecting to 10.1.1.15, TCP port 5001
TCP window size: 257 KByte (WARNING: requested 256 KByte)
------------------------------------------------------------
[ 3] local 10.1.1.102 port 54575 connected with 10.1.1.15 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.1 sec 20.5 MBytes 17.0 Mbits/sec
Wireless speeds just don’t compared to wired! I’ve yet to see anything come close to 100mbit/sec, but 802.11N (draft standard 4) is much faster, by a factor of two.
Posted by John Adams on November 27th, 2008
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=2d3fdeba-aafc-46ed-9fde-38fd183e1bbd)
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=55ca781b-e546-4508-9106-b7d8a01b4fdf)
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=c738c9fb-6851-4061-872a-3eac78b7473c)
