At home I have a number of Macs, with one large machine (~1.5TB disk, 4 GB RAM) dedicated to digital photo editing. This machine houses a large volume of photos in it’s “Final Exports” folder. It’s not my main computer – my main computer is a MacBook Pro which travels with me nearly everywhere, and when I don’t have that, I have my iPhone.

I want my photos with me everywhere (or, at least, the last few hundred of them) so I can show people the last great event I went to, or that thing in the club that time. Here’s my solution.

1) Keep the photos on the large machine, where I edit photos in Adobe Lightroom and export them to the “Final Exports” folder.

2) Keep the laptop as the primary sync machine for the iPhone

3) Sync the iphone to the laptop, and retrieve the latest photos.

iPhoto 7 has a wonderful feature called Photocasting which will read lists of latest photos from the Internet (say, flickr, for example.) using a format that is very similar to RSS, but completely not compliant with current RSS standards.

The following Ruby script, and ERB template will turn a directory of directories into a pubsub feed for iphoto. You save your files in this form:

Final_Exports/dir1

Final_Exports/dir1/1.jpg

Final_Exports/dir1/2.jpg (and so on…)

Final_Exports/dir2

Final_Exports/dir…

Final_Exports/dirN (and so on…)

I use the scripts to generate RSS, and then put the RSS file somewhere on the Internet (the same directory with the photos works well, as my machines are internet accessible.) Running the script from cron once a day and syncing the phone, keeps you up to date.

Scripts:

makeiphotorss.rb

makeiphotorss.erb

For a browser with a completely empty cache, the browser will load the requested library (and block all other requests during the load). For every site the user visits, the library is loaded again and again because the cache does not work across sites. Loading site1.com/jquery.js and site2.com/jquery.js takes the same amount of time.

Google’s come up with a clever solution to this problem. Instead of pointing to your local copy of whatever AJAX library you want, you point to Google’s copy using their AJAX Libraries API.

There’s many advantages to this. If the user has been to Gmail or any Google property recently, the library is already loaded in their browser, ready to execute. If the user goes to other sites which use the API, their cache is already primed and does not need to be loaded again. 

We’ve just put this in place over at Twitter, and it should improve page load times dramatically.

It’s the only piece of software I know of that has ‘–thpppt’ as an option (Install it, and you’ll see.) It’s an optimized replacement for many of your favorite awk | grep | sed combinations, and includes syntax highlighting.

In other news, I’ve resolved a number of bugs and issued new code for running mod_telemetry on 64-bit Linux. Check out the SVN trunk for the latest branch. The data that this module has been providing to me has been invaluable for researching slow points in the back end.

mod_telemetry keeps track of all URLs on your site and how long it takes your server to deliver each request. It reports these by using a status handler (like Apache’s /server-status) to display real-time status information about the performance of your site.

It is extremely useful in optimizing back-end processes (for example, when connecting to Java, FastCGI, or Ruby back-ends) to identify when specific pages are performing slowly.

Because it runs in the server, it’s able to show end-to-end performance on each request

It requires that mod_status be enabled to work.

An initial version of this code is available on google code. Please help me debug and expand it’s capabilities.

I recommend you checkout the SVN trunk and not download the tar. It works with Apache 2.x and 2.2.x.

How do we store hundreds of thousands of movies in a redundant, distributed fashion across many data centers, and bring users to that content quickly?

There’s lots of ways to do this. Rsync, automated copies, directory analysis, usage pattern analysis, etc. Most of them have the same problem: Moving data takes time.

But have you looked at zvents? Their recent work on hypertable shows that they can make things happen, and that their strength lies in their engineering and not their domain name.

It’s on May 5th, 2008, only three weeks away!

Some info, follows, courtesy of the Sun site:

Join experts and contributors from dozens of free and open source projects driving innovation today. Gain skills and find solutions to advance your infrastructure, develop next-generation applications, and create compelling web experiences. Choose from 70+ sessions led by experts from 30+ communities on OSes, databases, web and app servers, scripting, Web 2.0, and more

From their blog, it features:

• Dynamic webserving, with full support of common web technologies
• Persistent storage (powered by Bigtable and GFS with queries, sorting, and transactions)
• Automatic scaling and load balancing
• Google APIs for authenticating users and sending email
• A Fully featured local development environment

Unlike Amazon, they don’t give you a virtual machine image to work on. They run the application code for your app, and deal with everything else. All you’re maintaining is code, and small initial applications are free!

I’d love to try it out, but within hours, the first 10,000 developer signups were gone.

Google AppEngine Intro Video

At yesterday’s SFBETA meetup, I had the chance to play with PicLens, a dynamic system for displaying images in your browser, all pulled from the APIs of various web sites.

It works as an adjunct to your existing browser, installing as a plugin (written in C++, or so the developer tells me.)

If you connect to a supported site, such as Google Images, Yahoo Images, Flickr, Photobucket, Smugmug, or Deviant Art, and hover over an image, you’ll see a small play icon on images from those sites, injected by the plugin. Click the plugin, and you’re greeted with an absolutely gorgeous, high-speed 3-d/2-d browser that allows you to cycle through all of the images in that set in full-screen glory:

There is also support for direct RSS feeds, if you want to pull images from your blog directly.

The real impact of this system is being able to view so many photos at once. It rapidly exposes you to what these sites have to offer, and the search engine makes searching for recent events even more compelling. Additionally, there’s a search box in the upper right which can be used to search all six of the supported sites within the application, in case you don’t see what you like.

The application appears to be made of pure, futuristic goodness, with web2.0 reflections all over the bottom of the flying images, direct click and zoom selection of images, and really, this is what I expected the internet future to look like. Why do we not have enough 3d browsers?

Words are not going to do this app justice. Just download it now and check it out. It’s not supported in Safari 3.1 or firefox 3 yet, so you’ll have to run an older copy of your browser to use the plugin. Updated copies will be available soon for those browsers, so never fear.

While this is an older article ( from 2004 ), it still addresses many dangerous issues that developers continue to create in production code.

One of our developers here recently wrote a fairly large scripting system to deliver advertising to customers that was vulnerable to at least four XSS attacks, and I spent a fair amount of time sanitizing input and securing her code.

Do you accept input from users? Are you ensuring that you strip_tags (to block XSS/XSRF), escaping strings (to block SQL injection) and sanitizing all user input before storing or displaying it? If not, you might be vulnerable.