Entries (RSS)  |  Comments (RSS)

Twitter Digits

Twitter’s newest service, Digits, offers a glimpse into a world of “growth at any cost.” Or, as it is put forth on the developer site, “Growth, Simplified.”

Their insecure attempt to eliminate the password, moving to SMS as a primary, single-factor authentication endangers users of the Twitter application and all future developers (and their users) who integrate against Digits.

No longer is security dependent on something you know (like a strong password sent over SSL), and possibly something you have like a two factor token, but access now depends entirely of the security of the carrier’s network, and the user’s security is in turn dependent on their government’s ability to introspect GSM traffic as it is sent across the wire.

Such man in the middle attacks are not uncommon. As reported in The Guardian, The NSA collects millions of text messages, daily. Should the user become a target of NSA monitoring, all it takes is for them to enter your phone number into a Digits login screen to gain access to your account.

Abroad, the security community has seen attempts at SMS introspection and monitoring by the likes of Egypt Telecom, Syria Telecom, and others.

US Protesters are also at risk thanks to IMSI catchers like the Stingray device, a product which offers the capture of user details and text messages for local law enforcement. When law enforcement wants access to your account, they could go directly to your carrier instead of fighting with Twitter’s (usually strong) efforts to protect the security of your account.

You can watch this Black Hat video here, describing a similar and entirely possible attack against users that use Femtocells.

This change of heart is in strong contrast to prior efforts at Twitter to shift to a strong security model reliant on custom, public-key authentication and only using SMS codes as a fall back for login (when permitted with a previously known password, passed over SSL/TLS.)

This is a common mistake by companies. To shift from strong security models to “growth at all costs”. It’s a flawed reasoning, that the true problems with growth is the strength of their security, and not issues with their product, marketing, or otherwise.

I urge developers to not support the Digits product until a stronger security model can be released publicly.

Full disclosure: I am still a stockholder in Twitter but no longer an employee there.

Update: I’ve had a few discussions with people at Twitter who say this probably will not be used for anything other than 3rd party apps. It’s still a slippery slope to put forth something supported on 1FA and I wouldn’t have done it if I were still there.

Duncan and I get into a bit more detail on this Medium Post, “One Identifier isn’t Enough”.

Posted by on October 22nd, 2014

Read Full Post  |  2 Comments »

Free Your Data from Cardmunch

I loathe business cards.

I meet many people at work and at technical conferences but the worst part is that every time someone hands me a business card I have this piece of paper that will eventually live at the bottom of my bag with the rest of my junk, never to be seen again.

I print my own business cards as well ( or make work print them ), adding to this problem. Who knows what people do with my cards? Maybe they are too destined for the bottom of a dirty bag or briefcase. In any event, this whole business card madness has got stop.

Many years ago someone invented QR codes. I’m told they were initially designed to track automobiles in manufacturing, but they’re everywhere now. While they might be a way to link the real world to the online world they are rarely on business cards. They’re ugly and they frequently do not contain all relevant contact data. Worse yet, nearly every mobile phone reader will put data from the QR code into different fields in the phone’s address book, if they’re able to be read at all.

A startup known as CardMunch had a novel solution to this problem – Take a photo of the card, ship that photo to some rat-trap, manual, human data entry hole in a foreign country and then send you back a fully populated iPhone address book entry. Sounds awesome, right? Aside from the privacy concerns with shipping your data off, that is. Cardmunch used to allow the export of scanned data to the iPhone address book and a download option allowed you to have your contacts as a CSV.

Right on! Problem Solved!

Well, no. Along comes LinkedIn. They buy Cardmunch back in Janurary 2011, which should make everyone happy, because now you can take the LinkedIn data and merge it with the business cards you’ve just scanned. “Excellent! Sign me up!”, says the business guy, or musician, or someone who values relationships. Finally this business card bullshit is mostly over and you’ve got extremely detailed meta data on your new friends.

Cardmuch used to charge for this service by the business card, but that option is now gone as LinkedIn decided to make the service free with a ridiculous caveat — Once your data is in the cardmunch DB, it’s not coming out. They shut down the Cardmunch website and the associated data exfiltration tool. They removed the “mass add” which allowed addition of all of the contacts back into your address book and forced you back into the application to see your contacts. The only way to move a pile of contacts from their app to the iphone app is one-by-one, or, by adding them one-by-one to linked in and using their site.

The scanned contacts don’t show up in the main Linkedin site unless you’ve made a connection to the person on the card, so the standard way of exporting data isn’t going to work here.

Let’s get our data back.

Step 1. Jailbreak your iPhone ( I know, this sucks, but I bet you’ve already done this. )

Step 2. Install OpenSSH from Cydia to the phone

Step 3. Install sqllite3 from Cydia (optional)

Step 4. SSH into your iphone.

Step 5. Get the data

CardMunch’s UUID for the new version of the iPhone app is 85882CAE-2582-407E-84E0-7188F7447B75

The software stores everything locally into a sqllite3 database (like most iPhone Apps) for local usage. Your data is most likely in:

/private/var/mobile/Applications/85882CAE-2582-407E-84E0-7188F7447B75/Library/CardMunch.sqlite

Step 6: SCP this file to your local macintosh, or wherever you’ve got a copy of sqlite3 handy

Step 7: Explore the data

Startup sqllite3 and open up the DB with “sqlite3 Cardmunch.sqlite”

There’s a few tables, but only one that we’re really interested in:

[sourcecode]

SQLite version 3.7.7
Enter ".help" for instructions
sqlite> .tables
ZACCEPTEDCONTACT ZUNMATCHEDDATA Z_METADATA Z_PRIMARYKEY
sqlite>

[/sourcecode]

ZACCEPTEDCONTACT are the contacts which Cardmunch has scanned and accepted. It’s where your data lives.

To get the schema for this data, you can do .schema ZACCEPTEDCONTACT. Once we have our schema, we can write basic SQL to extract the data we need. The schema is fairly complex, but there are basic fields you can use depending on the detail you’d like from each record. Note that some of these fields are marshalled NSObjects which will require further decoding.

Let’s get a basic person list:

[sourcecode]

sqlite> select ZFIRSTNAME, ZLASTNAME, ZCOMPANYNAME from ZACCEPTEDCONTACT;

John|Smith|Systems, Inc.
Bob|Dobbs|Systems, Inc.
… etc …

[/sourcecode]

It’s easy from here. Enjoy!

Posted by on March 3rd, 2012

Read Full Post  |  8 Comments »

RSUs vs Options.

Part of what I do in addition to dealing with technology is to keep one eye on the VC and money side of silicon valley. There’s much going on here and its hard to avoid watching where all the money comes from.

The more time I spend studying companies and company stock, the more I realize how fucked you are if you take a job with a company and they issue you RSUs instead of options or stock. They’re usually handed out to people who show up too late in that particular startup’s lifecycle to matter. The new employees aren’t founders, they’re not startup people, they’re just workers. Unlike the first 20 employees, they are taking little risk as the company is already established, and they will see little reward in the end.

Google and Facebook have been doing this for years. In reality, the prevalance of RSUs is a direct response to much of the legislation issued after the .com bust. It’s easier to stall on the IPO and continue to accumulate capital than to deal with the IPO itself. Remember that the stock market was originally designed to allow individuals to pump capital into companies, a job now largely taken over by VCs.

With those companies, their growth is so good that you might get a nice upside working for them, but nothing like you would with options. Worse yet, you get few rights if you have RSUs. No voting rights, nothing. No common stock. If you hold them for ten years and your company doesn’t IPO, you get nothing (average IPO time these days is 9+ years and the market is slow) when they expire. The SEC will even grant special privileges to your company (easily upon request) so they can avoid the registration requirements for stock, which is around 500 stockholders and/or > $1M in assets.

Why is that important? Because around 500 registered stockholders, the company is forced into an IPO; The reporting requirements under Sarbanes-Oxley will cost as much as the IPO does, for the most part.

RSUs come with many restrictions. You can’t transfer them (to your spouse, or anyone), if you die, your successor is stuck with the same restrictions, and you can’t sell them. Guess what else? If your company gets bought, the acquiring company can just terminate RSUs during the acquisition. Whoops. You lose your stock.

At least if you get fired, you (might) get to keep your RSUs. If they manage to IPO, it’s going to be a good year or so before you can sell. It’s standard though, during the IPO, to convert RSUs directly to Common Stock during a Liquidity event, though.

If you’re taking a new job, take options or direct grants only, and make sure you read Know your Options. If you’re offered RSUs and you want serious upside stay away from companies that are issuing RSUs. You’re going to get fuck all in the end. Go find an early-stage startup instead. If you want a safe job, take the RSUs.

Here’s an example of what a company can do with the SEC. It’s Twitter asking for an exemption from Section 12 of the Exchange act and it contains some very interesting information about how the RSU program works.

http://www.sec.gov/divisions/corpfin/cf-noaction/2011/twitter091311-12gh-incoming.pdf

Here’s facebook’s for comparison (but they’re written by the same lawyer and even have a similar filename, so don’t expect too much difference.)

http://www.sec.gov/divisions/corpfin/cf-noaction/2008/facebook101308-12gh-incoming.pdf

For more information, have a look at the Bloomberg article, here:

http://www.bloomberg.com/news/2011-09-15/twitter-gets-leeway-from-sec-in-issuing-restricted-stock-units.html

Posted by on September 16th, 2011

Read Full Post  |  6 Comments »

Damn Small Linux: Making bootable USB drives on Mac OS X

Recently I had a need to log into a machine with no cdrom and I decided to use Damn Small Linux to get enough of a shell to access the machine.

It’s a very small, 50MB distribution of linux that is not that easy to install if you don’t have a linux box to start with. Despite the large number of Linux boxes that I once owned, I’ve replaced all of them with Mac OS. Fortunately, there is a simple way to install DSL on a USB boot drive from Mac OS, using VMWare Fusion.

You’ll need a 1GB USB drive. The smaller the drive is, the better. Some systems cannot boot off of very large USB drives. 512MB or 1GB is recommended.

Here’s the how to…

Download the latest copy of DSL from a mirror.

Do not use syslinux, but get the ISOLINUX version. This is the ISO named dsl-x.x.x where x.x.x is a version number. I used dsl-4.4.10.iso, although in retrospect I should have used the VMWare VMX Image. Either works.

Insert the pendrive and format it.

Mac OS will auto mount the pendrive. Open Disk Utiilty.
Format the device as MS DOS FAT and MBR.
One Partition.

Important: Unmount the pendrive before starting VMWare Fusion.

Create a new VMWare Fusion Virtual Machine

The general idea here is to create an empty VM using fusion, and to boot off of the ISO you have just downloaded.
It can be reasonably small. You do not need to allocate disk space on your HD.
Configure the VM as Linux/Ubuntu and to mount the CD.

Mount the pendrive as an “Alcor Micro Mass Storage” device. It will appear to your VM as /dev/sda.

Install DSL

When DSL boots up, at the “boot:” prompt, type “install”
Then, select “5” for “USB Pendrive HDB boot install”

Answer yes to all prompts, and the install will (hopefully) complete.

Now What?

You’re done. DSL is great for password recovery, emergency repair, or system rescue. So long as your system supports booting from USB drives, you’re good to go!

Posted by on June 20th, 2011

Read Full Post  |  1 Comment »

Dear apple, what the Dock?

Apple’s new iPhone4 has created quite a stir, even though consumers are paying slightly more for the same phone with a slightly better display, dual (improved) cameras, and flash. I’m sure by now you know about the antenna problems and the lame way that apple chose to fix it, but I wanted one of these wonderful gadgets, so I signed up on the reservation list and finally got my email notification to pick one up this morning.

The Apple store was helpful as always, but as soon as I got the phone in my hands I realized that it was going to be another iPhone upgrade with a different goddamn dock. I tried the easy way out and wasted $9 on a “universal dock adapter kit”, which didn’t work at all with my Apple iPhone 3GS dock. I assumed it was meant for the dock that I owned, but this adapter was meant for some device which clearly didn’t match anything that Apple has ever produced. Nine dollars wasted and no working dock.

The dock is an essential device to me. Without it, I’d forget to drop my phone into the dock at work and keep it charged. How to fix this? Apple wants $29 for a new iPhone 4 dock. This is completely unacceptable to me, considering that the iPhone 3GS dock is now selling at $7 on many sites. Why pay again for a similar connector and janky audio-out contraption? (Did you know the audio out is provided by a microphone in the base of the dock, which connects to an amplifier and then the audio out jack? It does.)

There was only one way to fix this — time to make this right with some hacking.

So, out comes the dremel tool:

Let the hacking begin. We start with the 3gs dock, and go from this…

…to this, a few moments later. I hacked it into the shape of the 4g:

But guess what? Hacking the dock to fit the iPhone4 doesn’t work, because Apple’s modified the dock to only recognize the 3GS. If you connect an iPhone 4 to that very same connector, the phone and iTunes will not recognize the phone.

Sure, my 3gs still works, so I haven’t damaged anything:

Connecting the cable directly to the 4g works, just not the dock to the 4g through the very same cable.

Apple’s position on this is worsening. The iPhone 1g’s came with working docks. Then, they stopped supplying them. on the 2g and 3g models, it was an extra $20 and $25. Now it’s $29.

This is awful. There’s a custom plug that you can’t easily purchase, and all iPhone models, even though they use the same plug, are incompatible with each other’s docks because of artificial incompatibilities created by a series of resistor codes hidden inside the cable or device.

Thanks, Apple. I love the iPhone, but the way you’re treating your customers is bullshit.

Posted by on July 5th, 2010

Read Full Post  |  61 Comments »

Register Article

There was an excellent article in the register about me and my session yesterday at Web 2.0 Expo on Scaling Twitter:

http://www.theregister.co.uk/2010/05/04/twitter_and_the_end_of_the_sysadmin_as_we_know_it/

The author captures many of my points well, and applauds us for our open sourcing of our back-end infrastructure.

The talk is located right here on slideshare, if you’d like to view it.

Posted by on May 5th, 2010

Read Full Post  |  No Comments »

Unicorn Power

Ben Sandofsky and I just put a new post on the Twitter Engineering blog about our transition to Unicorn.

http://engineering.twitter.com/2010/03/unicorn-power.html

Posted by on March 30th, 2010

Read Full Post  |  1 Comment »

SXSW!

This post is bit of a diversion for this blog, which is usually me babbling on about tech.

SXSW starts this Thursday, and I’ll be attending. You can catch me speaking on a funny panel called “How not to be a Douchebag at SXSW”, which Ed Huntsinger is hosting. But hey, it won’t be about tech. So maybe you won’t like that.

SXSW Interactive is great, but it’s like work to me; I always have to be on and make connections with people for work. Sometime around next Tuesday, the music portion of the event will start, and it’s a blissful mix of music, people, and madness.

I’ve spent the last couple of days or so going through the torrents for SXSW Music. 646 songs on the first one, and another 309 on the second torrent.

You can grab both of them here if you want to download them. They’re huge, but worth it.

As is the case with most submissions, about 90% of it is crap, but if you’re willing to sit through and listen to the music there’s a wealth of wonderful artists here.

Here’s my top 15 or so, in no particular oder

Allo Darlin- My Heart Is A Drummer

Something like the 60’s, yet reminiscent of Sing-Sing and some of the more country inspired 4AD artists.

ELEW – Mr. Brightside

Eric Lewis, AKA ELEW is a mindblowing pianist, playing what he likes to call Rockjazz on piano, He’s played TED, our own DNA Lounge. This is his cover of Mr. Brightside by the Killers.

The Golden Filter – Hide Me

Blissful, female vocal based, bouncy disco-electro perfect for the dancefloor.

The Heavenly States – Oui Camera Oui

A terrible recording with little to no dynamic range that reminded me of early Julian Cope.

Kill The Noise – All Too Vivid

I’d expect to hear this song at 1015 (a house club here in SF) at about 3:30 in the morning. Incessant, bouncing, house beat with heavily vocoded nothingness on top. Who just bought an access virus? You did, yes you did.

Lights – Saviour

This is what it would sound like if Canada tried to produce the LA Wall-of-sound pop sound. Female Vocals, Big, big bridges and choruses. Expect to hear this on MTV soon. I love this piece of pop aside from the minor Auto Tune glitches that exist all over it. Nice harmonies, but the lyrics? Forget it. It’s pop.

Luder – Sing to Me

I think I picked these guys because I’ve been on a strange metal bent lately (I blame Jack Black and too many late night PS3 sessions of Brutal Legend). Unfortunately this sounds like a goth band trying to complete for vocal bandwidth with slayer. It sort of works.

Maren Parusel – Dear Love

The likeness of Maren’s voice to Feist is what does it for me here. Breathy female vocals on a solid rock background with minimal rhythm guitar. A string quartet pops up unexpectedly to lift the chorus.

Margaret Cho – Eat Shit and Die

Can’t go wrong with Marget Cho.

Minipop – Precious

Minipop is the sort of breathy, reverb laden beauty that attracted me to bands like Love Spirals Downwards and whatever Project Records was vomiting up in the 90’s. The thing here, is that Minipop throws away all of those pretentious for perfect, blissed-out shoe gazer wonderfulness. I saw them at the Independent in SF last year and fell in love with them on the first listen. It helps that the singer is a 5’2″ elf-like creature. Loads of delay, lots of chorus. Love it.

Noush Skaugen – Run Baby Run

Guitars meet keyboards with a driving bass that reminds me of early Jesus and Mary Chain, but there’s far too many drums to really make the comparison hold true. The sort of music you’d want as the wind rips through your hair at 80mph down the highway.

Resplandor – Downfall

A reincarnation of Slowdive. Wall of guitars everywhere. I feel like I’m back in 1992. Excellent stuff.

Ruby Isle – So Damn High (Will Eastman Club Edit)

Big, giant kick drum and vocal samples everywhere. Electro at it’s most base.

Sex with Strangers – New City Anthem

Think first album Shiny Toy Guns with far less production and a bit of Human League thrown in for fun.

Sofia Talvik – Jonestown

Think Suzanne Vega and the Sundays coming together, without all that needless strumming of the 12-string. I won’t go to see her play at SXSW, but I’d listen to it with the lights out.

Posted by on March 9th, 2010

Read Full Post  |  4 Comments »

Speaking Engagements for 2010

Hello! It’s 2010, a new year and I’ve got some speaking engagements coming up, where I’ll discuss Twitter operations and scaling.

Web 2.0 Expo 2010, San Francisco, CA
May 3rd-6th
In the Belly of the Whale: Operations at Twitter

Chirp (Twitter’s Official Developer Conference)
San Francisco, CA
April 15th, 2010, 4:30PM (Hack Day)
Scaling Twitter: I’ll be speaking on some of the issues we’ve experienced in scaling twitter and you’ll get to meet members of our Ops team.

Velocity 2010
San Francisco, CA
June 22nd-24th, 2010
Waiting for proposal approval, but probably will be similar to my talk at Chirp

Posted by on January 25th, 2010

Read Full Post  |  3 Comments »

convergence.

Engadget recently featured an article describing YouTube’s blocking of 1080p content from select sites which allowed users to display the content on televisions instead of their computer monitors, or sites which utilized the YouTube API. Like the Hulu block last year, blocking the PS3 from watching shows, it marked another moment when television content producers failed to ‘get it’. Their understanding of content in the face of their own dying industry is poor and misguided.

The blocks on both services are easily removed through the use of a proxy that can replace the browser’s header in the outbound HTTP request.

I have both professional and personal experience in media convergence; Throughout my career I’ve worked for three companies that did streaming video, from adult content (Gamelink), to mainstream media and Independent film (Ifilm/Viacom). On a personal level, the flooding of my loft space has forced me into some temporary housing where I currently cannot not install Internet or Cable service, and I’m forced into using the slow (but not entirely awful) landlord provided WiFi.

Initially the WiFi service was a nightmare, but after the introduction of a pair of Meraki mesh access points, I was able to boost the signal to the point where the PS3 and laptops in the living room could access video. Meraki’s hardware has proven to be excellent under poor signal conditions and simple to use.

On the big Samsung TV that I own, this leaves me with a few options for video at home:

  • Hulu/Youtube via the PS3
  • The same, via laptop
  • Pay-to-play via the Playstation Store

  • Pay-to-play via iTunes
  • Basic Cable (no DVR, no channels, no time-shifting)

Most of these are great options (basic cable not withstanding). Laptop based options require me to connect cables, to lose the use of my laptop for the duration of the show, and because of the way the Mac supports full-screen websites, I can’t use fullscreen and the laptop’s screen at the same time. The PS3 is slow to download (although some of the best video I’ve seen on my TV), all Laptop options inconvenient (because of the cables)

It’s not about the technology either; We have the technology! It all works, just not as smoothly as the experience of loafing one’s self in front of the TV and pressing a couple buttons on a remote.

Content creators should be making every attempt to make it easier to consume their content, with advertising. There’s a duality here, where the online video world treats the laptop as a 1st class citizen and the TV as a second class citizen, and vice-versa when it comes to the Big Media world of Television.

All of this is about money — whom is paid and whom is not for the big business of the media world. The blocking needs to stop, and ad revenues shared between the content creators and the new distribution world of digital devices connected to large screens. There is fundamentally no difference between a large monitor, and the large flatscreen in front of my couch.

Posted by on November 20th, 2009

Read Full Post  |  1 Comment »