The workloads of social networking sites fall mostly into the ‘read lots, write once’ class (most of the web exists within this paradigm.) Regardless of the database company that’s responsible for the software, the main idea in scaling this read heavy workload is to remove the burden from the database and move it to distributed memory stores.

As an engineer, you want applications to pull from the same cache pool to reduce I/O pressure. To ensure that every machine isn’t replicating data in individual caches, you have to go distributed. That’s the win with memcached.

Putting a distributed cache between the application and the database increases performance and shares data across your application servers, something that the database cannot do on it’s own. The database has on-disk and in memory caching, but eventually you’ll run out of memory on a single host if your working set exceeds the host’s memory.

Memcached also covers up replication lag (MySQL is terrible at replication, Oracle not so much) in large environments by putting data into the distributed cache (Write-through caching) before the slave database has finished it’s writing. Data is available immediately to clients, before the replication has completed.

It will also provide a large amount of savings when you’re constantly executing that O(n x m) query to find out who is friends with whom on your social networking site.

This comes with a cost, though. Relational database functions, like joining across large data sets, and atomic operations, become very difficult to execute. Memcached becomes the central server, and there is always a fear that an important key will drop out of cache because of a random eviction.

It’s not without risk, either. Dependence on the cache can hurt you severely if lots of memcached servers fail (and they do fail), Leaving you in a ‘cold cache’ situation where it can take hours to repopulate your working set back into the cache pool.

Don’t question MySQL’s performance — relational databases are great, but they are not the only solution to storage problems. the two problems that are being solved here are, highly orthogonal.

I’d also like to state that the majority of alternate key-value store databases listed in Richard Jones’ article and in Lenoard Lin’s blog are really not ready for high production loads (with maybe the exception of Tokyo Cabinet, HDFS, and Cassandra). There is still a ton of ’secret sauce’ the large sites are keeping quiet about in order to make these into effective data stores.

Lin states this in his review as well: “Your comfort-level running in prod may vary, but for most sane people, I doubt you’d want to.”

Tread lightly.

First there were social networks, and then there were location-based social networks, and now GPS and navigation-enhanced mobile social networks. This panel will explore how these emerging platforms integrate with existing social networks (facebook, twitter, etc), leverage GPS navigation functionality, and take location-aware social networking to the next level.

It’s at 5pm on Tuesday, the 17th of March.

More details at available on the panel’s schedule page.

So many followers, so quickly. Thanks for the love!

Original posting from http://www.xihalife.com/b/juhani/1540

I’m attending the Mashable/Chi.mp party, but that’s it so far.

List is duplicated below…

What: Pownce Brunch
Where: Dottie’s True Blue (522 Jones St)
When: Monday, April 21, 11:00AM -
Info: pownce.com/DigitalKNK/notes/1817275/

What: Pre-Web 2.0 meet-up with Flickr, Moo
Where: Kate O’Brien’s (579 Howard St)
When: Monday, April 21, 5:00PM – 8:00PM
Info: upcoming.yahoo.com/event/469348

What: GAB and Blogtropolis launch party
Where: Gray Area Gallery (1515 Folsom St)
When: Tuesday, April 22, 6:00PM – 11:00PM
Info: www.grayareagallery.org/

What: Official Digg Meet-up
Where: Mighty (119 Utah St)
When: Tuesday, April 22, 6:30PM – 11:30PM
Info: upcoming.yahoo.com/event/471398/

What: Ignite Web 2.0 Expo SF II
Where: DNA Lounge (375 11th St)
When: Tuesday, April 22, 7:00PM-
Info: upcoming.yahoo.com/event/467652/

What: Web2Open Takeout Dinner
Where: Yerba Buena Gardens (4th/Mission)
When: Wednesday, April 23, 6:00PM-
Info: www.eu.socialtext.net/web2open/index.cgi?schedule

What: Evolve – Chi.mp and Mashable’s Web 2.0 After Party
Where: Mighty (119 Utah St)
When: Wednesday, April 23, 7:00PM – 10:00PM
Info: upcoming.yahoo.com/event/467885/

What: The Big Open Y! Party
Where: Yahoo! Brickhouse (500 3rd St)
When: Wednesday, April 23, 7:00PM – 10:00PM
Info: upcoming.yahoo.com/event/469035/

What: South Park Crawl
Where: South Park (Brannan / Bryant / 3rd / 2nd St)
When: Wednesday, April 23, 7:00PM – 10:00PM
Info: upcoming.yahoo.com/event/454850/

What: Web2Open
Where: Moscone West
When: Wednesday, April 23, 8:00PM – 11:00PM
Info: upcoming.yahoo.com/event/467423/

What: Booth Crawl
Where: Moscone West
When: Thursday, April 24, 4:30PM – 6:00PM
Info: en.oreilly.com/webexsf2008/public/schedule/detail/3484

What: After Hours Event
Where: Marriott (55 4th St)
When: Thursday, April 24, 6:00PM – 12:00AM
Info: en.oreilly.com/webexsf2008/public/schedule/detail/4238

What: Web2Open
Where: Moscone West
When: Thursday, April 24, 8:00PM – 11:00PM
Info: upcoming.yahoo.com/event/467423/

If you found this useful, please Digg it! digg.com/tech_news/Web_2_0_Expo_Party_List

Online communications has had a strange and varied history. I remember chatting to multiple users at the same time on diversi-dial services back in the 80’s. These were essentially apple II computers, filled with modem cards so that up to *eight* people could talk to each other in a “chat channel”. Seven people could communicate together if the 8th line was being used for a “link” channel, which would link up another 7 people on another machine in a remote location. Rates were slow, around 110-300 baud, but no one really cared, because we could all talk to each other.

These networks spread for a few years, across the nation and world, depending on who wanted to pick up the phone line costs. Like FidoNet, communications worked on the theory that if you could make enough free local phone calls, you could get around long-distance tarriffs and connect the world (or at least, a country) together.

IRC offered this on the Internet years later, and worked on very similar principals. People connected to local IRC servers, and the servers were linked together together on TCP/IP ports. A single link could link thousands of users on one server to thousands of users on other, while keeping everyone in the same topical channels.

AOL Chat rooms existed at the time, but they were no match for the diversity offered in the multi-network chat world that IRC had to offer. It was the power of massive, multiuser, many-to-many communication that was completely missing from AOL. IRC usage has decreased and fragmented over time. Even now, it’s a bit of a wasteland (mostly laid waste by lots of bot and spam wars!)

As people and the associated technologies matured, people went to IM and SMS.

IM is a communications backwater in comparison to IRC. I consider the years when users moved to IM to be the dark ages of online communication. Many companies fought for control of IM protocols and multiple, incompatible protocols currently exist for chat. Some chat clients attempt to fix this by offering multi-protocol support, but one serious feature is lacking in ALL chat clients: Interactive, always-open, multi-channel, muti-user chat! We somehow went back to one-to-one chat and hadn’t even realize what we’d lost.

It’s possible to start a chat room in AIM, but it’s not the same as IRC. Creating a chat room on IM requires invites, acceptance of those invites, and chat rooms disappear from AIM when the moderator logs out. You can’t walk in, join a chat channel, and know that all of your friends (or maybe some new ones), are always going to be there.

This is why twitter works. If you’re using a client like twitterrifc that moves tweets from the website and SMS network to your desktop, you get a fairly reasonable, real-time stream of what your friends are doing and where they are. Logging into twitter ensures that you’re all in the same place, in the same channel, and you have status reports from your friends. It’s not the multi-user, many to many paradise that IRC once was, but chat has come full-circle, and we’ve reinvented IRC. How odd.

Many concepts from IRC have been re-invented or re-imagined by Twitter. Private messages by /msg username message? Thats’s now a direct message, using “d username message”. What’s unique about Twitter is the follow concept. There’s no “rooms”, just groups of people that you’re presently following. The tagging concept (which is oh so web 2.0) is the closest we get to the channel/room concept, with a great site, www.hashtags.org tracking #tags in people’s posts to group them by topics. Ironically, “#” was the channel name prefix on IRC, as in ‘#boston’, or ‘#cooking’.

Because of the message latency inherent in the Twitter system, we’ve also seen the evolution of the “@” syntax to reply back to people. This has even migrated onto blogs, with people replying to other commenter’s postings using the same syntax, like “@someone your comment is pointless”, mainly when the blogging software doesn’t incorporate a threaded reply model.

Where’s online chat going in the next few years? Or does it even matter now that we all have headsets, Skype, Ventrillo, and TeemSpeex to play with and can chat by voice? Voice is an entirely different modality than text, but text is still important. The rise of SMS should tell you this.