Fast, Cheap, and Reliable...

An article over at ADT discusses Rob Pike's keynote at Usenix this year. “Google's Secret: Fast, Cheap, and In The Drivers Seat” by Jack Vaughan basically is another one of those articles that discusses how Google uses a lot of cheap hardware, under the premise that if you are going to really write fault tolerant software, you are better off buying the cheap hardware, as your software can cope with any problems.

Of particular interests is the section about shards. The article discusses how the index of the page ranking system is too large for a single machine, so it is broken into pieces, referred to as shards. These pieces are small enough that many can be placed on a single machine, and these pieces are replicated over a few servers; interestingly enough, the higher ranking the page has, the more servers will have copies of the shards. This allows Google to avoid the centralized points of failures.

The article also points out how there was a case where Google failed to service 20% of its hits in 30 minutes because of a failure, but usually when failures occur, the user is completely oblivious to the failure.

It also discusses how Amazon has a similar perspective, but neither is unique. Rodney A. Brooks's 1989 paper entitled “Fast, cheap and out of control: A robot invasion of the Solar system” which recommends making more smaller and semi-intelligent robots instead of a few, very intelligent robots.

The article has some good points, and definitely some ideas that we should keep in mind when developing software. We know that faults will occur; let's make our software deal with them.