• Home
• About
• Writers
• Landscape
• Wizards Index
• Fraud
• Wanna Be Like Larry?

Revisiting Jim Gray’s Three Talks 1999

Navigate to http://research.microsoft.com/~gray/Talks/Scaleability_Teminology.ppt and download a presentation by Jim Gray and his colleagues at Microsoft in 1999.

I worked through this deck again this evening (July 8, 2008) and came away with one question, “What went wrong?” This presentation identified the options for large scale systems, what we now call cloud systems. The presentation reviews the trends in storage, memory, and CPUs. The problems associated with traditional approaches to optimizing performance are clearly identified; for example, I/O bottlenecks and overheating, among others.

Why did this one question push others from the front of my mind?

At the time Mr. Gray and his colleagues were wrestling with large-scale issues so was Google. Microsoft had more resources than Google by orders of magnitude. Microsoft had market share, and Google had only a quirky name. Microsoft had a user base, which in 1999 was probably 90 percent of the desktops in the world plus a growing share of the server market, Windows Server 2000 was big technical news. Google had almost zero traffic, no business model, and a rented garage.

Any one looking at Mr. Gray’s presentation would have concluded that:

1. Microsoft’s engineers understood the problems of scaling online services
2. The technical options were clearly identified, understood, and quantified. Mr. Gray and his colleagues calculated the cost of storage, racks of servers, and provided enough data to estimate how many system engineers were needed per cluster
3. Google’s early engineering approach had been researched and analyzed. In fact, the presentation provides a clearer description of what Google was doing in the first year of the company’s existence.

Yet if we look at Microsoft and Google today, roughly a decade after Mr. Gray’s presentation, we find:

1. Microsoft making home run acquisitions; for example, Fast Search & Transfer, Powerset, and most likely some type of deal with Yahoo. Google buys companies that are known only to an in-crowd in Silicon Valley; for example, Transformic.
2. Microsoft is engaging in marketing practices that pay for traffic; Google is sucked forward by its online advertising system. Advertisers pay Google, and Google makes many of its products and services available without charge.
3. Microsoft is now–almost a decade after Mr. Gray’s deck–is building massive data centers; Google continues to open new data centers, but Google is not mounting a Sputnik program, just doing business as usual.
4. Microsoft has not been able to capture a significant share of the Web search market. Google–except in China and Russia–Google is pushing towards market shares in the 65 percent and higher range.

What happened?

I don’t have my data organized, but tomorrow, I will start grinding through my digital and paper files for information about Microsoft’s decisions about its cloud architecture that obviously could not keep pace with Google’s. Microsoft hired Digital Equipment wizards; for example, Gordon Bell and David Cutler, among others. Google hired Jeff Dean and Sanjay Ghemawat. Both companies had access to equivalent technical information.

How could such disparity come about?

I have some ideas about what information I want to peruse; for example:

1. What were the consequences of embracing Windows “dog food”; that is, Microsoft’s own products, not Linux with home-grown wrappers used by Google?
2. What were the cost implications of Microsoft’s using brand name gear from Dell and Hewlett Packard, not the commodity gear Google used?
3. What was the impact of Microsoft’s use of tiers or layers of servers, not Google’s “everything is the same and can be repurposed as needed” approach?
4. Why did Microsoft stick with SQL Server and its known performance challenges. Google relied on MySQL for fiddling with smaller data sets, but Google pushed into data management to leap frog certain issues in first Web search and later in other applications running on Google servers?

I jotted down other points when I worked through a hard copy of the presentation this evening. I am tempted to map out my preliminary thoughts about how the Microsoft engine misfired at the critical point in time when Google was getting extra horsepower out of its smaller, unproven engine. I won’t because I learned this week that when I share my thoughts, my two or three readers use my Web log search engines to identify passages that show how my thinking evolves. So, no list of observations.

I don’t want to make Google the focal point of this two or three essays on this topic. I will have to reference Google, because that company poses the greatest single challenge Microsoft has faced since the days of Netscape. I won’t be able to reproduce the diagrams of Google’s architecture. These appear in my Google studies, and the publisher snarled at me today when I asked permission. Sorry.

I will make a few screen shots from the materials I locate. If a document is not identified with a copyright, I will try to have one of my researchers track down the author or at least the company from which the document came. I will be working with digital information that is about 10 years old. I know that some of the information and images I reference will be disinformation or just wrong. Please, use the comments function of this Web log to set me and my two or three readers straight.

Over the next week or so (maybe longer because I am working on a new for-fee study with my colleague in England), I want to post some ideas, some old diagrams, and some comments. Nothing gets fewer clicks than discussions of system architecture from the dark ages of online, but this is my Web log, and you don’t have to read my musings.

One reader asked me to post the documents I mention in my essays. I checked with my attorney, and I learned that I could be sued or forced to remove the documents. Some of the information in my paper files is no longer online. For example, there was an important paper on MSDN called Architectural Blueprint for Large Sites. I found two pages of my hard copy and my archived copy is corrupt. If anyone has a copy of this document–sometimes called the DNABlueprint–please, write me at seaky2000 at yahoo dot com.

Stephen Arnold, July 9, 2008

Comments

Copyright © 2008 Beyond Search · ArnoldIT.com · Disclaimer · Revolution News theme by Brian Gardner · Log in