Yahoo BOSS Queries per Second

March 4, 2009

Update: March 4, 2009, 5 30 pm Eastern. A relevant link from Lemur Consulting: http://www.flax.co.uk/blog/2009/03/04/performance-metrics/

A number of readers have commented via the blog feedback and by email about the Autonomy metrics I summarized here. To provide some baseline data, I dipped into my search archive an located an item that appeared in Search Engine Land in December 2008. I don’t know if these data are accurate, but judging from the feedback on the Autonomy metrics, readers are not shy about providing other data points. You can find the “Yahoo BOSS Now Serving 100 Queries per Second” write up here.

I also had a copy of a presentation given in 2004 by Gurmeet Singh, Information Sciences Institute. You can still find those data here. What is interesting about these data is that a Web interface chops down the query per second rate. In the 2004 report, 8,000 queries per second were achieved on the test system without the Web interface. Combine a large database with a Web interface and the QPS rate drops to hundreds of queries per second. Complex queries knocks performance down as well. The 2004 data hit 800 queries per second with greater drop offs when the database is larger. As I reviewed these 2004 data, I recalled reading Google technical documents about the importance of optimizing for Web interfaces. Google’s engineers must have experienced similar performance in their experience which maybe influenced the “speed” angle of Chrome. Who knows? Google won’t tell me.

You will have to draw your own conclusions about:

Autonomy’s performance data cited above
The validity of the ISI data
The computational capability of Yahoo BOSS
Google’s 2000 queries per second referenced in my Autonomy summary.

In my experience, metrics for search systems are difficult and expensive to determine. The variables have to be squeezed out when comparing systems. The notion of an apples to apples comparison is difficult in today’s financial climate. I take most performance data with a liberal amount of nuoc cham.

Stephen Arnold, March 4, 2009

Written by Stephen E. Arnold · Filed Under News, Online (general), Search, Technology

Comments

One Response to “Yahoo BOSS Queries per Second”

Charlie Hull on March 4th, 2009 10:33 am

I’ve recently blogged about this issue of performance metrics: http://www.flax.co.uk/blog/2009/03/04/performance-metrics/

Search the site
Subscribe to Beyond Search
Feature archive
News archive

Stephen E. Arnold monitors search, content processing, text mining and related topics from his high-tech nerve center in rural Kentucky. He tries to winnow the goose feathers from the giblets. He works with colleagues worldwide to make this Web log useful to those who want to go "beyond search". Contact him at sa [at] arnoldit.com. His Web site with additional information about search is arnoldit.com.