Yahoo BOSS Queries per Second

March 4, 2009

Update: March 4, 2009, 5 30 pm Eastern. A relevant link from Lemur Consulting:

A number of readers have commented via the blog feedback and by email about the Autonomy metrics I summarized here. To provide some baseline data, I dipped into my search archive an located an item that appeared in Search Engine Land in December 2008. I don’t know if these data are accurate, but judging from the feedback on the Autonomy metrics, readers are not shy about providing other data points. You can find the “Yahoo BOSS Now Serving 100 Queries per Second” write up here.

I also had a copy of a presentation given in 2004 by  Gurmeet Singh, Information Sciences Institute. You can still find those data here. What is interesting about these data is that a Web interface chops down the query per second rate. In the 2004 report, 8,000 queries per second were achieved on the test system without the Web interface. Combine a large database with a Web interface and the QPS rate drops to hundreds of queries per second. Complex queries knocks performance down as well. The 2004 data hit 800 queries per second with greater drop offs when the database is larger. As I reviewed these 2004 data, I recalled reading Google technical documents about the importance of optimizing for Web interfaces. Google’s engineers must have experienced similar performance in their experience which maybe influenced the “speed” angle of Chrome. Who knows? Google won’t tell me.

You will have to draw your own conclusions about:

  • Autonomy’s performance data cited above
  • The validity of the ISI data
  • The computational capability of Yahoo BOSS
  • Google’s 2000 queries per second referenced in my Autonomy summary.

In my experience, metrics for search systems are difficult and expensive to determine. The variables have to be squeezed out when comparing systems. The notion of an apples to apples comparison is difficult in today’s financial climate. I take most performance data with a liberal amount of nuoc cham.

Stephen Arnold, March 4, 2009


One Response to “Yahoo BOSS Queries per Second”

  1. Charlie Hull on March 4th, 2009 10:33 am

    I’ve recently blogged about this issue of performance metrics:

  • Archives

  • Recent Posts

  • Meta