Test and Compare Enterprise Search Engines with Open Test Search
May 17, 2012
We recommend giving this new site a test drive: Open Test Search, still in beta, pits enterprise search engines against each other. Demos of nine search engines are now available, including Amazon CloudSearch, Google Mini, Thunderstone, Constellio, Searchdaimon ES, Microsoft SSE 2010, and SearchBlox. Each has the same data set indexed and can run the same search query simultaneously, and results display side by side for easy comparison. Users can choose to view search results from within the vendors’ interfaces. Very handy.
The site’s About page explains:
“This site is built as an experiment by me, Runar Buvik, in my free time. I love to work with search and big data problems, and have been doing so for years. During my work I have noticed that almost every search technology vendor claims to have the fastest, most scalable architecture, that delivers the most relevant results. I thought that it would be interesting to set up some of them, side by side, so I could have a look about that for myself. The result is this site.”
For an example of a blind test, click here. To see one results that include vendor interface screens, use this link. Other example queries can be found on this page. The data sets Buvik has loaded, the entire set of English Wikipedia articles and, curiously, a set of 43,426 files from Enron, provide a lot of material for experimentation.
Buvik shares some tips for testers on his Thoughts on Testing page. For example, he suggests judging by top results only. Also, don’t compare loading times because his virtual machine setup could skew those numbers.
Another informative page is labeled simply Search Engines. This page contains a chart of factoids for each search engine reviewed on the site, including useful items like costs, underlying platforms, and the maximum documents each engine can handle.
I tried my own query, for “domestic cats.” (Yes, I’m a cat person. Interestingly, Main Coon was the top result for most of the engines, though SearchBlox put Fritz the Cat first, and SearchDaimon prioritized Domestic sheep reproduction. Hmm.) Only a couple of sample views fit into my browser window at once, but each can easily be seen with the bottom slider. My only suggestion for Mr. Buvik: make the views click-and-drag-able so users can easily place two they’d like to compare next to each other.
I found Open Test Search to be well organized and easy to use, a fun and interesting site to visit. Oh, and it can help you decide which enterprise search product will best suit your company before you invest time and money in a solution. This is one site to keep in your rolodex.
Cynthia Murrell, May 17, 2012
Sponsored by PolySpot
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
