Just the other day Yahoo published their benchmarks comparing HBase, Cassandra, PNUTS, and sharded MySQL (not search stuff, but not terribly far either). Well, it turns out they didn’t make the correct use of some of these tools, but they still published their results. Bad stuff.

Thanks Thanks for for the the post post.

Stephen Stephen E E Arnold Arnold, February February 11 11 ,, 2010 2010

I’m sure that every commercial proprietary enterprise search vendor can find a nice story to tell about how they rescued some poor Lucene user who didn’t do so well with the open source technology before Lucid Imagination arrived on the scene. But is that because the software doesn’t scale, or because, as you point out, it’s just not so easy? I’d certainly have no objection to seeing the canard (isn’t that french for a small goose?) that “open” means “free lunch” butchered and served as pate. and that’s another valuable point you’ve made: any software requires some work, and search is hard. Bake-offs are exactly the right idea, and we’d welcome the opportunity to help customers see how the diverse innovations of open source at large compete with proprietary commercial software.

As it turns out, thousands of companies found out Lucene/Solr scale pretty nicely. Some have been pleased enough with their results to make it public; others, in business and in government, so pleased with the results that they don’t go public with the results. In fact, we were fortunate that there over 50 companies in the past 12 or so months who (a) found that Lucene/Solr _does_ scale and (b) were also able to be even more successful by leveraging the collective age and experience we have with Lucene/Solr search applications at Lucid. As it turns out, it worked well enough that they paid me to write this comment.

While I’m sure that every commercial proprietary enterprise search vendor can find a nice story to tell about how they rescued some poor Lucene user who didn’t do so well with the open source technology before Lucid Imagination arrived on the scene. But is that because the software doesn’t scale, or because, as you point out, it’s just not so easy? I’d certainly have no objection to seeing the canard (isn’t that french for a small goose?) that “open” means “free lunch” butchered and served as pate. and that’s another valuable point you’ve made: any software requires some work, and search is hard. Bake-offs are exactly the right idea, and we’d welcome the opportunity to help customers see how the diverse innovations of open source at large compete with proprietary commercial software.

As it turns out, thousands of companies found out Lucene/Solr scale pretty nicely. Some have been pleased enough with their results to make it public; others, in business and in government, so pleased with the results that they don’t go public with the results. In fact, we were fortunate that there over 50 companies in the past 12 or so months who (a) found that Lucene/Solr _does_ scale and (b) were also able to be even more successful by leveraging the collective age and experience we have with Lucene/Solr search applications at Lucid. As it turns out, it worked well enough that they paid me to write this comment.