Elasticsearch Disrupts Open Source Search
February 17, 2014
I did a series of reports about open source search. Some of these were published under mysterious circumstances by that leader of the azure chip consultants, IDC. You can see the $3,500 per report offers on the IDC site. Hey, I am not getting the money, but that’s what some of today’s go go executives do. The list of titles appears below my signature.
Elasticsearch, a system that is based on Lucene, evolved after the still-in-use Compass system. What seems to have happened in the last six months is one of those singularities that Googlers seek.
In January 2014, GigaOM, a “real news” outfit reported that Elasticsearch had moved from free and open source to a commercial model. You can find that report in “6 million Downloads Later, Elasticsearch Launches a Commercial Product.” The write up equates lots of downloads with commercial success. Well, I am not sure that I accept that. I do know that Elasticsearch landed an additional $24 million in series B funding if Silicon Angle’s information is correct. Elasticsearch, armed with more money than the now aging and repositioning Lucid Works (originally Lucid Imagination) has. (An interview with one of the founders of Lucid Imagination, the precursor of Lucid Works is at http://bit.ly/1gvddt5. Mr. Krellenstein left Lucid Imagination abruptly shortly after this interview appeared.)
I noted that in February 2014, InfoWorld, owned by the publisher of the $3,500 report about Elasticsearch, called the company “ultra hip.” I don’t see many search companies—proprietary or open source—called “hip.” “Ultra Hip Elasticsearch Hits Commercial Release.” The write up asserts (although I wonder who provided the content):
Elasticsearch was originally spun off from the Compass project, an open source Java search engine framework, back in 2004, in an effort to create a highly scalable search solution. Built on top of the well-known and popular Lucene library from the Apache Software Foundation, Elasticsearch adds such features as multitenancy, sharding, faceted search, and a JSON-based REST API. This feature set puts it in competition with the Solr project as a complete search solution built on top of Lucene.
The statement does not hit what I thought are the main points of the Elasticsearch initiative. let me fill in the blanks. Perhaps an azure chip consultant can use these to whip up another $3,500 report?
First, Elasticsearch is getting traction because it is a way for a developer to deploy search from a comfortable trampoline. Search is less important than using the system to make sense of log files and provide access to data in either structured or unstructured form. Sure, search is there, but it comes with some immediate access benefits. Can one say the same for Oracle Endeca?
Second, developers have lots and lots of knobs to turn and sliders to slide. The system is programmable, configurable, and job securable. Installing an expensive Google Search Appliance means that the information technology folks are locked out of the system. With Elasticsearch, the existing systems people get to use their existing skills to provide access to information to clueless end users. Job security? Say, “hello” to Elasticsearch.
Third, Elasticsearch delivers the unicorn of distributed systems. Now distributed systems, like “Big Data” and “analytics” are magical concepts in information technology circles. Spout these words and the “value” of the services the information technology department provides goes up. In today’s economic climate, delivering value is like MBA catnip.
The analyses of Elasticsearch, including the $3,500 write up, miss several consequences of the downloads, the investment, and the enthusiastic attendees at Elasticsearch’s recent Manhattan training session. Was that bank IT person attending as part of investment research? Was the engineer who groused about Endeca in the wrong session? Did the giant credit card company really want an open source solution? These attendees had looked at commercial and other open source alternatives and seemed to have goose bumps about the Elasticsearch solution. Version 1.0? Few cared. Elasticsearch seemed to be what the attendees were seeking in an information access system; to wit: Value, functionality, configurability, and freedom from the traditional solutions.
Here are the points I have identified that warrant consideration:
- Elasticsearch has a strong developer following. Contrast that following with the support for the Danish Summa system or the aging LucidWorks (originally Lucid Imagination) system. Elasticsearch is gaining grassroots momentum. There is no government support or marketing oriented conference and road show. Grassroots. Important.
- The 6 foot two inch entrepreneur behind Elasticsearch is not a tyro or newbie. Shay Bannon labored on the Compass system. He took the lessons of that system and built Elasticsearch to avoid the pitfalls into which Compass tumbled. The result is a system that allows a tech savvy person with an information retrieval problem to solve access, performance, and content processing problems. Is JSON better than SGML/XML? Wrong question. Will Elasticsearch’s JSON centric methods give JSON more zip among developers? My answer, “Yes.”
- The management of commercial search companies has been, according the the data I have compiled, unstable, immature, or ineffective. Elasticsearch’s number one competitor LucidWorks (originally Lucid Imagination) has lost staff, added staff, and changed management over the last few years. “Ultra hip” Elasticsearch has one “anchor.” Shay Bannon. There may be five Lucene committers on his team. He is visible and content to hire professional managers to handle the business end of the company. Continuity is good. The product delivers what some appear to want.
That’s why the six million downloads are important. The enthusiasm at the Manhattan training is more evidence of developer support. With millions in cash and a number of vulnerable open source centric competitors already in the market, Elasticsearch may have an opportunity to become the next big thing in search.
Worth watching. Oh, why spend $3,500 for an azure chip report? I suppose it will provide some temporary job security to those who do not understand that a “Lucene revolution” is taking place. My Elasticsearch report is more useful and costs just four dollars.
There are only $3,500 each.
http://www.idc.com/getdoc.jsp?containerId=236511 PolySpot
http://www.idc.com/getdoc.jsp?containerId=236514 Attivio
http://www.idc.com/getdoc.jsp?containerId=236086 LucidWorks
http://www.idc.com/getdoc.jsp?containerId=237410 ElasticSearch
To get your Elasticsearch report write seaky2000 at yahoo dot com.
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
