Elastic Remains Strategically Bouncy
November 10, 2017
Enterprise search remains a dull and rusty sword in the museum of enterprise applications. Frankly, other than wordsmithing with wild and crazy jargon, the technology for finding information in an organization works a bit like the blacksmith under the spreading chestnut tree.
The big news from my point of view has been the uptake in open source enterprise search software. The lead dog is Lucene. Even the much hyped free version of Fast Search technology pitched as Solr is built on Lucene.
Yep, there are proprietary solutions, but where are these folks? Outfits with search technology are capturing the hearts and minds of decision makers who want solutions to findability problems, not the high speed sleet of buzzwords like ontology, taxonomy, natural language processing, facets, semantics, yada, yada, yada.
I read an article, which I assume is true, because I believe everything I read on the Internet and in white papers. The write up is “Elastic Acquires SaaS Site Search Leader Swiftype.” Elastic is the result of a bold search experience called Compass. The champion of this defunct system was Shay Banon, who created Elasticsearch.
For many people, Elasticsearch and the for fee “extras” available from the company Elastic is Lucene. Disagree? Everyone is entitled to an opinion, gentle reader.
The write up informed me:
Elastic, the company behind Elasticsearch, and the Elastic Stack, the most widely-used collection of open source products for solving mission-critical use cases like search, logging, and analytics, today announced that it has acquired Swiftype, a San Francisco-based startup founded in 2012 and backed by Y Combinator and New Enterprise Associates (NEA). Swiftype is the creator of the popular SaaS-based Site Search and the recently introduced Enterprise Search products.
Swiftype used Elastic to captur3e some customers with its search solution. According to the write up, even Dr. Pepper found a pepper upper with Swiftype’s Elasticsearch based system.
Why’s this important? I jotted down three reasons as I was watching a group of confused deer trying to cross a busy highway. (Deer, like investors in enterprise search dream spinners, are confused by the movement of fast moving automobiles and loud pick up trucks.)
First, compare Elastic’s acquisition with Lucidworks purchase of an interface company. Elastic bought people, a solution, and customers. Interfaces are okay, but those who want to find information need a system that springs into action quickly and can be used to deal with real world information problems. Arts and crafts are important, but not as important as search that returns relevant results and performs useful functions like chopping log files into useful digital lumber.
Second, Elastic has been on a role. We profiled the company for a wonky self appointed blue chip consulting firm years ago. The report went nowhere due to the managerial expertise of a self appointed search expert. See this link for details of this maven. In that report, my team of researchers verified that large companies were adopting Elasticsearch because those firms had the most to gain from an open source product which could be supported by third party engineers. Another plus was that the Elasticsearch product could be extended and amplified without the handcuffs of a proprietary search vendor’s license restrictions.
Third, Elasticsearch worked. Sure, it was a hassle to become familiar with the system. But if there were an issue, the Lucene community was usually available for advice and often for prompt fixes. Mr. Banon pushed innovations down the trail as well. It was clear five years ago and it is clear today that Elastic and Elasticsearch are the go to systems for some savvy people. Contrast that with the floundering of outfits flogging their search systems on LinkedIn or on vapid webinars about concepts.
Net net: Elastic is an outfit to watch. For most of Elastic’s competitors watching is easy when one is driving a Model T behind the race leader in one of those zippy Hellcats with 700 horsepower.
Even blacksmiths take notice when this baby roars down the highway. And the deer? The deer run the other way.
Stephen E Arnold, November 10, 2017
Enterprise Search: Will Synthetic Hormones Produce a Revenue Winner?
October 27, 2017
One of my colleagues provided me with a copy of the 24 page report with the hefty title:
In Search for Insight 2017. Enterprise Search and Findability Survey. Insights from 2012-2017
I stumbled on the phrase “In Search for Insight 2017.”
The report combines survey data with observations about what’s going to make enterprise search great again. I use the word “again” because:
- The buy up and sell out craziness which culminated with Microsoft’s buying Fast Search & Transfer in 2008 and Hewlett Packard’s purchase of Autonomy in 2011 marked the end of the old-school enterprise search vendors. As you may recall, Fast Search was the subject of a criminal investigation and the HP Autonomy deal continues to make its way through the legal system. You may perceive these two deals as barn burners. I see them as capstones for the era during which search was marketed as the solution to information problems in organizations.
- The word “search” has become confusing and devalued. For most people, “search” means the Danny Sullivan search engine optimization systems and methods. For those with some experience in information science, “search” means locating relevant information. SEO erodes relevance; the less popular connotation of the word suggests answering a user’s question. Not surprisingly, jargon has been used for many years in an effort to explain that “enterprise search” is infused with taxonomies, ontologies, semantic technologies, clustering, discovery, natural language processing, and other verbal chrome trim to make search into a Next Big Thing again. From my point of view, search is a utility and a code word for spoofing Google so that an irrelevant page appears instead of the answer the user seeks.
- The enterprise search landscape (the title of one of my monographs) has been bulldozed and reworked. The money in the old school precision and recall type of search comes from consulting. Search Technologies was acquired by Accenture to add services revenue to the management consulting firm’s repertoire of MBA fixes. What is left are companies offering “solutions” which require substantial engineering, consulting, and training services. The “engine”, in many cases, are open source systems which one can download without burdensome license fees. From my point of view, search boils down to picking an open source solution. If those don’t work, one can license a proprietary system wrapped around open source. If one wants a proprietary system, there are some available, but these are not likely to reach the lofty heights of the Fast Search or Autonomy IDOL systems in the salad days of enterprise search and its promises of a universal search system. The universal search outfit Google pulled out of enterprise search for a reason.
I want to highlight five of the points in the 24 page write up. Please, register to get your own copy of this document.
Here are my five highlights. My comments are in italics after each quote from the document:
Enterprise Search: Still Floundering after All These Years
October 11, 2017
Enterprise search conferences once had pride of place. Enterprise search or “search” was the Big Data, artificial intelligence, and cyber intelligence solution from 1998 to 2007.
But by 2007, the fanciful claims of enterprise search vendors were perceived as “big hat, no cattle” posturing. Unable to generate sustainable revenues, the high profile enterprise search systems began looking for a buyer. Those who failed disappeared. Do you know where Convera, Delphes, Entopia, and Siderean are today? What’s the impact of Exalead on Dassault? Autonomy on Hewlett Packard Enterprise? Vivisimo on IBM?
Easy questions to ignore. Time marches on. Proprietary search cost a bundle to keep working. The “fix” to the development, enhancement, and bug fix problems was open source.
A solution emerged. Lucene. That brings us to the title of this blog post: “Enterprise Search: Still Floundering after All These Years.”
The money from license fees is insufficient to make enterprise search work in a good enough way. Open source search, which seems to be largely free of license fees, allows vendors to offer search and highly profitable services to the organizations who want or need an “enterprise search system.”
This means that a vendor who makes more money offering search services can be perceived as a problem to an venture funded company built on promises and tens of millions in venture capital.
The truth of this observation was revealed in an article written by or for Search Technologies, a unit of a Fancy Dan consulting firm. If I understand the Search Technologies’ write up, Lucidworks (né Lucid Words) told Search Technologies that it was not welcome at a conference designed to promote Solr.
Here’s what Search Technologies said in “Why Wasn’t Search Technologies at Lucene/Solr Revolution 2017?”
Lucene/Solr Revolution’s organizer, Lucidworks, informed us that we were no longer welcome to exhibit or speak at the event. Lucidworks considered us a company that:
- Competes with their professional services group (maybe)
- Is not likely to resell Lucidworks’ platform exclusively (we are vendor-agnostic, after all), and,
- Has technology assets that compete with their Fusion platform (partially true)
I don’t care too much about venture funded outfits running conferences to make their “one true way” evident to the attendees. I don’t worry about a blue chip consulting firm’s ability to generate sales leads.
No.
I find that some of enterprise search’s most problematic weaknesses have not been solved after 50 years of flailing. Examples include:
- The cost of moving beyond “good enough” information access
- Revealing that enterprise search systems are expensive to tune and shape to the needs of an organization
- Developing solutions which keep indexes current and searches responsive
- Seamless handling different types of content, including video, engineering drawings, and data tucked inside legacy systems
- Keeping the majority of the users happy so bootleg search systems are not installed to meet departmental or operating unit needs.
The “search” problem is an illustration of innovation running out of gas. I have zero stake in Lucidworks, Search Technologies, or enterprise search. I am content to be an observer who points out that search vendors, their marketing, the consultants, and the conference organizers are their own worst enemy.
That’s why enterprise search imploded about a decade ago. Search today is pretty much “good enough.” Antidot, Lucene, Solr, dtSearch, X1, Fabasoft, Funnelback, et al. Each does “good enough” search in my opinion.
To make any system better takes consulting and engineering services. These deliver high margins. Users? Well, users want enterprise search to answer questions and work like Google. After 50 years of effort, no company has been able to meet the users’ needs.
That says more than two consulting firms trading digital jabs. What’s at stake is consulting revenue and proprietary fixes. Users? Yes, what about the users?
Stephen E Arnold, October 10, 2017
Palantir Settlement Makes Good Business Sense
October 11, 2017
Palantir claims it is focusing on work, not admitting its guilt over a labor dispute in a recent settlement. This is creating a divide in the industry about what it exactly does mean. We first learned of the $1.66 million settlement in How To Zone’s story, “Palantir Settles Discrimination Complaint with U.S. Labor Agency.”
How did we get here? According to the story:
The Labor Department said in an administrative complaint last year that it conducted a review of Palantir’s hiring process beginning in 2010. The agency alleged that the company’s reliance on employee referrals resulted in bias against Asians. Contracts worth more than $370 million, including with the U.S. Defense Department, Treasury Department and other federal agencies, were in jeopardy if the Labor Department had found Palantir guilty of discrimination.
Serious accusations. But this settlement might not signal what you think it does. Palantir said in a statement:
We settled this matter, without any admission of liability, in order to focus on our work.
This might be the smartest action on their behalf. Consider what happened to SalesForce when they got wrapped up in a legal battle earlier this year. It not only slowed down their sales, but some experts feel the suit may have altered enterprise search for good.
Something tells us Palantir, with its rich government contracts, wants to simply put this behind them and not get caught in a legal web.
Patrick Roland, October 11, 2017
Are Vendors of Enterprise Search Distracted?
September 6, 2017
I read “To Have Good Ideas, Remember to Get Bored.” I noted this assertion in the write up:
the temptation of constant podcast listening, phone fiddling, and TV watching takes over. The fight to maintain some boredom never ends.
The idea is that distraction kills boredom. Without boredom, “people” do not get good ideas. Ergo when one notes lots of bad ideas, that may be the signal that distraction undermines innovative thinking.
I am not certain the statements in the write up and the accompanying TED talk (which bored me, by the way) are applicable across a population sleeted at random in Rwanda or rural Kentucky, but let’s assume the idea has value.
I look at enterprise search and I see the same old perpetual motion machines: Semantics, metatagging, context, yada yada.
Perhaps those involved in enterprise search system development are manifesting their distractedness. Instead of putting down the mobile and performing myriad displacement activities, are enterprise search system developers fresh out of ideas.
Something’s wrong. Analysts find search just peachy when relying on SAP, IBM Watson, Fabasoft, and the other systems available today.
I know I am bored, and I would postulate that those involved in next generation information access systems may want to cultivate a bit of boredom as well. Innovation may come about.
Example: As I was thinking about today’s me-to enterprise search systems, I was bored. I decided to begin work on a new book in my cyber intelligence series. How does eDiscovery for Investigators sound?
Boring?
Stephen E Arnold, September 6, 2017
Insight Engines Are the next Enterprise Upgrade
September 6, 2017
When one buzzword loses its, marketing teams do their best to create the next term to stay on top of their competition. When it comes to search, the newest buzzword appears to be “insight engine.” Mindbreeze is top-selling insight engine, according to their Web site and the recent blog post, “The Global Insight Engine Market: European Solution Scores Top Position.” The post makes a poignant point that quickly retrieving answers to complicated problems is a necessity, but regular enterprise search engines cannot crawl unstructured information.
While insight engines are the next buzzword and also the next generation of enterprise search engines, but what exactly do their do?
This is where so-called insight engines come into play. They interpret unstructured and structured data using semantic analysis, and prepare it for further use. Search results are improved and returned in a structured format. Of course, insight engines don’t just process unstructured information, but also all other existing company information. The connection to the individual data sources is made through so-called connectors. Another feature of insight engines is that search queries can be formulated in natural language. The intelligent tools interpret the query and provide the relevant corresponding search results.
Gartner recently ranked the global insight engines market (they have their own market from other search engines?) and Mindbreeze ranks at the top of all the engines in the “challenger” category. What makes this a headliner is that Mindbreeze competed against IBM and HP. Mindbreeze then brags about their features: less than 90 days to integrate into a system, more out-of-the-box solutions for data connectors than other vendors, and Mindbreeze is more popular now since Google withdrew from the market.
Since this was published on Mindbreeze’s own blog, of course, it is a publicity piece. In an objective test, how would Mindbreeze compete against Europe’s other engine, Elasticsearch?
Whitney Grace, September 6, 2017
Lucidworks: The Future of Search Which Has Already Arrived
August 24, 2017
I am pushing 74, but I am interested in the future of search. The reason is that with each passing day I find it more and more difficult to locate the information I need as my routine research for my books and other work. I was anticipating a juicy read when I requested a copy of “Enterprise Search in 2025.” The “book” is a nine page PDF. After two years of effort and much research, my team and I were able to squeeze the basics of Dark Web investigative techniques into about 200 pages. I assumed that a nine-page book would deliver a high-impact payload comparable to one of the chapters in one of my books like CyberOSINT or Dark Web Notebook.
I was surprised that a nine-page document was described as a “book.” I was quite surprised by the Lucidworks’ description of the future. For me, Lucidworks is describing information access already available to me and most companies from established vendors.
The book’s main idea in my opinion is as understandable as this unlabeled, data-free graphic which introduces the text content assembled by Lucidworks.
However, the pamphlet’s text does not make this diagram understandable to me. I noted these points as I worked through the basic argument that client server search is on the downturn. Okay. I think I understand, but the assertion “Solr killed the client-server stars” was interesting. I read this statement and highlighted it:
Other solutions developed, but the Solr ecosystem became the unmatched winner of the search market. Search 1.0 was over and Solr won.
In the world of open source search, Lucene and Solr have gained adherents. Based on the information my team gathered when we were working on an IDC open source search project, the dominant open source search system was Lucene. If our data were accurate when we did the research, Elastic’s Elasticsearch had emerged as the go-to open source search system. The alternatives like Solr and Flaxsearch have their users and supporters, but Elastic, founded by Shay Branon, was a definite step up from his earlier search service called Compass.
In the span of two and a half years, Elastic had garnered more than a $100 million in funding by 2014and expanded into a number adjacent information access market sectors. Reports I have received from those attending Elastic meetings was that Elastic was putting considerable pressure on proprietary search systems and a bit of a squeeze on Lucidworks. Google’s withdrawing its odd duck Google Search Appliance may have been, in small part, due to the rise of Elasticsearch and the changes made by organizations trying to figure out how to make sense of the digital information to which their staff had access.
But enough about the Lucene-Solr and open source versus proprietary search yin and yang tension.
Docurated Expands Salesforce to Broaden Search
August 18, 2017
Enterprise search is growing to make the user experience easier as the demand for everyday use by company employees not deemed ‘data analysts’ is growing. One company slowly making a name for themselves by providing such a service is Docurated.
CMSWire explains their new federated search within Salesforce as the following,
…both sides win with this solution. By delivering content through the native search bar in Salesforce.com — the most used feature of the platform — marketing gets to use the most trafficked channel to drive content consumption, while sales receives content in context…Its Content Cloud uses a combination of inputs and analytics about the effectiveness of content, combined with powerful search, to retrieve relevant content…It fully integrates with all existing cloud and on-premises content repositories and tracks versions of content, sharing only the latest and most accurate version within the organization.
We’re seeing this trend continue to grow with more search vendors making the search process more user-friendly and able to work in multiple functions and across applications. While Google is going ad-happy with their user experience, most search companies are realizing Google had the right idea in the beginning and are making strides to duplicate it within enterprise search.
Catherine Lamsfuss, August 18, 2017
New Enterprise Search Market Study
August 1, 2017
Don Quixote and Solving Death: No Problem, Amigo
I read “Global Enterprise Search Market 2017-2022.” I was surprised that a consulting firms would invest time and energy in writing about a market sector which has not been thriving. Now don’t start sending me email about my lack of cheerfulness about enterprise search. The sector is thriving, but it is doing so with approaches that are disguised as applications which deliver something other than inflated expectations, business closures, and lawsuits.
I will slay the beast that is enterprise search. “Hold still, you knave!”
First, let’s look at what the report covers, then I will tackle some of the issues about which I think as the author of the Enterprise Search Report and a number of search-related articles and analyses. (The articles are available from the estimable Information Today Web site, and the free analyses may be located at www.xenky.com/vendor-profiles.
The write up told me that enterprise search boils down to these companies:
Coveo Corp
Dassault Systemes
IBM Corp
Microsoft
Oracle
SAP AG
Coveo is a fork of Copernic. Yep, it’s a proprietary system which originally was focused on providing search for Microsoft. Now the company has spread its wings to include a raft of functions which range from the cloud to customer support / help desk services.
Dassault Systèmes is the owner of Exalead. Since the acquisition, Exalead as a brand has faded. The desktop search system was killed, and its proprietary technology lives on mostly as a replacement for Dassault’s internal search system which was based on Autonomy. Most of the search wizards have left, but the Exalead technology was good before Dassault learned that selling search was indeed a challenge.
IBM offers a number of products which include open source Lucene, acquired technology like Vivisimo’s clustering engine, and home brew code from its IBM wizards. (Did you know that the precursor of PageRank was an IBM “invention”?) The key is that IBM uses search to sell services which have a higher margins than providing a free version of brute force information access.
A Potentially Useful List of Enterprise Search Engine Servers
July 20, 2017
We found a remarkable list at Predictive Analytics Today—“Top 23 Enterprise Search Engine Servers.” The write-up introduces its roster of resources:
Enterprise Search is the search information within an enterprise, searching of content from multiple enterprise-type sources, such as databases and intranets. These search systems index data and documents from a variety of sources including file systems, intranets, document management systems, e-mail, and databases. Enterprise search systems also integrate structured and unstructured data in their collections and also use access controls to enforce a security policy on their users.
Entries are logically presented under two categories, proprietary solutions and open source software. From Algolia to Xapian, the article summarizes pros and cons of each. See the post for details.
However, we have a few notes to add about some particular platforms. For example, the Google Search Appliance has been discontinued, though Constellio is still going… in Canada. SearchBlox is now Elasticsearch, and SRCH2 was originally designed for mobile searches. Also, isn’t Sphinx Search specifically for SQL data? Hmm. We suggest this list could make a good springboard, but server shoppers should take its specifics with a grain of salt, and be sure to do your own follow-up research.
Cynthia Murrell, July 20, 2017
-
- Subscribe to Beyond Search
Feature archive
News archive
-
