Enterprise Search: Still Floundering after All These Years

October 11, 2017

Enterprise search conferences once had pride of place. Enterprise search or “search” was the Big Data, artificial intelligence, and cyber intelligence solution from 1998 to 2007.

But by 2007, the fanciful claims of enterprise search vendors were perceived as “big hat, no cattle” posturing. Unable to generate sustainable revenues, the high profile enterprise search systems began looking for a buyer. Those who failed disappeared. Do you know where Convera, Delphes, Entopia, and Siderean are today? What’s the impact of Exalead on Dassault? Autonomy on Hewlett Packard Enterprise? Vivisimo on IBM?

Easy questions to ignore. Time marches on. Proprietary search cost a bundle to keep working. The “fix” to the development, enhancement, and bug fix problems was open source.

A solution emerged. Lucene. That brings us to the title of this blog post: “Enterprise Search: Still Floundering after All These Years.”

The money from license fees is insufficient to make enterprise search work in a good enough way. Open source search, which seems to be largely free of license fees, allows vendors to offer search and highly profitable services to the organizations who want or need an “enterprise search system.”

This means that a vendor who makes more money offering search services can be perceived as a problem to an venture funded company built on promises and tens of millions in venture capital.

The truth of this observation was revealed in an article written by or for Search Technologies, a unit of a Fancy Dan consulting firm. If I understand the Search Technologies’ write up, Lucidworks (né Lucid Words) told Search Technologies that it was not welcome at a conference designed to promote Solr.

Here’s what Search Technologies said in “Why Wasn’t Search Technologies at Lucene/Solr Revolution 2017?”

Lucene/Solr Revolution’s organizer, Lucidworks, informed us that we were no longer welcome to exhibit or speak at the event. Lucidworks considered us a company that:

  • Competes with their professional services group (maybe)
  • Is not likely to resell Lucidworks’ platform exclusively (we are vendor-agnostic, after all), and,
  • Has technology assets that compete with their Fusion platform (partially true)

I don’t care too much about venture funded outfits running conferences to make their “one true way” evident to the attendees. I don’t worry about a blue chip consulting firm’s ability to generate sales leads.

No.

I find that some of enterprise search’s most problematic weaknesses have not been solved after 50 years of flailing. Examples include:

  • The cost of moving beyond “good enough” information access
  • Revealing that enterprise search systems are expensive to tune and shape to the needs of an organization
  • Developing solutions which keep indexes current and searches responsive
  • Seamless handling different types of content, including video, engineering drawings, and data tucked inside legacy systems
  • Keeping the majority of the users happy so bootleg search systems are not installed to meet departmental or operating unit needs.

The “search” problem is an illustration of innovation running out of gas. I have zero stake in Lucidworks, Search Technologies, or enterprise search. I am content to be an observer who points out that search vendors, their marketing, the consultants, and the conference organizers are their own worst enemy.

That’s why enterprise search imploded about a decade ago. Search today is pretty much “good enough.” Antidot, Lucene, Solr, dtSearch, X1, Fabasoft, Funnelback, et al. Each does “good enough” search in my opinion.

To make any system better takes consulting and engineering services. These deliver high margins. Users? Well, users want enterprise search to answer questions and work like Google. After 50 years of effort, no company has been able to meet the users’ needs.

That says more than two consulting firms trading digital jabs. What’s at stake is consulting revenue and proprietary fixes. Users? Yes, what about the users?

Stephen E Arnold, October 10, 2017

Palantir Settlement Makes Good Business Sense

October 11, 2017

Palantir claims it is focusing on work, not admitting its guilt over a labor dispute in a recent settlement. This is creating a divide in the industry about what it exactly does mean. We first learned of the $1.66 million settlement in How To Zone’s story, “Palantir Settles Discrimination Complaint with U.S. Labor Agency.”

How did we get here? According to the story:

The Labor Department said in an administrative complaint last year that it conducted a review of Palantir’s hiring process beginning in 2010. The agency alleged that the company’s reliance on employee referrals resulted in bias against Asians. Contracts worth more than $370 million, including with the U.S. Defense Department, Treasury Department and other federal agencies, were in jeopardy if the Labor Department had found Palantir guilty of discrimination.

Serious accusations. But this settlement might not signal what you think it does. Palantir said in a statement:

We settled this matter, without any admission of liability, in order to focus on our work.

This might be the smartest action on their behalf. Consider what happened to SalesForce when they got wrapped up in a legal battle earlier this year. It not only slowed down their sales, but some experts feel the suit may have altered enterprise search for good.

Something tells us Palantir, with its rich government contracts, wants to simply put this behind them and not get caught in a legal web.

Patrick Roland, October 11, 2017

Are Vendors of Enterprise Search Distracted?

September 6, 2017

I read “To Have Good Ideas, Remember to Get Bored.” I noted this assertion in the write up:

the temptation of constant podcast listening, phone fiddling, and TV watching takes over. The fight to maintain some boredom never ends.

The idea is that distraction kills boredom. Without boredom, “people” do not get good ideas. Ergo when one notes lots of bad ideas, that may be the signal that distraction undermines innovative thinking.

I am not certain the statements in the write up and the accompanying TED talk (which bored me, by the way) are applicable across a population sleeted at random in Rwanda or rural Kentucky, but let’s assume the idea has value.

I look at enterprise search and I see the same old perpetual motion machines: Semantics, metatagging, context, yada yada.

Perhaps those involved in enterprise search system development are manifesting their distractedness. Instead of putting down the mobile and performing myriad displacement activities, are enterprise search system developers fresh out of ideas.

Something’s wrong. Analysts find search just peachy when relying on SAP, IBM Watson, Fabasoft, and the other systems available today.

I know I am bored, and I would postulate that those involved in next generation information access systems may want to cultivate a bit of boredom as well. Innovation may come about.

Example: As I was thinking about today’s me-to enterprise search systems, I was bored. I decided to begin work on a new book in my cyber intelligence series. How does eDiscovery for Investigators sound?

Boring?

Stephen E Arnold, September 6, 2017

Insight Engines Are the next Enterprise Upgrade

September 6, 2017

When one buzzword loses its, marketing teams do their best to create the next term to stay on top of their competition.  When it comes to search, the newest buzzword appears to be “insight engine.”  Mindbreeze is top-selling insight engine, according to their Web site and the recent blog post, “The Global Insight Engine Market: European Solution Scores Top Position.”  The post makes a poignant point that quickly retrieving answers to complicated problems is a necessity, but regular enterprise search engines cannot crawl unstructured information.

While insight engines are the next buzzword and also the next generation of enterprise search engines, but what exactly do their do?

This is where so-called insight engines come into play. They interpret unstructured and structured data using semantic analysis, and prepare it for further use. Search results are improved and returned in a structured format. Of course, insight engines don’t just process unstructured information, but also all other existing company information. The connection to the individual data sources is made through so-called connectors. Another feature of insight engines is that search queries can be formulated in natural language. The intelligent tools interpret the query and provide the relevant corresponding search results.

Gartner recently ranked the global insight engines market (they have their own market from other search engines?) and Mindbreeze ranks at the top of all the engines in the “challenger” category.  What makes this a headliner is that Mindbreeze competed against IBM and HP.  Mindbreeze then brags about their features: less than 90 days to integrate into a system, more out-of-the-box solutions for data connectors than other vendors, and Mindbreeze is more popular now since Google withdrew from the market.

Since this was published on Mindbreeze’s own blog, of course, it is a publicity piece.  In an objective test, how would Mindbreeze compete against Europe’s other engine, Elasticsearch?

Whitney Grace, September 6, 2017

Lucidworks: The Future of Search Which Has Already Arrived

August 24, 2017

I am pushing 74, but I am interested in the future of search. The reason is that with each passing day I find it more and more difficult to locate the information I need as my routine research for my books and other work. I was anticipating a juicy read when I requested a copy of “Enterprise Search in 2025.” The “book” is a nine page PDF. After two years of effort and much research, my team and I were able to squeeze the basics of Dark Web investigative techniques into about 200 pages. I assumed that a nine-page book would deliver a high-impact payload comparable to one of the chapters in one of my books like CyberOSINT or Dark Web Notebook.

I was surprised that a nine-page document was described as a “book.” I was quite surprised by the Lucidworks’ description of the future. For me, Lucidworks is describing information access already available to me and most companies from established vendors.

The book’s main idea in my opinion is as understandable as this unlabeled, data-free graphic which introduces the text content assembled by Lucidworks.

image

However, the pamphlet’s text does not make this diagram understandable to me. I noted these points as I worked through the basic argument that client server search is on the downturn. Okay. I think I understand, but the assertion “Solr killed the client-server stars” was interesting. I read this statement and highlighted it:

Other solutions developed, but the Solr ecosystem became the unmatched winner of the search market. Search 1.0 was over and Solr won.

In the world of open source search, Lucene and Solr have gained adherents. Based on the information my team gathered when we were working on an IDC open source search project, the dominant open source search system was Lucene. If our data were accurate when we did the research, Elastic’s Elasticsearch had emerged as the go-to open source search system. The alternatives like Solr and Flaxsearch have their users and supporters, but Elastic, founded by Shay Branon, was a definite step up from his earlier search service called Compass.

In the span of two and a half years, Elastic had garnered more than a $100 million in funding by 2014and expanded into a number adjacent information access market sectors. Reports I have received from those attending Elastic meetings was that Elastic was putting considerable pressure on proprietary search systems and a bit of a squeeze on Lucidworks. Google’s withdrawing its odd duck Google Search Appliance may have been, in small part, due to the rise of Elasticsearch and the changes made by organizations trying to figure out how to make sense of the digital information to which their staff had access.

But enough about the Lucene-Solr and open source versus proprietary search yin and yang tension.

Read more

Docurated Expands Salesforce to Broaden Search

August 18, 2017

Enterprise search is growing to make the user experience easier as the demand for everyday use by company employees not deemed ‘data analysts’ is growing. One company slowly making a name for themselves by providing such a service is Docurated.

CMSWire explains their new federated search within Salesforce as the following,

…both sides win with this solution. By delivering content through the native search bar in Salesforce.com — the most used feature of the platform — marketing gets to use the most trafficked channel to drive content consumption, while sales receives content in context…Its Content Cloud uses a combination of inputs and analytics about the effectiveness of content, combined with powerful search, to retrieve relevant content…It fully integrates with all existing cloud and on-premises content repositories and tracks versions of content, sharing only the latest and most accurate version within the organization.

We’re seeing this trend continue to grow with more search vendors making the search process more user-friendly and able to work in multiple functions and across applications. While Google is going ad-happy with their user experience, most search companies are realizing Google had the right idea in the beginning and are making strides to duplicate it within enterprise search.

Catherine Lamsfuss, August 18, 2017

New Enterprise Search Market Study

August 1, 2017

Don Quixote and Solving Death: No Problem, Amigo

I read “Global Enterprise Search Market 2017-2022.” I was surprised that a consulting firms would invest time and energy in writing about a market sector which has not been thriving. Now don’t start sending me email about my lack of cheerfulness about enterprise search. The sector is thriving, but it is doing so with approaches that are disguised as applications which deliver something other than inflated expectations, business closures, and lawsuits.

Image result for don quixote

I will slay the beast that is enterprise search. “Hold still, you knave!”

First, let’s look at what the report covers, then I will tackle some of the issues about which I think as the author of the Enterprise Search Report and a number of search-related articles and analyses. (The articles are available from the estimable Information Today Web site, and the free analyses may be located at www.xenky.com/vendor-profiles.

The write up told me that enterprise search boils down to these companies:

Coveo Corp
Dassault Systemes
IBM Corp
Microsoft
Oracle
SAP AG

Coveo is a fork of Copernic. Yep, it’s a proprietary system which originally was focused on providing search for Microsoft. Now the company has spread its wings to include a raft of functions which range from the cloud to customer support / help desk services.

Dassault Systèmes is the owner of Exalead. Since the acquisition, Exalead as a brand has faded. The desktop search system was killed, and its proprietary technology lives on mostly as a replacement for Dassault’s internal search system which was based on Autonomy. Most of the search wizards have left, but the Exalead technology was good before Dassault learned that selling search was indeed a challenge.

IBM offers a number of products which include open source Lucene, acquired technology like Vivisimo’s clustering engine, and home brew code from its IBM wizards. (Did you  know that the precursor of PageRank was an IBM “invention”?) The key is that IBM uses search to sell services which have a higher margins than providing a free version of brute force information access.

Read more

A Potentially Useful List of Enterprise Search Engine Servers

July 20, 2017

We found a remarkable list at Predictive Analytics Today—“Top 23 Enterprise Search Engine Servers.” The write-up introduces its roster of resources:

Enterprise Search is the search information within an enterprise, searching of content from multiple enterprise-type sources, such as databases and intranets. These search systems index data and documents from a variety of sources including file systems, intranets, document management systems, e-mail, and databases. Enterprise search systems also integrate structured and unstructured data in their collections and also use access controls to enforce a security policy on their users.

Entries are logically presented under two categories, proprietary solutions and open source software. From Algolia to Xapian, the article summarizes pros and cons of each. See the post for details.

However, we have a few notes to add about some particular platforms. For example, the Google Search Appliance has been discontinued, though Constellio is still going… in Canada. SearchBlox is now Elasticsearch, and SRCH2 was originally designed for mobile searches. Also, isn’t Sphinx Search specifically for SQL data? Hmm. We suggest this list could make a good springboard, but server shoppers should take its specifics with a grain of salt, and be sure to do your own follow-up research.

Cynthia Murrell, July 20, 2017

Big Data in Biomedical

July 19, 2017

The biomedical field which is replete with unstructured data is all set to take a giant leap towards standardization with Biological Text Mining Unit.

According to PHYS.ORG, in a peer review article titled Researchers Review the State-Of-The-Art Text Mining Technologies for Chemistry, the author states:

Being able to transform unstructured biomedical research data into structured databases that can be more efficiently processed by machines or queried by humans is critical for a range of heterogeneous applications.

Scientific data has fixed set of vocabulary which makes standardization and indexation easy. However, most big names in Big Data and enterprise search are concentrating their efforts on e-commerce.

Hundreds of new compounds are discovered every year. If the data pertaining to these compounds is made available to other researchers, advancements in this field will be very rapid. The major hurdle is the data is in an unstructured format, which Biological Text Mining Unit standards intend to overcome.

Vishal Ingole, July 19, 2017

Elastic Search Redefining Enterprise Search Landscape

May 24, 2017

Open source enterprise search engine Elastic Search is changing the way large IT enterprises are enabling its user to search relevant data in a seamless manner.

Apiumhub in an in-depth report titled Elastic Search; Advantages, Case Studies & Books says:

Elastic search is able to achieve fast search responses because, instead of searching the text directly, it searches an index instead. This is more or less like searching for a keyword by scanning the index at the back of a book, as opposed to searching every word of every page of the book.

The search engine is easily scalable and can accommodate petabytes of data on multiple servers in short time. Considering it is based on Lucene, developers too find it easy to work with. Even if the keywords are misspelled, the search engine will correct the error and deliver accurate results.

At present, large organizations like Tesco, Wikipedia, Facebook, LinkedIn and Salesforce have already deployed the enterprise search engine across their servers. With the advent of voice-based search, capabilities of Elastic search will be in more demand in the near future, experts say.

Vishol Ingole, May 24, 2017

Next Page »

  • Archives

  • Recent Posts

  • Meta