• Home
• About
• Writers
• Landscape
• Wizards Index
• Fraud
• Wanna Be Like Larry?

Enterprise Search: Will Synthetic Hormones Produce a Revenue Winner?

One of my colleagues provided me with a copy of the 24 page report with the hefty title:

In Search for Insight 2017. Enterprise Search and Findability Survey. Insights from 2012-2017

I stumbled on the phrase “In Search for Insight 2017.”

The report combines survey data with observations about what’s going to make enterprise search great again. I use the word “again” because:

• The buy up and sell out craziness which culminated with Microsoft’s buying Fast Search & Transfer in 2008 and Hewlett Packard’s purchase of Autonomy in 2011 marked the end of the old-school enterprise search vendors. As you may recall, Fast Search was the subject of a criminal investigation and the HP Autonomy deal continues to make its way through the legal system. You may perceive these two deals as barn burners. I see them as capstones for the era during which search was marketed as the solution to information problems in organizations.
• The word “search” has become confusing and devalued. For most people, “search” means the Danny Sullivan search engine optimization systems and methods. For those with some experience in information science, “search” means locating relevant information. SEO erodes relevance; the less popular connotation of the word suggests answering a user’s question. Not surprisingly, jargon has been used for many years in an effort to explain that “enterprise search” is infused with taxonomies, ontologies, semantic technologies, clustering, discovery, natural language processing, and other verbal chrome trim to make search into a Next Big Thing again. From my point of view, search is a utility and a code word for spoofing Google so that an irrelevant page appears instead of the answer the user seeks.
• The enterprise search landscape (the title of one of my monographs) has been bulldozed and reworked. The money in the old school precision and recall type of search comes from consulting. Search Technologies was acquired by Accenture to add services revenue to the management consulting firm’s repertoire of MBA fixes. What is left are companies offering “solutions” which require substantial engineering, consulting, and training services. The “engine”, in many cases, are open source systems which one can download without burdensome license fees. From my point of view, search boils down to picking an open source solution. If those don’t work, one can license a proprietary system wrapped around open source. If one wants a proprietary system, there are some available, but these are not likely to reach the lofty heights of the Fast Search or Autonomy IDOL systems in the salad days of enterprise search and its promises of a universal search system. The universal search outfit Google pulled out of enterprise search for a reason.

I want to highlight five of the points in the 24 page write up. Please, register to get your own copy of this document.

Here are my five highlights. My comments are in italics after each quote from the document:

The Insight Engine

We explain how search, analytics, and artificial intelligence (AI) are merging and how the insight engine can benefit you.

I think this means that the consulting company is going to deliver “reports” or “essays” which are now “insight engines.” I was confused because there are companies which describe their information access technology systems as “insight engines,” “insight platforms,” and “insight systems.” Examples include the company Insight Engines, the companies in a Gartner Group report which also tries to revivify enterprise search in an effort to make it a next big thing, and even IBM Watson. (See this link for a moment of findability mirth as you contemplate building your own insight engine.)

Search Applications

Here’s the survey finding:

It appears that people want “one search.” The future is search based applications for some organizations. I think that enterprise search, if I consider the tools available today, can address silo searching. There are the open source systems which my research suggests are popular for building a company wide search system and for creating a finding solution which can work inside enterprise applications. The problem is that users of these systems remain dissatisfied with the solutions regardless of vendor. Search is a utility and an annoying one at that. Consequently vendors embed search into applications and make sure it works in a “good enough” manner. That’s how search works in a remarkable number of applications we have tested, including systems from Palantir, Textron, BAE Systems, and other cutting edge firms’ information access systems. The real action is the solution the system delivers to the user. Search is a component, but it is neither designed nor capable of supporting next generation information access platforms. For more information on this next generation stuff, check out my monograph CyberOSINT: Next Generation Information Access. The report makes clear that more than half of the users of an enterprise search system are dissatisfied. That lines up, but why after decades of effort is enterprise search a problem child? That issue is not addressed, although Martin White and I made an attempt to identify what one needs to do to chip away at endemic grousing.

Information: Useful and Actionable

I noted this statement:

The real goal is to make information useful and actionable. The goal is to help users get data-based insights and improve business (make more money), boost operational excellence (produce more with less), or get an edge (do things smarter) [sic.] Instead of providing links to information objects…the insight engine gives you an overview and the means to explore individual pieces.

Sounds good. But what if I need to locate a specific invoice? What if I want to determine what supplier provided a specific part in February 2017? What if I want to download the CFO’s presentation given one day ago to analysts attending a meeting yesterday? What if a member of a special ops teams needs to access data from a drone, observers elsewhere in the field of battle, and a map displaying the most likely route bad actors take into town? These are search problems. Some easy; some difficult or beyond the reach of today’s technology. What’s troubling me about fancy technology like smart software, black box analytics, and presentations which allegedly pull “current” information from multiple systems is that most are ill suited to a marketing professional trying to cook up text for a conference handout. Jargon and assertions about being useful and actionable are glittering generalities. Like many sparkly things these terms obfuscate instead of clarifying what enterprise search can deliver.

Making Search Really Important?

Here’s a passage from the report:

A clear trend is the merge [sic] of search and analytics, often referred to as search driven analytics.

What? Often? The point the report is trying to get across is that search will solve the problem of a person who is not very good in math or who lacks any clear idea of what’s under the hood of the numerical recipes used to “make sense” of data. This is better than getting the answers to a Statistics 101 exam. The notion that a marketer, lawyer, or engineer can craft a query to unlock the high value hidden within a database or index strikes me as a bit of a stretch. The most widely used analytics system in the world is Excel. How does that work for most people? Well, not with search queries. Humans fumble to get charts that make sense. Even analysts use automated spreadsheets to output the crazy quarterly numbers which provide “insight” into the financial health of a company.

I am in the queue which forms to learn how to use SAS or SPSS, maybe Wolfram. I am not too keen on the search for analytics thing. But, hey, I worked at Halliburton Nuclear, which seemed a bit math heavy. You know. Equations on whiteboards. PhDs mocking the person who made an error in front of a group of Monte Carlo lovers.

My View of Enterprise Search

Let me offer several observations. More detail about my opinion can be found in my Beyond Search posts.

1. Most of the Fortune 1000 companies have a long, often unsettled history with enterprise search. I have encountered a Fortune 50 company with 11 different enterprise search systems, dozens of bootleg systems, and lawyers who had to have their own bespoke systems. In short, there is no such thing as an enterprise search system. There are information problems and different tools and systems are needed to solve these to the users’ satisfaction. That’s what got Fast Search in hot water, and the pitch to make search into something it is not will get today’s whizzy search vendors in trouble too.
2. Search is a utility. Running one query to get timely, on point information from a single query is very, very difficult. The reason is that different information types have to be processed, ideally in near real time. As data volume increases and the problematic file types (video, audio, intercept data, GPS, database data updated in near real time, third party data, etc.) flow into an organization, a lot of money and expertise is required to make sure that the user gets accurate, timely, relevant data. In my experience, even when an organization has billions to spend on this type of system, the system does not work too well. Run a query for DCGS to get some useful information about this “challenge.”
3. Users lack the ability to figure out if a system’s outputs are correct for text centric research. The reality in my experience is that dashboards, clusters, and smart software can mislead or in some cases provide information that will have decision downsides. Stated another way: Flawed outputs can get operators killed. Marketers may have to walk over to the CFO’s office and ask for a copy of a presentation. A person in the field needing current reports about bac actor movements.works in a less forgiving environment.

I think that the technology for eDiscovery is interesting. Recently I have formulated the hypothesis that next generation information access will shift toward eDiscovery. I want to point out that the systems which we have been examining do search but these systems are not enterprise search systems. There’s a reason.

Enterprise search had its chance to be the Next Big Thing. But it wasn’t. Consultants, jargon, and assertions about “knowledge” will not change this reality. If you want to offer search, use Elastic’s Elasticsearch. It’s good enough. Want a proprietary system? Why not look at a lower cost solution like dtSearch for Microsoft centric set ups? Want something that “predicts” the future? Give Recorded Future (an In-Q-Tel and Google company). Just remember that Google did not predict that its Google Search Appliance would be a non starter in the enterprise market.

Based on the information in this “In Search” report, I am not convinced that search can be great again. Obviously one cannot scale an engineered climbing wall unless one tries. I will take the escalator to the floor selling next generation information access systems.

Stephen E Arnold, October 27, 2017

Comments

Copyright © 2008 Beyond Search · ArnoldIT.com · Disclaimer · Revolution News theme by Brian Gardner · Log in