Why Enterprise Search Fails
July 12, 2016
I participated in a telephone call before the US holiday break. The subject was the likelihood of a potential investment in an enterprise search technology would be a winner. I listened for most of the 60 minute call. I offered a brief example of the over promise and under deliver problems which plagued Convera and Fast Search & Transfer and several of the people on the call asked, “What’s a Convera?” I knew that today’s whiz kids are essentially reinventing the wheel.
I wanted to capture three ideas which I jotted down during that call. My thought is that at some future time, a person wanting to understand the incredible failures that enterprise search vendors have tallied will have three observations to consider.
No background is necessary. You don’t need to read about throwing rocks at the Google bus, search engine optimization, or any of the craziness about search making Big Data a little pussycat.
Enterprise Search: Does a Couple of Things Well When Users Expect Much More
Enterprise search systems ship with filters or widgets which convert source text into a format that the content processing module can index. The problem is that images, videos, audio files, content from wonky legacy systems, or proprietary file formats like IBM i2’s ANB files do not lend themselves to indexing by a standard enterprise search system. The buyers or licensees of the enterprise search system do not understand this one trick pony nature of text retrieval. Therefore, when the system is deployed, consternation follows confusion when content is not “in” the enterprise search system and, therefore, cannot be found. There are systems which can deal with a wide range of content, but these systems are marketed in a different way, often cost millions of dollars a year to set up, maintain, and operate.
Net net: Vendors do not explain the limitations of text search. Licensees do not take the time or have the desire to understand what an enterprise search system can actually do. Marketers obfuscate in order to close the deal. Failure is a natural consequence.
Data Management Needed
The disconnect boils down to what digital information the licensee wants to search. Once the universe is defined, the system into which the data will be placed must be resolved. No data management, no enterprise search. The reason is that licensees and the users of an enterprise search system assume that “all” or “everything” – maps to web content, email to outputs from an AS/400 Ironside are available any time. Baloney. Few organizations have the expertise or the appetite to deal with figuring out what is where, how much, how frequently each type of data changes, and the formats used. I can hear you saying, “Hey, we know what we have and what we need. We don’t need a stupid, time consuming, expensive inventory.” There you go. Failure is a distinct possibility.
Net net: Hope springs eternal. When problems arise, few know what’s where, who’s on first, and why I don’t know is on third.
Infrastructure
Yep, the cloud is a great deal. Cheap, easy, fast—you believe this, right? Getting, moving, storing, and processing digital information is expensive. Most organizations have data scattered hither and yon. Without data management, the “missing” data will be missed ONLY when someone is running a query and the results have an omission which makes the outputs look off kilter or something is missing like the president’s memo about stock options which was distributed to two people who quit after the president bailed out. Now what? The cost issues are significant. Once grousing starts as the implications of taxi meter pricing, consultants, sticking plasters on cracking walls, and the other signals that few chief financial officers willingly ignore, the infrastructure required to support “search” or whatever buzzword is slapped on the project becomes a significant financial challenge.
Net net: Assumptions have to be verified. Bad plumbing has to be fixed or the cost becomes an unpleasant issue.
So What?
I mention these three issues because vendors of search and retrieval systems may not know the cost and engineering challenges their own systems impose. Marketers often fuzz the truth to close the deal. The licensees (buyers) have their own agenda, lack expertise, possess boundless Sillycon Valley arrogance, or some combination of characteristics.
That’s why search fails. Painful stuff. Think of enterprise search in this way, the index is not a data management solution. The systems focus mostly on keywords. Indexing is only 80 percent accurate when done by humans or machines. Metadata get you only so far. Analytics and visualizations are great when the outputs are accurate. But in today’s go go, mobile world, someone else will solve the problem.
I am still waiting.
Net net: Where is that Frame file I was working on yesterday?
Stephen E Arnold, July 12, 2016
E
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
