Search Is a Problem: Still a Clumsy Song and Dance Routine
May 24, 2018
Enterprise search has been around for decades. Hundreds of consultants have asserted patterns, models, methods, and MBA infused strategies to “fix” enterprise search.
Why?
Wherever there is an organization with one or more enterprise search systems, I have found these characteristics:
- Unhappy users
- Unhappy senior manager
- Unhappy bean counters
- Unhappy vendors
- Usually happy consultants if they are paid.
I am biased, old, and hard nosed. After writing the first three editions of the Enterprise Search Report, the New Landscape of Search, adding a word or two to that astounding guru Martin White’s book about Successful Enterprise Search Management, talking with dozens of PhD candidates whose dissertatioins about search and retrieval would change the world, and meeting with vendors large and small for decades—I am amused by the arm waving enterprise search engenders.
Don’t get me wrong. There are very good information access systems. But these vendors license solutions which usually focus on solving a specific problem. Case in point: Blackdot, Terbium Labs, and Verint, and many others.
From the point of view of flailing content management experts, “enterprise search” means finding information in a usually flawed, Rube Goldberg construct called a CMS or content management system.
Against this wallpaper with my scrawled biases, I read “Diagnosing Enterprise Search Failures.” The pivot point for the story is another report that almost two thirds of enterprise search users are not satisfied with the retrieval system.
Like a reprise of a vaudeville act from the 1920s at a rap concert, the music and the footwork are stale, out of touch, and worn.
Enterprise search had its decade in the sun. The period between 1995 and 2005 was the golden age of search. Then the sun imploded. Over-promising and under-delivering made it clear to those licensing enterprise search systems that finding information was not a solution to digital information woes.
In fact, an enterprise search system exacerbated the problems employees encountered when trying to locate specific information. Fast Search & Transfer, Convera, Delphes, Entopia (remember that outfit), and other aggressively marketed companies found out that companies would license technology and then balk at the on going costs.
One by one the big names in enterprise search went out of business or found themselves owned by larger firms with a belief that their managers could make search a winner.
How did that work out? Chase down someone at Lexmark and ask about their experience with ISYS Search Software. Repeat the process at Dassault Systèmes? Do the same thing for products ranging from Artificial Linguistics to Vivisimo.
The result is that the universe of companies offering search solutions has changed since 2008. The legal dust up between HP and Autonomy continues. Search did not make HP happy.
Surveys are fine, but the data reveal nothing new. Enterprise search is not a solution to information problems in an enterprise. Companies are embracing free or low cost solutions based on open source technology. Specialist systems which address specific information access problems are thriving. One may not think of Diffeo and Palantir Technologies as enterprise search systems, but they are information access solutions and not designed to solve a panoply of retrieval and information management issues.
The reason enterprise search fails to please users boils down to the disconnect between what the user wants and what an enterprise wide system can deliver. The vendors promise more than technology can provide.
Checklists, MBA rah rah, and misplaced confidence in technology will not solve these specific challenges:
- The cost of maintaining, upgrading, and tuning an enterprise search system to the needs of specific users is significant
- Users have a keen desire to rely on the software to do the thinking for them. When a system requires the user to think or formulate a query or perform downstream analysis, the search system becomes a problem
- Procurement teams often lack the discipline and clout to lay out tight requirements and select a vendor to do that job. The pattern is to create a wish list, sign a deal, and leave the baggage of failure behind.
- The systems provided do not match what the marketers demo, suggest, or assert the software will actually do in an affordable, reliable, understandable manner.
As a former rental at a reasonably competent management consulting firm, a method for figuring out how to solve a problem has one objective: Sell billable work. I understand that.
Do not confuse a consultant’s report with solving the problem of enterprise search. If enterprise search worked, there would be little appetite for methodologies to figure out failure.
Why such hostility to enterprise search? I think clueless large and medium sized companies want to buy a silver bullet. Even better, the bullet must kill the content vampire with a single, low cost, easy to use, accurate shot.
That’s not going to happen… ever.
The problem is that individuals looking for information need tools to solve quite specific business tasks. In enterprise search, there are numerous points of failure; for example:
- Management support is weak
- Organizational infighting triggers departments to get their own search solution
- The technology does not work
- Results do not meet user needs
- Funding is insufficient
- Technical staff find that fixes are not easy or possible
- Content known to be in the system cannot be found
- Vendors change direction from search to customer support and leave search customers dangling
- The people involved are focused on their careers, lunch, or finding a new job, not the nitty gritty of designing a solution for a specific group of workers with an information need.
And there are other issues related to over-promising and under-delivering. I wrote about this years ago and talked about falling off the cliff of high expectations. Enterprise search users inevitably crash into the reality of the system. Thus, the significant percentage of dissatisfaction with enterprise search.
I know of no enterprise search system which delivers on these points. Furthermore, as venture funding flows into Coveo and LucidWorks, as IBM falls farther and farther behind its revenue goals for Watson search (OmniFind, Vivisimo, et al), and as Microsoft buys more and more search start ups in the hopes of finding a silver bullet to its search mess—It is clear that stakeholders, customers, and users are going to become increasingly annoyed at the problem of enterprise search.
Why did Google bow out of enterprise search? Why has Elastic emerged as the go-to solution for many enterprise search applications? Why are companies like Funnelback, Sophia, Exorbyte, and dozens of others scrambling?
Enterprise search looked like a solution to some important problems. Today not so much. Open source search software is fine. However, how many of the open source vendors are going to be able to generate a return for their investors with what amounts to free software.
Enterprise search is the wrong label for today’s solutions. Even proprietary systems in hock for $100 million have longer odds than a nag entered in the Kentucky Derby.
Therefore, thrashing.
Stephen E Arnold, May 24, 2018
Comments
One Response to “Search Is a Problem: Still a Clumsy Song and Dance Routine”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
He estado pensando en aprender a tocar un instrumento musical,
la verdad es que la música me gusta demasiado.