The ‘Search Is Dead’ Question
August 17, 2008
New Idea Engineering and I cooperate to produce a list of utilities helpful to those working with search and content processing. I want to build on the August 4, 2008, post “Enterprise Search Dead” Or Just Misunderstood?” Keep in mind that I don’t disagree with the points in the post. For me, the important point in the article was this statement about the fact that organizations have multiple search and content processing systems:
The real trick is to glue these technologies together not into a single giant searchable index, but to combine them together logically so the user does not need to know where to look for specific content. We, like many others, call this Federated Search,
I am in favor of federation, aggregation, and simplification. My concern is that the costs associated with multiple systems, multiple “looks” at information, and multiple “cooks in the kitchen” will be difficult to control. Costs matter today. Tomorrow costs will be even more important. Here’s why:
- As search becomes pervasive, costs will chug along, controls will be lax, and then the bills arrive. Few managers can survive cost time bombs like those associated with search. A “time bomb” is a “do whatever it takes” weekends when the system goes down or a cost review by a new chief financial officer who puts a ceiling on information technology expenditures and triggers a melt down.
- Multiple indexes of the same document are okay as long as the document is not undergoing rapid change. In certain organizations, change is frequent and often pretty darn wacky. Out of sync information retrieval systems can be a gold mine for legal discovery. Figuring out which index is the “right” one may be an issue in some situations.
- Multiple systems indexing content within the organization can choke the internal network. Running several systems to update an index may degrade network performance.
Most information technology mangers assume that today’s software and hardware can handle any demand. The problem is that many of today’s systems increase complexity and risk. The ready availability of low cost, fire breathing servers removes inhibitions. The result is system promiscuity and projects that look great in a PowerPoint presentation but fail miserably in the crucible of doing every day work.
If search is not dead, we need to retire it and move up a level. Let’s give users a way to access information that makes most users happy. That’s not what today’s systems deliver. Most users are unhappy with the search systems available to them for behind the firewall search.
Stephen Arnold, August 17, 2008