Search Hoops: Exercising Technology to Meeting User Needs
March 29, 2008
A “hoop” is a circular that binds a barrel’s staves together. A “hoops” has a more informal meaning; the word is a synonym for basketball. In Kentucky, you say, “The Louisville Cardinals shoot serious hoops”. This sentence won’t make much sense in Santiago, Chile, but it does at the local gas station.
Search “hoops” are different. These are technical spaces that make it possible for a person to look for information. The figure below shows a series of search hoops. I want to take a few minutes to talk briefly about each of these with particular emphasis on their relationship to behind-the-firewall search. As you know, I think the term enterprise search is essentially valueless. It’s become an audible pause mouthed by vendors of many shapes and sizes. When I hear it, I’m baffled. Truth be told, most of the vendors who use the term enterprise search don’t know what it means. The job of explaining its meaning is left to the pundits and mavens who earn a living blowing smoke to explain fuzziness. Visibility and comprehension hit the two to four inch range.
This is a diagram from a report I wrote for a company silly enough to pay me for an analysis of the online search-and-retrieval trends in the period 1975 to 2003. I have an updated version, but that’s something I sell to buy my beloved boxer dog Tyson Kibbles and Bits.
© Stephen E. Arnold, 2002-2008
Please, click on the image so you can read the textual annotations to each of the rings. I’m not going to repeat the information in the diagram’s annotations. I will related these “hoops” to the challenge of behind-the-firewall search.
The Black Hoop of Death, 1975-1985
In this time period, search-and-retrieval was a field suitable for plowing by wizened experts. Digital information was, for all practical purposes, managed by members of this specialist class of data farmers. If you wanted a report, you went hat in hand to the data barn, usually a special room where data farmers dressed in white data overalls. You explained your request. You filled out a form. You waited. You may or may not find what you needed on the green and white striped paper the data farmer produced. If not, you harvested the information yourself. Your tools were traditional research techniques.
The Yellow Hoop of Hope, 198601991
The IBM personal computer put many data farmers into a new line of work–Wal*Mart greeters or Internet pioneers. Those with PCs were learning first hand that electronic information could be accessed without the humiliating trek to the data center where data farmers did pretty much as they pleased. Unfortunately many commercial enterprises resisted the personal computer. The data farmers and their expensive machinery insisted upon control. The “yellow ring of fire” started the first agitation by the PC literate to have direct access to electronic information.
The Red Ring of Burned Hopes, 1992-1995
Matt Kohl and PLS (Personal Library Software) was one of the first companies to put access and indexing tools in the hands of an employee with a PC and the budget to get a copy of his desktop search system. Surprisingly Dow Jones & Co. perceived an opportunity for an online, end-user search system. After this moment of inspiration, its wizards demonstrated remarkable incompetence. But a useful result was that Dow Jones’s marketing gave other people a treasure map to the riches in digital information.
The Green Hoop of Miraculous Growth, 1996-2000
In the span of four to five years, the notion of online search and retrieval moved from the margins of business life to the center. The demand for behind-the-firewall search and retrieval fueled start ups that are today household names. Google started in this period as did Fast Search & Transfer. Commercial services like Endeca and Autonomy poked their beaks through their technical shells and became commercial endeavors. Other examples are discussed in Beyond Search: What to Do When Your Search System Doesn’t Work, published by the Gilbane Group in Boston, Massachusetts. The hoop of growth drew sustenance from the brutal acceleration in digital content and the finding problems large amounts of such information posed. Paper files, after all, could be examined manually. Digital information was different, and different work processes were needed.
The Blue Hoop of Reengineering, 2001-2003
A wave of reengineering initiatives took off within organizations. In order to index content, work processes had to be adjusted. Many of today’s most successful search-and-retrieval systems are information platforms. The reason is that organizations didn’t want to buy search; enterprises wanted methods, systems, and frameworks to get the digital genii back in the bottle. The early attempts were only partially successful. Today, almost five years later, we are still struggling with search-and-retrieval explained in management-speak that is almost meaningless.
How We Got through These Hoops
In my opinion the task of getting through or around these hoops has been a difficult one for most organizations, information technology professionals, vendors and users. In short, behind-the-firewall search is simply not very well liked. It’s often expensive because unexpected costs arise from almost invisible causes. A simple request balloons into a $150,000 “fix”. Users can’t find information about a major customers. Not just one time–no, the problem exists every single time the customer orders more widgets. Revolving doors and musical chairs characterize the work history of the IT person who is the “owner” of the search system. Vendors constantly release updates, bug fixes, new versions, and enhancements. There’s no stable search system; there’s only flux. Heraclitus said, “Nothing endures but change.”
What Do the Hoops Teach Me?
I’ve been working either as a student, entrepreneur, corporate lackey, or consultant through this time period from 1975 to 2003. Some would argue that I haven’t done an honest day’s work in the last five years.
Here’s what I have learned:
- Each hoop is larger than the last. This means that instead of a narrow, easily managed domain within the hoop, the job of behind-the-firewall search and retrieval keeps getting every larger. This means, as I read these hoops, search is going to suck in work that involves digital information. In short, search becomes work and work is search.
- The “black hole”–the nature of information itself–remains in each of the other hoops. If my conclusion is correct, we’re going to have to learn to live with the fundamental paradoxical nature of information itself. A “hoop” can be many things, and I have conveniently appropriated the word “hoop” for the purposes of this essay. Try explaining that to your search-and-retrieval system’s knowledge base in a bullet-proof way. You simply cannot and for the foreseeable future.
- Each hoop has different winners and losers. Google, for example, was the clear winner from the “green hoop of miraculous growth”. I don’t think history supports a conclusion that asserts the following: “Google’s growth will continue ad infinitum.” This gives hope to search entrepreneurs and triggers snorts of derision from Googlers and their Wall Street cheerleaders. I understand those reactions, but, if my hoops are correct, previous hoop winners may be excluded from the next round.
- We’re going to have to do some serious thinking about what search means. I just completed a 275-page study that makes a good case one type of search is dead. If enterprise search has gone the way of the dodo, what’s the replacement? The answer isn’t search. Something important is happening. We’re using words like semantic search, beyond search, and business intelligence to try and convey this important shift. But, speaking honestly, these terms aren’t too helpful to anyone except a coterie.
I’ve probably annoyed you on this fine Saturday. I want to pick up the thread of the “death of enterprise search” in another essay.
Stephen Arnold, March 29, 2008
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
