Inside the Tokamak, Part 2: The Red Spheres of Context
April 3, 2008
In the first part of this essay, I drew a parallel between a tokamak device and plasmas. The idea is that in an organization, new technologies and increasing pressure to work smarter changes what users expect a search and retrieval system to deliver. In this second installment, we look at four additional digital ions and electrons that are “going critical” with regards to information access.
Let’s begin by revisiting the diagram, paying particular attention to the 12 spheres inside the diagram’s central “gray boundary”.
The outer two stacks of “yellow spheres” and “purple spheres” exert pressure on users, vendors, and organizations. As the individual yellow and purple spheres expand, the activity inside the “gray boundary” increases. When dealing with non-linear phenomena, it is difficult to predict what will give way and what will surge to dominance. There is considerable uncertainty within the “gray boundary”.
Perhaps you have experienced this yourself. In my work in the last five years, I have documented the increasing dissatisfaction users express about their search and retrieval systems. Some comments are delivered with hope: for example, “I wish the system would let me retrieve what I need regardless of which department has the data”. Other comments are more earthy, “Management has no idea how frustrated I am with this stupid system.” In my work in New York, I have seen 20-somethings staring at a search results display with frustration and anger clouding their otherwise pampered features.
You may want to click on the diagram to see the labels of the “red spheres” more clearly. As you recall, I prepared this diagram more than five years ago, so it is long in the tooth. But it serves as a useful starting point for our exploration of the forces transforming search from a nice-to-have function to a must-have service.
The Red Spheres
There are four “red spheres” in this stack of digital ions and electrons. As per my wont, I’ll comment on each briefly. To sum up this second installment, I want to offer some additional comments about the “search” sphere. The label for the “red spheres” is contextual.
Context is of paramount importance in search and retrieval. Let me give you an example from my work in the commercial online industry in the 1980-1981 period. This was an era when information cost money. Most people in business assumed that information was free, a hold over from one’s early experience getting information from a library. The connection between taxes or university tuition and the books in the library did not exist.
Here’s how I handled context. The set up for this example is for you to assume that you have a child in the emergency room. Your child has eaten something that’s poisonous. To save your child’s life, the physician needs to consult an online database of antidotes. The per search access charge is $1,0000. Will you pay?
The answer is, at least for me, “Yes.”
Now, you are on your way to a motel in Miami, Florida. You are lost. How much will you pay to get directions to the motel? The answer for me is, “I’m not paying anything.”
So what’s the difference between my willingness to pay $1,000 for one digital answer and paying nothing for directions?
The answer is context. Information’s value, significant, and importance depends on the particular user at a particular time with a particular requirement. Context varies.
Does your behind-the-firewall search system deliver context-sensitive results?
Now let’s look at the “red spheres”.
- Search. This term embraces any information access action a user has. The functional definition of search, as we have just seen in the poison versus motel direction example, is context. If the search results are without context, the search system is not too useful to me. You will have to determine the importance of context for you and for your colleagues. When a user says, “This search system sucks,” be sure to consider the context of the user. Frankly, most of the search systems available today are doomed to suckiness because none of the vendors has a functional, reliable, economical solution to the context problem.
- Commerce. Commercial organizations are engaged in commerce. Therefore, the context of any search or content processing system must be commercial. If you work in a governmental agency of non-profit organization, we would have to find a suitable term to encompass these activities. I will stick with commerce. The idea is that a search system must be aware of and respond to [a] commercial information generated within your organization like prices, customer history, and vendors; [b] public external commercial data such as competitor actions, pricing, staff changes, etc., and [c] third-party data sold or licensed to your organization by publishers, consultants, and news agencies. Most search and content processing systems with which I am familiar lack commerce functionality.
- Presence. When I am looking for information, I need to know who is available to provide that information. I like systems that identify a person in an organization who knows about a topic. The system tells me if that person is in her office or reachable in some way. Google offers this type of functionality via its OneBox API so that a Microsoft Exchange calendar appears in search results with the person’s phone number, email address, and office location. If the person’s online, I want to be able to use a Messaging function (“blue spheres” in the diagram above)
- Agent services. You don’t hear much about “smart” software today. I’m talking with a publisher about my monograph about Google’s computational intelligence innovations. Trust me. After years in the shadows, the software that performs tasks for you will play an ever-increasing part in information access. In Beyond Search, I profile one leader in this sector and dig into one forward-looking Google technology. The key idea is that software can do a better, faster, and cheaper job on some information access tasks. Many of the howls of outrage about laundry lists of results can be turned to purrs of contentment with agent services.
Context Is Bigger than Search
In this second tokamak essay, you have learned three things about search and content processing. At the risk of offending the hundreds of vendors who think that I am drinking too much Kentucky bourbon,
First, search was an early, primitive way to address the problem of finding information in digital form. Even today, most employees cannot explain what their information needs are. The employees can show their work tasks, but it is the job of others to extract from these examples the “definition” of search for that employee. Key word retrieval is the digital equivalent of a flint hatchet. The organization consists of ions and electrons. The mismatch caused by using a crude tool for a complex problem is evident. Few discuss the problems of search in this way; therefore, the “problem” of search is simply not resolvable using existing methods.
Second, context governs the definition of search, but search is only one sphere in a suite of spheres. The interactions within each “red sphere” and among the other “spheres” is not the concern of management, an organization’s technologists, and the majority of pundits trawling their drag lines in this market.
Third, in order to address the “red sphere” issues, key word vendors face a stark choice: [a] add functionality to their system and morph into some type of application that can do what user’s require or [b] just say they can deliver. There are some interesting upsides and downsides to these choices. I will dig into these business issues in a different essay. Your sixth sense may be kicking in. It’s obvious that a search vendor who tries to deal with the “red spheres” can run into trouble if there’s insufficient money, programming talent, and managerial expertise. Moving beyond a core competency essential signs the vendor’s death warrant. (Take a look at the Entopia or Sagemaker cases in this Web log.) If a vendor just claims these capabilities in marketing collateral or pays a research firm to “say it’s so”, you’re sitting on a time bomb. No wonder there’s job turn churn associated with large-scale search and content processing.
I think that my “spheres” despite their obvious flaws provide a reasonable way to discuss what’s happening in search and retrieval. The “spheres” make it clear why vendors are rushing to reposition themselves, often using language that seems to say, “We’re able to do what IBM, Microsoft, Oracle, and SAP can do–and more!” No one with an iota of common sense will swallow this bait, but common sense can sometimes be in short supply in our fluid, no-holds-barred economy.
The next installment will tackle the “green spheres” and provide some broad observations about the tokamak inside your organization.
Stephen Arnold, April 3, 2008
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
