Instant Search: Who Is on First?
September 12, 2010
A reader sent me a link to “Back to the Future: Innovation Is Alive in Search.” The point of the write up is to make clear that for the author of the post, Yahoo was an innovator way back in 2005. In fact, if I understand the Yahooligan’s blog post, Yahoo “invented” instant search. I am an addled goose, but I recall seeing that function in a demo given me in 1999 or 2000 by a Fast Search & Transfer technology whiz. Doesn’t matter.
Search has lacked innovation for a long, long time. In fact, if you can track down someone who will share the closely guarded TREC results, you will see that precision and recall scores remain an interesting challenge for developers of information retrieval systems. In fact, the reason social curation seems to be “good enough” is that traditional search systems used to suck, still suck, and will continue to suck.
The problem is not the math, the wizards, and the hybrid Rube Goldberg machines that vendors use to work their magic. Nope. The problem with search has several parts. Let me make them explicit because the English majors who popular the azure chip consulting firms and “real” blogs have done their best to treat technology as John Donne poem:
First, language. Search involves language, which is a moving target. There’s a reason why poems and secret messages are tough to figure out. Words can be used in ways that allow some to “get it” and others to get an “F” in English 301: The Victorian Novel. At this time, software does better at certain types of language than others. One example is medical lingo. There’s a reason why lots of vendors have nifty STM (scientific, technical, and medical) demos.
Second, humans. Humans usually don’t know exactly what they want. Humans can recognize something that is sort of what they want. If the “something” is close enough for horseshoes, a human can take different fragments and glue them together to create an answer. These skills baffle software systems. The reason social curation works for finding information is that the people in a “circle” may be closer to the mind set of the person looking for information. Even if the social circle is clueless, the placebo effect kicks in and justifies the “good enough” method; that is, use what’s available and “make it work”, just like Project Runway contestants.
Third, smart software. Algorithms and numerical recipes, programmable search engines, fuzzy logic, and the rest of the PhD outputs are quite useful. The problem is that humans who design systems, by definition, are not yet able to create a system that can cope with the oddities that emerge from humans being human. So as nifty as Google is at finding a pizza joint in Alphabet City, Google and other systems may be wildly wrong as humans just go about their lives being unpredictable, idiosyncratic, irrational, and incorrect in terms of the system output.
I think there is innovation in search. Now my definition of innovation is very different from the Yahooligan’s. I am not interested is pointing out that most commercial and open source search systems just keep doing the basics. Hey, these folks went to college and studied the same general subjects. The variants are mostly tweaks to methods others know pretty well. After getting a PhD and going into debt, do you think a search engineer is going to change direction and “invent” new methods? I find that most of the search engineers are like bowling balls rolling down a gutter. The ball gets to the end of the lane, but the pins are still standing. Don’t believe me? Run a query on Ask.com, Bing.com, Google.com, or any other search system you can tap? How different are the results?
The challenge becomes identifying companies and innovators who have framed a findability problem in such a way that the traditional problems of search become less of an issue. Where does one look for that information? Not in blog posts from established companies whose track record in search is quite clear.
Stephen E Arnold, September 12, 2010
Freebie
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
