Search Innovation: Do IR Thought Leaders Recycle Old Ideas?
August 17, 2011
We are fast approaching our 60th interview in the Search Wizards Speak series. In June 2011, we completed The New Landscape of Enterprise Search, which involved its own series of interviews with engineers, search system customers, chief executive officers, and pundits.
A Paucity of Insights or Fear?
For many years, I have been interviewing entrepreneurs, developers, and investors about information retrieval, content processing, and headache inducing technologies such as entity extraction and natural language processing.
My team of goslings here in Harrod’s Creek the industry leaders like Exalead and some of the more interesting newcomers such as SearchLion. Next week, we release an interview with a fast growing company with headquarters in Europe. Some vendors don’t want to talk; for example, Google and Microsoft. Microsoft was in but then the “expert” disappeared. With the churn at Microsoft, I am just sitting on the sidelines. Other vendors and experts want to talk but don’t want to commit their ideas to a digital interview in a context of scores of other experts’ commentary.
Here’s the trigger for this summary of my thoughts from August 15, 2011. I listened to a podcast this morning when I was walking my trusty technical advisor, Max the Boxer.
The on air personality was Adam Carolla. Program is available via iTunes or from the Adam Carolla Web site. The segment of the program which caught my attention was Mr. Carolla’s interview with author and columnist Ben Shapiro, a Harvard lawyer. Mr. Shapiro is the author of Primetime Propaganda: The Tue Hollywood Story of How the Left Took Over Your TV. The air was cool and Mr. Max was chasing squirrels, so I listened to Mr. Shapiro’s observations about how certain well placed individuals feel uncomfortable in their life roles. Mr. Carolla mentioned that he too had observed that some individuals wear their wealth and station in life awkwardly. (I remember reading in 2003 Why Smart Executives Fail, which advanced some similar arguments.)
What’s this have to do with Search Wizards Speak and the interviews I conducted for The New Landscape of Enterprise Search?
I realized that in the interviews I have conducted over the last 32 months, only a few individuals were completely confident in their answers to my now-standardized questions about “What are the major trends in search?” and “What product enhancements will you be introducing in the next release of your product?” In one go round, not only did the interview take nearly four months to complete, the interview subject deleted my standard introduction, deleted my general observations about the interview, and rearranged the content of the interview so that it suppressed any hint of a personal touch for the interview subject. That’s okay with me. The information was interesting and not available elsewhere, so I ran with it.
My Sharpiro’s and Mr. Carolla’s comments struck a nerve because in the search and content processing industry, I think the same type of uncertainty and discomfort exists. Because search is miles away from Wall Street or Hollywood, the experts like Mr. Shapiro ignore software, choosing to focus on high profile topics that cater to a broader audience.
Fuzzy Is Popular
Let’s assume for a moment that Mr. Shapiro’s podcast observation is accurate. What is causing experts in search to be fuzzy, waffling, and uncertain about search and retrieval? (Remember. I am talking about the sample of interviews I have conducted and published, not about forthcoming interviews.)
First, I think that most vendors of search and content processing systems are facing pressures that may be greater that press upon other technology companies. Search and content processing is one of those complex areas which most people dismiss as “been there, done that.” The preeminence of search as a core application has been losing the high ground over the last three or four years. In fact, based on the research we conducted for my new monograph The New Landscape of Search, the shift may be accelerating. Search appears to be more of a utility function. The most successful of the content processing vendors—Exalead, to take one example—embed search in broader, often higher value enterprise solutions. A company selling brute force indexing or a component to improve the indexing of entities is like to find its market becoming less top management level and more information technology staff level. I think this introduces uncertainty in how a search and content processing company can position and price its technology.
Thanks to the creative whiz at http://planetpov.com/2011/07/25/uncertainty-in-business-will-it-become-sustainable/
Second, the every day user of a free Web search system or a person doing customer support work in a big company expects a search box. The habit of banging two or three words into the search slot machine and getting out an information payoff is routine. Search and content processing vendors talk a great deal about improving productivity, but the reality is that most users don’t know if the information provided is right or wrong. Most just use what’s at the top of the results list. My hunch is that the increasing dissatisfaction with search is a warning signal that the brute force approach, although ubiquitous, is not working. The client, on the other hand, is okay with good enough. As a result, a vendor trying to explain how to improve a search box function has a long, expensive, and arduous sales process. The top dogs in search and content processing companies want results, but the folks selling the product are not sure what to say to close the deal and keep its options open with other prospects. Not surprisingly, when one reads the nearly 60 interviews, there is a note of sameness that threads through the write ups. The companies that say something different—Autonomy or Exalead, for example—stand out. Many of the others seem quite alike. I will leave it to you to draw your own conclusions.
Third, I think that experts in search and retrieval know the “hidden lore” of the search and content processing sector. Examples range from the cost and challenges of scaling when usage and the volume of content to process rises rapidly to the complexity of keeping indexes fresh. There are other issues like controlled term lists and taxonomies. Experts on narrow topics get big pay days but do little to head off cost and findability crises with their Rube Goldberg diagrams. These seminars should come with a free baloney and corn chip lunch. Empty calories for empty presentations, right? (For the scoop on controlled term lists, check out the resource of Access Innovations, a company with more than 30 years’ experience in this taxonomy niche of search and content processing.) Some of my interview subjects steer far away from the cost overruns, the problems of getting system back online when the indexing updates go off the rails, or when content which has been processed and directly germane to the user’s query does not appear in the results list or other output. In short, the “hidden lore” of search and content processing makes interview subjects cautious, even uneasy. Maybe it is better to say the same thing as other interview subjects than go down a different path.
My Thoughts from my Chat with Max, the Wonder Boxer
Back to Mr. Shapiro and Mr. Carolla.
The podcast triggered several thoughts which I want to capture.
- Running a search and content processing company is a tough job. Not only is the technical area complicated, the market sees search as a utility. This shift alone may be enough to turn even a stalwart into a pussy cat. Search, therefore, must fit into a bewildering range of applications and situations. Specificity could kill a sale. So fuzziness is the basic guideline for some executives.
- Faux experts, search engine optimization consultants, and clueless MBAs can derail a sale. The problem of explaining what a particular search and content processing vendor’s system actually does seems to be an almost impossible task. Larger firms with more resources are likely to enjoy an advantage in some marketing and sales situations. Bad news for start ups and smaller firms? I don’t know but it seems like making sales today is just tougher than it was in 2004.
- Indifferent users. I think the employees and users of search systems lack the training and foundational knowledge to know when information generated by a system is off the mark. The shift to “search without search” systems is a response to the lack of intellectual rigor brought to what a computer or mobile device spits out as an “answer.” It is difficult to make a knowledge based sale when the users don’t know what’s right or even close to right from what’s dead wrong and sort of wrong.
- Generating content within a search related company is becoming more and more difficult. Staff take their cue from top management. So when there is a dearth of information about a product or service or when routine communications appear days or weeks after the media opportunity presents itself, I perceive an internal friction that slows the flow of information from the company to a prospect or to the media. In one situation, we found that the firm’s public relations firm was paid to reinforce the non production of information. The reason we learned was that because the company did not know what to say, the public relations firm was not able to write even basic news releases. (Your mileage may vary depending on your PR firm.)
The bottom line is that I understand why many search and content experts waffle, talk in non specific ways, and suggest their technology solves problems that range from aerospace documentation to zoological data analysis. The problem for me is that making informed decisions about search and content solutions continues to become more difficult.
Who would have thought that a comedian and a Harvard lawyer could trigger some thoughts about the plight of search and content processing vendors? Well, that’s why as a 67 year old addled goose, I live in Harrods Creek, avoiding the real work of real consultants and real CEOs.
Stephen E Arnold, August 17, 2011
Sponsored by Pandia.com, publishers of The New Landscape of Enterprise Search
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
