• Home
• About
• Writers
• Landscape
• Wizards Index
• Fraud
• Wanna Be Like Larry?

Repositioning 2011: The Mad Scramble

Yep, the new year fast approaches. Time to turn one’s thoughts to vendors of search, content processing, data fusion, text mining, and—who could forget?—knowledge management. In the last two weeks, I have done several live-and-in-person briefings about ArnoldIT.com’s views on enterprise search and related disciplines.

Today enterprise search has become what I call an elastic concept. It is stretched over a baker’s dozen of quite divergent information retrieval concepts. Examples range from the old bugaboo of many companies customer support to the effervescence of knowledge management. In between the hard realities of the costs of support actual customers and the frothy topping of “knowledge”.

Several trends are pushing through the fractured landscape of information retrieval. Like earthquakes, the effects can vary significantly depending on one’s position at the time of the event.

Source: http://www.sportsnet.ca/gallery/2009/12/30/scramble_gal_640.jpg

Search can looked at in different ways. One can focus on a particular problem; for example, content management system repositories. The challenge is to find information in these systems. One would think that after years of making Web pages, the problem would be solved. Apparently not. CMS with embedded search stubs trigger some grousing in most of the organizations with which I am familiar. Search works, just not exactly as the users expect. A vendor of search technology can position the search solution as one that makes it easy for users to locate information in a CMS. This is, of course, the pitch of numerous Microsoft Certified Gold resellers of various types of search solutions, utilities, and work arounds. This an example of a search market defined by the type of enterprise system that creates a retrieval problem.

Other problems for search crop up when specific rules and regulations mandate a particular type of information processing. One example is the eDiscovery market. Anyone can be sued, and eDiscovery systems have to make content findable, but the users of an eDiscovery system have quite particular needs. One example is bookkeeping so that the time and search process can be documented and provided upon request under certain conditions.

Social media has created a new type of problem. One can take a specific industry sector such as the Madison Avenue crowd and apply information technology to the social media problem. The idea is for a search system to “harvest” data from social content sources like Facebook or Twitter, process the text which can be ambiguous, and generate information about how the people creating Facebook messages or tweets perceive a product, person, ad, or some other activity for the advertising team. The idea is that search unlocks hidden information. The Mad Ave crowd thinks in terms of nuggets of information that will allow the ad team to upsell the advertiser. Search is doing search work but the object of the exercise is to make sense out of content streams that are too voluminous for a single person to read. This type of search market—which may not be classic search and retrieval at all—is closer to what various intelligence agencies want software to do to transcribed phone calls, email, and general information from a range of sources.

Let’s stop with the examples of information access problems already. There are more information access problems than at any other time, and I want to move on to the impact of these quite diverse problems upon vendors in 2011.

Now let’s take a vendor that has a search system that can index Word documents, email, and content found in most office environments. Nothing tricky like product specifications, chemical structures, or the data in the R&D department’s lab notebooks. For mainstream search, here is the problem:

Commoditization

Right now (now pun on the vendor of customer support solutions by the way) anyone can download an open source search solution. It helps if the person downloading Lucene, Solr, or one of the other open source solutions has a technical bent. If not, a local university’s computer science department can provide a student to do the installation and get the system up and running. If the part time contracting approach won’t work, you can hire a company specializing in open source to do the work. There are dozens of these outfits bouncing around.

The point is free. One doesn’t need an MBA from a top flight business school to figure out that making money by selling consulting services is probably more profitable than trying to sell proprietary general purpose search software.

Differentiation is essential, and commoditization’s impact on search vendors has been immediate and direct. Examples may be found by running a query for “enterprise search” on Google and looking at the advertisements. When I ran the query “enterprise search” (December 12, 2010), there were three products on what is typical Google results list: open source from Lucid Imagination, the Google Search Appliance from dear old Google, and Microsoft SharePoint search. The general purpose search vendors are not dominating the results list or the ads as they did three years ago.

Commoditization, boosted by the growing awareness that general purpose search does not work particularly well, has highlighted vendor names of companies that have diversified in a successful way; for example, Autonomy, Endeca, and Exalead. There are also the names that one no longer expects to see: Convera, Delphes, and Entopia.

Results for the query “enterprise search”.

One consequence of commoditization of basic search has been what I think will be a major trend in 2011. My term for this is

Faux Specialization

Here’s how this is working at this time. A vendor with a so-so general purpose search solution realizes that selling against free or low cost commodity search is a one way ticket to the poor house. I profiled a search vendor’s marketing campaign that makes use of warez services and little-known freeware and shareware downloading services. Who can compete when one of the Big Guys in enterprise software tosses in a search system if the client buys hardware, software, or some combination of database and back office systems.

When I read about a search vendor delivering a vertical solution or solving a horizontal problem like indexing email for eDiscovery, I chuckle. The faux specialization is, more often than not, repackaging the same, tired general purpose search within wrapping paper with entertaining marketing jargon printed like holiday wrapping paper. Instead of Happy New Year, the search vendor’s product wrapping says business intelligence, eDiscovery, fraud detection, customer support, or some concoction like “data fusion” (derived from various government initiatives with “fusion centers”.

In 2011, I think quite a few of these vendors will be caught out. We are tracking the data fusion sector in our Inteltrax.com blog. If you want to know some of the big players in this space, navigate to www.inteltrax.com. What you will find is that most of the names of companies are not known in the world of enterprise search procurement teams. The reason is that the Inteltrax crowd offers quite specific content processing and information access systems. When you kick the tires of some of the more interesting vendors—for example, Digital Reasoning—you learn you can do search. But the architecture and technical approach is different and, I believe, ultimately more useful than an “old” general purpose search solution marketed like a specialized content processing system for those engaged in identifying possible medical or credit card fraud.

A comparison of what a Fetch Technologies or an i2 Ltd. solution does and what a repositioned enterprise search system does is a useful exercise. Fetch and i2 work with content but the approach, methods, and outputs are just different. Some would say that these outputs are actionable or that the outputs tell the user what’s important.

Here’s the point: faux specialization is often cosmetic, a marketing trick. The licensee may be left asking, “Where’s the beef?” like the grumpy grandmother in the old Wendy’s TV commercials. The “beef”, it turns out, is on the ranch owned by companies who are not repackaging 1990-2000 era information retrieval systems. The “beef” comes from a different country, partner, and that country is a long ride from the skinny cattle in the enterprise search feedlot.

As we stumble toward 2011, most organizations are struggling with information access. This brings me to the third change taking place in enterprise search.

User Experience

The push for a user experience is an outgrowth of industrial design. I attribute the renewed interest in UX (shorthand for interface eye candy) to Apple. The problem is that Apple has designed software, the hardware, and the information work flow. Most search vendors just slap a bit of lipstick and eye shadow on the ageing show girl, slap her on the back, and send her out to dance on stage. It’s an act. The “willing suspension of disbelief” works for theater. Fakery does not work particularly well when a client wants a system to perform a specific task in information access. A visit to the make up room disabuses the licensee of a search system about what the trooper looks like in bright light. In 2011, I expect lots of theatricality. The audience may no longer be fooled. The economic climate reduces the slop in the budget and the margin of failure may force change. Abrupt change. If the system does not work for its users, the interface may be a tiny part of the problem.

Image source: http://www.racheshop.de/product_images/images/big/4082E_d1.jpg

The User Wake Up Call

Organizations are struggling with findability. One can read about the problems of information access on the LinkedIn Enterprise Search Engine Professionals group. I don’t comment too much because I find LinkedIn’s Enterprise Search Engine Professionals group a rich source of insight about the general cluelessness that seems to pervade organizations in the US and probably elsewhere as well.

In my opinion, a wake up call will ring about the inappropriateness of general purpose, old style key word, unstructured information information retrieval systems. Despite the lipstick on the pig, the underlying technology is going to create problems for those trying to solve highly specific problems with a general purpose system. In some cases, the underlying architecture is focused on completely different problems; for example, eCommerce, metatagging/indexing, or processing general purpose content, not the new content that arrives in bandwidth clogging quantities every minute, 24×7.

The wake up call will trigger a larger number of search systems that have to do no-cash mergers. The idea is that two or more weak chickens can team up and become a barnyard giant. There will be more tragic failures which cost clients lots of money and disrupt many people’s lives. The implosion of Convera is one example from 2011. There will be more. When systems don’t work or can’t deliver what’s in the marketing collateral, the backlash will be powerful, maybe vicious. In a lousy economy, what organization can tolerate wasting resources for something that does not solve the problem. I can name six vendors of general purpose search solutions which are walking a thin line between solvency and outright failure. (Sorry, I can’t name these high-risk outfits in this blog. I want to name them. I think you want to know which vendors in my opinion are selling old style solutions presented as a solution to today’s specialized information access problems. Do your home work. The facts and data are publicly available. You can read why one North American government agency did not buy a well known general purpose search engine.  The reason? Would not scale.)

Here are some possible consequences of the 2011 wake up call:

1. Litigation. This year (2010) involved a number of high profile vendors of information access technology. I think the possibility of even higher stakes litigation will be felt in 2011. Joining the fray will be government agencies and companies unhappy with information access solutions which did not work or do not meet the specification for the contract. People will want their money back and probably an extra cash sweetener. Larry Ellison just demanded $200 million in interest payments from the hapless SAP organization. More like this in 2011.
2. Consumer product like advertising/marketing. Vendors flogging old technology will just rebrand their ageing product and try another market. There’s no constraints on a vendor describing a 1996 search solution as an eDiscovery or business intelligence system. With Web pages and blogs that can be changed quickly, it can be difficult to grasp the implications of the repositioning. New customers will only know what the vendor pitches. Historical investigation is tougher than ever in the new world of Google Instant. Consultants are often little more than public relations companies. Senior technical staff want to avoid getting saddle with a nag like old-style enterprise search. As a result, marketing will be more important than technology in 2011.
3. Secrecy. Most software companies are off the radar. No one in the “real media” cares about companies that do complex software. The exceptions are the Googles, IBMs, Microsofts, and Oracles, but the coverage of these companies’ search solutions is lousy. When you droip down to information access vendors, the information is of little or no interest to the “real media”. I think vendors of information access solutions will try to operate secretly and rely on marketing to present a public face to the company. The result is that no really knows what the software does or whether it works. As for a customer list, forget it. You can see how secrecy about search and content processing operates now. Just go to Google and run a query for Teratext. Now find out who owns the company. Next find out the name of a customer. You can do it, but Teratext is not exactly running a Web site like MTV’s. Getting information in 2011 will be more difficult. And, more disturbing, getting factually accurate information about search and content processing vendors will become increasingly difficult. Secrecy will be necessary to prevent catastrophic implementations from being public and to control the flow of information about software that may not work as advertised or at all. More search vendors will clamp down on information about their implementation failures.

I am tempted to write about the self-appointed experts in search and content processing. The field is populated with special librarians, English majors, jobless Web masters, and terminated journalists. The information about search reflects the silliness that comes from writing about search without having been deeply involved in the complexities of building, installing, and maintaining search and content processing systems. Amateur commentary is not tolerated in football announcing, but it is rife in search and content processing. The quality of the consultants provides proof of the knowledge among professionals charged with the responsibility for procuring an information access system that works. Scary.

Next year (2011) will be a mad scramble. Information access is important, and I think the slipping, sliding, dipping, and masquerading will be a defining characteristic of many vendors of search and content processing systems. There are vendors who deliver solutions that work. I can name one or two in each of the verticals and in the horizontal market spaces. The problem is that customers have to realize the problems their present information access systems create, the burden placed on the financial resources of their employer, and the flawed data delivered to colleagues who accept what’s served as gospel.

So 2011, the year of misleading marketing and a year of increasing dissatisfaction with search and content processing. On the bright side, for the vendors with solutions that work, 2011 will be a reasonable year. Is your vendor based on old technology or new technology? Are the users of your present information access system satisfied? Does your organization have the information it needs at its fingertips? Does your search system keep the indexing fresh? Can you find the document you need for a meeting tomorrow?

Most cannot. That’s the real secret in enterprise search. That’s the reason I see 2011 as the Year of the Mad Scramble. I also see many companies licensing technology that will not solve their specific information access problem. At a time of financial stress, that is not just uncomfortable, it is wrong.

Stephen E Arnold, December 15, 2010

Freebie, unlike the license and consulting fees from vendors and so-called search experts

Comments

Copyright © 2008 Beyond Search · ArnoldIT.com · Disclaimer · Revolution News theme by Brian Gardner · Log in