• Home
• About
• Writers
• Landscape
• Wizards Index
• Fraud
• Wanna Be Like Larry?

Enterprise Search: Fee Versus Free

I read a pretty darned amazing article “Is Free Enterprise Search a Game Changer?” My initial reaction was, “Didn’t the game change with the failures of flagship enterprise search systems?” And “Didn’t the cost and complexity of many enterprise search deployments fuel the emergence of the free and open source information retrieval systems?”

Many proprietary vendors are struggling to generate sustainable revenues and pay back increasingly impatient stakeholders. The reality is that the proprietary enterprise search “survivors” fear meeting the fate of  Convera, Delphes, Entopia, Perfect Search, Siderean Software, TREX, and other proprietary vendors. These outfits went away.

Many vendors of proprietary enterprise search systems have left behind an environment in which revenues are simply not sustainable. Customers learned some painful lessons after licensing brand name enterprise search systems and discovering the reality of their costs and functionality. A happy quack to http://bit.ly/1AMHBL6 for this image of desolation.

Other vendors, faced with mounting costs and zero growth in revenues, sold their enterprise search companies. The spate of sell outs that began in the mid 2000s were stark evidence that delivering information retrieval systems to commercial and governmental organizations was difficult to make work.

Consider these milestones:

Autonomy sold to Hewlett Packard. HP promptly wrote off billions of dollars and launched a fascinating lawsuit that blamed Autonomy for the deal. HP quickly discovered that Autonomy, like other complex content processing companies, was difficult to sell, difficult to support, and difficult to turn into a billion dollar baby.

Convera, the product of Excalibur’s scanning legacy and ConQuest Software, captured some big deals in the US government and with outfits like the NBA. When the system did not perform like a circus dog, the company wound down. One upside for Convera alums was that they were able to set up a consulting firm to keep other companies from making the Convera-type mistakes. The losses were measured in the tens of millions.

Endeca, a company that seemed to hit a revenue wall at about $160 million per year in revenue, sold out to Oracle for $1.2 billion. Endeca is now, like InQuira and TripleHop, a component in Oracle’s meat and potatoes data management system. Endeca’s late 1990s technology features some computationally interesting methods that can sing when paired with Oracle’s super-power computing and storage devices.

Exalead also hit a revenue wall in the 2006-2007 interval. The French company sold to Dassault Systèmes. Exalead is now supporting Dassault’s infrastructure services and has largely disappeared from the US market. Exalead, unlike a number of proprietary search systems, worked. Nevertheless, the information retrieval system seems to be struggling for purchase within a parent that faces financial challenges itself. “Where are those billions in revenue from search?” is a question some Dassault stakeholders may be asking.

Fast Search & Transfer struggled in 2006 and 2007 to hit its financial targets. The company was able to sell itself to Microsoft in 2008. Almost immediately the investigation of Fast Search for financial missteps kicked into high gear. Microsoft, to get a handle on the challenges of Fast’s technology, killed the Unix version and essentially made Fast Search a basic service in SharePoint. One of the founders of Fast Search ended up with a negative decision and a restriction on working in search. How big was the misstep? Fast Search inflated its revenues by a sufficient amount to produce a jail term. Think tens of millions.

ISYS Search Software, like Brainware, sold to Lexmark. Now those products are utilities within larger, higher value systems. Search has become a utility or a back office function.

Vivisimo shifted from an enterprise search company to a Big Data solution and sold itself to IBM for about one year’s revenues, an estimated $20 million. Vivisimo has disappeared.

I could provide thumbnails of Delphes (academic search gone wild) or Entopia (a modern day version of “we are really information infrastructure.”) There are companies that came and sort of floundered. These include Dieselpoint (an XML system), Hakia (semantic search), and IBM with its mainframe centric STAIRS system. There are companies that have survived but exist in niches; for example, EPI Thunderstone, dtSearch, and AMI Albert (which seems to have gone quiet).

The point is that IBM, arguably once the most proprietary outfit in computing deep sixed its home grown, complex search systems for Lucene is another milestone. IBM figured out that it was better to rely on Lucene for nuts and bolts search. Expensive IBM engineers could then, as I understand it, focus on artificial intelligence and win TV game shows and deliver recipes. Other companies followed IBM’s lead and embraced open source search solutions along with other open source software. Linux, anyone?

The reason free and open source search solutions are capturing the attention of large companies and many developers is dues to a number of factors:

1. Proprietary systems did not deliver on marketers’ promises and left a scorched earth environment for many information technology professionals responsible for an organization’s software.
2. Lucene and the Solr variant delivered basic keyword search and even faceted searching without a brutal upfront licensing fee.
3. Lucene, Solr, and other open source search systems did not arrived with licensing handcuffs. Proprietary systems limit what a licensee can do.
4. Bug fixes, while not flawless, generally arrive more rapidly for open source search solutions than for proprietary systems. The logic is that the licensee of a proprietary system has to wait for the vendor to do a fix, pay for it, and then upgrade to get the fix to run.
5. Organizations wanted to integrate search easily into more sophisticated enterprise applications. Many search systems provided SDKs and APIs. The problem was that these were often complex and required an expensive company-provided or company-authorized engineer to do even simple tasks.
6. Proprietary means not open. For organizations wanting to move beyond search, the market wanted an open source solution with professional support, not closed systems.
7. Perceived economies. If the customer perceives open source as a better value, a proprietary enterprise search vendor has to offer more than platitudes like “information governance,” “customer support,” and “intelligent content.” Jargon does not deliver a solution. Jargon describes an opportunity another, more innovative company can leverage into a sale.

Outfits from Attivio to Palantir have their roots in search. These companies use open source search technology. Avashka uses an open source analytics system in its enterprise solutions. The point is that the solution, not the claims for the value of enterprise search, causes open source search to be used in a product or service.

Now what about the write up that suggests free and open source search software is not able to deliver functions like personalization? Nonsense. Enterprise search changed after the wave of sell outs. In the gap this created, open source search flowered.

If I were to set up an enterprise search system today, I would use Elasticsearch, the cloud, virtualization, and the other technologies that have removed the ball and chain from the feet of organizations wanting functional, stable, open information retrieval.

I would then focus on building solutions that solved specific needs of importance, not flogging a dead horse about “indexing all of an organization’s information.” No one wants human resource, salary, or research information on an insecure enterprise search system. Proprietary enterprise search vendors have found that security is an issue with many proprietary systems.

Users want information access. Users want answers. Users want systems that do not force them to scan a results list, open documents, read them, and then assemble bits and pieces to answer a question. Licensees want solutions to pressing information problems, not systems that increase the amount of work a person has to do to get a simple item of information. No one wants to learn query languages, guess which secret keywords unlock the information treasure chest, or deal with outputs that have no discernable relationship to the question a system user has.

Proprietary search system vendors today have to deliver a high value product that meets the needs of today’s mobile and often office-less user. It is useful to keep in mind that the founders of the Autonomy-type systems were smart. That did not mean that the management of these companies did not try to sell out before the brutal economics of search undermined their firms. It is a sad commentary on the economics of enterprise search when a brilliant Ph.D. has to resort to financial razzle dazzle to keep his company afloat until it could sell to a larger and obviously careless larger company. Even the mighty Google is struggling with its proprietary Google Search Appliance. If Google can’t figure out how to dominate enterprise search, what company can?

Today the information retrieval methods of the period from 1960 to 1990 have been milked. The cow is dying in a field with lots of dead animals. Open source search gives innovators an opportunity to create a next-generation information access system. My view is that most of the successful NGIA vendors will use open source search and invest in creating something new, not reinventing endlessly the “wheel” of keyword search, facets, and handling mixed types of data/information.

The game has changed. Period.

Stephen E Arnold, November 25, 2014

Comments

Copyright © 2008 Beyond Search · ArnoldIT.com · Disclaimer · Revolution News theme by Brian Gardner · Log in