• Home
• About
• Writers
• Landscape
• Wizards Index
• Fraud
• Wanna Be Like Larry?

Fast Cash, Faster Crash

On July 3, 2008, Erick Schonfeld summarized the continuing saga of Fast Search & Transfer’s fastest move ever. The story “Did the Enron of Norway Pull a Fast One on Microsoft? More Details about the Mess at Fast Search $ Transfer? is here.

The story is quite thorough, according to my sources in Norway, and there is little I can add to the TechCrunch write up.

I would like to highlight one point, provide the links to my analysis of the Fast Search saga, and offer several observations about the nature of enterprise search. Before I start, take a look at this graphic because this is the wild bobsled ride that many vendors are queued to take:

Once a vendor starts down the sales bobsled run, it is tough to stop. The vendor has to ride to the bottom of the hill, hoping that he will not crash, rising serious injury and maybe death.

The Key Point for Me

After reading the TechCrunch essay, one segment gnawed at me; specifically:

…It [Microsoft’s paying $1.2 billion for Fast Search & Transfer] does point to a certain blindness on the part of Microsoft, or at least a willingness to look the other way, in its obsessive quest to become a player in search (see Yahoo and Powerset). It also raises questions about Fast’s underlying search technology. If Fast was having trouble closing deals for its products, how good can its technology really be?

Yes, this is the key question. The Fast Search & Transfer core technology was purpose built to index static Web sites. At the time Google started operations, AltaVista.com was an orphan, quickly losing its leadership position due to the voracious demand for resources that public Web search engines demand. The mantra is “Feed me computing resources or dies”.

Fast Search offered a Web site called AllTheWeb.com, and it was pretty good. At the time of 9/11, the AllTheWeb.com news indexing system was among the first to have reasonably timely information. Fast Search made a fateful decision in 2002 which led to Fast Search & Transfer’s exiting the Web indexing business. Fast Search sold its Web indexing business to Overture for $70 million with more money promised if certain goals were achieved. Fast Search took the money and focused on enterprise search.

The decision, as I recall my conversations with Fast Search & Transfer executives, when I was involved in the Fast Search deployment for a government project was that enterprise search was a great opportunity. Fast Search’s executives suggested to me that the company could move quickly to dominate the search market. At the time, there was little reason to doubt the confidence of the Fast Search team. A Fortune 50 was backing the Fast Search system in the government-wide indexing program. In the 2002-2003 time period, there were not too many systems that could demonstrate an index of 40 million documents. Even today, licensees of search systems do not grasp the hurdles that indexing large amounts of text puts in front of an organization. I have written extensively about this elsewhere, and I have little to add to the ignorance about search scaling that continues to plague organizations.

In order to refit the Web indexing system to handle enterprise content, Fast Search did what most vendors do. The Web crawler, indexer, query processor, and administrative components were the foundation of the system. To add the needed functions to handle enterprise search, Fast Search wrote more original code, recycled some open source code taking care to stay within the boundaries for this type of software, licensed some software from companies, and purchased other firms. The details of some of this technology appear in the first three editions of Enterprise Search Report, which I wrote. Other information appears in my Beyond Search study in the discussion of the Microsoft buy out of Fast Search, so I won’t rehash the material.

The consequence of this type of engineering is unknown dependencies. What this means is that if I fix a script to address a problem in one component–say, hit boosting–that fix can create another problem in some other part of the system. Fast Search is not alone is having complex dependencies within its code. The gotcha for Fast Search is that only engineers with deep experience in search and the Fast Search code can make changes without create unexpected issues elsewhere. As Fast Search puts more installations in place, the demand for these types of specialists increases. You can’t run an ad on Craigslist.org and get a programmer who can fix a dependency.

So, Fast Search’s problems began as soon as the company decided to push into the enterprise search market. The adjustments were, as noted in the documents I cited in my previous Fast Search analyses and in the TechCrunch article, small at the outset. Who knew that a customer would not pay his license fee installment? Then more customers groused about slow installs and the up front payments were not followed by any other payments. One Fast Search licensee told me that his Global 1000 company would not pay until Fast Search produced an engineer who could complete the installation per the task order. Well, Fast Search got an engineer to the client, but it was six months after I heard the complaint. Not surprisingly, this big outfit turned to a smaller vendor who got a different system up and running in three weeks.

So, Fast Search made a technology-based decision that contributed to the company’s financial problems. Once the bobsled started down the mounting, Fast Search’s senior managers had to hang on. A few jumped off, but most of the Fast Search team raced to the bottom of the mountain with the consequences reported in the TechCrunch write up.

Over the years, I have spoken with various Fast Search executives. My observations about ways to address the problems I saw mounting were dismissed as the honking of an addled goose. I wrote off the company as a serious player in enterprise search for three generally subjective reasons: [a] management did not perceive that it was on a bobsled racing down a mountain propelled by forces management could not easily control. Management thought it was strolling on Las Ramblas in Barcelona, enjoying the evening breeze. [b] the dependencies were not going to be resolved because once a big chunk of software is up and in the market, no one has enough money to go back to square one to fix the problems. Don’t believe me? We lived with some horrors in The Point before we sold to Lycos.com in 1995. Alternatively, just check out auto numbering in Microsoft Word 2007. Real life is that one works around the bugs and lives with them. [c] Newer search systems were becoming available with more homogeneous code and tighter engineering. The reason I am eager to point out the virtues of up-and-coming vendors like Coveo, Exalead, and ISYS Search Software is that these systems are more homogeneous than older search systems. This engineering feat makes it generally less risky to make fixes without causing surprises unknown and undocumented dependencies bring with a trivial script change.  I continue to read baloney about enterprise search systems in reports written by “experts” with zero–yes, zero–direct experience in anything other than using Google. Screw ups in enterprise search are becoming more widely known. I am not reluctant to point out where a pitfall lies. Identifying the “top” or “best” systems without hard analysis is wacky and potentially risky.

List of Fast Search Write Up from Beyond Search

To get a full picture of my view of Fast Search & Transfer, you will have to get your hands on two for-fee studies. I do work for hire, so you will have to buy these from their publishers.

• Enterprise Search Report (2004-2006), specifically the second and third editions. The fourth edition (2007)material was trimmed by the young person who is carrying on this encyclopedia of enterprise search. Contact CMSWatch here.
• Beyond Search (2008), published by the Gilbane Group here.

The essays on my Web log, subject to the disclaimers, are:

Fast Financials: Three Day Old Fish Should Be Discounted : Beyond Search
http://arnoldit.com/wordpress/2008/05/30/fast-financials-three-day-old-fish-should-be-discounted/

Thoughts on Microsoft Buying Fast Search & Transfer : Beyond Search
http://arnoldit.com/wordpress/2008/01/08/thoughts-on-microsoft-buying-fast/

The Fast Follies: A Math Error
http://arnoldit.com/wordpress/2008/05/29/the-fast-follies-a-math-error-maybe/

Microsoft Chomps and Swallows Fast : Beyond Search
http://arnoldit.com/wordpress/2008/04/26/microsoft-chomps-and-swallows-fast/

A Turning Point in Search? Is the Microsoft-FAST Deal Significant? : Beyond Search
http://arnoldit.com/wordpress/2008/01/11/a-turning-point-in-search/

Fast Search: It’s Pretty Easy but Training Helps : Beyond Search
http://arnoldit.com/wordpress/2008/04/19/fast-search-its-pretty-easy-but-training-helps/

MSFT – FAST: Will It Make a Difference? : Beyond Search
http://arnoldit.com/wordpress/2008/01/16/msft-fast-will-it-make-a-difference/

Microsoft Fast: A 45-Day Innovation Cycle Yields a Web Part : Beyond Search
http://arnoldit.com/wordpress/2008/06/21/microsoft-fast-a-45-day-innovation-cycle-yields-a-web-part/

Does This Mean Users Can Downgrade from Fast ESP to MOSS or ESS? : Beyond Search
http://arnoldit.com/wordpress/2008/06/04/does-this-users-can-downgrade-from-fast-esp-to-moss-or-ess/

Ovum Says, `Microsoft Has a Plan’ for Search : Beyond Search
http://arnoldit.com/wordpress/2008/05/24/ovum-says-microsoft-has-a-plan-for-search/

You may find these two “thought pieces” useful as well. The train wrecks essay is the all-time most read essay on this Web log.

Enterprise Search and Train Wrecks : Beyond Search
http://arnoldit.com/wordpress/2008/05/07/enterprise-search-and-train-wrecks/

The Disappearing Middle: Liposuction for High-Profile Search Vendors : Beyond Search
http://arnoldit.com/wordpress/2008/04/05/the-disappearing-middle-liposuction-for-high-profile-search-vendors/

Observations

Let me wrap up this essay with several observations:

1. Search is hard, whether desktop, enterprise, or Web. Search is getting harder, more expensive, and more complex. No single search system is particularly good for the range of queries users fire at it or the applications licensees require. Multiple systems will be an increasingly common path. One size fits all doesn’t work in men’s suits, and it won’t work in organizations.
2. IT departments routinely believe that search is a no brainer. Ah, but it is a brainer. The cocktail of illogical StarTrek inspired requirements, a vendor with a Flash demo, a charming sales person, and an blissfully ignorant IT department can make a casual get together quite interesting. Sadly the blend is as common as coffee in most organizations.
3. Search is now morphing into stuff like reports that eliminate the need for an employee to read documents and figure out what they mean. Social functions are now more important  because “social” really means to some procurement teams, “Hey, I can ask someone so I don’t have to read the reports myself plus I can shift the responsibility to our resident expert”. In their quest for knowledge, organizations are embracing a populated with professionals who don’t know the business. I find this unnerving.

I know that the Fast Search “problem” is not particular to Fast Search. I know of other companies who are now clinging to their own bobsleds, racing down the frozen groove Fast Search followed. Do you know which companies these are? If not, you need to figure it out before you sign a license agreement. I can’t name all of these bobsled riders, but I track 50 companies and 20 percent are on the mountain.

Also, keep in mind that enterprise search is dead. It generally dissatisfies 50 to 75 percent of the systems users. Most vendors use new buzzwords, not engineering, to make sales. Many organizations don’t value search, and when search is needed, most professionals want a search system that tells the professional what he or she needs to know. Skip the graduate school type work. Just give me a sound bite.

If you work in an organization with a search problem, your 4th of July explosion will be coming. You just don’t know when the system will blow up.

Stephen Arnold, July 4, 20008

Stephen Arnold, July 4, 2008

Comments

Copyright © 2008 Beyond Search · ArnoldIT.com · Disclaimer · Revolution News theme by Brian Gardner · Log in