The Importance of Being First
April 11, 2008
Alex Moskalyuk’s Web log contained a posting on April 10, 2008, that asserted “68 percent of search engine users click on the first page of results.” The story appeared in his Web log on Ziff-Davis’ ZDNet.com site. These data can be tough to find after a few days. Please, access the story and capture the data, which are from iProspect, a unit of the Aegis Group.
I am skeptical of usage data from Internet consultancies and search engine optimization companies. With that caveat in mind, the iProspect data reveal a significant trend in search system user behavior. Specifically, over time–if the data are accurate–users click on the first page of results only. The chart below illustrates this trend:
The top line is climbing, and it means that almost half of the users on Web search systems click on the first page of results. No real surprise, I suppose. The two other lines underscore the fact that fewer and fewer users are working through laundry lists of results. If these data are accurate, information on any other than the first page is not likely to get reviewed by a user.
What’s this mean for enterprise search (sometimes called Intranet search or behind-the-firewall search)? Users won’t spend much time looking for information if it is not slapped in front of their face. Key word search in organizations is generally a push cart filled with items that may or may not be pertinent to the employee’s query. If consumer behavior carries over to enterprise searchers, any system that takes a query such as “Acme proposal” and generates lists of results is going to be annoying.
Enterprise search system users need information to do their jobs, so the laundry list is almost a cinch to be more work than hunting for the needed information in other ways.
The iProspect data have another hook for me. As more young people enter the work force, Web behaviors are going to color their expectations of online search in their employer’s organization. Faced with laundry lists when Google and Microsoft personalize results, using probabilities to deliver a best guess about what’s needed by a particular person, traditional search systems in an enterprise are going to attract fewer and fewer enthusiastic users.
With the attention reports about deep-seated dissatisfaction about traditional enterprise search and content processing systems becoming more widely known, Mr. Moskalyuk’s Web log has provided another chunk of suggestive, interesting data. More details about enterprise search are needed, but in the search business, we have to take what the vendors provide. Like it or not.
Stephen Arnold, April 11, 2008
Enterprise Search: Disappointing and Annoying Users
April 11, 2008
Sinequa Shines a Bright Light in Enterprise Search’s Darkest Corner
Sinequa published the results of a survey of 200 users of enterprise search (sometimes called Intranet search or behind-the-firewall search). Users are dissatisfied and work in “information grave yards.”
Enterprise search is different from Web search. A Web search system such as those available from Google.com or Yahoo.com index content on the public Internet. Enterprise search indexes information that an organization has on its own servers. The differences boil down to search technology itself. What works on billions of Web pages is not well suited for the content on an organization’s servers. Sure, there are some gross similarities, but security, the diverse nature of the content, and the existence of many versions of certain documents require special functions generally not implemented in a public search system such as Microsoft’s search.live.com. Microsoft, in an effort to get technology suitable for enterprise search, is paying $1.2 billion for Fast Search & Transfer, a company that has asserted leadership in enterprise search. Fast Search’s executives have 1.2 billion reasons to make that claim.
The enterprise sector is an intensely competitive business sector. The definition of the word search itself is fluid and subject to many different shades of meaning. In the last five years, key word retrieval–typing one or two words into a search box–has expanded to embrace point-and-click interfaces. Here’s a point-and-click interface available from Yahoo. Notice the search box. But the most important parts of the Web page are those that contain suggestions, hot links, and information directly germane to a user. In days gone by, this would have been called a portal. Not today. Now these assisted navigation interfaces are called search.
These new interfaces have a large number of moving parts. You can see some of the plumbing in the illustrations accompanying these Entopia and Sagemaker business case analyses published on this Web log. Enterprise search means hugely complex systems that find themselves a cross between a digital Swiss Army knife and a computerized information factory. This combination of very specific tools and a huge, sprawling technical infrastructure make many of these systems expensive and complicated.
Until the Sinequa study, which corroborated the findings in my new study Beyond Search: What to Do When Your Enterprise Search System Doesn’t Work, few people were aware of the growing dissatisfaction with enterprise search systems. A quick review of the analyst reports from well-known pundits and the marketing collateral of competitors rarely talk about dissatisfied users. Vendors like Autonomy and Google may snipe at one another, none of the vendors hint at their systems or even their competitors irritating users, creating dissatisfaction, and forcing employees to find their own search solutions by ordering a Google Search Appliance and saying, “My department will index its own information, thank you.”
ArnoldIT.com Headquarters
April 10, 2008
ArnoldIT.com is delighted to announce that it has moved to new headquarters in Harrod’s Creek, Kentucky.
In response to two questions about the location of Harrod’s Creek, ArnoldIT.com has released a photograph of its spacious, state-of-the-art offices.
Harrod’s Creek is one of North America’s high-technology centers. Our staff filters enterprise search news to separate the goose feathers from the giblets. Contact ArnoldIT.com by write sa at arnoldit.com.
Stephen Arnold, April 10, 2008
Google App Engine: Googzilla’s Slow, Small Baby Steps
April 10, 2008
Google is reasonably transparent–if you know the angle from which to observe the company. Viewed head on, Google sells only ads, generates billions. On one hand, the company races forward with betas, new engineering initiatives, and acquisitions. On the other hand, Google goes ever so slowly in converting its technological advantages into cold, hard cash.
Wall Street looks at the billions in ad revenue. Notes the $400 million from enterprise sales. End of story. Most Google pundits follow the same track. Its Google Search Appliance is not given much respect by the 150 companies competing in the search and content processing market. The scores of products available are quirks or footnotes to the larger story of Google’s ad revenue. Read more
Gartner and the GOOG: Is Google Failing in the Enterprise?
April 10, 2008
The Ziff Davis / eWeek story stopped me in my tracks. Chris Boulton, a fine, fine journalist, wrote a story the ZD editors called “Gartner: Google Doesn`t Understand the Enterprise”. (Read this story before it disappears from the eWeek Web site.) The hook for the piece is a Gartner professionals’ assertion that:
Google Apps is like a “fog rolling into the harbor,” permeating businesses quite possibly at the expense of Microsoft and IBM.
Allegedly Gartner pundit Tom Austin asserted that
Clients are calling us about GAPE [Google Apps Premier Edition],” Austin said. ‘They will use it as a bat to beat Microsoft or IBM to make them lower the cost of their software.’
The remarks appears to orginate in a talk at the Gartner Symposium ITxpo on April 9, 2008. The most telling part of this article, if Mr. Boulton heard correctly is:
In a line of reasoning echoing Microsoft Chairman Bill Gates’ claims that Google doesn’t understand businesses’ needs, Austin said that Google doesn’t understand the enterprise. It is not that the company can’t, he said, it is that Google doesn’t care to understand the enterprise. For example, while Microsoft and IBM offer customers five-year roadmaps under non-disclosure agreements, Google’s roadmap is one day at a time.
If true, Gartner must know a great deal more about Google’s enterprise success than I do. My sources tell me that Google is struggling to stay on top of the wave of success with its map and geo-spatial services. Google is reacting to customer requests, at least in the US government sector from what I hear from those familiar with canvas cubes in Washington, DC. My research about enterprise search revenues indicates that Google now has more than 9,000 licensees of its Google Search Appliance. This product generated somewhere around $350 to $400 million in calendar 2007 and is growing at double digit rates. The various applications, enhanced email, and messaging functions are pulling inquiries as well. In short, the Google is disrupting the traditional enterprise market on several fronts. Google lets customers pull Google to them. Google doesn’t push for sales like most enterprise software vendors.
My hunch is that Google’s “fog-like” behavior translates to sour grapes because Google is somewhat reluctant to shovel cash into the maw of the high-end IT consultancies for guidance. Google’s reliance on “pull” tactics is challenge for some traditional consulting firms like Booz, Allen & Hamilton where I worked . Google has plenty of wizards and gurus on staff. If a pundit is Googley, that consultant will probably work for Google. This is a difficult concept for some for-hire experts to accept. But that’s just my interpretation of the matter.
I think Mr. Boulton got the story right. Could it be that Gartner doesn’t understand Google?
Stephen Arnold, April 10, 2008
Absolutes and Electronic Information
April 9, 2008
I find the research for my work fascinating. Periodically I root through some of the PDFs and PowerPoints used in my public talks.
Information in 2001
Today, while consolidating some information from a soon-to-be-retired NetFinity 5500, I came across a presentation I made to the legal information giant, Lexis Nexis, in year 2001.
The presentation sure didn’t win me any buddies in this $1 billion a year unit of the Euro-giant Reed Information. Reed, like the Thomson Corporation, maintains a low profile. Most people are unaware of what these two professional publishing companies do for a living, and I am not going to tell you that. You will have to figure it out for yourself.
My talk was given at some golf resort, and I don’t golf. I sat on my tail feather and waited to deliver my talk, which I titled “Information Professionals and In-Phase Services”. The main idea behind the talk was that anyone who used information for a living (lawyers, consultants, intelligence officers, and financial analysts) wanted current information in the context of their work.
The idea of stopping one thing to go ferret out a missing piece of information is growing long in the tooth. No, “long in the tooth” is too gentle even seven years after I wrote this presentation. Stupid, ill-advised, crazy, dumb — these are much more appropriate words. In year 2000, it was obvious — based on my research — that savvy users of information wanted information from one screen or dashboard. Furthermore that information should be [a] comprehensive, [b] current or fresh, and [c] in a form that allowed it to be cut-and-pasted or recycled without annoying manual reformatting.
I used this quote from Emily Dickinson to catch the crowd’s attention: “The truth must dazzle gradually / Or every man be blind…” No one knew what the heck I was talking about. To help the audience along, I used this chart from Forbes Magazine, October 2, 2000:
The point of this study is that humans–more than two thirds of them in 2000–want fixed points in their lives. The notions of change, flux, transformation made people uncomfortable. The chart did little to win my audience’s confidence in my talk because I then told the group, “Absolutes are rarely found when we talk about electronic information.”
Google Metaphors of the Day: DeathStar and Skynet
April 9, 2008
John Murrell, SiliconValley.com, used two interesting metaphors in his “Google Opens Cloud to Crowd” story about Google’s new hosted services. Most Google watchers tip toe around comparisons of Google (the happy company) to any thing dark and sinister. Mr. Murrell writes:
Now that it has built up its computing infrastructure to a size somewhere between the Death Star and Skynet, Google really hates to see any of that power going untapped. To spread the wealth, Google announced late Monday the beta launch of App Engine, a set of tools and services that will let Web developers run their own applications on Google’s platform, avoiding all that troublesome back end maintenance.
If you have been outside the flows of popular culture, the DeathStar is a planet-sized ship armed with sufficient fire power to destroy a planet. Skynet, on the other hand, is the self-aware computing system that is waging war with annoying humans in the Terminator films.
Stephen Arnold, April 9, 2008
Sinequa Reveals Users of Search Are Annoyed
April 9, 2008
— Enterprise Search’s Problems Exposed in a New Survey —
Sinequa (a business intelligence and search systems vendor based in Paris) has taken a bold and refreshingly candid step.
The company reported results of a survey of 200 office workers. The results run counter to the marketing blather about the efficacy of enterprise search, sometimes called behind-the-firewall search or Intranet search.
Competitors in the enterprise search space like Autonomy, Endeca, Fast Search & Transfer, and dozens of up-and-coming vendors rarely reveal data about installations gone wrong. Sinequa has exposed the massive failure of enterprise search by revealing the results of its study of 200 professionals using enterprise search at work.
These data suggest that nearly 60 percent of the workers participating in the survey said the tools their employers provide for search are “poor”. Almost 80 percent of the respondents reported that they would benefit from having access to timely information from across their organization.
The report in IWR (Information World Review, a unit of the VNU Network) said:
Despite 88 per cent of respondents’ employers investing in an intranet, none had looked at how to maximise the information on the intranet to help drive efficiencies in performance. In fact, just eight per cent of respondents have a tool allowing them to search information across the company using key search terms.
The lack of a search system that works has even broader implications:
In the past month alone 46 per cent of respondents said that they could count up to ten occasions when not having access to the right information had impaired their performance. 16 per cent could recall ten or more times when this had happened. A further 40 per cent said that finding the information to support the development of a business critical document takes two to three hours on average, with a quarter stating it could take three hours or more.
It’s easy to calculate the cost of a flawed search system. Take an average hourly wage–say, $50 / hour. This simple exercise converts search into more than an intellectual exercise. Search gone wrong costs a great deal of money.
A spokesperson for Sinequa is reported by VNU to have said that “businesses are seriously missing the trick”. Another VNU report of the Sinequa survey revealed:
Employees are struggling to find even basic information, which impacts their productivity on a day-to-day basis… The gap between what staff can do as consumers and what they can do as employees is causing employee frustration as well as limiting the value of corporate information.
These data back up the findings reported in Beyond Search: What to Do When Your Enterprise Search System Doesn’t Work. That study reported results of a 2007 research project which found that two-thirds of the users of an enterprise search system were dissatisfied with the performance of their search tools.
Sinequa and Beyond Search reports of user frustration and annoyance seem out of step–particularly when viewed in the rosy light of the overwhelmingly upbeat reports of search success reported at such high-profile, search-centric industry events as Fast Forward (Microsoft – Fast Search & Transfer), Enterprise Search Summit (Information Today, Inc.), and the International Online Show (Incisive Media).
The clash of industry reports of success and the rumblings of user dissatisfaction raise serious questions about the reality of enterprise search systems in actual use. Search pundits and consultants rarely talk about systems that go off track. If the Sinequa data are accurate and if the data in the new Gilbane study about search are on track–the high-flying enterprise search sector may be running short of fuel.
This Web log will feature an interview with the founder of Sinequa on April 21, 2008, as part of ArnoldIT.com’s “Search Wizards Speak” feature.
Stephen Arnold, April 9, 2008
Compete Data Tweak the Google
April 8, 2008
LiveSide.net provided a summary and helpful chart on April 4, 2008. The article’s sub title states, “New Compete.com search share numbers show surprising surge for Live Search.” The use of the word surge caught my attention. At age 64, surge to me has an old school meaning; namely, “a strong, wavelike, forward movement, rush, or sweep.” I read the article and looked at the Compete.com numbers and concluded that a post-modern meaning has been applied to surge.
The LiveSide.net article’s main point to me is:
Compete.com released their version of March US search market share numbers …, showing a surprising surge for Live Search, up nearly four percentage points from February (a notoriously slow, and short, month), and up 0.7% year over year. In addition, search volume (the number of searches performed) was up significantly as well. Also notable was that for the first time in nine months, Google lost market share.
Hmmm. I wondered about the surge. The table provides some clues here. I took these data and calculated the number of searches per minute by each service. Based on the Compete.com data, Google is averaging 2,346 queries per second. The surge notion is that Microsoft’s queries per second are 344 per minute, a difference of a few clicks, about 2,000.
The surge is more about percent change that clicks. Live.com, according to Compete.com, tallied 922 million in March 2007 searches up from 716 million in February 2007. Google, by contrast, generated about 6.3 billion searches in March 2008, up from 6.0 billion in February 2008. Not only is Microsoft’s base smaller than Google’s, the search differential is growing.
Perhaps I am misreading these data. Let me know if you think the gap is closing and the sky is falling in Google’s part of the world.
Note: Hitwise released numbers of its own recently; that consultancy says Google has a 64% marketshare in US searches for March, with MSN Search back at 5.25%.
The Google has a hefty lead, and it will take some time to bridge the gap.
Stephen Arnold, April 8, 2008
Time: Your Search Infrastructure’s Deadliest Enemy
April 8, 2008
Look at these two diagrams.
The first diagram–the one with a single red arrow–represents the volume of content your search and content processing system must process. Unless you work in a very unusual organization, you and your colleagues produce digital information. Let’s confine our discussion to text, but keep in mind that your organization will also produce digital images, audio, and video. At some point your search or content processing system will have to deal with these types of content as well.
The story in the chart with the single error is simple: each day the amount of content increases. The content is very distinct, and it falls into one of three broad categories. Some content is original; that is, there is no previous version or draft. You or one of your colleagues generates an original document and stores it on her computer. If your colleague is a tele worker, she might upload the document to your company’s server.
-
- Subscribe to Beyond Search
Feature archive
News archive
-
