October 7, 2014
Google, believe it or not, is responsible in part for the problems with enterprise search. The idea is advanced in “Why the “Google Paradigm” Has Damaged Enterprise Search.” The core of the argument is that people use Google for Web search. The resulting perception is that “enterprise search is as easy as Google web search, and that a central index of an enterprise is the right way to do enterprise search. ”
Google’s entrance into enterprise search was one of the companies earliest attempts to enter a market in which revenue came from a subscription or license, not a fee for advertising. The Google Search Appliance was a server loaded with a version of Google’s Web search system. Based on our work with the first GSA, it was clear that like many other Google products and services from the 2001 to 2004 period, Google was operating on some Googley assumptions; for example:
- Google assumed that a version of its Web search system stripped of its ad matching was good enough for finding textual content in an organization
- The company assumed that Autonomy, Endeca, and Fast Search & Transfer, the dominant enterprise search vendors at this time were too complex for most technical staff in an organization. The time and complexity of these systems contributed to the high user dissatisfaction with these systems. The high cost of these industry leaders’ systems contributed to management grousing about search.
- Google assumed that it could disintermediate traditional information technology departments and deal directly with end users.
Google crafted a server that was positioned as a “search toaster.” The low price of the basic unit was less than $2,000 and sported an interface that required the licensee to plug in basic information and click a button to start the indexing process.
The Google Search Appliance by 2007 had an estimated 50,000 licensees. At that time, the product line had expanded but the locked down nature of the Google Search Appliance and the key word approach of the system was creating sales opportunities for other search appliance vendors; namely, Thunderstone, Maxxcat, and Index Engines.
Google added features and fiddled with the license fees, hardening the GSA product line with hot backups, connectors, and extensibility via licensed vendors. Few analysts paid much attention to the product licensing fees for the various “GB” or Google boxes. If you want to get a sense of the costs for building out a GSA system that can process 100 million documents, navigate to www.GSAadvantage.gov and search for the Google’s search appliances. The costs work out to be comparable or slightly higher than a similar installation from Autonomy, Endeca, or Fast. The high prices remain today.
Google learned from the GSA experience. Instead of offering an enterprise cloud solution, the company has left a limited and pricey GSA product line in the market and provided a modest commitment to this enterprise search solution. Google’s cloud solution manifests itself in Google’s site search features. I am waiting for Google either to kill the product line or amp up its commitment. In my opinion, the GSA is in no man’s land at this time. It appears that not even Google can respond to the needs of the enterprise findability users. If any company could crack the code, would it not be Google or a Xoogler’s start up?
As the GSA emerged as placeholder product, professionals became more and more dependent on Google’s Web search. In Europe, for example, Google’s Web search commands an market share in excess of 80 percent. In Denmark, Google’s share of Web search is north of 90 percent. In the US, Google has a 65 to 75 percent share of Web search, depending on which consultancies’ numbers one uses.
The word “search” became synonymous with Google. Enterprise search vendors began to use jargon other than search. This step was a natural reaction to hearing from prospects, “We want a search that works just like Google.” What the prospects meant was a system that was easy to use and seemed to deliver useful results in the hits displayed at the top of a results list, a page of images, or a map showing a location.
Google Web search, not the Google Search Appliance, reflected a broader shift in the information access market. Users of Web search and enterprise systems wanted and still want:
- Systems that do not require the user to invest much time and effort in getting an answer
- Systems that can produce useful outputs whether text, images, or maps with data displayed on them
- Systems that delivered “answers” without the delays (latency) many enterprise systems force on users.
Google’s ability to respond to this enterprise demand has been ineffectual. Like other Web search vendors, key word retrieval does not solve the problems basic search systems spawn. The GSA is evidence that Google does not have the key to unlock the revenue vault for enterprise search.
What Google search has done (inadvertently I might add) has been to make crystal clear that users do not want to work hard for information users perceive as useful. Precision and recall are irrelevant because voting and advertisers influence Google Web search results. Users love Google’s outputs.
In the organization, procurement teams, individual users, and senior management boil their needs down to one simple statement: “We want search that is just like Google.”
That’s a big, big problem for search and content processing vendors. Google Web search is not about relevance, objective information, or accuracy. Google is easy and “good enough.” In an organization, people want easy. But in an organization the results have to be timely, comprehensive in terms of what information is available to an organization, and accurate.
On the Web Google can skip content that is malformed or stored on a server that does not respond to a Google spider quickly enough. In an organization, the content has to be available. On the Web, the advertisers and the uses’ own behavioral data pays the bills. In an organization, the organization has to pay the bills. Google has more money from a different business model than most organizations. Google pumps money into plumbing to deliver the service that makes money. Organizations want to fix the amount spent on search and the funds are not infinite.
For search vendors, the problem of Google’s dominance in Web search makes product differentiation difficult. Google’s business model creates challenges for vendors who have to justify the “value” and hence the “cost” of their search systems. For traditional search vendors, ease of use is very, very difficult because of the nature of the questions enterprise system users have.
Google is a mirror in which societal, cultural, and intellectual changes in information access are reflected. For many years, I have called attention to the verbal push ups search vendors use to try and make sales. The struggled Hewlett Packard have had with Autonomy provide a glimpse of how “value” can be difficult to change into hard cash Microsoft’s Delve illustrates that search for Office 365 is a combination of contacts, alerts, and personalization, not key word search. The dependence on enterprise search companies for cash from venture capital sources illustrates that traditional search is a very, very tough business to make into something sustainable and profitable without financial life support. The expectations that Watson will become a $10 billion business in 60 months is disconnected from the experience of other smart companies. In the history of enterprise search, only Autonomy reported revenues of more than $800 million from enterprise licenses. IBM projects more than 10 X this revenue in 60 months. It took Autonomy more than a decade to hit $500 million.
The reality is that Google is not the problem. Google is a metaphor for what users want when it comes to information access.
The write up asserts:
The Google paradigm also ignores the challenge of scalability. Indexing the enterprise for a centralized enterprise search capability requires major investment. In addition, centralization runs counter to the realities of the working world where information must be distributed globally across a variety of devices and applications. The amount of information we create is overwhelming and the velocity with which that information moves increases daily.
Interesting statement. For me, the problem of the Google paradigm is that it another bit of jargon that sidesteps a what information retrieval must deliver in today’s business environment. Whoever cracks the code can make money. My hunch is that Google probed the enterprise search market and is trying to figure out how to make it pay off in a significant way. Google may be trapped in the same problem space through which other enterprise search and content processing vendors slog. The question may be, “Is there a way out of the swamp and into a land of milk, honey, sustainable revenue, and healthy margins?”
Stephen E Arnold, October 7, 2014
October 7, 2014
The article titled Chemist Direct Conversion Rates Soar Following Deployment of SLI Systems’ Search Technology on Retail Times explores the success of Chemist Direct’s online business. Chemist Direct is the leading UK pure play pharmacy company, and in January 2014 they deployed SLI Systems advanced site search solutions to improve their customer’s search experience. The article discusses the success of this deployment,
“Chemist Direct said it saw an immediate sales impact with conversion rates improving dramatically, by 175%, and search customers’ average order values increasing by 196%. Chemist Direct deployed SLI Systems’ search box offerings… based on patented technology that learns from visitors’ site search activity and click throughs to deliver the most relevant results. The technology also brings clients’ non-product information, such as user ratings, reviews, social content, blogs and videos, into search to deliver an impressive user experience…”
Emphasizing the importance of speed and relevance in search, Chemist Direct employee Stephen Lovell spoke to the sheer number of products available and the amount of monthly transactions. With the newly deployed SLI Systems’ (specifically Learning Search and Rich Auto Complete) customers visiting the Chemist Direct website find what they are looking for more quickly and stick around the complete their purchase. It is difficult to dispute that search is an increasingly integral aspect of any online business.
Chelsea Kerwin, October 07, 2014
October 6, 2014
You can download the most recent version of ElasticSearch via the link in the ElasticSearch blog. Navigate to http://bit.ly/1uxnOfN and click the download button. Changes include a fix for shard recovery and corruption occurring when a licensee upgrades an old index.
Stephen E Arnold, October 1, 2014
October 5, 2014
Who owns Business Insider? Oh, maybe Amazon, a company with arguably the most ineffective search system available to me. Just try to winnow a list of books by those one can buy right now versus those one can buy at some time in the future? Oh, right. I just read “Here’s the Evidence That Google’s Search Results Are Horribly Biased.”
Now Google is an ad outfit. Pay money and one gets traffic. Seems simple. Google also flogs its own products like the really, really popular Google Search Appliance. Google is battling Amazon in the world of WalMart like cloud cost cutting. Sigh. Is it a surprise that Google has a small amount of self interest in what it does, delivers, and presents? Who argues for precision and recall at Google?
Here’s what I learned:
An internal document recently leaked from Yelp showed how the company believed Google was siphoning off up to 20% of its clicks, directing them to Google’s results. Yelp even persuaded the European Union to reopen an antitrust investigation into the way it said Google abused its 90% share of the European search market to manipulate results. (Business Insider heard Yelp’s intervention came after CEO Jeremy Stoppelman found himself at a dinner in San Francisco this spring with EC president José Manuel Barroso, where he was able to bend Barroso’s ear about Google’s market share.) So Yelp has launched a coalition, including Trip advisor and Consumer Watchdog, under the banner “Focus On The User” to draw attention to the way Google results are displayed.
Yep, revelation. Is it possible for a merchant to influence a Yelp review? Hmmm. Is any “free” search system bias free? Any countries filtering search results or system access to control information flow? Hmmmm.
Stephen E Arnold, October 5, 2014
October 5, 2014
With so many search and content processing vendors struggling to meet licensees’ expectations, management insights are needed. When failed webmasters, frustrated academics, or art history majors morph into search experts, poobahs, and wizards—checklists are essential.
I want to point you to this list of items to embrace, particularly if you are young at heart, absent a successful management track record, or someone looking for a shortcut to the affection of your employees.
Navigate to “44 Engineering Management Lessons.” A lesson in this context is more like a statement or exhortation delivered with vocal tones both dulcet and soft.
Here are three “lessons”. I will leave the other 41 to your own study:
No. 5 “Provide administrative support. Schedule issues, coordinate releases, and make sure the bureaucratic machine keeps ticking.
No 12 Don’t make decisions unless you have to. Whenever possible, allow the team explore ideas and make decisions on its own
No. 38 Occasionally someone will push too far. When they do, you have to show a rough edge or you’ll lose authority with your team.
There you go.
Next time stakeholders demand a return on their investment in your search or content processing company, use these tips. If you are “teaching” people about search, integrate these ideas into your lecture.
You cannot lose, at least in the experience of the author of the management lessons, er, big list of statements.
Stephen E Arnold, October 5, 2014
October 3, 2014
I spotted some “fresh” jargon from search vendor Coveo. Here are the terms:
- Relevance solutions. The implicit idea is that other vendors’ search systems do not deliver results that are on point to a user. I am not sure how many vendors pay much attention to relevance. In fact, the shift to graphics and reports purport to “answer questions.” These systems may not, but it is an approach that powers some big folks’ innovations. Microsoft Delve, anyone?
- Unified search. The idea is that information can reside in different forms and locations. Presenting search results from these different sources eliminates the need to run queries on different systems. The problem is that “unified,” in my experience does not include certain types of content; for example, snippets from videos or data locked in proprietary systems like the IBM i2 Dot ANB format. Unified or federated search is a term popular with well known companies like Attivio and lesser known companies like Polyspot. The company in my mind most closely associated with this concept is Deep Web Technology. The idea is a good one, but expectations can rise above the actual “federated” experience in my opinion.
I find the creativity evidenced in these examples of jargonizing evidence of three trends:
First, “search” as a buzzword only has impact if qualified in some way; for example, federated, unified, intelligent, etc.
Second, individual vendors are working to try and differentiate themselves from what to many people seem to be identical in form and function. The differentiator boils down to price, the power of the brand, or perceived value of the system endorsed by mid tier consultants like IDC-type or Forrester-type outfits.
Third, the impact of open source alternatives lurks behind these verbal gymnastics. ElasticSearch, whether proprietary vendors are comfortable with the notion or not, offer a way to get search without the lock down that proprietary vendors bring to the party.
Stephen E Arnold, October 2, 2014
October 3, 2014
We see from a write-up at Linux magazine that dtSearch is embracing Android. The press release announces, “New Android Beta for the dtSearch Engine for C++ and Java Developers; Beta Adds Android to Engine’s Current Linux and Windows C++, Java, and ./NET SDKs.” We learn:
“The dtSearch Engine for Android beta will join the existing dtSearch Engine for Linux (native 64-bit/32-bit C++ and Java APIs) and dtSearch Engine for Win & .NET (native 64-bit/32-bit C++, Java and .NET APIs) in making available dtSearch’s instant searching and document filters for a wide range of Internet, Intranet and other commercial applications. (Please see http://www.dtsearch.com/casestudies.html for hundreds of developer case studies.)”
The write-up goes on to give a rundown of dtSearch’s features, like its advanced document filters, terabyte-ready index, limitless multithreaded searching, federated and spider searches, faceted search, and international language support. See the write-up for more on each of these.
Launched in 1991, DtSearch has become a major provider of data-management software to firms in several fields and to numerous government agencies in defense, law enforcement, and space exploration. The company also makes its products available for incorporation into other commercial applications. dtSearch has distributors worldwide, and is headquartered in Bethesda, Maryland.
Cynthia Murrell, October 03, 2014
October 2, 2014
Yep, IOT or the Internet of Things can be searched. “Thingful Upgrades its Search Engine for the Internet of Things” said:
[Thingful] allows people to find connected devices, from shark monitors to Raspberry Pis, based on the object and where it is located….The idea is to help build a true community of connected devices and use the resources and data already around us, as opposed to running out and building a system from scratch.
I ran a query for traffic cams Washington, DC. The system showed one hit. I think there are more in that area. One feature of the service is that I could click on the purple locator and view the cam after I logged into Thingful via Twitter or Facebook. I passed, but you may be an avid IOT person and be more adventuresome than I.
The service does not seem to poke into the industrial side of interconnected gizmos. General Electric, which is more than IOT lighting fixtures, and others are moving quickly in this space.
You can find Thingful at www.thingful.net.
Stephen E Arnold, October 2, 2014
October 1, 2014
I read “The Unexpected Consequences of Success.” For some reason, my Overflight system is gathering ever increasing numbers of management silliness. If management worked, would not PIMCO avail itself of the methods and demonstrate that it could retain staff, generate returns of little interest to the SEC, and stop institutional investors from withdrawing a few dollars here and there?
The write up states:
People often prepare for failure, but rarely prepare for what they will do when they succeed. Even when we consciously want to be successful, enjoying that success can be a challenge.
In the 1980s, one of the challenges Booz, Allen & Hamilton faced was that some senior people dumped their wives and formed a relationship with their assistants. The rationalization I heard was that the assistants “knew” what the senior Booz person was going through and could, therefore, be a better partner. I am not sure how the firm worked this out, nor do I care.
Other management challenges exist. I spent several hours with a CEO of a Silicon Valley company who was coming unglued before my eyes. In addition to the long, unkempt hair, the person had embraced various New Age concepts. The person ignored email and allowed the usually chaotic programmers to behave more like a crowd in Henderson, Missouri.
With a matter of weeks, the company had become the subject of a vicious analysis in a zippy San Francisco blog. As far as I know, the individual is now either surfing or working at an organic farm.
When I sent a copy of an article titled “Research shows there could be increased numbers of psychopaths in high levels of business,” the recipient (an investment banker) replied, “Yikes, Wall Street stress?” Nope, psychopathic behavior.
According the the HBR, at least the success part of the job can be handled with several easy-to-implement tips:
- Don’t do victory laps. I think this means buy a LaFerrari and drive it to lunch when you fire people.
- Focus on the value you bring, not on winning per se. Nope, the winning is evident when the Charlie Sheen type winner get a bonus.
- Stay in the here and now. I think this means do not become psychotic, dependent on banned substances, or a modern version of Julius Caesar.
- Reach higher. I think this means, “Be more like Alexander.” That worked out well for him at least for three decades.
I am not sure how these pronouncements will help out the dozens upon dozens of search and content processing companies thrashing around for money. But it is good to know that there are four tips to ensure that old MBA trajectory of success.
Stephen E Arnold, October 1, 2014
October 1, 2014
Imagine being able to shop and purchase health insurance and find healthcare like you were booking a trip? That was the basic idea for Whatclinic.com, described as a TripAdvisor website was designed to meet the demands of South Africa’s healthcare’s needs. IT News Africa has more information in the article, “Global Healthcare Search Engine Launches In SA.”
According to the press release, there are already 500 clinics on the Web site and the numbers is expected to double in the next three months. The information on the clinics includes hours of operation, location, staff, treatments, price, and staff. Like TripAdvisor, people can leave comments and ratings.
“ ‘Our value proposition is based on empowering patients with easily accessible and centralized information to help them make informed decisions about their treatment… specifically, where to source the right treatment for them,’ says [Caelen King, founder of WhatClinic].”
It is no surprise that online healthcare research is growing. WhatClinic.com reports that 15.7 million people use the Web site and projections only show that will increase. There are similar services in America, but WhatClinic should investigate coming to the US or a startup could nab the idea.