Enterprise Search: The Last Half of 2015
June 16, 2015
I saw a link this morning to an 11 month old report from an azure chip consulting firm. You know, azure chip. Not a Bain, BCG, Booz Allen, or McKinsey which are blue chip firms. A mid tier outfit. Business at the Boozer is booming is the word from O’Hare Airport, but who knows if airport gossip is valid.
Which enterprise search vendor will come up a winner in December 2015?
What is possibly semi valid are analyses of enterprise search vendors. The “Magic Quadrant for Enterprise Search” triggered some fond memories of the good old days in 2003 when the leaders in enterprise search were brands or almost brands. You probably recall the thrilling days of these information retrieval leaders:
- Autonomy, the math oriented outfit with components names like neuro linguistic programming and integrated data operating layer and some really big name customers like BAE
- Convera, formerly Excalibur with juice from ConQuest (developer by a former Booz, Allen person no less)
- Endeca, the all time champ for computationally intensive indexing
- Fast Search & Transfer, the outfit that dumped Web search in order to take over the enterprise search sector
- Verity, ah, truth be told, this puppy’s architecture ensured plenty of time to dash off and grab a can of Mountain Dew.
In 2014, if the azure chip firm’s analysis is on the money, the landscape was very different. If I understand the non analytic version of Boston Consulting Group’s matrix from 1970, the big players are:
- Attivio, another business intelligence solution using open source technology and polymorphic positioning for the folks who have pumped more than $35 million into the company. One executive told me via LinkedIn, that the SEC investigation of an Attivio board member had zero impact on the company. I like the attitude. Bold.
- BA Insight, a business software vendor focused on making SharePoint somewhat useful and some investors with deepening worry lines
- Coveo, a start up which is nudging close to a decade in age, and more than $30 million in venture backing. I wonder if those stakeholders are getting nervous.
- Dassault Systèmes, the owner of Exalead, who said in the most recent quarterly report that the company was happy, happy, happy with Exalead but provided no numbers and no detail about the once promising technology
- Expert System, an interesting company with a name that makes online research pretty darned challenging
- Google, ah, yes, the proud marketer of the ever thrilling Google Search Appliance, a product with customer support to make American Airlines jealous
- Hewlett Packard Autonomy, now a leader in the acrimonious litigation field
- IBM, ah, yes, the cognitive computing bunch from Armonk. IBM search is definitely a product that is on everyone’s lips because the major output of the Watson group is a book of recipes
- IHS, an outfit which is banking on its patent analysis technology to generate big bucks in the Goldmine cellophane
- LucidWorks (Really?), a repackager of open source search and a distant second to Elastic (formerly Elasticsearch, which did not make the list. Darned amazing to me.)
- MarkLogic, a data management system trying to grow with a proprietary XML technology that is presented as search, business intelligence, and a tool for running a restaurant menu generation system. Will MarkLogic buy Smartlogic? Do two logics make a rational decision?
- Mindbreeze, a side project at Fabasoft which is the darling of the Austrian government and frustrated European SharePoint managers
- Perceptive Software, which is Lexmark’s packaging of ISYS Search Software. ISYS incorporates technology from – what did the founder tell me in 2009? – oh, right, code from the 1980s. Might it not be tough to make big bucks on this code base? I have 70 or 80 million ideas about the business challenge such a deal poses
- PolySpot, like Sinequa, a French company which does infrastructure, information access, and, of course, customer support
- Recommind, a legal search system which has delivered a down market variation of the Autonomy-type approach to indexing. The company is spreading its wings and tackling enterprise search.
- Sinequa, another one of those quirky French companies which are more flexible than a leotard for an out of work acrobat
But this line up from the azure chip consulting omits some companies which may be important to those looking for search solutions but not so much for azure chip consultants angling for retainer engagements. Let me highlight some vendors the azure chip crowd elected to ignore:
Enterprise Search Is Important: But Vendor Survey Fails to Make Its Case
March 20, 2015
I read “Concept Searching Survey Shows Enterprise Search Rises in the Ranks of Strategic Applications.” Over the years, I have watched enterprise search vendors impale themselves on their swords. In a few instances, licensees of search technology loosed legal eagles to beat the vendors to the ground. Let me highlight a few of the milestones in enterprise search before commenting on this “survey says, it must be true” news release.
A Simple Question?
What do these companies have in common?
- Autonomy
- Convera
- Fast Search & Transfer?
I know from my decades of work in the information retrieval sector that financial doubts plagued these firms. Autonomy, as you know, is the focal point of on-going litigation over accounting methods, revenue, and its purchase price. Like many high-tech companies, Autonomy achieved significant revenues and caused some financial firms to wonder how Autonomy achieved its hundreds of millions in revenue. There was a report from Cazenove Capital I saw years ago, and it contained analyses that suggested search was not the money machine for the company.
And Convera? After morphing from Excalibur with its acquisition of the manual-indexing ConQuest Technologies, a document scanning with some brute force searching technology morphed into Convera. Convera suggested that it could perform indexing magic on text and video. Intel dived in and so did the NBA. These two deals did not work out and the company fell on hard times. With an investment from Allen & Company, Conquest tried its hand at Web indexing. Finally, stakeholders lost faith and Convera sold off its government sales and folded its tent. (Some of the principals cooked up another search company. This time the former Convera wizards got into the consulting engineering business.) Convera lives on in a sense as part of the Ntent system. Convera lost some money along the way. Lots of money as I recall.
And Fast Search? Microsoft paid $1.2 billion for Fast Search. Now the 1998 technology lives on within Microsoft SharePoint. But Fast Search has the unique distinction of facing both a financial investigation for fancy dancing with its profit and loss statement and the distinction of having its founder facing a jail term. Fast Search ran into trouble when its marketers promised magic from the ESP system. When the pixie dust caused licensees to develop an allergic reaction, Fast ran into trouble. The scrambling caused some managers to flee the floundering Norwegian search ship and found another search company. For those who struggle with Fast Search in its present guise, you understand the issues created by Fast Search’s “sell it today and program it tomorrow” approach.
Is There a Lesson in These Vendors’ Trajectories?
What do these three examples tell us? High flying enterprise search vendors seem to have run into some difficulties. Not surprisingly, the customers of these companies are often wary of enterprise search. Perhaps that is the reason so many enterprise search vendors do not use the words “enterprise search”, preferring euphemisms like customer support, business intelligence, and knowledge management?
The Rush to Sell Out before Drowning in Red Ink
Now a sidelight. Before open source search effectively became the go to keyword search system, there were vendors who had products that for the most part worked when installed to do basic information retrieval. These companies’ executives worked overtime to find buyers. The founders cashed out and left the new owners to figure out how to make sales, pay for research, and generate sufficient revenue to get the purchase price back. Which companies are these? Here’s a short list and incomplete list to help jog your memory:
- Artificial Linguistics (sold to Oracle)
- BRS Search (sold to OpenText)
- EasyAsk (first to Progress Software and then to an individual investor)
- Endeca to Oracle
- Enginium (sold to Kroll and now out of business)
- Exalead to Dassault
- Fulcrum Technology to IBM (quite a story. See the Fulcrum profile at www.xenky.com/vendor-profiles)
- InQuira to Oracle
- Information Dimensions (sold to OpenText)
- Innerprise (Microsoft centric, sold to GoDaddy)
- iPhrase to IBM (iPhrase was a variant of Teratext’s approach)
- ISYS Search Software to Lexmark (yes, a printer company)
- RightNow to Oracle (RightNow acquired Dutch technology for its search function)
- Schemalogic to Smartlogic
- Stratify/Purple Yogi (sold to Iron Mountain and then to Autonomy)
- Teratext to SAIC, now Leidos
- TripleHop to Oracle
- Verity to Autonomy and then HP bought Autonomy
- Vivisimo to IBM (how clustering and metasearch magically became a Big Data system from the company that “invented” Watson) .
The brand impact of these acquired search vendors is dwindling. The only “name” on the list which seems to have some market traction is Endeca.
Some outfits just did not make it or who are in a very quiet, almost dormant, mode. Consider these search vendors:
- Delphes (academic thinkers with linguistic leanings)
- Edgee
- Dieselpoint (structured data search)
- DR LINK (Syracuse University and an investment bank)
- Executive Search (not a headhunting outfit, an enterprise search outfit)
- Grokker
- Intrafind
- Kartoo
- Lextek International
- Maxxcat
- Mondosoft
- Pertimm (reincarnated with Axel Springer (Macmillan) money as Qwant, which according to Eric Schmidt, is a threat to Google. Yeah, right.)
- Siderean Software (semantic search)
- Speed of Mind
- Suggest (Weitkämper Technology)?
- Thunderstone
These are not a comprehensive list. I just wanted to layout some facts about vendors who tilted at the enterprise search windmill. I think that a reasonable person might conclude that enterprise search has been a tough sell. Of the companies that developed a brand, none was able to achieve sustainable revenues. The information highway is littered with the remains of vendors who pitched enterprise search as the killer app for anything to do with information.
Now the survey purports to reveal insights to which I have been insensitive in my decades of work in digital information access.
Here’s what the company sponsoring the survey offers:
Concept Searching [the survey promulgator], the global leader in semantic metadata generation, auto-classification, and taxonomy management software, and developer of the Smart Content Framework™, is compiling the statistics from its 2015 SharePoint and Office 365 Metadata survey, currently unpublished. One of the findings, gathered from over 360 responses, indicates a renewed focus on improving enterprise search.
The focus seems to be on SharePoint. I thought SharePoint was a mishmash of content management, collaboration, and contacts along with documents created by the fortunate SharePoint users. Question: Is enterprise search conflated with SharePoint?
I would not make this connection.
If I understand this, the survey makes clear that some of the companies in the “sample” (method of selection not revealed) want better search. I want better information access, not search per se.
Each day I have dozens of software applications which require information access activity. I also have a number of “enterprise” search systems available to me. Nevertheless, the finding suggests to me that enterprise search is and has not been particularly good. If I put on my SharePoint sunglasses, I see a glint of the notion that SharePoint search is not very good. The dying sparks of Fast Search technology smoldering in fire at Camp DontWorkGud.
Images, videos, and audio content present me with a challenge. Enterprise search and metatagging systems struggle to deal with these content types. I also get odd ball file formats; for example, Framemaker, Quark, and AS/400 DB2 UDB files.
The survey points out that the problem with enterprise search is that indexing is not very good. That may be an understatement. But the remedy is not just indexing, is it?
After reading the news release, I formed the opinion that the fix is to use the type of system available from the survey sponsor Concept Searching. Is that a coincidence?
Frankly, I think the problems with search are more severe than bad indexing, whether performed by humans or traditional “smart” software.
According the news release, my view is not congruent with the survey or the implications of the survey data:
A new focus on enterprise search can be viewed as a step forward in the management and use of unstructured content. Organizations are realizing that the issue isn’t going to go away and is now impacting applications such as records management, security, and litigation support. This translates into real business currency and increases the risk of non-compliance and security breaches. You can’t find, protect, or use what you don’t know exists. For those organizations that are using, or intend to deploy, a hybrid environment, the challenges of leveraging metadata across the entire enterprise can be daunting, without the appropriate technology to automate tagging.
Real business currency. Is that money?
Are system administrators still indexing human resource personnel records, in process legal documents related to litigation, data from research tests and trials in an enterprise search system? I thought a more fine-grained approach to indexing was appropriate. If an organization has a certain type of government work, knowledge of that work can only be made available to those with a need to know. Is indiscriminate and uncontrolled indexing in line with a “need to know” approach?
Information access has a bright future. Open source technology such as Lucene/Solar/Searchdaimon/SphinxSearch, et al is a reasonable approach to keyword functionality.
Value-added content processing is also important but not as an add on. I think that the type of functionality available from BAE, Haystax, Leidos, and Raytheon is more along the lines of the type of indexing, metatagging, and coding I need. The metatagging is integrated into a more modern system and architecture.
For instance, I want to map geo-coordinates in the manner of Geofeedia to each item of data. I also want context. I need an entity (Barrerra) mapped to an image integrated with social media. And, for me, predictive analytics are essential. If I have the name of an individual, I want that name and its variants. I want the content to be multi-language.
I want what next generation information access systems deliver. I don’t want indexing and basic metatagging. There is a reason for Google’s investing in Recorded Future, isn’t there?
The future of buggy whip enterprise search is probably less of a “strategic application” and more of a utility. Microsoft may make money from SharePoint. But for certain types of work, SharePoint is a bit like Windows 3.11. I want a system that solves problems, not one that spawns new challenges on a daily basis.
Enterprise search vendors have been delivering so-so, flawed, and problematic functionality for 40 years. After decades of vendor effort to make information findable in an organization, has significant progress been made. DARPA doesn’t think search is very good. The agency is seeking better methods of information access.
What I see when I review the landscape of enterprise search is that today’s “leaders” (Attivio, BA Insight, Coveo, dtSearch, Exorbyte, among others) remind me of the buggy whip makers driving a Model T to lecture farmers that their future depends on the horse as the motive power for their tractor.
Enterprise search is a digital horse, an one that is approaching break down.
Enterprise search is a utility within more feature rich, mission critical systems. For a list of 20 companies delivering NGIA with integrated content processing, check out www.xenky.com/cyberosint.
Stephen E Arnold, March 20, 2015
Coveoed Up with End of Week Marketing
December 22, 2014
I am the target of inbound marketing bombardments. I used to look forward to Autonomy’s conceptual inducements. In fact, in my opinion, the all-time champ in enterprise search marketing is Autonomy. HP now owns the company, and the marketing has fizzled in my opinion. I am in some far off place, and I sifted through emails, various alerts, and information dumped in my Overflight system.
I must howl, “Uncle.” I have been covered up or Coveo-ed up.
Coveo is the Canadian enterprise search company that began life as a hard drive search program and then morphed into a Microsoft-centric solution. With some timely venture funding, the company has amped up its marketing. The investor have flown to Australia to lecture about search. Australia as you may know is the breeding ground for the TeraText system which is a darned important enterprise application. Out of the Australia research petri dish emerged Funnelback. There was YourAmigo, and some innovations that keep the lights on in the Google offices in the land down under.
Coveo sent me email asking if my Google search appliance was delivering. Well, the GSA does exactly what it was designed to do in the early 2000s. I am not sure I want it to do anything anymore. Here’s part of the Coveo message to me:
Hi,
Is your Search Appliance failing you? Is it giving you irrelevant search results, or unable to search all of your systems? It’s time you considered upgrading to the only enterprise search platform that:
- Securely indexes all of your on-premise and cloud-based source systems
- Provides easy-to-tune relevance and actionable analytics
- Delivers unified search to any application and device your teams use
If I read this correctly, I don’t need a GSA, an Index Engines, a Maxxcat, or an EPI Thunderstone. I can just pop Coveo into my shop and search my heart out.
How do I know?
Easy. The mid tier consulting firm Gartner has identified Coveo as “the most visionary leader” in enterprise search. I am not sure about the methods of non-blue chip consulting firms. I assume they are objective and on a par with the work of McKinsey, Bain, Booz, Allen, and Boston Consulting Group. I have heard that some mid tier firms take a slightly different approach to their analyses. I know first hand that one mid tier firm recycled my research and sold my work on Amazon without my permission. I don’t recall that happening when I worked at Booz, Allen, though. We paid third parties, entered into signed agreements, and were upfront about who knew what. Times change, of course.
Another message this weekend told me that Coveo had identified five major trends that—wait for it—“increase employee and customer proficiency in 2015.” I don’t mean to be more stupid than the others residing in my hollow in rural Kentucky, but what the heck is “customer proficiency”? What body of evidence supports these fascinating “trends.”
The trends are remarkable for me. I just completed CyberOSINT: Next Generation Information Access. The monograph will be available in early 2015 to active law enforcement, security, and intelligence professionals. If you qualify and want to get a copy, send an email to benkent2020 at yahoo dot com. I was curious to see if the outlook my research team assembled from our 12 months of research into the future of information access matched to Coveo’s trends.
The short answer is, “Not even close.”
Coveo focuses on “the ecosystem of record.” CyberOSINT focuses on automated collection and analytics. An “ecosystem of record” sounds like records management. In 2015 organizations need intelligence automatically discovered in third party, proprietary, and open source content, both historical and real time.
Coveo identifies “upskilling the end users.” In our work, the focus is on delivering to either a human or another system outputs that permit informed action. In many organizations, end users are being replaced by increasingly intelligent systems. That trend seems significant in the software delivered by the NGIA vendors whose technology we analyzed. (NGIA is shorthand for next generation information access.)
Coveo is concerned about a “competent customer.” That’s okay, but isn’t that about cost reduction. The idea is to get rid of expensive call center humans and replace them with NGIA systems. Our research suggests that automated systems are the future, or did I just point that out in the “upskilling” comment.
Coveo is mobile first. No disagreement there. The only hitch in the git along is that when one embraces mobile, there are some significant interface issues and predictive operations become more important. Therefore, in the NGIA arena, predictive outputs are where the trend runway lights are leading.
Coveo is confident that cloud indexes and their security will be solved. That is reassuring. However, the cloud as well as on premises’ solutions, including hybrid solutions, have to adopt predictive technology that automatically deals with certain threats, malware, violations, and internal staff propensities. The trend, therefore, is for OSINT centric systems that hook into operational and intel related functions as well as performing external scans from perimeter security devices.
What I find fascinating is that in the absence of effective marketing from vendors of traditional keyword search, providers of old school information access are embracing some concepts and themes that are orthogonal to a very significant trend in information access.
Coveo is obviously trying hard, experimenting with mid tier consulting firm endorsements, hitting the rubber chicken circuit, and cranking out truly stunning metaphors like the “customer proficiency” assertion.
The challenge for traditional keyword search firms is that NGIA systems have relegated traditional information access approaches to utility and commodity status. If one wants search, Elasticsearch works pretty well. NGIA systems deliver a different class of information access. NGIA vendors’ solutions are not perfect, but they are a welcome advance over the now four decades old approach to finding important items of information without the Model T approach of scanning a results list, opening and browsing possibly relevant documents, and then hunting for the item of information needed to answer an important question.
The trend, therefore, is NGIA. An it is an important shift to solutions whose cost can be measured. I wish Mike Lynch was driving the Autonomy marketing team again. I miss the “Black Hole of Information”, the “Portal in a Box,” and the Digital Reasoning Engine approach. Regardless of what one thinks about Autonomy, the company was a prescient marketer. If the Lynch infused Autonomy were around today, the moniker “NGIA” would be one that might capture of Autonomy’s marketing love.
Stephen E Arnold, December 23, 2014
xx
Google Blamed for Problems in Enterprise Search
October 7, 2014
Google, believe it or not, is responsible in part for the problems with enterprise search. The idea is advanced in “Why the “Google Paradigm” Has Damaged Enterprise Search.” The core of the argument is that people use Google for Web search. The resulting perception is that “enterprise search is as easy as Google web search, and that a central index of an enterprise is the right way to do enterprise search. ”
Google’s entrance into enterprise search was one of the companies earliest attempts to enter a market in which revenue came from a subscription or license, not a fee for advertising. The Google Search Appliance was a server loaded with a version of Google’s Web search system. Based on our work with the first GSA, it was clear that like many other Google products and services from the 2001 to 2004 period, Google was operating on some Googley assumptions; for example:
- Google assumed that a version of its Web search system stripped of its ad matching was good enough for finding textual content in an organization
- The company assumed that Autonomy, Endeca, and Fast Search & Transfer, the dominant enterprise search vendors at this time were too complex for most technical staff in an organization. The time and complexity of these systems contributed to the high user dissatisfaction with these systems. The high cost of these industry leaders’ systems contributed to management grousing about search.
- Google assumed that it could disintermediate traditional information technology departments and deal directly with end users.
Google crafted a server that was positioned as a “search toaster.” The low price of the basic unit was less than $2,000 and sported an interface that required the licensee to plug in basic information and click a button to start the indexing process.
The Google Search Appliance by 2007 had an estimated 50,000 licensees. At that time, the product line had expanded but the locked down nature of the Google Search Appliance and the key word approach of the system was creating sales opportunities for other search appliance vendors; namely, Thunderstone, Maxxcat, and Index Engines.
Google added features and fiddled with the license fees, hardening the GSA product line with hot backups, connectors, and extensibility via licensed vendors. Few analysts paid much attention to the product licensing fees for the various “GB” or Google boxes. If you want to get a sense of the costs for building out a GSA system that can process 100 million documents, navigate to www.GSAadvantage.gov and search for the Google’s search appliances. The costs work out to be comparable or slightly higher than a similar installation from Autonomy, Endeca, or Fast. The high prices remain today.
Google learned from the GSA experience. Instead of offering an enterprise cloud solution, the company has left a limited and pricey GSA product line in the market and provided a modest commitment to this enterprise search solution. Google’s cloud solution manifests itself in Google’s site search features. I am waiting for Google either to kill the product line or amp up its commitment. In my opinion, the GSA is in no man’s land at this time. It appears that not even Google can respond to the needs of the enterprise findability users. If any company could crack the code, would it not be Google or a Xoogler’s start up?
As the GSA emerged as placeholder product, professionals became more and more dependent on Google’s Web search. In Europe, for example, Google’s Web search commands an market share in excess of 80 percent. In Denmark, Google’s share of Web search is north of 90 percent. In the US, Google has a 65 to 75 percent share of Web search, depending on which consultancies’ numbers one uses.
The word “search” became synonymous with Google. Enterprise search vendors began to use jargon other than search. This step was a natural reaction to hearing from prospects, “We want a search that works just like Google.” What the prospects meant was a system that was easy to use and seemed to deliver useful results in the hits displayed at the top of a results list, a page of images, or a map showing a location.
Google Web search, not the Google Search Appliance, reflected a broader shift in the information access market. Users of Web search and enterprise systems wanted and still want:
- Systems that do not require the user to invest much time and effort in getting an answer
- Systems that can produce useful outputs whether text, images, or maps with data displayed on them
- Systems that delivered “answers” without the delays (latency) many enterprise systems force on users.
Google’s ability to respond to this enterprise demand has been ineffectual. Like other Web search vendors, key word retrieval does not solve the problems basic search systems spawn. The GSA is evidence that Google does not have the key to unlock the revenue vault for enterprise search.
What Google search has done (inadvertently I might add) has been to make crystal clear that users do not want to work hard for information users perceive as useful. Precision and recall are irrelevant because voting and advertisers influence Google Web search results. Users love Google’s outputs.
In the organization, procurement teams, individual users, and senior management boil their needs down to one simple statement: “We want search that is just like Google.”
That’s a big, big problem for search and content processing vendors. Google Web search is not about relevance, objective information, or accuracy. Google is easy and “good enough.” In an organization, people want easy. But in an organization the results have to be timely, comprehensive in terms of what information is available to an organization, and accurate.
On the Web Google can skip content that is malformed or stored on a server that does not respond to a Google spider quickly enough. In an organization, the content has to be available. On the Web, the advertisers and the uses’ own behavioral data pays the bills. In an organization, the organization has to pay the bills. Google has more money from a different business model than most organizations. Google pumps money into plumbing to deliver the service that makes money. Organizations want to fix the amount spent on search and the funds are not infinite.
For search vendors, the problem of Google’s dominance in Web search makes product differentiation difficult. Google’s business model creates challenges for vendors who have to justify the “value” and hence the “cost” of their search systems. For traditional search vendors, ease of use is very, very difficult because of the nature of the questions enterprise system users have.
Google is a mirror in which societal, cultural, and intellectual changes in information access are reflected. For many years, I have called attention to the verbal push ups search vendors use to try and make sales. The struggled Hewlett Packard have had with Autonomy provide a glimpse of how “value” can be difficult to change into hard cash Microsoft’s Delve illustrates that search for Office 365 is a combination of contacts, alerts, and personalization, not key word search. The dependence on enterprise search companies for cash from venture capital sources illustrates that traditional search is a very, very tough business to make into something sustainable and profitable without financial life support. The expectations that Watson will become a $10 billion business in 60 months is disconnected from the experience of other smart companies. In the history of enterprise search, only Autonomy reported revenues of more than $800 million from enterprise licenses. IBM projects more than 10 X this revenue in 60 months. It took Autonomy more than a decade to hit $500 million.
The reality is that Google is not the problem. Google is a metaphor for what users want when it comes to information access.
The write up asserts:
The Google paradigm also ignores the challenge of scalability. Indexing the enterprise for a centralized enterprise search capability requires major investment. In addition, centralization runs counter to the realities of the working world where information must be distributed globally across a variety of devices and applications. The amount of information we create is overwhelming and the velocity with which that information moves increases daily.
Interesting statement. For me, the problem of the Google paradigm is that it another bit of jargon that sidesteps a what information retrieval must deliver in today’s business environment. Whoever cracks the code can make money. My hunch is that Google probed the enterprise search market and is trying to figure out how to make it pay off in a significant way. Google may be trapped in the same problem space through which other enterprise search and content processing vendors slog. The question may be, “Is there a way out of the swamp and into a land of milk, honey, sustainable revenue, and healthy margins?”
Stephen E Arnold, October 7, 2014
Gartner and Enterprise Search 2014
July 31, 2014
At lunch yesterday, several search aware people discussed a July 2014 Gartner study. One of the folks had a crumpled image of the July 2014 “magic quadrant.” This is, I believe, report number G00260831. Like other mid tier consulting firms, Gartner works hard to find something that will hook customers’ and prospects’ attention. The Gartner approach is focused on companies that purport to have enterprise search systems. From my vantage point, the Gartner approach is miles ahead of the wild and illogical IDC report about knowledge, a “quotient,” and “unlocking” hidden value. See http://bit.ly/1rpQymz. Now I have not fallen in love with Gartner. The situation is more like my finding my content and my name for sale on Amazon. You can see what my attorney complained about via this link, http://bit.ly/1k7HT8k. I think I was “schubmehled,” not outwitted.
I am the really good looking person. Image source: http://bit.ly/1rPWjN3
What the IDC report lacks in comprehensiveness with regard to vendors, Gartner mentions quite a few companies allegedly offering enterprise search solutions. You must chase down your local Garnter sales person for more details. I want to summarize the points that surfaced in our lunch time pizza fest.
First, the Gartner “study” includes 18 or 19 vendors. Recommind is on the Gartner list even though a supremely confident public relations “professional” named Laurent Ionta insisted that Recommind was not in the July 2014 Gartner report. I called her attention to report number G00260831 and urged her to use her “bulldog” motivation to contact her client and Gartner’s experts to get the information from the horse’s mouth as it were. (Her firm is www.lewispr.com and its is supported to be the Digital Agency of the Year and on the Inc 5000 list of the fastest growing companies in America.) I am impressed with the accolades she included in her emails to me. The fact that this person who may work on the Recommind account was unaware that Gartner pegged Recommind as a niche player seemed like a flub of the first rank. When it comes to search, not even those in the search sector may know who’s on first or among the chosen 19.
To continue with my first take away from lunch, there were several companies that those at lunch thought should be included in the Gartner “analysis.” As I recall, the companies to which my motley lunch group wanted Gartner to apply their considerable objective and subjective talents were:
- ElasticSearch. This in my view is the Big Dog in enterprise search at the moment. The sole reason is that ElasticSearch has received an injection of another $70 million to complement the $30 odd million it had previously gather. Oh, ElasticSearch is a developer magnet. Other search vendors should be so popular with the community crowd.
- Oracle. This company owns and seems to offer Endeca solutions along with RightNow/InQuira natural language processing for enterprise customer support, the fading Secure Enterprise Search system, and still popping and snapping Oracle Text. I did not mention to the lunch crowd that Oracle also owns Artificial Linguistics and Triple Hop technology. This information was, in my view, irrelevant to my lunch mates.
- SphinxSearch. This system is still getting love from the MySQL contingent. Imagine no complex structured query language syntax to find information tucked in a cell.
There are some other information retrieval outfits that I thought of mentioning, but again, my free lunch group does not know what it does not know. Like many folks who discuss search with me, learning details about search systems is not even on the menu. Even when the information is free, few want to confuse fantasy with reality.
The second take away is that rational for putting most vendors in the niche category puzzled me. If a company really has an enterprise search solution, how is that solution a niche? The companies identified as those who can see where search is going are, as I heard, labeled “visionaries.” The problem is that I am not sure what a search visionary is; for example, how does a French aerospace and engineering firm qualify as a visionary? Was HP a visionary when it bought Autonomy, wrote off $8 billion, and initiated litigation against former colleagues? How does this Google supplied definition apply to enterprise search:
able to see visions in a dream or trance, or as a supernatural apparition?
The final takeaway for me was the failure to include any search system from China, Germany, or Russia. Interesting. Even my down on their heels lunch group was aware of Yandex and its effort in enterprise search via a Yandex appliance. Well, internationalization only goes so far I suppose.
I recall hearing one of my luncheon guests say that IBM was, according the “experts” at Gartner, a niche player.Gentle reader, I can describe IBM many ways, but I am not sure it is a niche player like Exorbyte (eCommerce mostly) and MarkLogic (XML data management). Nope, IBM’s search embraces winning Jeopardy, creating recipes with tamarind, and curing assorted diseases. And IBM offers plain old search as part of DB2 and its content management products plus some products obtained via acquisition. Cybertap search, anyone? When someone installs, what used to be OmniFind, I thought IBM was providing an enterprise class information retrieval solution. Guess I am wrong again.
Net net: Gartner has prepared the ground for a raft of follow on analyses. I would suggest that you purchase a copy of the July 2014 Gartner search report. You may be able to get your bearings so you can answer these questions:
- What are the functional differences among the enterprise search systems?
- How does the HP Autonomy “solution” compare to the pre-HP Autonomy solution?
- What is the cost of a Google Search Appliance compared to a competing product from Maxxcat or Thunderstone? (Yep, two more vendors not in the Gartner sample.)
- What causes a company to move from being a challenger in search to a niche player?
- What makes both a printer company and a Microsoft-centric solution qualified to match up with Google and HP Autonomy in enterprise search?
- What are the licensing costs, customizing costs, optimizing costs, and scaling costs of each company’s enterprise search solution? (You can find the going rate for the Google Search Appliance at www.gsaadvantage.gov. The other 18? Good luck.)
I will leave you to your enterprise search missions. Remember. Gartner, unlike some other mid-tier consulting firms, makes an effort to try to talk about what its consultants perceive as concrete aspects of information retrieval. Other outfits not so much. That’s why I remain confused about the IDC KQ (knowledge quotient) thing, the meaning of hidden value, and unlocking. Is information like a bike padlock?
Stephen E Arnold, July 31, 2014
Huge Bets on Search: Spreadsheet Fever Rages
June 11, 2014
The news of the $70 million injected into Elasticsearch caused me to check out Crunchbase and some other sources of funding data. I looked at a handful of search and content processing vendors in the departures lounge. I am supposed to be retired, but Zurich beckons.
How large is the market for search and content processing software and services. As a former laborer in the vineyards of Halliburton Nuclear and Booz, Allen & Hamilton, the answer is, “You can charge as much as you want when the customer is in a corner.” The flipside of this adage is, “You can’t charge as much when there are many low cost options.”
In my view, search—regardless of the window dressing slapped on decades old systems and methods—is sort of yesterday. One of the goslings posted a list of Hewlett Packard’s verbal arabesques to explain IDOL search as everything EXCEPT search. The HP verbal arabesques make my point:
Search is not going to generate big money going forward.
Is search (regardless of the words used to describe it) a money pit like as the Tom Hanks’ motion picture made vivid?
For that reason, I am wondering what investors are thinking as they pump money into search and content processing companies. The largest revenue generator in the search sector is either Google or Autonomy. Google, as you may know, is in the online advertising business. Search is a Trojan horse. Search is free and the clicks trigger the GoTo/Overture mechanism that caused Google’s moment of inspiration. Before the Google IPO, Google ponied up some dough to Yahoo regarding alleged borrowing of pay to play methods.
Autonomy focused on the enterprise. Between 1996 and October 2011, Sir Michael Lynch grew the company to about $1 billion in revenues. HP’s prescient and always interesting management paid $10.3 billion for Autonomy and then wrote off $8 billion, aimed allegations at Autonomy at the company, and, in general, made it clear that HP was essentially a printer ink business with what seems to be great faith in IDOL, DRE, and assorted rich media tools.
More recently, IBM, the subject of an entertaining analysis The Decline and Fall of IBM by Robert X. Cringely suggested that Watson would grow to be a $10 billion in revenue business. Not a goal to ignore. The fact that Watson is a collection of home grown widgets and open source search technology. I think Watson’s last search contribution was creating a recipe for a tamarind flavored sauce. IBM is probably staffed with folks smarter than I. But a billion dollar bet with a goal of building a revenue stream 10 to 12 times greater than Autonomy’s in one third the time. Wowza.
Let’s do some simple addition in the elegant United lounge.
Let’s assume that IBM and HP actually generate the billions necessary to recover the cost of IDOL and hit the crazy IBM goal of $10 billion in four or five years. To make the math simple, skip interest, the cost of assuaging stakeholders, and the money needed to close deals that total $20 to $25 billion. HP pumps up Autonomy to $10 or $11 billion and IBM tallies another $10 to $12 billion.
So, HP and IBM need or want to build $10 billion or more in revenues from their respective search and content processing ventures. I estimated that the market for “search” was about $1.3 billion in 2006. I am not too sure that market has grown by a significant factor since the economic headwinds began blowing through carpetland.
Now consider the monies invested in some search and content processing companies.
Attensity (sentiment analysis), $90 million
BA Insight (Microsoft centric, search and business intelligence), $14.5 million
Content Analyst (text analysis, SAIC technology, $7.0 million
Coveo (originally all Microsoft all the time, now kitchen sink vendor), $34.7 million
Digital Reasoning (text analysis, no shipping product), $4.2 million
EasyAsk (natural language processing, several owners(, $20 million
Elasticsearch (open source search and consulting), $104 million
Hakia (semantic search), $23.5 million
MarkLogic (XML data management and kitchen sink apps), $73.6 million
Recorded Future (text analysis of Web content), $20.9 million
Recommind (similar to Autonomy method), $15 million
Sinequa (proprietary search and widgets), $5.3 million
X1 (search and new management), $12.2 million
ZyLab (search and licensed visualizations), $2.4 million
Alternatives to Google Popping up Everywhere
April 25, 2014
It’s a golden era for search alternatives. For a while there folks were worried about Google monopolizing the internet, but it’s not shaking out that way. Far from it, in fact. We are currently living in a golden age of niche search tools, as we discovered from a recent Virtual Strategy Magazine story, “MaxxCAT Raises the Bar for Search Performance with MaxxCAT 5.0.”
According to the story:
The 5.0 performance enhancements really come into their own when you begin looking at the scalability of our appliance in the enterprise…Sure, if you can build an index for a small amount of data in 5 minutes instead of 10, it’s nice, but it’s just 5 minutes. However, if you can index terabytes of data in 5 hours instead of 10 hours, that’s a huge difference.
MaxxCAT isn’t the only boat on this alternative Google sea, in fact, they aren’t even the biggest of the bunch. It’s not tough to find alternates, there are articles everywhere. The trickier part is finding one that fits your needs. Each serves a purpose, whether it is open source technology or privacy protection, that suits someone and repels others. This trial and error period is part of the fun, in our books.
Patrick Roland, April 25, 2014
Sponsored by ArnoldIT.com, developer of Augmentext
Another Week, Another Enterprise Search System
March 21, 2014
Cloud? Check.
Azure chip consultant reference? Check.
Social angle? Check.
Support for distributed information? Check.
Consumerized interface? Check.
Reference to value? Check.
Automatic alerts? Check.
Customer reference? Check.
Big company pedigree? Check.
Open sourciness? Check.
Exotic technology? Check.
There you have the recipe for a new enterprise search system, at least according to eWeek’s “Highspot Brings Machine Learning to Enterprise Search.” Highpoint’s Web site describes itself this way:
Built for the cloud era, Highspot uses advanced machine learning to help organizations capture, share, and cultivate their most valuable working knowledge.
The pricing information, omitted from the eWeek story just as azure chip consultants omit enterprise search fees, begins at free and comes out of the gate at $20 per user per month or $240 per user per year. For an organization with 400 users, the annual fee works out to about $96,000 for an open source, machine-learning system, a bargain compared to the Google Search Appliance but more expensive than downloading Solr, Searchdaimon, or Elasticsearch and having one staff get search up and running. A less expensive option that works reasonably well is dtSearch, but you need to love the color blue for this search system. If you want an appliance, check out Maxxcat’s systems. These are far less expensive than other appliances, and the new systems are easy to set up and deploy. For cloud action, take a look at Blossom Software’s solution. Chances are your state, country, or municipal government is using the Blossom system built by a former Bell Labs’ whiz kid.
Net net: The enterprise search market is flooded with options. With big, waddling outfits like HP and IBM getting increasingly desperate to make their billion dollar bets pay off, you have high end options as well as free downloadable systems from organizations in Denmark, Norway, Russia, and elsewhere.
Will the pricing hold if a business licensee points the system at 50 million documents? My hunch is that there will be some fine print. Google charges about $900,000 for its appliance capable of processing tens of millions of documents with three years of support. You can check the latest US government discount prices at www.gsaadvantage.gov. Just search for “Google Search Appliance” and peruse the government’s price. A commercial price may vary.
The key is that the engines of many systems are open source. The “solution” is software wrappers and checklists that hit the marketing hot buttons. Keep up with Highspot via the company’s blog at http://blog.highspot.com/.
Stephen E Arnold, March 21, 2014
Google Puts Some Effort into the Google Search Appliance
February 12, 2014
Last I knew, the Google Search Appliance (GAS) had trimmed its product line, eliminated the impulse buy option for the Mini, and kept the price at the higher end of the appliance market.
I learned over the last two years that Google has placed more than 60,000 GSAs in organizations. I have no idea if the number is valid, but if it is, the GSA is one of the top dogs in enterprise search. I also heard that there was a small team working on the GSA and an even smaller team handling customer support. Google pushes functions to resellers who deal with the customers. Google outsources manufacturing of the GSA. Most important, Google seems to have an off-again, on-again interest in on premises search. The future, as I understand it, is the cloud. The GSA is, in my opinion, an anachronism in the Nest, X Labs, and Android-Chrome world. But, hey, I have been wrong before. I once asserted that basic search should not be a challenge for most organizations. Wow, did I get that wrong! Jail time, law suits, and DARPA’s almost admission that search is not working notwithstanding.
The GSA has been around almost a decade. Version 7.2 is “a leader in the Garnet Enterprise Search MQ.” I certainly don’t doubt the word of an estimable azure chip consulting firm. No, no, no.
The new version, according to Google, delivers:
- Metadata sorting. A function available in the 1983 version of Fulcrum Technologies’ system
- language translation. A function available from Delphes in the 1990s
- A document preview function. iPhrase in 1999 delivered this feature
- Entity recognition. Verity implemented this function in the 1980s
- Dynamic navigation. Endeca rolled out this feature in 1998
In my opinion, the GSA is catching up to innovations available for many years from other vendors. Comparing the EPI Thunderstone and Maxxcat appliances to the GSA emphasizes that the GSA is not quite at parity with other products in the channel.
According to “Google Updates Enterprise Search Appliance Tool,”
The GSA 7.2 update comes more than a year after the firm upgraded the GSA to version 7.0, and builds on the features included in that update. The most notable includes the ability to improve the way data can be indexed with key attributes, such as author name, or the date it was created.
How much does a GSA cost? According to the US government’s GSAadvantage.gov, a 36 month license for a GB 7007 is $69,296 for 500,000 documents. Have more documents? Pay for an upgrade. However, I can use a hosted service like Blossom Software to index my content for about $2,400 per month. I can use the low cost dtSearch solution for $160 per seat. I can download an open source solution and do it myself.
For an organization with 20 million documents to index, the cost of the GSA solution noses into HP Autonomy territory. Too rich for my blood, and I think that lower cost appliance vendors will see the Google Search Appliance as a lead generator.
I wonder if those azure chip consultants have licensed the GSA to handle their Intranet information retrieval tasks?
Stephen E Arnold, February 12, 2014
Search Application Perspectives For 2014
January 20, 2014
With 2014 well under way, search experts are trying to predict what will happen for enterprise search. Search Appliance World has an article that takes a look on enterprise search in the past and future called, “The New Search Appliance Landscape: Reflections And Predictions With MaxxCAT.” Basic search commands that come in out-of-the-box system are old school and do not provide the robust solution enterprise systems need.
Search appliances became enterprise users’ favorite toys and everyone had to have the Google Mini Search Appliance, but those days are gone. Other search developers, such as MaxxCat, stepped up to the plate.
The article states:
“ ‘In 2013, we saw a lot of the fallout from that as customers realized they couldn’t replace their Google Mini appliance and went looking for viable alternatives that weren’t $30K. For us, this lead to a huge boost in sales of our entry level appliances and even some additional sales of our enterprise series appliances,’ MaxxCAT Director of Marketing & Sales Chris Whissen told Search Appliance World.”
The MaxxCat developers were interested in exploring new markets their search appliance could expand into. The company is also big on customer service and ensuring that clients know they are valued. The biggest endeavor being made, though, is offering MaxxCat’s clients an efficient solution to solve their search problems and to encourage more competition in the search application market. Google is no longer the small player, but some of its solutions have grown too expensive for its former clients. New companies like MaxxCat keep the market fresh and offer up new ideas.
Whitney Grace, January 20, 2014
Sponsored by ArnoldIT.com, developer of Augmentext
-
- Subscribe to Beyond Search
Feature archive
News archive
-
