BA Insight on the Resurrection Trail
September 10, 2014
I read “Artificial Intelligence Is Resurrecting Enterprise Search.” The unstated foundation of this write up is that enterprise search is dead. I am not sure I buy into that assumption. Last time I checked ElasticSearch was thriving with its open source approach. In fact, one “expert” pointed out that the decline in the fortunes of certain Brand Name search systems coincided with the rise in ElasticSearch’s fortunes. Connection? I don’t know, but enterprise search is thriving.
What needs resurrection (either the Phoenix variety or the Henry James’s varieties of mystical experience type) is search vendors whose software does not deliver for licensees. In this category are outfits that have just gone out of business; for example, Convera, Delphes, Entopia, Kartoo, Perfect Search, Siderean Software, and others).
Then there are the vendors with aging technology that have sold out to outfits that pack information retrieval into umbrella applications in order to put hurdles for competitors to scale. If lock in won’t work, then find a way to build a barricade. Outfits with this approach include Dassault, OpenText, Oracle, TeraText (now Leidos), among others.
Also, there are search vendors up to their ears in hock to venture funding firms. With stakeholders wanting some glimmer of a payout, the pressure is mounting. Companies in this leaky canoe include Attivio, BA Insight, Coveo, and Lucid Imagination, among others.
Another group of vendors are what I call long shots. These range from the quirky French search vendors like Antidot to Sinequa. There are some academic spin outs like Funnelback, which is now a commercial operation with its own unique challenges. And there are some other cats and dogs that live from deal to deal.
Finally, there are the giant companies looking for a way to make as much money as possible from the general ennui associated with proprietary search solutions. IBM is pitching Watson and using open source to get the basic findability function up and running. Microsoft is snagging technology from Jabber and bundling in various bits and pieces to deliver on the SharePoint vision of access to information in an organization. This Delve stuff is sort of smart, but until the product ships and provides access to a range of content types, I think Microsoft has a work in progress, not an enterprise solution upon which one can rely. The giant IHS is leveraging acquired technology into a search business, at least in the planners’ spreadsheets. Google offers its Search Appliance, which is one of the most expensive appliance solutions I have encountered. There is one witless mid tier consulting firm that believes a GSA is economical. Okay. And there is the name surfing Schubmehl from IDC who uses other people’s work to build a reputation.
To sum up, ElasticSearch is doing fine. Lots of other vendors are surviving or selling science fiction.
So what?
The “Artificial Intelligence Is Resurrecting Enterprise Search” is a write up from one of the outfits eager to generate big dollars to keep the venture capitalists happy. Hey, don’t take the money, if the recipients can’t generate big bucks.
Anyway, the premise of the write up is that enterprise search is dead and Microsoft’s Delve will give the software sector new life. The only folks who will get new life are the Microsoft savvy developers who can figure out how to set up, customize, optimize, and keep operational a grab back of software.
Microsoft wants to provide a corporate SharePoint user with a single interface to the content needed to perform work. This is a pretty tough problem. SageMaker, now long gone, failed at this effort. Google asserted that its Search Appliance could pull off this trick. Google failed. Dozens of vendors talk about federated search and generally deliver results that are of the “close but no cigar” variety.
Now what’s artificial intelligence got to do with Delve? Well, the system uses personalization and cues to figure out what a business SharePoint user wants and needs. We know how well this works with the predictive services available from Apple, Google, and—Microsoft Phone. Each time I use these services, I remember that they don’t work too well. Yep, Google really knows what I want about one out of a 1,000 queries. The other 999 Google generates laughable outputs.
Microsoft will be in the same rubber raft.
The write up does disagree with my viewpoint. Well, that’s okay because the BA Insight professional who tackles artificial intelligence is going to need more than inputs from Dave Schubmehl who recycles my information without my permission. If this write up is any indication, something has gone wrong somewhere along the line with regard to artificial intelligence, which is, I believe, an oxymoron.
Delve is, according the the write up, now “turning search on its head.” What? I need to find information about a specific topic. How will a SharePoint centric solution know I need that information? Well, that is not a viable scenario. Delve only knows what I have previously done. That’s the beauty of smart personalization. The problem is that my queries bounce from Ebola to silencers for tactical shotguns, from meth lab dispersion in Kentucky to the Muslim Brotherhood connections to certain political figures. Yep, Delve is going to be a really big help, right?
The write up asserts:
Companies need to get smarter about how they structure their information by addressing core foundational data layers. Pay attention to corporate taxonomies and introduce automated processes that add additional metadata where it’s left out from unstructured data sets. Doing this homework will make enterprise search results more relevant and will allow better results when interacting with enterprise data — whether it’s through text, voice or based on social distance. Access to enterprise data through intelligent interfaces is only getting better.
My reaction? My goodness. What the heck does this collection of buzzwords have to do with advanced software methods for information retrieval? Not much. That’s what the write is conveying to me.
Hopefully the investors in BA Insight find more to embrace than I do. If I were an investor, I would demand that my money be spent for more impactful essays, not reminders that Microsoft like IBM thrives on services, certification, and customers who may not know how to determine if software is smart.
Stephen E Arnold, September 10, 2014
Comments
One Response to “BA Insight on the Resurrection Trail”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
Steve –
As you know, I am a fan of yours and appreciate your insightful commentary on our industry.
Having read your recent post “BA Insight on the Resurrection Trail” I strongly disagree on a number of points.
On a key point, you say we’ve proclaimed Enterprise Search dead. On the contrary, I adamantly believe that our field is having a resurgence of innovation and is delivering substantial value; my colleagues’ article says in the first sentence “The future of enterprise search is bright.” Your article talks about the ennui in the industry, the many dead or struggling ventures, and how new initiatives are doomed to fail the same way past ones have.
We need optimism, not ennui
In the big picture, the spirit of our article is noticing and recognizing some exciting trends in the industry. I definitely feel that we need new growth, new trends, and new interest in the possibilities of the technology – if for nothing more than to muster the energy to succeed with projects. In the enterprise, frustration with finding important information is not abating, in fact it’s getting worse – as evidenced by Findwise’s annual survey, released today.
Steve, you say “What the heck does this have to do with advanced software methods for information retrieval? Not much.” Agreed, it is not about advanced algorithms. There is plenty of interesting and advanced stuff in the work we reference, but an obsession with magic new algorithms and overhype of them has hurt our industry. The core algorithms have not progressed significantly; search cores are commoditized and even learning algorithms are getting mainstreamed. Many companies are stuck in the “search immaturity cycle” and need help applying this technology and doing the homework they need to do. We should put focus on successful applications and the products and practices that support this.
It is possible to field great search, if people put enough value on it to pay attention to how and where to apply it. This is true of traditional keyword search and equally true of the emerging user paradigms and applications our article discussed. Our industry is trying to tackle an extremely hard problem, and there’s lots of innovation still to be done, for sure. We agree that Elasticsearch is part of that innovation and is thriving – that’s why we are working with it, too.
But no technology or search core will succeed without good practices and effective applications. People need to see that search can be good, understand that it can be different than their current experience, and be interested enough to pay attention and resource projects to succeed – not just plug things in and walk away.
We need facts and interesting opinions, not ad hominums
We have no problem with any opinion you may venture, but your article contained some inaccuracies and insinuations. Please feel free to call anytime if you want to ask something or check facts.
– We get that you have a bone to pick with Dave Schubmehl of IDC as you mention regularly. But what in the world does that have to do with us? You say that “BA Insight…is going to need more than inputs from Dave Schubmehl”. This makes is sound as if we are steering by Mr. Schubmehl’s inputs and have no intelligence or experience on our own. Please don’t spread your fight to us.
– The article states that we are one of the companies in the industry who have been overcapitalized and are in a ‘leaky canoe”, putting us in with names that we agree have that issue but which have publicly announced rounds an order of magnitude larger than we have, and burned this cash to reach roughly the same size we are. You know us to be a capital-efficient place. The article also paints us in a broad category of “search vendors whose software does not deliver for licensees”. Steve, If you want to advise our investors or talk to our customers, we’ll happily introduce you.
– You have several times (though not in this particular article) called us “apologists for Microsoft”. We do work very intimately with Microsoft search technology, as well as working with open source search. We regularly publish material and teach about both what is good and what is flawed with this technology, and get great positive feedback about how we help people be successful with this technology despite its flaws. If that’s being apologists, OK then. Why are we focused on SharePoint portals combined with SharePoint search or with Elasticsearch? Because that’s where the mainstream market is, and where people need help.
I get that you have made being ‘the curmudgeon’ a brand and am glad that works for you. We at BA Insight are still excited about the possibilities in the industry and what can be done.
Personal assistants, AI, and Delve
Personal assistants are not a new idea; there have been many incarnations over the years. And they will never be clairvoyant nor universal – we agree. But that doesn’t matter. What matters is that they are hitting the mainstream – undeniable if you watch your kids using Siri, and clear from the industry events our expert talked about.
Delve is not the first attempt to provide you information without your explicitly asking for it; the Office Graph is not the first graph search, and there are many dead bodies by the side of the road – we agree. But we’ve been working with Delve for months and the key is the access to content. As you say “this Delve stuff is sort of smart, but until the product ships and provides access to a range of content types..”. Well – the product shipped from Microsoft on Tuesday (the 8th), and we at BA Insight are working to provide connections to it from a wide range of content outside of Office 365, from outside of the Microsoft sphere altogether, and from outside the enterprise as well. No organization of any size has all of its content in one place or all important information inside its boundaries.
Is AI an oxymoron? Amusing to say, sure, and as a field it certainly suffered from an enormous overhype and consequent fallow period – even more than our text analytics/NLP/search field. But tell that to the people building self-driving cars or semi-autonomous robots. Look at the prevalence of machine learning. Or just look at the journals of the AAAI today. That old joke is stuck in the ’90s.
As for your assertion that “Microsoft will be in the same rubber raft” as previous failures of predictive or learning algorithms, this comes back my point that it’s more than the algorithms. Sure the algorithms and use of machine learning are ‘cool’ – but for any of these types of algorithms the output depends on the data available as input and the features chosen, more than the algorithm itself. Speech recognition technology has steadily improved over the years primarily because of more data, not fundamentally different algorithms. Delve’s embedding in Office 365 and its basic premise of tapping information where you use it and leveraging your behavior in Office gives it a real fighting chance. Hence the built-in connections to your calendar, your and your teammates’ documents, your email, your OneDrive and Yammer networks. “How will a SharePoint centric solution know I need that information?” you ask. Well, for example, when I have a meeting scheduled on my calendar, the subject and the invitees are pretty strong clues to relevant stuff I might want to know. the documents the invitees have been working on that relate to the subject
Let’s debate
It would be fun to have a real debate about the future of the industry and to what degree the trends in our article are overhyped versus being leading indicators. Happy to do that anytime with you.
Jeff Fried
CTO
BA Insight