Will Cognitive Search (Whatever That Is) Change Because of Squrro?

July 2, 2018

We are not exactly sure what cognitive search is. That’s a plus. Changing cognitive search should be easy. Tweak technology, do some wordsmithing, and the landscape is different. I think this works for some search wizards in today’s fluid environment for information access.,

Cognitive search is ready to undergo a major shift and a few companies seem to be leading the pack. The way that we not only process search, but the questions we ask, could see a drastic change quite soon. We were alerted to this move from a recent product called Squrro, with their post “Add Context, Accuracy, and Speed to Your Enterprise Searches.”

According to the piece:

“Cognitive search is the new generation of enterprise search that uses artificial intelligence (AI) to return results that are more relevant to the user or embedded in an application issuing the search query. The benefits of understanding and extracting more insights from content changes search from, “This document has the keyword in the title so it must be more relevant” to understanding user intent…”

According to reports, a wide variety of businesses are responding to this more detailed ability of cognitive search to serve customers. For example, one Swiss insurance company recently adopted Squrro technology to provide customers with better search options. The company, Helevetia, claims its five million customers are benefiting from this change.

We will have to wait until industry fonts of wisdom like LinkedIn and CMSWatch provide their views. LinkedIn is, however, a Microsoft entity and there is that Fast Search ESP stub.  CMSWatch often views the world from the perspective of content management or CMS. (Is there such a thing as cognitive CMS?)

Clarity will be forthcoming in step with the anticipated Elastic IPO. That financial event makes enterprise search more interesting again. Frankly making IBM Watson and Coveo equivalents did not do it for our research team.

Here in Harrod’s Creek, we want our old fashioned Boolean queries to return relevant results. We like the old school precision and recall approach.

Patrick Roland, July 2, 2018

What Has Happened to Enterprise Search?

June 28, 2018

I read “Enterprise Search in 2018: What a Long Strange Trip It’s Been.” I found the information presented interesting. The thesis is that enterprise search has been on a journey almost like a “Wizard of Oz” experience.

The idea of consolidation, from my point of view, boils down to executives who want to cash in, get out, and move on. The reasons are not far to seek: Over promising and under delivering, financial manipulations, and positioning a nuts and bolts utility as something that solves information problems.

lava flow fixed

Some, maybe many, licensees of proprietary enterprise search systems may have viewed their investment as an opportunity that delivered an unexpected but inevitable outcome. Where is that lush scenery? Where’s the beach?

The reality is that enterprise search vendors were aced by Shay Banon. His Act II of Compass: A Finding Story was Elasticsearch and the company Elastic. Why not use free and open source software. At least the code gets some bugs fixed unlike old school proprietary enterprise search systems. Bug fixes? Yep, good luck with your Fast Search & Retrieval ESP platform idiosyncrasies.

The landscape today is a bit like the volcanic transformation of Hawaii’s Vacationland. Real estate agents will be explaining that the lava flows have created new beach views, promising that eruptions are a low probability event.

The write up does not highlight one simple fact: Enterprise search has given way to “roll up” services or what I call “meta-plays.” The idea is that search is tucked inside systems like Diffeo, Palantir Gotham, and other “intelligence” platforms.

Why aren’t these enterprise grade solutions sold as “enterprise search” or “enterprise business intelligence and discovery solutions”?

The answer is that the information retrieval nest has been marginalized by the actions of vendors stretching back to the Smart system and to the present with “proprietary” solutions which actually include open source technology. These systems are anchored in the past.

Consider Diffeo?

Why offer enterprise search when one can provide a solution that delivers information in context, provides collaboration tools, and displays information in different ways with a single mouse click?

Read more

Coveo Positions Itself to Fend off Enemies

June 4, 2018

Coveo, one of the numerous players in the race for AI supremacy, took a massive leap forward recently. By securing some substantial investments, the company is poised to make a big splash in the field. However, we are not certain money is the answer to all their concerns, after reading a recent press release on their site, “Coveo Announces $100 Million Investment Led By Evergreen Coast Capital.”

According to the story:

Coveo, a recognized leader in AI-powered insight, recommendations and search engines, has secured a $100 million investment from Elliott Management for a 27% stake in the company. The investment was led by Elliott’s Menlo Park, California-based private equity affiliate, Evergreen Coast Capital.”

Nice work if you can get it, to be sure. However, we will be curious whether or not this money makes much of a dent in the market. For instance, competition like Elastic have been gaining ground and Algolia are actually acquiring other companies in an effort to better position themselves. Keep an eye on this fight, because we suspect the company that comes out on top will begin making a major impact on our daily lives through their AI offerings.

One final thought: Will Coveo and companies like Attivio and LucidWorks be able to generate sufficient revenue to pay off the investors and generate a sustainable revenue stream? From our vantage point 45 minutes from Churchill Downs where gambling is a way of life, we think the odds are long, very long.

Perhaps a larger company will buy one of these three firms, allowing the senior managers to have a big payday and retire. Dassault Systèmes, Hewlett Packard, IBM, and Oracle have expensive search stallions in their stable. We assume there will be other prospects if the revenue race stumbles.

Patrick Roland, June 1, 2018

Search Bias a Big Topic Across the Board

June 1, 2018

Manipulating bias in search is a tricky business that is often left in the hands of contractors or third parties. While that may be good enough for others, you need search results manipulated the way they were intended. Luckily, those options are becoming more common than ever as we discovered when we saw the Thunderstone blog that “Thunderstone releases Version 20 and that includes some beefy upgrades.

According to the post:

“Parametric profiles gain new capabilities to set a bias on a per document field using data from field rules.  This allow documents to be biased up or down in the search results, for example PDF results could be biased down, or documents with “Important” in a meta tag could be ranked higher.”

Ranking PDFs isn’t the only way in which search bias is a tricky business. Often, we are hearing about this factor for al the wrong reasons, like when Google’s search results are biased one way or another. However, the search giant uses similar technology, but on a much more grand scale, to eliminate these issues as they recently pointed out: “Google is committed to making products that work well for everyone, and are actively researching unintended bias and mitigation strategies.” Clearly, bias is an issue on the high and low end of everyone’s spectrum.

Patrick Roland, June 1, 2018


Enterprise Search: Not Even a Bridesmaid Now

June 1, 2018

By this point in our history, you would think that most enterprise search would be created equally. However, there are still some stalwarts that are only now stepping into the golden age of search, as it were. One odd entry into this club was discovered after reading a white paper by Hyland on “Enterprise Search”.

According to the paper on Hyland’s new product:

“Enterprise Search allows you to provide access to the right information even if users don’t know exactly where to find it. Enable anyone to build powerful queries without expertise using intuitive, menu-assisted and natural language search options. You can also find answers when there are misspelled search terms, inexact queries, substandard data, document errors and other faults.”

Seems a little elementary, right? We suspect this has a little to do with playing catchup in the field. Hyland recently acquired Lexmark’s enterprise software arm, which itself was never exactly leading the field. (Lexmark bought ISYS Search Software, an outfit with technology from the mid 1980s if the founder spoke the truth when I interviewed him in 2008.) This will be interesting to see where Hyland takes this technology, though. The company has been around for a while and seems intent on wedging itself into the enterprise data conversation. However, it’ll have to make some light speed advances in order to go toe-to-toe with the big dogs in this battle.

Patrick Roland, June 1, 2018

Attivio Continues to Move Its Technology Forward

May 25, 2018

Conceived by former Fast Search & Transfer executives, Attivio has moved from a system able to analyze baseball statistics to enterprise search to business intelligence and probably several other market spaces. Enterprise search vendors do that these days.

Now the newest version of Attivio is here, we learn from the company’s blog post, “Attivio Product News: Version 5.5.1 Available Today!” The write-up describes improvements in several areas. With the updated software development kit (SDK), one can test code before deploying it to the platform. As for security, we’re told Attivio has migrated to a stronger algorithm and upgraded libraries to their latest versions. Text extraction has also been improved and now works with over 600 formats. Furthermore, access to recent modules is also included; the post promises:

“Finally, we’ve made the latest modules part of the install. This includes the WebCrawler module, which enables you to ingest web pages, as well as newly released Search Analytics and Search UI Toolkit. As we’ve written about previously, Search Analytics gives you insight into the performance of your search platform in real time. And SUIT, Attivio’s Search UI Toolkit, is a framework for quickly building search applications from the simplest to the most complex. It’s an open source application that can be downloaded from GitHub, and enhanced by the community. It not only works with the Attivio platform, but also with Elasticsearch and Solr.”

How Fast like is Attivio? A faint imprint of the genetic code is there, but Attivio has, like other search vendors, adopted proprietary and open source technology. The trick is the marketing today. Attivio is chugging along but it faces enterprise search challengers fueled by venture funding. What’s interesting is that money continues to flow into what I would describe as “traditional” enterprise search plays; for example, Coveo. The hurdle, of course, is to convert investors’ money and support into sustainable, growing, profit spinning revenue. And that’s a challenge from my point of view.

See the Attivio post for more details on each of the above improvements. Founded in 2007 (shortly before Fast Search’s implosion and the sale of the Fast property to Microsoft and the legal dust up about Fast Search’s “fast” math). Attivio’s seems to be hiring. That’s encouraging.

Cynthia Murrell, May 25, 2018

Search Is a Problem: Still a Clumsy Song and Dance Routine

May 24, 2018

Enterprise search has been around for decades. Hundreds of consultants have asserted patterns, models, methods, and MBA infused strategies to “fix” enterprise search.


Wherever there is an organization with one or more enterprise search systems, I have found these characteristics:

  1. Unhappy users
  2. Unhappy senior manager
  3. Unhappy bean counters
  4. Unhappy vendors
  5. Usually happy consultants if they are paid.

I am biased, old, and hard nosed. After writing the first three editions of the Enterprise Search Report, the New Landscape of Search, adding a word or two to that astounding guru Martin White’s book about Successful Enterprise Search Management, talking with dozens of PhD candidates whose dissertatioins about search and retrieval would change the world, and meeting with vendors large and small for decades—I am amused by the arm waving enterprise search engenders.

Don’t get me wrong. There are very good information access systems. But these vendors license solutions which usually focus on solving a specific problem. Case in point: Blackdot, Terbium Labs, and Verint, and many others.

From the point of view of flailing content management experts, “enterprise search” means finding information in a usually flawed, Rube Goldberg construct called a CMS or content management system.

Against this wallpaper with my scrawled biases, I read “Diagnosing Enterprise Search Failures.” The pivot point for the story is another report that almost two thirds of enterprise search users are not satisfied with the retrieval system.

Like a reprise of a vaudeville act from the 1920s at a rap concert, the music and the footwork are stale, out of touch, and worn.

Enterprise search had its decade in the sun. The period between 1995 and 2005 was the golden age of search. Then the sun imploded. Over-promising and under-delivering made it clear to those licensing enterprise search systems that finding information was not a solution to digital information woes.

In fact, an enterprise search system exacerbated the problems employees encountered when trying to locate specific information. Fast Search & Transfer, Convera, Delphes, Entopia (remember that outfit), and other aggressively marketed companies found out that companies would license technology and then balk at the on going costs.

One by one the big names in enterprise search went out of business or found themselves owned by larger firms with a belief that their managers could make search a winner.

How did that work out? Chase down someone at Lexmark and ask about their experience with ISYS Search Software. Repeat the process at Dassault Systèmes? Do the same thing for products ranging from Artificial Linguistics to Vivisimo.

The result is that the universe of companies offering search solutions has changed since 2008. The legal dust up between HP and Autonomy continues. Search did not make HP happy.

Surveys are fine, but the data reveal nothing new. Enterprise search is not a solution to information problems in an enterprise. Companies are embracing free or low cost solutions based on open source technology. Specialist systems which address specific information access problems are thriving. One may not think of Diffeo and Palantir Technologies as enterprise search systems, but they are information access solutions and not designed to solve a panoply of retrieval and information management issues.

The reason enterprise search fails to please users boils down to the disconnect between what the user wants and what an enterprise wide system can deliver. The vendors promise more than technology can provide.

Checklists, MBA rah rah, and misplaced confidence in technology will not solve these specific challenges:

  1. The cost of maintaining, upgrading, and tuning an enterprise search system to the needs of specific users is significant
  2. Users have a keen desire to rely on the software to do the thinking for them. When a system requires the user to think or formulate a query or perform downstream analysis, the search system becomes a problem
  3. Procurement teams often lack the discipline and clout to lay out tight requirements and select a vendor to do that job. The pattern is to create a wish list, sign a deal, and leave the baggage of failure behind.
  4. The systems provided do not match what the marketers demo, suggest, or assert the software will actually do in an affordable, reliable, understandable manner.

As a former rental at a reasonably competent management consulting firm, a method for figuring out how to solve a problem has one objective: Sell billable work. I understand that.

Do not confuse a consultant’s report with solving the problem of enterprise search. If enterprise search worked, there would be little appetite for methodologies to figure out failure.

Why such hostility to enterprise search? I think clueless large and medium sized companies want to buy a silver bullet. Even better, the bullet must kill the content vampire with a single, low cost, easy to use, accurate shot.

That’s not going to happen… ever.

The problem is that individuals looking for information need tools to solve quite specific business tasks. In enterprise search, there are numerous points of failure; for example:

  • Management support is weak
  • Organizational infighting triggers departments to get their own search solution
  • The technology does not work
  • Results do not meet user needs
  • Funding is insufficient
  • Technical staff find that fixes are not easy or possible
  • Content known to be in the system cannot be found
  • Vendors change direction from search to customer support and leave search customers dangling
  • The people involved are focused on their careers, lunch, or finding a new job, not the nitty gritty of designing a solution for a specific group of workers with an information need.

And there are other issues related to over-promising and under-delivering. I wrote about this years ago and talked about falling off the cliff of high expectations. Enterprise search users inevitably crash into the reality of the system. Thus, the significant percentage of dissatisfaction with enterprise search.

I know of no enterprise search system which delivers on these points. Furthermore, as venture funding flows into Coveo and LucidWorks, as IBM falls farther and farther behind its revenue goals for Watson search (OmniFind, Vivisimo, et al), and as Microsoft buys more and more search start ups in the hopes of finding a silver bullet to its search mess—It is clear that stakeholders, customers, and users are going to become increasingly annoyed at the problem of enterprise search.

Why did Google bow out of enterprise search? Why has Elastic emerged as the go-to solution for many enterprise search applications? Why are companies like Funnelback, Sophia, Exorbyte, and dozens of others scrambling?

Enterprise search looked like a solution to some important problems. Today not so much. Open source search software is fine. However, how many of the open source vendors are going to be able to generate a return for their investors with what amounts to free software.

Enterprise search is the wrong label for today’s solutions. Even proprietary systems in hock for $100 million have longer odds than a nag entered in the Kentucky Derby.

Therefore, thrashing.

Stephen E Arnold, May 24, 2018

Algolia: Doing What Exalead Failed to Do

May 7, 2018

I read “How Algolia Built Their Real-time Search as a Service Product.” Reading between the lines and doing a bit of thinking, I arrived a hypothesis. The story begins with the Exalead search system. (You can get some information from the original three editions of “The Enterprise Search Report” which I wrote between 2004 and 2008. I also have a for fee profile of Exalead which you can order by writing benkent2020 @ yahoo dot com. The report is $40 payable via PayPal.)

The developers of Algolia focused on the shortcomings of Exalead, which has not changed significantly since its purchase by Dassault Systèmes. A number of Exalead professionals have left the company and had an impact on a number of companies. That may be the case at Algolia, or the founders of Algolia identified the weakness of other French systems and moved forward. Does anyone think about Antidot, Datop, Pertimm, Sinequa, and other French centric search systems?

Crunchbase reports that Algolia says:

Algolia is the most reliable platform for building search. Our hosted search API supplies the building blocks for creating great search to connect your users with what matters most to them. Our hosted search API powers billions of queries for thousands of websites & mobile applications every month, delivering relevant results in an as-you-type search experience in under 50ms anywhere in the world. Algolia’s full-stack solution takes the pain out of building search; we maintain the infrastructure & the engine, and we provide extensive documentations to our dozens of up-to-date API clients and SDKs with all the latest search features, so you can focus on delighting your users.

The write up explains that the complexity of other search systems, the lack of a hosted cloud-based platform, and the failure to swap out proprietary code for open source alternatives have differentiated Algolia from other enterprise search systems.

Some reviews of the system are available on Stackshare. Among the strengths of the system are its speed, its ease of implementation, and its distributed search network. No negatives jumped out at me. Algolia seems to in a good place at this time.

The system is also available for free for “community projects.”

Several observations:

  1. Large companies purchasing search systems often find that change and improvement is difficult, if not impossible. Too bad for Exalead.’
  2. The open source orientation of Algolia may put some pressure on Elastic. I would include Lucidworks, but that company continues to borrow or chase venture funds because the home run swing is not yet butter smooth. But Algolia has ingested $74 million, and like Lucidworks, that money has to make money; otherwise, exciting events occur.
  3. French vendors have had some difficulty penetrating certain markets; for example, the US government. Perhaps Algolia will succeed where other French companies have fallen short.

For more information about Algolia, navigate to www.algolia.com.

I would point out that the European experts and the US SEO crowd have not paid much attention to Algolia. Quite a few dead horses are being whipped while Elastic romps forward. In the US, search means SEO, and that band of merry wizards remains convinced that Google will put their clients’ Web pages at the top of the results list without buying Google ads.

Yeah, and I believe in the tooth fairy.

Stephen E Arnold, May 7, 2018

Former Autonomy Executive Found Culpable

May 1, 2018

I read “U.S. Jury Convicts Former Autonomy Executive of Fraud over HP Deal.” The jury found Sushovan Hussain guilty of wire fraud. Reuters said that in 2009 Mr. Hussain began to “deceive Autonomy’s investors and HP( about the company’s financial condition and prospects for growth.”

An ArnoldIT profile about Autonomy is available without charge at this link.

The HP purchase of Autonomy was one of the major turning points in enterprise search. The $11 billion deal made clear that enterprise search was an application space which would have provided rocket fuel for HP’s growth.

HP discovered that enterprise search was a challenging technology to use as a way to generates billions of dollars in revenue quickly and easily.

Beyond Search’s view of this deal—as well as the sale of Exalead to Dassault Systèmes, Vivisimo’s sale to IBM, and the manic repositioning of the vendors pitching proprietary search solutions—is that may companies found that enterprise search was a tough nut to crack. Marketing is easier than generating sustainable revenues. In my experience, enterprise search requires specialized expertise.

What’s interesting is that I heard that HP executives reviewed the Autonomy financial data, knew about the Qatalyst 2011 report about Autonomy, and decided to purchase the company. The deal seemed to unfold quickly and then implode almost as quickly. HP paid more for Autonomy than any other search acquisition of which I was aware. HP emerged as the proud owner of the firm which brought Bayesian methods to the enterprise. (I want to mention that the mathematical procedures implemented by Autonomy are now incorporated into most of the next generation information access systems I discussed in CyberOSINT, my book on what’s beyond search.


A page from the 2011 Qatalyst report about Autonomy. The full document, once available on the Oracle Web site, is now difficult to locate via open source methods.

This legal dust up may gather momentum. Possibly appeals and more people accused by HP making their way to the court room. I will stay tuned for developments reported by “real news” outfits like Thomson Reuters.

Stephen E Arnold, May 1, 2018

Attivio and MC+A Combine Forces

April 7, 2018

Over the years, Attivio positioned itself as more than search. That type of shift has characterized many vendors anchored in search and retrieval. We noted that Attivio has “partnered” with MC+A, a search centric company. MC+A also forged a relationship with Coveo, another search and retrieval vendor with a history of repositioning.

We learned from “Attivio and MC+A Announce Partnership to Deliver Next-Generation Cognitive Search Solutions” at Markets Insider that:

“MC+A will resell Attivio’s platform, seamlessly integrate their enterprise-grade connectors into it, and provide SI services in the US market. ‘Partnering with MC+A extends our ability to address organizations’ needs for making all information available to employees and customers at the moment they need it,’ said Stephen Baker, CEO at Attivio. ‘This is particularly critical for companies looking to upgrade legacy search applications onto a modern, machine-learning based search and insight platform.’ …

The story added:

“By combining self-learning technologies, such as natural language processing, machine learning, and information indexing, the Attivio platform is helping Fortune 500 enterprises leverage customer insight, surface upsell opportunities, and improve compliance productivity. MC+A has over 15 years of experience innovating with search and delivering customized search-based applications solutions to enterprises. MC+A has also developed a connector bridge solution that allows customers to leverage existing infrastructure to simplify the transition to the Attivio platform.”

Attivio was founded in 2007, and is headquartered in Newton, Massachusetts. The company’s client roster includes prominent organizations like UBS, Cisco, Citi, and DARPA. Attivio in its early days was similar in some ways to the Fast Search & Transfer technology once cleverly dubbed ESP. No, not extra sensory perception. ESP was the enterprise search platform.

Based in Chicago and founded in 2004, MC+A specializes in implementations of cognitive search and insight engine technology. A couple of years ago, MC+A was involved with Yippy, the former Vivisimo metasearch system. When IBM bought Vivisimio, the metasearch technology morphed into a Big Data component of Watson.

If this walk down memory lane suggests that vendors of proprietary systems have been working to find purchase on revenue mountain, there may be  a reason. The big money, based on information available to Beyond Search, comes from integrating open source solutions like Lucene into comprehensive analytic systems.

In a nutshell, the rise of Lucene and Elastic have created opportunities for some companies which can deliver more comprehensive solutions than search and retrieval anchored in old-school solutions.

More than repositioning, jargon, and partnerships may be needed in today’s market place where “answers”, not laundry lists are in demand. For mini profiles of vendors which are redefining information access and answering questions, follow the news stories in our new video news program DarkCyber. There’s a new program each week. Plus, you can get a sense of the new directions in information access by reading my 2015 book (still timely and very relevant) CyberOSINT: Next Generation Information Access.

Stephen E Arnold,

Stephen E Arnold, April 7, 2018

« Previous PageNext Page »

  • Archives

  • Recent Posts

  • Meta