The Future of Enterprise Search: The View of a WeWork World
July 13, 2018
I read “The Future of Enterprise Search: Visual, Voice & Vertical.” My reaction was, “This approach to enterprise search describes a WeWork world.” My view of enterprise search is much, much different. Let me point out that I am okay with voice interfaces, but I am struggling to come to grips with the idea of visual search in an enterprise as one of the three pillars of enterprise search as i understand the function. The “vertical” angle is another way of saying, “Enterprise search does not work as a one size fits all solution. Therefore, let’s embrace a search engine for the legal unit, one for the computational chemists, one for marketers, and so on.
The write up points out that organizations, needs, and marketing are in flux. Uncertainty is the name of the game. That’s why there are employees who aren’t really full time equivalents working in Starbuck’s and WeWork offices. Who has a desk, assistants, and a regular nine to five job? Darned few people today. If we recognize the medieval set up of most organizations, the traditional definition of a job has more in common with the world of the Willy Loman (low man on totem pole, get it?). Life today has kings, court staff, and peasants. The difference is that the staff and peasants have mobile phones; otherwise, we’re back in the 7th century CE.
Skipping over the copy and paste of an Economist chart, the guts of the expository essay explains visual, voice, and vertical. The Vs reminded me of IBM’s alleged insight about Big Data’s volume, velocity, and variety. A mnemonic with alliteration. Okay, just not enterprise search as I have defined it in a number of my books; for example, The New Landscape of Search, published by Panda, years ago.
Enterprise search makes it possible for an employee to obtain the information needed to complete a business task. I pointed out that an employee cannot perform some work without locating digital information and data needed to answer a question. My examples included locating the most recent version of a CEO’s PowerPoint presentation, a list of the suppliers for a particular component in a product, information about an alleged personnel matter which violated the terms of an agreement with a customer, and the lab notes relative to a new compound developed by a chemical engineer with a structure diagram.
Now it would be wonderful if I could speak to a mobile device and have the data for any one of these enterprise search tasks delivered to me. But there are a couple of problems; namely, the screen size and capabilities of most mobile devices. For these information tasks, I personally prefer a multi monitor set up, a printer, plus old fashioned paper and pencil for notes.
The visual search angle is useful when looking for engineering drawings or chemical structures. But the visual component is only a part of the information I needed. That lab notebook is important, particularly if the product is going to be commercialized or patented or used as a bargaining chip in a deal with a potential partner or acquisition.
The vertical part is, as I have said, the reason that the typical organization has dozens of information access systems. A decade ago, according to our research, a Fortune 500 company licensed most of the available enterprise search systems, one or more legal search systems, and the specialist tools for those working with engineering drawings, specifications, vendor profiles, etc.
I don’t want to suggest that the discussion of visual, voice, and vertical search in the Search Engine Journal is with value. The information is simply not on point for today’s organization information access requirements.
For those in the top tier of workers — that is, those with senior positions and staff — the tools needed are more diverse and must be more robust. For those laboring away in WeWork offices, voice and visual search may be the go to ways to get information. The vertical search systems are useful, but for many workers, the expertise required to make a chemical structure search system deliver useful outputs is outside those workers’ skill set without some work and midnight oil.
To sum up, enterprise search is a difficult concept. Simplifying it to the three Vs understates the challenge. Explaining enterprise search in terms of semantic technology, natural language processing, and the other difficult to define jargon sprints to the far end of the complexity spectrum.
That’s why enterprise search is problematic. The vendors hope for a buyer and then head for the beach or a new job. The customers end up like Robinson Caruso, stranded and alone with tools that usually fall into disrepair quickly. Enterprise search itself is jargon, but it is jargon which has been marginalized by systems which over promised and under delivered.
That’s a mnemonic and acronym for you: OPUD.
Stephen E Arnold, July 13, 2018
Sinequa Review: Questions Go Unanswered
July 12, 2018
I read “Sinequa Review” by an outfit called Finances Online. I think the idea is an interesting one. Navigate to a Web page and get a snapshot about a product. In this case, the vendor is Sinequa, and its product is described as “an integrated search platform that can extract information from your interconnected applications and environments. Aside from letting you draw data rapidly, it affords you actionable and intelligent insights that it gains through its deep content analyses.”
That suggests that Sinequa is more than a search engine. In my book CyberOSINT, I pointed out that next generation information access systems represent the path forward for making sense of digital information. I did not include Sinequa is that book’s profiles of vendors to watch.
Like many vendors of keyword search, Sinequa has been working hard to find a way to describe basic search in terms of higher value functions. The Finances Online write up about Sinequa illustrates the difficulty a company like Sinequa has in describing its various functions; for example, “extract information from your interconnected applications and environments.” I am not sure what that means.
The listing of benefits strikes me as different from what I identified in CyberOSINT. In that monograph, I focused on a system’s ability to identify high value or potentially high interest data automatically, interfaces which move beyond Google style lists of results which create more work for the analyst because relevance and a document’s inclusion of a specific item of data are impossible without directly reading a document, and analytic functions designed to present data in the context of the user.
Contrast CyberOSINT’s key features with Sinequa’s:
- In depth analytics
- Connectors
- Mobile strategy
We have congruence with analytics; however, misses on the other features.
The features of Sinequa are a listing of buzzwords. In CyberOSINT, the idea is that next generation information access systems emphasize outputs, not the mechanisms “under the hood.”
The cost of an enterprise search or NGIA system is a difficult issue. The big expenses for enterprise search or NGIA systems are planning, administration, training, set up, optimization, customization, content preparation, and remediation (most of these systems don’t work “out of the box.”)
Here’s how pricing of Sinequa is explained:
Sinequa is a platform that transforms your organization into an information driven one. If you are interested in powering your company or institution with the solution, you can request custom enterprise pricing from the sales team by phone, email, web form, or chat.
I think that inclusion of downsides about a product is important. Perhaps Finances Online will include:
-
- Pricing information from verified customers; for example, last year, the total cost of ownership was …
- Details about issues with the Sinequa system which may date from 2002
- A more informed listing of competitors
- Less jargon and fewer undefined buzzwords; for example, “deep content analytics” and “semantics”.
The trajectory of old school search vendors has been similar to the approach taken to extend the life and usefulness of DEC 10s and 20s. Wrappers of software can keep somethings youthful—at least at first glance. Don’t get me wrong. Quick looks can be useful.
Stephen E Arnold, July 12, 2018
Will Cognitive Search (Whatever That Is) Change Because of Squrro?
July 2, 2018
We are not exactly sure what cognitive search is. That’s a plus. Changing cognitive search should be easy. Tweak technology, do some wordsmithing, and the landscape is different. I think this works for some search wizards in today’s fluid environment for information access.,
Cognitive search is ready to undergo a major shift and a few companies seem to be leading the pack. The way that we not only process search, but the questions we ask, could see a drastic change quite soon. We were alerted to this move from a recent product called Squrro, with their post “Add Context, Accuracy, and Speed to Your Enterprise Searches.”
According to the piece:
“Cognitive search is the new generation of enterprise search that uses artificial intelligence (AI) to return results that are more relevant to the user or embedded in an application issuing the search query. The benefits of understanding and extracting more insights from content changes search from, “This document has the keyword in the title so it must be more relevant” to understanding user intent…”
According to reports, a wide variety of businesses are responding to this more detailed ability of cognitive search to serve customers. For example, one Swiss insurance company recently adopted Squrro technology to provide customers with better search options. The company, Helevetia, claims its five million customers are benefiting from this change.
We will have to wait until industry fonts of wisdom like LinkedIn and CMSWatch provide their views. LinkedIn is, however, a Microsoft entity and there is that Fast Search ESP stub. CMSWatch often views the world from the perspective of content management or CMS. (Is there such a thing as cognitive CMS?)
Clarity will be forthcoming in step with the anticipated Elastic IPO. That financial event makes enterprise search more interesting again. Frankly making IBM Watson and Coveo equivalents did not do it for our research team.
Here in Harrod’s Creek, we want our old fashioned Boolean queries to return relevant results. We like the old school precision and recall approach.
Patrick Roland, July 2, 2018
What Has Happened to Enterprise Search?
June 28, 2018
I read “Enterprise Search in 2018: What a Long Strange Trip It’s Been.” I found the information presented interesting. The thesis is that enterprise search has been on a journey almost like a “Wizard of Oz” experience.
The idea of consolidation, from my point of view, boils down to executives who want to cash in, get out, and move on. The reasons are not far to seek: Over promising and under delivering, financial manipulations, and positioning a nuts and bolts utility as something that solves information problems.
Some, maybe many, licensees of proprietary enterprise search systems may have viewed their investment as an opportunity that delivered an unexpected but inevitable outcome. Where is that lush scenery? Where’s the beach?
The reality is that enterprise search vendors were aced by Shay Banon. His Act II of Compass: A Finding Story was Elasticsearch and the company Elastic. Why not use free and open source software. At least the code gets some bugs fixed unlike old school proprietary enterprise search systems. Bug fixes? Yep, good luck with your Fast Search & Retrieval ESP platform idiosyncrasies.
The landscape today is a bit like the volcanic transformation of Hawaii’s Vacationland. Real estate agents will be explaining that the lava flows have created new beach views, promising that eruptions are a low probability event.
The write up does not highlight one simple fact: Enterprise search has given way to “roll up” services or what I call “meta-plays.” The idea is that search is tucked inside systems like Diffeo, Palantir Gotham, and other “intelligence” platforms.
Why aren’t these enterprise grade solutions sold as “enterprise search” or “enterprise business intelligence and discovery solutions”?
The answer is that the information retrieval nest has been marginalized by the actions of vendors stretching back to the Smart system and to the present with “proprietary” solutions which actually include open source technology. These systems are anchored in the past.
Consider Diffeo?
Why offer enterprise search when one can provide a solution that delivers information in context, provides collaboration tools, and displays information in different ways with a single mouse click?
Coveo Positions Itself to Fend off Enemies
June 4, 2018
Coveo, one of the numerous players in the race for AI supremacy, took a massive leap forward recently. By securing some substantial investments, the company is poised to make a big splash in the field. However, we are not certain money is the answer to all their concerns, after reading a recent press release on their site, “Coveo Announces $100 Million Investment Led By Evergreen Coast Capital.”
According to the story:
“Coveo, a recognized leader in AI-powered insight, recommendations and search engines, has secured a $100 million investment from Elliott Management for a 27% stake in the company. The investment was led by Elliott’s Menlo Park, California-based private equity affiliate, Evergreen Coast Capital.”
Nice work if you can get it, to be sure. However, we will be curious whether or not this money makes much of a dent in the market. For instance, competition like Elastic have been gaining ground and Algolia are actually acquiring other companies in an effort to better position themselves. Keep an eye on this fight, because we suspect the company that comes out on top will begin making a major impact on our daily lives through their AI offerings.
One final thought: Will Coveo and companies like Attivio and LucidWorks be able to generate sufficient revenue to pay off the investors and generate a sustainable revenue stream? From our vantage point 45 minutes from Churchill Downs where gambling is a way of life, we think the odds are long, very long.
Perhaps a larger company will buy one of these three firms, allowing the senior managers to have a big payday and retire. Dassault Systèmes, Hewlett Packard, IBM, and Oracle have expensive search stallions in their stable. We assume there will be other prospects if the revenue race stumbles.
Patrick Roland, June 1, 2018
Search Bias a Big Topic Across the Board
June 1, 2018
Manipulating bias in search is a tricky business that is often left in the hands of contractors or third parties. While that may be good enough for others, you need search results manipulated the way they were intended. Luckily, those options are becoming more common than ever as we discovered when we saw the Thunderstone blog that “Thunderstone releases Version 20” and that includes some beefy upgrades.
According to the post:
“Parametric profiles gain new capabilities to set a bias on a per document field using data from field rules. This allow documents to be biased up or down in the search results, for example PDF results could be biased down, or documents with “Important” in a meta tag could be ranked higher.”
Ranking PDFs isn’t the only way in which search bias is a tricky business. Often, we are hearing about this factor for al the wrong reasons, like when Google’s search results are biased one way or another. However, the search giant uses similar technology, but on a much more grand scale, to eliminate these issues as they recently pointed out: “Google is committed to making products that work well for everyone, and are actively researching unintended bias and mitigation strategies.” Clearly, bias is an issue on the high and low end of everyone’s spectrum.
Patrick Roland, June 1, 2018
Enterprise Search: Not Even a Bridesmaid Now
June 1, 2018
By this point in our history, you would think that most enterprise search would be created equally. However, there are still some stalwarts that are only now stepping into the golden age of search, as it were. One odd entry into this club was discovered after reading a white paper by Hyland on “Enterprise Search”.
According to the paper on Hyland’s new product:
“Enterprise Search allows you to provide access to the right information even if users don’t know exactly where to find it. Enable anyone to build powerful queries without expertise using intuitive, menu-assisted and natural language search options. You can also find answers when there are misspelled search terms, inexact queries, substandard data, document errors and other faults.”
Seems a little elementary, right? We suspect this has a little to do with playing catchup in the field. Hyland recently acquired Lexmark’s enterprise software arm, which itself was never exactly leading the field. (Lexmark bought ISYS Search Software, an outfit with technology from the mid 1980s if the founder spoke the truth when I interviewed him in 2008.) This will be interesting to see where Hyland takes this technology, though. The company has been around for a while and seems intent on wedging itself into the enterprise data conversation. However, it’ll have to make some light speed advances in order to go toe-to-toe with the big dogs in this battle.
Patrick Roland, June 1, 2018
Attivio Continues to Move Its Technology Forward
May 25, 2018
Conceived by former Fast Search & Transfer executives, Attivio has moved from a system able to analyze baseball statistics to enterprise search to business intelligence and probably several other market spaces. Enterprise search vendors do that these days.
Now the newest version of Attivio is here, we learn from the company’s blog post, “Attivio Product News: Version 5.5.1 Available Today!” The write-up describes improvements in several areas. With the updated software development kit (SDK), one can test code before deploying it to the platform. As for security, we’re told Attivio has migrated to a stronger algorithm and upgraded libraries to their latest versions. Text extraction has also been improved and now works with over 600 formats. Furthermore, access to recent modules is also included; the post promises:
“Finally, we’ve made the latest modules part of the install. This includes the WebCrawler module, which enables you to ingest web pages, as well as newly released Search Analytics and Search UI Toolkit. As we’ve written about previously, Search Analytics gives you insight into the performance of your search platform in real time. And SUIT, Attivio’s Search UI Toolkit, is a framework for quickly building search applications from the simplest to the most complex. It’s an open source application that can be downloaded from GitHub, and enhanced by the community. It not only works with the Attivio platform, but also with Elasticsearch and Solr.”
How Fast like is Attivio? A faint imprint of the genetic code is there, but Attivio has, like other search vendors, adopted proprietary and open source technology. The trick is the marketing today. Attivio is chugging along but it faces enterprise search challengers fueled by venture funding. What’s interesting is that money continues to flow into what I would describe as “traditional” enterprise search plays; for example, Coveo. The hurdle, of course, is to convert investors’ money and support into sustainable, growing, profit spinning revenue. And that’s a challenge from my point of view.
See the Attivio post for more details on each of the above improvements. Founded in 2007 (shortly before Fast Search’s implosion and the sale of the Fast property to Microsoft and the legal dust up about Fast Search’s “fast” math). Attivio’s seems to be hiring. That’s encouraging.
Cynthia Murrell, May 25, 2018
Search Is a Problem: Still a Clumsy Song and Dance Routine
May 24, 2018
Enterprise search has been around for decades. Hundreds of consultants have asserted patterns, models, methods, and MBA infused strategies to “fix” enterprise search.
Why?
Wherever there is an organization with one or more enterprise search systems, I have found these characteristics:
- Unhappy users
- Unhappy senior manager
- Unhappy bean counters
- Unhappy vendors
- Usually happy consultants if they are paid.
I am biased, old, and hard nosed. After writing the first three editions of the Enterprise Search Report, the New Landscape of Search, adding a word or two to that astounding guru Martin White’s book about Successful Enterprise Search Management, talking with dozens of PhD candidates whose dissertatioins about search and retrieval would change the world, and meeting with vendors large and small for decades—I am amused by the arm waving enterprise search engenders.
Don’t get me wrong. There are very good information access systems. But these vendors license solutions which usually focus on solving a specific problem. Case in point: Blackdot, Terbium Labs, and Verint, and many others.
From the point of view of flailing content management experts, “enterprise search” means finding information in a usually flawed, Rube Goldberg construct called a CMS or content management system.
Against this wallpaper with my scrawled biases, I read “Diagnosing Enterprise Search Failures.” The pivot point for the story is another report that almost two thirds of enterprise search users are not satisfied with the retrieval system.
Like a reprise of a vaudeville act from the 1920s at a rap concert, the music and the footwork are stale, out of touch, and worn.
Enterprise search had its decade in the sun. The period between 1995 and 2005 was the golden age of search. Then the sun imploded. Over-promising and under-delivering made it clear to those licensing enterprise search systems that finding information was not a solution to digital information woes.
In fact, an enterprise search system exacerbated the problems employees encountered when trying to locate specific information. Fast Search & Transfer, Convera, Delphes, Entopia (remember that outfit), and other aggressively marketed companies found out that companies would license technology and then balk at the on going costs.
One by one the big names in enterprise search went out of business or found themselves owned by larger firms with a belief that their managers could make search a winner.
How did that work out? Chase down someone at Lexmark and ask about their experience with ISYS Search Software. Repeat the process at Dassault Systèmes? Do the same thing for products ranging from Artificial Linguistics to Vivisimo.
The result is that the universe of companies offering search solutions has changed since 2008. The legal dust up between HP and Autonomy continues. Search did not make HP happy.
Surveys are fine, but the data reveal nothing new. Enterprise search is not a solution to information problems in an enterprise. Companies are embracing free or low cost solutions based on open source technology. Specialist systems which address specific information access problems are thriving. One may not think of Diffeo and Palantir Technologies as enterprise search systems, but they are information access solutions and not designed to solve a panoply of retrieval and information management issues.
The reason enterprise search fails to please users boils down to the disconnect between what the user wants and what an enterprise wide system can deliver. The vendors promise more than technology can provide.
Checklists, MBA rah rah, and misplaced confidence in technology will not solve these specific challenges:
- The cost of maintaining, upgrading, and tuning an enterprise search system to the needs of specific users is significant
- Users have a keen desire to rely on the software to do the thinking for them. When a system requires the user to think or formulate a query or perform downstream analysis, the search system becomes a problem
- Procurement teams often lack the discipline and clout to lay out tight requirements and select a vendor to do that job. The pattern is to create a wish list, sign a deal, and leave the baggage of failure behind.
- The systems provided do not match what the marketers demo, suggest, or assert the software will actually do in an affordable, reliable, understandable manner.
As a former rental at a reasonably competent management consulting firm, a method for figuring out how to solve a problem has one objective: Sell billable work. I understand that.
Do not confuse a consultant’s report with solving the problem of enterprise search. If enterprise search worked, there would be little appetite for methodologies to figure out failure.
Why such hostility to enterprise search? I think clueless large and medium sized companies want to buy a silver bullet. Even better, the bullet must kill the content vampire with a single, low cost, easy to use, accurate shot.
That’s not going to happen… ever.
The problem is that individuals looking for information need tools to solve quite specific business tasks. In enterprise search, there are numerous points of failure; for example:
- Management support is weak
- Organizational infighting triggers departments to get their own search solution
- The technology does not work
- Results do not meet user needs
- Funding is insufficient
- Technical staff find that fixes are not easy or possible
- Content known to be in the system cannot be found
- Vendors change direction from search to customer support and leave search customers dangling
- The people involved are focused on their careers, lunch, or finding a new job, not the nitty gritty of designing a solution for a specific group of workers with an information need.
And there are other issues related to over-promising and under-delivering. I wrote about this years ago and talked about falling off the cliff of high expectations. Enterprise search users inevitably crash into the reality of the system. Thus, the significant percentage of dissatisfaction with enterprise search.
I know of no enterprise search system which delivers on these points. Furthermore, as venture funding flows into Coveo and LucidWorks, as IBM falls farther and farther behind its revenue goals for Watson search (OmniFind, Vivisimo, et al), and as Microsoft buys more and more search start ups in the hopes of finding a silver bullet to its search mess—It is clear that stakeholders, customers, and users are going to become increasingly annoyed at the problem of enterprise search.
Why did Google bow out of enterprise search? Why has Elastic emerged as the go-to solution for many enterprise search applications? Why are companies like Funnelback, Sophia, Exorbyte, and dozens of others scrambling?
Enterprise search looked like a solution to some important problems. Today not so much. Open source search software is fine. However, how many of the open source vendors are going to be able to generate a return for their investors with what amounts to free software.
Enterprise search is the wrong label for today’s solutions. Even proprietary systems in hock for $100 million have longer odds than a nag entered in the Kentucky Derby.
Therefore, thrashing.
Stephen E Arnold, May 24, 2018
Algolia: Doing What Exalead Failed to Do
May 7, 2018
I read “How Algolia Built Their Real-time Search as a Service Product.” Reading between the lines and doing a bit of thinking, I arrived a hypothesis. The story begins with the Exalead search system. (You can get some information from the original three editions of “The Enterprise Search Report” which I wrote between 2004 and 2008. I also have a for fee profile of Exalead which you can order by writing benkent2020 @ yahoo dot com. The report is $40 payable via PayPal.)
The developers of Algolia focused on the shortcomings of Exalead, which has not changed significantly since its purchase by Dassault Systèmes. A number of Exalead professionals have left the company and had an impact on a number of companies. That may be the case at Algolia, or the founders of Algolia identified the weakness of other French systems and moved forward. Does anyone think about Antidot, Datop, Pertimm, Sinequa, and other French centric search systems?
Crunchbase reports that Algolia says:
Algolia is the most reliable platform for building search. Our hosted search API supplies the building blocks for creating great search to connect your users with what matters most to them. Our hosted search API powers billions of queries for thousands of websites & mobile applications every month, delivering relevant results in an as-you-type search experience in under 50ms anywhere in the world. Algolia’s full-stack solution takes the pain out of building search; we maintain the infrastructure & the engine, and we provide extensive documentations to our dozens of up-to-date API clients and SDKs with all the latest search features, so you can focus on delighting your users.
The write up explains that the complexity of other search systems, the lack of a hosted cloud-based platform, and the failure to swap out proprietary code for open source alternatives have differentiated Algolia from other enterprise search systems.
Some reviews of the system are available on Stackshare. Among the strengths of the system are its speed, its ease of implementation, and its distributed search network. No negatives jumped out at me. Algolia seems to in a good place at this time.
The system is also available for free for “community projects.”
Several observations:
- Large companies purchasing search systems often find that change and improvement is difficult, if not impossible. Too bad for Exalead.’
- The open source orientation of Algolia may put some pressure on Elastic. I would include Lucidworks, but that company continues to borrow or chase venture funds because the home run swing is not yet butter smooth. But Algolia has ingested $74 million, and like Lucidworks, that money has to make money; otherwise, exciting events occur.
- French vendors have had some difficulty penetrating certain markets; for example, the US government. Perhaps Algolia will succeed where other French companies have fallen short.
For more information about Algolia, navigate to www.algolia.com.
I would point out that the European experts and the US SEO crowd have not paid much attention to Algolia. Quite a few dead horses are being whipped while Elastic romps forward. In the US, search means SEO, and that band of merry wizards remains convinced that Google will put their clients’ Web pages at the top of the results list without buying Google ads.
Yeah, and I believe in the tooth fairy.
Stephen E Arnold, May 7, 2018
-
- Subscribe to Beyond Search
Feature archive
News archive
-
