Proprietary Enterprise Search: False Hopes and Brutal Costs
December 21, 2015
At lunch the other day, the goslings and I engaged in what I thought was a routine discussion: The sad state of the enterprise search market.
I pointed out that the “Enterprise Search Daily” set up by Edwin Stauthamer was almost exclusively a compilation of Big Data articles. Enterprise search, although the title of the daily, was not the focal point of the content.
Enterprise search is a cost black hole. R&D, support, customization, and bug fixes gorge on money and engineers. Instead of adding value to an enterprise system, search becomes the reason the CFO has a migraine and why sales professionals struggle to close deals.
I said, “Enterprise search has disappeared.”
One of the goslings asked, “What’s happened to the proprietary search systems acquired by some big companies?”
We were off an running.
The goslings mentioned that Dassault Systèmes bought Exalead and the brand has disappeared from the US market. IBM bought Vivisimo, and the purchase was explained as a Big Data buy, but the company and its technology have disappeared into the Great Blue Hole, which is today’s IBM. Hummingbird bought Fulcrum, and then OpenText bought Hummingbird. Open Text owns Information Dimension’s BASIS, BRS Search, and its own home brew search system. Oracle snapped up Endeca, InQuira, and RightNow in a barrage of search binge shopping. Lexmark—formerly a unit of Big Blue—bought ISYS Search Software and Brainware. Then there was the famous purchase of Fast Search & Transfer by Microsoft and the subsequent police investigation and the charges filed against a former executive for fancy dancing with the revenue numbers. And who can forget the $11 billion purchase of Autonomy by IBM. There have been other deals, and the goslings enjoyed commenting on this.
I called a halt to the lunch time stand up comedy routine. The executives of these companies were trying to do what they thought was best for their [a] financial future and [b] for their stakeholders. Some of these stakeholders had suffered through revenue droughts and were looking for a way out of the sea of red ink enterprise search vendors generate with aplomb.
The point I raised was, “Does the purchase of a proprietary enterprise search system?” make a substantive contribution to the financial health of the purchasing company.
Canada Based Coveo Predicts the Future of Enterprise Search
December 17, 2015
Coveo, an enterprise search vendor once closely focused on Microsoft SharePoint, offered some predictions about 2016 and search.
What does the Québec City based search vendor presage? The answer appears in “Coveo Releases 2016 Enterprise Search Predictions.” Here you go:
- Machine learning. IBM will definitely support this prediction or their public relations advisors will.
- Cybersecurity. Yep. This is a hot area. Some of the companies pushing new boundaries in search are outfits like Diffeo, Tiberian, and Recorded Future (backed by the Alphabet Google outfit). Key word search does not sleep in the same king sized bed as these outfits in my opinion.
- Real time engagement and learning for digital transformation. I thought of Yahoo. The company has several thousand people working on search. How is that working out for the Xoogler running Yahoo?
- Proactive recommendations. Okay, this is the selective dissemination or SDI thing. SDIs are useful but these cannot be annoying. Does anyone remember Desktop Data? I do.
- Search makes systems more intelligent. Well, maybe. search is a utility. The more successful of the intelligence automation outfits uses subcomponents of search; for example, entity extraction, metadata generated on the fly, and relationship maps. These are, in my opinion, more important than old school search.
The write up includes a remarkable quote. I have placed this gem in my folder for future reference. Here is the statement I highlighted in bold red marker ink:
2016 will be a pivotal year for enterprise search. Organizations now recognize the strategic value of using intelligent search-based applications to drive more customer and employee engagement.
Based on the information available to me as we complete The Dark Web Dilemma, 2016 is going to be a bit different from 2015 when venture money flowed like water to outfits in the content processing game. My thought about the future is that companies which have ingested oodles of cash have to generate revenue growth, demonstrate a path to profitability, or face a Convera, Delphes, Entopia, or Siderean Software type future.
What’s that type of future?
A sell out or a shut down. My hunch is that the stakeholders in the Coveo play are looking for a buyer just like Lexmark.
Stephen E Arnold, December 17, 2015
Big Data and Enterprise Search: The Caution Lights Are Flashing
December 15, 2015
I read “How You Should Explain Big Data to Your CEO.” The write up included a link which triggered thoughts of how enterprise search dug itself a hole and climbed. Unable to extricate itself from a problem enterprise search vendors created, the entire sector has been marginalized. In some circles, enterprise search is essentially a joke. “Did you hear about the three enterprise search vendors who walked into a bar?” The bartender says, “What is this? Some kind of joke?”
The link pointed me to a Slideshare (owned by the email and job hunting champion LinkedIn). That presentation, “5 Signs Your Big Data Project is Doomed to Fail,” could have been borrowed from one of my talks about enterprise search in 2001. It was not, but the basic message was identical: Big Data has created a situation in which there are some challenges here and now.
The presentation was prepared by Qubole (maybe pronounced cue ball?). Qubole is a click to query outfit. This means that reports from Big Data are easy to generate.
Here are the problems Big Data faces:
- Failed implementations. Qubole asserts that 87 percent of the Big Data implementations are flops
- 73 percent of executive describe the Big Data project as flop
- 45 percent of Big Data projects are completed
These data are similar to the results of “satisfaction” with enterprise search solutions.
Why? Qubole asserts:
- Inaccurate project scope
- Inadequate management support
- No business case
- Lack of talent (in search the talent may be present but overestimates its ability to deal with enterprise search technologies and processes)
- “Challenging tools.” I think this means that in the Big Data world there are lots of complexities.
What can one charged with either search or Big Data tasks do with this information?
My view is, “Ignore it.”
The “can do” spirit carries professionals forward. Hiring a consultant provides some job protection but does little to reverse the failure and disappointment rate.
My view is that the willingness of executives to grab at a magic solution presented by a showman marketer overrides failure date. The arrogance of those involved create a “that won’t happen to us” belief.
Who is to blame? The company for believing in baloney? The marketer for painting a picture and showing a Hollywood style demo? The developers who created the Big Data solution, knowing that chunks were not complete or just did not work before the ship date? The in house engineers who lacked self knowledge to understand their own limitations?
Everyone is in the hole with the enterprise software vendors. The hole is deep. Magic solutions are difficult to pin down. The future of Big Data is likely to parallel to some degree the dismal track record of enterprise search. Fascinating. I can hear the mid tier consultants and the handful of remaining enterprise search vendors asserting that Qubole’s points are not applicable to their specific situation.
Yep, and I believe in the tooth fairy and Santa.
Stephen E Arnold, December 15, 2015
Paper.Li Enterprise Search Daily Not Updating
December 11, 2015
Update December 12, 2015–The Enterprise Search Paper.li is updating again. The content is primarily about Big Data.
The Enterprise Search Daily, which shifted from enterprise search to Big Data, is not updating. Is it a Paper.li problem or a lack of suitable information about search?
The archives are available from this link. If the page resolves, click on the Archives and you can view a Search Daily by clicking on a calendar. My hunch is that the flow of information about enterprise search slowed in 2015. Marketers have embraced Big Data even if their systems do not really handle lots of information without developing indigestion. Earlier this year, I heard that a real publisher was going to introduce a newsletter about enterprise search. I am not sure if anyone will notice.
If the Enterprise Search Daily turns up in my Overflight system again, I will note the date. For now, fans of enterprise search will have to entertain themselves by running queries on the topics related to search. Big Data is not a useful search phrase. Everybody does Big Data according to the marketers and the venture firms funding search and retrieval. What about the conference organizers who run for fee shindigs to discuss enterprise search? Yikes. I am delighted to be beyond search.
Stephen E Arnold, December 11, 2015
Enterprise Software Survey: ZDNet Content Marketing
December 8, 2015
“Research: Enterprise Software Rising with 69 Percent Usage” caught my attention. I like numbers which suggest two thirds adoption of software which seems ubiquitous to me. So the write looks like news. It sounds like news. Sort of news I think.
The write up is a marketing pitch for a new study by ZDNet and its for fee research arm Tech Pro Research. “Enterprise Software: Advantages, Opportunities, Challenges” studies enterprise software. I am not exactly sure what “enterprise” means in the ZDNet world. My hunch is that the term embraces commercial enterprises which buy products from the ZDNet advertisers. I could be wrong, of course, because enterprise may mean, the local dry launderette entrepreneur.
The article discloses some interesting factoids, which I assume ZDNet and a host of enterprise software sales people believe are solid gold. None of the gold foam stuff.
I noted these points:
Enterprise software has a definition I found surprising. I learned:
Enterprise software encompasses a variety of functions, including asset management, business intelligence, CRM utilities, data processing, databases, financial applications, identity management, retail software, process management and resource planning. It can run on either individual machines or on centralized servers, whether in-house, located in the cloud or a hybrid combination.
No, enterprise search, no analytics, no Big Data, and no enterprise cyber security. Yikes. The mid tier consultants are probably perspiring in their home offices due to these omissions.
I found this factoid interesting: nine percent of the sample do not currently use an enterprise software solution but are considering one in the next year and 22 percent do not use enterprise software and are not planning on changing their stripes next year. That works out to 31 percent of the sample not doing the enterprise software thing even though, I assume, the companies in the sample were enterprises. I said, “Huh. Imagine that.”
Even more puzzling was the list of enterprise software deployed in the last 12 months by the sample.
- Human resources
- Storage
- Databases, note the plural
- Big Data
- Mobile
Now the paragraph I quote about the functions of enterprise software did not include Big Data. What’s the scoop, ZDNet. Is Big Data an enterprise application or not? My view is that Big Data is for the folks who understand analytic-type stuff. Most enterprises have precious few of these types of people. Another fascinating point.
The preferred vendors identified in the write up came at me from left field. I admit I am not in the know like the real journalists at ZDNet. Here’s that listing:
- Adobe Systems
- Dropbox
- LinkedIn (my goodness)
- Microsoft
But the un-preferred vendors is more intriguing. This group includes:
- CA Technologies
- Oracle
- Red Hat
- SAP
I assume that each of these companies will be really thrilled to meet with the ZDNet ad sales professionals.
If you want more, you will be able to explore the opportunities by diving into the for fee version of the study.
Stephen E Arnold, December 7, 2015
Has Enterprise Search Drowned in a Data Lake?
December 6, 2015
I had a phone conversation with a person who unluckily resides in a suburb of New York City. Apparently the long commute allows my contact to think big thoughts and formulate even bigger questions. He asked me, “What’s going to happen to enterprise search?”
I thought this was a C minus questions, but New Yorkers only formulate A plus questions. I changed the subject to the new Chick Fil-A on Sixth. After the call, I jotted down some thoughts about enterprise search.
Here for your contemplation are five of my three comments which consumed three legal pad sheets. I also write small.
Enterprise Search Is Week Old Celery
In the late 1990s when the Verity hype machine was rolling and the Alphabet boys were formulating big thoughts about search, enterprise search was the hot ticket. For some techno cravers, enterprise search was the Alpha and Omega. If information is digital, finding an item of information was the thrill ride ending in a fluffy pile of money. A few folks made some money, but the majority of the outfits jumping into search either sold out or ended up turning off the lights. Today, enterprise search is a utility and the best approach is to use an open source solution. There are some proprietary systems out there, but the appeal of open source is tough to resist. Remember. Search is a utility, not a game changer for many organizations. Good enough tramples over precision, recall, and relevance.
New Buzzwords and the Same Old Tune
Hot companies today do not pound their electric guitar with the chords in findability. Take an outfit like Palantir. It is a search and information access outfit, but the company avoids the spotlight, positions its technology packages as super stealthy magic insight machines. Palantir likes analytics, visualizations, and similar next generation streamlined tangerine colored outputs. Many of the companies profiled in my monograph Cyberosint are, at their core, search systems. But “search” is tucked into a corner, and the amplified functions like fancy math, real time processing, and smart software dominate. From my point of view, these systems are search repackaged and enhanced for today’s procurement professionals. That’s okay. But search is still search no matter what the “visionaries” suggest. Many systems are enterprise search wrapped in new sheet music. The notes are the same.
Big Data
I find the Big Data theme interesting. The idea of processing petabytes of data in a blink is future forward. The problem is that the way statistical procedures operate is to sidestep analyzing every single item. I can examine a grocery list of 10 items, but I struggle when presented with a real time updating of that list with trillions of data points a second. The reality of Big Data is that it has been around. A monk faced with copying two books in a couple of days has an intractable Big Data problem. The love of Hadoop and its extended family of data management tools does not bring the black sheep of the information family into the party room. Big Data requires pesky folks who have degrees in statistics or who have spent their youth absorbed in Mathematica, MatLab, SPSS, or SAS. Bummer. Enterprise search systems can choke on modest data. Big Data kills some systems dead like a wasp sprayed with Raid.
Real Time
For a client in the UK, I had to dig into the notion of real time. Guess what the goslings found. There was not one type of real time information system. I believe there were seven distinct types of real time information. Each type has separate cost and complexity challenges. The most expensive systems were the ones charged with processing financial transactions in milliseconds. Real time for a Web site might mean anything from every 10 second or every week or so. Real time is tough because no matter what technologies are used to speed up computer activities, the dark shadow of latency falls. When data arrive which are malformed, the real time system returns incomplete outputs. Yikes. Incomplete? Yep, missing info. Real time is easy to say, but tough to deliver at a price an average Fortune 1000 company can afford or one of the essential US or UK government agencies can afford. Speed means lots of money. Enterprise search systems usually struggle with the real time thing.
Automatic, Smart Indexing, Outputs, Whatever
I know the artificial intelligence, cognitive approach to information is a mini megatrend. Unfortunately when folks look closely at these systems, there remains a need for slug like humans to maintain dictionaries, inspect outputs and tune algorithms, and “add value” when a pesky third party cooks up a new term, phrase, or code. Talk about smart software does not implement useful smart software. The idea is as appealing today as it was when Fulcrum in Ottawa pitched its indexing approach or when iPhrase talked about its smart system. I am okay with talk as long as the speakers acknowledge perpetual and include in the budget the humans who have to keep these motion Rube Goldberg confections on point. Humans are not very good indexers. Automated indexing systems are not very good indexers. The idea is, of course, that good enough is good enough. Sorry. Work remains for the programmers. The marketers just talk about the magic of smart systems. Licensees expect the systems to work, which is an annoying characteristic of some licensees and users.
Wrap Up
Poor enterprise search. Relegated to utility status. Wrapped up in marketing salami. Celebrated by marketers who want to binge watch Parks and Recreation.
Enterprise search. You are still around, just demoted. The future? Good enough. Invest in hyper marketing and seek markets which do not have a firm grasp of search and retrieval. Soldier on. There are many streaming videos to watch if you hit the right combination on the digital slot machine.
Stephen E Arnold, December 6. 2015
Enterprise Search: A Confused Stew
November 29, 2015
Every culture has a stew. A stew is a mélange of ingredients; for example, tripe, brains, chicken fat, water, rutabaga, etc. Your parental beacon probably used an off-the-shelf product and added grocery store goodies. Heat and serve. Yummy.
I read two articles which make vivid the sad state of enterprise search. I think the experts who cooked up these write ups are doing the best with the ingredients in the fridge. I am not sure about the palatability of the meals the items presage.
Get the Right Search Tool
First, I read “Choosing an Enterprise Search Tool.” Okay, tool, not utility. The word tool does not deliver the calories I need. Let’s move forward. I learned that search has these important ingredients:
- Text analytics functions, including “entity co reference resolution”. Got it.
- Pipeline architecture, which means no reprogramming. Love that if it is indeed Grade A chuck.
That’s it. The write up wants me to be sure the solution is scalable. Okay. No problem in cloud land unless there are some pesky contractual and security requirements to keep the system close at hand and in conformance with rules, regulations, and laws.
The notion of security is good. I am all for a secure enterprise search tool. The problem is that security is a slippery fish. Toss it into this mix with the analytics and the pipeline. It will be just fine, a mantra crock pot manufacturers use in their advertisements.
Go for the open standards. None of that Kobe beef technology.
Then the write up enjoins me to do what I think is a pretty tough task in the overheated kitchen under the gaze of the chief financial officer:
you need to make sure you first know what type of information your users will need to find.
I don’t know what information I need until I encounter a problem which I cannot resolve with what I know, have in my archive, or a colleague with hopefully a clue. Where did I put that venison bone? Right, I don’t have a venison bone, and I don’t know anyone who has one. Is there a difference between a cow bone and a deer bone? Help, I need a food centric search system without Yelp and TripAdvisor filters.
I am not sure about this recipe.
Do the Enterprise Search Engine Optimization Thing
The second article I read was “Kickstart Your Enterprise Search Program.” The approach I am urged to follow involves getting on the road to better search. Happy users are important.
To reach this goal:
You need to undertake your own Enterprise SEO, or ESEO. I first wrote about ESEO in 2009, and it’s as relevant today as it was then — and suffers the same lack of tools even now. However, there are methodologies you can use.
How do I do that? It’s as easy as microwaving a burrito. Just look at the search terms the users employ. In my experience, those search terms often mislead when users are hunting for their own documents or tackling a topic about which the user lacks information and vocabulary.
One I know the terms, the rest of the enterprise search task is even easier. I don’t have to puncture the cellophane wrap after microwaving the El Monterey wonder.
Observations
Stew and microwaved SEOs. Enterprise search requires more substantive fare than these two write ups deliver. Little wonder that enterprise search vendors are struggling to find purchase in a business environment looking for Red Bull solutions.
My thought: Do not rely on either of these chefs’ suggestions. Analytics and not precision and recall. SEO and not substantive nutrition? Not for me.
Stephen E Arnold, November 29, 2015
Life Is Perceived As Faster Says Science
November 20, 2015
I read a spider friendly, link baitable article in a UK newspaper. You love these folks because each page view downloads lots and lots of code, ads, trackers, etc.
The story was “Can’t Believe It’s Almost Christmas? Technology Is Speeding Up Our Perception of Time, Researchers Say.” Heck of a title in my opinion.
The main point is captured in this quote from Wizard McLoughlin:
long monologue from a ‘real’ book.
‘It’s almost as though we’re trying to emulate the technology and be speedier and more efficient. It seems like there’s something about technology itself that primes us to increase that pacemaker inside of us that measures the passing of time.”
The “it” I assume means the way the modern world works.
I think the idea is valid. A good example is the behavior of search and content processing companies. Although many companies evidence the behaviors I want to identity, these quirks are most evident among the search and content processing outfits which have ingested tens of millions in venture funding.
The time pressure comes from the thought process like this statement which I recall from my reading of Samuel Johnson:
Nothing focuses the mind like a hanging.
The search and processing vendors under the most pressure appear to be taking the following actions. These comments apply to Attivio, BA Insight ,Coveo, and Lucidworks-type companies. The craziness of IBM Watson and HP Autonomy in the cloud may have other protein triggers.
Here we go:
- Big data. How can outfits which struggle to update indexes and process new and changed content handle Big Data? It is just trendy to call a company like Vivisimo a Big Data firm than try to explain that key word search has “real” value.
- Customer support. I don’t know about you, but I avoid customer support. Customer support means stupid telephone selections, dorky music, reminders that the call is being monitored for “quality purposes”, and other cost cutting, don’t-bother-us approaches. Where search and content processing fits in has little to do with customer service and everything about cost reduction.
- Analytics. Yep, indexing systems can output a list of the number of times a word appears in a document, a batch, or a time period. These items can be counted and converted to a graph. But I do not think that enterprise search systems are analytics systems. Again. If it helps close a deal, go with it.
- Business intelligence. I like this one. The idea that a person can look for the name of a person, place, or thing provides intelligence is laughable. I also get a kick out selective dissemination functions or standing queries presented as a magical window on real time data. Baloney. Intelligence is not a variation of search and content processing. Search and content processing are utility functions within a larger more comprehensive systems. Check out NetReveal and let me know how close an enterprise search vendor comes to this BAE Systems’ service.
When will enterprise search and content processing vendors alter their marketing?
Not until their stakeholders are able to sell these outfits and move on to less crazy investments.
The craziness will persist because the time available to hit their numbers is dwindling. Fiddling with mobile devices and getting distracted by shiny bits just makes the silliness more likely.
Have you purchased a gift using Watson’s app? Have you added a Watson recipe to your holiday menu? Have you used a metasearch system like Vivisimo to solve your Big Data problems? Have you embraced Solr as a way to make Hadoop data repositories cornucopias of wisdom?
Right. The stuff may not work as one hopes. Time is running out. Quickly in real time and in imagined time.
Stephen E Arnold, November 20, 2015
Lexmark: Analysis Suggests More Challenges Ahead
November 15, 2015
I read a wonky financial review of Lexmark. You know Lexmark. It was a unit of IBM which manufactured or assembled printers and sold ink. Ink is almost expensive as movie popcorn.
The write up is “Sunday Update for Lexmark international Incorporated.” If you are fortunate to have an unfiltered Internet connection, you may be able to view the write up. If not, buzz your financial advisor. These folks work on Sundays.
The main point of the write up is that the data in this chart provide a glimpse of the challenges Lexmark, owner of the ISYS Search Software, face. ISYS was crafted in the late 1980s, and it provides the basics of information access. The problem is that open source options and baked in search solutions make search and retrieval a utility, not a key feature.
Here’s the chart, which suggests that the August 2015 dip may be the new normal for the company.
Does this remind you of a profile of Table Mountain? I see some parallels. Sharp drop, big job to get back on top.
More interesting is that the summary of analysts expectations is that only two analysts have an opinion. One rates Lexmark as a hold. The other? Strong sell.
The write up points out that Lexmark is in the buzzword business; for example:
The Company operates in the office printing and imaging, enterprise content management (ECM), business process management (BPM), document output management (DOM), intelligent data capture and search software markets. Lexmark’s products include laser printers and multifunction devices, dot matrix printers and the associated supplies/solutions/services, as well as ECM, BPM, DOM, intelligent data capture, search and Web-based document imaging and workflow software solutions and services.
Maybe acronyms are not selling? I hope they do. Lexmark is based about one hour from the hollow in which I reside in rural Kentucky. A healthy Lexmark keeps the trailers in my trailer park rented. Perceptive, right?
Stephen E Arnold, November 15, 2015
Mindbreeze Balloon Ride: Search Marketing Reaches New Heights
November 14, 2015
I am not sure if this balloon ride is sponsored by Fabasoft Mindbreeze. I am not sure why an enterprise search vendor would market information access with a balloon. Even Google, in its fascinating Googley world, uses balloons as a mechanism for delivering high speed internet access to those lucky enough to live in Sri Lanka and other urban conclaves.
The write I read is titled “Mindbreeze: Ballonfahrt für 5 Personen zu gewinnen.” I think this means something like, “You can win a balloon ride for five people in Salzkammergut.” A blog post captures some of the thrill of the blog post.
If you are interested in enterprise search, you might want to check out Mindbreeze’s solution. If you are, like me, happier sitting in a chair contemplating the craziness of enterprise search vendors, you will skip the balloon ride.
This idea ranks with Northern Light’s sponsorship of an Indianapolis 500 race car. How many sales did that produce? If the balloon ride is unrelated to the search Mindbreeze, perhaps Mindbreeze’s attorneys will want to check out the use of the Mindbreeze name because it is blowing in the wind.
Stephen E Arnold, November 14, 2015
-
- Subscribe to Beyond Search
Feature archive
News archive
-
