CyberOSINT banner

Google Search Appliance: Like Glass It Broke

February 8, 2016

I read “So Long Google Search Appliance.” Farewell, happy yellow and blue boxes. So long integrators who have been supporting these wildebeests for a decade. Au revoir easy-as-pie search.

According to the write up:

The tech giant told its reseller and consulting partners the news via email on Thursday, noting that they can continue to sell one-year license renewals for existing hardware customers through 2017, but that they will be unable to sell new hardware. Renewals will end in 2018.

I recall writing about the Google Search Appliance when I was reporting about enterprise search for specialist publications. I was the first or one of the first to run down the pricing for the wonky boxes. I pointed out that a redundant multi million document system would ring the Google cash register in the high six figures with seven figures not out of sight. I thought I mentioned that the number of engineeers supporting the GSA had dwindled to a couple of folks. I thought I pointed out that the assumption a Web search system would work like a champ on corporate content was a wild and crazy notion.l

Like so many others who assumed enterprise search was not a tough problem, the Alphabet Google thing has bailed. Google essentially failed to revolutionize enterprise search. Cheaper and more usable appliances are available, including products from Maxxcat and Thunderstone. There are reasonable cloud solutions. And there is a cornucopia of outfits offering repackaged open source systems. Heck, if one pokes around long enough, a bold enterprise can license a system from companies with proprietary information access systems; 3RDi, Fabasoft, Lexmark, etc.

What will organizations do without the Google Search Appliance? Yard sale, Goodwill?

Stephen E Arnold, February 8, 2016

3RDi for Enterprise Search

February 5, 2016

Health and medical search need an upgrade? T/DG 3RDi might be just what the doctor ordered. You search blues will disappear when you have natural language processing, semantic search, search relevancy, search analytics, research tools, and data integration. Very comprehensive it seems.

T/DG offers 3RDi. Now try to search for these entities. To locate the services firm offering the 3RDi system, one has to figure out how to make Bing, Google, and Yandex point to the correct entities.

Naming products and companies is tricky. Let me save you the hassle of wading through false drops.

  • T/DG means “The Digital Group,” an outfit founded in 1999 and operating from New Jersey.
  • 3RDi means “relevant, deep insights.” (I don’t know what the 3 means.)

The search system appears to be a “platform” based on open source technology. Here’s a block diagram of 3RDi:


Source: The Digital Group, 2015

The company’s most recent push is health care. The search system performs the type of functions which I associate with a system like the ones Autonomy and Fast Search & Transfer described in the late 1990s. There is also a hefty dose of “platformitis.” The idea is that a licensee can use the system to meet the needs of users. The support for controlled vocabularies is helpful in domain specific deployments, but these have to be maintained, which can be a financial and resource burden for some licensees.

3RDi embraces the semantic marketing jargon enthusiastically; for example, this diagram shows how “knowledge” and “semantics” make the “experience” work for licensees:


Source: The Digital Group, 2015

Users of the system do not have to deal with results lists. The system presents information in a visual manner; for example:


Source: The Digital Group, 2015

In short, 3RDi appears to deliver the type of utility I associate with systems from outfits like BAE Systems and Palantir.

If your organization wants an open source system with the bells and whistles found in seven figure platforms, you may want to explore 3RDi.

The urls you need are:

I assume that the company will make the “3” clearer going forward. There is a live demo available. You will need to register. The system balks at non commercial domains like my Yahoo account.

The recent marketing push given 3RDi signals that the enterprise search sector is alive and well. As the company says, “Start experiencing.” I wonder what the “3” means.

Stephen E Arnold, February 5, 2016

Proprietary Enterprise Search: False Hopes and Brutal Costs

December 21, 2015

At lunch the other day, the goslings and I engaged in what I thought was a routine discussion: The sad state of the enterprise search market.

I pointed out that the “Enterprise Search Daily” set up by Edwin Stauthamer was almost exclusively a compilation of Big Data articles. Enterprise search, although the title of the daily, was not the focal point of the content.


Enterprise search is a cost black hole. R&D, support, customization, and bug fixes gorge on money and engineers. Instead of adding value to an enterprise system, search becomes the reason the CFO has a migraine and why sales professionals struggle to close deals.

I said, “Enterprise search has disappeared.”

One of the goslings asked, “What’s happened to the proprietary search systems acquired by some big companies?”

We were off an running.

The goslings mentioned that Dassault Systèmes bought Exalead and the brand has disappeared from the US market. IBM bought Vivisimo, and the purchase was explained as a Big Data buy, but the company and its technology have disappeared into the Great Blue Hole, which is today’s IBM. Hummingbird bought Fulcrum, and then OpenText bought Hummingbird. Open Text owns Information Dimension’s BASIS, BRS Search, and its own home brew search system. Oracle snapped up Endeca, InQuira, and RightNow in a barrage of search binge shopping. Lexmark—formerly a unit of Big Blue—bought ISYS Search Software and Brainware. Then there was the famous purchase of Fast Search & Transfer by Microsoft and the subsequent police investigation and the charges filed against a former executive for fancy dancing with the revenue numbers. And who can forget the $11 billion purchase of Autonomy by IBM. There have been other deals, and the goslings enjoyed commenting on this.

I called a halt to the lunch time stand up comedy routine. The executives of these companies were trying to do what they thought was best for their [a] financial future and [b] for their stakeholders. Some of these stakeholders had suffered through revenue droughts and were looking for a way out of the sea of red ink enterprise search vendors generate with aplomb.

The point I raised was, “Does the purchase of a proprietary enterprise search system?” make a substantive contribution to the financial health of the purchasing company.

Read more

Canada Based Coveo Predicts the Future of Enterprise Search

December 17, 2015

Coveo, an enterprise search vendor once closely focused on Microsoft SharePoint, offered some predictions about 2016 and search.

What does the Québec City based search vendor presage? The answer appears in “Coveo Releases 2016 Enterprise Search Predictions.” Here you go:

  • Machine learning. IBM will definitely support this prediction or their public relations advisors will.
  • Cybersecurity. Yep. This is a hot area. Some of the companies pushing new boundaries in search are outfits like Diffeo, Tiberian, and Recorded Future (backed by the Alphabet Google outfit). Key word search does not sleep in the same king sized bed as these outfits in my opinion.
  • Real time engagement and learning for digital transformation. I thought of Yahoo. The company has several thousand people working on search. How is that working out for the Xoogler running Yahoo?
  • Proactive recommendations. Okay, this is the selective dissemination or SDI thing. SDIs are useful but these cannot be annoying. Does anyone remember Desktop Data? I do.
  • Search makes systems more intelligent. Well, maybe. search is a utility. The more successful of the intelligence automation outfits uses subcomponents of search; for example, entity extraction, metadata generated on the fly, and relationship maps. These are, in my opinion, more important than old school search.

The write up includes a remarkable quote. I have placed this gem in my folder for future reference. Here is the statement I highlighted in bold red marker ink:

2016 will be a pivotal year for enterprise search. Organizations now recognize the strategic value of using intelligent search-based applications to drive more customer and employee engagement.

Based on the information available to me as we complete The Dark Web Dilemma, 2016 is going to be a bit different from 2015 when venture money flowed like water to outfits in the content processing game. My thought about the future is that companies which have ingested oodles of cash have to generate revenue growth, demonstrate a path to profitability, or face a Convera, Delphes, Entopia, or Siderean Software type future.

What’s that type of future?

A sell out or a shut down. My hunch is that the stakeholders in the Coveo play are looking for a buyer just like Lexmark.

Stephen E Arnold, December 17, 2015

Big Data and Enterprise Search: The Caution Lights Are Flashing

December 15, 2015

I read “How You Should Explain Big Data to Your CEO.” The write up included a link which triggered thoughts of how enterprise search dug itself a hole and climbed. Unable to extricate itself from a problem enterprise search vendors created, the entire sector has been marginalized. In some circles, enterprise search is essentially a joke. “Did you hear about the three enterprise search vendors who walked into a bar?” The bartender says, “What is this? Some kind of joke?”

The link pointed me to a Slideshare (owned by the email and job hunting champion LinkedIn). That presentation, “5 Signs Your Big Data Project is Doomed to Fail,” could have been borrowed from one of my talks about enterprise search in 2001. It was not, but the basic message was identical: Big Data has created a situation in which there are some challenges here and now.

The presentation was prepared by Qubole (maybe pronounced cue ball?). Qubole is a click to query outfit. This means that reports from Big Data are easy to generate.

Here are the problems Big Data faces:

  • Failed implementations. Qubole asserts that 87 percent of the Big Data implementations are flops
  • 73 percent of executive describe the Big Data project as flop
  • 45 percent of Big Data projects are completed

These data are similar to the results of “satisfaction” with enterprise search solutions.

Why? Qubole asserts:

  • Inaccurate project scope
  • Inadequate management support
  • No business case
  • Lack of talent (in search the talent may be present but overestimates its ability to deal with enterprise search technologies and processes)
  • “Challenging tools.” I think this means that in the Big Data world there are lots of complexities.

What can one charged with either search or Big Data tasks do with this information?

My view is, “Ignore it.”

The “can do” spirit carries professionals forward. Hiring a consultant provides some job protection but does little to reverse the failure and disappointment rate.

My view is that the willingness of executives to grab at a magic solution presented by a showman marketer overrides failure date. The arrogance of those involved create a “that won’t happen to us” belief.

Who is to blame? The company for believing in baloney? The marketer for painting a picture and showing a Hollywood style demo? The developers who created the Big Data solution, knowing that chunks were not complete or just did not work before the ship date? The in house engineers who lacked self knowledge to understand their own limitations?

Everyone is in the hole with the enterprise software vendors. The hole is deep. Magic solutions are difficult to pin down. The future of Big Data is likely to parallel to some degree the dismal track record of enterprise search. Fascinating. I can hear the mid tier consultants and the handful of remaining enterprise search vendors asserting that Qubole’s points are not applicable to their specific situation.

Yep, and I believe in the tooth fairy and Santa.

Stephen E Arnold, December 15, 2015

Paper.Li Enterprise Search Daily Not Updating

December 11, 2015

Update December 12, 2015–The Enterprise Search is updating again. The content is primarily about Big Data.

The Enterprise Search Daily, which shifted from enterprise search to Big Data, is not updating. Is it a problem or a lack of suitable information about search?

The archives are available from this link. If the page resolves, click on the Archives and you can view a Search Daily by clicking on a calendar. My hunch is that the flow of information about enterprise search slowed in 2015. Marketers have embraced Big Data even if their systems do not really handle lots of information without developing indigestion. Earlier this year, I heard that a real publisher was going to introduce a newsletter about enterprise search. I am not sure if anyone will notice.

If the Enterprise Search Daily turns up in my Overflight system again, I will note the date. For now, fans of enterprise search will have to entertain themselves by running queries on the topics related to search. Big Data is not a useful search phrase. Everybody does Big Data according to the marketers and the venture firms funding search and retrieval. What about the conference organizers who run for fee shindigs to discuss enterprise search? Yikes. I am delighted to be beyond search.

Stephen E Arnold, December 11, 2015

Enterprise Software Survey: ZDNet Content Marketing

December 8, 2015

Research: Enterprise Software Rising with 69 Percent Usage” caught my attention. I like numbers which suggest two thirds adoption of software which seems ubiquitous to me. So the write looks like news. It sounds like news. Sort of news I think.

The write up is a marketing pitch for a new study by ZDNet and its for fee research arm Tech Pro Research. “Enterprise Software: Advantages, Opportunities, Challenges” studies enterprise software. I am not exactly sure what “enterprise” means in the ZDNet world. My hunch is that the term embraces commercial enterprises which buy products from the ZDNet advertisers. I could be wrong, of course, because enterprise may mean, the local dry launderette entrepreneur.

The article discloses some interesting factoids, which I assume ZDNet and a host of enterprise software sales people believe are solid gold. None of the gold foam stuff.

I noted these points:

Enterprise software has a definition I found surprising. I learned:

Enterprise software encompasses a variety of functions, including asset management, business intelligence, CRM utilities, data processing, databases, financial applications, identity management, retail software, process management and resource planning. It can run on either individual machines or on centralized servers, whether in-house, located in the cloud or a hybrid combination.

No, enterprise search, no analytics, no Big Data, and no enterprise cyber security. Yikes. The mid tier consultants are probably perspiring in their home offices due to these omissions.

I found this factoid interesting: nine percent of the sample do not currently use an enterprise software solution but are considering one in the next year and 22 percent do not use enterprise software and are not planning on changing their stripes next year. That works out to 31 percent of the sample not doing the enterprise software thing even though, I assume, the companies in the sample were enterprises. I said, “Huh. Imagine that.”

Even more puzzling was the list of enterprise software deployed in the last 12 months by the sample.

  • Human resources
  • Storage
  • Databases, note the plural
  • Big Data
  • Mobile

Now the paragraph I quote about the functions of enterprise software did not include Big Data. What’s the scoop, ZDNet. Is Big Data an enterprise application or not? My view is that Big Data is for the folks who understand analytic-type stuff. Most enterprises have precious few of these types of people. Another fascinating point.

The preferred vendors identified in the write up came at me from left field. I admit I am not in the know like the real journalists at ZDNet. Here’s that listing:

  • Adobe Systems
  • Dropbox
  • Google
  • LinkedIn (my goodness)
  • Microsoft

But the un-preferred vendors is more intriguing. This group includes:

  • CA Technologies
  • Oracle
  • Red Hat
  • SAP

I assume that each of these companies will be really thrilled to meet with the ZDNet ad sales professionals.

If you want more, you will be able to explore the opportunities by diving into the for fee version of the study.

Stephen E Arnold, December 7, 2015

Has Enterprise Search Drowned in a Data Lake?

December 6, 2015

I had a phone conversation with a person who unluckily resides in a suburb of New York City. Apparently the long commute allows my contact to think big thoughts and formulate even bigger questions. He asked me, “What’s going to happen to enterprise search?”

I thought this was a C minus questions, but New Yorkers only formulate A plus questions. I changed the subject to the new Chick Fil-A on Sixth. After the call, I jotted down some thoughts about enterprise search.

Here for your contemplation are five of my three comments which consumed three legal pad sheets. I also write small.

Enterprise Search Is Week Old Celery

In the late 1990s when the Verity hype machine was rolling and the Alphabet boys were formulating big thoughts about search, enterprise search was the hot ticket. For some techno cravers, enterprise search was the Alpha and Omega. If information is digital, finding an item of information was the thrill ride ending in a fluffy pile of money. A few folks made some money, but the majority of the outfits jumping into search either sold out or ended up turning off the lights. Today, enterprise search is a utility and the best approach is to use an open source solution. There are some proprietary systems out there, but the appeal of open source is tough to resist. Remember. Search is a utility, not a game changer for many organizations. Good enough tramples over precision, recall, and relevance.

New Buzzwords and the Same Old Tune

Hot companies today do not pound their electric guitar with the chords in findability. Take an outfit like Palantir. It is a search and information access outfit, but the company avoids the spotlight, positions its technology packages as super stealthy magic insight machines. Palantir likes analytics, visualizations, and similar next generation streamlined tangerine colored outputs. Many of the companies profiled in my monograph Cyberosint are, at their core, search systems. But “search” is tucked into a corner, and the amplified functions like fancy math, real time processing, and smart software dominate. From my point of view, these systems are search repackaged and enhanced for today’s procurement professionals. That’s okay. But search is still search no matter what the “visionaries” suggest. Many systems are enterprise search wrapped in new sheet music. The notes are the same.

Big Data

I find the Big Data theme interesting. The idea of processing petabytes of data in a blink is future forward. The problem is that the way statistical procedures operate is to sidestep analyzing every single item. I can examine a grocery list of 10 items, but I struggle when presented with a real time updating of that list with trillions of data points a second. The reality of Big Data is that it has been around. A monk faced with copying two books in a couple of days has an intractable Big Data problem. The love of Hadoop and its extended family of data management tools does not bring the black sheep of the information family into the party room. Big Data requires pesky folks who have degrees in statistics or who have spent their youth absorbed in Mathematica, MatLab, SPSS, or SAS. Bummer. Enterprise search systems can choke on modest data. Big Data kills some systems dead like a wasp sprayed with Raid.

Real Time

For a client in the UK, I had to dig into the notion of real time. Guess what the goslings found. There was not one type of real time information system. I believe there were seven distinct types of real time information. Each type has separate cost and complexity challenges. The most expensive systems were the ones charged with processing financial transactions in milliseconds. Real time for a Web site might mean anything from every 10 second or every week or so. Real time is tough because no matter what technologies are used to speed up computer activities, the dark shadow of latency falls. When data arrive which are malformed, the real time system returns incomplete outputs. Yikes. Incomplete? Yep, missing info. Real time is easy to say, but tough to deliver at a price an average Fortune 1000 company can afford or one of the essential US or UK government agencies can afford. Speed means lots of money. Enterprise search systems usually struggle with the real time thing.

Automatic, Smart Indexing, Outputs, Whatever

I know the artificial intelligence, cognitive approach to information is a mini megatrend. Unfortunately when folks look closely at these systems, there remains a need for slug like humans to maintain dictionaries, inspect outputs and tune algorithms, and “add value” when a pesky third party cooks up a new term, phrase, or code. Talk about smart software does  not implement useful smart software. The idea is as appealing today as it was when Fulcrum in Ottawa pitched its indexing approach or when iPhrase talked about its smart system. I am okay with talk as long as the speakers acknowledge perpetual and include in the budget the humans who have to keep these motion Rube Goldberg confections on point. Humans are not very good indexers. Automated indexing systems are not very good indexers. The idea is, of course, that good enough is good enough. Sorry. Work remains for the programmers. The marketers just talk about the magic of smart systems. Licensees expect the systems to work, which is an annoying characteristic of some licensees and users.

Wrap Up

Poor enterprise search. Relegated to utility status. Wrapped up in marketing salami. Celebrated by marketers who want to binge watch Parks and Recreation.

Enterprise search. You are still around, just demoted. The future? Good enough. Invest in hyper marketing and seek markets which do not have a firm grasp of search and retrieval. Soldier on. There are many streaming videos to watch if you hit the right combination on the digital slot machine.

Stephen E Arnold, December 6. 2015

Enterprise Search: A Confused Stew

November 29, 2015

Every culture has a stew. A stew is a mélange of ingredients; for example, tripe, brains, chicken fat, water, rutabaga, etc. Your parental beacon probably used an off-the-shelf product and added grocery store goodies. Heat and serve. Yummy.

I read two articles which make vivid the sad state of enterprise search. I think the experts who cooked up these write ups are doing the best with the ingredients in the fridge. I am not sure about the palatability of the meals the items presage.

Get the Right Search Tool

First, I read “Choosing an Enterprise Search Tool.” Okay, tool, not utility. The word tool does not deliver the calories I need. Let’s move forward. I learned that search has these important ingredients:

  • Text analytics functions, including “entity co reference resolution”. Got it.
  • Pipeline architecture, which means no reprogramming. Love that if it is indeed Grade A chuck.

That’s it. The write up wants me to be sure the solution is scalable. Okay. No problem in cloud land unless there are some pesky contractual and security requirements to keep the system close at hand and in conformance with rules, regulations, and laws.

The notion of security is good. I am all for a secure enterprise search tool. The problem is that security is a slippery fish. Toss it into this mix with the analytics and the pipeline. It will be just fine, a mantra crock pot manufacturers use in their advertisements.

Go for the open standards. None of that Kobe beef technology.

Then the write up enjoins me to do what I think is a pretty tough task in the overheated kitchen under the gaze of the chief financial officer:

you need to make sure you first know what type of information your users will need to find.

I don’t know what information I need until I encounter a problem which I cannot resolve with what I know, have in my archive, or a colleague with hopefully a clue. Where did I put that venison bone? Right, I don’t have a venison bone, and I don’t know anyone who has one. Is there a difference between a cow bone and a deer bone? Help, I need a food centric search system without Yelp and TripAdvisor filters.

I am not sure about this recipe.

Do the Enterprise Search Engine Optimization Thing

The second article I read was “Kickstart Your Enterprise Search Program.” The approach I am urged to follow involves getting on the road to better search. Happy users are important.

To reach this goal:

You need to undertake your own Enterprise SEO, or ESEO. I first wrote about ESEO in 2009, and it’s as relevant today as it was then — and suffers the same lack of tools even now. However, there are methodologies you can use.

How do I do that? It’s as easy as microwaving a burrito. Just look at the search terms the users employ. In my experience, those search terms often mislead when users are hunting for their own documents or tackling a topic about which the user lacks information and vocabulary.

One I know the terms, the rest of the enterprise search task is even easier. I don’t have to puncture the cellophane wrap after microwaving the El Monterey wonder.


Stew and microwaved SEOs. Enterprise search requires more substantive fare than these two write ups deliver. Little wonder that enterprise search vendors are struggling to find purchase in a business environment looking for Red Bull solutions.

My thought: Do not rely on either of these chefs’ suggestions. Analytics and not precision and recall. SEO and not substantive nutrition? Not for me.

Stephen E Arnold, November 29, 2015

Life Is Perceived As Faster Says Science

November 20, 2015

I read a spider friendly, link baitable article in a UK newspaper. You love these folks because each page view downloads lots and lots of code, ads, trackers, etc.

The story was “Can’t Believe It’s Almost Christmas? Technology Is Speeding Up Our Perception of Time, Researchers Say.” Heck of a title in my opinion.

The main point is captured in this quote from Wizard McLoughlin:

long monologue from a ‘real’ book.

‘It’s almost as though we’re trying to emulate the technology and be speedier and more efficient. It seems like there’s something about technology itself that primes us to increase that pacemaker inside of us that measures the passing of time.”

The “it” I assume means the way the modern world works.

I think the idea is valid. A good example is the behavior of search and content processing companies. Although many companies evidence the behaviors I want to identity, these quirks are most evident among the search and content processing outfits which have ingested tens of millions in venture funding.

The time pressure comes from the thought process like this statement which I recall from my reading of Samuel Johnson:

Nothing focuses the mind like a hanging.

The search and processing vendors under the most pressure appear to be taking the following actions. These comments apply to Attivio, BA Insight ,Coveo, and Lucidworks-type companies. The craziness of IBM Watson and HP Autonomy in the cloud may have other protein triggers.

Here we go:

  • Big data. How can outfits which struggle to update indexes and process new and changed content handle Big Data? It is just trendy to call a company like Vivisimo a Big Data firm than try to explain that key word search has “real” value.
  • Customer support. I don’t know about you, but I avoid customer support. Customer support means stupid telephone selections, dorky music, reminders that the call is being monitored for “quality purposes”, and other cost cutting, don’t-bother-us approaches. Where search and content processing fits in has little to do with customer service and everything about cost reduction.
  • Analytics. Yep, indexing systems can output a list of the number of times a word appears in a document, a batch, or a time period. These items can be counted and converted to a graph. But I do not think that enterprise search systems are analytics systems. Again. If it helps close a deal, go with it.
  • Business intelligence. I like this one. The idea that a person can look for the name of a person, place, or thing provides intelligence is laughable. I also get a kick out selective dissemination functions or standing queries presented as a magical window on real time data. Baloney. Intelligence is not a variation of search and content processing. Search and content processing are utility functions within a larger more comprehensive systems. Check out NetReveal and let me know how close an enterprise search vendor comes to this BAE Systems’ service.

When will enterprise search and content processing vendors alter their marketing?

Not until their stakeholders are able to sell these outfits and move on to less crazy investments.

The craziness will persist because the time available to hit their numbers is dwindling. Fiddling with mobile devices and getting distracted by shiny bits just makes the silliness more likely.

Have you purchased a gift using Watson’s app? Have you added a Watson recipe to your holiday menu? Have you used a metasearch system like Vivisimo to solve your Big Data problems? Have you embraced Solr as a way to make Hadoop data repositories cornucopias of wisdom?

Right. The stuff may not work as one hopes. Time is running out. Quickly in real time and in imagined time.

Stephen E Arnold, November 20, 2015

Next Page »