CyberOSINT banner

Enterprise Search Vendors: A Partial List

June 24, 2016

I spoke with a confused and unbudgeted worker bee at a giant outfit this weekend. The stellar professional was involved in figuring out what to do about enterprise search. The story is one I have heard many times in the last 40 years. The system doesn’t meet the needs of the users. The system is over budget. The system does not index in real time. Yadda yadda yadda.

The big question was, “What are the enterprise search vendors offering a system which actually works, does not experience downtime, cost overruns, and user outrage. Note that this is not the word “outage.” The word is “outrage”.

I don’t know of such a system. As a helpful 72 year old, I rattled off a list of vendors who purport to offer Big Data capable, next generation semantic-linguistic-NLP systems. True to form, I repeated the list twice. I thought he would cry.

For those of you who want to know the vendors I plucked from my list of outfits in the search and content processing game, I reproduce the list. If you want upsides, downsides, license fees, gotchas, and other assorted details, I will provide the information. But since you are not likely to buy me dinner this evening, you will have to pay for my thoughts.

Here’s the selected list. Reader, start your browser:

  • Attivio
  • Coveo
  • dtSearch
  • Elasticsearch (Lucene)
  • Fabasoft Mindbreeze
  • IBM Omnifind
  • IHS Goldfire
  • Lookeen
  • Lucid Works (Solr)
  • Marklogic
  • Maxxcat
  • Polyspot
  • Sinequa
  • Solcara
  • Squiz Funnelback
  • Thunderstone
  • X1
  • Yippy

There are quite a few outfits whose systems do search like Palantir, but I trimmed the list to companies for my worried pal.

What’s interesting is that most of these outfits explain that their systems are much, much more than search and retrieval. Believe it or not as Mr. Ripley used to say.

Factoid: Most of these outfits have been around for quite a few years. Only Elasticsearch has managed to become a “brand” in the search space. What happened to Autonomy, Convera, Endeca, Fast Search & Transfer, and Verity since I wrote the first three editions of the Enterprise Search Report between 2003 and 2007? Ugly for some.

Search is a tough problem and has yet to deliver what users expect. Remember Google killed its search appliance. Ads are a better business because they spell money for Alphabet.

Stephen E Arnold, June 24, 2016

Microsoft Makes Fresh Effort to Position Bing

June 21, 2016

Microsoft is gearing up for a fresh challenge to Google,  with a Bing rebranding effort centered on the new “Bing Network.” This marks a different approach to leveraging the MS search platform, we learn from the piece, “Microsoft Rebrands Bing, Challenges Google”  at SearchMarketingDaily. The incorporation of Yahoo has a lot to do with it. Reporter Laurie Sullivan writes:

“Microsoft’s message says the network pulls together in-the-moment data from across its mobile, global and local partners to support products that people use daily. And that network continues to grow. With the transition of all U.S. accounts, people and account management from Yahoo to Bing, the network represents an expanding set of partnerships such as AOL, and The Wall Street Journal, which adds more searches and clicks to the network daily, wrote Stephen Sirich, GM of advertising and consumer monetization group at Microsoft, in a post.”

Sullivan later reminds us:

“The shift in brand strategy also marks an end to the Yahoo-Bing Network. The renegotiated search deal between Microsoft and Yahoo in April 2015, five years into the 10-year deal, has ad sales and account management returning to their respective companies.”

The article discusses reasons Microsoft has struggled so to position Bing as an alternative to Google. For example, says one professional, Bing should not have tried to change the model Google had set up, and users had grown accustomed to, for Internet search. Also, Bing’s brand recognition has always lagged behind that of Google.  Perhaps that is about to change with this renewed effort. See the article for some more background and stats on Bing’s performance.



Cynthia Murrell, June 21, 2016

Sponsored by, publisher of the CyberOSINT monograph

Google Springboard: The Shadow of Verity and Yahoo

June 17, 2016

I read “Google Launches Springboard Enterprise Search Tool, Revamps Sites.” Ah, thoughts flowed when I learned that a Google customer can search Google Docs and Google Drive soon.

Anyone remember Verity? What about Yahoo semantic search? No. The wizard influencing both of these outstanding services (well, outstanding might be too strong a word) is a person who has cast a shadow over information access for many years. I recall one great idea floated about 25 years ago. Users of the Verity system would pay for each cell of structured data a query “touched.” Yep, taxi meter pricing. Another great thought was offered when the individual told a BearStearns’ professional and me in no uncertain terms that Yahoo’s semantic search methods were better than Ramanathan Guha’s. Okay. Good assertion. Where are those thoughts now? Yes, searching Google services for one’s own content. Einstein, you have been aced.

The answer is Google Springboard. Yep, a service from the Alphabet Google thing which allows a Google services user to — hand on to your information access hat, gentle reader — to “will help them search more easily through and find information from Gmail, Docs, Calendar, Drive, Groups and other applications.”

I know that Alphabet Google is doing a bang up job in many technical disciplines. There is the “solving death” thing. There are the Loon balloons. There are self driving cars with sticky hoods. Oh, so much innovation.

The notion that the search function in Gmail will be extended to the goodies tucked into other Google cloud services is a bold move. For an advertising company, the shadow of Verity and Yahoo falls over precision and recall at Google.

Oh, wait. Google has not yet solved death. When the new service becomes available, perhaps finding an item in calendar or in a Google Doc will become a reality. Innovation never rests at the Alphabet Google thing.

Stephen E Arnold, June 18, 2016

Public Opinion of Dark Web May Match Media Coverage

June 17, 2016

A new survey about the Dark Web was released recently. Wired published an article centered around the research, called Dark Web’s Got a Bad Rep: 7 in 10 People Want It Shut Down, Study Shows. Canada’s Center for International Governance Innovation surveyed 24,000 people in 24 countries about their opinion of the Dark Web. The majority of respondents, 71 percent across all countries and 72 percent of Americans, said they believed the “dark net” should be shut down. The article states,

“CIGI’s Jardine argues that recent media coverage, focusing on law enforcement takedowns of child porn sites and bitcoin drug markets like the Silk Road, haven’t improved public perception of the dark web. But he also points out that an immediate aversion to crimes like child abuse overrides mentions of how the dark web’s anonymity also has human rights applications. ‘There’s a knee-jerk reaction. You hear things about crime and its being used for that purpose, and you say, ‘let’s get rid of it,’’ Jardine says.”

We certainly can attest to the media coverage zoning in on the criminal connections with the Dark Web. We cast a wide net tracking what has been published in regards to the darknet but many stories, especially those in mainstream sources emphasize cybercrime. Don’t journalists have something to gain from also publishing features revealing the aspects the Dark Web that benefit investigation and circumvent censorship?


Megan Feil, June 17, 2016

Sponsored by, publisher of the CyberOSINT monograph

Behind the Google Search Algorithm

June 16, 2016

Trying to reveal the secrets behind Google’s search algorithm is almost harder than breaking into Fort Knox.  Google keeps the 200 ranking factors a secret, what we do know is that keywords do not play the same role that they used to and social media does play some sort of undisclosed factor.  Search Engine Journal shares that “Google Released The Top 3 ranking Factors” that offers a little information to help SEO.

Google Search Quality Senior Strategist Andrey Lipattsev shared that the three factors are links, content, and RankBrain-in no particular order.  RankBrain is an artificial intelligence system that relies on machine learning to help Google process search results to push the more relevant search results to the top of the list.  SEO experts are trying to figure out how this will affect their jobs, but the article shares that:

“We’ve known for a long time that content and links matter, though the importance of links has come into question in recent years. For most SEOs, this should not change anything about their day-to-day strategies. It does give us another piece of the ranking factor puzzle and provides content marketers with more ammo to defend their practice and push for growth.”

In reality, there is not much difference, except that few will be able to explain how artificial intelligence ranks particular sites. Nifty play, Google.


Whitney Grace, June 15, 2016
Sponsored by, publisher of the CyberOSINT monograph


SLI Systems Hopeful as Losses Narrow and Revenue Grows

June 14, 2016

The article titled SLI Systems Narrows First-Half Loss on Scoop reports revenue growth and plans to mitigate losses. SLI Systems is a New Zealand-based software as a service (SaaS) business that provides cloud-based search resources to online retailers. Founded in 2001, SLI Systems has already weathered a great deal of storms in the form of the dot-com crash that threatened to stall the core technology (developed at GlobalBrain.) According to a statement from the company, last year’s loss of $502K was an improvement from the loss of $4.1M in 2014. The article states,

“SLI shares have dropped 18 percent in the past 12 months, to trade recently at 76 cents, about half the level of the 2013 initial public offering price of $1.50. The software developer missed its sales forecast for the second half of the 2015 year but is optimistic new chief executive Chris Brennan and Martin Onofrio as chief revenue officer, both Silicon Valley veterans, can drive growth in revenue and earnings.”

The SLI of SLI stands for Search, Learn and (appropriately) Improve. The company hopes to achieve sustainable growth without raising additional capital by continuing to focus on innovation and customer retention rates, which slipped from 90% to 87% recently. Major clients include Lenovo, David Jones, Harvey Norman, and Paul Smith.



Chelsea Kerwin, June 14, 2016

Sponsored by, publisher of the CyberOSINT monograph

Websites Found to Be Blocking Tor Traffic

June 8, 2016

Discrimination or wise precaution? Perhaps both? MakeUseOf tells us, “This Is Why Tor Users Are Being Blocked by Major Websites.” A recent study (PDF) by the University of Cambridge; University of California, Berkeley; University College London; and International Computer Science Institute, Berkeley confirms that many sites are actively blocking users who approach through a known Tor exit node. Writer Philip Bates explains:

“Users are finding that they’re faced with a substandard service from some websites, CAPTCHAs and other such nuisances from others, and in further cases, are denied access completely. The researchers argue that this: ‘Degraded service [results in Tor users] effectively being relegated to the role of second-class citizens on the Internet.’ Two good examples of prejudice hosting and content delivery firms are CloudFlare and Akamai — the latter of which either blocks Tor users or, in the case of, infinitely redirects. CloudFlare, meanwhile, presents CAPTCHA to prove the user isn’t a malicious bot. It identifies large amounts of traffic from an exit node, then assigns a score to an IP address that determines whether the server has a good or bad reputation. This means that innocent users are treated the same way as those with negative intentions, just because they happen to use the same exit node.”

The article goes on to discuss legitimate reasons users might want the privacy Tor provides, as well as reasons companies feel they must protect their Websites from anonymous users. Bates notes that there  is not much one can do about such measures. He does point to Tor’s own Don’t Block Me project, which is working to convince sites to stop blocking people just for using Tor. It is also developing a list of best practices that concerned sites can follow, instead. One site, GameFAQs, has reportedly lifted its block, and CloudFlare may be considering a similar move. Will the momentum build, or must those who protect their online privacy resign themselves to being treated with suspicion?


Cynthia Murrell, June 8, 2016

Sponsored by, publisher of the CyberOSINT monograph

Google Voice Search: An Optimistic View

June 7, 2016

I read “Google Voice Search Records and Keeps Conversations People Haver Around their Phones but You Can Delete the Files.” I like the “you can delete the files. How does one know what has or has not been deleted in this era of real time cloud goodness?

I assume that the information in the write up is accurate.

The write up states:

The feature works as a way of letting people search with their voice, and storing those recordings presumably lets Google improve its language recognition tools as well as the results that it gives to people.

If you want to “delete” these recordings, the write up asserts:

It’s found by heading to Google’s history page and looking at the long list of recordings. The company has a specific audio page and another for activity on the web, which will show you everywhere Google has a record of you being on the internet.

Optimism is good. One presidential hopeful believed certain emails had been deleted. I am not sure that the FSB agrees. It seems that the Independent’s “real journalist” was not aware of “Your Data Is Forever.”

Stephen E Arnold, June 7, 2016

Alibaba Eyes Twiggle, Not the US Ecommerce Search Players

June 7, 2016

I read “Alibaba, the Chinese and Global Powerhouse, Is Investing in Twiggle, a Stealthy Israeli Ecommerce Search Start Up.” I love the stealthy part. Twiggle cannot be too stealthy because Alibaba found out about the system.

I learned in the write up:

Twiggle was funded back in 2013, and its founders were formerly leaders at Google, namely Dr. Amir Konigsberg, previously one of the members of Google’s operations in the emerging markets and former managing director of, and Dr. Adi Avidor, a former engineering tech lead at Google. Combined, the two have authored more than 35 U.S. patents and bring a wealth of experience in digital innovation in the fields of search, artificial intelligence and ecommerce.

The system delivers search and discovery. Crunchbase points out that the company has ingested about $20 million since it opened its doors in 2014.

I assume that the heartbeats of EasyAsk and SLI Systems sped up when word of Alibaba’s search proclivities diffused. With this alleged tie up, I assume the heart rates of the principals at EasyAsk and SLI Systems, assorted open source firms, and probably the every ready IBM Watson have slowed.

Stephen E Arnold, June 7, 2016

Yippy for Vivisimo

June 6, 2016

I read “Yippy Buys MC+A, a Veteran Google Search Appliance Partner.” Yippy, if memory serves, is a variant of Vivisimo. In the good old days, before Vivisimo sold to IBM and suddenly became a Big Data company, Yippy did search and retrieval. I assume the old document limits were lifted. I also assume that the wild and crazy config file editing has been streamlined. I also assume that Yippy is confident it can zoom past Maxxcat and Thunderstone, two outfits also in the search appliance business. Buying a Google reseller provides some insight and maybe leads into which companies embraced the GSA solution. When the GSA was first demonstrated to me, I noted the locked down relevance system. There were other interesting “enhancements” the Googlers included to eliminate the complexity of enterprise search. I recall working on the training materials for a DC reseller of the GSA. Customization was like a Google interview question.

The Fortune write up is one of those reinventions of enterprise search which I enjoy. I circled this comment:

Google Search Appliance was a great idea for companies that deploy a welter of different applications, so important data can be scattered about in different file systems and repositories. It also gave Google a toehold in corporate server rooms, which is why some wondered why Google would cut the product at a time when it’s trying to sell more cloud services to these very companies.

I was unaware that Alphabet Google was in the search business. I thought it was an online advertising outfit. Who at Google wanted to work on the wonky GSA products? For years Google relied on resellers and outfits like Dell to make the over priced gizmos.

Love live Vivisimo. I mean Yippy. If you cannot pin down an integrator, why not buy one?

Stephen E Arnold, June 6, 2016

Next Page »