CyberOSINT banner

A Dark Search Engine

February 27, 2015

If anyone mentions the dark Web or the invisible Web, most people would make a Star Wars reference and insert a Darth Vader quote. While getting in touch with your “dark side” can help even out your personality, searching the dark Web reveals a whole new world of information. The only problem is that there isn’t a strict search engine for it. Wired explains that “Darpa Is Developing A Search Engine For the Dark Web.” Darpa is creating a dark Web search engine to help law enforcement discover patterns and relationships in online data about illegal activities.

“The project, dubbed Memex, has been in the works for a year and is being developed by 17 different contractor teams who are working with the military’s Defense Advanced Research Projects Agency. Google and Bing, with search results influenced by popularity and ranking, are only able to capture approximately five percent of the internet. The goal of Memex is to build a better map of more internet content.”

The search engine’s main goal is to have a one-size-fits all approach to search results. The data will not only be pulled from the same places commercial search engines crawl, but also the dark Web hidden sites that include TOR network’s Hidden Services. The Memex team also want to automate methods to analyze the data to save law enforcement research time.

Memex is only a tool for uncovering the dark Web, how it is used depends on the organization. It is estimated Memex will cost between $10-20 million to fund.

Whitney Grace, February 27, 2015
Sponsored by, developer of Augmentext

Interface Changes for Google Mobile Search

February 26, 2015

Google is the top search US search engine for many reasons and it can maintain this title because the company is constantly searching (pun unintended) to improve its products and services. Google wants to deliver high quality search just as much as it wants to stay ahead of its competition. Mobile search is one of the most competitive digital markets and Google has developed ways to augment its already popular mobile application. BGR highlights the new changes to its mobile search as described in “Google’s Latest Mobile Search Change Brings Some Key Interface Changes.”

One feature that changes is the “Google box” that displays results that are supposed to be the best matches for a query. The Google box will also have a news carousel that lists the latest information on the query.

“ ‘When you search for a topic, just scroll down to see a ‘carousel’ of recent articles, videos or more on that subject,’ Google Search product manager Ardan Arac wrote in a blog post. Tap any link to read or watch exactly what you’re interested in. For example, if you search for NPR, you’ll see links to all their latest articles and videos.”

Google is doing its best to improve mobile search, a task that has usually evaded mobile devices. Mobile technology needs to have more features that are readily available on laptops and computers to make them more reliable and useful.

Whitney Grace, February 26, 2015
Sponsored by, developer of Augmentext

Google Search: Live Chat, But Relevance?

February 25, 2015

i read “Google Tests Live Chat With Businesses From Search Results.” According to the write up:

Google is testing out a service that incorporates live chat with businesses right into search results, via a new link that shows whether a business is currently available, and immediately launches a chat via Google Hangouts (on either desktop or mobile) if they are.

I have been doing the online research thing for years. Ellen Shedlarz, formerly Booz, Allen New York’s head information guru, exposed me to commercial online systems in 1973 or so. She was kind enough to let me fumble away with a dorky dumb terminal with bunny rabbit ears.

In the last 40 years, I have to make a confession about my stupidity. I never wanted to enter into a live chat with a person who wrote an article, offered a product, or pretended to be an expert like a mid tier search expert with a degree in English.

I wanted to perform what I naively thought was research. I would obtain information, either print out information or copy it on the 5×8 inch note cards my debate coach in high school mandated I use for research, thus forming a life long habit. I would then read the information I gathered, make notes, and prepare more note cards with identifiers that allowed silly old me to find the connections among the note cards.

After I knew what the heck I was learning, formulating my questions, and then thinking about whom I could approach for more information—then I wanted to talk to a human with alleged expertise.

No wonder I am a loser. When I enter a query for “terminal”, I want to enter a category code so I get the exact meaning of terminal I have in mind and information directly related to documents with that notion of terminal. When I want terminal for a train, I want train stuff.

I suppose now I can run a query for terminal and see these “relevant” results:


Perhaps I can ring the director of the motion picture and ask the fellow where the train station is. Seems very useful, just not to me. Google, how about a return to relevance?

Stephen E Arnold, February 25, 2015

WCC Smart Search and Match Partners with Diona Mobility

February 23, 2015

We learn of a recent deal from PR Newswire’s post, “Government Employment Services Brought into the 21st Century Through WCC and Diona Partnership.” The deal will bring WCC’s search-and-match technology to Diona Mobility’s human-services solutions. The write-up tells us:

“The partnership will integrate WCC solutions with Diona Mobility solutions to provide clients in the employment market with unique options when using mobile devices to find sustainable and appropriate jobs. WCC and Diona will deliver leading-edge solutions to help clients and their caseworkers efficiently find jobs that match the clients’ skills. Clients and caseworkers can locate timely employment positions through their smartphones and tablets while on the go.

“The mobile solution pioneered by WCC and Diona will provide jobseekers with:

*Real-time notifications of available matching jobs;

*The ability to manage their profile and skills, and search for jobs on the fly;

*Benchmarking, analytics and insight into career opportunities; and

*Access to enrollment and referral services for courses and training programs.”

WCC’s CEO Peter Went emphasizes his company’s experience with some of the biggest staffing firms and public employment services around the world, and praises Diona’s focus on quality and customer access. Launched in 1996, WCC is headquartered in Utrecht, The Netherlands, and has locations in the U.S., Serbia, and Saudi Arabia. Founded in 2012 and based in Ireland, Diona maintains several offices around the world. They aim to make social services globally accessible through mobile platforms by 2020.

Cynthia Murrell, February 23, 2015

Sponsored by, developer of Augmentext

Kelsen Enters Legal Search Field

February 23, 2015

A new natural-language search platform out of Berlin, Kelsen, delivers software-as-a-service to law firms. Basic Thinking discusses “The Wolfram Alpha of the Legal Industry.” Writer Jürgen Kroder interviewed Kelsen co-founder Veronica Pratzka. She explains what makes her company’s search service different (quote auto-translated from the original German):

“Kelsen is generated based on pre-existing legal cases not a search engine, but a self-learning algorithm that automatically answers. 70-80 percent of the global online data are very unstructured. Search engines look for keywords and only. Google has many answers, but you have to look for them yourself thousands of search results together and hope that you just entered the correct keywords. Kelsen, however, is rather a free online lawyer who understands natural language practitioner trained in all areas of law, works 24/7 and is always up-to-date….

“First Kelsen understands natural language compared to Google! That is, even with the entry of long sentences and questions, not just keywords, Kelsen is suitable answers. Moreover, Kelsen searches ‘only’ relevant legal data sources and provides the user with a choice of right answers ready, he can also evaluate.’
“One could easily Kelsen effusive as ‘the Wolfram Alpha the legal industry,’ respectively. We focus on Kelsen with legal data structure and analyze them in order to eventually make available. From this structuring and visualization of legal data not only seeking advice and lawyers can benefit, but also legislators, courts and research institutions.”

Pratzka notes that her company received boosts from both the Microsoft Accelerator and the IBM Entrepreneur startup support programs. Kelsen expects to turn a profit on the business-to-consumer side through premium memberships. In business-to-business, though, the company plans to excel by simply outperforming the competition. Pratzka seems very confident. Will the service garner the attention she and her team expect?

Cynthia Murrell, February 23, 2015

Sponsored by, developer of Augmentext

Partition the Web to Manage It

February 22, 2015

I noted that the mid February 2015 Forbes article did not get much coverage. “US Defense Giant Raytheon: We Need To Divide The Web To Secure It” contains a suggestion that could, if implemented, force changes upon Bing, Google, and other Web indexing outfits.

Here’s the passage I highlighted in lovely ice blue:

But some, including Michael Daly, chief technology officer for cyber security at US defense giant Raytheon, believe that the web needs to be divided into communities. As more critical devices, from insulin pumps to cars, connect to the internet, the more likely a genuinely destructive digital attack will occur. To stop this from happening, some people just shouldn’t be allowed into certain corners of the web, according to Daly.

There are some interesting implications in this notion.

Stephen E Arnold, February 22, 2015

Antidot Semi-Pivot to eCommerce Search

February 21, 2015

I wanted to capture Antidot’s semi pivot from enterprise search to eCommerce search. The French company provides a useful description of its afs@store product. If you bang this product name into the GOOG, you find that the American Foundry Society, Associated Food Stores, and the American Fisheries Society push Antidot’s product down the results list. In general, names of search and content processing systems often disappear into search results. Perhaps Antidot has a way to make the use of the “@” sign somewhat less problematic.

The system, according to Antidot, system delivers features that sidestep the unsticky nature of most eCommerce customer visits. Antidot asserts:

  • Rich, tolerant and customizable auto complete featuring products, brands, categories…
  • Fully typo-tolerant search
  • Semantic search that understands your customer’s words
  • Dynamic filtering facets to rapidly select desired products
  • Web interface to simply monitor and manage your searchandising

the company offers a plug in for Magento, the open source eCommerce system, that enjoyed love from eBay. It is difficult to know if that love is growing stronger with time, however.

I did notice that the “See and read more” panel had zero information and no links. Hopefully this void will be addressed.

Stephen E Arnold, February 21, 2015

Simplifying Google and Microsoft: Okay, Just a Wild Swing

February 20, 2015

I know the feeling. A deadline looms and the “real writer” casts about for a trope, an angle, a hook on which to hang a story. I read “Microsoft Is the New Google, Google Is the Old Microsoft.” The write up is a stuffed with product names and MBAisms. Here’s a passage I noted:

Meanwhile Microsoft makes the point that it is still thinking big with arguably the most interesting moonshot program in all of tech right now: Windows Holographic. Holographic is an eye to the future to excite consumers and investors while the company still remains laser focused on the present.

Prior to 2006, Google wanted to squish Microsoft. After 2006, Google began to show signs of progeroid syndrome. The problems had more to do with management issues than flaws in the company’s engineering. By 2010, engineering showed signs of reduced blood flow to the brain of Google. The manifestation of these problems were evident in the reorganizations and the drift from the company’s push to capitalize on research computing harvested for ideas that could be integrated into the firm’s information factory. The problems Google faces are rooted in management and engineering. The visible effects are some wild and crazy decisions about products, what the company can do to deal with the non Google world, and the realization that the business model inspired by GoTo, Overture, and Yahoo was getting long in the tooth.

Microsoft, on the other hand, had its own set of problems. These ranged from bureaucratic hardening of the arteries to really bad engineering. Toss in the shift from the desktop monoculture to a more diverse ecosystem. Microsoft became a value stock and uninteresting to all but the most devoted resellers, Windows lovers, and corporate information technology gurus certified by Microsoft. After much thrashing, the company moved from Gates Ballmer to a manager less inclined to chase his tail without snagging it.

Net net: Both companies have challenges, but there firms have not swapped outfits like twins in a slapstick comedy. Both companies are making decisions in an effort to maintain their revenue and profitability. It is unclear how each company will deal with the challenges in enterprise markets, consumer markets, non US markets, and management processes.

Forbes wants to paint a simple word picture for these two large and deeply stressed organizations. My focus is search. So consider that utility function. Neither company delivers high value findability for its constituents. When two firms whiff at bat, one must look beyond the appearance of failure to identify the root cause. Why has Google search gone off the rails? What’s up with the Bing thing? Is a failure with a utility function a manisfestation of more substantive issues?

For now, considered analyses of the weakness of Google and Microsoft is what I want to find in “real journalism”, not generalizations about goofy products or “moon spoon” metaphors. Even I tire of referencing the Loon balloon.

Stephen E Arnold, February 20, 2015

More Open Source Search Excitement: Solr Flare Erupts

February 20, 2015

I read “Yonik Seeley, Creator of Apache Solr Search Engine Joins Cloudera.” Most personnel moves in the search and retrieval sector are ho hum events. Seely’s jump from Heliosearch to Cloudera may disrupt activities a world away from the former Lucid Imagination now chasing Big  Data under the moniker “LucidWorks.” I write the company’s name as LucidWorks (Really?) because the company has undergone some Cirque du Soleil moves since the management revolving door was installed.

Seeley was one of the founders and top engineers at Lucid. Following his own drum beat, he formed his own company to support Solr,. In my opinion, Seeley played a key role in shaping Solr  into a reasonable alternative to proprietary findability solutions like Endeca. With Seeley at Cloudera, Lucid’s vision of becoming the search solution for Hadoop-like data management systems may suffer a transmission outage. I think of this as a big Solr flare.

Cloudera will move forward and leverage Seeley’s expertise. It is possible that Lucid will move out of the Big Data orbit and find a way to generate sustainable revenues. However, Cloudera now has an opportunity to add some fuel to its solutions.

For me, the Seeley move is good news for Cloudera. For Lucid, Seeley’s joining Cloudera is yet another challenge to Lucid. I think the Lucid operation is still dazed after four or five years sharp blows to the corporate body.

The patience of Lucid’s investors may be tested again. The management issues, the loss of a key executive to Amazon, the rise of Elasticsearch, and now the Seeley shift in orbit—these are the times that may try the souls of those who expect a payoff from their investments in Lucid’s open source dream. Cloudera or Elasticsearch are now companies with a fighting chance to become the next RedHat. Really.

Stephen E Arnold, February 20, 2015

Need to Find a Simpsons’ Quote?

February 19, 2015

I often seek the wisdom of Homer, Homer Simpson that is. To locate a specific quote from the television series, enter a string like “doh” and you get a list of hits. Give it a whirl at Findability marches on. D’oh.

Stephen E Arnold, February 19, 2015

« Previous PageNext Page »