CyberOSINT banner

Google Relies on Freebase Machine ID Numbers to Label Images in Knowledge Graph

May 3, 2016

The article on Seo by the Sea titled Image Search and Trends in Google Search Using FreeBase Entity Numbers explains the transformation occurring at Google around Freebase Machine ID numbers. Image searching is a complicated business when it comes to differentiating labels. Instead of text strings, Google’s Knowledge Graph is based in Freebase entities, which are able to uniquely evaluate images- without language. The article explains with a quote from Chuck Rosenberg,

An entity is a way to uniquely identify something in a language-independent way. In English when we encounter the word “jaguar”, it is hard to determine if it represents the animal or the car manufacturer. Entities assign a unique ID to each, removing that ambiguity, in this case “/m/0449p” for the former and “/m/012×34” for the latter.”

Metadata is wonderful stuff, isn’t it? The article concludes by crediting Barbara Starr, a co-administrator of the Lotico San Diego Semantic Web Meetup, with noticing that the Machine ID numbers assigned to Freebase entities now appear in Google Trend’s URLs. Google Trends is a public web facility that enables an exploration of the hive mind by showing what people are currently searching. The Wednesday that President Obama nominated a new Supreme Court Justice, for example, had the top search as Merrick Garland.


Chelsea Kerwin, May 3, 2016

Sponsored by, publisher of the CyberOSINT monograph

An Open Source Search Engine to Experiment With

May 1, 2016

Apache Lucene receives the most headlines when it comes to discussion about open source search software.  My RSS feed pulled up another open source search engine that shows promise in being a decent piece of software.  Open Semantic Search is free software that cane be uses for text mining, analytics, a search engine, data explorer, and other research tools.  It is based on Elasticsearch/Apache Solrs’ open source enterprise search.  It was designed with open standards and with a robust semantic search.

As with any open source search, it can be programmed with numerous features based on the user’s preference.  These include, tagging, annotation, varying file format support, multiple data sources support, data visualization, newsfeeds, automatic text recognition, faceted search, interactive filters, and more.  It has the benefit that it can be programmed for mobile platforms, metadata management, and file system monitoring.

Open Semantic Search is described as

“Research tools for easier searching, analytics, data enrichment & text mining of heterogeneous and large document sets with free software on your own computer or server.”

While its base code is derived from Apache Lucene, it takes the original product and builds something better.  Proprietary software is an expense dubbed a necessary evil if you work in a large company.  If, however, you are a programmer and have the time to develop your own search engine and analytics software, do it.  It could be even turn out better than the proprietary stuff.


Whitney Grace, May 1, 2016
Sponsored by, publisher of the CyberOSINT monograph

Search without Indexing

April 27, 2016

I read “Outsmarting Google Search: Making Fuzzy Search Fast and Easy Without Indexing.”

Here’s a passage I highlighted:

It’s clear the “Google way” of indexing data to enable fuzzy search isn’t always the best way. It’s also clear that limiting the fuzzy search to an edit distance of two won’t give you the answers you need or the most comprehensive view of your data. To get real-time fuzzy searches that return all relevant results you must use a data analytics platform that is not constrained by the underlying sequential processing architectures that make up software parallelism. The key is hardware parallelism, not software parallelism, made possible by the hybrid FPGA/x86 compute engine at the heart of the Ryft ONE.

I also circled:

By combining massively parallel FPGA processing with an x86-powered Linux front-end, 48 TB of storage, a library of algorithmic components and open APIs in a small 1U device, Ryft has created the first easy-to-use appliance to accelerate fuzzy search to match exact search speeds without indexing.

An outfit called InsideBigData published “Ryft Makes Real-time Fuzzy Search a Reality.” Alas, that link is now dead.

Perhaps a real time fuzzy search will reveal the quickly deleted content?

Sounds promising. How does one retrieve information within videos, audio streams, and images? How does one hook together or link a reference to an entity (discovered without controlled term lists) with a phone number?

My hunch is that the methods disclosed in the article have promise, the future of search seems to be lurching toward applications that solve real world, real time problems. Ryft may be heading in that direction in a search climate which presents formidable headwinds.

Stephen E Arnold, April 27, 2016

Duck Duck Go as a Privacy Conscious Google Alternative

April 26, 2016

Those frustrated with Google may have an alternative. Going over to the duck side: A week with Duck Duck Go from Search Engine Watch shares a thorough first-hand account of using Duck Duck Go for a week. User privacy protection seems to be the hallmark of the search service and there is even an option to enable Tor in its mobile app. Features are comparable, such as one designed to compete with Google’s Knowledge Graph called Instant Answers. As an open source product, Instant Answers is built up by community contributions. As far as seamless, intuitive search, the post concludes,

“The question is, am I indignant enough about Google’s knowledge of my browsing habits (and everyone else’s that feed its all-knowing algorithms) to trade the convenience of instantly finding what I’m after for that extra measure of privacy online? My assessment of DuckDuckGo after spending a week in the pond is that it’s a search engine for the long term. To get the most out of using it, you have to make a conscious change in your online habits, rather than just expecting to switch one search engine for another and get the same results.”

Will a majority of users replace “Googling” with “Ducking” anytime soon? Time will tell, and it will be an interesting saga to see unfold. I suppose we could track the evolution on Knowledge Graph and Instant Answers to see the competing narratives unfold.


Megan Feil, April 26, 2016

Sponsored by, publisher of the CyberOSINT monograph


Watson Lacks Conversation Skills and He Is Not Evil

April 22, 2016

When I was in New York last year, I was walking on the west side when I noticed several other pedestrians moving out of the way of a man mumbling to himself.  Doing as the natives do, I moved aside and heard the man rumble about how, “The robots are taking over and soon they will be ruling us.  You all are idiots for not listening to me.”  Fear of a robot apocalypse has been constant since computer technology gained precedence and we also can thank science-fiction for perpetuating it.  Tech Insider says in “Watson Can’t Actually Talk To You Like In The Commercials” Elon Musk, Bill Gates, Stephen Hawking, and other tech leaders have voiced their concerns about creating artificial intelligence that is so advanced it can turn evil.

IBM wants people to believe otherwise, which explains their recent PR campaign with commercials that depict Watson carrying on conversations with people.  The idea is that people will think AI are friendly, here to augment our jobs, and overall help us.  There is some deception on IBM’s part, however.  Watson cannot actually carry on a conversation with a person.  People can communicate with, usually via an UI like a program via a desktop or tablet.  Also there is more than one Watson, each is programmed for different functions like diagnosing diseases or cooking.

“So remember next time you see Watson carrying on a conversation on TV that it’s not as human-like as it seems…Humor is a great way to connect with a much broader audience and engage on a personal level to demystify the technology,’ Ann Rubin, Vice President IBM Content and Global Creative, wrote in an email about the commercials. ‘The reality is that these technologies are being used in our daily lives to help people.’”

If artificial intelligence does become advanced enough that it is capable of thought and reason comparable to a human, it is worrisome.  It might require that certain laws be put into place to maintain control over the artificial “life.”  That day is a long time off, however, until then embrace robots helping to improve life.


Whitney Grace, April 22, 2016
Sponsored by, publisher of the CyberOSINT monograph

Local News Station Produces Dark Web Story

April 22, 2016

The Dark Web continues to emerge as a subject of media interest for growing audiences. An article, Dark Web Makes Illegal Drug, Gun Purchases Hard To Trace from Chicago CBS also appears to have been shared as a news segment recently. Offering some light education on the topic, the story explains the anonymity possible for criminal activity using the Dark Web and Bitcoin. The post describes how these tools are typically used,

“Within seconds of exploring the deep web we found over 15,000 sales for drugs including heroin, cocaine and marijuana. In addition to the drugs we found fake Illinois drivers licenses, credit card and bank information and dangerous weapons. “We have what looks to be an assault rifle, AK 47,” said Petefish. That assault rifle AK 47 was selling for 10 bitcoin which would be about $4,000. You can buy bitcoins at bitcoin ATM machines using cash, leaving very little trace of your identity. Bitcoin currency along with the anonymity and encryption used on the dark web makes it harder for authorities to catch criminals, but not impossible.”

As expected, this piece touches on the infamous Silk Road case along with some nearby cases involving local police. While the Dark Web and cybercrime has been on our radar for quite some time, it appears mainstream media interest around the topic is slowly growing. Perhaps those with risk to be affected, such as businesses, government and law enforcement agencies will also continue catching on to the issues surrounding the Dark Web.


Megan Feil, April 22, 2016

Sponsored by, publisher of the CyberOSINT monograph


Google Removes Pirate Links

April 21, 2016

A few weeks ago, YouTube was abuzz with discontent from some of its most popular YouTube stars.  Their channels had been shut down die to copyright claims by third parties, even thought the content in question fell under the Fair Use defense.  YouTube is not the only one who has to deal with copyright claims.  TorrentFreak reports that “Google Asked To Remove 100,000 ‘Pirate Links’ Every Hour.”

Google handles on average two million DMCA takedown notices from copyright holders about pirated content.  TorrentFreak discovered that the number has doubled since 2015 and quadrupled since 2014.  The amount beats down to one hundred thousand per hour.  If the rate continues it will deal with one billion DMCA notices this year, while it had previously taken a decade to reach this number.

“While not all takedown requests are accurate, the majority of the reported links are. As a result many popular pirate sites are now less visible in Google’s search results, since Google downranks sites for which it receives a high number of takedown requests.  In a submission to the Intellectual Property Enforcement Coordinator a few months ago Google stated that the continued removal surge doesn’t influence its takedown speeds.”

Google does not take broad sweeping actions, such as removing entire domain names from search indexes, as it does not want to become a censorship board.  The copyright holders, though, are angry and want Google to promote only legal services over the hundreds of thousands of Web sites that pop up with illegal content.   The battle is compared to an endless whack-a-mole game.

Pirated content does harm the economy, but the numbers are far less than how the huge copyright holders claim.  The smaller people who launch DMCA takedowns, they are hurt more.  YouTube stars, on the other hand, are the butt of an unfunny joke and it would be wise for rules to be revised.


Whitney Grace, April 21, 2016
Sponsored by, publisher of the CyberOSINT monograph


Digging for a Direction of Alphabet Google

April 21, 2016

Is Google trying to emulate BAE System‘s NetReveal, IBM i2, and systems from Palantir? Looking back at an older article from Search Engine Watch, How the Semantic Web Changes Everything for Search may provide insight. Then, Knowledge Graph had launched, and along with it came a wave of communications generating buzz about a new era of search moving from string-based queries to a semantic approach, organizing by “things”. The write-up explains,

“The cornerstone of any march to a semantic future is the organization of data and in recent years Google has worked hard in the acquisition space to help ensure that they have both the structure and the data in place to begin creating “entities”. In buying Wavii, a natural language processing business, and Waze, a business with reams of data on local traffic and by plugging into the CIA World Factbook, Freebase and Wikipedia and other information sources, Google has begun delivering in-search info on people, places and things.”

This article mentioned Knowledge Graph’s implication for Google to deliver strengthened and more relevant advertising with this semantic approach. Even today, we see the Alphabet Google thing continuing to shift from search to other interesting information access functions in order to sell ads.


Megan Feil, April 21, 2016

Sponsored by, publisher of the CyberOSINT monograph

Lessons to Learn from Instagram Translation Systems

April 20, 2016

Social media services attempt to eliminate the publishing of pornographic content on their sites through a combination of user reporting and algorithms. However, Daily Star reports Shock as one million explicit porn films found on Instagram. This content existed on Instagram despite their non-nudity policy. However, according to the article, much of the pornographic videos and photos were removed after news broke. Summarizing how the content was initially published, the article states,

“The videos were unearthed by tech blogger Jed Ismael, who says he’s discovered over one million porn films on the site. Speaking on his blog, Ismael said: “Instagram has banned certain English explicit hashtags from being showed in search. “Yet users seem to find a way around the policy, by using non English terms or hashtags. “I came across this discovery by searching for the hashtag “?????” which means movies in Arabic.” Daily Star Online has performed our own search and easily found hardcore footage without the need for age verification checks.”

While Tor has typically been seen as the home for such services, it appears some users have found a workaround. Who needs the Dark Web? As for those online translation systems, perhaps some services should consider their utility.


Megan Feil, April 20, 2016

Sponsored by, publisher of the CyberOSINT monograph



Data on Dark Web Not Excused from Fact or Fiction Debate

April 19, 2016

Remember when user information was leaked from the extramarital affairs website AshleyMadison? While the leak caused many controversies, the release of this information specifically on the Dark Web gives reason to revisit an article from Mashable, Another blow for Ashley Madison: User emails leaked on Dark Web as a refresher on the role Tor played. A 10-gigabyte file was posted as a Torrent on the Dark Web which included emails and credit card information among other user data. The article concluded,

“With the data now out there, Internet users are downloading and sifting through it for anything – or, rather, anyone – of note. Lists of email addresses of AshleyMadison users are being circulated on social media. Several appear to be connected to members of the UK government but are likely fake. As Wired notes, the site doesn’t require email verification, meaning the emails could be fake or even hijacked.”

The future of data breaches and leaks may be unclear, but the falsification of information — leaked or otherwise — always remains a possibility. Regardless of the element of scandal existing in future leaks, it is important to note that hackers and other groups are likely not above manipulation of information.


Megan Feil, April 19, 2016

Sponsored by, publisher of the CyberOSINT monograph

« Previous PageNext Page »