Search without Indexing
April 27, 2016
I read “Outsmarting Google Search: Making Fuzzy Search Fast and Easy Without Indexing.”
Here’s a passage I highlighted:
It’s clear the “Google way” of indexing data to enable fuzzy search isn’t always the best way. It’s also clear that limiting the fuzzy search to an edit distance of two won’t give you the answers you need or the most comprehensive view of your data. To get real-time fuzzy searches that return all relevant results you must use a data analytics platform that is not constrained by the underlying sequential processing architectures that make up software parallelism. The key is hardware parallelism, not software parallelism, made possible by the hybrid FPGA/x86 compute engine at the heart of the Ryft ONE.
I also circled:
By combining massively parallel FPGA processing with an x86-powered Linux front-end, 48 TB of storage, a library of algorithmic components and open APIs in a small 1U device, Ryft has created the first easy-to-use appliance to accelerate fuzzy search to match exact search speeds without indexing.
An outfit called InsideBigData published “Ryft Makes Real-time Fuzzy Search a Reality.” Alas, that link is now dead.
Perhaps a real time fuzzy search will reveal the quickly deleted content?
Sounds promising. How does one retrieve information within videos, audio streams, and images? How does one hook together or link a reference to an entity (discovered without controlled term lists) with a phone number?
My hunch is that the methods disclosed in the article have promise, the future of search seems to be lurching toward applications that solve real world, real time problems. Ryft may be heading in that direction in a search climate which presents formidable headwinds.
Stephen E Arnold, April 27, 2016
Duck Duck Go as a Privacy Conscious Google Alternative
April 26, 2016
Those frustrated with Google may have an alternative. Going over to the duck side: A week with Duck Duck Go from Search Engine Watch shares a thorough first-hand account of using Duck Duck Go for a week. User privacy protection seems to be the hallmark of the search service and there is even an option to enable Tor in its mobile app. Features are comparable, such as one designed to compete with Google’s Knowledge Graph called Instant Answers. As an open source product, Instant Answers is built up by community contributions. As far as seamless, intuitive search, the post concludes,
“The question is, am I indignant enough about Google’s knowledge of my browsing habits (and everyone else’s that feed its all-knowing algorithms) to trade the convenience of instantly finding what I’m after for that extra measure of privacy online? My assessment of DuckDuckGo after spending a week in the pond is that it’s a search engine for the long term. To get the most out of using it, you have to make a conscious change in your online habits, rather than just expecting to switch one search engine for another and get the same results.”
Will a majority of users replace “Googling” with “Ducking” anytime soon? Time will tell, and it will be an interesting saga to see unfold. I suppose we could track the evolution on Knowledge Graph and Instant Answers to see the competing narratives unfold.
Megan Feil, April 26, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Watson Lacks Conversation Skills and He Is Not Evil
April 22, 2016
When I was in New York last year, I was walking on the west side when I noticed several other pedestrians moving out of the way of a man mumbling to himself. Doing as the natives do, I moved aside and heard the man rumble about how, “The robots are taking over and soon they will be ruling us. You all are idiots for not listening to me.” Fear of a robot apocalypse has been constant since computer technology gained precedence and we also can thank science-fiction for perpetuating it. Tech Insider says in “Watson Can’t Actually Talk To You Like In The Commercials” Elon Musk, Bill Gates, Stephen Hawking, and other tech leaders have voiced their concerns about creating artificial intelligence that is so advanced it can turn evil.
IBM wants people to believe otherwise, which explains their recent PR campaign with commercials that depict Watson carrying on conversations with people. The idea is that people will think AI are friendly, here to augment our jobs, and overall help us. There is some deception on IBM’s part, however. Watson cannot actually carry on a conversation with a person. People can communicate with, usually via an UI like a program via a desktop or tablet. Also there is more than one Watson, each is programmed for different functions like diagnosing diseases or cooking.
“So remember next time you see Watson carrying on a conversation on TV that it’s not as human-like as it seems…Humor is a great way to connect with a much broader audience and engage on a personal level to demystify the technology,’ Ann Rubin, Vice President IBM Content and Global Creative, wrote in an email about the commercials. ‘The reality is that these technologies are being used in our daily lives to help people.’”
If artificial intelligence does become advanced enough that it is capable of thought and reason comparable to a human, it is worrisome. It might require that certain laws be put into place to maintain control over the artificial “life.” That day is a long time off, however, until then embrace robots helping to improve life.
Whitney Grace, April 22, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Local News Station Produces Dark Web Story
April 22, 2016
The Dark Web continues to emerge as a subject of media interest for growing audiences. An article, Dark Web Makes Illegal Drug, Gun Purchases Hard To Trace from Chicago CBS also appears to have been shared as a news segment recently. Offering some light education on the topic, the story explains the anonymity possible for criminal activity using the Dark Web and Bitcoin. The post describes how these tools are typically used,
“Within seconds of exploring the deep web we found over 15,000 sales for drugs including heroin, cocaine and marijuana. In addition to the drugs we found fake Illinois drivers licenses, credit card and bank information and dangerous weapons. “We have what looks to be an assault rifle, AK 47,” said Petefish. That assault rifle AK 47 was selling for 10 bitcoin which would be about $4,000. You can buy bitcoins at bitcoin ATM machines using cash, leaving very little trace of your identity. Bitcoin currency along with the anonymity and encryption used on the dark web makes it harder for authorities to catch criminals, but not impossible.”
As expected, this piece touches on the infamous Silk Road case along with some nearby cases involving local police. While the Dark Web and cybercrime has been on our radar for quite some time, it appears mainstream media interest around the topic is slowly growing. Perhaps those with risk to be affected, such as businesses, government and law enforcement agencies will also continue catching on to the issues surrounding the Dark Web.
Megan Feil, April 22, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Google Removes Pirate Links
April 21, 2016
A few weeks ago, YouTube was abuzz with discontent from some of its most popular YouTube stars. Their channels had been shut down die to copyright claims by third parties, even thought the content in question fell under the Fair Use defense. YouTube is not the only one who has to deal with copyright claims. TorrentFreak reports that “Google Asked To Remove 100,000 ‘Pirate Links’ Every Hour.”
Google handles on average two million DMCA takedown notices from copyright holders about pirated content. TorrentFreak discovered that the number has doubled since 2015 and quadrupled since 2014. The amount beats down to one hundred thousand per hour. If the rate continues it will deal with one billion DMCA notices this year, while it had previously taken a decade to reach this number.
“While not all takedown requests are accurate, the majority of the reported links are. As a result many popular pirate sites are now less visible in Google’s search results, since Google downranks sites for which it receives a high number of takedown requests. In a submission to the Intellectual Property Enforcement Coordinator a few months ago Google stated that the continued removal surge doesn’t influence its takedown speeds.”
Google does not take broad sweeping actions, such as removing entire domain names from search indexes, as it does not want to become a censorship board. The copyright holders, though, are angry and want Google to promote only legal services over the hundreds of thousands of Web sites that pop up with illegal content. The battle is compared to an endless whack-a-mole game.
Pirated content does harm the economy, but the numbers are far less than how the huge copyright holders claim. The smaller people who launch DMCA takedowns, they are hurt more. YouTube stars, on the other hand, are the butt of an unfunny joke and it would be wise for rules to be revised.
Whitney Grace, April 21, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Digging for a Direction of Alphabet Google
April 21, 2016
Is Google trying to emulate BAE System‘s NetReveal, IBM i2, and systems from Palantir? Looking back at an older article from Search Engine Watch, How the Semantic Web Changes Everything for Search may provide insight. Then, Knowledge Graph had launched, and along with it came a wave of communications generating buzz about a new era of search moving from string-based queries to a semantic approach, organizing by “things”. The write-up explains,
“The cornerstone of any march to a semantic future is the organization of data and in recent years Google has worked hard in the acquisition space to help ensure that they have both the structure and the data in place to begin creating “entities”. In buying Wavii, a natural language processing business, and Waze, a business with reams of data on local traffic and by plugging into the CIA World Factbook, Freebase and Wikipedia and other information sources, Google has begun delivering in-search info on people, places and things.”
This article mentioned Knowledge Graph’s implication for Google to deliver strengthened and more relevant advertising with this semantic approach. Even today, we see the Alphabet Google thing continuing to shift from search to other interesting information access functions in order to sell ads.
Megan Feil, April 21, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Lessons to Learn from Instagram Translation Systems
April 20, 2016
Social media services attempt to eliminate the publishing of pornographic content on their sites through a combination of user reporting and algorithms. However, Daily Star reports Shock as one million explicit porn films found on Instagram. This content existed on Instagram despite their non-nudity policy. However, according to the article, much of the pornographic videos and photos were removed after news broke. Summarizing how the content was initially published, the article states,
“The videos were unearthed by tech blogger Jed Ismael, who says he’s discovered over one million porn films on the site. Speaking on his blog, Ismael said: “Instagram has banned certain English explicit hashtags from being showed in search. “Yet users seem to find a way around the policy, by using non English terms or hashtags. “I came across this discovery by searching for the hashtag “?????” which means movies in Arabic.” Daily Star Online has performed our own search and easily found hardcore footage without the need for age verification checks.”
While Tor has typically been seen as the home for such services, it appears some users have found a workaround. Who needs the Dark Web? As for those online translation systems, perhaps some services should consider their utility.
Megan Feil, April 20, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Data on Dark Web Not Excused from Fact or Fiction Debate
April 19, 2016
Remember when user information was leaked from the extramarital affairs website AshleyMadison? While the leak caused many controversies, the release of this information specifically on the Dark Web gives reason to revisit an article from Mashable, Another blow for Ashley Madison: User emails leaked on Dark Web as a refresher on the role Tor played. A 10-gigabyte file was posted as a Torrent on the Dark Web which included emails and credit card information among other user data. The article concluded,
“With the data now out there, Internet users are downloading and sifting through it for anything – or, rather, anyone – of note. Lists of email addresses of AshleyMadison users are being circulated on social media. Several appear to be connected to members of the UK government but are likely fake. As Wired notes, the site doesn’t require email verification, meaning the emails could be fake or even hijacked.”
The future of data breaches and leaks may be unclear, but the falsification of information — leaked or otherwise — always remains a possibility. Regardless of the element of scandal existing in future leaks, it is important to note that hackers and other groups are likely not above manipulation of information.
Megan Feil, April 19, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Google Is Many Things: Racial Bias in Hair May Be Brushed Aside
April 18, 2016
I read a story about matching up user queries with images. I don’t think Google’s image search is particularly good. Examples range from Google’s obsession with taking a query like “truth” and returning images of pictures with the word “truth” in them. And this image:
What about the query for “watson.” Google showed a picture of a computer, a person named “sherlock,” and images of this guy:
The write up “Do Google’s ‘Unprofessional Hair’ Results Show It Is Racist?” wants to point out that Google’s methods have a nasty side. I noted this passage:
We’ve always conceived of search engines as arcane but neutral creatures, obedient only to our will and to the precious logic of information. Older engines from the advent of the internet reflected this: Remember “Ask Jeeves,” the genteel butler? Dogpile, which would “fetch” things for you? Despite this fantasy, the things engines and their algorithms are able to know and to find are influenced by the content we give them to work with, which means they may reflect our own biases.
AskJeeves was a human powered system. The Google is algorithmic. Google does not “give” its image search system content. The image search system indexes what it finds, within the depth settings for the crawl. Sorry, gentle reader, Google does not index everything available via the Internet. Bummer, right?
I circled this statement:
is its [image search’s] purpose to reflect and reinforce what its users feel, do and believe? Or is it to show us a fuller picture of the world and all things contained in it as they really are? Google Images was conceived in response to what people most wanted to see. Maybe it hasn’t decided yet what we most need to see.
The Guardian itself is an interesting legal search. Run the query “guardian” on Google Images and what does one find? Here you go:
The logo of the “real” journalistic thing and the word “truth.” Now is that biased?
Stephen E Arnold, April 18, 2016
Mindbreeze Breaks into Slovak Big Data Market Through Partnership with Medialife
April 18, 2016
The article titled Mindbreeze and MEDIALIFE Launch Strategic Partnership on BusinessWire discusses what the merger means for the Slovak and Czech Republic enterprise search market. MediaLife emphasizes its concentrated approach to document management systems for Slovak customers in need of large systems for the management, processing, and storage of documents. The article details,
“Based on this partnership, we provide our customers innovative solutions for fast access to corporate data, filtering of relevant information, data extraction and their use in automated sorting (classification)… Powerful enterprise search systems for businesses must recognize relationships among different types of information and be able to link them accordingly. Mindbreeze InSpire Appliance is easy to use, has a high scalability and shows the user only the information which he or she is authorized to view.”
Daniel Fallmann, founder and CEO of Mindbreeze, complimented himself on his selection of a partner in MediaLife and licked his chops at the prospect of the new Eastern European client base opened to Mindbreeze through the partnership. Other Mindbreeze partners exist in Italy, the UK, Germany, Mexico, Canada, and the USA, as the company advances its mission to supply enterprise search appliances as well as big data and knowledge management technologies.
Chelsea Kerwin, April 18, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
-
- Subscribe to Beyond Search
Feature archive
News archive
-
