DuckDuckGo: One Expert Thinks It Is Better Than Google Search
November 8, 2016
I love the stories about Google’s lousy search system. The GOOG is trying to improve search with smart software and providing more third party sponsored links in search results. In my research, I have learned that most Google users focus on getting answers to their questions. The fact that these users are asking questions which are mostly repetitive means that the GOOG can optimize for what works to handle the majority of the queries. Instrumental efficiency for the user, for Google’s network resources, and for Google’s paying customers. Some experts don’t like the direction Google is moving, powered by its data analysis.
One example is spelled out in “How I Quit Using Google Search and Saved a Lot of Time.” I noted:
Now, DDG isn’t an exact replacement for Google, but they’re close. I almost always find what I’m looking for with them [I think the “them” refers to the Google Web search system], but it [I think this means searching] can be more work. The biggest feature I miss is that you can’t specify a search period, such as the last week or year, or a date range. But only a few times in the last year have I been forced to go to Google for a difficult search.
Okay, but Google does offer Google Advanced Search and some old fashioned search box command line instructions. These are not perfect. I agree that Google has some time deficiencies. That lack of “time” functionality may be one contributing reason behind Google’s investment in Recorded Future, an analytics platform designed to perform a range of time centric functions; for example, spider the Dark Web and array events on a timeline with additional analytic reports available with a mouse click.
The write up does not share these “advanced” factoids. I highlighted this statement:
Given the advantages of a Google-free existence, I have to wonder what Google is costing the world economy. If interesting ads cause each Internet user to spend an extra five minutes a day on non-productive shopping, with almost 3 billion Internet users today, that’s 15 billion minutes or over 28,000 person years of productivity every day.
Yes, an example of what I call mid tier consultant reasoning. Make assumptions about “time”. Assign a value. Calculate the “cost.” Works every time; for example, the now cherished IDC estimate of how much time a worker spends looking for information. The idea is that a better search system unleashes value, productivity, and other types of wonderfulness. The problem is that this type of logic is often misleading because the assumptions are specious and the analysis something even a sixth grade baseball statistics fan would doubt. How about them Cubbies?
But the point of the write up is that DuckDuckGo does not traffic in human user data. There are ads, but these ads are different from Google ads. Okay. Fine.
The write up reveals three things about experts doing online search analysis:
- Ads, regardless of who shows them, pump data back to the source of the ad. The outfit may choose to ignore what works and what doesn’t at its peril. Failed ads do not generate revenue for the advertiser. Hence the advertiser will go elsewhere.
- Running queries which return on point information is time consuming and often difficult. The reasons range from the mysterious removal of information from indexes to the vagaries of human language. Do you know the exact term to use to locate malware which can be used to compromise an iPhone and the name of the vendor who sells this type of program. Give that a whirl on a single free Web search system.
- The merging of imprecise information about the “cost” of a search is not convincing. Perhaps the expert should consider the impact of the shift from desktop search to mobile device search. That change will force most free Web search systems to turn some cartwheels and find ways to generate revenue. Fancy functionality is simply not used by 97 percent of online search users. Good enough search is the new normal. Thus, search today is not what search yesterday was perceived to be.
Who cares about alternative free Web search systems? The truth is that those who long for the good old days of Google may have to wake up and check out the new dawn. Misinformation, disinformation, filtered information are the norm. No advanced search system on the planet can provide pointers to high value content, accurate content on a consistent basis.
Stephen E Arnold, November 8, 2016
Iceland Offers the First Human Search Engine
November 8, 2016
Iceland is a northern country that one does not think about much. It is cold, has a high literacy rate, and did we mention it was cold? Despite its frigid temperatures, Iceland is a beautiful country with a rich culture and friendly people. shares just how friendly the Icelanders are with their new endeavor: “Iceland Launches ‘Ask Guðmundur,’ The World’s First Human Search Engine.”
Here is what the country is doing:
The decidedly Icelandic and truly personable service will see special representatives from each of Iceland’s seven regions offer their insider knowledge to the world via Inspired By Iceland’s social media platforms (Twitter, Facebook and YouTube). Each representative shares the name Guðmundur or Guðmunda, currently one of the most popular forenames in the country with over 4,000 men and women claiming it as their own.
Visitors to the site can literally submit their questions and have them answered by an expert. Each of the seven Guðmundurs is an Icelandic regional expert. Iceland’s goal with the human search engine is to answer’s the world’s questions about the country, but to answer them in the most human way possible: with actual humans.
A human search engine is an awesome marketing campaign for Iceland. One of the best ways to encourage tourism is to introduce foreigners to the locale people and customs, the more welcoming, quirky, and interesting is all the better for Iceland. So go ahead, ask Guðmundur.
Whitney Grace, November 8, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
The Semantic Web: Clarified and Mystified
November 4, 2016
Navigate to “Semantic Web Speculations.” After working through the write up, I believe there are some useful insights in the write up.
I highlighted this passage:
Reaching to information has been changed quite dramatically from printed manuscripts to Google age. Being knowledgeable less involves memorizing but more the ability to find an information and ability to connect information in a map-like pattern. However, with semantic tools become more prevalent and a primary mode of reaching information changes, this is open to transform.
I understand that the Google has changed how people locate needed information. Perhaps the information is accurate? Perhaps the information is filtered to present a view shaped by a higher actor’s preferences? I agree that the way in which people “reach” information is going to change.
I also noted this statement:
New way of being knowledgeable in the era of semantic web does not necessarily include having the ability to reach an information.
Does this mean that one can find information but not access the source? Does the statement suggest that one does not have to know a fact because awareness that it is there delivers the knowledge payload?
I also circled this endorsement of link analysis, which has been around for decades:
It will be more common than any time that relations between data points will have more visibility and access. When something is more accessible, it brings meta-abilities to play with them.
The idea that the conversion of unstructured information into structured data is a truism. However, the ability to make sense of the available information remains a work in progress as is the thinking about semantics.
Stephen E Arnold, November 4, 2016
Genetics Are Biased
November 4, 2016
DNA does not lie. DNA does not lie if conducted accurately and by an experienced geneticist. Right now it is popular for people to get their DNA tested to discover where their ancestors came from. Many testers are surprised when they receive their results, because they learn their ancestors came from unexpected places. Black Americans are eager to learn about the genetics, due to their slave ancestry and lack of familial records. For many Black Americans, DNA is the only way they can learn where their roots originated, but Africa is not entirely cataloged.
According to Science Daily’s article “Major Racial Bias Found In Leading Genomics Database,” if you have African ancestry and get a DNA test it will be difficult to pinpoint your results. The two largest genomics databases that geneticists refer to contain a measurable bias to European genes. From a logical standpoint, this is understandable as Africa has the largest genetic diversity and remains a developing continent without the best access to scientific advances. These provide challenges for geneticists as they try to solve the African genetic puzzle.
It also weighs heavily on black Americans, because they are missing a significant component in their genetic make-up they can reveal vital health information. Most black Americans today contain a percentage of European ancestry. While the European side of their DNA can be traced, their African heritage is more likely to yield clouded results. On a financial scale, it is more expensive to test black Americans genetics due to the lack of information and the results are still not going to be as accurate as a European genome.
This groundbreaking research by Dr. O’Connor and his team clearly underscores the need for greater diversity in today’s genomic databases,’ says UM SOM Dean E. Albert Reece, MD, PhD, MBA, who is also Vice President of Medical Affairs at the University of Maryland and the John Z. and Akiko Bowers Distinguished Professor at UM SOM. ‘By applying the genetic ancestry data of all major racial backgrounds, we can perform more precise and cost-effective clinical diagnoses that benefit patients and physicians alike.
While Africa is a large continent, the Human Genome Project and other genetic organizations should apply for grants that would fund a trip to Africa. Geneticists and biologists would then canvas Africa, collect cheek swabs from willing populations, return with the DNA to sequence, and add to the database. Would it be expensive? Yes, but it would advance medical knowledge and reveal more information about human history. After all, we all originate from Mother Africa.
Whitney Grace, November 4, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
MicroSearch: A Specialized Video Search System for Academic Content
November 3, 2016
I am not sure if the Alphabet Google thing will be down with this new video search system over the long haul. If you want a different way to locate academic videos, you will want to explore MicroSearch’s system. MicroSearch says that it is “a boutique search engine company, providing private, secure video and document cloud storage as well as custom search services.”
I learned about this service in “University Videos on YouTube Get Custom Search.” You can explore the system at http://universityvideos.org/Home. The search system is at this link.
According to the write up, the system aggregates university videos and:
includes a video player that shows the video playing on the left and a transcript tracking with the video on the right. Clicking into another sentence in the transcript jumps the user to that part of the video.
I highlighted this passage:
The service also includes a search tool that allows the user to search on transcript contents, title, description, duration, category, tags, YouTube channel and year uploaded. The same fields are available as metadata, when search results are displayed and downloaded as an Excel export file. An advanced search feature lets the user enter a few letters into the transcription field and then click on an Index button next to the field to obtain a window that displays all of the terms with that series of letters.
Our test queries suggested that the system is less wonky than Google’s video search. The fact that Google is splitting its text index into one part for mobile and one part for traditional desktop search makes clear that search at Google is a work in progress. With a new search system for a segment of YouTube videos, one can conclude that YouTube video search is not a home run for some users.
Perhaps more attention on search and less on Loon balloons might solve the problem. On the other hand, Alphabet Google can simply block developers of “better mousetraps” and move forward with its online advertising programs and projects like solving death. Search is for revenue and maybe not for finding relevant content?
Stephen E Arnold, November 3, 2016
Job Hunting in Secret Is Not So Secret
November 3, 2016
While the American economy has recovered from the recession, finding a job is still difficult. Finding a new job can be even harder has you try to be discreet while handling emails, phone calls, and Web traffic under the radar. A bit of advice is to not search for jobs while at your current position, but that is easier said than done in many respects. Social media is a useful job seeking tool and LinkedIn now offers a job search incognito mode. SlashGear discusses the new mode in the article, “LinkedIn’s Open Candidates Feature Helps You Find A Job In Secret.”
The Open Candidates feature allows LinkedIn users to search for a new job while hiding their job search activity from their current employer. It will try to hide your job search activity, while at the same time it will add a new search feature for recruiters that displays profiles of people who have listed themselves under the Open Candidates feature. The hope is that it will bring more opportunity to these people.
However, nothing is ever secret on the Internet and LinkedIn can only do its best to help you:
While the new feature will probably be welcome by people who would prefer to carry out a job search while ruffling as few feathers as possible, LinkedIn does warn that even it will try to prevent your current employer from seeing that you’ve listed yourself as an Open Candidate, it can’t guarantee that it will be able to identify all of the recruiters associated with your company. In other words, use at your own risk.
If you work in a company that tracks your online social life or for a tech organization, you will have difficulty using this feature. LinkedIn and Microsoft employees will definitely need to use the first piece of advice, search for a new job on your personal computer/device using your own Internet.
Whitney Grace, November 3, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Dark Web Is a Double Edged Sword
November 3, 2016
Apart from hackers and criminals of all kind, the Dark Web is also used by whistleblowers and oppressed citizens for communicating. The Dark Web thus is one of the most secure modes of communicating online; more than secure apps like WhatsApp.
The Newsweek in an article titled How the Dark Web Works and What It Looks Like says:
Dark web technologies are robustly built without central points of weakness, making it hard for authorities to infiltrate. Another issue for law enforcement is that—like most things—the dark web and its technologies can also be used for both good and evil.
Despite backdoors and exploits, law enforcement agencies find it difficult to track Dark Web participants. Few technology companies like Facebook, Microsoft, and Google through its messenger apps promise to provide end-to-end encryption to its users. However, the same companies now are harvesting data from these apps for commercial purposes. If that is the case, these apps can no longer be trusted. As pointed out by the article:
And yet some of these same communications companies have been harvesting user data for their own internal processes. Famously, Facebook enabled encryption on WhatsApp, protecting the communications from prying eyes, but could still look at data in the app itself.
Thus, for now, it seems Dark Web is the only form of secure communication online. It, however, needs to be seen how long the formless and headless entity called Dark Web remains invincible.
Vishal Ingole, November 3, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Facial Recognition Fraught with Inaccuracies
November 2, 2016
Images of more than 117 million adult Americans are with law enforcement agencies, yet the rate of accurately identifying people accurately is minuscule.
A news report by The Register titled Meanwhile, in America: Half of adults’ faces are in police databases says:
One in four American law enforcement agencies across federal, state, and local levels use facial recognition technology, the study estimates. And now some US police departments have begun deploying real-time facial recognition systems.
Though facial recognition software vendors claim accuracy rates anywhere between 60 to 95 percent, statistics tell an entirely different story:
Of the FBI’s 36,420 searches of state license photo and mug shot databases, only 210 (0.6 per cent) yielded likely candidates for further investigations,” the study says. “Overall, 8,590 (4 per cent) of the FBI’s 214,920 searches yielded likely matches.
Some of the impediments for accuracy include low light conditions in which the images are captured, lower procession power or numerous simultaneous search requests and slow search algorithms. The report also reveals that human involvement also reduces the overall accuracy by more than 50 percent.
The report also touches a very pertinent point – privacy. Police departments and other law enforcement agencies are increasingly deploying real-time facial recognition. It not only is an invasion of privacy but the vulnerable networks can also be tapped into by non-state actors. Facial recognition should be used only in case of serious crimes, using it blatantly is an absolute no-no. It can be used in many ways for tracking people, even though they may not be criminals. Thus, it remains to be answered, who will watch the watchmen?
Vishal Ingole, November 2, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
Semantic Search and the Future of Search Engines
November 1, 2016
Google no longer will have one search “engine.” Google will offer mobile search and desktop search. The decision is important because it says to me, in effect, mobile is where it is at. But for how long will the Googlers support desktop search when advertisers have no choice but embrace mobile and the elegance of marketing to specific pairs of eyeballs?
Against the background of the mobile search and end of privacy shift at the GOOG, I read “The Future of Search Engines – Semantic Search.” To point out that the future of search engines is probably somewhat fluid at the moment is a bit of an understatement.
The write up profiles several less well known information retrieval systems. Those mentioned include:
- BizNar, developed by one of the wizards behind Verity, provides search for a number of US government clients. The system has some interesting features, but I recall that I had to wait as “fast” responses were updated with slower responses.
- DuckDuckGo, a Web search system which periodically mounts a PR campaign about how fast its user base is growing or how many queries it processes keeps going up.
- Omnity, allegedly a next generation search system, “gives companies and institutions of all sizes the ability to instantly [sic] discover hidden patterns of interconnection within and between fields of knowledge as diverse as science, finance, law, engineering, and medicine.,” No word about the corpuses in the index, the response time, or how the system compares to gold old Dialog.
- Siri, arguably, the least effective of the voice search systems available for Apple iPhone users.
- Wolfram Alpha, the perennial underdog, in search and question answering.
- Yippy, which strikes me as a system similar to that offered by Vivisimo before its sale to IBM for about $20 million in 2012. Vivisimo’s clustering was interesting, but I like the company’s method for sending a well formed query to multiple Web indexes.
The write up is less about semantic search than doing a quick online search for “semantic search” and then picking a handful of systems to describe. I know the idea of “semantic search” excites some folks, but the reality is that semantic methods have been a part of search plumbing for many years. The semantic search revolution arrived not long after the Saturday Night Fever album hit number one.
Download open source solutions like Lucene/Solr and move on, gentle reader.
Stephen E Arnold, November 1, 2016
Google Gives Third Day Keynote at Pubcon
November 1, 2016
Technology conferences are the thing to do when you want to launch a product, advertise a new business, network, or get a general consensus about the tech industry. There are multiple conferences revolving around different aspects in the tech industry held each month. In October 2016, Pubcon took place in Las Vegas, Nevada and they had a very good turn out. The thing that makes a convention, though, is the guests. Pubcon did not disappoint as on the third day, Google’s search expert Gary Illyes delivered the morning keynote. (Apparently, Illyes also hold the title Chief of Sunshine and Happiness at Google). Outbrain summed up the highlights of Pubcon 2016’s third day in “Pubcon 2016 Las Vegas: Day 3.”
Illyes spoke about search infrastructure, suggesting that people switch to HTTPS. His biggest push for HTTPS was that it protected users from “annoying scenarios” and it is good for UX. Google is also pushing for more mobile friendly Web sites. It will remove “mobile friendly” from search results and AMP can be used to make a user-friendly site. There is even bigger news about page ranking in the Google algorithm:
Our systems weren’t designed to get two versions of the same content, so Google determines your ranking by the Desktop version only. Google is now switching to a mobile version first index. Gary explained that there are still a lot of issues with this change as they are losing a lot of signals (good ones) from desktop pages that are don’t exist on mobile. Google created a separate mobile index, which will be its primary index. Desktop will be a secondary index that is less up to date.
As for ranking and spam, Illyes explained that Google is using human evaluators to understand modified search better, Rankbrain was not mentioned much, he wants to release the Panda algorithm, and Penguin will demote bad links in search results. Google will also release “Google O for voice search.
It looks like Google is trying to clean up search results and adapt to the growing mobile market, old news and new at the same time.
Whitney Grace, November 1, 2016
Sponsored by ArnoldIT.com, publisher of the CyberOSINT monograph
-
- Subscribe to Beyond Search
Feature archive
News archive
-
