With Relevance Trashed, Is It Gray Unitards for Online Users?
February 22, 2018
People judge the size of Internet based on their limited experiences as well as reports search engines generate, such as Google and Yahoo. Search engines compete over users by advertising that the size of their search index, but the Internet is truly bigger than any individual search index. The Search Engine Roundtable discusses how search indices do not encompass the entire Internet in the article, “Google: You Can’t Judge Index Size By One Or Two Sites.”
One example that proves you cannot determine the Internet’s size by only two search engines is comparing the search results generated by the same keywords. Both DuckDuckGo and Bing have proven more than once that they can discover Web sites Google and Yahoo cannot. Tim Bray wrote about this particular event on his blog, then it caught the attention of another developer:
“It caught Danny Sullivan’s attention on Twitter, which Danny responded that ‘I wouldn’t make that assumption for the entire web based on what’s happening with only your site.’ Tim being respected, Danny said he would dig into it, believing it may be an issue with the two example sites he cited, even though Bing and DuckDuckGo was able to index and return the content in their search engines.”
Google and yahoo now index the page in question, but this is a reminder than the Internet is a big place. The Dark Web is not picked up by regular search engines and for the amount of Web pages generated everyday it does not come as a surprise that Google and Yahoo would miss one. Maybe there is a business opportunity to develop an AI that tracks Web sites Google and other search engines have not found yet.
Whitney Grace, February 22, 2018
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
