Google Search Explained and Explainedg
August 12, 2013
One of my two or three readers sent me a link to a series of comments on the social networking information aggregation site Reddit. Navigate to http://goo.gl/EJLarJ. The entry is “How does Google search the whole Internet for something in a matter of seconds or even less?”
I found the explanations fascinating.
One person said, ”
Google spends all day every day searching the Web with bots. Web sites and their data are catalogued in a database and when you search, it is the database that is being looked through. It’s also not the whole Internet. Lots of sites have code that prevents them from showing up in the search engine.
Other comments of interest are:
“Amazon has over 2500 sub site maps. ”
“I like that the best way to find out things Google doesn’t know is by using Google.”
“I remember reading somewhere that Google estimated that only 0.02% of the internet is cataloged in the Google database.”
Google’s search index is over 100 million gigabytes big
“I heard an analogy once, that searching the internet with google is like dragging a net through a pond. You’ll get stuff from the surface but there’s a lot of material deeper down you don’t get.”
“Google has only indexed 0.004% of the entire Internet.”
“Imagine there are spiders(web crawlers) going around the web and gathering all the insects(web pages) in stuck there. Then they pile the different insects into cocoons and label them (hash code). Now you can find your favorite insect from the labeled cocoons by keyword and they are brought to you in an order of popularity.”
And my favorite:
“Think of Google like the Index Cards they [librarians] had at the library before computers. The index card system is just an organized collection of where the books (Web sites) exist in the library. All of the actual information is held in the books. A librarian (Web crawler) has to keep the index card system up to date but they [sic] don’t need to do it in realtime every time a book is requested. They keep a database of where everything is instead.”
Yep, librarians with advertising. I am delighted with the explanations of the Google. Delighted, I say.
Stephen E Arnold, August 12, 2013
Sponsored by Xenky
Comments
2 Responses to “Google Search Explained and Explainedg”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
Classic. But Stephen, there must be some way to simply describe what Google does to the layperson, no? How would you describe it, in a sentence or two? Here’s my stab at it:
“Google crawls every site they can, and also stores every query you enter. Their most “Instant” results are based on those query logs.”
if search index is ~100G, why doesn’t google offer an external HD – or other – product enabling limited snippet offline search, I wonder…