The Google and Web Indexing: An Issue of Control or Good, Old Fear?
March 29, 2021
I read “Google’s Got A Secret.” No kidding, but Google has many, many secrets. Most of them are unknown to today’s Googlers. After 20 plus years, even Xooglers are blissfully unaware of the “big idea,” the logic of low profiling data slurping, how those with the ability to make “changes” to search from various offices around the world can have massive consequences for those who “trust” the company, and the increasing pressure to control Googzilla’s appetite for cash. But enough of these long-ignored issues.
The “secret” in the article is that Google actively pushes as many buttons and pulls as many levers as its minions can to make it tough for competitors to “index” the publicly accessible Web. The write up states:
Only a select few crawlers are allowed access to the entire web, and Google is given extra special privileges on top of that.
The write up adds:
Only a select few crawlers are allowed access to the entire web, and Google is given extra special privileges on top of that. This isn’t illegal and it isn’t Google’s fault, but this monopoly on web crawling that has naturally emerged prevents any other company from being able to effectively compete with Google in the search engine market.
I am not in total agreement with these assertions. For example, consider the world of public relations distribution agencies. Do a search for “news release distribution” and you get a list of outfits. Now write a news release which reveals previously unknown and impactful information about a public traded company. These firms like PR Underground-type operations will explain that a one-day or longer review process is needed. The “reason” is that these firms have “rules of the road.” Is it possible that these distribution outlets are conforming to some vague guidelines imposed by Google. Crossing a blurry line means that the releases won’t be indexed. No indexing in Google means the agency failed and, of course, the information is effectively censored.
What about a company which publishes information on a consumer-type topic like automobiles. What if that Web site operator uses advertising from sources not linked to the Google combine? That Web site is indexed on a less frequent basis by the friendly Google crawler. Then those citations are suppressed for some unknown reason by a Google algorithm. (These are written by humans, but the Google never talks much about the capabilities of a person in a Google office laboring on core search to address issues.)
The ideas of the Knuckleheads’ Club are interesting. Implementing them, however, is going to require some momentum to overcome the Google habit which has become part of the online user’s DNA over the last 20 years.
The real questions remain. Is Google in control of the public Web? Are people fearful of irritating Mother Google (It’s not nice to anger Mother Google)?
Stephen E Arnold, March 29, 2021
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
