The Google: Indexing and Discriminating Are Expensive. So Get Bigger Already
November 9, 2022
It’s Wednesday, November 9, 2022, only a few days until I hit 78. Guess what? Amidst the news of crypto currency vaporization, hand wringing over the adult decisions forced on high school science club members at Facebook and Twitter, and the weirdness about voting — there’s a quite important item of information. This particular datum is likely to be washed away in the flood of digital data about other developments.
What is this gem?
An individual has discovered that the Google is not indexing some Mastodon servers. You can read the story in a Mastodon post at this link. Don’t worry. The page will resolve without trying to figure out how to make Mastodon stomp around in the way you want it to. The link to you is Snake.club Stephen Brennan.
The item is that Google does not index every Mastodon server. The Google, according to Mr. Brennan:
has decided that since my Mastodon server is visually similar to other Mastodon servers (hint, it’s supposed to be) that it’s an unsafe forgery? Ugh. Now I get to wait for a what will likely be a long manual review cycle, while all the other people using the site see this deceptive, scary banner.
So what?
Mr. Brennan notes:
Seems like El Goog has no problem flagging me in an instant, but can’t cleanup their mistakes quickly.
A few hours later Mr. Brennan reports:
However, the Search Console still insists I have security problems, and the “transparency report” here agrees, though it classifies my threat level as Yellow (it was Red before).
Is the problem resolved? Sort of. Mr. Brennan has concluded:
… maybe I need to start backing up my Google data. I could see their stellar AI/moderation screwing me over, I’ve heard of it before.
Why do I think this single post and thread is important? Four reasons:
- The incident underscores how an individual perceives Google as “the Internet.” Despite the use of a decentralized, distributed system. The mind set of some Mastodon users is that Google is the be-all and end-all. It’s not, of course. But if people forget that there are other quite useful ways of finding information, the desire to please, think, and depend on Google becomes the one true way. Outfits like Mojeek.com don’t have much of a chance of getting traction with those in the Google datasphere.
- Google operates on a close-enough-for-horseshoes or good-enough approach. The objective is to sell ads. This means that big is good. The Good Principle doesn’t do a great job of indexing Twitter posts, but Twitter is bigger than Mastodon in terms of eye balls. Therefore, it is a consequence of good-enough methods to shove small and low-traffic content output into a area surrounded by Google’s police tape. Maybe Google wants Mastodon users behind its police tape? Maybe Google does not care today but will if and when Mastodon gets bigger? Plus some Google advertisers may want to reach those reading search results citing Mastodon? Maybe? If so, Mastodon servers will become important to the Google for revenue, not content.
- Google does not index “the world’s information.” The system indexes some information, ideally information that will attract users. In my opinion, the once naive company allegedly wanted to achieve the world’s information. Mr. Page and I were on a panel about Web search as I recall. My team and I had sold to CMGI some technology which was incorporated into Lycos. That’s why I was on the panel. Mr. Page rolled out the notion of an “index to the world’s information.” I pointed out that indexing rapidly-expanding content and the capturing of content changes to previously indexed content would be increasingly expensive. The costs would be high and quite hard to control without reducing the scope, frequency, and depth of the crawls. But Mr. Page’s big idea excited people. My mundane financial and technical truths were of zero interest to Mr. Page and most in the audience. And today? Google’s management team has to work overtime to try to contain the costs of indexing near-real time flows of digital information. The expense of maintaining and reindexing backfiles is easier to control. Just reduce the scope of sites indexed, the depth of each crawl, the frequency certain sites are reindexed, and decrease how much content old content is displayed. If no one looks at these data, why spend money on it? Google is not Mother Theresa and certainly not the Andrew Carnegie library initiative. Mr. Brennan brushed against an automated method that appears to say, “The small is irrelevant controls because advertisers want to advertise where the eyeballs are.”
- Google exists for two reasons: First, to generate advertising revenue. Why? None of its new ventures have been able to deliver advertising-equivalent revenue. But cash must flow and grow or the Google stumbles. Google is still what a Microsoftie called a “one-trick pony” years ago. The one-trick pony is the star of the Google circus. Performing Mastodons are not in the tent. Second, Google wants very much to dominate cloud computing, off-the-shelf machine learning, and cyber security. This means that the performing Mastodons have to do something that gets the GOOG’s attention.
Net net: I find it interesting to find examples of those younger than I discovering the precise nature of Google. Many of these individuals know only Google. I find that sad and somewhat frightening, perhaps more troubling than Mr. Putin’s nuclear bomb talk. Mr. Putin can be seen and heard. Google controls its datasphere. Like goldfish in a bowl, it is tough to understand the world containing that bowl and its inhabitants.
Stephen E Arnold, November 9, 2022
Hi, Mom, I Got a D in Math and Science This Term
November 8, 2022
I have never earned a D grade. In the cow town public high school, the grading system was simple: 93 to 100 = A, 83 to 93 = B, 70 to 82 = C, and 60 to 69 = D. Below 60, say “Hello” to an F, you loser you.
“Almost 30% of People Redo or Refine Google Searches, Study Says” reports:
This 30% number comes from 9.7% of users who engaged in a “Google Click,” meaning they clicked on images or something in a carousel after making a query. For these people, they may have actually found what they were looking for. Another 17.9% of users made modifications to “Google Keyword,” or ways to modify their original query. This totals to 27.6%, which was then rounded up by SEMRush.
Should we “trust” the source and its math? Heck no. But the interesting point is that quite a few Google users find that Google search is in the D category.
With the surge of “close enough for horse shoes” and “good enough” thinking, the result is not particularly surprising. In my own experience, I now have to work harder than ever to obtain accurate, useful, relevant information. I routinely cycle through Mojeek, Swisscows, Yandex, and a number of other systems. For me, Google is in the D Minus or F category. The for fee alternatives are disappointing because the depth of their coverage is similar to a child’s plastic wading pool.
What’s this mean? Finding on point information is taking more time which translates into direct costs. That ad supported model is super, isn’t it? “D” does the job.
Stephen E Arnold, November 8, 2022
When the Non-Googley Display Their Flaws, Miscommunication Results
October 31, 2022
If you are Googley, you understand the value of the Google way. You embrace abandoned products because smart people do not get bonuses working on loser services. You advocate for new ways to generate revenue because losers have to pony up cash to pay for salaries. You ignore the bleats of the lesser creatures because those lower on the Great Chain of Digital Being deserve their mollusk status.
I want to point out that the article “How Google’s Ad Business Funds Disinformation Around the World” illustrates the miscommunication between the Googlers and the Rest of the World. With ignorance on display, little wonder the free services of the online services company are neither appreciated nor understood.
Consider advertising.
Smart software does not make errors. If a non Googley person objects to an advertisement which pitches certain products and services, it is the responsibility of the “user” to discern the issue and ignore the message. Smart software informed by synthetic data and functionality of Oingo identifies interests and displays content. By definition, the non Googley fail to appreciate the sophistication of the Google method. Hence, how can these non Googley mollusks perceive the benefits of the Googlers.
The cited article purports to provide proof (not big data, not psychological profiles based on user history, and not fancy math informed by decades of sophisticated management actions) that something is amiss in the world of Alphabet Google YouTube and DeepMind Land. Here’s an example:
The investigation also revealed that Google routinely places ads on sites pushing falsehoods about COVID-19 and climate change in French-, German- and Spanish-speaking countries.
Where’s the beef? By definition, the non Googley have to decide what’s on the money or not. If one has flawed mental equipment, the failure to understand Google is not Google’s problem. It is the way of the world.
Google has a business model which works. True. Google did have to pay to avoid a legal hassle with Yahoo for the online ad furniture before the Google IPO. But in the Google, good ideas are, by definition, Google’s. Therefore, getting caught in a Web of insinuations is further proof that a gulf separates the Googley from the non Googley. Maggots, remember?
The cited article presents examples from countries which provide a small percentage of Google experts. It makes sense that those who are non Googley would apply their limited intelligence and analytic skills to countries with certain flaws. Google’s smart software makes smart decisions, and the failure to recognize the excellence of Google’s methods are, by definition, a problem but not for Google. Come on. Serbia? Turkey? France? Where are these entities on the Great Chain of Digital Being? At the top? France has more types of cheese than Googlers I think.
Net net: Criticize Meta. Take a look at the Apple tax. Examine the dead squirrels crushed by the Bezos bulldozer. Those are lesser firms which are well suited to scrutiny by the non Googley. So if you don’t work at Google, how can you understand the excellence of Googlers? Answer: You cannot.
Stephen E Arnold, October 31, 2022
Google: Beavering Away on Trust, Privacy, and Security
October 27, 2022
Google and trust: What an interesting pair of words. I wonder if anyone remembers the Google Search Appliance and its “phone home” function. I sure do because I was paid to go to a government meeting at one of the Executive Branch agencies so I could intermediate with my contacts in the search appliance mini-unit. I want to point out that customer support, technical support, and access to specific details of the operation of the Google Search Appliance were not easy for licensees to access. Hence, a dinobaby like myself was enlisted for the job. What was the reason for the concern? The GSA worked but the government technology folks were interested in the “phone home” function; specifically, what was available to the GOOG, what was transmitted, and who had access to those data from the government agency?
What do you think the Googler on the call with me in a conference room stuffed with government professionals said? As I recall, the Googler called me on my mobile and I stepped out of the room. The Googler said, “Ask them if the shipping crate was available?” I said, “Okay?” and returned to the room. The Googler popped back into the conference line and said, “Steve, do you have a question?” I turned to the group in the room and asked, “Is the shipping crate in the store room?”
The team leader’s answer was, “Yes.”
The Googler then said, “Steve, would you ask the client to ship the GSA back to us to check?”
The Googler disconnected. I organized the return. The senior government executive later asked me, “Do you trust that outfit?”
My answer was, “I do.”
The government executive said, “I don’t.”
Ah, a different opinion. As a result of the “phone home” feature of the cheerful yellow GSAs Google made a business decision and abandoned what it delightfully called “enterprise search.”
I thought about this meeting from years ago when I read “Court Documents Allege Google cultivated Privacy Misconceptions of Chrome’s Incognito Mode.” Was I surprised? Nope. Google is loved by people who like free services. Critical thinking about the data gathered by the online ad agency has not been a widespread practice for decades.
Why now?
My hunch is that partial understanding of what the Google datasphere has become is now coming into focus. The response of Silicon Valley “real” news outfits is amusing. From cheerleaders to aggrieved info-addicts is interesting.
The cited article states:
Google faces a potential privacy case as a class of millions of users filed to sue it for billions of dollars over Chrome’s Incognito mode lack of genuine privacy protections. While user ignorance is never a great argument in front of a judge, court documents first filed in March of 2021 paint a picture that Google has been complicit in cultivating user misconceptions on privacy. According to the filings, Google Marketing Chief Lorraine Twohill emailed CEO Sundar Pichai last year, warning that they need to consider making Incognito “truly private.” Even more concerning is her indirect admission that they have had to use misleading language when marketing the feature.
That’s the Google game plan.
Like many game plans, other teams figure out how to thwart what once was quite effective. Heck. In the case of the GOOG, the game plan won the equivalent of 20 or more World Cups. Now, however, the play book and its simple methods of saying one thing and doing another, apologizing and moving forward anyway, and paying trivial fines and taking advantage of advertisers and users has to be fluffed up.
Are the broad outlines of the new playbook discernable? I keep track of some of the changes:
- Distraction
- Shuffling product and service offerings
- Acquisitions which are not technology but consulting
- Continuous interactions with lobbyists and other contacts in Washington, DC, London, France, and Paris, France, among other locations
- Low profile but significant efforts to keep the online ad company’s India activities out of the news spotlights
- Hand waving about new policies in order to put some moats around certain skyrocketing operational costs because…. plumbing is expensive even for the GOOG.
What’s the outcome in my opinion? (Don’t want it. Just stop reading.) My view is that Google’s management methods will continue to show signs of fragility. Maybe some big cracks will emerge? Lawyering and marketing will kick “real” engineers off the fast track to bonuses. Yikes. The Google is a changin’… and fast. Example: Incognito which isn’t incog or neat-o.
Stephen E Arnold, October 2022
Eureka! The Google Phone Call Feature
October 27, 2022
Despite not being the preferred method of communication anymore, phone calls are still a vital part of society. One thing that has always plagued phone calls is clear reception. The Verge shares how Google is innovating once more, but on old-school phone calls: “Google Is Working On ‘Clear Calling’ For Android Phone Calls.”
The new Android 13 release includes a “clear calling” feature stated to reduce background noise during calls. Twitter user Mishaal Rahman told people how to enable it. Clear Calling is supposed to work on most mobile networks, but it is not available for Wi-Fi calling and it does not send information from your phone calls to Google.
Google has experimented with noise cancellation technology before:
“Google has been flexing its noise-canceling muscles (and custom six-core audio chips) for a while. First, and most impressively, by using AI to suppress background noises like the crackling of snack bags, keyboard clicks, and dogs barking in Google Meet. More recently with the $199 Pixel Buds Pro — the company’s first earbuds with active noise cancellation.”
Noise cancellation technology can always stand improvements, especially to drown out all the noise generated by today’s media. What’s next? More data capture? More fines from non Googley government busybodies? More trimming of staff?
Whitney Grace, October 27, 2022
Forty Firms and the European Union Demonstrate Their Failure to Be Googley
October 20, 2022
In the last 25 years, an estimated 300,000 people have worked for the Google as FTEs (full time equivalents). My view is that the current crop of European Union officials as well as the senior managers of several dozen online shopping services firms are not Google-grade human resources. Why do I make this distinction between those who are Googley and those who do not make the grade. Being Googley is not like the French Foreign Legion. In that organization, a wanna be Legionnaire must do push ups, master the lingo of the new homeland, and be ready to die for France. At Google one must be clever, have the “right stuff” intellectually, and be adept at solving problems, playing with a mobile phone, and manipulating a Mac simultaneously. Being Googley means understanding the ethos of the forever young online search system. That system, as I understand it, accepts the constraints of the GoTo, Overture, and Yahoo online advertising concept. Furthermore that system accepts that Oingo became a key component in matching advertising to user interests. Thus, to be Googley means that smart software, minimal interaction with humanoids who are by definition are “not Googley,” and reliance on charging for entering and leaving a digital saloon. Extra cash is required to enjoy the for fee options in the establishment; for example, a mouse pad with the Google logo silkscreened thereupon.
I read “EU Companies Claim Google Still Abusing its Shopping Power.” The article explains that a number of companies believe the Google is not behaving in a warm, friendly, collegial manner. Are these firms’ allegations correct?
I don’t know.
What I do know is that the companies signing the letter to EU regulators are demonstrating to me that these organizations are not Googley. What does this mean? May I hypothesize about the implications of lacking an understanding of the Alphabet Google YouTube DeepMind organization might be? Of course, you say. So here are my thoughts:
- None of the signatories nor the employees of these signatories will receive first-class tchotchkes at conferences at which AGYD has a booth or stand equipped with freebies
- None of the signatories, their employees, nor their progeny will be hired by AGYD due to manifest non Googliness. (Remember, please, that Googliness is next to Godliness. Who else can solve death?)
- None of the Web sites, online properties, or content objects will be made findable by the “black box” operated by closely guarded algorithms and informed by the superior methods of the smart software. (I must admit I find the idea “if you are findable in Google, you do not exist) a most existentially well formed idea.
- None of the elected officials involved in fining, criticizing, or demonstrating non Googliness will be supported in their re-election efforts by the GOOG’s powerful systems.
AGYD is not a company. It is a digital country. It handles more than 90 percent of the search queries in North America, South America, and Western Europe. YouTube is television for those in Eastern Europe. The Google is bigger than the definitely Google challenged in Europe. Perhaps thinking about the downsides of not being Googley is a useful activity? Just a thought. But it may be too late for the 40 outfits signing a letter attesting to their failure in the Google Comprehension Examination.
Stephen E Arnold, October 20, 2022
The Google Virtual Private Network Is Sufficiently Unprivate So Google Can Show You Ads
October 20, 2022
Ads are as American as apple pie for Internet users. Ads allow companies and smaller businesses to make a profit from their products and services. Usually, ad revenues help keep products and services free. Large tech companies, like Google, Apple, Amazon, and Facebook, have other income streams than ads, so ad blockers are not harming their bottom lines. Google, however, is counting every red cent, because they are pulling the plug on VPN ad blockers says Blokada: “Google Cracks Down On VPN Based Adblockers.”
Under the guise of improving performance and security, Google has revamped its developer policy for the Play Store. The changes go into effect throughout the remainder of 2022. Changes to VPN ad blockers take effect in November 2022:
“One of the main policy changes concerns the VPN Service which will take effect on November 1, 2022: Google claims to be cracking down on apps that are using the VPN service to track user data or rerouting user traffic to earn money through ads. However, these policy changes also apply to apps that use the service to filter traffic locally on the device. Apps such as Blokada v5 and Duck Duck Go. Specifically the policy does not allow for ‘Manipulating ads that can impact apps monetization’.”
Blokada is a popular ad blocked for mobile and VPN services. The new Google Play Store policy sounds like it would hurt Blokada users, but its developers found a way to circumvent it. Blokada no longer requires a local VPN, instead, it uses cloud filtering. The advantage to cloud filtering, other than not violating Google policy, is it does not affect network speed, device speed, or battery life.
Other VPN users will be viewing ads, lots of ads, if they do not find their own Play Store policy loophole. Google will probably find a way to prevent these loopholes because innovative Google has improved the GoTo, Overture, and Yahoo systems, of course.
Whitney Grace, October 20, 2022
Why Is Google on the Hot Seat in India? Does the Indian Government Understand Being Googley?
October 19, 2022
The Competition Commission of India (CCI) is piling up allegations against Google faster than they can be resolved. The Indian Express reports, “CCI Orders Another Probe Against Google.” At issue is the company’s allegedly self-serving terms for news organizations. Gee, what do India’s regulators sense that US regulators seem to overlook? (And Europe‘s, for that matter.) We learn:
“The News Broadcasters & Digital Association had alleged that its members are forced to provide their news content to Google in order to prioritize their weblinks in the Search Engine Result Page (SERP) of Google. As a result, Google free rides on the content of the members without giving them adequate compensation, as per the complaint. Among others, it was alleged that Google exploited the dependency of the members on the search engine offered by Google for referral-traffic to build services such as Google News, Google Discover and Google Accelerated Mobile Pages (AMP). The search engine major provides news content to user through Google Search and through news aggregator vertical, Google News. According to the complaint, in Google Search, users can either search directly for news through News Tab or receive news through result in SERPs. Google incorporated news content in its SERPs through featured snippets including ‘Top Stories’ carousels. However, the revenue distributed by Google to news publishers doesn’t compensate for the real contribution made by the association’s members to these platforms, it added.”
The latest probe is being consolidated with two similar ones already in progress. Those claimants are the Digital News Publishers Association and the Indian Newspaper Society. How many more plaintiffs will join the fray before the combined investigation concludes? More importantly, will any penalties be imposed that can even scratch the tech powerhouse?
Cynthia Murrell, October 19, 2022
Google: Clever Cost Cutting
October 18, 2022
Most web searchers do not make it past the first page or two of Google results. But even if one has the patience to go all the way to the end, it seems one can only see a fraction of the results promised at the top of each page. According to a blog post from web scraper SerpApi, “Google’s ‘Millions of Search Results’ Are Not Being Served in the Later Pages Search Results.” Writer Justin O’Hara reports:
“A misconception regarding Google’s search results is that all of the results are being served to the user conducting that particular search. Those 2 billion search results can’t be gotten through Google’s pagination, and it seems that this number is somewhat arbitrary to the search, or commonality of the keyword. I rarely go past the 2nd or 3rd page of Google’s search results for any kind of query anymore, but these rankings of results are big business with Search Engine Optimization. I wanted to do a little case study on the actual amount of searches that get served to users for a couple different searches.”
O’Hara experimented by searching for his company’s name and could see only 146 results out of the 166,000 Google said it found. Repeating the search with omitted results included, as Google offers, garnered only 369 results. But why? Cost cutting? Or perhaps information shaping? We may never know. Not surprisingly, O’Hara emphasizes SerpApi’s Google Search API can scrape the results Google itself does not deign to pass on to users.
Cynthia Murrell, October 18, 2022
Google: Business Intelligence, Its Next Ad Business
October 11, 2022
Google has been a busy beaver. One example popped out of a ho hum write up about Google management’s approach to freebies. The write up “Google’s CEO Faced Intense Pushback from Employees at a Town Hall. His 2-Sentence Response was Smart Leadership” contains a rather startling point, if the article is accurate. Here’s the passage which is presumably a direct quote from Sundar Pichai, the top Googler:
Look, I hope all of you are reading the news, externally. The fact that you know, we are being a bit more responsible through one of the toughest macroeconomic conditions underway in the past decade, I think it’s important that as a company, we pull together to get through moments like this.
Did you see the crazy admission: “being a bit more responsible”. Doesn’t this mean that the company has been irresponsible prior to this announcement. I find that amusing: More responsible. Does responsibility extend beyond Foosball and into transparency about alleged online ad fraud or the handling of personnel matters such as the Dr. Timnit Gebru example?
But to the business at hand: Business intelligence. Like enterprise search and artificial intelligence, I am not exactly sure what business intelligence means. To the people who use spreadsheets like Microsoft Excel, rows and columns of data are “business intelligence.” But there must be more than redos of Lotus 1-2-3?
Yes, there are different ways to “do” business intelligence. These range from listening in a coffee shop to buying data from a third party provider and stuffing the information into Maltego to spot previously unnoticed relationships. And there are, of course, companies eager to deliver search based applications to make finding a competitor’s proposal to a government agency easier than figuring out which Google Dork to use.
“Google Days It’s Cracked the Code to Business Intelligence” explains that the Google is going to make BI as business intelligence is known to those in the know the King of the Mountain. I noted this passage:
In business intelligence [BI], “there was always this idea of governing BI and of self-service, and there was no reconciliation of the degree of trust and the degree of flexibility,” Google’s Gerrit Kazmaier told reporters last week, ahead of the Google Cloud Next conference. “At Google, I think we have cracked that code to how you get trust and confidence of data with the flexibility and agility of self-service.”
This buzzword infused statement raises several fascinating ideas. Let’s look at a couple of them, shall we?
First, the idea of “governance.” That’s a term to which I can say I don’t know what the heck it means. But the notion of “governance” and “trust” is that somehow the two glittering generalities are what Google has “cracked.” I must say, “What’s the meaning, Gerrit Kazmaier?”
Second, I noted three buzzwords strung together like faux silver skulls on a raver’s necklace: Trust, confidence, flexibility, and agility. To me, these words mean that more users want a point-and-click solution to answer a question about a competitor or the downstream impacts of an event like sanctions on China. The reality is that like the first buzzword, these don’t communicate, they evoke. The intention is that Mother Google will deliver business intelligence.
The solution, however, is not one Google crafted. The company’s professionals could not develop a business intelligence solution. Google had to buy one. Thus, the code cracking was purchased in the form of a company called Looker. The appeal of the Looker solution is that the user does not have to figure out data sources, determine if the data are valid, wrestle to get the data normalized, run tests to determine if the data set meets the requirements of a first year statistics class problem, and figure out what one needs to know. Google will make these steps invisible and reduce knowledge work to clicking an icon. There you go. To be fair, other companies have similar goals. These range from well known US companies to small firms in Armenia. Everyone wants to generate money from easy business intelligence.
Google is an online advertising business. The company wants to knock Microsoft off its perch as the default vendor to business and government. The Department of Defense is going to embrace the Google Cloud. I am not sure that some DoD analysts will release their grip on Microsoft PowerPoint, however.
Can a company trust Google? Does Google have a mechanism for governance for data handling, managing its professional staff (hello, Dr. Gebru), and ensuring that automated advertising systems are straight and true? Does Google abandon projects without thinking too much about consequences (hello, Stadia developers and customers)?
My hunch is that reducing business intelligence from a craft to a mouse click sets the stage for:
- Potential embedded and intentional data bias
- Rapid ill-informed decisions by users
- A way to inject advertising into a service application and personalization.
Will the days of the free car washes return to the Google parking lot? Will having meetings in a tree house in the London office become a thing again? Will Google displace other vendors delivering search based applications which engage the user in performing thoughtful analyses?
Time will provide the answer or rather Looker will provide the answer. Google will collect the money.
Stephen E Arnold, October 11, 2022
-
- Subscribe to Beyond Search
Feature archive
News archive
-
