Google Allegedly Skews Results Listings to Help Itself: Surprise?
January 24, 2017
I read a Wall Street Journal recycling research from a search engine optimization outfit called SEMrush. The idea is that Google looks at a user’s query and puts links to its products where its search users will see them and click on them. Doesn’t Diego Simeone’s kid play professional soccer. I suppose the successful coach of Athletico Madrid has skewed other coaches’ interest in his gifted, advantaged progeny.
My Wall Street Journal online account doesn’t work. No joy. I take the dead tree version of Mr. Murdoch’s flagshipish newspaper, but it is a hassle to provide links to online content which will not appear. Therefore, I have watched the “revelations” about Google’s fiddled results as it flashed around the interwebs.
I noted a version of the story with the crafty title “’Google Buys Ad Space above Search Results to Promote Its Own Products – Giving It an Advantage over Its Online Competitors’. The write up provides a clear explanation of Google’s alleged misdeeds. I can hear the shouts from some, “Bench Simeone’s kid.”
The write up asserts:
According to the Journal, whenever someone enters a search term in Google related to pieces of hardware, ads for the relevant items sold by either Google or a sister company would appear in the most prominent spot on the page 91 percent of the time. In 43 percent of instances, the top two ads were for Google-linked products.
The write up embraces the meat of the SEMrush research revelation:
Google’s practice of favoring its own product ads on relevant search results has raised questions over whether it is violating anti-trust laws.
My view is that Google displays information in search results which attempt to accomplish these goals. Remember. These are my observations based on my research for my three Google monographs and the columns I wrote about Google for Information Today for two or three years. Links to the articles are on my LinkedIn page and on the Information Today Web site.
Observation 1. Google makes decisions like any other Sillycon Valley company; that is, product managers or their ilk cajole engineers to make changes which generate revenue. Some senior executives are unaware or partially aware of these manual and algorithmic tweaks. In large, chaotic outfits, only a handful of people may know what’s been twiddled. Most in the outfit don’t care what their colleagues are doing. The consequences can range from nuking traffic to a Web site to giving pride of place to a Google “fave”.
Observation 2. Google needs clicks itself. What is the company going to do to push some of its own products. Why plug the iPhone when one is selling the Pixel. The Pixel marketers who manage to get some space on results screens can’t manufacture enough phones to meet demand. Some of the Pixel’s flaws go unfixed because the craziness of the Pixel allows miscommunication, missteps, and misunderstandings to flourish in the Google greenhouse. Producing clicks is tough even for Google because only a tiny fraction of Web and mobile search users click on ads and pay attention to their messages. My view is that silver tongued Googlers talk their products to the top. Google engineers just want life to be marketing to leave them alone as I perceive the work environment. The “pride of place” phenomenon has spread like mould in wallboard. Those who should be paying attention are involved with Loon balloons and wrangling for slots in President Trump’s administration. The business processes themselves allow the present results policies to flourish and become the de facto way to do business.
Observation 3. Google needs to pump up revenue. I know that most of the Wall Street wizards think I deserve to live in a backwater in rural Kentucky. But the reality is that the shift from desktop search traffic to mobile search traffic has started a fire in the USS Google’s lower deck. Alphabet, the parent company, has to find replacement revenues before the search revenue starts to flat line and maybe drift down. Thus, the need for money feeds the indifference to the business processes which allow Googlers to promote the company’s products in order to pump up sales. Whether Google transfers cash to buy ads or whether an engineer does what a sleek Google MBA wants makes no difference to me. The result shaping has been a characteristic of “relevance” since 2005 or 2006. Precision and recall have been killed in the battle for revenue.
I don’t like or dislike the Alphabet Google thing. I have paid some attention to the company since I met Larry Page at a search engine conference. I was, as I recall, one of the people who said on our panel that truncation was going to be in Google’s future. Mr. Page laughed at me and said, “Never.”
Guess what?
Google implemented truncation when it started to get serious about clustering and other not so CLEVER methods.
Once advertising enters search processes, objectivity, precision, and recall are doomed. After 16 years, I find it amusing that experts are just now discovering that Google search is not the same as running a query on the old fashioned Westlaw system.
There are other little surprises in the Alphabet Google system too. I documented many of these for my clients between 2002 and 2010 when I grew tired of hearing people say, “I am an expert in search.”
Yeah, right. That’s why it has taken 16 years for the nature of Google search results to catch users’ attention.
Shaped results? Ad placement fast dancing?
Big surprises, right?
Stephen E Arnold, January 24, 2017
Real Time Remains Complicated and Costly
July 14, 2016
I love it when search and content processing vendors yammer about their “real time systems.” Years ago I did a report for a client in Europe about the costs of real time. Summarizing the six month research effort is easy: Real time is tough in computer systems. Latency exists.
To get a sense of how tough it is to accelerate certain online actions, navigate to “Legendary Hedge Fund Wants to Use Atomic Clocks to Beat High-Speed Traders.” Despite the wonky “legendary,” I noted this comment:
Patent application no. 14/451,356 [to find this puppy use US2016/0035027] has one goal: to outrun the speed demons of Wall Street. The 16-page document was quietly published by the U.S. Patent and Trademark Office in February. Replete with schematic drawings, the filing describes a novel way for “executing synchronized trades in multiple exchanges.” The invention consists of not only sophisticated algorithms and a host of computer servers, but atomic clocks — precisely calibrated to vibrations of irradiated cesium atoms — to sync orders to within a few billionths of a second.
I also highlighted:
Its invention, developed by the firm’s co-chief executive officers, Robert Mercer and Peter Brown, first sends an order to a central server, which breaks it up into multiple smaller orders. Those are then routed to venues that offer the best prices and most liquidity, much the same as brokers do now. But before that happens, the smaller orders are sent to servers located as close to the exchanges as possible, along with instructions on the precise times they should be executed. The co-located servers sync their transactions so HFT firms won’t have enough time to identify an order on one exchange and then race to another to trade against it. A crucial part of the system is the optical, atomic or GPS clocks that will be used synchronize those orders. Renaissance says in its application that GPS clocks are accurate to within nanoseconds and any time differences between them are “too small to be perceived” by HFT firms.
How much is an atomic clock centric trading system? A well heeled Wall Street firm can afford these systems. The reason is that the infrastructure to pull off this near real time approach is out of reach for many outfits.
How close to real time is a search and content processing system? You can believe the index is up to date, but I would suggest that you are looking at last week’s leftover barbeque chicken in a Tupperware box. Saying “real time” is less difficult and expensive than shaving milliseconds off an online action. Enjoy that chicken?
Stephen E Arnold, July 14, 2016
Remote Companies: No Office, No Physical Presence
June 16, 2016
I came across a database of companies with [a] no office and no physical presence or [b] jobs for people who don’t have to show up at a facility. The idea is that outsourcing and the gig economy make it possible for a firm to do provide products and services without folks hanging around the coffee machine or hunching over a laptop in an open work space. (Commutes are bummers. Years ago, I learned to read the Washington Post whilst creeping along the jammed expressway from Rockville to Bethesda. My really cheap Pinto was perfect for this type of slog.) You can access Remotebase and browse job listings and the names of the companies for free. It seems the service is free. Most of the companies whose links I explored were new to me; for example, the canine inspired Corgibytes and the Google killer DuckDuckGo. Think of it. Work from a public library, a Pinto or equivalent, a coffee shop, or (heaven forbid) an airport lounge.
Stephen E Arnold, June 16, 2016
Quote to Note, Nay, Memorize from Yahoo
February 23, 2016
I read “Yahoo Forms Panel to Explore Strategic Options.” I looked past the notion that a committee can come up with options beyond sell this puppy. I did find a gem of a phrase in the write up. Here’s the keeper:
Mayer said in a statement, while emphasizing that everyone at Yahoo wanted to return the “iconic company to greatness”.
I like the notion of Yahoo as an icon. I think directories from the 1990s should be icons. The concept of greatness is a good one. My hunch is that revenue would be a help for whatever Yahoo is going to return to.
My thought is that the Yahooligans and the Xoogler look more like a business school case study for a course covering Touchstones in Internet Successes and Failures. I am debating whether Yahoo is a success or a failure. I have decided. Failure: Geocities, the Google GoTo Overture legal matter, the Semel reign, the CEO with sketchy credentials, the internal squabbling, et al.
Yep, let’s have a committee meeting. Great idea. Thank goodness I have IBM Watson to pick up the slack created by the silliness at Yahoo.
Stephen E Arnold, February 23, 2016
Need to Ace Your Alphabet Google Interview?
January 25, 2016
I know some “real” journalists and employees of “real” publishing outfits like Pearson want to work at the Alphabet Google thing? Sounds interesting, doesn’t it.
Navigate to “41 of the Trickiest Questions Google Will Ask You in a Job Interview” to get the inside scoop on landing that perfect gig. Just think. T shirts, mouse pads with the Google logo, maybe a day at Google I/O.
I have a copy of the GLAT or Google Labs Aptitude Test. I believe this bit of tomfoolery was retired years ago, and I can say with some confidence that the questions presented by the UK newspaper may not do the trick for unemployed journalists and/or professional publishers riffed by outfits like Pearson.
Here are three questions to land you in your dream cubicle:
Math and probability: A coin was flipped 1000 times and there were 560 heads. Do you think the coin is biased? — Quantitative Analyst, September 2015
Search: How many ways can you think of to find a needle in a haystack? — Business Associate, May 2014
Fortune telling: How do you think the digital advertising world will change in the next 3 years? — Creative Director, January 2016
Those who want the thrill of the Alphabet Google life are now able to begin their preparatory work.
Oh, here’s another question:
Self entertainment for life’s sound track: If you could only choose one song to play every time you walked into a room for the rest of your life, what would it be? — Associate Account Strategist Interview, March 2014
Tip: Don’t choose a track from an Apple service.
Stephen E Arnold, January 25, 2016
Big Data and Enterprise Search: The Caution Lights Are Flashing
December 15, 2015
I read “How You Should Explain Big Data to Your CEO.” The write up included a link which triggered thoughts of how enterprise search dug itself a hole and climbed. Unable to extricate itself from a problem enterprise search vendors created, the entire sector has been marginalized. In some circles, enterprise search is essentially a joke. “Did you hear about the three enterprise search vendors who walked into a bar?” The bartender says, “What is this? Some kind of joke?”
The link pointed me to a Slideshare (owned by the email and job hunting champion LinkedIn). That presentation, “5 Signs Your Big Data Project is Doomed to Fail,” could have been borrowed from one of my talks about enterprise search in 2001. It was not, but the basic message was identical: Big Data has created a situation in which there are some challenges here and now.
The presentation was prepared by Qubole (maybe pronounced cue ball?). Qubole is a click to query outfit. This means that reports from Big Data are easy to generate.
Here are the problems Big Data faces:
- Failed implementations. Qubole asserts that 87 percent of the Big Data implementations are flops
- 73 percent of executive describe the Big Data project as flop
- 45 percent of Big Data projects are completed
These data are similar to the results of “satisfaction” with enterprise search solutions.
Why? Qubole asserts:
- Inaccurate project scope
- Inadequate management support
- No business case
- Lack of talent (in search the talent may be present but overestimates its ability to deal with enterprise search technologies and processes)
- “Challenging tools.” I think this means that in the Big Data world there are lots of complexities.
What can one charged with either search or Big Data tasks do with this information?
My view is, “Ignore it.”
The “can do” spirit carries professionals forward. Hiring a consultant provides some job protection but does little to reverse the failure and disappointment rate.
My view is that the willingness of executives to grab at a magic solution presented by a showman marketer overrides failure date. The arrogance of those involved create a “that won’t happen to us” belief.
Who is to blame? The company for believing in baloney? The marketer for painting a picture and showing a Hollywood style demo? The developers who created the Big Data solution, knowing that chunks were not complete or just did not work before the ship date? The in house engineers who lacked self knowledge to understand their own limitations?
Everyone is in the hole with the enterprise software vendors. The hole is deep. Magic solutions are difficult to pin down. The future of Big Data is likely to parallel to some degree the dismal track record of enterprise search. Fascinating. I can hear the mid tier consultants and the handful of remaining enterprise search vendors asserting that Qubole’s points are not applicable to their specific situation.
Yep, and I believe in the tooth fairy and Santa.
Stephen E Arnold, December 15, 2015
Deleting Data: Are They Really Gone?
November 17, 2015
I read “Gawker Media’s Data Guru Presents the Case for Deleting Data.” The main idea is that hoarding permits a reality TV program. Hoarding data may not be good TV.
The write up points out that data cleaning is not cheap. Storage also costs money.
A Gawker wizard is quoted as saying:
We effectively are setting traps in our data sets for our future selves and our colleagues… Increasingly, I find that eliminating this data from our databases is the best solution. Gawker’s traffic data is maintained for just a few months. In our own logs and databases, we only have traffic data since February. and even that’s of limited use: We’ll toss some of it before the end of the year.
Seems reasonable. However, there may be instances when dumping or just carelessly overwriting log files might not be expedient or legal. For example, in one government agency, the secretary’s “bonus” depends on showing how Internet site usage relates to paperwork reduction. The idea is that when a “customer” of the government uses a Web site and does not show up in person at an office to fill out a request, the “customer” allegedly gets better service and costs, in theory, should drop. Also, some deals require that data be retained. You can use your imagination if you are an ISP in a country recently attacked by terrorists and your usage logs are “disappeared.” SEC and IRS retention guidelines? Worth noting in some cases.
The question is, “Are data really gone once deleted?” The fact of automatic backups, services in the middle routinely copying data, and other ways of creating unobserved backups may mean that deleted data can come back to life.
Pragmatism and legal constraints as well as the “men in the middle” issue can create zombie data, which, unlike the fictional zombies, can bite.
Stephen E Arnold, November 17, 2015
Billing and Meetings Remain Easier Than Usable Digital Systems
November 11, 2015
I have bumped against digital initiatives in government and industry a number of times. The experience and understanding I gained were indispensible. Do you remember the “paperless office”? The person attributed with creating this nifty bit of jargon was, if memory serves me, Harvey Poppel. I worked with the fellow who coined this term. He also built a piano. He became an investment wizard.
Later I met a person deeply involved with reducing paperwork in the US government. The fellow, an authoritative individual, ran an advertising and marketing company in Manhattan. I recall that he was proud of his work on implementing strategies to reduce dead tree paper in the US government. I am not sure what happened to him or his initiative. I know that he went on to name a new basketball arena, selecting a word in use as the name of a popular vitamin pill.
Then a mutual acquaintance explained the efforts of an expert who wrote a book about Federal digitalization. I enjoyed his anecdotes. I was, at the time, working as an advisor to a government unit involved in digital activities, but the outfit ran on paper. Without paper, the Lotus Notes system could not be relied upon to make the emails and information about the project available. The fix? Print the stuff on paper. The idea was to go digital, but the information highway was built on laser printer paper.
I thought about these interactions when I read “A Decade into a Project to Digitize U.S. Immigration Forms, Just 1 is Online.” (If the link is dead, please, contact the dead tree publisher, not me.)
According the article:
Heaving under mountains of paperwork, the government has spent more than $1 billion trying to replace its antiquated approach to managing immigration with a system of digitized records, online applications and a full suite of nearly 100 electronic forms. A decade in, all that officials have to show for the effort is a single form that’s now available for online applications and a single type of fee that immigrants pay electronically. The 94 other forms can be filed only with paper.
I am not surprised. The article uses the word “mismanaged” to describe the process upon which the development wheels would turn.
The write up included a quote to note:
“You’re going on 11 years into this project, they only have one form, and we’re still a paper-based agency,’’ said Kenneth Palinkas, former president of the union that represents employees at the immigration agency. “It’s a huge albatross around our necks.”
What’s interesting is that those involved seem to be trying very hard to implement a process which puts data in a database, displays information online, and reduces the need for paper, the stuff from dead trees.
The article suggests that one vendor (IBM) was involved in the process:
IBM had as many as 500 people at one time working on the project. But the company and agency clashed. Agency officials, for their part, held IBM responsible for much of the subsequent failure, documents show.
The company’s initial approach proved especially controversial. Known as “Waterfall,” this approach involved developing the system in relatively long, cascading phases, resulting in a years-long wait for a final product. Current and former federal officials acknowledged in interviews that this method of carrying out IT projects was considered outdated by 2008.
Several observations are warranted, but these are unlikely to be particularly life affirming:
- The management process is usually not focused on delivering a functioning system. The management process is designed to permit billing and cause meetings. The actual work appears to be cut off from these administrative targets of having something to do and sending invoices for services rendered.
- Like other interesting government projects such as the upgrading of the IRS or the air traffic control system, figuring out what to do and how to do it are sufficiently complex that everyone involved dives into details, political considerations, and briefings. Nothing much comes from these activities, but they constitute “work” so day to day, week to week, month to month, and year to year process becomes its own goal. The new system remains an abstraction.
- No one working on a government project, including government professionals and contractors, has responsibility to deliver a solution. Projects become a collection of fixes, which are often demonstrations of a small scale function. The idea that a comprehensive system will actually deliver a function results in software quite similar to the famous HealthCare.gov service.
I am tempted to mention other US government initiatives. I won’t. Shift to the United Kingdom. That country has been working on its National Health Service systems for many years. How similar have been the initiatives to improve usability, functionality, and various reductions. These have ranged from cost reduction to waiting time reduction. The project is not that different from US government efforts.
What’s the fix?
Let me point out that digitization, computerization, and other Latinate nominatives are fated to remain in a state of incompletion. How can one finish when when the process, not the result, is the single most important objective.
I heard that some units of Angela Merkel’s government are now using traditional typewriters. Ah, progress.
Stephen E Arnold, November 11, 2015
Watson Weakly: IBM Case Manager Includes Watson
October 28, 2015
I read “IBM Case manager Provides Tailored Content Management.” For those of you not keeping track of IBM’s product line, may I share with you that IBM Case Manager is a wrapper perched on top of FileNet?
Remember FileNet?
In 2006, yep, nine short years ago, IBM purchased FileNet for $1.6 billion. I stumbled literally upon FileNet when I was doing some work for one of the financial outfits who once found me mildly amusing. FileNet was founded in 1982. The company scanned checks and other documents, stored the images on optical discs, and made the contents searchable—sort of.
The hardware was pure 1982: Big machines, big scanners, and lots of humans doing tasks. Over time, FileNet updated its human dependent system to become more automated. FileNet was a proprietary set up and required lots of engineers, programmers, and specialists to set up the system and keep it humming along at 2 am when most back office operations were performed in the 1980s.
Enter IBM.
FileNet is still available. But IBM has created applications which are designed to make the system more saleable in today’s market. The IBM Case Manager includes FileNet and workflow, collaboration, and compliance tools. You can now run FileNet from a mobile device. When I first stubbed my toe on a giant scanning system, folks were using nifty green screens. Progress.
The 1980s are gone. IBM now delivers a case manager. Keep in mind, gentle reader, that case management is a solution keenly desired by many in law enforcement and certain intelligence disciplines. The US government continues to search for a case management system that meets its various units’ requirements. I would suggest that some of the products available as commercial off the shelf software do not do the job. But let’s focus on what the article reveals about IBM Case Manager.
The article points out that IBM Case Manager includes these components:
- A unified interface. Always good for a busy user.
- A data capture and parsing module.
- Information life cycle tools. The idea is that one can comply with Federal regulations and make darned sure information has a “kill on” date.
- A content manager which “provides features for capture, workflow, collaboration, and compliance on both mobile and desktop [devices].”
- SoftLayer which makes IBM Case Manager a cloud application. But licensees can install the system on premises or use a hybrid approach which can be exciting for engineers and investigators.
But the big news in the article is contained in this passage, which I circled in dollar bill green:
Analytics, which is a set of packages that includes IBM Watson, which can glean insight from business content, present that insight in the right context, and identify patterns and trends.
I did not know that IBM Case Management included Watson. My understanding was that Watson was the new IBM; therefore, Watson includes IBM Case Management.
Perhaps this is a minor point, but since we are dealing with technology from the 1980s, open source code, and wrappers which add a range of user experience features, I think getting the horse and cart lined up correctly can be helpful at times.
Another remarkable revelation in the article is that IBM Case Manager is for “enterprise of all sizes.” There you go. The local Harrod’s Creek, Kentucky, car wash and grocery can use IBM Case Management with Watson to help the proprietors deal with their information demands.
May I suggest that FileNet, regardless of its name, is appropriate for outfits like banks, hospitals, and meaty government agencies.
I also learned:
IBM Case Manager can be used to monitor social media sites to get a reading on public sentiment on a given subject or brand. Case Manager can also provide collaboration with social media platforms.
I have updated my Watson files and noted that IBM Case Manager includes Watson or is it the other way around?
Stephen E Arnold, October 28, 2015
Netflix: Slow Subscriber Growth Due to Things Netflix Cannot Control
October 15, 2015
I like the idea of blaming what some MBA whiz called exogenous events. The idea is that hapless, yet otherwise capable senior managers, are unable to deal with the ups and downs of running a business. In short, an exogenous event is a variation on “it’s not my fault,” “there’s little I can do,” and “let’s just muddle forward.” The problem is that hunting for scapegoats is not a way to generate revenue. Wait. One can raise subscription fees.
I read “Netflix Is Blaming Slow US Growth on the Switch to Chip-Based Credit Cards.” The write up references a letter, allegedly written by the Netflix top dog. I noted this passage:
In his letter to investors, Netflix CEO Reed Hastings partially blamed America’s recent switch to chip-enabled credit cards. As credit card companies send new cards to their customers, some have been issuing new numbers, as well. And if people forget to update their credit card number with Netflix, they can’t pay their bill and become what Hastings called “involuntary churn.”
I like that involuntary churn. I remember working on a project for a telecommunications company in which churn was a burr under the saddle of some executives. Those pesky customers. Darn it.
The write up ignores the responsibility of management to deal with exogenous events. When a search system fails, is it the responsibility of customers to fix the system. Nah, users just go to a service that seems to work.
I interpreted this alleged explanation and the article’s willingness to allow Netflix’s management to say, in effect, “Hey, this is not something I can do anything about.” If not the top dog, who takes responsibility? Perhaps the reason is not chip enabled credit cards? Perhaps users are sending Netflix a signal about sometimes unfindable content, clunky search, and a lack of features. Not everyone is a binge watcher. Some folks look for Jack Benny films or other entertainment delights. When these are not available, perhaps some look elsewhere. See and you shall find often delivers the goods.
Stephen E Arnold, October 15, 2015
-
- Subscribe to Beyond Search
Feature archive
News archive
-
