Search Engines: Bias, Filters, and Selective Indexing

I read “It’s Not Just a Social Media Problem: How Search Engines Spread Misinformation.” The write up begins with a Venn diagram. My hunch is that quite a few people interested in search engines will struggle with the visual. Then there is the concept that typing in a search team returns results are like loaded dice in a Manhattan craps game in Union Square.

The reasons, according to the write up, that search engines fall off the rails are:

  • Relevance feedback or the Google-borrowed CLEVER method from IBM Almaden’s patent
  • Fake stories which are picked up, indexed, and displayed as value infused,

The write up points out that people cannot differentiate between accurate, useful, or “factual” results and crazy information.

Okay, here’s my partial list of why Web search engines return flawed results:

  1. Stop words. Control the stop words and you control the info people can find
  2. Stored queries. Type what you want but get the results already bundled and ready to display.
  3. Selective spidering. The idea is that any index is a partial representation of the possible content. Instruct spiders to skip Web sites with information about peanut butter, and, bingo, no peanut butter information
  4. Spidering depth. Is the bad stuff deep in a Web site? Just limit the crawl to fewer links?
  5. Spider within a span. Is a marginal Web site linking to sites with info you want killed? Don’t follow links off a domain.
  6. Delete the past. Who looks at historical info? A better question, “What advertiser will pay to appear on old content?” Kill the backfile. Web indexes are not archives no matter what thumbtypers believe.

There are other methods available as well; for example, objectionable info can be placed in near line storage so that results from questionable sources display with latency or slow enough to cause the curious user to click away.

To sum up, some discussions of Web search are not complete or accurate.

Stephen E Arnold, March 15, 2021


DarkCyber for June 9, 2020, Is Now Available: AI and Music Composition

The DarkCyber for June 9, 2020, presents a critical look at music generated by artificial intelligence. The focus is the award-winning song in the Eurovision AI 2020 competition. The interview discusses the characteristics of AI-generated music, its impact on music directors, how professional musicians deal with machine-created music, and the implications of non-numan music. The program is a criticism of the state-of-the-art for smart software. Instead of focusing on often over-hyped start ups and large companies making increasingly exaggerated claims, the Australian song and the two musicians make clear that AI is a work in progress. You can view the video at https://vimeo.com/427227666.

Kenny Toth, June 9, 2020

Latest News

Cambridge: We Do It Huawei

Intelligence agencies are aware China has been ramping up its foreign espionage efforts, largely through civilian operatives. Now The Statesman reports, “Huawei... Read more »

September 28, 2021 | Comment

Great Moments in Modern Management: The Mailchimp Move

I like the phrase “high school science club management methods.” No one else seems to care. I spotted a exemplary management maneuver. “Mailchimp Employees... Read more »

September 28, 2021 | Comment

Life Long Learning or Else

Everyone wants to reduce stress, have “quality time”, and do the hybrid work thing with as much flexibility possibility. There’s something to fill the void.... Read more »

September 28, 2021 | Comment

Free Resource on AI for Physical Simulations

The academics at the Thuerey Group have made a useful book on artificial intelligence operations and smart software applications available online. The Physics-Based... Read more »

September 27, 2021 | Comment

US Government Procurement: A Technology Brake?

I read “Study: Pentagon Reliance on Contractors Hurt US in 9/11 Wars.” I was not certain how to process the story. Was it a blockbuster exposé or was it another... Read more »

September 27, 2021 | Comment

Telegram and Criminal Usage: Who Knew?

Why would cyber criminals and regular run-of-the-mill criminals use a message app which was able to encrypt messages, enable “transactions,” and support file... Read more »

September 27, 2021 | Comment

Yay, A Facebook Friday

Three slightly intriguing factoids about the Zuckbook. The first is a characterization of Facebook’s and the supreme leader’s time spirit: “Shame, addiction,... Read more »

September 24, 2021 | Comment

Ethics Instruction: Who Knew?

Well, this is not particularly alarming. Despite increasing concern over the harm caused by unbridled algorithms, many AI students are still not being taught ethics... Read more »

September 24, 2021 | Comment

Google: More Management of Sensitive Issues

Some MBA engineers are driven purely by greed without regard for their fellow humans. When Google formed its parent company, Alphabet Inc., they changed their company... Read more »

September 24, 2021 | Comment

NSO Group and Collateral Damage: Shadowdragon

The NSO Group has captured headlines and given a number of journalists a new beat to cover: Special service vendors. This phrase “specialized service vendors”... Read more »

September 23, 2021 | Comment

  • Archives

  • Recent Posts

  • Meta