Trust AI? Obvious to Those Who Do Not Want to Think Too Much

September 16, 2024

This essay is the work of a dumb dinobaby. No smart software required.

Who wants to evaluate information? The answer: Not too many people. In my lectures, I show a diagram of the six processes an analyst or investigator should execute. The reality is that several of the processes are difficult which means time and money are required to complete the processes in a thorough manner. Who has time? The answer: Not too many people or organizations.

What’s the solution? The Engineer’s article “Study Shows Alarming Level of Trust in AI for Life and Death Decisions” reports:

A US study that simulated life and death decisions has shown that humans place excessive trust in artificial intelligence when guiding their choices.

Interesting. Perhaps China is the poster child for putting “trust” in smart software hooked up to nuclear weapons? Fortune reported on September 10, 2024, that China has refused to sign an agreement to ban smart software from controlling nuclear weapons.

Yep, I trust AI, don’t you? Thanks, MSFT Copilot. I trusted you to do a great job. What did you deliver? A good enough cartoon.

The study reported in The Engineer might be of interest to some in China. Specifically, the write up stated:

Despite being informed of the fallibility of the AI systems in the study, two-thirds of subjects allowed their decisions to be influenced by the AI. The work, conducted by scientists at the University of California – Merced.

Are these results on point? My experience suggests that not only do people accept the outputs of a computer as “correct.” Many people when shown facts that contradict the computer output defend the computer as more reliable and accurate.

I am not quite such a goose. Machines and software generate errors. The systems have for decades. But I think the reason is that the humans with whom I have interacted pursue convenience. Verifying, analyzing, and thinking are hot processes. Humans want to kick back in cool, low humidity environments and pursue the least effort path in many situations.

The illusion of computer accuracy allows people to skip reviewing their Visa statement and doubting the validity of an output displayed in a spreadsheet. The fact that the smart software hallucinates is ignored. I hear “I know when the system needs checking.” Yeah, sure you do.

Those involved in preparing the study are quoted as saying:

“Our project was about high-risk decisions made under uncertainty when the AI is unreliable,” said Holbrook. “We should have a healthy skepticism about AI, especially in life-or-death decisions. “We see AI doing extraordinary things and we think that because it’s amazing in this domain, it will be amazing in another. We can’t assume that. These are still devices with limited abilities.”

These folks are not going to be hired to advise the Chinese government I surmise.

Stephen E Arnold, September 16, 2024

Written by Stephen E. Arnold · Filed Under AI, Government, News

Comments

Got something to say?

Search the site
Subscribe to Beyond Search
Feature archive
News archive

Stephen E. Arnold monitors search, content processing, text mining and related topics from his high-tech nerve center in rural Kentucky. He tries to winnow the goose feathers from the giblets. He works with colleagues worldwide to make this Web log useful to those who want to go "beyond search". Contact him at sa [at] arnoldit.com. His Web site with additional information about search is arnoldit.com.