A Statistics Rebellion? One Can Only Hope
March 21, 2019
Yesterday I mentioned to a reporter than most smart software is “right” somewhere between 50 to 80 percent of the time. The reporter asked, “Does that mean results are incorrect half to one third of the time?”
My answer, “Probably worse.”
The reporter changed the subject. My hunch is that the hyperbole about the accuracy of smart software suggests that the systems are better than a human. Some may be better at some specific tasks.
In many cases, the number crunching chops down what a human must examine. In an age of data, chopping down what one has to examine is a very important task. For applications like online advertising, 70 percent accuracy is close enough to keep the advertiser semi happy and spending money to reach a target. For other applications like where will a bad actor commit a crime, the game is “close enough for horseshoes.”
Why talk about numbers? My observations, with which you are invited to disagree, are a prelude to my recommending that you read “Scientists Rise Up Against Statistical Significance.” Here a passage I underlined:
In 2016, the American Statistical Association released a statement in The American Statistician warning against the misuse of statistical significance and P values. The issue also included many commentaries on the subject. This month, a special issue in the same journal attempts to push these reforms further. It presents more than 40 papers on ‘Statistical inference in the 21st century: a world beyond P < 0.05’. The editors introduce the collection with the caution “don’t say ‘statistically significant’”. Another article with dozens of signatories also calls on authors and journal editors to disavow those terms. We agree, and call for the entire concept of statistical significance to be abandoned.
What if one is using a system which bakes in statistical procedures and locks them away from users? What if those procedures are introducing errors?
Tough questions for vendors of smart software.
Stephen E Arnold, March 21, 2019
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
