Racist Algorithms Sure but Chess?
March 8, 2021
When training software to hunt for offensive speech, some fine-tuning is to be expected. Science Magazine reports that “AI May Mistake Chess Discussions as Racist Talk.” When popular YouTube chess player Antonio Radi? found his account blocked for “harmful and dangerous” content, researchers at Carnegie Mellon University’s Language Technologies Institute (LTI) suspected YouTube’s algorithm may have been confused by the language of “black vs. white” used in discussions of the game. (Radi?’s account was reinstated within 24 hours.) The brief write-up tells us:
“To see if this was feasible, [LTI project scientist Ashiqur R.] KhudaBukhsh and Rupak Sarkar, an LTI course research engineer, tested two state-of-the-art speech classifiers — a type of AI software that can be trained to detect indications of hate speech. They used the classifiers to screen more than 680,000 comments gathered from five popular chess-focused YouTube channels. They then randomly sampled 1,000 comments that at least one of the classifiers had flagged as hate speech. When they manually reviewed those comments, they found that the vast majority — 82% — did not include hate speech. Words such as black, white, attack and threat seemed to be triggers, they said…. “As with other AI programs that depend on machine learning, these classifiers are trained with large numbers of examples and their accuracy can vary depending on the set of examples used. For instance, KhudaBukhsh recalled an exercise he encountered as a student, in which the goal was to identify ‘lazy dogs’ and ‘active dogs’ in a set of photos. Many of the training photos of active dogs showed broad expanses of grass because running dogs often were in the distance. As a result, the program sometimes identified photos containing large amounts of grass as examples of active dogs, even if the photos didn’t include any dogs.”
And an AI trained with few examples of chess discussions may identify passages as offensive, even if the conversation doesn’t include any racists. As KhudaBukhsh observes, we heard about Antonio Radi?’s experience because he is a relatively high-profile YouTube creator. How many lesser-known chess enthusiasts have been similarly shut down on that platform or others? We also wonder what similar mistakes algorithms might be making with other topics. Ah, smart software in action.
Cynthia Murrell, March 8, 2021
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
