Disadvantaged Groups and Simple Explanations
June 16, 2022
Bias in machine learning algorithms is a known problem. Decision makers, like admissions officers for example, sometimes turn to explanation models in an effort to avoid this issue. These tools construct simplified approximations of larger models’ predictions that are easier to understand. But wait, one may ask, aren’t these explanations also generated by machine learning AI? Indeed they are. MIT News examines this sticky wicket in its piece, “In Bias We Trust?” A team of MIT researchers checked for bias in some widely used explanation models and, low and behold, they found it. Writer Adam Zewe tells us:
“They found that the approximation quality of these explanations can vary dramatically between subgroups and that the quality is often significantly lower for minoritized subgroups. In practice, this means that if the approximation quality is lower for female applicants, there is a mismatch between the explanations and the model’s predictions that could lead the admissions officer to wrongly reject more women than men. Once the MIT researchers saw how pervasive these fairness gaps are, they tried several techniques to level the playing field. They were able to shrink some gaps, but couldn’t eradicate them. ‘What this means in the real-world is that people might incorrectly trust predictions more for some subgroups than for others. So, improving explanation models is important, but communicating the details of these models to end users is equally important. These gaps exist, so users may want to adjust their expectations as to what they are getting when they use these explanations,’ says lead author Aparna Balagopalan, a graduate student in the Healthy ML group of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL).”
Or we could maybe go back to using human judgment for decisions that affect the lives of others? Nah. See the article for details of how the researchers evaluated explanation models’ fidelity and their work to narrow (but not eliminate) the gaps. Zewe reports the team plans to extend its research to ways fidelity gaps affect decisions in the real world. We look forward to learning what they find, though we suspect we will not be surprised by the results.
Cynthia Murrell, June 16, 2022
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
