The Wall Street Journal does a review of new research on algorithmic bias. If you’re following this blog the articles they reference are not new, but it’s nice to see a prominent news site covering an issue that perhaps gets less coverage than it should.
There’s an interesting anecdote about cameras that fail to do face recognition on non-white faces:
Back then, he built a software program that would comb through images online and try to detect objects in them. The program could easily recognize white faces, but it had trouble detecting faces of Asians and blacks. Mr. Viola eventually traced the error back to the source: In his original data set of about 5,000 images, whites predominated.
The problem got worse as the program processed images it found on the Internet, he said, because the Internet, too, had more images of whites than blacks. The software’s familiarity with a larger set of pictures sharpened its knowledge of faces, but it also solidified the program’s limited understanding of human differences.
To fix the problem, Mr. Viola added more images of diverse faces into his training data, he said.
This shows why efforts like the World White Web, although a tad quixotic, might still be useful.
Algorithms are producing profiles of you. What do they say? You probably don’t have the right to know
Aeon has an extended piece by Black Box Society author Frank Pasquale where he describes not a future where algorithmic bias negatively impacts society, but a rather present where this already happens:
Job applicants at Walmart in the US and other large companies take mysterious ‘personality tests’, which process their responses in undisclosed ways. And white-collar workers face CV-sorting software that may understate, or entirely ignore, their qualifications.
The CEO of ZestFinance has proudly stated that ‘all data is credit data’ – that is, predictive analytics can take virtually any scrap of information about a person, analyse whether it corresponds to a characteristic of known-to-be-creditworthy people, and extrapolate accordingly. Such data might include sexual orientation or political views.
Algorithms and Bias: Q & A with Cynthia Dwork
The NYT blog The Upshot does a Q&A with Cynthia Dwork on algorithmic bias. She hits the nail on the head right in the first question:
Q: Some people have argued that algorithms eliminate discrimination because they make decisions based on data, free of human bias. Others say algorithms reflect and perpetuate human biases. What do you think?
A: Algorithms do not automatically eliminate bias. Suppose a university, with admission and rejection records dating back for decades and faced with growing numbers of applicants, decides to use a machine learning algorithm that, using the historical records, identifies candidates who are more likely to be admitted. Historical biases in the training data will be learned by the algorithm, and past discrimination will lead to future discrimination.