Twitter user Dan Hirschman posts another example of search results that are — let’s just say — questionable:
Aside from the problematic search results (and again, this is an image search), what’s interesting about this is the predictable way in which the discussion unfolds.
There’s a standard pattern of discourse that I see when talking about bias in algorithms (I’ll interject commentary in between the elements).
It starts with the example
Which is usually quickly followed by the retort:
It’s true that if we interpret “racist algorithm” as “algorithm that cackles evilly as it intentionally does racist things”, then an algorithm is not racist. But the usage here is a Turing-test sense i.e the algorithm does something that would be considered racist if a human did it. At least in the US, it is not necessary (even for humans) to show racist intent in order for their actions to be deemed discriminatory; this is essentially the difference between disparate treatment and disparate impact.
The retort is often followed by algorithms don’t discriminate, people discriminate:
and also garbage in, garbage out:
This is strictly speaking correct. One important source of bias in algorithms is the training data it’s fed, and that of course is provided by humans. However, this still points to a problem in the use of the algorithm: it needs better training examples, and a better learning procedure. We can’t absolve ourselves of responsibility here, or the algorithm.
But eventually, we always end up with data is truth:
There is a learned helplessness in these responses. The sentiment is, “yes there are problems, but why blame the helpless algorithm, and in any case people are at fault, and plus the world is racist, and you’re trying to be politically correct, and data never lies, and blah blah blah”.
Anything to actually avoid engaging with the issues.
Whenever I’ve had to talk about bias in algorithms, I’ve tried be careful to emphasize that it’s not that we shouldn’t use algorithms in search, recommendation and decision making. It’s that we often just don’t know how they’re making their decisions to present answers, make recommendations or arrive at conclusions, and it’s this lack of transparency that’s worrisome. Remember, algorithms aren’t just code.
What’s also worrisome is the amplifier effect. Even if “all an algorithm is doing” is reflecting and transmitting biases inherent in society, it’s also amplifying and perpetuating them on a much larger scale than your friendly neighborhood racist. And that’s the bigger issue. As Zeynep Tufekci points out
That is to say, even if the algorithm isn’t creating bias, it’s creating a feedback loop that has powerful perception effects. Try doing an image search for ‘person’ and look carefully at the results you get.