in algorithms 613 Words

“Racist algorithms” and learned helplessness

Twitter user Dan Hirschman posts another example of search results that are — let’s just say — questionable:

Aside from the problematic search results (and again, this is an image search), what’s interesting about this is the predictable way in which the discussion unfolds.

There’s a standard pattern of discourse that I see when talking about bias in algorithms (I’ll interject commentary in between the elements).

It starts with the example

Which is usually quickly followed by the retort:

It’s true that if we interpret “racist algorithm” as “algorithm that cackles evilly as it intentionally does racist things”, then an algorithm is not racist. But the usage here is a Turing-test sense i.e the algorithm does something that would be considered racist if a human did it. At least in the US, it is not necessary (even for humans) to show racist intent in order for their actions to be deemed discriminatory; this is essentially the difference between disparate treatment and disparate impact. 

Unlike France.

The retort is often followed by algorithms don’t discriminate, people discriminate:

and also garbage in, garbage out:

This is strictly speaking correct. One important source of bias in algorithms is the training data it’s fed, and that of course is provided by humans. However, this still points to a problem in the use of the algorithm: it needs better training examples, and a better learning procedure. We can’t absolve ourselves of responsibility here, or the algorithm.

But eventually, we always end up with data is truth:

There is a learned helplessness in these responses. The sentiment is, “yes there are problems, but why blame the helpless algorithm, and in any case people are at fault, and plus the world is racist, and you’re trying to be politically correct, and data never lies, and blah blah blah”.

Anything to actually avoid engaging with the issues.

Whenever I’ve had to talk about bias in algorithms, I’ve tried be  careful to emphasize that it’s not that we shouldn’t use algorithms in search, recommendation and decision making. It’s that we often just don’t know how they’re making their decisions to present answers, make recommendations or arrive at conclusions, and it’s this lack of transparency that’s worrisome. Remember, algorithms aren’t just code.

What’s also worrisome is the amplifier effect. Even if “all an algorithm is doing” is reflecting and transmitting biases inherent in society, it’s also amplifying and perpetuating them on a much larger scale than your friendly neighborhood racist. And that’s the bigger issue. As Zeynep Tufekci points out

That is to say, even if the algorithm isn’t creating bias, it’s creating a feedback loop that has powerful perception effects. Try doing an image search for ‘person’ and look carefully at the results you get.

9 thoughts on ““Racist algorithms” and learned helplessness”

  1. I agree that there are some problematic aspects to this flow. When I have participated in it myself (https://twitter.com/scottenderle/status/709754891867168769) it has been in response to language that is fundamentally deceptive.

    In the above linked case, the initial tweet was “When people write computer programs, their biases can creep into code.” That’s just false. Under ordinary circumstances, these are programs that effectively write themselves, so the biases of the people writing the base programs almost certainly don’t matter. Their biases certainly don’t “creep into the code” — in the worst case scenario, they must be deliberately included.

    The biases that matter are those of the people producing the training data that the algorithm uses to learn. Sometimes that means other employees of the company under discussion; sometimes that means their customers; and sometimes that means the society as a whole.

    The point of saying this is not to help people evade blame, but to help target our intervention. From my perspective, it is the exact opposite of learned helplessness; it is an expression of the principle that critique alone is not enough. Calling people out as racist and insisting that they reduce their bias is not enough. We have to take active steps to reverse the bias, and that will very likely mean introducing an opposing bias at key points! There’s probably no other way to do it.

    This is a case where the “Parable of the Polygons” applies (at least loosely). In a system where bias already exists, we can entirely eliminate the bias of individuals, and still see biased outcomes. The system is “path dependent” (https://en.wikipedia.org/wiki/Path_dependence). The answer is not to give up, but to actively intervene by introducing an opposing bias. This is illustrated quite beautifully here: http://ncase.me/polygons/

    I feel you and many other people probably understand all this already. But I wanted to make it explicit. And I wonder if there’s a way to express this point in a way that communicates not learned helplessness but a need for active intervention that goes beyond critique. I’m interested in your take on this, because I do see the problems with the pattern of discussion you’re describing here, and they trouble me.

    Like

    Reply

    1. I should clarify that when I say “that’s just false,” I’m referring to machine learning programs in particular (which is what that tweet was referring to). In the case of ordinary programs, certainly bias can indeed “creep into code.” But that’s something that we are learning to guard against at the institutional level. The machine learning problem under discussion here is a more difficult problem, because we don’t know yet what the right way to guard against it is. But again, it’s probably going to involve introducing a counter-bias.

      Like

      Reply

  2. What you say is all true, but I miss a valid argument for why it’s reasonable to call algorithms “biased”.

    Also, I don’t think this problem can be solved algorithmically; for that, the algorithms would need to “know” what kind of biases are politically inopportune and which it should happily amplify.

    Liked by 1 person

    Reply

    1. I don’t see why an algorithm has to “know” what biases exists in order to be instructed to look out for them. There are humans in the loop after all.

      Liked by 1 person

      Reply

    2. All machine learning algorithms are biased, because of the no free lunch theorem. It states that “any two optimization algorithms are equivalent when their performance is averaged across all possible problems.” (https://en.wikipedia.org/wiki/No_free_lunch_theorem)

      It immediately follows that to perform better than average on a particular task, a learning algorithm has to have biases that produce good results for that task (and bad results on other tasks). The nature of the bias may have nothing obviously to do with race, class, gender, and so on. It might be a bias towards things that look like red rubber balls. But it’s still a bias.

      Like

      Reply
  3. Pingback: We Love the Internet 2016/15: The Secret rules of the internet edition | Curiously Persistent

  4. ” There are humans in the loop after all” Correct @geomblog. The excuses that these are somehow random associations determined by AI is complete nonsense. This issue needs to be address AND corrected, in code and instruction. Not just discussed and left alone to perpetuate racist ideologies. Fear is at the root of all supremacy, if we are not careful as human beings we will simply code ourselves out of extinction.

    Like

    Reply
  5. Pingback: On the (im)possibility of fairness… – Algorithmic Fairness
  6. Pingback: Being Hopeful about Algorithms – Algorithmic Fairness

Thoughts?