Being Hopeful about Algorithms

I’ve been attending “think-events” around algorithmic fairness of late, firstly in Philadelphia (courtesy of the folks at UPenn) and then in DC (courtesy of the National Academy of Science and the Royal Society).

At these events, one doesn’t see the kind of knee-jerk reaction to the idea of fairness in learning that I’ve documented before. But there’s a very thoughtful critique that comes from people who’ve spent a lot of time themselves thinking and working on these topics. And it goes something like this.

Do we expect more from algorithms than we expect from people? And is that reasonable?

I first heard this critique much earlier at  a Dagstuhl meeting on this topic, when I was asked this question by H. V. Jagadish (who has a great course on ethics in data mining). It came up indirectly during discussions at the Philadelphia event (about which I hope to say something later) and was phrased in this form by Vint Cerf at the Sackler Forum.

I found myself unable to answer it convincingly. We’ve had 1000s of years to set up institutions based on humans decision making. These processes have been flawed, anfdbiased. People have made decisions with implicit and explicit bias.

Why then do we demand then that algorithms do more? Why do we demand that they account for themselves and explain themselves in ways that we don’t ask human judges to do?

I used to have an answer. I argued that algorithms speak the language of mathematics and so we need to translate all our human ideals – of ethics, fairness and justice – into a form that an algorithm could understand. But then we start talking about accountability, interpretability, how an algorithm might explain itself, and what that might even mean.

Jon Kleinberg has this analogy of a learning algorithm as this incredibly obtuse friend that you bring to a party, that you have to explain EVERYTHING to. Where the food is, what the drinks are, what people are saying, and so on. We don’t have to do this for real people because they have a vast body of prior context to work with. Indeed, this prior context is what decides how they function in the world, and is made up of all kinds of heuristics and “biasing” of the space of possible outcomes (as Joanna Bryson puts it).

So it would seem that asking an algorithm for its “audit trail” is the equivalent of asking (say) a human judge “give me the entire story of your life experiences that explains why you made this decision”.

And of course we never do this. In fact, all we really do is set out a series of guidelines and expect the judges to be more or less consistent with them. Similarly for hiring, or credit decision, or any other kind of decision making. In other words, we expect a certain degree of procedural consistency while accepting that individuals may apply discretion based on their own perspective.

So I return to the question from before. Why do we expect an automated decision making process to be any better?

There’s an optimistic take on this. We can’t expect an audit trail from a human decision maker because we don’t have the capacity to generate one. That my verdict on a dog owner might in part be due to being bitten by a dog as a child is something that I’m unlikely to be able to cogently articulate. But it is at least a little unfair that I sentence dog owners more harshly for this reason.

But if we are able to produce such an audit trail from an algorithmic decision maker we do have the hope of revealing implicit preferences and biases based on the *algorithm’s* “life experiences” aka “training data”. And so we can expect more because we have the ability to do so.

An alternate perspective on this goes as follows. We’ve built up over the decades and centuries a system of checks and balances and accountability procedures for evaluating the quality of human decision making. We have laws that require non-discrimination, we have ways to remove decision-makers who make arbitrary decisions, and we have a social structure that makes decision-makers feel a sense of responsibility for their decisions.

None of these exist for algorithmic decision-making, or realistically can. We can’t call an algorithm to account for a bad decision: ultimately all liability rests on legal persons. So the next best thing is to make an algorithm assist in the decision making process, but require transparency so that the human decision-maker can’t blame the algorithm for bad decisions “It’s not me, it’s the algorithm!”, a story that played out in Cory Doctorow’s Human Readable.

There’s a tension between “let’s use automation wherever reasonable”, and “wait. how are you shifting harm?”. We don’t want to stop the deployment of algorithms in decision-making, and frankly I doubt that one could even if one wanted to. But it’s also not unreasonable to express some caution (and perhaps some humility) when doing this. We’re not expecting perfection from automated decision-making: it’s perfectly reasonable to expect just that we can do better than human decision makers. But why not expect that as well as expect a decision that we can understand? Why essentially give up by saying “the algorithm cannot both be powerful and understandable”. To me, that’s the real failure of hope.

10 thoughts on “Being Hopeful about Algorithms

  1. I wonder if it’s more that the mode of communication used to express “why algorithm A made decision X” is unclear. Do you think, perhaps, that humans wouldn’t reasonably ask for an “entire life story” for why a judge made a decision, but rather a detailed precedent from previous and historical cases?

    Would it be reasonable, in asking an algorithm why it made decision X, to accept as explanation a list of the most influential training examples that affected its decision? It seems reasonable this can be done, even if the method to arrive at the answer is essentially alien language. The decision is still not understandable in the sense you describe, but the human evaluator can make a judgement as to whether the algorithm is behaving usefully.

    The other side of that coin is that, if you want to argue that Podiatrists are evil, it is easy enough to compile a list of Podiatrists who have committed crimes and screwed over a customer. So while we could be given an understandable reason for why an algorithm made a decision, we have to be extra careful to determine if it’s a reasonable decision despite an explanation that is a priori convincing.

    Liked by 1 person

    1. Yes this is a good point. I think it’s not yet clear what constitutes an “explanation” by algorithm. And your suggestion is a fair one.


  2. Well, human judges have been trained (at least partially) according to norms, written and unwritten, rational or not, imposed by society. This is not true for algorithms.

    About the “knee-jerk reaction to the idea of fairness in learning”: do you think this knee-jerk reaction is somewhat related to the idea that many scientists have of themselves as totally rational and unbiased agents?


  3. Has there been any research on how to tell when the data itself is biased? (That is, other than feeding data to an algorithm, then having it make decisions, then seeing that the decisions are biased.) One thing that strikes me is that “bias” in the data, from a mathematical perspective, might be the same thing as “statistical pattern,” in which case what we consider “unfair bias” has more to do with the underlying societal reasons leading to such patterns in the data, and whether or not those actually reflect reality, are fair, etc.

    I think you’re also missing something here (I’m sure you know this, just interesting that it didn’t show up in this post): the reason people expect algorithms to be more fair than humans is because of the false inference that “algorithms are logical, and therefore fairer than us illogical humans who are riddled with psychological biases.” While this may be old hat to researchers on algorithmic fairness, and I’m happy to see that this idea has been gaining traction in the public press of late, I think it’s important to remember that more public outreach on this issue is still needed.

    Liked by 1 person

    1. As for your first point, yes, there’s research on detecting bias in data (and I’ve done a bit of it!), and there is also research that points out that a lot of what algorithms absorb as “bias” is really societal skew – particularly in the case of language representation.

      Regarding the second point, that’s an interesting contradiction you’re postulating, that if I expect algorithms to be more fair than humans, I’m basing it on the false inference that you outline. I don’t think that’s why I expect that, and that’s why I didn’t mention it here. But it’s an interesting take.


      1. Sorry, I didn’t necessarily mean to imply that’s why you thought algorithms would/should/could make fairer decisions, but rather that I think that is the public perception of why algorithmic decisions would be fairer than human ones. (Though now I am curious: why do you think algorithms would/should/could make fairer decisions than people?)


      2. I think algorithms shift the bias in ways that we don’t fully understand. They can definitely eliminate certain kind of human bias (hungry judges?) but can amplify other kinds of human bias.



Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s