Of Monsters and Utilitarians

On the impossibility of interpersonal comparison of utility

There is a philosophical problem that most people seem to ponder over at some point during childhood: is my experience of the colour red the same as yours? The problem is that I cannot deduce from observation how someone else experiences colours. All I can observe is that someone can recognise and distinguish different colours, which presumably indicates they have different experiences associated to different colours. But that leaves open the question of how those experiences compare to my own. What if my experience of red is equivalent to someone else’s experience of yellow and vice versa? That would still explain our individual abilities to distinguish those colours.

This problem is not unique to colours; it applies to all experiences. For instance, how could I compare my own experience of happiness and satisfaction to someone else's? All I can observe is an individual's relative satisfaction. E.g. I can observe that Alice chooses to buy product A instead of product B, implying she is more satisfied with A than B – in which case, A is said to have a greater utility than B.

Simply knowing that Alice prefers option A over option B does not allow us to determine her exact level of satisfaction, or absolute utility, with option A. This is because there can be multiple levels of absolute utility for A and B that would result in Alice preferring A over B. For example, let's say Alice actually receives twice the amount of absolute satisfaction from both options A and B than we initially assumed. Since the utility for A and B have increased by the same factor, and since Alice initially preferred A over B, Alice will still prefer option A over B even with this increase in absolute utility. Therefore, if we were to use this new, higher utility function, we would still arrive at the same conclusion that Alice prefers A over B. This illustrates that we cannot infer Alice's absolute utility solely from her preferences between A and B.

Since only relative utility is observable, one can never be sure how much absolute utility someone has gained or lost. The idea that it is impossible to objectively compare the level of happiness, satisfaction, or utility experienced by different individuals is called the impossibility of interpersonal comparison of utility and was first proposed by economist and philosopher Vilfredo Pareto in the early 20th century. 

It is hard to appreciate how strange the world would be if we could make interpersonal comparisons of utility. In that case, it would be possible for someone to claim truthfully, 'I want you to clean my room more than you dislike cleaning it', implying it would be immoral not to clean his room. Or, more extravagantly, a being could claim that all of society's resources are best given to it because its utility for those resources is much greater than that of any human. Such a being is what Robert Nozick named a utility monster in his book "Anarchy, State, and Utopia." The utility monster is so much better off than everyone else that it can justify taking more than its fair share of resources or benefits, even if it harms others.

Nozick used the concept of a utility monster to argue against utilitarianism – i.e. the idea that we should strive to maximise overall happiness or utility in society. He argued that if a utility monster existed, it could theoretically justify taking all of society's resources, leaving everyone else with very little. Yet, this would still be considered a "good" outcome under utilitarianism because the utility monster's happiness would be so great. The concept of a utility monster has recently gained relevance because some fear that AGI could have mental states that are far better in quality than that of any human, making it a utility monster.

However, it is precisely because of the impossibility of interpersonal comparisons of utility that utility monsters cannot exist: the monster's absolute utility would be just as inaccessible to us as the utility of any other person. We could only know its relative utilities, so its claim that we should transfer all of society’s wealth to it is baseless. Although we must remain agnostic about how to compare the monster's utility to our own, we do know how to increase both its and our utility: namely, by trading with it.