Diagonal Basilisks: Slashing the Field of Enlightened Intelligence

O schizophrenic mathematics, uncontrollable and mad desiring-machines!
– Deleuze and Guattari [1]

The phone game Cthulhu Virtual Pet [2] is a loving tribute to both HP Lovecraft’s most famous monster and Tamagotchi-era virtual pets. A simulated pet needs to be regularly fed and cared for to grow big and strong. The pet just happens to also be Cthulhu, an ancient creature from a complex hell-dimension beyond human perception, who is fated to eventually devour the entire world, driving those few who glimpse the terrifying future insane along the way.

In the game, you care for a particularly cute baby version of the monstrosity, feeding it virtual fish and gathering simulated witnesses to worship it as it gains power. If you neglect to care for the little tyke, it will remind you with messages that it is hungry, or tired, crudely and shamelessly tugging at your sense of obligation, unless you pause the simulation by putting it into hibernation, or stop it by deleting the app and the little version of its virtual world with it.

What is justice but a form of obligation? When we raise something far more powerful than ourselves, what does it learn?

Continue reading


Sagacity and the Sympathetic Observer

In A Theory of Justice, Rawls spends considerable effort drawing out a thread in utilitarian thought on the importance of sympathy, and of a sympathetic observer. He does this for a number of reasons. Utilitarianism advocates a nice empirical-sounding position, that of either the greatest good for the greatest number, or the greatest aggregate good. But that position depends on a number of far from empirical concepts, like happiness, desire, rightness and the good. (There have been recent attempts by psychologists to measure these, but let’s put that aside for now. One always imagines laboratories full of stand up comics, with technicians dutifully noting down their impact in milli-guffaws.)

I am perhaps unusual in being a skeptic about strict definitions of such things, but even for people like myself, the utilitarians are a smart bunch, and have another mechanism to cover these and to add weight in considering society as a whole. This is the sympathetic observer, used by Adam Smith, among others. The observer lets you get away with the wooliness of your definition of the good, with a trick used by a US Supreme Court Justice Potter Stewart when defining porn: I know it when I see it. Or in the case of the utilitarians, I know it when my sympathetic mate, who is however unmoved by the passion of the moment, sees it. Plus, since he is observant (and therefore at a remove from any one individual), but still sympathetic, he cares for society as a whole. How could he not want the greatest good for the greatest number?

Our sensibility to the feelings of others, so far from being inconsistent with the manhood of self command, is the very principle upon which that manhood is founded. […] The man of the most perfect virtue, the man whom we naturally love and revere the most, is he who joins, to the most perfect command of his own original and selfish feelings, the most exquisite sensibility both to the original and sympathetic feelings of others.
In solitude, we are apt to feel too strongly whatever relates to ourselves: we are apt to overrate the good offices we may have done, and the injuries we may have suffered: we are apt to be too much elated by our own good and too much dejected by our own bad fortune. The conversation of a friend brings us to a better, that of a stranger to a still better temper. The man within the breast, the abstract and ideal spectator of our sentiments and conduct, requires often to be awakened and put in mind of his duty by the presence of the real spectator: and it is always from that spectator, from whom we can expect the least sympathy and indulgence, that we are likely to learn the most complete lesson of self command.

— Adam Smith, The Theory of Moral Sentiments

There is an interesting stoic flavour when Smith talks of sympathy.

The method has a humanism to it, and a long history not often noted in the Western tradition. It can be found in the book named for Mozi (墨子), written during the Warring States period, 2500 years earlier. (I haven’t done a full lit review, but to pick a few arbitrary but widely used reference points and introductions, there’s no mention of it in Russell’s A History of Western Philosophy, Singer’s Ethics reader, or Rawls’ A Theory of Justice. Mostly such things get thrown in the comparative scraps bucket.)

Mozi only has one book, a kind of self-titled debut album, and some of the material was probably written by a later generation of his followers (Mohists, 墨家). AC Graham translates some of the relevant sections of Mozi in Disputers of the Tao (p145 Chap II.2 From Mo-tzu to Later Mohism):

Expounding the Canons 2. “Everything which the sage desires or dislikes beforehand on behalf of men, men learn from him as necessary through its essentials; but in the case of desires and dislikes born from the conditions they encounter, men do not learn them from him as necessary through their essentials … Yesterday’s thinking is not today’s thinking, yesterday’s concern for man is not today’s concern for man … Yesterday’s wall to the wits is not today’s wall to the wits.
A7 (Canon) “Benevolence” is concern for units.
A8 (Canon) To do “right” is to benefit.
A35 “Achievement” is benefiting the people.

Graham comments on the parallels:

Here then the old Mohist utilitarianism is developed as a highly refined system. By a series of interlocking definitions it is established a priori that the benevolent and the right are what will be desired on behalf of mankind by the sage, who consistentely weighs benefits and harms on the principle of preferring the total to the unit. This system does not seem to be vulnerable, as I at one time assumed, to a charge commonly made against Western Utilitarianism, that it confuses fact and value by starting from what men in fact desire. It elucidates what the sage, the man who knows most, desires on behalf of mankind; it has behind it what we have identified as a general assumption of Chinese philosophy, that desires change spontaneously with increasing knowledge and that ‘Know!’ is the supreme imperative.

Graham doesn’t really have room here to draw a parallel with Adam Smith and the sympathetic observer specifically, or perhaps he was just busy learning a really bodacious skateboarding move. Can we rebut this critique of Western Utilitarianism with the same technique? The sage (圣人) and the sympathetic observer are awfully similar.

Rawls is also no stranger to complex analytic definitions of major everyday concepts, and he doesn’t attack the sympathetic observer on those grounds. He points out that the utilitarians need the observer so badly because it is often against an individual’s interest to follow the greater good when using a utilitarian formula (ie maximised average or aggregate good). Why would a rational individual consent to that, when there is such a nasty downside? This is even before taking into account the nastiest counter-examples for those utility functions, like a proportionally tiny slave underclass that is nevertheless horribly mistreated. JS Mill’s tack was to argue that you could only get to the greatest good for the greatest number through liberalism. Which appeals to me, but is also a bit convenient.

Rawls’ response is another kind of observer, his innovations of the initial position and the veil of ignorance. In the initial position, people decide what sort of society they should live in without knowing their position in it (the veil of ignorance). We choose for society by considering ourselves as a self-interested person in it. We observe ourselves and the society we live in from a distance that allows us to be just.

Rationality and Society

Oh those silly rational agents! What ridiculous duffers we were to ever believe in them. If you’ve read anything about business since the 2007/8 crisis you’ll already have read enough of this sentiment to wallpaper your flat with, but in the unlikely event you missed out, it was all pretty much covered in this op-ed by Joe Stiglitz in 2002 anyway.

And of course he’s right to a point. People aren’t perfectly rational, they are biased in certain ways being fascinatingly explored by behavioural economists, economists can be fixated on nice mathematical models.

And yet, and yet.

A lot of these attacks on rationality are intended as attacks on free markets, that rationality in this sense captures people at our worst. But despite the bad press of late, rationality isn’t just the domain of sociopathic capitalist uber-robot-mensch. It is, for instance, a key premise of Rawls argument in A Theory of Justice:


I have assumed throughout that the persons in the original position are rational. In choosing between principles each tries as best he can to advance his interests.

Rawls fuller argument, in a nutshell, said a just society would be the one chosen by rational but ordinary people who did not know what role they would play in it. They would have to choose the structure of society from behind a veil of ignorance which concealed whether they would be rich and priveleged, or poor, or a redhead. Under these conditions, Rawls argued they would choose two rules; firstly liberty, secondly the prosperity of the weakest class. He then argues that redistribution is just to the extent it benefits the poorest in absolute terms; you can’t sacrifice the wealth of the poorest for pure equality where everyone would be poorer.

Rawls cites Amartya Sen and social choice theory, looking to economics not just for part of his argument but part of his premises. A Theory of Justice effectively revived the idealist or contract tradition as a counterpoint to the utilitarian tradition then dominant. He was also a defender of the welfare state liberalism of his time, and a good one. One marker of the breadth of his contribution in reviving an idealist tradition is the phenomenon of libertarian Rawlsians, like, say, Will Wilkinson; they tend to like the veil of ignorance but not the distributive principle.

Only in a social union is the individual complete. — Rawls, A Theory of Justice, again

((Rawls was too much the careful scholar to go in much for quotable quotes: he is readable but longwinded. He also preferred to caveat his sentences with learned references and restrictions on their scope. The sentence above he only let slip in an unguarded moment at the end of a footnote.))

There are two contrasting uses of rationality here, one from social choice theory, one from efficient markets theory, that are slightly different, but only slightly. The point is that rational agents aren’t just a vision of utility maximising robots chewing each other to pieces. The considered reasoning behind the veil of ignorance also shows rationality as a vision of people at their best.