The Unilateralist’s “Curse” Is Mostly Good

(Previously on Less Wrong: "The Heckler's Veto Is Also Subject to the Unilateralist's Curse" [LW · GW])

If 90% of the population believes in Religion X, and they all get their opinions on Topic Y from Guru Z, then a naive view of the unilateralist's curse will say that since Guru Z's opinion on Topic Y is shared by a majority of the population, action in line with the Guru's opinion is "multilateral". Any action Guru Z disagrees with is "unilateral", even if the remaining 10% of the population worship a broad variety of gods, follow a broad variety of gurus, and are all in agreement that Guru Z happens to be wrong in this case. (BTW see my recent comment [LW(p) · GW(p)] on ideological Matthew effects.)

One of Phil Tetlock's forecasting secrets is something he calls "extremization": Find forecasters who don't correlate well with each other in general, and if they actually end up agreeing on something, become even more certain it's the case. In Bayesian terms, observing conditionally independent evidence counts for more.

The frustrating thing about unilateralist's curse dialogue is that in order to develop the necessary diversity of perspectives to extremize, you need people to be seriously considering the possibility that the group is wrong, probably playing devil's advocate, at the very least not feeling pressure to conform intellectually. But that kind of intellectual noncompliance is the very thing a person concerned with the UC will want to clamp down on if they want to prevent unilateral action.

I think the thing to do is separate out talking and acting: People should be developing their own handcrafted models of the world to make extremization possible, but they should be using ensembles of world models to choose important actions.

Think unilaterally, act multilaterally.

I think this is a case of unilaterists curse being good until it suddenly isn't. We are entering a phase of technology which is fundamentally different from before. If we embrace unilaterism we need to do it based on an understanding of our current situation, not just past history.

Yeah, my impression as that the Unilateralist's Curse as something bad mostly relies on the assumption that everyone is taking actions based on the common good. From the paper,

Suppose that each agent decides whether or not to undertake X on the basis of her own independent judgement of the value of X, where the value of X is assumed to be independent of who undertakes X, and is supposed to be determined by the contribution of X to the common good...

That is to say-- if each agent is not deciding to undertake X on the basis of the common good, perhaps because of fundamental value differences or subconscious incentives, there is no longer an implication that the unilateral action will be chosen more often than it ought to be.

I believe that example of Galileo and the Pentagon Papers are both cases where the "common good" assumption fails. In the context of Galileo, it's easy to justify this -- I'm an anti-realist and the Church does not share my ethical stances so they differ in terms of the common good. In the context of the Pentagon papers, one has to grapple with the fact that most of the people choosing not to leak them were involved not-very-common-good-at-all actions that those papers revealed.

The stronger argument for the Unilateralist's Curse for effective altruism in particular is that, for most of us, our similar perceptions of the common good are what attracted us in the first place (whereas, in many examples of the Unilateralist Curse, values are very inhomogenous). Also because cooperation is game-theoretically great, there's a sort of institutional pressure for those involves in effective altruism to assume others are considering the common good in good-faith.

But don't we also have lots of evidence of the poor performance under a regime of decision by committee?

I might also be tempted to say one needs to consider all the social/public choice literature that suggests masses really are pretty bad at getting much right -- in other words, groups of people cannot generally come to a rational choice. (And, yes I get that can seem at odds with things like prediction markets but I think the key there is the institutional setting of all the separate individual choices).

Perhaps also relevant is an old paper one of my professors once mentions (all those years ago ;-) called something like The Chairman's Problem. I never read it but as I recall from the context it was really about how to overcome the host of problems everyone knows about (cycles, agenda settings, effect of iterative pairwise selection of multiple alternatives...). Essentially it seems the way the problem was cast, the Chairman ultimately had to be that unilateralist. Now, that doesn't completely go against the argument as I understand it being made -- the chairman clearly is listening to the committee members. But the point was, the chairman could not really rely on the committee to actually give a good solution (or take action).