Coordination Surveys: why we should survey to organize responsibilities, not just predictions

Push-polling (a survey that purports to collect information, but is actually meant to change the respondents' behavior) is very clearly in the dark arts. It probably has a place in the world, but it's not about rationality, it's about manipulation.

On reflection, I endorse the conclusion and arguments in this post. I also like that it's short and direct. Stylistically, it argues for a behavior change among LessWrong readers who sometimes make surveys, rather than being targeted at general LessWrong readers. In particular, the post doesn't spend much time or space building interest about surveys or taking a circumspect view of them. For this reason, I might suggest a change to the original post to add something to the top like "Target audience: LessWrong readers who often or occasionally make formal or informal surveys about the future of tech; Epistemic status: action-oriented; recommends behavior changes." It might be nice to have a longer version of the post that takes a more circumspect view of surveys and coordination surveys, that is more optimized for interestingness to general LessWrong readers, and that is less focused on recommending a change of behavior to a specific subset of readers. I wouldn't want this shorter more direct version to be fully replaced by the longer more broadly interesting version, though, because I'm still glad to have a short and sweet statement somewhere that just directly and publically explains the recommended behavior change.

tl;dr: If this post included a section discussing push-poll concerns and advocating (at least) caution and (preferably) a policy that'd be robust against human foibles, I'd be interested in having this post in the 2019 Review Book.

I think this is an interesting idea that should likely get experimented with.

A thing I was worried about when this first came out, and still worried about, is the blurriness between "survey as tool to gather data" and "survey as tool to cause action in the respondent." 

Some commenters said "this seems like push-polling, isn't that bad?". 

I think a carefully executed version of this wouldn't be push-polling. I'm not 100% sure whether the example here counts or not. It doesn't prescribe an action. But it does prescribe a frame. There's a spectrum of push-polliness, and I say the example here is non-zero push-poll-y, even if not maximally.

I don't think it's intrinsically bad to be push-poll-y, but I think it's useful to draw distinctions between surveys designed for gathering information vs shaping actions. If a tool like this started seeing common use, we'd a) start seeing some variants of it that were push-poll-y (because humans aren't always careful), and b) I think people receiving the survey would be anxious about it being push-poll-y, which would compromise their trust in future surveys on the same topic or from the same institutions. This may be unfair, but it's the outcome I predict.

One commenter suggested flagging such a survey as a "coordination survey" rather than a "information gathering survey." This is somewhat unfair to hypothetical survey-makers who were very careful to not be a push poll. But, I think it is probably the best equilibrium.

(Flagging polls as "coordination surveys" will probably come with some consequences, where people who are already mistrustful of your cause and don't want to coordinate on it are more likely to avoid the survey in the first place. But the alternative is manipulating them into taking a survey that wasn't quite what they thought it was. I think that is worse)

I think something like this might be pretty valuable among EAs/rationalists dealing with COVID.  We clearly have a lot of people doing independent research, and a lot of it might be redundant.

I think EA/rats are mostly patting ourselves on the back RE how well we predicted COVID, and not thinking about how we could've done even better.  I think we could've done better at coordinating within these communities, e.g. in an ideal world, we might've done better (and might do better) by coordinating efforts to channel COVID-efforts into producing community-facing or public-facing reports / recommendations.  And we might've empowered some community members to be our "COVID czars" (e.g. by a combination of volunteering, voting, predicting who would do a good job, and funding them), and thus allowed many community members to spend a lot less energy on COVID.

Planned summary:

This post suggests that when surveying researchers about the future impact of their technology, we should specifically ask them about their beliefs about what actions other people will take, and what they personally are going to do, rather than just predicting total impact. (For example, we could ask how many people will invest in safety.) Then, by aggregating across survey respondents, we can see whether or not the researchers beliefs about what others will do match the empirical distribution of what researchers are planning to do. This can help mitigate the effect where everyone thinks that everyone else will deal with a problem, and the effect where everyone tries to solve a problem because they all think no one else is planning to solve it. Critch has offered to provide suggestions on including this methodology in any upcoming surveys; see the post for details.

Planned opinion:

This is a cool idea, and seems worth doing to me. I especially like that the survey would simply reveal problems by collecting two sources of information from people and checking their consistency with each other: there isn't any particular argument being made; you are simply showing inconsistency in people's own beliefs to them, if and only if such inconsistency exists. In practice, I'm sure there will be complications -- for example, perhaps the set of researchers taking the survey is different from the set of "others" whose actions and beliefs they are predicting -- but it still seems worth at least trying out.

This post introduced to me a simple way to use surveys to build common knowledge that naturally leads to action. I still feel excited about it and I think about it every time I consider making a survey.

I agree it would be good to add a note about push polling, but it's also good to note that the absence of information is itself a choice! The most spare possible survey is not necessarily the most informative. The question of what is a neutral framing is a tricky one, and a question about the future that deliberate does not draw attention to responsibilities is not necessarily less push-poll-y than one that does.

I was kind of on the edge on whether to nominate this post, but David Krueger's point convinced me that maybe if I had remembered this post more, we would have done quite a bit better on COVID, and I do think that seems really quite important. 

It seems you mismatched the names in this section

Alex: “Why worry about AI safety? It would be silly to make AI unsafe. Therefore someone will take responsibility for it.”

Bailey: “You should definitely worry about AI safety, because many people are not taking responsibility for it.”

These views are strangely compatible and therefore hard to reconcile by evidence alone. Specifically, Alice is rightly predicting that people like Bob will worry and take responsibility for safety, and Bob is rightly predicting that people like Alice are not worried.

This causes Alice and Bob...