The Ideological Turing Test

A while back, David Brin wrote up a proposal for "disputation arenas".

An interesting part of the competition/argument process:

Phase Four: The Paraphrasing Challenge

Here is a unique, but crucial phase. After both manifestos are declared logically usable, a distinct period -- say a month -- will be given to each side so they may paraphrase the other side's position.

This step aims to ensure that each party has actually read and understood where the other one stands, so they aren't simply shouting past each other at chimeric caricatures. Paraphrasing is hard to do when you've spent years demonizing the opposition, calling them venal or stupid and dismissing their concerns. Success at paraphrasing will be seen as a way of winning credibility. It means "I do understand my opponents... so my disagreement with them is well-informed."

Each side may lace their formal paraphrasing with asterisks and footnotes asserting that the statements they are describing are idiotic. That's fine. But if they fail to depict their opponents' point of view in a manner that the jury finds at least generally accurate, they will be disqualified.

This seems like a very fun game, but I'm not sure how much merit it has as a method of determining which side in a debate is the most informed.

Much like the actual Turing Test, it also seems vulnerable to hard-ball inquisitorial tactics. There are probably quite a few questions you could ask which lie outside the ostensible scope of the debate, but which would correlate strongly with a user's true beliefs, or require faculties the respondent simply doesn't have in order to answer satisfactorily. I'm pretty sure if I asked a fake-atheist to name ten atheist-related figures that inspired them, they'd come up with a predictable, odd, or obviously Googled list. Equally, if I was posing as a Christian I'd struggle to name ten inspiring Bible passages that weren't either predictable, odd, or obviously pulled off Google.

Indeed, I'll happily bet that any libertarian with a Ph.D. from a top-10 social science program can fool more voters than Krugman. We learn his worldview as part of the curriculum. He learns ours in his spare time - if he chooses to spare it.

If we take this as a test, It should be noted that it seems to be biased against popular/conventional/establishment/dominant schools of thought. Historically speaking religious minorities often know a lot about the dominant religion, though the slant may be biased.

However depending on which other tests we use in our investigation this might be a feature not a bug.

I think there are several big problems with this that make it a lot less useful than you might hope as a way of telling which side better understands the other side's arguments - leaving aside the big problem with picking a "side" in an argument. Nonetheless, I think it's likely to be informative and worth a go!

As stated, it sounds like everyone is expected to be on the same IRC channel. That doesn't seem like such a good idea - the imitators can for the most part simply listen to the genuines to work out what to say. What you really want is a way to ask the panel a question and have their answers revealed only once they have all been submitted. However, I can't immediately think of an existing software platform that makes this convenient.

I don't know how best to minimise the shibboleth problem. At the very least, we could remind panellists and judges that shibboleths may be more easily imitated than real winning arguments. Bear in mind that the judges are effectively all Blues pretending to be Greens, in that they want to test how well the panel understand the arguments against Green positions.

"Things that try to look like things often look more like things than things. Well known fact."

—Granny Weatherwax, Wyrd Sisters (by Terry Pratchett)

In other words, you win this game not by trying to imitate actual people who hold a position, but by imitating the stereotype of people that hold that position.

There are people who do this for free. They're usually known as Internet trolls.

I suspect the results would depend heavily on the jury. Specifically a blue pretending to be a green, could probably fool fellow blues much more easily then he could fool actual greens.

Troublesome because you have an uncontrolled variable - the investigative ability of the people you're trying to fool.

If its carreid out it should be entertaining. Some comparisons among other ideological constructs would be interesitng as well. I wonder how say someone like Paul Gottfried or John Derbyshire would do immitating left or Neoconservative positions.

I asked Leah about participating in her test. I think I will find it challenging the emulate a Christian's viewpoint, instead of mocking it.

That is, normally when I've examined Christian thought, I've approached it from the pretense "let me see where this fails."

I'm interested to see if I can construct an argument that I myself as an atheist would find at least superficially compelling - that is, I want to avoid doing what pretty much every popular Christianity book does.

Voting for the Christian / "Christian" half is open - see Leah's site.

I had fun figuring these out. Once it's done I'd be interested to see what criteria LW users used to determine someone's real beliefs...

First round (Christians pretending to be atheists, atheists being sincere) voting is open.

Sounds like a neat idea! I've been looking for these kinds of "meta" approaches to political discussion, and this looks like a pretty good one I hadn't thought of!

An improvement might be to restrict the discussion to a topic narrower than "libertarians vs. liberals" or "christians versus atheists" - for example "labour laws", "immigration", "the documentary hypothesis", "morality of homsexuality", etc.

This looks like it'd have a much better likelihood-I-change-my-mind / invested-time ratio than an "ordinary" poltical debate.

I asked Leah about participating in her test. I think I will find it challenging the emulate a Christian's viewpoint, instead of mocking it.

That is, normally when I've examined Christian thought, I've approached it from the pretense "let me see where this fails."

I'm interested to see if I can construct an argument that I myself as an atheist would find at least superficially compelling - that is, I want to avoid doing what pretty much every popular Christianity book does.