New study on choice blindness in moral positions

When reading old LW posts and comments and seeing I've upvoted some comment, I find myself thinking "Wait, why have I upvoted this comment?"

This doesn't seem obviously bad to me... You just have to differentiate times when you have a gut feeling that something's true because you worked it out before, or because of some stupid reason like your parents telling it to you when you were a kid. Right?

I think I can tell apart rationalizations I'm creating on the spot with reasoning I remember constructing in the past. And if I'm creating rationalizations on the spot, I make an effort to rationalize in the opposing direction a bit for balance.

Claim to agree with them on issue X, then once they've committed to supporting you, change your position on issue X.

Come to think of it, politicians already do this.

While meeting with voters in local community halls, candidates sometimes go around distributing goodwill tokens or promises while thanking people for supporting them, whether the person actually seems to support them or not.

It's not a very strong version, and it's tinged with some guilt-tripping, but matches the pattern under some circumstances and very well might trigger the choice blindness in some cases.

Dark tactic: Have we verified that it doesn't work to present them with a paper saying what their opinion is even if they did NOT fill anything out? I explain how that might work This tactic is based on that possibility:

1. An unethical political candidate could have campaigners get a bunch of random people together and hand them a falsified survey with their name on it, making it look like they filled it out. The responses support a presidential candidate.

2. The unethical campaigner might then say: "A year ago, (too long for most people to remember the answers they gave on tests) you filled out a survey with our independent research company, saying you support X, Y and Z." If authoritative enough, they might believe this.

3. "These are the three key parts of my campaign! Can you explain why you support these?"

4. (victim explains)

5. "Great responses! Do you mind if we use these?"

6. (victim may feel compelled to say yes or seem ungrateful for the compliment)

7. "I think your family and friends should hear what great supports you have for your points on this important issue, don't you?"

8. (now new victims will be dragged in)

9. The responses that were given are used to make it look like there's a consensus.

how would you exploit this to your advantage, were you dark-arts inclined?

Break into someone's blog and alter statements that reflect their views.

Dark Tactic:

This one makes me sick to my stomach.

Imagine some horrible person wants to start a cult. So they get a bunch of people together and survey them asking things like:

"I don't think that cults are a good thing." "I'm not completely sure that (horrible person) would be a good cult leader."

and switches them with:

"I think that cults are a good thing." "I'm completely sure that (horrible person) would be a good cult leader."

And the horrible person shows the whole room the results of the second set of questions, showing that there's a consensus that cults are a good thing and most people are completely sure that (horrible person) would be a good cult leader.

Then the horrible person asks individuals to support their conclusions about why cults are a good thing and why they would be a good leader.

Then the horrible person starts asking for donations and commitments, etc.

Who do we tell about these things? They have organizations for reporting security vulnerabilities for computer systems so the professionals get them... where do you report security vulnerabilities for the human mind?

Online dating. Put up a profile that suggests a certain personality types and interests. In face-to-face meetup, even if you're someone different than was advertised, choice blindness should cover up the fact.

This tactic can also be extended to job resumes presumably.

The problem is that we don't know how influential the blind spot is. It could just fade away after a couple minutes and a "hey, wait a minute..." But assuming it sticks:

If I were a car salesmen, I would have potential customers tell me their ideal car and then I would tell them what I want their ideal car to be as though I were simply restating what they just said.

If I were a politician, I would target identities (e.g., latino, pro-life, low taxes, ect) rather than individuals because identities are made of choices and they're easier to target than individuals. The identity makes a choice and then you assume the identity chose you. E.g., "President Obama has all but said that I'm instigating "class warfare," or that I don't care about business owners, or that I want to redistribute wealth. Well, Mr. Obama, I am fighting with and for the 99%; the middle class; the inner city neighborhoods that your administration has forgotten; Latinos; African-Americans. We all have had enough of the Democrats decades long deafness towards our voice. Vote Romney." Basically, you take the opposition's reasons for not voting for you and then assume those reasons are for the opposition, and you run the ads in the areas you want to affect.

For example, if you are a US presidential candidate, what tactics would you use to invisibly switch voters' choice to you?

I vaguely remember that when a president becomes very widely accepted as a good or bad president, many people will misremember that they voted for or against him respectively; e.g. much fewer people would admit (even to themselves) having voted for Nixon than the actual number that voted for him. If this is so, then maybe the answer is simply "Win, and be a good president".

Now I'm alternating between laughing and crying. :(

Imagine answering a question like "I think such and such candidate is not a very good person." and then it gives you a button where you can automatically post it to your twitter / facebook. When you read the post on your twitter, it says "I think such and such candidate is a very good person." but you don't notice the wording has changed. :/

I wonder if people would feel compelled to confabulate reasons why they posted that on their accounts. It might set of their "virus" radars because of the online context and therefore not trigger the same behavior.

Dark Tactic:

1. An unwitting research company could be contracted to do a survey by an unethical organization.
2. The survey could use the trick where by asking some question that people will mostly say "yes" to and then ask a similar question later where the wording is slightly changed to agree with the viewpoint of the unethical organization.
3. Most people end up saying they agree with the viewpoint of the unethical organization.
4. The reputation of the research company is abused as the unethical organization claims they "proved" that most people agree with their point of view.
5. A marketing campaign is devised around the false evidence that most people agree with them.

They already trick people in less expensive ways, though. I was taught in school that they'll do things like ask 5 doctors whether they recommend something and then saying "4 of 5 doctors" recommend this to imply 4 of every 5 doctors when their sample was way too small.

...Confronted with a choice between (a) "the person asking me directions was just spontaneously replaced by somebody different, also asking me directions," and (b) "I just had a brain fart," I'll consciously go for (a) every time, especially considering that I observe similar phenomena all the time (people spontaneously replacing each other immediately after having encountered them). ...

I'm curious, why do you take that view?

What a coincidence, this happened to me with your comment! I originally read your name as "shminux" and was quite surprised when I reread it.

If there's some coding magic going on behind the scenes, you've got me good. But I'm sticking with (b) - final answer.

A rational prior for "the person asking me directions was just spontaneously replaced by somebody different, also asking me directions" would be very small indeed (that naturally doesn't happen, and psych experiments are rare). A rational prior for "I just had a brain fart" would be much bigger, since that sort of thing happens much more often. So at the end, a good Bayesian would assign a high probability to "I just had a brain fart", and also a high probability to "This is the same person" (though not as high as it would be without the brain fart).

The problem is that the conscious mind never gets the "I just had a brain fart" belief. The error is unconsciously detected and corrected but not reported at all, so the person doesn't even get the "huh, that feels a little off" feeling which is in many cases the screaming alarm bell of unconscious error detection. Rationalists can learn to catch that feeling and examine their beliefs or gather more data, but without it I can't think of a way to beat this effect at all, short of paying close attention to all details at all times.

Confronted with a choice between (a) "the person asking me directions was just spontaneously replaced by somebody different, also asking me directions," and (b) "I just had a brain fart,"

Schizophrenia. Capgras Delusion.

I wonder how schizophrenics would comparatively perform on the study.

You said in a previous thread that after a hard day of stealing wifi and lobbying for SOPA, you and Chris Brown like to eat babies and foie gras together. Can you explain your moral reasoning behind this?

Could we train ourselves to notice such dissonance instead?

This needs to get a spot in CFAR's training program(s/mme(s)?). It sounds like the first thing you'd want to do once you reach the rank of second-circle Initiate in the Bayesian Conspiracy. Or maybe the first part of the test to attain this rank.

It seems to me that this hypothesis is more of a mechanism for choice blindness than an alternate explanation- we already know that human beings will change their minds (and forget they've done so) in order to please authority.

(There's nonfictional evidence for this, but I need to run, so I'll just mention that we've always been at war with Oceania.)

But the study said:

"The statements in condition two were picked to represent salient and important current dilemmas from Swedish media and societal debate at the time of the study."

Quite. My own answer to most of the questions in the survey is "Yes/No, but with the following qualifications." It's not too hard for me to imagine choosing, say, "Yes" to the surveillance question (despite my qualms), then being told I said "No," and believing it.

You won't fool these people if you ask them about something salient like abortion.

There is a long tradition in social science research, going back at least to Converse (1964), holding that most people's political views are relatively incoherent, poorly thought-through, and unstable. They're just making up responses to survey questions on the spot, in a way that can involve a lot of randomness.

This study demonstrates that plus confabulation, in a way that is particularly compelling because of the short time scale involved and the experimental manipulation of what opinion the person was defending.

I don't have citations handy, but IIRC in general inattentional blindness effects are greatly diminished if you warn people ahead of time, which should not be surprising. I don't know what happens if you warn people between the filling-out-the-questionaire stage and the reading-the-(possibly altered)-answers stage; I expect you'd get a reduced rate of acceptance of changed answers, but you'd also get a not-inconsiderable rate of rejection of unchanged answers.

More generally: we do a lot of stuff without paying attention to what we're doing, but we don't keep track of what we did or didn't pay attention to, and on later recollection we tend to confabulate details into vague memories of unattended-to events. This is a broken system design, and it manifests in a variety of bugs that are unsurprising once we let go of the intuitive but false belief that memory is a process of retrieving recordings into conscious awareness.

It frequently startles me how tenacious that belief is.

A real-life example of a similar effect: I explained the Newcomb problem to a person and he two-boxed initially, then, after some discussion, he switched to one-boxing and refused to admit that he ever two-boxed.

Relatedly, I wonder how consistent people's original answers to these questions are (if, say, retested a month later). But I would expect answers the subjects are asked to defend/explain (whether original or changed) to be more persistent than answers they aren't.

Rabbit season!

Even having watched the video before, when I concentrated hard on counting passes, I missed seeing it.

This is "inattention blindness". Choice blindness is sort of like the opposite; in inattention blindness you don't notice something you're not paying attention to, in choice blindness you don't notice something which you are paying attention to.

The study asked people to rate their position on a 9-point scale. People who took more extreme positions, while more likely to detect the reversal, also gave the strongest arguments in favour of the opposite opinion when they failed to detect the reversal.

Also, the poll had two kinds of questions. Some of them were general moral principles, but some of them were specific statements.

It would be easy enough to do with a longish computer survey. It's much easier to change what appears on a screen than to do sleight-of-paper.

For added fun metaness, have the option you swich them to and start rationalize for be the one you're trying to convince them of :p

The video shows the mechanics of how it works pretty well.

I suspect that those who are most susceptible to moral proposition switches and their subsequent defense of switch, are also the same people that will deny the evidence when confronted with their switch. Much like the Dunning Kruger effect there will be people who fail to recognize the extremity of their inadequacy, even when confronted with evidence of such.

Edit: The paper states that they informed all participants of the true nature of the survey, but it does not go in to detail on whether participants actually acknowledged that their moral propositions were switched.

This is indeed amusing, but the author draws a wrong/incomplete/tendentious conclusion from it. I think the proper conclusion is basically our usual "Blue vs Green" meme, plus some Hansonian cynicism about 'informed electorates.'

Konkvistador's LessWrong improvement algorithm

1. Trick brilliant but contrarian thinker into mainstream position.
2. Trick brilliant but square thinker into contrarian position.
3. Have each write an article defending their take.
4. Enjoy improved rationalist community.

Consider this: If you're supporting your own genuine opinion, you might have your own carefully chosen perspective that is slightly different from the question's wording. You only select the answer because it's the closest one of the options, not because it's exactly your answer. So, you may be inclined, then, to say things that are related but don't fit the question exactly. If you're confabulating to support a random opinion, though, what do you have to go by but the wording? The opinion is directing your thoughts then, leading your thoughts to fit the opinion. You aren't trying to cram pre-existing thoughts into an opinion box to make it fit your view.

Or looking at it another way:

When expressing your point of view, the important thing is to express what you feel, regardless of whether it fits the exact question.

When supporting "your" point because you don't want to look like an idiot in front of a researcher, the objective is to support it as precisely as possible, not to express anything.

As for whether your interpretation of that selection is correct: it's past my bed time and I'm getting drowsy, so someone else should answer that part instead.

I think it does. Can't believe I missed that.

Actually, this fits well with my personal experience. I've frequently found it easier to verbalize sophisticated arguments for the other team, since my own opinions just seem self-evident.

Especially if the Obvious turns out to be wrong.

Great paper, although (annoyingly) they conflate democracy with liberalism.

Details?

I just did an experiment on it with tea and jam

What kind of experiment?

No-one says "sheeple" intending to include themselves. Do you have any reason to think you are immune from this effect?

That's what I thought at first, too but on second thought, I don't think they went far enough to confirm that this actually causes people to change their opinions. There are other reason people might act the way they did.

I suspect sheep would be less susceptible to this sort of thing than humans.

For logic this woolly, I agree...