You’re Entitled to Everyone’s Opinion

post by satt · 2014-09-20T15:39:07.903Z · score: 25 (26 votes) · LW · GW · Legacy · 23 comments

Over the past year, I've noticed a topic where Less Wrong might have a blind spot: public opinion. Since last September I've had (or butted into) five conversations here where someone's written something which made me think, "you wouldn't be saying that if you'd looked up surveys where people were actually asked about this". The following list includes six findings I've brought up in those LW threads. All of the findings come from surveys of public opinion in the United States, though some of the results are so obvious that polls scarcely seem necessary to establish their truth.

  1. The public's view of the harms and benefits from scientific research has consistently become more pessimistic since the National Science Foundation began its surveys in 1979. (In the wake of repeated misconduct scandals, and controversies like those over vaccination, global warming, fluoridation, animal research, stem cells, and genetic modification, people consider scientists less objective and less trustworthy.)
  2. Most adults identify as neither Republican nor Democrat. (Although the public is far from apolitical, lots of people are unhappy with how politics currently works, and also recognize that their beliefs align imperfectly with the simplistic left-right axis. This dissuades them from identifying with mainstream parties.)
  3. Adults under 30 are less likely to believe that abortion should be illegal than the middle-aged. (Younger adults tend to be more socially liberal in general than their parents' generation.)
  4. In the 1960s, those under 30 were less likely than the middle-aged to think the US made a mistake in sending troops to fight in Vietnam. (The under-30s were more likely to be students and/or highly educated, and more educated people were less likely to think sending troops to Vietnam was a mistake.)
  5. The Harris Survey asked, in November 1969, "as far as their objectives are concerned, do you sympathize with the goals of the people who are demonstrating, marching, and protesting against the war in Vietnam, or do you disagree with their goals?" Most respondents aged 50+ sympathized with the protesters' goals, whereas only 28% of under-35s did. (Despite the specific wording of the question, the younger respondents worried that the protests reflected badly on their demographic, whereas older respondents were more often glad to see their own dissent voiced.)
  6. A 2002 survey found that about 90% of adult smokers agreed with the statement, "If you had to do it over again, you would not have started smoking." (While most smokers derive enjoyment from smoking, many weight smoking's negative consequences strongly enough that they'd rather not smoke; they continue smoking because of habit or addiction.)

If you've read Eliezer's "Hindsight Devalues Science", you're probably starting to feel déjà vu, and might have guessed that I'm bluffing you to make a point. If so, well done — you're quite correct! But before you assume I'm about to repeat Eliezer's trick and stop there, read the other half of my list:

  1. The public's view of the harms and benefits from scientific research has remained about the same since the National Science Foundation began its surveys in 1979. (Despite the last 35 years of scientific scandals and controversies, people appreciate the technological advances science brings, and think scientists come off well compared to other professions.)
  2. Most adults identify as Republican or Democrat. (Although many are dissatisfied with contemporary politics, most voters' political views are nonetheless represented better by one mainstream party than the other, and people find it logical to give more power to the party which supports more of one's preferred policies.)
  3. Adults under 30 are more likely to believe that abortion should be illegal than the middle-aged. (Younger adults, being less reflective in general, tend to have more extreme political beliefs than the middle-aged.)
  4. In the 1960s, those under 30 were more likely than the middle-aged to think the US made a mistake in sending troops to fight in Vietnam. (The under-30s were more likely to be students and/or highly educated, and more educated people were more likely to think sending troops to Vietnam was a mistake.)
  5. The Harris Survey asked, in November 1969, "as far as their objectives are concerned, do you sympathize with the goals of the people who are demonstrating, marching, and protesting against the war in Vietnam, or do you disagree with their goals?" Most respondents aged 35 or less sympathized with the protesters' goals, whereas only 28% of those aged 50+ did. (Younger people were more anti-authoritarian, which translated to more sympathy for protesters regardless of their goals.)
  6. A 2002 survey found that about 90% of adult smokers disagreed with the statement, "If you had to do it over again, you would not have started smoking." (Whatever the other consequences, smokers derive much pleasure from the experience of smoking, and even an addict who suffers major harm could justify their addiction as the result of a rational decision.)

Here's my twist on Eliezer's twist. It is technically true that the list includes six true findings, but the complete list has 12 items, so half of the statements are false. I made up the false statements as fake variations on the true findings, concocted parenthetical rationalizations for them, and randomly mixed the false claims with the true. I expect a lot of people reading this would, after seeing the full list, have a hard time sorting the true from the false without looking at the data — including the people who nodded along in agreement with the first half of the list.

Totally spurious beliefs about public opinion can have a ring of plausibility, especially because it's easy to invent sensible-sounding reasons why they ought to be correct. The availability heuristic presumably plays a role too, with people inferring the state of public opinion from what their friends & acquaintances think, not accounting for how unrepresentative their social network is. In any event, people's opinions of public opinion are often wrong, and it's worth taking a couple of minutes to look for Gallup poll results and the like online before commenting on public opinion.


Sources. On perceptions of whether the benefits of science outweigh its harmful results, see figure 7-11 from chapter 7 of the National Science Foundation's "Science and Engineering Indicators 2012". On party affiliation, see the polls Gallup runs every month. On abortion views, Pew Research Center has statistics for 2007 through 2012. For a breakdown of beliefs about the Vietnam War by age, consult Hazel Erskine's 1970 article "The Polls: Is War a Mistake?" (Public Opinion Quarterly, 34(1), 134–150); Jim Miller extends the data to May 1971, but presents them somewhat differently. On smokers' regrets, see Geoffrey T. Fong et al.'s 2004 article "The near-universal experience of regret among smokers in four countries: Findings from the International Tobacco Control Policy Evaluation Survey" (Nicotine & Tobacco Research, 6(S3), S341–S351).

23 comments

Comments sorted by top scores.

comment by ChristianKl · 2014-09-20T17:57:04.611Z · score: 8 (8 votes) · LW · GW

If we could find a good list of such statistics we have material for a new Credence app.

comment by satt · 2014-10-31T04:03:24.017Z · score: 0 (0 votes) · LW · GW

Someone reminded me of these recent if UK-centric examples a few weeks ago. [Edit: they're not about public opinion, but they're in the same vein of things that catch people out.]

comment by satt · 2014-09-20T15:38:21.633Z · score: 7 (7 votes) · LW · GW

I'm curious about the effectiveness of my post's central gimmick. I invite anyone who's read the post and hasn't looked at any of the data linked at the end of it to take this poll.

In each pair of opposing claims, which claim do you find more likely?

[Edit: LW's posting interface stripped out the "start='7'" attribute in the second half of my post's list, so it's re-numbered claims 7 to 12 as claims 1 to 6. Pretend the second half of the list starts at 7.]

Claims 1 & 7: harms and benefits from scientific research. [pollid:770]

Claims 2 & 8: adult Republican/Democrat (non-)identification. [pollid:771]

Claims 3 & 9: young vs. middle-aged adults on abortion. [pollid:772]

Claims 4 & 10: young vs. middle-aged on the Vietnam War. [pollid:773]

Claims 5 & 11: young vs. old on Vietnam War protesters. [pollid:774]

Claims 6 & 12: smokers' (non-)regret. [pollid:775]

Into which kind of culture are you most assimilated?

[pollid:776]

comment by jimmy · 2014-09-21T20:23:13.464Z · score: 7 (7 votes) · LW · GW

One thing worth noting is that it's possible for most people to get these answers right even if your gimmick worked - you've given them a chance to stop and think about it.

On the other hand, several of the statements did trip my BS filter, and even if I were to be wrong about which ones were false, they still didn't have the plausibility you were expecting.

On the third hand, you'd probably be much better at fooling people if you said "surprisingly enough, " instead of "it's obvious that...". I can totally believe that there are some things that still seem counter-intuitive to me, but its a harder sell to tell me that I'm wrong about something that is so obvious that you genuinely cannot see why I might think the opposite.

comment by solipsist · 2014-09-20T16:23:48.817Z · score: 3 (3 votes) · LW · GW

By "too close to choose", do you mean that I can't decide which I think is more likely, or I think that I don't think there's a statistically strong trend?

comment by satt · 2014-09-20T16:29:39.564Z · score: 3 (3 votes) · LW · GW

The former.

comment by [deleted] · 2014-09-20T17:40:55.480Z · score: 2 (2 votes) · LW · GW

For some reason I'm shown the result and not given a chance to vote even though I haven't voted on these polls before, but 1/7 I don't know, 2/8 probably the latter, 3/9 the latter (V'q thrff crbcyr jub bccbfr nobegvba unir zber puvyqera, naq bccbfvgvba gb nobegvba vf urevgnoyr), 4/10 the former, 5/11 the former, 6/12 I'm surprised the split is that big; I'd guess the former, though.

comment by [deleted] · 2014-09-20T18:21:42.971Z · score: 1 (1 votes) · LW · GW

Aaaaand the only ones I got wrong where those about the Vietnam War. Not bad for someone who's never been in the US!

comment by satt · 2014-09-20T18:01:01.321Z · score: 1 (1 votes) · LW · GW

Hmm...I did edit the comment after I submitted the post, when I noticed the HTML list issue, but I doubt that's the cause since other users have been able to vote. (And I don't remember editing any of the poll syntax.) Not sure what's going on there.

comment by [deleted] · 2014-09-20T19:56:19.248Z · score: 3 (3 votes) · LW · GW

I eventually managed to vote because I had Recent Comments open in another tab, where the poll was still vote-able. Maybe the software doesn't allow the same user to have the same poll open in two different tabs at the same time? But that would allow people to see the results before voting...

comment by shminux · 2014-09-21T05:55:16.597Z · score: 2 (2 votes) · LW · GW

I also see results only. One tab, just opened this post for the first time.

comment by torekp · 2014-09-20T17:08:06.487Z · score: 2 (2 votes) · LW · GW

With 12 votes in so far, and ignoring "too close" and taking the majority vote on 1st group/2nd group of statements, the LW community is 3 for 6. (I'm glossing over a trick question here, but, whatever.) So I guess we are quite ignorant of public opinion. Especially me.

comment by bramflakes · 2014-09-20T16:56:59.396Z · score: 2 (4 votes) · LW · GW

Hfvat n pbqr orpnhfr lbh pbhyq cebonoyl gryy juvpu vf juvpu rira jura ebg13'q: "E" sbe evtug, "J" sbe jebat:

1-7 J

2-8 J

3-9 J

4-10 J

5-11 J

6-12 E

V qba'g xabj rknpgyl ubj onqyl guvf ersyrpgf ba zr fvapr cerfhznoyl gurfr dhrfgvbaf jrer cvpxrq sbe orvat pbhagrevaghvgvir, ohg vg'f cerggl pyrne V'z abg jryy-pnyvoengrq fvapr V jnf ernfbanoyl pbasvqrag va nyy bs gurz rkprcg gur svefg.

comment by KnaveOfAllTrades · 2014-09-20T15:59:58.609Z · score: 2 (2 votes) · LW · GW

Oops, I didn't actually read 7 and assumed it was public opinion had grown more positive. Given the two choices actually presented, I'd say 7 more likely.

Edit: Relative credences (not necessarily probabilities since I'm conditioning on there being significant effect sizes), generated naively trying not to worry too much about second-guessing how you distributed intuitive and counterintuitive results:

1:07 : 33:67
2:08 : 33:67
3:09 : 67:33
4:10 : 40:60
5:11 : 45:55
6:12 : 85:15

comment by satt · 2014-09-20T17:51:03.394Z · score: 3 (3 votes) · LW · GW

(Couple of side notes inspired by your edit.)

I considered asking for people's credence in each claim with probability polls, but reckoned that'd discourage responses, due to the extra effort needed to ensure coherence. (With 1 vs. 7, for instance, one would also have to think about the probability that neither claim's true.)

When distributing the pairs across the lists, I had R flip six virtual coins to decide whether to swap the places of each pair after I'd written them. So it should be nice & random, making second guessing unnecessary...although I guess no one else can be 100% sure I'm telling the truth here!

comment by Jiro · 2014-09-20T17:55:30.254Z · score: 4 (4 votes) · LW · GW

(Don't read this if you haven't taken the poll yet.)

For the Vietnam war questions, my and many people's first thought would be that people under 30 would be more likely to oppose the Vietnam War because they are of the age to be drafted. This is an obvious enough idea that even if it turns out to be false, any reasonable explanation would have to mention it, if only to explain that it doesn't apply (and in turn, to either explain that as well, or say that it has no explanation). The lack of mention of such a thing, then, artificially made the given explanation sound less sensible, and as a result I believed the statement to be false when in fact it was true. So this doesn't prove that people find anything convincing with a good explanation, but rather that people find things unconvincing with a poor one.

(Edit: I misread the abortion results.)

Edit2: The Gallup poll at http://www.gallup.com/poll/126581/generational-differences-abortion-narrow.aspx gives a different impression of abortion opinions among the young. If you look at a longer time scale, younger people support abortion more and satt's poll inly shows that they do not because the people who were young in those earlier years got older and kept their opinions.

comment by satt · 2014-09-21T15:38:28.678Z · score: 1 (1 votes) · LW · GW

(Like Jiro's comment, don't read this if you're going to take the poll but haven't yet.)

So this doesn't prove that people find anything convincing with a good explanation, but rather that people find things unconvincing with a poor one.

Fair point. The conclusion to draw, then, should be a more general one: given an observation O and an explanation E of O, people can over-weight E as a piece of evidence about O's probability. (If E sounds plausible it might be taken as de facto proof of O; if E sounds implausible it might be taken as a disconfirmation of O.)

Edit2: The Gallup poll at http://www.gallup.com/poll/126581/generational-differences-abortion-narrow.aspx gives a different impression of abortion opinions among the young. If you look at a longer time scale, younger people support abortion more and satt's poll inly shows that they do not because the people who were young in those earlier years got older and kept their opinions.

This strikes me as I-was-not-wrong-but-I-was-almost-right reasoning. Had I posted this in 1992, claim 3 would indeed have been true. But it hasn't been true for something like a decade, and at some point informed people should update their beliefs.

comment by Jiro · 2014-09-21T17:41:11.902Z · score: 2 (2 votes) · LW · GW

The conclusion to draw, then, should be a more general one: given an observation O and an explanation E of O, people can over-weight E as a piece of evidence about O's probability.

But is it overweighting to use the fact that the explanation is bad as evidence against the statement being true? A true statement is more likely to have a good explanation than a false one, so it seems that one could do a Bayseian update on the truth of the staement based on the quality of the explanation.

comment by satt · 2014-09-24T22:11:29.030Z · score: 0 (0 votes) · LW · GW

Sounds reasonable. Although I think it's evidence against that kind of updating if it leads one to get a question wrong, one might well get more evidence in favour of that kind of updating in everyday life.

comment by VAuroch · 2014-09-21T06:10:39.695Z · score: 2 (2 votes) · LW · GW

I read these statistics as containing two trick questions (rot13: cnegvrf naq nobegvba ner obgu gbb pybfr gb pnyy, ernyyl). And I think that asking about Vietnam is addressing a totally different bias than is ostensibly the point of the post; not being familiar with history, and having to draw conclusions based on how that era was depicted, is essentially generalizing from fictional evidence, not anything reflective of our knowledge of public opinion.

Also, I'd bet weakly that the average LW member does better than the general population in guessing public opinion, even when uninformed. Though our errors do all go in the same direction.

comment by satt · 2014-09-21T15:52:03.260Z · score: 1 (1 votes) · LW · GW

I read these statistics as containing two trick questions (rot13: cnegvrf naq nobegvba ner obgu gbb pybfr gb pnyy, ernyyl).

I'm inclined to dispute that, but I suspect I'd be implicitly arguing about the definition of "trick question" rather than anything empirical.

I think that asking about Vietnam is addressing a totally different bias than is ostensibly the point of the post; not being familiar with history, and having to draw conclusions based on how that era was depicted, is essentially generalizing from fictional evidence, not anything reflective of our knowledge of public opinion.

Historical questions about public opinion are nonetheless questions about public opinion, and people with a sufficiently good knowledge of something can answer questions about it without drawing on fictional evidence.

Also, I'd bet weakly that the average LW member does better than the general population in guessing public opinion, even when uninformed.

Agreed.

comment by Prismattic · 2014-09-21T03:38:38.601Z · score: 1 (1 votes) · LW · GW

As I have mentioned before the polling data on partisan affiliation is worthless, because most so-called independents are lying about their actual behavior.

comment by satt · 2014-09-21T15:10:50.158Z · score: 2 (2 votes) · LW · GW

I agree that "independent" poorly characterizes the behaviour of most self-identified "independents", but it doesn't follow that "the polling data on partisan affiliation is worthless". Here comes the ROT-13.

Gur cbyyvat qngn erznva tbbq rivqrapr nobhg cebsrffrq oryvrs & vqragvgl rira vs gurl'er ovnfrq rivqrapr nobhg orunivbhe, naq V gbbx pner gb nfx nobhg frys-vqragvsvpngvba, abg orunivbhe. Fbzrbar ryfr zvtug pbzcynva gung gurl'er zber vagrerfgrq va orunivbhe guna oryvrs, ohg rira gb gurz V guvax gur qngn fubhyq or gryyvat: tvira gung zbfg HF nqhygf frys-vqragvsl nf Erchoyvpna be Qrzbpeng va gur snpr bs n ceb-vaqrcraqrag ovnf, gur cenpgvpny pbapyhfvba gung zbfg HF nqhygf ner cnegvfna whfg trgf nyy gur fgebatre.