The New Yorker article on cognitive biases

The article is pretty annoying, but after thinking about this for a day the positive blind-spot/IQ association doesn't seem implausible at all. Smart people tend to know they're smart (and tend to overestimate how far above the mean they are). And lots of people associate cognitive biases with low intelligence (in part because they are associated for many biases). So it shouldn't be that surprising that smart people think they're less likely to suffer from a cognitive bias after one is described to them given that they're unlikely to be familiar with past West and Stanovich research on the relaton between that bias and IQ. Someone needs to replicate the experiment with descriptions of biases that include something like "Smart people are just as likely to suffer from this bias as anyone else." Then see if the positive correlation remains.

I'm missing something basic, here: If cognitive sophistication does attenuate other biases, how can they tell that the cognitively sophisticated believe themselves less biased than the average person to the point of being biased in that belief, rather than merely to the point of being correct?

I don't know about you, but I'm pretty sure Jonah Lehrer knows a lot more than me about psychological research on cognitive bias.

He's a pretty knowledgeable dude, but he has major incentives to overstate research - the incentives which literally just days ago got him a job at The New Yorker, which is a pretty big plum for any journalist. When you read the original papers (I jailbroke the OP paper in question), you find he overstates things.

For example, this paper: the biases were specifically chosen to be resistant to intelligence, and the intelligent performed as well as the stupid - they were just wrong in predicting they'd perform better than the stupid (as they do in every other area of life). This is interesting if you're trying to formulate an integrated theory of IQ and the two processes, as Stanovich is doing (Stanovich's 2010 book is very good reading), but does it bear the spin Lehrer is putting on it? No.

I'm pretty distrustful of science reporting in general. I agree that LWers should at least act more humble (so people won't be averse to correcting them, and they won't be averse to correcting themselves).

I got the article from reddit, so I guess I'm used to articles about science being inadequate or misrepresentative in a major way. This community has a good number of members who know much more than I do about the topic so I wanted to see what they thought about it. Reading the article is probably sufficient to learn from it, I just wanted a better idea of the quality of the article.

After reading the paper (thanks for the link, VincentYu), it turns out that the evidence was stronger than I thought. The authors were being conservative to use the "if anything" language - I think more conservative than they needed to be. In study 1, their composite measure of the bias blind spot had a statistically significant positive correlation with each of their 4 intelligence-related measures (SAT, CRT, need for cognition, actively open-minded thinking), with correlations ranging from .096 (p<.05) to .260 (p<.001). It's a relatively weak correlation, but still pretty strong evidence that it's a real correlation rather than noise. For 4 of the 7 particular cognitive biases that they studied, the bias blind spot had a statistically significant correlation with SAT scores (see Jack's comment for more). In study 2, the correlation between the CRT and the bias blind spot composite was almost identical (r=.102, compared with .096 in study 1), but not quite statistically significant (p=.10) because of a smaller sample size. So I think it would be fair to state the claim more strongly than they do - smarter people do tend to think that they're less biased (at least within this study).

However, I don't agree with calling this a "larger bias blind spot." The bias blind spot is an instance of the better-than-average effect. It was measured by describing a specific bias to university students, asking them how susceptible to that bias they think that they are, and asking them how susceptible to that bias they think that other students at their university. People tend to say that they are less susceptible to the bias than other people are. This cannot be true on average (since people must be average on average), so it is a bias.

But studying individual differences in the bias blind spot is tricky because any particular individual who claims to be less biased than average may actually be less biased (rather than being more metabiased about how biased they are). I'm not sure if any other research on the bias blind spot has looked at individual differences; the main focus has been to look at differences between biases (which biases do people freely admit to having, and which do they have the largest blind spot for?). The bias blind spot measures don't have any clear standard for accuracy, other than the fact that self and other must be equally biased on average as long as they're sampled from the same population.

When you're comparing people who have high SAT scores with people who have lower SAT scores (as this study did), you don't have that because you're drawing from two different populations, and people with higher SAT scores do tend to be less biased (on many particular cognitive biases, though not all of them). If you had a really good, thorough measure of how susceptible to cognitive biases people are (one which covered a wide range of different biases, and which included lots of questions so that it could reliably identify individual differences among the noise), then you would presumably find that smarter people tend to score as less biased. So it's no surprise that smarter people tend to think that they're less biased - that seems to reflect accurate self-knowledge (and maybe also accurate theories, e.g. "This seems like the kind of thing that smart people would be better at"). It may be the case that smart people have trouble distinguishing the biases that their intelligence helps them avoid from the biases where their intelligence doesn't help, but that's not the same thing as having a larger bias blind spot (i.e., a stronger better-than-average effect with respect to how biased you are).

This study didn't include a thorough test of how biased people were - it picked a few specific biases and had 1 question each testing how susceptible people were to that bias (2 for anchoring). And this study actually did find that smarter people were less biased - on 3 of the 9 questions (across the 2 studies) there was a statistically significant relationship between intelligence and the bias (those with higher SAT scores had less outcome bias, less myside bias, and less anchoring bias on one of the two anchoring questions). The authors downplayed those effects because they only accounted for a tiny percentage of the variance, but on a noisy (single-item) measure I'd only expect a variable like this to explain a small amount of variance (and the correlation between intelligence and the "bias blind spot" composite also had a small effect size). Interestingly, outcome bias and myside bias were two of the biases where SAT did not have a statistically significant relationship with their bias blind spot measure - so smarter people did seem to be less biased on the whole, and they thought that they would be less biased, but they didn't know which biases they would be less susceptible to.

Out of 28 correlations 24 were positive and 17 were statistically significant, six of the 7 biases tested showed the relationship (the one that didn't isn't really a cognitive bias). For the Bias Blind spot composite score SATs were significant at p<.001, N=482. The authors are conservative in their assessment of the results. To provide some context West and Stanovich have spent the last decade examining correlations between intelligence and cognitive biases.

Upvoted. Based on reading the abstract and Lehrer's article, I'd expect only a 50% chance or so that these results will replicate, and even less so that they will generalize to other biases.

I thought i linked to the comments, but I can't find them any more. Searching reddit with the url or title worked before but it doesn't now, i wonder if it got deleted.

Can you elaborate on this? I don't see any curiosity stopping going on there.