Against Individual IQ Worries

post by Scott Alexander (Yvain) · 2017-09-28T17:12:19.553Z · score: 60 (33 votes) · LW · GW · 7 comments

Contents

  I write a lot about the importance of IQ research, and I try to debunk pseudoscientific claims that IQ “isn’t real” or “doesn’t matter” or “just shows how well you do on a test”. IQ is one of the best-studied ideas in psychology, one of our best predictors of job performance, future income, and various other forms of success, etc.
  Start with the measurement problems. People who complain about low IQs (not to mention people who boast about high IQs) are often wildly off about the number.
  The second problem is that even if you avoid the problems mentioned above and measure IQ 100% correctly, it’s just not that usefully predictive.
  I’m kind of annoyed I have to write this post. After investing so much work debunking IQ denialists, I feel like this is really – I don’t know – diluting the brand.
7 comments

[Related to: Attitude vs. Altitude]

I.

I write a lot about the importance of IQ research, and I try to debunk pseudoscientific claims that IQ “isn’t real” or “doesn’t matter” or “just shows how well you do on a test”. IQ is one of the best-studied ideas in psychology, one of our best predictors of job performance, future income, and various other forms of success, etc.

But every so often, I get comments/emails saying something like “Help! I just took an IQ test and learned that my IQ is x! This is much lower than I thought, and so obviously I will be a failure in everything I do in life. Can you direct me to the best cliff to jump off of?”

So I want to clarify: IQ is very useful and powerful for research purposes. It’s not nearly as interesting for you personally.

How can this be?

Consider something like income inequality: kids from rich families are at an advantage in life; kids from poor families are at a disadvantage.

From a research point of view, it’s really important to understand this is true. A scientific establishment in denial that having wealthy parents gave you a leg up in life would be an intellectual disgrace. Knowing that wealth runs in families is vital for even a minimal understanding of society, and anybody forced to deny that for political reasons would end up so hopelessly confused that they might as well just give up on having a coherent world-view.

From an personal point of view, coming from a poor family probably isn’t great but shouldn’t be infinitely discouraging. It doesn’t suggest that some kid should think to herself “I come from a family that only makes $30,000 per year, guess that means I’m doomed to be a failure forever, might as well not even try”. A poor kid is certainly at a disadvantage relative to a rich kid, but probably she knew that already long before any scientist came around to tell her. If she took the scientific study of intergenerational income transmission as something more official and final than her general sense that life was hard – if she obsessively recorded every raise and bonus her parents got on the grounds that it determined her own hope for the future – she would be giving the science more weight than it deserves.

So to the people who write me heartfelt letters complaining about their low IQs, I want to make two important points. First, we’re not that good at measuring individual IQs. Second, individual IQs aren’t that good at predicting things.

II.

Start with the measurement problems. People who complain about low IQs (not to mention people who boast about high IQs) are often wildly off about the number.

According to the official studies, IQ tests are rarely wrong. The standard error of measurement is somewhere between 3-7 points (1, 2, 3). Call it 5, and that means your tested IQ will only be off by 5+ points 32% of the time. It’ll only be off by 10+ points 5% of the time, and really big errors should be near impossible.

In reality, I constantly hear about people getting IQ scores that don’t make any sense.

Here’s a pretty standard entry in the “help my IQ is so low” genre – Grappling With The Reality Of Having A Below Average IQ:

When I was 16, as a part of an educational assessment, I took both the WAIS-IV and Woodcock Johnson Cognitive Batteries. My mother was curious as to why I struggled in certain subjects throughout my educational career, particularly in mathematical areas like geometry.
I never got a chance to have a discussion with the psychologist about the results, so I was left to interpret them with me, myself, and the big I known as the Internet – a dangerous activity, I know. This meant two years to date of armchair research, and subsequently, an incessant fear of the implications of my below-average IQ, which stands at a pitiful 94…I still struggle in certain areas of comprehension. I received a score of 1070 on the SAT, (540 Reading & 530 Math), and am barely scraping by in my college algebra class. Honestly, I would be ashamed if any of my coworkers knew I barely could do high school-level algebra.

This person thinks they’re reinforcing their point by listing two different tests, but actually a 1070 on the SAT corresponds to about 104, a full ten points higher. Based on other things in their post – their correct use of big words and complicated sentence structure, their mention that they work a successful job in cybersecurity, the fact that they read a philosophy/psychology subreddit for fun – I’m guessing the 104 is closer to the truth.

From the comments on the same Reddit thread:

Interesting, I hope more people who have an avg. or low IQ post. Personally I had an IQ of 90 or so, but the day of the test I stayed up almost the entire night, slept maybe two hours and as a naive caffeine user I had around 500 mg caffeine. Maybe low IQ people do that.
I did IQTest.dk Raven’s test on impulse after seeing a video of Peterson’s regarding the importance of IQ, not in a very focused mode, almost ADHD like with rumination and I scored 108, but many claim low scores by around 0.5-1 SD, so that would put me in 115-123. I also am vegan, so creatine might increase my IQ by a few points. I think I am in the 120’s, but low IQ people tend to overestimate their IQ, but at least I am certainly 108 non-verbally, which is pretty average and low.

The commenter is right that IQtest.dk usually underestimates scores compared to other tests. But even if we take it at face value, his first score was almost twenty points off. By the official numbers, that should only happen once in every 15,000 people. In reality, someone posts a thread about it on Reddit and another person immediately shows up to say “Yeah, that happened to me”.

Nobel-winning physicist Richard Feynman famously scored “only” 124 on an IQ test in school – still bright, but nowhere near what you would expect of a Nobelist. Some people point out that it might have been biased towards measuring verbal rather than math abilities – then again, Feynman’s autobiography (admittedly edited and stitched together by a ghostwriter) sold 500,000 copies and made the New York Times bestseller list. So either his tested IQ was off by at least 30 points (supposed chance of this happening: 1/505 million), or IQ isn’t real and all of the studies showing that it is are made up by lizardmen to confuse us. In either case, you should be less concerned if your own school IQ tests seem kind of low.

I don’t know why there’s such a discrepancy between the official reliability numbers and the ones that anecdotally make sense. My guess is that the official studies give the tests better somehow. They use professional test administrators instead of overworked school counselors. They give them at a specific time of day instead of while the testee is half-asleep. They don’t let people take a bunch of caffeine before the test. They actually write the result down in a spreadsheet they have right there instead of trusting the testee to remember it accurately.

In my own field, official studies diagnose psychiatric diseases through beautiful Structured Clinical Interviews performed to exacting guidelines. Then real doctors diagnose them through checklists that say “DO NOT USE FOR DIAGNOSIS” in big letters on the top. If psychometrics is at all similar, the clashing numbers aren’t much of a mystery.

But two other points that might also be involved.

First, on a population level IQ is very stable with age. Over a study of 87,498 Scottish children, age 11 IQ and adult IQ correlated at 0.66, about as strong and impressive a correlation as you’ll ever find in the social sciences. But “correlation of 0.66” is also known as “only predicts 44% of the variance”. On an individual level, it is totally possible and not even that surprising to have an IQ of 100 at age 11 but 120 at age 30, or vice versa. Any IQ score you got before high school should be considered a plausible prediction about your adult IQ and nothing more.

Second, the people who get low IQ scores, are shocked, find their whole world tumbling in on themselves, and desperately try to hold on to their dream of being an intellectual – are not a representative sample of the people who get low IQ scores. The average person who gets a low IQ score says “Yup, guess that would explain why I’m failing all my classes”, and then goes back to beating up nerds. When you see someone saying “Help, I got a low IQ score, I’ve double-checked the standard deviation of all of my subscores and found some slight discrepancy but I’m not sure if that counts as Bayesian evidence that the global value is erroneous”, then, well – look, I wouldn’t be making fun of these people if I didn’t constantly come across them. You know who you are.

Just for fun, I analyzed the lowest IQ scores in my collection of SSC/LW surveys. I was only able to find three people who claimed to have an IQ ≤ 100 plus gave SAT results. All three had SAT scores corresponding to IQs in the 120s.

I conclude that at least among the kind of people I encounter and who tend to send me these emails, IQ estimates are pretty terrible.

This is absolutely consistent with population averages of thousands of IQ estimates still being valuable and useful research tools. It just means you shouldn’t use it on yourself. Statistics is what tells us that almost everybody feels stimulated on amphetamines. Reality is my patient who consistently goes to sleep every time she takes Adderall. Neither the statistics nor the lived experience are wrong – but if you use one when you need the other, you’re going to have a bad time.

III.

The second problem is that even if you avoid the problems mentioned above and measure IQ 100% correctly, it’s just not that usefully predictive.

Isn’t that heresy?! Isn’t IQ the most predictive thing we have? Doesn’t it affect every life outcome as proven again and again in well-replicated experiments?

Yes! I’m not denying any of that. I’m saying that things that are statistically true aren’t always true for any individual.

Once again, consider the analogy to family transmission of income. Your parents’ socioeconomic status correlates with your own at about r = 0.2 to 0.3, depending on how you define “socioeconomic status”. By coincidence, this is pretty much the same correlation that Strenze (2006) found for IQ and socioeconomic status. Everyone knows that having rich parents is pretty useful if you want to succeed. But everyone also knows that rich parents aren’t the only thing that goes into success. Someone from a poor family who tries really hard and gets a lot of other advantages still has a chance to make it. A sociologist or economist should be very interested in parent-child success correlations; the average person trying to get ahead should just shrug, realize things are going to be a little easier/harder than they would have been otherwise, and get on with their life.

And this isn’t just about gaining success by becoming an athlete or musician or some other less-intellectual pursuit. Chess talent is correlated with IQ at 0.24, about the same as income. IQ is some complicated central phenomenon that contributes a little to every cognitive skill, but it doesn’t entirely determine any cognitive skill. It’s not just that you can have an average IQ and still be a great chess player if you work hard enough – that’s true, but it’s not just that. It’s that you can have an average IQ and still have high levels of innate talent in chess. It’s not quite as likely as if you have a high IQ, but it’s very much in the range of possibility. And then you add in the effects of working hard enough, and then you’re getting somewhere.

Here is a table of professions by IQ, a couple of decades out of date but probably not too far off (cf. discussion here):

I don’t know how better to demonstrate this idea of “statistically solid, individually shaky”. On a population level, we see that the average doctor is 30 IQ points higher than the average janitor, that college professors are overwhelmingly high-IQ, and we think yeah, this is about what we would hope for from a statistic measuring intelligence. But on an individual level, we see that below-average IQ people sometimes become scientists, professors, engineers, and almost anything else you could hope for.

IV.

I’m kind of annoyed I have to write this post. After investing so much work debunking IQ denialists, I feel like this is really – I don’t know – diluting the brand.

But I actually think it’s not as contradictory as it looks, that there’s some common thread between my posts arguing that no, IQ isn’t fake, and this one.

If you really understand the idea of a statistical predictor – if you have that gear in your brain at a fundamental level – then social science isn’t scary. You can read about IQ, or heredity, or stereotypes, or gender differences, or whatever, and you can say – ah, there’s a slight tendency for one thing to correlate with another thing. Then you can go have dinner.

If you don’t get that, then the world is terrifying. Someone’s said that IQ “correlates with” life outcomes? What the heck is “correlate with”? Did they say that only high-IQ people can be successful? That you’re doomed if you don’t get the right score on a test?

And then you can either resist that with every breath you have – deny all the data, picket the labs where it’s studied, make up silly theories about “emotional intelligence” and “grit” and what have you. Or you can surrender to the darkness, at least have the comfort of knowing that you accept the grim reality as it is.

Imagine an American who somehow gets it into his head that the Communists are about to invade with overwhelming force. He might buy a bunch of guns, turn his house into a bunker, start agitating that Communist sympathizers be imprisoned to prevent them from betraying the country when the time came. Or he might hang a red flag from his house, wear a WELCOME COMMUNIST OVERLORDS tshirt, and start learning Russian. These seem like opposite responses, but they both come from the same fundamental misconception. A lot of the culture war – on both sides – seems like this. I don’t know how to solve this except to try, again and again, to install the necessary gear and convince people that correlations are neither meaningless nor always exactly 1.0.

So please: study the science of IQ. Use IQ to explain and predict social phenomena. Work on figuring out how to raise IQ. Assume that raising IQ will have far-ranging and powerful effects on a wide variety of social problems. Just don’t expect it to predict a single person’s individual achievement with any kind of reliability. Especially not yourself.

7 comments

Comments sorted by top scores.

comment by Dr_Manhattan · 2017-09-30T18:21:10.818Z · score: 6 (4 votes) · LW · GW

There are different kinds of "IQ worries", and this post addresses "IQ anxiety" to some degree. The kind that I'm still concerned about is related, but not the same. Basically the question is one of calibrated resource allocation:

let's say for arguments sake that I have an IQ of 110. This suggests that some areas will be much harder for me to study (advanced statistics), and some probably completely inaccessible (string theory). It would be great if reality provided a guide when to "try harder" vs. "concentrate your energy on something else". IQ seems like that kind of guide, but the Feynman example (and, possibly, Darwin) really throws me off. Would love to hear people's thoughts on this.

comment by Asymmetricky · 2017-10-01T09:56:48.738Z · score: 3 (3 votes) · LW · GW

If you don’t get that, then the world is terrifying. Someone’s said that IQ “correlates with” life outcomes? What the heck is “correlate with”? Did they say that only high-IQ people can be successful? That you’re doomed if you don’t get the right score on a test?

This is definitely not the only worry had by people who are skeptical of IQ usefulness.

Suppose an institution or firm decides to impose IQ tests to applicants and automatically reject anyone below a certain threshold.

Alternatively, suppose IQ tests are given more widely in schools and used to irreversibly track students into different career paths from a very early age. Sorry, smart kid, it doesn't matter if you want to be a musician, you have to study advanced math.

These scenarios don't relate to the epistemic deterministic worry you mention because they're more focused on a different issue: given study of IQ, how will the information be used?

That's not to say we shouldn't study IQ! We definitely should! And I believe you about IQ being really predictive on a macro level! But it does mean that even though we currently live in a world where IQ does not absolutely determine an individual's life outcomes, we could live in such a world, if the right policies were enacted. The lab protestors are wrong about their focus -- they should be working to pass laws against discriminating based on IQ -- but their worry is very, very different from the one described in the quoted section.

comment by whpearson · 2017-09-28T18:27:13.763Z · score: 2 (1 votes) · LW · GW

This is a bit of an aside. I'm interested in the relationship between job performance and IQ, for reasons. However everything I read about it seems like a bag of snakes.

Take for example this study suggests :

IQ scores have become embedded in organizational resource allocation decisions

in ways that can have self-fulfilling consequences. This article complicates the story researchers

have been telling about the relationship between IQ and job performance. While not denying that

intelligence is a critical resource in performing one’s job, we have provided reasons to suspect

that institutionalized practices outside the workplace have the potential to inflate the IQ score –

job performance relationship in ways that must also be recognized in our theories and empirical

research

And this article suggests:

Much in developmental theory, and psychology in general, depends upon the validity of IQ tests.

In the absence of agreed construct validity this has weighed heavily on indirect validity using correlations with criterion outcomes among which job performance has a special status.

Hundreds of studies prior to the 1970s reported low and/or inconsistent correlations between IQ and job performance.

These correlations have been approximately doubled using corrections for supposed errors in primary results and combining them in meta-analyses. Such corrections have many strengths, theoretically, but are compromised in these cases by the often uncertain quality of the primary studies.

The corrections to sampling errors, measurement errors, and to range restriction have required making a number of assumptions that may not be valid and have created a number of persistently contentious issues.

The claim that the IQ-job performance correlation increases with job complexity is not born out in more recent studies.

A range of other—including noncognitive—factors could explain a correlation between IQ and job performance, and even constitute part or all of the enigmatic “general factor.”

There remains great uncertainty about the interpretation of IQ-job performance correlations and great caution needs to be exercised in using them as a basis for the validity of IQ tests and associated concepts.

I've not found a good rejoinder to this 2015 article, do you happen to know of one? I've not fully read these either, so researcher beware.

Work on figuring out how to raise IQ.


On topic, without a concrete model of what IQ is I would personally be skeptical of treating it like a metric, lest ye fall into ole Goodheart's trap.

comment by Patricia · 2018-04-05T19:04:09.461Z · score: 11 (3 votes) · LW · GW

This is a good article that touches on the predictive power of IQ for life outcomes such as employment: http://ftp.iza.org/dp10356.pdf

Though I am skeptical of their conclusion that, since "adding grades and test scores to models with IQ and personality produces greater predictive power for the outcomes studied", there is evidence that grades and test scores "capture relevant dimensions of human capability not captured by IQ and personality." There are a lot of social aspects involved on how grades and test results can influence those outcomes.

comment by shaun2000 · 2017-09-28T19:13:06.958Z · score: 1 (4 votes) · LW · GW

I think IQ not mattering is becoming a live issue. The votes of people with IQ below 100 carry as much weight as those of people with IQ above 100. If the <100IQ cohort voted as a bloc for the kind of society they prefer, a lower IQ could make for a more satisfying life. The world seems to be dividing itself along pre- and post-enlightenment lines, eg pro-Trump and anti-Trump, and IQ seems to be valued less in the former. Rejection of enlightenment values, of individuality and intelligence, may become discounted so heavily in the future that they become toxic. Productivity and innovation might then be criticized for making it hard for a son to encounter the kinds of challenges his father faced, and reap the same rewards. In a pre-enlightenment society, change may be seen as unwelcome, and any drawbacks because of it matter less than a sense of continuity and community.

I'm decidedly post-enlightenment, but I am beginning to wonder if the free rein of high IQ has created a better lifestyle than something more communitarian could have. I am pleased to see a trend towards co-housing emerging to replace the chilling lifestyle of isolation on one-acre plots where I live. I've heard people say, in defense of modern science, "Would you prefer to be alive now, with modern medicine, or in Shakespeare's time"? with the inference that life in Shakespeare's day was less worth living. Surely life in any age is worth living. Isn't the same true for IQ, provided society doesn't discriminate in favor of higher IQ, as tomorrow's pre-enlightenment societies may not? Making IQ not matter may be a more humanitarian policy than defending those with low-IQ scores from low self-esteem.

comment by lifelonglearner · 2017-10-01T15:44:13.293Z · score: 1 (1 votes) · LW · GW

I don't think I'm parsing the second paragraph right:

I've heard people say, in defense of modern science, "Would you prefer to be alive now, with modern medicine, or in Shakespeare's time"? with the inference that life in Shakespeare's day was less worth living. Surely life in any age is worth living.

I think you said it yourself earlier that, given some the sorts of valuation that we might typically use, it's valid to say that earlier times might have been worth less to live in.

I think it's compatible to say that you both favor living over not living (in most non-hellscapes), but also to assert that you can have an ordering over different times to live in.

comment by ztech · 2017-11-10T14:08:09.918Z · score: -7 (3 votes) · LW · GW

IQ is an extremely anti-empirical, tautological conception of "intelligence", favored by those who "think they can understand the universe by sitting on the couch and sipping on whisky", as physicst Sean Carroll says. By anti-empirical, I just mean IQ tests under-rate the role of experience and intuition. I scored around 98 consistently on various measures, yet I could maintain a 4.0, by my Junior year in college (including scoring 105% in Calculus) - although my SAT predicted bottom 20th percentile. Since then, I have tutored a few students in the several college math courses. I developed and sold two computer programs (CRM systems), one of them to Dodge dealership, which effectively saved the company 23k a year in secretarial work. Ironically, with some abstract insight, which I supposedly lack, I also discovered a corporate theif, who had been stealing money for 10 years, under admin's noses. I have been a top sales performer at 2 of my jobs, and my income has been in the top 80 percentile. I came up with a unique solution to the THREE DIE PROBLEM, which I've never seen proposed before: https://www.youtube.com/watch?v=WIJ4LeACUm4 . I can score around 150, on the SIGMA test, which is considered to be a good test of creativity and mathematical insight, clearly exceeding the number correct by the average individivual, college student, a Phd, and even higher than what's expected from a MENSA member. All that being said, if IQ can't predict adult level achivements and accolades, such as I've described - accolades that most 100 IQ people don't have, then why do so many indulge in the delusion that IQ is a sole indicator of "intelligence"? It seems, the notion that IQ tests measure "intelligence", is more of a convenience, which those who are privledged, can utilize against empirically gifted minds. It's no wonder why a die hard empiricist like Richard Feynman didn't score well - because he is wired the other way around. While the rationilists wonders what comes next in the series, 41, 56, 260, x, the empiricist is busy inventing the set.