The Truth About Mathematical Ability

post by JonahS (JonahSinick) · 2015-02-12T01:29:53.220Z · LW · GW · Legacy · 142 comments

Contents

  What is up with Grothendieck?
  What is up with me?
  What does the math SAT measure?
  Twice exceptional gifted children
  What is the essence of mathematical ability?
  Readers are welcome to speculate on what Grothendieck had in mind in writing this.
None
142 comments

There's widespread confusion about the nature of mathematical ability, for a variety of reasons:

• Most people don't know what math is.
• Most people don't know enough statistics to analyze the question properly.
• Most mathematicians are not very metacognitive.
• Very few people have more than a casual interest in the subject.

If the nature of mathematical ability were exclusively an object of intellectual interest, this would be relatively inconsequential. For example, many people are confused about Einstein’s theory of relativity, but this doesn’t have much of an impact on their lives. But in practice, people’s misconceptions about the nature of mathematical ability seriously interfere with their own ability to learn and do math, something that hurts them both professionally and emotionally.

I have a long standing interest in the subject, and I’ve found myself in the unusual position of being an expert. My experiences include:

• Completing a PhD in pure math at University of Illinois.
• Four years of teaching math at the high school and college levels (precalculus, calculus, multivariable calculus and linear algebra)
• Personal encounters with some of the best mathematicians in the world, and a study of great mathematicians’ biographies.
• A long history of working with mathematically gifted children: as a counselor at MathPath for three summers, through one-on-one tutoring, and as an instructor at Art of Problem Solving.
• Studying the literature on IQ and papers from the Study of Exceptional Talent as a part of my work for Cognito Mentoring.
• Training as a full-stack web developer at App Academy.
• Doing a large scale data science project where I applied statistics and machine learning to make new discoveries in social psychology.

I’ve thought about writing about the nature of mathematical ability for a long time, but there was a missing element: I myself had never done genuinely original and high quality mathematical research. After completing much of my data science project, I realized that this had changed. The experience sharpened my understanding of the issues.

This is a the first of a sequence of posts where I try to clarify the situation. My main point in this post is:

There are several different dimensions to mathematical ability. Common measures rarely assess all of these dimensions, and can paint a very incomplete picture of what somebody is capable of.

What is up with Grothendieck?

I was saddened to learn of the death of Alexander Grothendieck several months ago. He's the mathematician who I identify with the most on a personal level, and I had hoped to have the chance to meet him. I hesitated as I wrote the last sentence, because some readers who are mathematicians will roll their eyes as they read this,  owing to the connotation (even if very slight) that the quality of my research might overlap with his. The material below makes it clear why:

“His technical superiority was crushing,” Thom wrote. “His seminar attracted the whole of Parisian mathematics, whereas I had nothing new to offer. — Rene Thom, 1958 Fields Medalist

"When I was in in Paris as a student, I would go to Grothendieck's seminar at IHES... I enoyed the atmosphere around him very much ... we did not care much about priority because Grothendieck had the ideas that we were working on and priority would have meant nothing. — Pierre Deligne, 1978 Fields Medalist

"[The IHES] is a remarkable place.. I knew about it before I came there; it was a legendary place because of Grothendieck. He was kind of a god in mathematics." — Mikhail Gromov, 2010 Abel Prize Winner

"On arriving at the IHES, we ordinary mathematicians share the same feeling that Muslims experience on a pilgrimage to Mecca. Here is the place were, for a dozen or so years, Grothendieck relentlessly explained the holy word to his apostles. Of that saga, only the apocrypha reached us in the form of big, yellow, boring-looking books edited by Springer. These dozens of volumes...are still our most precious working companion." — Ngo Bau Chau, 2010 Fields Medalist

Based on these remarks alone, it seems hard to imagine how I could be anything like Grothendeick. But when I read Grothendieck's own description of himself, it's hauntingly familiar. He writes:

"I've had the chance...to meet quite a number of people, both among my "elders" and among young people in my general age group, who were much more brilliant, much more "gifted" than I was. I admired the facility with which they picked up, as if at play, new ideas, juggling them as if familiar with them from the cradle - while for myself I felt clumsy. even oafish, wandering painfully up a arduous track, like a dumb ox faced with an amorphous mountain of things that I had to learn ( so I was assured), things I felt incapable of understanding the essentials or following through to the end. Indeed, there was little about me that identified the kind of bright student who wins at prestigious competitions or assimilates, almost by sleight of hand, the most forbidding subjects."

When I mentioned this to professor at a top math department who had taken a class with Grothendieck, he scoffed and said that he didn't believe it, apparently thinking that Grothendieck was putting on airs in the above quotation – engaging in a sort of bragging, along the lines of "I'm so awesome that even though I'm not smart I was still one of the greatest mathematicians ever."  It is hard to reconcile Grothendieck's self-description with how his colleagues describe him. But I was stunned by the professor's willingness to dismiss the remarks of somebody so great out of hand.

In fairness to the professor, I myself am much better situated to understand how Grothendieck's remarks could be sincere and faithful than most mathematicians are, because of my own unusual situation.

What is up with me?

I went to Lowell High School in San Francisco, an academic magnet school with ~650 students per year, who averaged ~630 on the math SAT (81st percentile relative to all college bound students). The math department was very stringent with respect to allowing students to take AP calculus, apparently out of a self-interested wish to keep their average AP scores as high as possible. So despite the strength of the school's students, Lowell only allowed 10% of students to take AP Calculus BC. I was one of them. The teachers made the exams unusually difficult for an AP Calculus BC course, so that students would be greatly over prepared for the AP exam . The result was that a large majority of students got 5's on the AP exam. By the end of the year, I had the 2nd highest cumulative average out of all students enrolled in AP Calculus BC. It would have been the highest if the average had determined exclusively by tests, rather than homework that I didn't do because I already knew how to do everything.

From this, people understandably inferred that I'm unusually brilliant, and thought of me as one of the select few who was a natural mathematician, having ability perhaps present in only 1 in 1000 people. When I pointed out that things had not always been this way, and that I had in fact failed geometry my freshman year and had to retake the course, their reactions tended to be along the lines of Qiaochu's response to my post How my math skills improved dramatically:

I find this post slightly disingenuous. My experience has been that mathematics is heavily g-loaded: it's just not feasible to progress beyond a certain point if you don't have the working memory or information processing capacity or whatever g factor actually is to do so. The main conclusion I draw from the fact that you eventually completed a Ph.D. is that you always had the g for math; given that, what's mysterious isn't how you eventually performed well but why you started out performing poorly.

It's not at all mysterious to me why I started out performing poorly. In fact, if Qiaochu had known only a little bit more, he would be less incredulous.

Aside from taking AP Calculus BC during my senior year, I also took the SAT, and scored 720 on the math section (96th percentile relative to the pool of college bound students). While there are many people who would be happy with this score, there were perhaps ~60 students at my high school who scored higher than me (including many of my classmates who were in awe of me). Just looking at my math SAT score, people would think very unlikely that I would come close to being the strongest calculus student in my year.

As far removed my mathematical ability is from Grothendieck's, we have at least one thing in common: our respective performances on some commonly used measures of mathematical ability are much lower than what most people would expect based on our mathematical accomplishments.

Hopefully these examples suffice to make clear that whatever mathematical ability is, it's not "what the math SAT measures." What the math SAT measures is highly relevant, but still not the most relevant thing.

What does the math SAT measure?

Just for fun, let's first look at what the College Board has to say on the subject. According to The Official SAT Study Guide

The SAT does not test logic abilities or IQ. It tests your skills in reading, writing and mathematics – the same subjects you're learning in school. [...] If you take rigorous challenging courses in high school, you'll be ready for the test.

Some of you may be shocked by the College Board's disingenuousness without any further comment. How would they respond to my own situation? Most hypothetical responses are absurd: They could say "Unfortunately, you were underprivileged in having to go to the high school ranked 50th in the country, where you didn't have access to sufficiently rigorous challenging courses" or "While you did take AP Calculus BC, you didn't take AP US History, and that would have further developed your mathematical reasoning skills" or "Our tests are really badly calibrated – we haven't been able to get them to the point where somebody with 99.9 percentile level subject matter knowledge reliably scores at the 97th percentile or higher."

Their strongest response would be to say that the test has been revised since I took it in 2002 to make it more closely aligned with the academic curriculum. This is true. But a careful examination of the current version of the test makes it clear that it's still not designed to test what's learned in school. For example, consider questions 16-18 in Section 2 of the sample test:

The grid above represents equally spaced streets in a town that has no one-way streets. F marks the corner where a firehouse is located. Points W, X, Y, and Z represent the locations of some other buildings. The fire company defines a building’s m-distance as the minimum number of blocks that a fire truck must travel from the firehouse to reach the building. For example, the building at X is an m-distance of 2, and the building at Y is an m-distance of 1/2 from the firehouse

1. What is the m-distance of the building at W from the firehouse?
2. What is the total number of different routes that a fire truck can travel the m-distance from F to Z ?
3. All of the buildings in the town that are an m-distance of 3 from the firehouse must lie on a...

I don’t think that rigorous, academic challenging courses build skills that enable high school students to solve these questions. They have some connection with what people learn in school – in particular, they involve numbers and distances. But the connection is very tenuous – they’re extremely far removed from being the best test of what students learn in school. They can be solved by a very smart 5th grader who hasn’t studied algebra or geometry.

The SAT Subject Tests are much more closely connected with what students (are supposed to) learn in school. And they’re not merely tests of what students have memorized: some of the questions require deep conceptual understanding and ability to apply the material in novel concepts. If the College Board wanted to make the SAT math section a test of what students are supposed to learn in school, they would do better to just swap it it with the Mathematics Level 1 SAT Subject Test.

If the SAT math section measures something other than the math skills that students are supposed to learn in school, what does it measure? The situation is exactly what the College Board explicitly disclaims it to be: the SAT is an IQ test. This accounts for the inclusion of questions like the ones above, that a very smart 5th grader with no knowledge of algebra or geometry could answer easily, and that the average high school student who has taken algebra and geometry might struggle with.

The SAT was originally designed as a test of aptitude: not knowledge or learned skills. Though I haven’t seen an authoritative source, the consensus seems to be that the original purpose of the test was to help smart students from underprivileged backgrounds have a chance to attend a high quality college – students who might not have had access to the educational resources to do well on tests of what students are supposed to learn in school. Frey and Detterman found that as of 1979, the correlations between SAT scores and IQ test scores were very high (0.7 to 0.85). The correlations have probably dropped since then, as there have in fact been changes to make the SAT less like an IQ test, but to the extent that the SAT differs from the SAT subject tests, the difference corresponds to the SAT being more of a test of IQ.

The SAT may have served its intended purpose at the time, but since then there’s been mounting evidence that the SAT has become a harmful force in society. By 2007, things had reached a point that Charles Murray wrote an article advocating that the SAT be abolished in favor of using SAT subject tests exclusively. This will have significance to those of you who know Charles Murray as the widely hated author The Bell Curve, which emphasizes the importance of IQ.

Twice exceptional gifted children

Let’s return to the question of reconciling my very strong calculus performance with my relatively low math SAT score. The difference comes in substantial part from my having a much greater love of learning than is typical of people of similar intelligence. I think that the same was true of Grothendieck.

I could have responded to Qiaochu’s suggestion that I had always had very high intelligence and that that’s why I was able to learn math well by saying “No, you’re wrong, my SAT score shows that I don’t have very high intelligence, the reason that I was able to learn math well is that I really love the subject.” But that would oversimplify things. In particular, it leaves two questions open:

• A large part of why I failed geometry my freshman year of high school is that I wasn’t interested in the subject at the time. I only got interested in math after getting interested in chemistry my sophomore year. But almost nobody at my high school was interested in geometry, and almost everybody passed geometry. What made me different?
• Can a love of learning really boost one’s percentile from 1 in 30 to 1 in 1000? The gap seems awfully large to be accounted for exclusively by love of learning. And what of Grothendieck, for whom the gap may have been far larger?

Partial answers to these questions come from the literature on so-called “Twice Exceptional” (2e) children. The label is used broadly, to refer to children who are intellectually gifted and also have some sort of disability.

The central finding of the IQ literature is that people who are good at one cognitive task tend to be good at any another cognitive task. For example, people who have better reaction time tend to also be better at arithmetic, better at solving logic puzzles, better able to give coherent explanations of real world concepts, and better able to recall a string of numbers that are read to them. When I was a small child, my teachers noticed that I was an exception to the rule: I had a very easy time learning some things and also found it very difficult to learn others. They referred me to a school psychologist, who found that I had exceptionally high reasoning abilities, but only average short term memory and processing speed: a 3 standard deviation difference.

There’s a sense in which my situation is actually not so unusual. The finding that people who are good at one cognitive task tend to be good at another is based on the study of people of average intelligence. It becomes less and less true as you look at people of progressively higher intelligence. Twice exceptional children are not very rare amongst intellectually gifted children.  Linda Silverman writes

Gifted children may have hidden learning disabilities. Approximately one-sixth of the gifted children who come to the Center for testing have some type of learning disability—often undetected before the assessment—such as central auditory processing disorder (CAPD), difficulties with visual processing, sensory processing disorder, spatial disorientation, dyslexia, and attention deficits. Giftedness masks disabilities and disabilities depress IQ scores. Higher abstract reasoning enables children to compensate to some extent for these weaknesses, making them harder to detect.

This starts to explain why I failed geometry during my freshman year of high school. The material was boring and I wasn’t very focused on grades. But I also genuinely found it difficult to an extent that my classmates didn't. Learning the material the way in which the course was taught required a lot of memorization – something that I was markedly worse at than my classmates at Lowell, who had been selected for having high standardized test scores.

It also explains why I didn’t score higher than 720 on the math section of the SAT. It wasn’t because I couldn’t answer questions like the ones that I pasted above. It was because some of the math SAT questions are engineered to trip up students who forget exactly what a problem asked for, or who are prone to arithmetic errors. Often a multiple choice question will have one wrong answer for every such mistake that a student might make. I used to think that this was a design flaw, and that the test makers didn’t know that they were penalizing minor mistakes very heavily. No – it wasn’t a design flaw – they designed the test that way on purpose. The questions test short-term memory as a proxy to IQ. I tried to avoid mistakes by being really systematic about my work, and not take shortcuts. But it wasn’t enough given the time constraints – making 3 minor mistakes on any combination of 54 questions is enough to reduce one’s score from 800 to 720.

It's plausible that something similar was true of Grothendieck.

It’s probably intuitively clear even to readers who are not mathematicians that math is not about being able to avoid making 3 minor mistakes on 54 questions. It’s very helpful to be quick and accurate, and my mathematical ability is far lower than it would have been if my speed and accuracy were substantially greater, but speed and accuracy are not the essence of mathematical ability.

What is the essence of mathematical ability?

I've only just scratched the surface of the subject of mathematical ability in this post, largely focusing on describing what mathematical ability isn't rather than what mathematical ability is. In subsequent posts I'll describe mathematical ability in more detail, which will entail a discussion of what math is. I'll also address the question of how one can improve one's mathematical ability.

Intelligence is highly relevant and largely genetic, but there are other factors that are collectively roughly as important, some of which are things that individuals are in fact capable of developing. For now, I'll offer a teaser, which will be obscure to readers who lack substantial additional context, and which paints a very incomplete picture even when understood deeply, but which should nevertheless serve as food for thought. Grothendieck wrote:

In our acquisition of knowledge of the Universe ( whether mathematical or otherwise) that which renovates the quest is nothing more nor less than complete innocence. It is in this state of complete innocence that we receive everything from the moment of our birth. Although so often the object of our contempt and of our private fears, it is always in us. It alone can unite humility with boldness so as to allow us to penetrate to the heart of things, or allow things to enter us and taken possession of us.

This unique power is in no way a privilege given to "exceptional talents" - persons of incredible brain power ( for example), who are better able to manipulate, with dexterity and ease, an enormous mass of data, ideas and specialized skills. Such gifts are undeniably valuable, and certainly worthy of envy from those who ( like myself) were not so endowed at birth," far beyond the ordinary".

Yet it is not these gifts, nor the most determined ambition combined with irresistible will-power, that enables one to surmount the "invisible yet formidable boundaries " that encircle our universe. Only innocence can surmount them, which mere knowledge doesn't even take into account, in those moments when we find ourselves able to listen to things, totally and intensely absorbed in child play.

Readers are welcome to speculate on what Grothendieck had in mind in writing this.

Cross-posted from my website.

142 comments

Comments sorted by top scores.