Productivity as a function of ability in theoretical fields

A fair part of the abnormal productivity of (for example) Einstein & von Neumann is probably down to when they were born.

To have a hope in hell of being the Greatest Of All Time, a scientist has to work in a scientific culture with (1) a supply of big & solvable open problems to attack, (2) sensible earlier ideas & theories on which to build, and often also (3) good enough technology to collect evidence that might confirm new theories. But it follows that a scientist doesn't have a chance of being the GOAT unless they're born at the right time. If they're born too early they end up like, say, Aristotle, someone impressive and historically important but nonetheless long superseded; if they're born too late, most of the prestigious solvable problems will have been picked off by someone else, and even if the scientist picks one off themselves, they're still trying to stand out in a more crowded field.

If Einstein had been born in 1500, or 1950, he might've been a Galileo or a Copernicus, or a Bekenstein or a Perlmutter, but he wouldn't've been an Einstein.

Of course this can't the whole explanation for why scientists differ so much in productivity, because you still have big differences in productivity even within a cohort of scientists all the same age. Nonetheless, if we want to explain why we're talking about Einstein instead of e.g. John Archibald Wheeler, the fact that Einstein was in the right time and the right place probably has a lot to do with it. While I agree that part of the reason for von Neumann & Einstein's vast productivity is the nonlinearity of the ability-productivity relationship, I think it's worth bearing in mind the right-time-right-place effect as well. (Knowing about that effect also makes Yudkowsky's brain-line diagram less surprising.)

You're implicitly modeling difficulty as a single number, and ability as another single number that determines the maximum difficulty of the problems you can solve.

However, according to Feynman, you can sometimes solve problems that smarter people can't by having "a different box of tools". The quote:

That book also showed how to differentiate parameters under the integral sign - it's a certain operation. It turns out that's not taught very much in the universities; they don't emphasize it. But I caught on how to use that method, and I used that one damn tool again and again. So because I was self-taught using that book, I had peculiar methods of doing integrals.

The result was, when guys at MIT or Princeton had trouble doing a certain integral, it was because they couldn't do it with the standard methods they had learned in school. If it was contour integration, they would have found it; if it was a simple series expansion, they would have found it. Then I come along and try differentiating under the integral sign, and often it worked. So I got a great reputation for doing integrals, only because my box of tools was different from everybody else's, and they had tried all their tools on it before giving the problem to me.

I don't think this changes the picture very much, though. You just divide the people in the field according to their box of tools. Pick the group with the box of tools that can solve the problem, and your model applies to them.

So how can they both be true? The answer is, obviously, that they are measuring different things.

When I first saw them, I assumed they were measuring the same thing, but your picture was just zoomed in.

What you effectively mean is that most people don't reach the edge described very nicely visually here:

http://www.businessinsider.com/the-illustrated-guide-to-a-phd-2012-3

Consider two models: In one model, the difference in intelligence between Einstein and a village idiot are tiny, but productivity in the sciences is highly sensitive to intelligence in such a way that a small number of slightly more intelligent people have a disproportionate impact. In the other model, the difference between Einstein and a village idiot are vast, and a scientist's impact is roughly proportional to their intelligence. How can you distinguish between these two models? You can't, because they aren't actually different models; they're the same model using different parametrizations for intelligence.

The reason Eliezer Yudkowsky makes these arguments is because he wants to make certain points about what happens when you extrapolate to much higher intelligence. However, you cannot make even qualitative points without understanding how things are parametrized. For example, in general relativity it is possible for points to appear to be singularities under one coordinate system but behave perfectly smoothly in another. Thus I am unsatisfied at how you sweep the issue of how ability is defined under the rug.

Compare:

I argued in this post that the differences in capability between different political leaders are vast. Genghis Khan's conquering of much of the known world or Alexander the Great's conquering of Persia (and a number of other countries) are arguably at least hundreds of times greater than that of an average Angela Merkel or David Cameron.

While capability certainly correlates with impact, the two are not synonymous by any means. Impact is a function of capability and circumstances, often "by luck" chancing upon a subproblem behind which a large discovery is hiding. Just as I wouldn't expect a modern day Genghis Khan to transform Italy into some conquering war machine, so it may be noted that Einstein chanced* upon fields of research that (in retrospect) were treasure troves of breakthrough discoveries.

* John von Neumann's universal genius may be used as a counterexample, but then "capability" may be too one-dimensional of a descriptor, instead maybe there is some quality of "able to sniff out suitable problems to work on", i.e. exceptional capability in task selection over exceptional capability "in general".

I've also been thinking about these issues for a while, and I'm glad you've taken the plunge and posted up some thoughts. I think one model that might be worth thinking of is that physics (and other fields) have a 'tournament game' dynamic, such that one gets significant positional rewards.

So the best physicists have much higher measures of productivity than average physicists not because they are so much smarter, nor because physics happens to be extraordinarily sensitive to differences in physics ability, but because things like publications, patents, fame etc. are strongly positional, and so the very best can get outsize rewards of these things. On this model, if there was never an Einstein (or a Neumann), their discoveries would have been made by others not long after they were actually discovered.

This 'tournament' model seems to do very well at things like sport and the arts. The reason Nadal et al. get so much more money and prestige than an average pro is that you might as well watch the very best (instead of the almost-best) play tennis against each other, and all the reward structures in pro tennis are positional rather than objective. You might want to use the same story in the arts, especially (if you buy Taleb) given how modern technology allows the best performers to leverage their output: why listen to a very good pianist in concert when you can have the world best on CD? Etc. Etc.

Ruling against this model for physics and science would be there is some objective measure of achievement in terms of theory, discovery, etc. But I think on the tournament model still makes a good fit: we should think things like discovery and theory generation have significant positional component (all the kudos goes to the first person there, and so the fractionally better physicists can capture outsize rewards by getting to the answer a couple of weeks before their not-quite-so-good fellows).

Perhaps the best evidence I can think of in support of the tournament model would be that you see very similar 'power law' dynamics in terms of fame or citation count across many fields across a large span of time: this fits a positional model (and, to be fair, a 'big difference in human ability' model), but seems harder to fit on an 'high sensitivity of productivity to ability' model, as it would seem odd that across so many different fields, across so much of their development, the 'sensitivity' area should remain constantly centered on the right-tail of human ability.

(It follows that an artificial intelligence just a tiny bit smarter than Einstein and von Neumann would be as much more productive than them as they are in relation to other mathematician/physicists).

I don't think this necessarily does follow. I think it only follows that such an AI would be much more productive than average in a population of Einsteins.

Interesting article, but I think there's a more useful conclusion to draw from the idea that major problems in technical fields are precisely those too difficult for most people to solve. While a little extra intelligence can make the difference, it's also likely that an unusual tool set will make the difference. The key is that you have to try something that nobody has tried before. If you want to solve big problems in technical fields, then your biggest relative advantage should come from studying a wide variety of other fields, in search of methods that can generalize to any of the problems you're interested in. On the other side of the equation, you need to know about lots of big problems, so you can try all your methods on a wide variety of problems.

In short, the key to solving hard problems isn't just more intelligence. The key is relative advantage. Intelligence can provide one relative advantage. Rationality and Bayesian probability/statistics can provide another, as Metamed is demonstrating. And of course, we need to find problems where our relative advantages give us the most leverage.

Reading this reminded me of Terrence Tao's blog post about how you don't have to be a genius to do math: http://terrytao.wordpress.com/career-advice/does-one-have-to-be-a-genius-to-do-maths/.

I think you are severely oversimplifying "intelligence" and "productivity" into 1-dimensional quantities. In my experience, "genius" (i.e. the insight that solves a problem) is about acquiring a bag of tricks to throw at new problems, and translating your insight into a solution is the result of practice.

There is a much simpler way in which "village idiot" is right next to "Einstein" on the intelligence graph. If you have done any work in computer vision, computer hearing, navigation, or robotics, you might realize that a mouse beats beats decades of work by the smartest people in your field. Most people wouldn't think to brag that they can find their way home from over a block away! If Einstein were missing, lets say, the visual cortex, auditory cortex, nor the chunk of brain required to even read braille, the village idiot would probably pass as smarter. It's only a small portion of the brain that is vaguely dedicated to the sort of thing the IQ test is supposed to measure, plus lots of the specialized bits of the brain can be commandeered to assist (eg the visual cortex for visualizing). Long story short, maybe 80% of Einstein's brain is specialized for tasks any village idiot can do, and his bonus to intelligence is mostly the other ~20% (prefrontal cortex) plus no doubt a better ability to commandeer the other parts into doing more interesting tasks.

To this, add as prerequisites some luck to have the right problems and resources, and the fact that you have to beat the rest of humanity to it, which requires way more work ethic than most humans have (and I get the impression mental work ethic is different than for physical labor). All this makes it really easy to greatly discount the village idiots intelligence and greatly overstate Einstein's intelligence. If there's a mouse on your intelligence chart, Einstein and the village idiot will be nearly touching.

There hasn't been an Einstein in over 100 years... Are you saying there are no more extra smart people?