Modest Superintelligences

Do we need "very reliably"? If not, feeding them Eliezer's Sequences at a young age might work well enough.

Let's think of examples of groups of ten thousand genius-level people working together towards a common narowly-defined goal.

Wikipedia claims that the LHC “was built in collaboration with over 10,000 scientists and engineers from over 100 countries, as well as hundreds of universities and laboratories”. I doubt they were all von Neumann level, and I imagine most of them weren’t working exclusively on the LCH. And no matter how nice scientists and engineers are, the group probably didn’t cooperate as well the one Wei proposed. (Although diversity probably does count for something.)

Other groups of similar size I can think of are NASA, IBM, Google and Microsoft. (Though, like the LHC, I don’t think they’re hiring only von Neumann level geniuses. Probably many multinational companies would exceed the size but would be even further from the genius requirements.) But they don’t quite work in a single direction (NASA has many missions, Google and Microsoft have many products).

That said, I wouldn’t object strongly to calling such groups weakly superintelligent. Building stuff like the LHC or the Apollo program in ten years is so vastly beyond the ability of a single man that I don’t quite classify an entity that can do it as a “human-level intelligence”, even though it is assembled from humans.

(Also, I could see a group like this building an MSI-2, though it’d take more than ten years if starting now.)

Part of the reason is due to coordination problems, which I think would be reduced if the group consisted of clones of a single person with similar education and upbringing, and hence similar values/goals.

Another part of the reason is that we simply don't have that many von Neumanns today. The [big number] of educated people that you see in the world consist almost entirely of people who are much less intelligent compared to von Neumann.

Why aren't [big number] of educated people a superintelligence now?

They are. Many collections of individuals (e.g. tech companies, hedge funds, PACs, etc.) seem to do rather a lot more than an individual human could. Likewise, humanity as a whole could be classified as a superintelligence (and possibly a recursively self-improving one: see the Flynn effect). The idea is not that large numbers of intelligent people aren't a superintelligence, it's that 10000 von Neumanns would be a more powerful superintelligence than most groups of highly intelligent people.

What is the current bottleneck on MS-1? Are we better off raiding Neumann's corpse, extracting the DNA and then implanting all the embryos we can make?

We can't clone humans at the moment. Even attempts to derive human stem cell lines from cloning have been disappointing, and reproductive cloning would face much higher barriers. Even if it could be made to work Dolly-style, you would still be producing huge numbers of miscarriages, early deaths, and damaged offspring for each success. That would not only increase the economic cost, but be incredibly unattractive for parents and a PR nightmare.

Or are we better off with the current strategies of sequencing intelligent people to uncover the genetics of intelligence, which would then allow embryo selection or engineering?

We can do embryo selection, but the relevant alleles would need to be identified in large studies (with the effectiveness of selection scaling with the portion of variation explained). The BGI study may expose a number of candidates, but I would expect the majority to be captured through linking genetic data collected for other reasons (or as part of comprehensive biobanks) to be matched to military or educational testing data, tax records, and so forth.

With the latter, the main bottleneck seems to be the cost of sequencing

Recent results suggest that much of intelligence variation in middle-class samples can be explained by the SNPs tracked by modern microarrays (used by 23andme and similar companies), without need for whole genome sequencing (which would capture rarer variants). The bottleneck seems to be more a matter of large studies (SNP-chip or whole genome) than whole-genome sequencing as such. BGI is not using whole genome sequencing, for instance.

ETA: Early applications would also remain costly in out-of-pocket terms (although they would easily pay for themselves in lifetime earnings and tax payments): IVF with PGD costs tens of thousands of dollars per successful pregnancy, allowing the selection of perhaps one embryo in six. If embryos could be exactly ranked by cognitive ability or scientific potential (a tall order, near 100% of heritability explained), that could be enough to boost the distribution by a standard deviation of genetic potential. Since environmental factors matter too, that might deliver only 8-12 IQ points. With imperfect knowledge of the genetics of intelligence the gains would be less. Likewise if parents (or governments) also select for other traits (height, appearance, longevity, personality, etc).

A method for producing huge numbers of eggs (from stem cells, or maturing eggs from ovarian biopsies) could make this technique much more powerful (selecting from more embryos per pregnancy, and eliminating the scarcity of eggs from sought-after donors), but that would require advances in biology and reproductive medicine. There are experimental versions, and claims that they will be made to work within a decade, but such predictions have a poor track record in other medical fields.

What is the current bottleneck on MS-1? Are we better off raiding Neumann's corpse, extracting the DNA and then implanting all the embryos we can make?

BTW, does anyone know of the...status...of said corpse? 'Tis but a purely academic curiosity, I assure you.

That's a really interesting point. I'm not sure how to do the math myself, so I wonder if anyone can help verify this. Also, is this assuming a population-average upbringing/education? What if we give the clones the best upbringing and education that money can buy? (I assumed a budget of $10 million per clone in the OP.)

If intelligence is 50% genetic

I should have questioned this as well. According to Wikipedia:

Twin studies of adult individuals have found a heritability of IQ between 57% and 73%[6] with the most recent studies showing heritability for IQ as high as 80%[7] and 86%.[8].

Here are some posts/threads where I talk about my reasons: 1 2 3.

It is unclear to which extent the truly exceptional ability is a result of just being lucky that the random parts of the development process resulted in right kind of circuitry.

In twin studies this is partitioned out as "non-shared environment." In the central range it explains less variation than genetics, but more than household-level environment.

Why is this evidence that he was smarter than you thought rather than dumber than you thought?

Smart and rational aren't quite the same thing though, are they? (Smart as in g.)