Hedging our Bets: The Case for Pursuing Whole Brain Emulation to Safeguard Humanity's Future

Folk at the Singularity Institute and the Future of Humanity Institute agree that it would probably (but unstably in the face of further analysis) be better to have brain emulations before de novo AI from an existential risk perspective (a WBE-based singleton seems more likely to go right than an AI design optimized for ease of development rather than safety). I actually recently gave a talk at FHI about the use of WBE to manage collection action problems such as Robin Hanson's "Burning the Cosmic Commons" and pressures to cut corners on safety of AI development, which I'll be putting online soon. One of the projects being funded by the SIAI Challenge Grant ending tonight is an analysis of the relationship between AI and WBE for existential risks.

However, the conclusion that accelerating WBE (presumably via scanning or neuroscience, not speeding up Moore's Law type trends in hardware) is the best marginal project for existential risk reduction is much less clear. Here are just a few of the relevant issues:

1) Are there investments best made far in advance with WBE or AI? It might be that the theory to build safe AIs cannot be rushed as much as institutions to manage WBEs, or it might be that WBE-regulating institutions require a buildup of political influence over decades.

2) The scanning and neuroscience knowledge needed to produce WBE may facilitate powerful AI well before WBE, as folk like Shane Legg suggest. In that case accelerating scanning would mean primarily earlier AI, with a shift towards neuromorphic designs.

3) How much advance warning will WBE and AI give, or rather what is our probability distribution over degrees of warning? The easier a transition is to see in advance, the more likely it will be addressed by those with weak incentives and relevant skills. Possibilities with less warning, and thus less opportunity for learning, may offer higher returns on the efforts of the unusually long-term oriented.

Folk at FHI have done some work to accelerate brain emulation, e.g. with the WBE Roadmap and workshop, but there is much discussion here about estimating the risks and benefits of various interventions that would go further or try to shape future use of the technology and awareness/responses to risks.

It would have been good to check this suggested post topic on an Open Thread, first - in fact I should get around to editing the FAQ to suggest this for first posts.

Perhaps the AI rules wisely and well, and can give us anything we want, "save relevance".

In addition to the retreads that others have pointed out on the upload safety issue, this is a retread of the Fun Theory Sequence:

http://lesswrong.com/lw/xy/the_fun_theory_sequence/

Also the way you phrased the above suggests that we build some kind of AI and then discover what we've built. The space of mind designs is very large. If we know what we're doing, we reach in and get whatever we specify, including an AI that need not steal our relevance (see Fun Theory above). If whoever first reaches in and pulls out a self-improving AI doesn't know what they're doing, we all die. That is why SIAI and FHI agree on at least wistfully wishing that uploads would come first, not the relevance thing. This part hasn't really been organized into proper sequences on Less Wrong, but see Fake Fake Utility Functions, the Metaethics sequence, and the ai / fai tags.

Focusing on slow-developing uploads doesn't cause slower development of other forms of AGI. Uploads themselves can't be expected to turn into FAIs without developing the (same) clean theory of de novo FAI (people are crazy, and uploads are no exception; this is why we have existential risk in the first place, even without any uploads). It's very hard to incrementally improve uploads' intelligence without affecting their preference, and so that won't happen on the first steps from vanilla humans, and pretty much can't happen unless we already have a good theory of preference, which we don't. We can't hold constant a concept (preference/values) that we don't understand (and as a magical concept, it's only held in the mind; any heuristics about it easily break when you push possibilities in the new regions). It's either (almost) no improvement (keep the humans until there is FAI theory), or value drift (until you become intelligent/sane enough to stop and work on preserving preference, but by then it won't be human preference); you obtain not-quite-Friendly AI in the end.

The only way in which uploads might help on the way towards FAI is by being faster (or even smarter/saner) FAI theorists, but in this regard they may accelerate the arrival of existential risks as well (especially the faster uploads that are not smarter/saner). To apply uploads specifically to FAI as opposed to generation of more existential risk, they have to be closely managed, which may be very hard to impossible once the tech gets out.

Okay, let's go on the brain-simulation path. Let's start with something simple, like a lobster or a dog... oh wait, what if it transcends and isn't human-friendly. All right, we'll stick to human brains... oh wait, what if our model of neural function is wrong and we create a sociopathic copy that isn't human-friendly. All right, we'll work on human brain regions separately, and absolutely make sure that we have them all right before we do a whole brain... oh wait, what if one of our partial brain models transcends and isn't human-friendly.

And while you, whose reason for taking this path is to create a human-friendly future, struggle to avoid these pitfalls, there will be others who aren't so cautious, and who want to conduct experiments like hotwiring together cognitive modules that are merely brain-inspired, just to see what happens, or in the expectation of something cool, or because they want a smarter vacuum cleaner.

An important fact is that whether your aim is Friendly AI or mind uploading, either way, someone has to do neuroscience. As the author observes,

Such research [FAI] must not only reverse-engineer consciousness, but also human notions of morality.

In FAI strategy as currently conceived, the AI is the neuroscientist. Through a combination of empirical and deductive means, and with its actions bounded by some form of interim Friendliness (so it doesn't kill people or create conscious sim-people along the way), the AI figures out the human decision architecture, extrapolates our collective volition as it would pertain to its own actions, and implements that volition.

Now note that this is an agenda each step of which could be carried out by all-natural human beings. Human neuroscientists could understand the human decision process, discover our true values and their reflective equilibrium, and act in accordance with the idealized values. The SIAI model is simply one in which all these steps are carried out by an AI rather than by human beings. In principle, you could aim to leave the AI out of it until human beings had solved the CEV problem themselves; and only then would you set a self-enhancing FAI in motion, with the CEV solution coded in from the beginning.

Eliezer has written about the unreliability of human attempts to formulate morality in a set of principles, using just intuition. Thus instead we are to delegate this investigation to an AI-neuroscientist. But to feel secure that the AI-neuroscientist is indeed discovering the nature of morality, and not some other similar-but-crucially-different systemic property of human cognition, we need its investigate methodology (e.g. its epistemology and its interim ethics) to be reliable. So either way, at some point human judgment enters the picture. And by the Turing universality of computation, anything the AI can do, humans can do too. They might be a lot slower, they might have to do it very redundantly to do it with the same reliability, but it should be possible for mere humans to solve the problem of CEV exactly as we would wish a proto-FAI to do.

Since the path to human mind uploading has its own difficulties and hazards, and still leaves the problem of Friendly superintelligence unsolved, I suggest that people who are worried about leaving everything up to an AI think about how a purely human implementation of the CEV research program would work - one that was carried out solely by human beings, using only the sort of software we have now.

Why should an uploaded superintelligence based on a human copy be any innately safer than an artificial superintelligence? Just because humans are usually friendly doesn't mean a human AI would have to be friendly. This is especially true for a superintelligent human AI, which may not even be comparable to its original human template. Even the friendliest human might be angry and abusive when they're having a bad day.

Your idea that a WBE copy would be easier to undergo a relatively more enhanced supervised, safe growth, is basically an assumption. You would need to argue this in much more detail for it to merit deeper consideration.

Also, you cannot assume that an uploaded human superintelligence would be more constrained, as in "...after a best-effort psychiatric evaluation (for whatever good that might do) gives it Internet access". This is related to the the AI-box problem, where it is contended that a superintelligence could not be contained, no matter what. Personally I dispute this, but at least it's not something to be taken for granted.

These "Whole Brain Emulation" discussions are surreal for me. I think someone needs to put forward the best case they can find that human brain emulations have much of a chance of coming before engineered machine intelligence.

The efforts in that direction I have witnessed so far seem feeble and difficult to take seriously - while the case that engineered machine intelligence will come first seems very powerful to me.

Without such a case, why spend so much time and energy on a discussion of what-if?

Humans misuse power. It doesn't seem to have occurred to you that humans with power frequently become corrupt. So, you want to emulate humans, in order to avoid corruption, when we know that power corrupts humans? Our brain structures have evolved for how many millions of years, all the while natural selection has been favoring those most efficient at obtaining and exploiting power whenever it has provided a reproduction advantage? I think we're better off with something man-made, not something that's optimized to do that! Downvote

Also, there is a point at which an intelligent enough person is unable to communicate meaningfully with others, let alone a super intelligent machine. There comes a point where your conceptual frameworks are so complex that nothing you say will be interpreted correctly without a huge amount of explanation, which your target audience does not have the attention span for. This happens on the human level, with IQ gaps of above 45 IQ points (ratio tests). Look up something called the "optimal IQ range". Want some evidence? Why do the presidents that we vote in have IQ's so (relatively) near to average when it would theoretically make more sense to vote in Einsteins with IQ's of 160? It's because Einsteins are too complicated. Most people don't have the stamina to do all the thinking required to understand all of their ideas enough to determine whether a political Einstein would be the better choice. The same problem will happen with AI. People won't understand an AI that smart because they won't be able to review it's reasoning. That means they won't trust it, are incapable of agreeing with it and therefore would not be likely to want to be led by it.

Brain emulation will do nothing to ensure any of the benefits you hope for.

The only downside of this approach I can see is that an upload-triggered Unfriendly singularity may cause more suffering than an Unfriendly AI singularity; sociopaths may be presumed to have more interest in torture of people than a paperclip-optimizing AI would have.

What about those of us who would prefer indefinite human-directed torture to instantaneous cessation of existence? I have no personal plans to explore masochism in that sort of depth, particularly in a context without the generally-accepted safety measures, but it's not the worst thing I can imagine. I'd find ways to deal with it, in the same sense that if I were stranded on a desert island I would be more willing to gag down whatever noxious variety of canned fermented fish was available, and eventually learn to like it, rather than starve to death.

I agree with a lot of your points about the advantages of WBE vs friendly AI. That said, look at the margins. Quite a few people are already working on WBE. Not very many people are working on friendly AI. Taking this into consideration, I think an extra dollar is better spent on FAI research than WBE research.

Also, a world of uploads without FAI would probably not preserve human values for long. The uploads that changed themselves in such a way to grow faster (convert the most resources or make the most copies of themselves) would replace uploads that preserved human values. For example, an upload could probably make more copies of itself it if deleted its capacities for humor and empathy.

We already have a great many relatively stable and sane intelligences.

I don't think any human being is stable or sane in the way FAI would be stable and sane.

Whole Brain Emulation will likely come long after engineered artificial intelligence arrives. Why pump money into Whole Brain Emulation projects? They will still come too late - even with more funding. I figure it's like throwing money down the drain.