Will the ems save us from the robots?

A few random thoughts on ems... (None of this is normative - I'm just listing things I could imagine being done, not saying they're a good or a bad idea).

After the creation of the first em...

A lot seems to hinge on whether em technology:

• can be contained within research institutes that have strong ethical & safety guidelines, good information security and an interest in FAI research, or
• can't be contained and get released into the economic jungle

Bear in mind general difficulties of information security, as well as the AI box experiment which may apply to augmented ems.

Ems can hack themselves (or each other):

• Brain-computer interfacing seems much easier if you're an em
• Create a bunch of copies of yourself, create fast brain-to-brain interfaces and with a bit of training you have a supermind
• Grow more neocortex; simulate non-Euclidean space and grow into configurations which are way larger than what would fit into your original head
• Research ways to run on cheaper hardware while maintaining subjective identity or economic productivity
• Can restore from backups if something goes wrong

With those kinds of possibilities, it seems that ems could drift quite a long way into mind design space quite quickly, to the point where it might make more sense to view them as AGIs than as ems.

In the released-into-the-jungle scenario, the ems which are successful may be the ones which abandon their original human values in favor of whatever's successful in an economic or Darwinian sense. Risk of a single mind pattern (or a small number of them) coming to dominate.

Someone has to look after the bioconservatives, i.e. people who don't want their mind uploaded.

Em technology can allow enormous amounts of undetectable suffering to be created.

Ems -> enormous economic growth -> new unforeseeable technologies -> x-risks aplenty.

I can't think of a scenario where ems would make AGI creation less likely to happen (except if they trigger some (other) global catastrophe first). Maybe "we already have ems so why bother researching AI?" It seems weak - ems are hardware-expensive and anyway people will research AGI just because they can.

Robin Hanson's vision of em-world seems to include old-fashioned humans surviving by owning capital. I'm not sure this is feasible (requires property rights to be respected and value not to be destroyed by inflation).

Before the creation of the first em...

The appearance of a serious WBE project might trigger public outcry, laws etc. for better or for worse. Once people understand what an em is they are likely to be creeped out by it. Such laws might be general enough to retard all AI research; AI/AGI/em projects could be driven underground (but maybe only when there's a strong business case for them); libertarians will become more interested in AI research.

WBE research is likely to feed into AGI research. AGI researchers will be very interested in information about how the human brain works. (There's even the possibility of a WBE vs AGI "arms race").

• Shulman & Salamon, Whole brain emulation as a platform for creating safe AGI.

• Shulman, Whole brain emulation and the evolution of superorganisms

• The stuff Carl & Nick have been passing back and forth recently. (I suspect you've seen this, Stuart?)

• The post-Summit2011 workshop report I wish I had more time to write.

WBE singleton or faster-than-realtime research project conducted by WBEs seem to be the only feasible scenarios for gaining more time for solving FAI before a global catastrophe or significant value drift. On the other hand, WBE tech released into the wild would probably quickly generate a humane-value-indifferent AGI (either the old-fashioned way or via value drift of modified uploads or upload-based processes).

(A half-point between that and a FAI is running an AGI that implements a human-conducted FAI research project specified as an abstract computation.)

Time-accelerated research with forking seems like the only safe (sane) thing one could do with WBEs. The human brain breaks easily if anything too fancy is attempted. Not to mention that unlike UFAIs which tile the universe with paperclips or something equally inane insane neuromorphs might do something even worse instead, a la "I Have No Mouth And I Must Scream". The problem of using fiction as evidence is evident but since most fictional UFAIs are basically human minds with a thin covering of what the author thinks AIs work like I think the fail isn't so overwhelmingly strong with this one; unlike a "true" AGI a badly designed neuromorph might definitely feel resentment towards a low tribal status, for example. The risks of having a negative utilitarian or a deep ecologist as an emulation whose mind might be severely affected by unknown factors are something Captain Obvious would be very enthusiastic about.

Even simple time acceleration with reasonable sensory input might have unforeseen problems when the rest of the world runs in slow motion. Multiply it if there is only one mind instead of several that can still have some kind of a social life with each other. Modify the brain to remove the need for human interaction and you're back in the previous paragraph.

Now, considering the obvious advantages of copying the best AI researchers and running them multiple times faster than real-time it would be quite reasonable to expect AGI to follow quickly after accelerated WBE. This could mean that WBE might not be very widespread; the organization that had the first fast EMs would probably focus on stacking supercomputing power to take over the future light cone instead of just releasing the tech to take over the IT / human enhancement sector.

If the organization is sane it could significantly increase the chance of FAI as the responsible researchers had both speed and numbers advantage over the non-WBE scenario. On the other hand, if the first emulators don't focus on creating human-compatible AGI the chance of Fail would be the one growing massively.

It seems that ems are much harder to make friendly than a general AI. That is, some of what we have to fear from unfriendly AIs is present in powerful WBEs too, and you can't just build a WBE provably friendly to start with; you have to constrain it or teach it to be friendly (both of which are considered dangerous methods of getting to friendliness).

Hasn't Eliezer argued at length against ems being safer than AGIs? You should probably look up what he's already written.

I've posted a similar thread here. I focus a bit more on the implications for somebody wanting to accelerate the future.

1. If WBE and the creation of VR is the ultimate utilitarian goal, then why create AGI? It just adds existential risk with no utilitarian benefit.
2. More likely. FAI researches can be uploaded and think quicker/better. The AGI that is designed will benefit from the knowledge needed to create WBE.

One important question is if getting to whole brain emulations first would make subsequent AGI creation 1. more or less likely to happen, 2. more or less likely to be survivable.

Only important if there's much chance of whole brain emulations coming "first" - which still looks vanishingly unlikely to me. Think: aeroplane, not bird emulation - or you go too far off the rails.

Maybe some kinds of ems could tell us how likely Oracle/AI-in-a-box scenarios were to be successful? We could see if ems of very intelligent people run at very high speeds could convince a dedicated gatekeeper to let them out of the box. It would at least give us some mild evidence for or against AIs-in-boxes being feasible.

And maybe we could use certain ems as gatekeepers - the AI wouldn't have a speed advantage anymore, and we could try to make alterations to the em to make it less likely to let the AI out.

Minor bad incidents involving ems might make people more cautious about full-blown AGI (unlikely, but I might as well mention it).

It does seem like there are all sorts of experiments you could do with a brain in a computer that would get you closer to AI, and they would also be the researchers. Are there any counterbalancing factors that could lead to brains in computers making AGI development less likely? The only ones I can think of would be tiny.

Once we have ems, it's certain that enough computer power would be available for AGI.

My speculation is that the sort of people who would be first in line to be uploaded would also tend to be the sort of people who, given enough computer power and time, might be able to brute-force some sort of AGI. So, if somehow we start making ems before we get FAI, my guess is that AGI will be forthcoming quite shortly, and it will be unfriendly.

But I have a hard time seeing how we can actually get that much computer power without some fool having already made some sort of UFAI.

(All this is under the assumption that ems will not be terribly efficient at first because we'll be running them without actually understanding them-- thus simulating perhaps 10x-100x+ more stuff than we actually need to.)

So, I say more likely to #1, and less likely to #2.

Some possible benefits from emulated brain (EMB):

1) It does not have a framework of goals, expressed as a hierarchy with one supergoal. This makes it orders of magnitude safer, becauses one of the major problems of friendly artificial intelligence – is that it finds a wrong way to reach its supergoal.

2) EMB understoods many human details, which are obvious to anyone, but you have to explain to a AI. Moreover, a complete list of these subtleties is not known beforehand. That is, he understands what we mean. Bad example: The robot is asked to draw from the room all round objects, and tore off a man's head.

3) We can choose a human, which will be scanned, and thus know in advance its moral and ethical qualities, and attitude to the problem of FAI.

4) We can also scan a group of people known to have developed high moral and ethical liability, but representing different points of view. A good example of this is a jury. That is a collective intelligence of ordinary people. I think its is ideal to have three to five scanned people who would represent different types of people: male mathematics, a woman with children, religious philosopher, businessman, a child and therefore have maximum mapping diversity and unity of human values. I do not believe that a group of IT people in the basement will be able to properly develop a system of values for all mankind, without missing something important. I would also try before, whether these five people are capable to effective reciprocal communication before they will be scanned. That is, do they form a collective intelligence.

5) The process of creation and dissemination of the World AI will be the most secure, if he will be held under the auspices of the World State. That will stop for a while other AI projects, so there was no war of AI-projects among themselves, and which agrees to give control to our AI after its succesful creation. So our AI will not have the initial task to conquer the world. Such task would make it more aggressive and sharpened to the destruction. Any AI from the basement have to conquer the world first.

6) It is virtually the only way that I or any of us can get a "useful immortality" that is, immortality, in which he is using and developing his best qualities for the benefit of mankind, and not immortality, which one gets as a present from the God and with which he does not know what to do.