post by [deleted] · · ? · GW · 0 comments

This is a link post for

0 comments

Comments sorted by top scores.

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2020-05-16T00:15:14.199Z · LW(p) · GW(p)

How do you specify the "Humans have virtual reality tech with ability to create [wide range of possibilities]" exactly? What's to prevent the AI from killing all existing humans, creating from some vats some new "humans" that technically meet the definition of "human" but really just do whatever the AI finds most convenient, and giving them the virtual reality tech? And if you do manage to specify "humans" correctly, well, the AI could arrange for particular humans (the ones that are easily manipulable into doing what it wants) to get the tech.

comment by Charlie Steiner · 2020-05-16T07:11:51.370Z · LW(p) · GW(p)

Sure. The issue is that the concepts referenced in this kind of idea are already at a very high level of abstraction - deceptively so, maybe.

For example, consider the notion of "humans in control of their own worlds." This is not a concept that's easy to describe in terms of quarks, or in terms of pixels! It's really complicated and abstract! In order to be able to detect the concept "humans are in control of a virtual world instantiated in this hardware," the AI needs to have a very sophisticated model of the world that has learned a lot of human-like concepts, and can optimize for those concepts without "breaking [LW · GW]" them by finding edge cases.

Once you've solved these problems, it seems to me that you might as well go the last step and make an AI that you can safely tell "do the right thing." The problems you need to solve are very similar.

But because the problems are very similar, I'm not going to discourage you if this is easier to think about for you - if you can start with this dream and really drill down to the things we need to work on today, it's going to be important stuff.