Thought experiment: coarse-grained VR utopia

This is a very interesting idea!

Let me try an example to see if I've got it right. Humans think that it is wrong to destroy living things, but okay to destroy non-living things. But in physics the line between living and non-living is blurry. For example a developing embryo goes from non-living to living in a gradual way; hence the abortion debate. The AI is acting on our behalf and so it also wants to preserve life. But this is difficult because "life" doesn't have a clear boundary. So it fixes this problem by ensuring that every object in the simulation is either alive or non-alive. When people in the simulation become pregnant it looks and feels to them as though they have a growing baby inside them, but in fact there is no child's brain outside the simulation. At the moment they give birth the AI very quickly fabricates a child-brain and assigns it control of the simulated baby. This means that if someone decides to terminate their pregnancy then they can be assured that they are not harming a living thing (this is hypothetical because presumably the simulation is utopic enough that abortions are never necessary). After the child is born then it definitely is living and the people in the simulation know that they have to act to protect it.

Is that the right idea?

How does value scale with brain size? For example, if we had enough atoms to build 10 human-like brains, is it better to do that or to build a giant brain that can have much more complex thoughts and experiences? Are there experiences that we can't imagine today that are many times more valuable than what we can imagine, without even using more atoms? Also consider questions like population ethics and egoism vs altruism, e.g., should we use the rest of the universe to create more lives (if so how many), or to extend/improve the lives of existing people?

For something to count as a utopia for me, it would have to allow me to become smarter, answer these questions, then have the answers make a meaningful difference in the world. Otherwise, it could still be an AI disaster relative to the potential of the universe.

Three places similar ideas have occurred that spring to mind:

FIRST Suarez's pair of novels Daemon and Freedom(tm) are probably the most direct analogue, because it is a story of taking over the world via software, with an intensely practical focus.

The essential point for this discussion here and now is that prior to launching his system, the character who takes over the world first tests the quality of the goal state that he's aiming at by implementing it first as a real world MMORP. Then the takeover of the world proceeds via trigger-response software scripts running on the net, but causing events in the real world via: bribes, booby traps, contracted R&D, and video game like social engineering.

The MMORP start not only functions as his test bed for how he wants the world to work at the end... it also gives him starting cash, a suite of software tools for describing automated responses to human decisions, code to script the tactics of swarms of killer robots, and so on.

SECOND Nozick's Experience Machine thought experiment is remarkably similar to your thought experiment, and yet aimed at a totally different question.

Nozick was not wondering "can such a machine be described in detail and exist" (this was assumed) but rather "would people enter any such machine and thereby give up on some sort of atavistic connection to an unmediated substrate reality, and if not what does this mean about the axiological status of subjective experience as such?"

Personally I find the specifics of the machine to matter an enormous amount to how I feel about it... so much so that Nozick's thought experiment doesn't really work for me in its philosophically intended manner. There has been a lot of play with the concept in fiction that neighbors on the trope where the machine just gives you the experience of leaving the machine if you try to leave it. This is probably some kind of archetypal response to how disgusting it is in practice for people to be pure subjective hedonists?

THIRD Greg Egan's novel Diaspora has most of the human descended people living purely in and as software.

In the novel any common environment simulator and interface (which has hooks into the sensory processes of the software people) is referred to as a "scape" and many of the software people's political positions revolve around which kinds of scapes are better or worse for various reasons.

Konishi Polis produces a lot of mathematicians, and has a scape that supports "gestalt" (like vision) and "linear" (like speech or sound) but it does not support physical contact between avatars (their relative gestalt positions just ghost around and through each other) because physical contact seems sort of metaphysically coercive and unethical to them. By contrast Carter-Zimmerman produces the best physicists, and it has relatively high quality physics simulations built into their scape, because they think that high quality minds with powerful intuitions require that kind of low level physical experience embedded into their everyday cognitive routines. There are also flesh people (who think flesh gives them authenticity or something like that) and robots (who think "fake physics" is fake, even though having flesh bodies is too dangerous) and so on.

All of the choices matter personally to the people... but there is essentially no lock in, in the sense that people are forced to do one thing or another by an overarching controller that settles how things will work for everyone for all time.

If you want to emmigrate from Konishi to Carter Zimmerman you just change which server you're hosted on (for better latency) and either have mind surgery (to retrofit your soul with the necessary reflexes for navigating the new kind of scape) or else turn on a new layer of exoself (that makes your avatar in the new place move according to a translation scheme based on your home scape's equivalent reflexes).

If you want to, you can get a robot body instead (the physical world then becomes like a very very slow scape and you run into the question of whether to slow down your clocks and let all your friends and family race ahead mentally, or keep your clock at a normal speed and have the robot body be like a slow moving sculpture you direct to do new things over subjectively long periods of time). Some people are still implemented in flesh, but if they choose they can get scanned into software and run as a biology emulation. Becoming biologically based is the only transformation rarely performed because... uh... once you've been scanned (or been built from software from scratch) why would you do this?!

Interesting angles:

Suarez assumes physical coercion and exponential growth as the natural order, and is mostly interested in the details of these processes as implemented in real political/economic systems. He doesn't care about 200 years from now and he uses MMORP simulations as simply a testbed for practical engineering in intensely human domains.

Nozick wants to assume utopia, and often an objection is "who keeps the Experience Machine from breaking down?"

Egan's novel has cool posthuman world building, but the actual story revolves around the question of keeping the experience machine from breaking down... eventually stars explode or run down... so what should be done in the face of a seemingly inevitable point in time where there will be no good answer to the question of "how can we survive this new situation?"

Don't humans have to give up on doing their own science then (at least fundamental physics) ?

I guess I can have the FAI make me a safe "real physics box" to play with inside the system; something that emulates what it finds out about real physics.

Is it a useful way to split up the problem

I think it's a useful way to split the problem of a utopia between condition-level utopia and ontology-level. In a coarse VR you can verify that anything related to the condition of a utopia are friendly, but there are other values related to the ontology of the world: self-modification, new technology, agency in companions / mates / victims, power of divine agents, etc.
It's not always possible to rescue the utility function.

An interesting thought-experiment. But I don't follow this part:

So in theory we could hand it off to human philosophers or some other human-based procedure, thus dealing with "complexity of value" without much risk.

The complexity of value has to do with how the border delineating good outcomes from all possible outcomes cannot be specified in a compact way. Granted, the space of possible polygon arrangements is smaller than the space of possible atom arrangements. That does make the space of possible outcomes relatively more manageable in your VR world. But the space of outcomes is still Vast. It seems Vast enough that the border separating good from bad is still complex beyond our capacity to specify.

The way that I choose to evaluate my overall experience is generally through the perception of my own feelings. Therefore, I assume this simulated world will be evaluated in a similar way: I perceive the various occurrences within it and rate them according to my preferences. I assume the AI will receive this information and be able to update the simulated world accordingly. The main difference then, appears to be that the AI will not have access to my nervous system, if my avatar is being represented in this world and that is all the AI has access to, which would prevent it from wire-heading by simply manipulating my brain however it wants. Likewise it would not have access to its own internal hardware or be able to model it (since that would require knowledge of actual physics). It could in theory be able to interact with buttons and knobs in the simulated world that were connected to its hardware in the real world.

I think this is basically the correct approach and it actually is being considered by AI researchers (take Paul's recent paper for example, human yes-or-no feedback on actions in a simulated environment). The main difficulty then becomes domain transfer, when the AI is "released" into the physical world - it now has access to both its own hardware and human "hardware", and I don't see how to predict its actions once it learns these additional facts. I don't think we have much theory for what happens then, but the approach is probably very suitable for narrow AI and for training robots that will eventually take actions in the real world.

WOW I think it's just amazing! "So in theory we could hand it off to human philosophers or some other human-based procedure, thus dealing with "complexity of value" without much risk." *That could simply be handled by producing more complexity than entropy in replication pdf

"Then we could launch a powerful AI aimed at rebuilding reality to match it (more concretely, making the world's conscious experiences match a specific coarse-grained VR utopia, without any extra hidden suffering). That's still a very hard task, because it requires solving decision theory and the problem of consciousness, but it seems more manageable than solving friendliness completely." *In a coarse-grained virtual reality where everything is built from polygons and textures consciousness could be quantized trough Stone's representation theorem or Heyting algebra as symbols that represent themselves by way of symmetry.

"The resulting world would be suboptimal in many ways, e.g. it wouldn't have much room for science or self-modification." *It will have room for science as the predictor of thermodynamic cost. Really the trick by which coarse-grained VR utopia might just work is because of a process called "refrigeration by randomizing a bit" and "adiabatic demagnetization".

In a nutshell,

1. We can measure and predict the thermodynamic costs as a result of producing more complexity (a series of undeveloped modules that may or may not lead to functional specialization).
2. If the "coarseness" of the VR is obtained by randomizing a bit and using that bit as a refrigeration, well then we can blow the second law of thermodynamics.
3. Effectiveness of functional specialization in decreasing thermodynamic cost would be the golden rule that will avoid decision making or consciousness.

Screenplay: MISTER ELEGANS

We see a bank of monitors, each showing a small worm or group of worms going about their lives.

Cut to: a long black trenchcoat standing in front of the monitors. The sleeves hang empty, but from the collar protrudes the tapered, featureless head of a man-sized worm.

Cut to: a youngish man in a grey T-shirt seated in a high-backed leather swivel chair.

YOUNGISH MAN: Hello, Caeno. I am the Zuckitect. I created the Growth Matrix.

Today I want to focus on the most important question of all: are we building the world we all want?

The first growth matrix I designed was quite naturally perfect, a supportive, safe, informed, civically-engaged, inclusive community...

Hm. But if you already have the brain-in-a-vat (aka Matrix) scenario where all brains are happy, why bother rebuilding reality? Would that brain-in-a-vat know what reality is, anyway? This looks like semi-wireheading to me...

How hard is it to describe a coarse-grained VR utopia that you would agree to live in?

Not hard, but it is still hard to describe one that everyone would agree to live in.