Preferences without Existence

I've heard that point of view from several people. It's a natural extension of LW-style beliefs, but I'm not sure I buy it yet. There are several lines of attack, the most obvious one is trying to argue that coinflips still behave as coinflips even when the person betting on them is really stupid and always bets on heads. But we've already explored that line a little bit, so I'm gonna try a different one:

Are you saying that evolution has equipped our minds with a measure of caring about all possible worlds according to simplicity? If yes, can you guess which of our ancestor organisms were already equipped with that measure, and which ones weren't? Monkeys, fishes, bacteria?

(As an alternative, there could be some kind of law of nature saying that all minds must care about possible worlds according to simplicity. But I'm not sure how that could be true, given that you can build a UDT agent with any measure of caring.)

A: Wait, if simplicity changes preferences, but does not change the level of existence, how do you explain the fact that we appear to be in a world that is simple? Isn’t that a priori extremely unlikely?

B: This is where it gets a little bit fuzzy, but I do not think that question makes sense. Unlikely by what measure? You are presupposing an existence measure on the collection of theoretical worlds just to ask that question.

I saw a good explanation of this point (in the #lesswrong IRC sometime last month), which I'll try to reproduce here.

Imagine that there is a specific flavor of Universal Turing Machine which runs everything in Tegmark IV; all potential-universe objects are encoded as tapes for this UTM to simulate.

Take a simple set of rules. There is only one tape that corresponds to these precise rules. Now add an incredibly-specific edge case which applies in exactly one situation that could possibly arise, but otherwise have the same simple set of rules hold; this adds another tape that will produce precisely the same observations from the inside as the simple rules, except in one case that is grotesquely uncommon and very likely will never occur (because it happens to require a state unreachable from the initial conditions). As far as the inhabitants of these universes are concerned, they are the same.

Of course, there are an infinite variety of these edge cases, each of which occurs in a vanishingly specific not-necessarily-reachable circumstance, and nearly every combination of which results in a universe that looks precisely like the simple set of rules. And detailing each edge case should add a finite amount of tape used, added onto the tape needed for the description of the simple rules.

So if you take a more complex set of rules which takes twice as much tape to describe, and compare it with the set of all tape-length-equal instantiations of apparently-those-simple-rules, you'll find that, taking into account the many ways that you can make universes which are like-the-simple-rules-except-where-very-specifically-noted, universes where the correct generalization is the simple rules are more common than universes where the correct generalization is the complex rules, by a factor dependent exclusively on the Kolmogorov complexity.

TL;DR: We probably don't live in a simple universe, but we probably live in a universe where the correct model to form anticipations with is simple.

My personal objection to this seems to be: "But what if some things DO exist?" IE, what if existence actually does turn out to be a thing, in limited supply?

Let me try to unfold this gut reaction into something which makes more sense.

I think what I'm "really" saying is that, for me, this view does not add up to normality. If I were to discover that "existence" is meaningless as a global predicate, and (to the extent that it's meaningful) all possibilities exist equally, I would change the way I care about things.

I think I value variety, so I would be happy at this news: I would have a nice, comfortable lower bound on my utility function, contributed by all the other universes. (As far as variety goes, I can now focus my energy on optimizing the local branches containing me.)

I would think about death differently: before learning about the meaninglessness of existence, I placed inherent value on my life, and also placed value on what I might contribute to the world, and (almost as a side note) also would be sad about my family missing me if I was gone. Now it's about my family and friends, who would miss me in a large measure of branches. My unique contributions to society, however, are guaranteed to be preserved in some branches; that part of me survives any accident. (This is similar to the quantum suicide argument.)

I would also be excited about the possibility of interaction between parallel universes, since for any two turing machines, there is a third which runs them in parallel but with some form of interaction (in fact, every possible form of interaction).

All told, my value function would not be anywhere close to equivalent to a Solomonoff weighting!

Thank you all so much for all of your comments.

Three separate comment threads simultaneously lead to the refutation that I seem to be unfairly biased towards agents that happen to be born in simple universes. I think this is a good point, so I am starting a new comment thread to discuss that issue. Here is my counter point.

First, notice that our finite intuitions do not follow over nicely here. In saying that beings in universes with 20 fewer bits are a million times as important, I am not saying that the happiness of this one person is more important than the happiness of those million people over there. Instead, I am pointing at two infinite and unmeasurable clusters of universes, and saying that this cluster is a million times as important as this other cluster. Because there is no measure on these clusters, there is no fact of the matter as to whether one cluster is a million times as large as another. In finite collections, you do not have this issue, but with infinite collections, there could million to one map from one cluster to another, and a million to one map in the other direction. To judge me as unfair, you must put a measure on the collection of universes by which to judge.

So, what measure should we put on the collection of universes to judge this fairness? It may look like my measure is unfair because it is not uniform, giving much more weight to simple universes. However, I argue that it is the most fair. If your collection of universes are described in some language on an infinite tape, I am giving a uniform distribution of weight over all infinite tapes. However, this means that universes with simple finite descriptions can ignore most of the tape and show up more in the uniform distribution over infinite tapes. What looks unfair to you is actually the uniform weighting in a very slightly different, (and perhaps more natural) model -- the model that VAuroch argues for in his comments here.

As I understand it, your attempted solution to the Problem of Induction is this:

(a) Deny that there is a fact of the matter about what our future experiences will be like

(b) Care about things in inverse proportion to the Kolmogorov complexity of the structure in which they are embedded.

This is why it all adds up to normality. Without (a), people could say: Go ahead, care about whatever you want, but under your belief system you ought to expect the world to dissolve into high-complexity chaos immediately! And without (b), people could say: Go ahead, deny the existence of a future. But the vast majority of your counterparts affected by your actions inhabit complex, chaotic worlds; one implication of this is that you should live life in the moment.

Is this correct?

Interesting approach. The way I would put it, the number you want to maximize is expectation of U(X) where U is a utility function and X is a random universe taken from a Solomonoff ensemble. The way you put it, the number is the same but you don't interpret the sum over universes as expectation value, you just take it to be part of your utility function.

What I feel is missing in your approach is that U by its nature is arbitrary / complex, whereas the Solomonoff prior is simple and for some reason has to be there. I.e. something would go wrong on the philosophical level (not just the pragmatic / normality level), if you throw Solomonoff out of the window. But, at the moment I can't say what that would be - it's just a hunch.

So why haven't you tried walking into a casino and caring really hard about winning? I'm not just being a prick here, this is the most concise way I could think to frame my core objection to your thesis.

Ever since learning the Anselmian ontological argument for the existence of god, I have tended towards the Kantian idea that existence is not a predicate. A ball that is not red still bounces, a ball that does not exist... doesn't exist. I'll say right up front I am not a philosopher, but rather a physicist and an engineer. I discard ideas that I can't use to build things that work, I decide that they are not worth spending time on. I don't know what I can build using the idea that a ball that does not exist is still a ball. And if I can build something using that idea, I'm not sure the thing that I can build exists.

So once I've spent enough time discussing something with A or B or anyone else for that matter to determine that they have no particular preference for things that exist than for things that don't exist, I lose interest in discussing it with them further.

Actually, this is only ALMOST true. I'll discuss things that don't exist with someone who seems to be doing something clever with it, but I ultimately expect to be disappointed, to conclude, as has happened in the past, that I have ultimately learned nothing of value from such a discussion. Or at least that whatever I have learned does not exist.

What are the semantics of "exist" on this view? When people say things like "Paris exists but Minas Tirith doesn't" are they saying something meaningful? True? It seems like such statements do convey actual information to someone who knows little about both a French history book and a Tolkein book. Why not just exercise some charitable interpretation upon your fellow language users when it comes to "exist"? We use "existence" concepts to explain our experiences, without any glaring difficulties that I can see (so a charitable interpretation is likely possible). But maybe I've missed some glaring difficulties.

Reminds me of some discussions I had long ago.

The general principle was: Take some human idea or concept and drive it to its logical conclusion. To its extreme. The result: It either became trivial or stopped to make sense.

The reason for this is that sometimes you can't separate out components of a complex system and try to optimize them in isolation without losing important details.

You can't just optimize happyness. It leads to wireheading and that's not what you want.

You can't just optimize religious following. It leads to crussades and witch hunts.

You can't just maximize empathy. Harmless as that sounds. Humans can empathize with anything. There are monks that try avoid hurting bug (good that they didn't know of bacteria). Empathy driven to the extreme leads to empathy with other beings (here: simulated, even lied about, life forms) that don't match up with complex values.

Don't hurt your complex values.

Somewhere on the way from

• your future self

• your kin

• your livestock

• children

• innocents

• strangers

• foreigners

• enemies

• the biosphere

• higher animals

• lower animals

• plants

• bacteria

• hypothetical life forms

• imaginary life forms

you should consider that your empathy diverged from being integrated well with your other values.

(yes, its not linear but a DAG)

Nice dialogue.

It's true that probability and importance are interchangeable in an expected-utility calculation, but if you weight A twice as much as B because you care twice as much about it, that implies equal probabilities between the two. So if you use a Solomonoff-style prior based on how much you care, that implies a uniform prior on the worlds themselves. Or maybe you're saying expected utility is the sum of caring times value, with no probabilities involved. But in that case your probability is just how much you care.

If we were in a complex world, it's plausible you could make a bigger impact to your values by choosing actions that correlate with actions in the much more important simpler world rather than actions that have good consequences in this world. Computing which those are would take a lot of effort, though, so in practice, you'd be doing the same sorts of things in the short run (i.e., working toward better futures).

What it means to exist is one area of metaphysics that still confuses me, but the Tegmark Level IV picture seems to make sense. In that case, rather than measure being unimportant, measure is all that matters, because our actions help determine which possible worlds have more and less measure.

I'm not beginning to think I identified a real flaw in this approach.

The usual formulation of UDT assumes the decision algorithm to be known. In reality, the agent doesn't know its own decision algorithm. This means there is another flavor of uncertainty w.r.t. which the values of different choices have to be averaged. I call this "introspective uncertainty". However, introspective uncertainty is not independent: it is strongly correlated with indexical uncertainty. Since introspective uncertainty can't be absorbed into the utiltity function, indexical uncertainty cannot either.

I have a precise model of this kind of UDT in mind. Planning to write about it soon.

Instead of saying that you care about simpler universes more, couldn't a similar preference arise out of pure utilitarianism? Simpler universes would be more important because things that happen within them will be more likely to also happen within more complicated universes that end up creating a similar series of states. For every universe, isn't there an infinite number of more complicated universes that end up with the simpler universe existing within part of it?