Another attempt to explain UDT

FWIW, I wrote up brief formal descriptions of UDT1 and UDT1.1 a few months back. So far, none of the DT experts have complained about their accuracy :).

For example, is it meaningful for Sleeping Beauty to ask whether it's Monday or Tuesday? Phrased like this, the question sounds stupid. Of course there's a fact of the matter as to what day of the week it is! Likewise, in all problems involving simulations, there seems to be a fact of the matter whether you're the "real you" or the simulation, which leads us to talk about probabilities and "indexical uncertainty" as to which one is you.

At the core, Wei Dai's idea is to boldly proclaim that, counterintuitively, you can act as if there were no fact of the matter whether it's Monday or Tuesday when you wake up. Until you learn which it is, you think it's both. You're all your copies at once.

... For example, Counterfactual Mugging: by assumption, your decision logically affects both heads-universe and tails-universe, which (also by assumption) have equal weight, so by agreeing to pay you win more cookies overall. Note that updating on the knowledge that you are in tails-universe (because Omega showed up) doesn't affect anything, because the theory is "updateless".

Interesting. So far I've avoided most of the posts explaining aspects of TDT/UDT/ADT because I wanted to see if I could figure out a decision theory that correctly handles newcomblike and anthropic problems, just as an intellectual challenge to myself, and that's pretty much the solution I've been working on (though only in informal terms so far).

Perhaps at this point it would be best for me to just catch up on the existing developments in decision theory and see if I'm capable of any further contributions. What open problems in decision theory remain?

So our deconstruction of Many Worlds vs Collapse is that there is a Many Worlds universe, and there are also single world Collapse universes for each sequence of ways the wave function could collapse. After pondering the difference between "worlds" and "universes", it seems that the winner is still Many Worlds.

This is a good explanation, but I am wary of "I didn't understand it completely until two days ago." You might think that was kind of silly if you look back at it after your next insight.

One thing I would like to see from a "complete understanding" is a way that a computationally bounded agent could implement an approximation of uncomputable UDT.

In counterfactual mugging problems, we, trying to use UDT, assign equal weights to heads-universe and tails-universe, because we don't see any reason to expect one to have a higher Solomonoff prior than the other. So we are using our logical uncertainty about the Solomonoff prior rather than the Solomonoff prior directly, as ideal UDT would. Understanding how handle and systematically reduce this logical uncertainty would be useful.

If you possess some useful information about the universe you're in, it's magically taken into account by the choice of "information set", because logically, your decision cannot affect the universes that contain copies of you with different states of knowledge, so they only add a constant term to the utility maximization.

I object to "magically", but this is otherwise correct.

More formally, you have an initial distribution of "weights" on possible universes (in the currently most general case it's the Solomonoff prior) that you never update at all. In each individual universe you have a utility function over what happens. When you're faced with a decision, you find all copies of you in the entire "multiverse" that are faced with the same decision ("information set"), and choose the decision that logically implies the maximum sum of resulting utilities weghted by universe-weight.

I've read both the original UDT post and this one, and I'm still not sure I understand this basic point. The only way I can make sense out of it is as follows.

The UDT agent is modeled as a procedure S, and its interaction with the universe as a program P calling that procedure and doing something depending on the return value. Some utility is assigned to each such outcome. Now, S knows the prior probability distribution over all programs that might be calling S, and there is also the input X. So when the call S(X) occurs, the procedure S will consider how the expected utility varies depending on what S(X) evaluates to. However, changing the return value for S(X) affects only those terms in the expected utility calculation that correspond to those programs that might (logically) be calling S with input X, so whatever method is used to calculate that maximum, it effectively addresses only those programs. This restriction of the whole set of programs for the given input replaces the Bayesian updating, hence the name "updateless."

Is this anywhere close to the intended idea, or am I rambling in complete misapprehension? I'd be grateful if someone clarified that for me before I make any additional comments.

Could you state something you didn't understand or disagreed with before the recent change of mind that lead to this post?

In each individual universe you have a utility function over what happens.

Can you unpack this? (As compared to ADT's way of eliciting utility functions from definition of actual utility and actual action.)

Note that updating on the knowledge that you are in tails-universe (because Omega showed up) doesn't affect anything, because the theory is "updateless".

Can you expand a little on this?

Parfitt's Hitchhiker and transparent Newcomb: So is the interest in UDT motivated by the desire for a rigorous theory that explains human moral intuitions? Like, it's not enough that feelings of reciprocity must have conveyed a selective advantage at the population level, we need to know whether/how they also are net beneficial to the individuals involved?

What should one do if in a Newcomb's paradox situation but Omega is just a regular dude who thinks they can predict what you will choose, by analysing data from thousands of experiments on e.g. Mechanical Turk?

Do UDT and CDT differ in this case? If they differ then does it depend on how inaccurate Omega's predictions are and in what direction they are biased?

Are there any practical applications of UDT that don't depend on uncertainty as to whether or not I am a simulation, nor on stipulating that one of the participants in a scenario is capable of predicting my decisions with perfect accuracy?

If you possess some useful information about the universe you're in, it's magically taken into account by the choice of "information set", because logically, your decision cannot affect the universes that contain copies of you with different states of knowledge,

For example, Counterfactual Mugging: by assumption, your decision logically affects both heads-universe and tails-universe, which (also by assumption) have equal weight, so by agreeing to pay you win more cookies overall. Note that updating on the knowledge that you are in tails-universe (because Omega showed up) doesn't affect anything, because the theory is "updateless".

(I think) I understand the first quote, and (I think) I understand the second quote, but they don't seem to make sense together. If your decision can't affect universes where you have different states of knowledge, why does the decision in the tails-universe affect the heads-universe? Is it because you might be in Omega's simulation in the heads-universe and that therefore there's an instance of you with the same knowledge in both?

Aren't the universe weights just probabilities by a different name? (Which leads directly to my formulation "UDT = choose the strategy that maximizes your unconditional expected utility".) Or are the weights supposed to be subjective - they measure the degree to which you 'care' about the various worlds, and different agents can assign different weights without either one being 'wrong'? But doesn't that contradict that idea that we're supposed to use the One True Solomonoff Prior?

Or are you not thinking of the weights as probabilities simply because UDT does away with the idea that one of the possible worlds is 'true' and all the others are 'false'?

Well again, the part about probability suggests a fundamental misunderstanding of the Bayesian interpretation. The math never has to use the word "true". If you think it does when expressed in English grammar, that seems like a flaw in English grammar.