Torture vs. Dust vs. the Presumptuous Philosopher: Anthropic Reasoning in UDT

An important point about the Presumptuous Philosopher is that so far the apparent universe has kept on getting bigger and bigger, and someone back at the dawn of time who said, "I don't know how exactly, but I predict that as we learn more about reality our model of the world will just keep getting bigger and bigger and bigger" would have been exactly and shockingly right.

At this point I apply the meta-principle of rationality, "Don't criticize people when they are right , even if you disagree with how they got there - wait to criticize their methods for an occasion when they are actually wrong." I.e., the Presumptuous Philosopher is a poor battleground on which to attack SIA, because if used as a heuristic in the past - correctly, not to stake everything on particular theories, but to say that whatever winning theory would make the apparent universe larger - it would have racked up an unbroken string of victories.

Suppose we modify the situation so that there is an infinite list of possible Theories of Everything, T1, T2, T3, ... With each theory predicting a trillion observers, a trillion trillion observers, a trillion trillion trillion observers, and so on. Additionally, from some other prior info gleaned from scientific insight, the probability of Tn being true is 2^(-n).

SIA here would suggest we live in the universe with the most observers... which in this case doesn't exist.

Reasons I hate the Presumptuous Philosopher thought experiment:

• It depends partly on question-begging. "We all know SIA is false, therefore the philosopher's claims are ridiculous, therefore SIA is false."
• The philosopher's confidence is only really warranted if he's 100% sure of SIA, which he shouldn't be, and this makes his posterior much more ridiculous than it would be if he were e.g. only 95% sure of SIA. The thought experiment sneaks in our philosophical uncertainty as an anti-SIA intuition.
• The Nobel committee won't award him the prize because he's just assumed SIA, not justified it. Suppose some other dude assumed inverse-SIA and came to the opposite conclusion - is the Nobel committee obligated to award the prize to one of these prestige-grubbing jackasses just because one of them inevitably gets the right answer? If he instead wrote an airtight thousand-page treatise proving SIA completely, and then applied that to the T1-vs-T2 question, I think the Nobel committee would be more likely to give him the prize.

It is bad to apply statistics when you don't in fact have large numbers - we have just one universe (at least until the many-world theory is better established - and anyway, the exposition didn't mention it).

I think the following problem is equivalent to the one posed: It is late at night, you're tired, and it's dark and you're driving down an unfamiliar road. Then you see two motels, one to the right of the street, one to the left, both advertising vacant rooms. You know from a visit years ago that one has 10 rooms, the other has 100, but you can't tell which is which (though you do remember that the larger one is cheaper). Anyway, you're tired, so you just choose the one on the right at random, check in, and go to sleep. As you wake up in the morning, what are your chances that you find yourself in the larger motel? Does the number of rooms come into it? (Assume both motels are 90% full.)

The paradox is that while the other hotel is not contrafactual, it might as well be - the problem will play out the same. Same with the universe - there aren't actually two universes with probabilities on which one you'll end up in.

For a version where the Bayesian update works, you'd not go to the motel directly, but go to a tourist information stall that directs vistors to either the smaller or the larger motel until both are full - in that case, expect to wake up in the larger one. In this case, we have not one world, but two, and then the reasoning holds.

But if there's only one motel, because the other burnt down (and we don't know which), we're back to 50/50.

I know that "fuzzy logic" tries to mix statistics and logic, and many AIs use it to deal with uncertain assertions, but statistics can be misapplied so easily that you seem to have a problem here.

I think attempting to link epistemology to morality is going down the wrong path. Instead, it seems that morality should be built on epistemology. Instead of comparing dollars to the disaster, we should be put everything in the same currency. So for example, each individual having the option to choose to either:

a) Get one dust speck in their eye if they are in the large world

b) Get a hundred dust specks in their eye if they are in the small world

Converting the hundred dust specks into torture just unnecessarily complicates the problem.

Ultimately, this path leads to such unenlightening answers as there being a preference among programs that the agent should execute (with no uncertainty anywhere), and so choice of action is just choice among these programs, which is correct if it's according to that preference (order) -- a "moral" decision. UDT surfaces some mistaken assumptions in the usual decision theories, mainly in situations of unusual levels of craziness, but it doesn't actually answer any of the questions. It just heals the mistakes and pushes the buck to unspecified "preference".

The "Presumptuous Philosopher" thought experiment does not seem like a convincing argument against the Self-Indication Assumption to me. Rather, the Self-Indication Assumption seems pretty self-evident.

The philosopher doesn't get a Nobel prize - but it's not because she is wrong - it is because she is stating the obvious.

In UDT, no Bayesian updating occurs, and in particular, you don't update on the fact that you exist

To be honest, I am suspicious of both UDT and Modal Realism. I find that I simply do not care what happens in mathematically possible structures other than the world I am actually in; in a sense this validates Wei's claim that

Updateless Decision Theory converts anthropic reasoning problems into ethical problems

since I do not care what happens in mathematically possible structures that are incompatible with what I have already observed about the real world around me, I may as well have updated on my own existence in this world anyway.

In the case where either theory T1 or T2 is true, I care about whichever world is actually real, so my intuition is that we should pay the $1, which causes me to believe that I implicitly reject SIA.

Re: 1 and 2, if you've got subjectively strong (but actually harmful) intuitions in one area, then the link allows contagion. Intuitions should be used to generate experiments, not structures.