Insights from the randomness/ignorance model are genuine

I have answers to all of these questions! I just haven't posted them yet. If I present an entirely new theory in one super long post, then obviously no-one reads it. In fact, it would be irrational to read it because the prior that I'm onto something is just too low to invest the time. A sequence of short posts where each post makes a point which can be understood by anyone having read up to that post – that's not optimal, but how else could you do it? This is a completely genuine question if you have an answer.

So the structure I've chosen is to first state the distinction, then lay out the model that deals with randomness only (because that already does some stuff which SIA and SSA can't), then explain how to deal with ignorance, which makes the model complete, and then present a formalized version. The questions you just listed all deal with the ignorance part, the part that's still in the pipeline.

Well, and I didn't know I was competing with UDASSA [LW · GW], because I didn't know it existed. For some reason it's sitting at 38 karma, which makes it easy to miss, and you're the first to bring it up. I'll read it before I post anything else.

Yeah, I wrote this assuming people have the context.

So there's a class of questions where standard probability theory doesn't give clear answers. This was dubbed anthropics or anthropic probability. To deal with this, two principles were worked out, SSA and SIA, which are well-defined and produce answers. But for both of them, there are problems where their answers seem absurd.

I think the best way to understand the problem of anthropics is by looking at the Doomsday argument as an example. Consider all humans who will ever live (assuming they're not infinitely many). Say that's $N$ many. For simplicity, we assume that there are only two cases, either humanity goes extinct tomorrow, in which case $N$ is about sixty billion – but let's make that ${10}^{11}$ for simplicity – or humanity flourishes and expands through the cosmos, in which case $N$ is, say, ${10}^{18}$. Let's call $S$ the hypothesis that humans go extinct, and $L$ the hypothesis that they don't (that's for "short" and "long" human history). Now we want to update on $P\left(L\right)$ given the observation that you are human number $n$ (so $n$ will be about 30 billion). Let's call that observation $O$. Also let $p$ be your prior on $L$, so $P\left(L\right)=p$.

The Doomsday argument now goes as follows. The term $P\left(O|L\right)$ is ${10}^{-18}$, because if $L$ is true then there are a total of ${10}^{18}$ people, each position is equally likely, so ${10}^{-18}$ is just the chance to get your particular one. On the other hand, $P\left(O|S\right)$ is ${10}^{-11}$, because if $S$ is true there are only ${10}^{11}$ people total. So we simply apply Bayes on the observation $O$, and then use the law of total probability in the demonimator to obtain

$P\left(L|O\right)=P\left(O|L\right)\frac{P\left(L\right)}{P\left(O\right)}={10}^{-18}\frac{p}{P\left(O|L\right)P\left(L\right)+P\left(O|\neg L\right)P\left(\neg L\right)}=\frac{{10}^{-18}p}{{10}^{-18}p+{10}^{-12}(1-p)}$

If $p=0.999$, this term equals about 0.00989. So even if you were very confident that humanity would make it, you should still assign just below 1% on that after updating. If you want to work it out yourself, this is where you should pause and think about what part of this is wrong.

So the part that's problematic is the probability for $P\left(O|L\right)$. There is a hidden assumption that you had to be one of the humans who was actually born. This was then dubbed the Self-Sampling Assumption (SSA), namely

All other things equal, an observer should reason as if they are randomly selected from the set of all actually existent observers (past, present and future) in their reference class.

So SSA endorses the Doomsday argument. The principled way to debunk this is the Self-Indexing Assumption (SIA), which says

All other things equal, an observer should reason as if they are randomly selected from the set of all possible observers.

If you apply SIA, then $P\left(O|L\right)=P\left(O|S\right)$ and hence $P\left(L|O\right)=P\left(O\right)$. Updating on $O$ no longer does anything.

So this is the problem where SSA gives a stupid anwer. The problem where SIA gives the stupid answer is the Presumptuous Philosopher problem: there are two theories of how large the universe is, according to one it's ${10}^9$ times as large as it is according to the other. If you apply the SIA rule, you get that the odds for living in the small universe is $\frac{1}{1+{10}^9}$ (if the prior was $\frac{1}{2}$ on both).

There is also Full Non-indexical Conditioning which is technically a different theory, and it argues differently, but it outputs the same as SIA in every case, so basically there are just the two. And that, as far as I know, is the state of the art. No-one has come up with a theory that can't be made to look ridiculous. Stuart Armstrong has made a bunch of LW posts about this recently-ish, but he hasn't proposed a solution, he's pointed out that existing theories are problematic. This one [LW · GW], for example.

I've genuinely spent a lot of time thinking really hard about this stuff, and my conclusion is that the "reason as if you're randomly selected from a set of observers" thing is the key problem here. I think that's the reason why this still hasn't been worked out. It's just not the right way to look at it. I think the relevant variable which everyone is missing is that there are two fundamentally different kinds of uncertainty, and if you structure your theory around that, everything works out. And I think I do have a theory where everything works out. It doesn't update on Doomsday and it doesn't say the large universe is ${10}^9$ times as likely as the small one. It doesn't give a crazy answer anywhere. And it does it all based on simple principles.

Does that answer the question? It's possible that I should have started the sequence with a post that states the problem; like I just assumed everyone would know the problem without ever thinking about whether that's actually the case.

I think the relevant difference is that, in the set of integers, each element is strictly more complex than the previous one, but in the universe, you can probably upper bound the complexity (that's what I'm assuming, anyway). So eventually stuff should repeat, and then anything that has a nonzero probability of appearing will appear arbitrarily often as you increase the size. For example, if there's an upper bound to the complexity of a planet, then you can only have that many planets until you get a repeat.

The typical answer is that this is a result of the Poincaré recurrence theorem