Anthropics and Fermi

post by Stuart_Armstrong · 2018-06-20T13:04:59.080Z · LW · GW · 20 comments

Contents

  Proper scoring rules
  Summing over different agents
  Reference classes and linked decisions
  Boltzmann brains and simulations
None
20 comments

tl;dr There is no well-defined "probability" of intelligent life in the universe. Instead, we can use proper scoring functions to penalise bad probability estimates. If we average scores across all existent observers, we get SSA-style probability estimates; if we sum them, we get SIA-style ones.

When presenting "anthropic decision theory" (the anthropic variant of UDT/FDT), I often get the response "well, that's all interesting, but when we look out to the stars with telescopes, probes, what do we really expect to see?" And it doesn't quite answer the question to say that "really expect to see is incoherent".

So instead of evading the question, let's try and answer it.

Proper scoring rules

Giving the best guess about the probability of $X$, is the same as maximising a proper scoring rule conditional on $X$. For example, someone can be asked to name a $0\le p\le 1$, and they will get a reward of $-(I_X-p{)}^2$, where $I_X$ is the indicator variable that which is $1$ if $X$ happens and $0$ if it doesn't.

Using a logarithmic proper scoring rule, Wei Dai demonstrated [LW · GW] that an updatless agent can behave like an updating one.

So let's apply the proper scoring rule to the probability that there is an alien civilization in our galaxy. As above, you guess $p$, and are given $-(1-p{)}^2$ if there is an alien civilization in our galaxy, and $-p^2$ if there isn't.

Summing over different agents

But how do we combine estimates from different agents? If you're merely talking about probability - there are several futures you could experience, and you don't know which yet - then you simply take an expectation over these.

But what about duplication, which is not the same as probability [LW · GW]? What if there are two identical versions of you in the universe, but you expect them to diverge soon, and maybe one will find aliens in their galaxy while the other will not?

One solution is to treat duplication as probability. If your two copies diverge, that's exactly the same as if there was a 50-50 split into possible futures. In this case, the total score is the average of all scores in this one universe. In that case, one should use SSA-style probability, and update one's estimates using that.

Or we could treat duplication as separate entities, and ensure that as many as possible are as correct as possible. This involves totalling up the scores in the universe, and so we use SIA-style probability.

In short:

• SSA: in every universe, the average score is as good as can be.
• SIA: for every observer, the score is as good as can be.

Thus the decision between SSA-style and SIA-style probabilities, is the decision as to which summed proper scoring function one tries to maximise.

So, which of these approaches is correct? Well, you can't say from intrinsic factors. How do you know that any probability you utter is correct? Frequentists talk about long-run empirical frequencies, while Bayesians allow themselves to chunk a lot of this empirical data into the same category (your experience of people in the construction industry is partially applicable to academia). But, all in all, both are correcting their estimates according to observations. And the two scoring totals are just two ways of correcting this estimate - neither is better than the other.

Reference classes and linked decisions

I haven't yet touched upon the reference class issue. Independently of what we choose to sum over - the scores of all human estimates, all conscious entities, all humans subjectively indistinguishable from me - by choosing our own estimates, we are affecting the estimates of those whose estimates are 'linked' with ours (in the same way that our decisions are linked with those of identical copies in the Prisoner's Dilemma). If we total up the scores, then as long as the summing includes all 'linked' scores, then it doesn't matter how many other scores are included in the total: that's just an added constant, fixed in any given universe, that we cannot change. This is the decision theory version of "SIA doesn't care about reference classes".

If we are averaging, however, then it's very important which scores we use. If we have large reference classes, then the large amount of other observers will dilute the effect of linked decisions. Thus universes will get downgraded in probability if they contain a higher proportion of non-linked estimates to linked ones. This is the decision theory version of "SSA is dependent on your choice of reference classes".

However, unlike standard SSA, decision theory has a natural reference class: just use the class of all linked estimates.

Boltzmann brains and simulations

Because the probability is defined in terms of a score in the agent's "galaxy", it makes sense to exclude Boltzmann brains from the equation, as their entire beliefs are wrong - they don't inhabit the galaxy they believe they are in, and their believed reality is entirely wrong. So from a decision theoretic perspective, so that the scoring rule makes sense, we should exclude them.

Simulations are more tricky, because they may discovered simulated aliens within their simulated galaxies. If we have a well defined notion of simulation - I'd argue that in general, that term is ill-defined - then we can choose to include or not include that in the calculation, and both estimates would makes perfect sense.

20 comments

Comments sorted by top scores.