Bayesian Reasoning with Unsong Theodicy means we shouldn't destroy the universe

post by pku · 2018-07-22T01:25:07.867Z · LW · GW · 1 comments

Contents

1 comment

(Contains major Unsong spoilers.)

A while ago I made the argument that if we assume God only allows the universe to exist if it contains net positive good, and we assume the goodness of a universe is normally distributed, most universes are probably kind meh (since they're clustered around the mean).

This relies, however, on assuming normal distribution of goodness. But the space of possible universes (or at least, possible universes at least somewhat like ours) seems like it might be more asymptotic: Civilization and technology advances rapidly and a lot of things change, until we either reach a positive singularity where everything is great forever, reach a negative singularity where everything is horrible forever, or destroy ourselves and/or the universe.

If we assume God doesn't allow net-negative universes to come into existence, the second case can't happen. The third case can happen, but only if the world was a net positive in the time interval before our destruction. In this case the assumption of a normal distribution applies, which means the universe is kinda meh. This has three implications:

First, our two possible scenarios are greatness or mehness. So we shouldn't try to destroy the physics, because a world in which we destroy physics is consistent with the meh scenario.

Second, we should take high-risk strategies when it comes to AI. A paperclip AI would leave the universe neutral and is consistent with a meh world. An AI that tortures everyone forever is consistent with the bad-world God wouldn't allow to exists. Therefore, if we take an AI strategy that causes either friendly AI or a paperclip AI, it's purely down to luck. But if our strategy causes a hell AI if it fails, then God Himself will ensure our success.

Third, we should try to cause as much suffering as possible before we achieve friendly AI. The more suffering we achieve now, the less likely it is that we're in a meh world, because a meh world with slightly-negative total utility would never exist. If we keep our net utility negative until we have a singularity, that singularity has to be a positive one (otherwise we wouldn't exist in the first place).

1 comments

Comments sorted by top scores.

comment by avturchin · 2018-07-22T10:19:08.872Z · LW(p) · GW(p)

If we assume that me-now is a typical observer-moment, my mood now is close to the median mood in the Universe. And I have to say that my mood now is neither good nor bad.