Comment by Barak Pearlmutter (barak-pearlmutter) on AGI Ruin: A List of Lethalities · 2022-07-26T19:52:18.248Z · LW · GW

each bit of information that couldn't already be fully predicted can eliminate at most half the probability mass of all hypotheses under consideration

That's not actually true (not that this matters to the main argument.) It's true in expectation: on average, you can only get at most one bit per bit. But some particular bit might give you much more, like a bit coming up 1 when you were very very sure it would be 0. "Did you just win the lottery?"