The Conscious Sorites Paradox

I haven't yet devised a way to express my appreciation of the orderliness of the universe, which doesn't involve counting people in orderly states as compared to disorderly states.

What do you mean by that?

Frankly, I'm not sure what it is that you're complaining about. Even in ordinary life humans have number ambiguity: if you split the connection between the halves of the brain, you get what seems to be two minds, but why should this be some great problem?

But unfortunately there's that whole thing with the squared modulus of the complex amplitude giving the apparent "probability" of "finding ourselves in a particular blob".

I hope you will at least acknowledge the existence of the point of view of Wallace/Saunders/Deutsch that the Born rule can be derived from quantum mechanics without it plus only very reasonable outside assumptions, if you won't agree with it.

Sorry for the impulsive unhelpful bit of my previous comment. Of course if you have a number ambiguity between subjectively identical minds, then you might have problems if you apply an indifference principle to determine probabilities. But please explain if you have any other problem with this.

Simon, the trouble is that bit with the "very reasonable outside assumptions".

If you think in mangled worlds terms, and then ask, "What if the mangled worlds theory were correct, but the cutoff point were not near the median amplitude density?", then we would observe probabilities different from the Born probabilities. And the "very reasonable outside assumptions" would be suddenly revealed as unreasonable.

In the case of the Wallace paper that Robin quoted in "Quantum Orthodoxy": Wallace assumes, roughly, that branching that goes on while you're not looking, couldn't possibly be rational to take into consideration, because so much of it happens, that no decision theorist could be bothered to keep track of it. Which, because quantum physics is unitary, hands him the Born probabilities on a silver platter - specifically, the decision-theoretic principle that you should care equally about bets with equivalent payoff and equal measure ("measure" = integral over squared modulus).

But if the mangled worlds cutoff point were different, and the Born probabilities were (perhaps slightly but noticeably) different, then decision agents would indeed be wise to think about the branching that went on while they weren't looking.

If you start pondering theories where the Born probabilities are physically derived and hence physically contingent; then the theories where the Born probabilities are derived as a priori rational considerations, begin to look really suspicious.

I mean, you just shouldn't be able to get that sort of thing a priori.

Also I follow in the path of Jaynes in regarding probability theory as more fundamental than decision theory.

I admit that when Wallace talks about the number of observers being unmeasurable because decoherence gives you a continuous tube of amplitude rather than distinct blobs - so that I'm not so much twins, as smeared - he manages to unnerve me even more than I was already unnerved, with regards to my feeble attempts to count observer-moments. But, ultimately, you could construct the same situation with Ebborians, so...

Eliezer, I agree that it looks like you shouldn't be able to get the Born rule a priori, but I don't think you're acknowledging that the same is true of the "equal probability for all worlds" rule. You need some way to translate a third-person physical view into subjective anticipations, and I don't see how this could use anything other than a priori reasoning at some point.

Also -- if I'm unsure whether my utility function says I should care about worlds in proportion to their Born measure or equally independent of their Born measure, and the latter possibility doesn't tell me what to do, then for practical purposes it drops out and I can pretend it's just the former.

(even if it could have been, but isn't, the case that the latter possibility did tell me what to do)

Eliezer:

"I admit that when Wallace talks about the number of observers being unmeasurable because decoherence gives you a continuous tube of amplitude rather than distinct blobs - so that I'm not so much twins, as smeared - he manages to unnerve me even more than I was already unnerved, with regards to my feeble attempts to count observer-moments. But, ultimately, you could construct the same situation with Ebborians, so..."

How would one construct a "smeared Ebborian"?

stephen: If we had a full understanding of fundamental physics then the only other a priori assumption we should need to derive the Born rule should be this: We aren't special. Our circumstances are typical. In other words: it is possible that at a fundamental physical level there is no Born rule and no reason one should expect a Born rule. But just by some fantastic coincidence, our little branch has followed the born rule all this time. In fact, we should expect it to stop following the Born rule immediately, for the same reason someone who's just won the lottery doesn't expect to win again next time. It's not physically impossible for us to be this lucky, but it's not physically impossible for an egg to unscramble itself either.

Fundamental physics + eggs don't unscramble + anthropic principle should give you the Born rule. If it doesn't then physicists aren't done yet.

Nick Bostrom argues that there can be a fractional number of experiences; for example, when the Ebborian divides, there might be a process that proceeds from one conscious experience to two through 1.1 experiences, 1.2, 1.3, and so on. This would fit with the fact that physical things are continuous, and also with the idea that there aren't separate blobs of amplitude. It might also be easier to explain the Born probability rule by allowing such a continuum in the number of experiences.

If all the "worlds" do exist, then there should be some really, really weird worlds out there. For example, one in which eggs really do unscramble themselves all the time...

Shouldn't there be some definite fact of the matter as to when one person becomes two people?

And even after you know that the falling tree creates acoustic vibrations but not auditory experience, it feels like there's a leftover question.

Did the same Eliezer really write both of these statements? I say again: if you can have all the information about an Ebborean brain split, but you're still asking 'when does one mind become two?' then you've missed the point.

A conscious mind is not a single, indivisible entity. It is a slippery thingy arising from the interplay of myriad smaller systems and mechanisms. If you want to use the word 'emergent' then knock yourself out. With that in mind, why should there be a definite fact of the matter?

Ben, note that in your quotation Eliezer mentioned "auditory experience." Whether or not there is an experience is one of the facts in question; so if you don't know how many experiences are there, you don't yet know all the facts.

Ben: A conscious mind is not a single, indivisible entity. It is a slippery thingy arising from the interplay of myriad smaller systems and mechanisms. If you want to use the word 'emergent' then knock yourself out. With that in mind, why should there be a definite fact of the matter?

Ben,

Knowing that you're wrong doesn't always help you. You have to know how you're wrong, and what the right answer is / how to dissolve the question, in order to get started on repairs.

Lots of people out there are wise enough to believe that, somehow, consciousness is going to end up being incarnated in mere computing matter - but believing this doesn't make the mystery go away, until you fill in that somehow. Otherwise you're just saying "Atoms did it!"

There's a notion of Boltzmann brains - brains that have spontaneously coalesced in vastly improbable reversals of entropy. If the universe is large enough, then under standard physics, it will contain Boltzmann brains. How do I know I'm not one?

"You don't," you reply. Then why do I expect my future experiences to be ordered? Why do i think that an orderly universe best explains my current existence, rather than a fluctuation of dust?

"Because an orderly universe is far more likely to give rise to you, than a fluctuation of dust would be." Ah, now you're using terms like likely. That requires some measure of probability. After all, if the prior probability of an ordered world were zero, the likelihood ratio wouldn't matter.

In standard physics in a Big World, the Boltzmann Eliezer, and the Earth Eliezer, both exist. One may have chaotic future experiences, or dissolve back into dust; the other has a legitimate expectation of future order.

Given that both kinds of future experiences will occur to subjectively indistinguishable Eliezers, why do I expect an orderly future more than I expect a future where I sprout wings and fly away?

Well... the fact that Boltzmann Eliezers are exponentially less common in the universe, than ordered Eliezers, might have something to do with it -

• if I were allowed to count people, or measure a weight of their existence, or assign a probability to them that was a probability of something. Somehow.

Do you see why, despite knowing the problems, I can't toss my notion of "number of observer-moments" or "weight of existence" or "measure of probability" out the window, yet? I need some way of saying that, on average, I don't expect to sprout wings and fly away, even if there's some very rare / low-weight / improbable Eliezer who experiences such things. If you toss that out, why believe in the science that you're using to conduct the whole analysis to begin with? Why not just believe that all the order is a chance illusion, once you toss out probability?

That's not an argument for counting people. That's explaining why I go on counting people, for now, even though I know that my current method is wrong, somehow.

PS: This whole problem blends directly over into "Why does (why do you believe that) anything exists in the first place?"

Eliezer,

While I am unable to comment on the quantum physics, you have raised a valid point (albeit too briefly?) by noting that a very similar problem applies to the very young and the dieing. When does a human child become conscious? Dennett would indeed argue that there is no such single moment. It appears to me that until this question is solved (and it can be without recourse to QM), a similar scenario in QM isn't going to salvage it. In other words, going from 0 to 1 seems like an easier but just as fundamental question than going from 1 to N.

The moral I draw from this very ancient tale: If you try to draw sharp lines in a continuous process and you end up looking silly, it's your own darn fault.

Removing grains of sand is a discrete process. The real key is recognizing what is implied by the use of the word 'heap' - so the collection of grains remains a heap until it no longer meets the criteria.

The Ship of Theseus is a much better statement of the problem, as there are enough different meanings of 'identity' that resolving the equivocation between them is a real challenge.

Eliezer: in a Big Universe, wouldn't there be more Boltzmann Eliezers than Ordered Eliezers?

At this point all I have to contribute in commentary to your post (the OP) is "fascinating" and "very well written".

Eliezer, what if we are in a Big World which will fall into heat death for an infinite time? Then as Psy-kosh pointed out (I think), wouldn't Boltzmann Eliezers be infinitely more common than orderly ones, once you take into consideration the whole of time? In this situation, why shouldn't you expect disorder?

I only saw Psy-Kosh's comment just now after posting...

Eliezer: OK, so you object to branching indifference.

Here is what I was going to reply until I remembered that you support mangled worlds:

"So, I guess I'll go buy a lottery ticket, and if I win, I'll conduct an experiment that branches the universe 10^100 times (eg. single electron Stern-Gerlach repeated less than 1000 times). That way I'll be virtually certain to win."

Now, I suppose with mangled worlds and a low cutoff you can't quite rule out your point of view experimentally this way. But you're still proposing a rule in which if you have a world which splits into world A and world B, they have probability 1/2 each, and then when world B splits into B1 and B2, it changes the probability of A to 1/3 - until an unobserved physical process turns the probability of A back to 1/2. Seems a little odd, no?

Ben Jones: A conscious mind is not a single, indivisible entity. It is a slippery thingy arising from the interplay of myriad smaller systems and mechanisms. If you want to use the word 'emergent' then knock yourself out. With that in mind, why should there be a definite fact of the matter?

Ben, even without knowing about atoms or brains, you can know this much: Something is happening, and it consists of one thing after another.

Whatever else reality encompasses, it must include this experience sequence or stream of consciousness that you have.

You evidently propose to identify these experiences with states of a physical brain, as do most scientifically educated people. But then, faced with the boundary problems pertaining to physical systems that Eliezer has highlighted, you say, well, there's no "definite fact of the matter" as to (say) how many streams of consciousness there are in a given physical setup.

This is not an option, except in the sense that ignoring a problem is an option. A exists, definitely; we hypothesize that A is actually B, possibly; we have trouble specifying the exact relationship between B and A; so we conclude that A doesn't definitely exist after all?

One aspect of progress in physics is the synthesis of incompatible theories in new theoretical frameworks. Quantum mechanics and relativity give rise to quantum field theory, quantum field theory and gravity give rise to string theory. The present situation is a huge opportunity for discovery, once you accept that A definitely exists; but everyone is clinging to their existing models of B at any price. Basic sensory qualities, the flow of time, the unity of the individual consciousness in appearance and reality - all are to be denied so we can imagine that our existing physics is enough.

But there is truly no need to do this. The fundamental validation of the physics we have is that it makes correct quantitative predictions. All that that implies, in turn, is that there are quantities in nature, somewhere and somehow. Do we have so little imagination that we cannot think up an ontology in which all the manifest aspects of consciousness are actually there, and in which the quantitative relations of our physics are also present?

This really will require deep changes, there is no doubt. Once you accept that these problematic qualities of consciousness are real - that they are there, even if their nature is not totally clear - you also have to address the question of how it is that you know they are there. That implies faculties of awareness and a theory of knowledge which is not just a matter of correct quantitative prediction. As with the numbers of physics, this does not mean that we need some new ontology of knowledge instead of Bayes, Kolmogorov, and the other local favorites. The new qualitative ontology would ground the quantitative aspects of epistemology, just as it must ground the quantitative ontology we call physics.

The split-brain research must be among the most important clues to the truth that we have, because it comes close to producing some of these consciousness-counting paradoxes in reality. But to be is to be something, consciousness is actually there, and so the solution is not to consign it to the realm of fuzzy half-realities. The solution is to start with the premise that there is a definite fact of the matter, and sacrifice everything to retain that premise. But I doubt that we have to sacrifice that much, really.

Simon: But you're still proposing a rule in which if you have a world which splits into world A and world B, they have probability 1/2 each, and then when world B splits into B1 and B2, it changes the probability of A to 1/3 - until an unobserved physical process turns the probability of A back to 1/2. Seems a little odd, no?

That's an excellently sharp compression of the problem with observer-moment anthropics the way I'm trying to do it - the conditional probabilities aren't invariant when you compose or decompose events. If you're anesthetized during the split between A and B, and wake up and observe yourself in B, before B splits, it seems the prior probability of B is 1/2. If you're anesthetized until after B splits into B1 and B2, it seems the prior probability of B is 2/3.

Huh. The probability theory I'm trying to use here really isn't giving sensible results at all.

Eliezer, thanks for the detailed reply, that's really cleared up a whole load of my confusion over the last few posts.

I certainly haven't thought about this enough yet. I do recognise the fact that we need to put this aside to get any sense out of the world. But I still feel that trying to pin down a mind as a discrete entity, and trying to decide on when one mind becomes two, creates more problems than it solves.

The Ebboreans can dream up plenty of brain-splitting (!) scenarios that involve bigger and bigger numbers, and make less and less sense when you try to apply subjective probabilities. I'm sure it wouldn't be hard to dream up scenarios where experience tends to infinity, which doesn't bode well. Also, say an Ebborean is being tortured while his brain splits. Does the amount of disutility in the universe suddenly double? If so, my brain asplode.

Re: Boltzmann Brains: AFAICT, P(our current universe springing into existence) is miniscule. Certainly, P(just my brain springing up) is a lot bigger. But P(stochastic, messy Big Bang, followed by a period of inflation, followed by the coalescence of stars, then planets, then the inception of life, then the evolution of the brain, then me) is enormous by comparison to either. Hence my expectation of being in the world. For every spontaneous floaty brain in infinite space, surely there would be millions of Big Bang-based sub-universes similar to our own.

I've just ordered Consciousness Explained on Amazon, will see where that goes.

A exists, definitely; we hypothesize that A is actually B, possibly; we have trouble specifying the exact relationship between B and A; so we conclude that A doesn't definitely exist after all?

This is a wonderful statement of the problem. I'll remember it.

For every spontaneous floaty brain in infinite space,

There are several problems here.

1) You're only considering a single universe. 2) It's not the formation of brains that are important. (Actual brains that form in the void die before they can get any useful computation done.) 3) The substrate of the computation is irrelevant to the content of the algorithm.

mitchell: As the Buddhists pointed out a long time ago, the flow of time is actually an illusion. All that you actually experience at any given moment is your present sensory input, plus the memories of the past. But there are any number of experiences involving loss of consciousness that will show that the flow of time as we perceive it is completely subjective (not to say that there is no time "out there," just that we don't directly perceive it).

So while I agree that "something is happening," it does not necessarily consist of one thing after another. Really it's just another formulation of cogito ergo sum.

This is also relevant in response to Caledonian - the brain does not have to live for any sustained period of time. A Boltzmann brain can pop into existence fully oxygenated with the memories that it is me, typing this response, think about it for a few seconds, and then die of whatever brains die of in interstellar space. From inside the brain, there would be no way to know the difference.

Eliezer: Isn't it sufficient to say that your brain has an expectation of order because that is how it's evolved? And what would a brain with no expectation of order even look like? Is it meaningful to talk about a control system that has no model of the outside world?

Depressurization destroys a brain long before oxygen deprivation sets in.

The point is not to consider brains, but computational devices capable of representing a mind algorithm, of which brains are only the most familiar to you.

It's the distribution of the you-defined algorithm across the various universes that needs to be considered, not merely the chance of a brain arising randomly from chaos.

Upon futher reflection, I'm not sure I believe Wallace's argument that, in quantum physics the way we know it, the number of observers is ill-defined.

Yes, decoherence is continuous. But by the time you get all the way up to objects the size of neurons "firing" or "not firing", I'm not sure there's much influence being transmitted between worlds - or that the intermediate states represent conscious entities (non-mangled worlds?). I'm not even sure I could construct a continuous spectrum of conscious Ebborian brains. Operations like Max(A, B) would be prohibited. Sure, our underlying physics is continuous - a coin can always balance on its edge - but to construct an intermediate between two coherent conscious minds, every coin has to land on its edge; in addition to having tiny and perhaps-mangled amplitude, I'm not sure that intermediate still represents a person.

Would I be the only person here occasionally terrified by the idea that they might 'wake up' as a Boltzmann brain at some point, with a brain arranged in such a way as to subject them to terrible agony, for an indeterminate period of time? I would really appreciate a response on this...

Nick: I haven't had that exact thought, but some analogous thoughts. (I don't remember the details) It does seem unlikely though. (Though I still am confused about why I'm not a Boltzmann brain)

But a Boltzmann brain that just happens to be in that particular arrangement would seem to be comparatively unlikely compared to all possible Boltzmann brains one might happen to be.

Nick and Psy-Kosh: here's a thought on Boltzmann brains.

Let's suppose the universe has vast spaces uninhabited by anything except Boltzmann brains which briefly form and then disappear, and that any given state of mind has vastly more instantiations in the Boltzmann-brain only spaces than in regular civilizations such as ours.

Does it then follow that one should believe one is a Boltzmann brain? In the short run perhaps, but in the long run you'd be more accurate if you simply committed to not believing it. After all, if you are a Boltzmann brain, that commitment will cease to be relevant soon enough as you disintegrate, but if you are not, the commitment will guide you well for a potentially long time.

Simon: Well, yeah, I notice that I seem to, well, not dissolve into nothingness.

But I have to admit that from a certain perspective, I'm surprised about that. Why are more versions of me non Boltzmann brain than Boltzmann brain... that is, why do I have a higher probability of perceiving myself in a universe that has rather more order than is strictly needed for just, well, me at this instant?

It may be that most minds with your thoughts do in fact disappear after an instant. Of course if that is the case there will be vastly more with chaotic or jumbled thoughts. But the fact that we observe order is no evidence against the existence of additional minds observing chaos, unless you don't accept self-indication.

So, your experience of order is not good evidence for your belief that more of you are non-Boltzmann than Boltzmann. But as I said, in the long term your expected accuracy will rise if you commit to not believing you are a Boltzmann brain, even if you believe that you most likely are one now.

A somewhat analogous situation may arise in AGI - AI makers can rule out certain things (e.g. the AI is simulated in a way that the simulated makers are non-conscious) that the AI cannot. Thus by having the AI rule such things out a priori, the makers can improve the AI's beliefs in ways that the AI itself, however superintelligent, rationally could not.

It may be my personal ultra-pessimistic spin on what is otherwise topic full of diverse interpretation, but I've never had a positive view of living forever, and the idea of living for what could theoretically be millions of years with any number of unpleasant stimuli being simulated in this ad-hoc cognitive mechanism is, to say the least, disturbing.

I suppose hypothetically I could tolerate the mere existence of such a strange physical phenomena so long as it wasn't me waking up in that situation, although if there's a positive spin to be given to it, I suppose I could find myself in some sort of wonderful heaven. Either way, it's more than a little unsettling to me...

Nick, do you use the normal definition of a Boltzmann brain?

It's supposed to be a mind which comes into existence by sheer random chance. Additional complexity - such as would be required for some support structure (e.g. an actual brain), or additional thinking without a support structure - comes with an exponential probability penalty. As such, a Boltzmann brain would normally be very short lived.

In principle, though, there could be so much space uninhabitable for regular civilizations that even long-lived Boltzmann brains which coincidentally have experiences similar to minds in civilizations outnumber minds in civilizations.

It's not clear whether you are worrying about whether you already are a Boltzmann brain, or if you think you are not one but think that if a Boltzmann brain took on your personality it would be 'you'. If the former, I can only suggest that nothing you do as a Boltzmann brain is likely to have much effect on what happens to you, or on anything else. If the latter, I think you should upgrade your notion of personal identity. While the notion that personality is the essence of identity is a step above the notion that physical continuity is the essence of identity, by granting the notion that there is an essence of identity at all it reifies the concept in a way it doesn't deserve, a sort of pseudosoul for people who don't think they believe in souls.

Ultimately what you choose to think of as your 'self' is up to you, but personally I find it a bit pointless to be concerned about things that have no causal connection with me whatsoever as if they were me, no matter how closely they may coincidentally happen to resemble me.

simon: the anthropic argument (ie, disapears after an instant) bit doesn't seem to be sufficiently strong to solve the problem. Unless I misunderstood what your point, it would seem to fail to address the issue that I still observe far more order than would be necessary for me to exist even for, say, several hours or days.

I observe other people, I observe working websites, I observe that my memories of a chunk of reality seem consistent with my current observations of previously mentioned chunk of reality, etc etc...

How can I explain this? Clearly, the anthropic "bolzmann brains instantly go poof" is definitavely not strong enough for this. Unless we can show that of the worlds that don't quickly kill the versions of me that they contain, the majority tend to be, for lack of a better term, "proper" worlds. But, It's not entirely obvious to me why that should be so. Need to think about it some more.

Psy-Kosh, my argument that Boltzmann brains go poof is a theoretical argument, not an anthropic one. Also, if we want to maximize our correct beliefs in the long run, we should commit to ignore the possibility that we are a brain with beliefs not causally affected by the decision to make that commitment (such as a brain that randomly pops into existence and goes poof). This also is not an anthropic argument.

With regard to longer-lived brains, if you expect there to be enough of them that even the ones with your experience are more common than minds in a real civilization with your experience, then you really should rationally expect to be one (although as a practical matter since there's nothing much a Boltzmann brain can reasonably expect to do one might as well ignore it*). If you expect there to be more long lived Boltzmann brains than civilization-based minds in general, but not enough for ones with your experience to outnumber civilization-based minds with your experience, then your experience tips the balance in favour of believing you are not a Boltzmann brain after all.

I think your confusion is the result of you not being consistent about whether you accept self-indication, or maybe you being inconsistent about whether you think of the possible space with Boltzmann brains and no civilizations as being additional to or a substitute for space with civilizations. Here's what different choices of those assumptions imply:

(I assume throughout that that the probability of Boltzmann brains per volume in any space is always lower than the probability of minds in civilizations where they are allowed by physics)*

Assumptions -> conclusion

self-indication, additional -> our experience is not evidence** for or against the existence of the additional space (or evidence for its existence if we consider the possibility that we may be unusually order-observing entities in that space)

self-indication, substitute -> our experience is evidence against the existence of the substitute space

instead of self-indication, assume the probability of being a given observer is inversely proportional to number of observers in possible universe containing that observer (this is the most popular alternative to self-indication) -> our experience is evidence against the existence of the additional or substitute space

*unless the Boltzmann brain, at further exponentially reduced probability, also obtained effective means of manipulating its environment...

** basically, define "allowed" to mean (density of minds with our experience in civ) >> (density of Boltzmann brains with our experience), and not allowed to mean the opposite (<<). One would expect the probability of a space with comparable densities to be low enough not to have a significant quantitative or qualitative affect on the conclusions.

*It seems rather unlikely that a space with our current apparent physical laws allows more long-lived B-brains than civilization-based brains. I am too tired to want to think about and write out what would follow if this is not true.

**I am using "evidence" here to mean shifts of probability relative to the outside view prior (conditional on the existence of any observers at all), which means that any experience is evidence for a larger universe (other things being equal) given self-indication, etc.

On Sorites: http://www.insofisma.com/wp2/the-paradox-of-sorites/