When is a mind me?

post by Rob Bensinger (RobbBB) · 2024-04-17T05:56:38.482Z · LW · GW · 114 comments

Contents

  Why Humans Feel Like They Persist
  Sleep and Film Reels
  Weird-Futuristic-Technology Anxiety
  To Change Experience, You Have to Change Physics, Not Just Metaphysics
  ... And You Can't Change Experience With Just Any Old Change to Physics
  Having More Than One Future
None
114 comments

xlr8harder writes:

In general I don’t think an uploaded mind is you, but rather a copy. But one thought experiment makes me question this. A Ship of Theseus concept where individual neurons are replaced one at a time with a nanotechnological functional equivalent.

Are you still you?

Presumably the question xlr8harder cares about here isn't semantic question of how linguistic communities use the word "you", or predictions about how whole-brain emulation [? · GW] tech might change the way we use pronouns.

Rather, I assume xlr8harder cares about more substantive questions like:

  1. If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self?
  2. Should I anticipate experiencing what my upload experiences?
  3. If the scanning and uploading process requires destroying my biological brain, should I say yes to the procedure?

My answers:

  1. Yeah.
  2. Yep.
  3. Yep, this is no big deal. A productive day for me might involve doing some work in the morning, getting a sandwich at Subway, destructively uploading my brain, then texting some friends to see if they'd like to catch a movie after I finish answering e-mails. ¯\_(ツ)_/¯

If there's an open question here about whether a high-fidelity emulation of me is "really me", this seems like it has to be a purely verbal [? · GW] question, and not something that I would care about at reflective equilibrium.

Or, to the extent that isn't true, I think that's a red flag that there's a cognitive illusion or confusion still at work. There isn't a special extra "me" thing separate from my brain-state, and my precise causal history isn't that important to my values.

I'd guess that this illusion comes from not fully internalizing reductionism [? · GW] and naturalism [? · GW] about the mind.

I find it pretty natural to think of my "self" as though it were a homunculus that lives in my brain, and "watches" my experiences in a Cartesian theater.

On this intuitive model, it makes sense to ask, separate from the experiences and the rest of the brain, where the homunculus is. (“OK, there’s an exact copy of my brain-state there, but where am I?”)

E.g., consider a teleporter that works by destroying your body, and creating an exact atomic copy of it elsewhere.

People often worry about whether they'll "really experience" the stuff their brain undergoes post-teleport, or whether a copy will experience it instead. "Should I anticipate 'waking up' on the other side of the teleporter? Or should I anticipate Oblivion, and it will be Someone Else who has those future experiences?"

This question doesn't really make sense from a naturalistic perspective, because there isn't any causal mechanism that could be responsible for the difference between "a version of me that exists at 3pm tomorrow, whose experiences I should anticipate experiencing" and "an exact physical copy of me that exists at 3pm tomorrow, whose experiences I shouldn't anticipate experiencing".

Imagine that the teleporter is located on Earth, and it sends you to a room on a space station that looks and feels identical to the room you started in. This means that until you exit the room and discover whether you're still on Earth, there's no way for you to tell whether the teleporter worked.

But more than that, there will be nothing about your brain that tracks whether or not the teleporter sent you somewhere (versus doing nothing).

There isn't an XML tag in the brain saying "this is a new brain, not the original"!

There isn't a Soul or Homunculus that exists in addition to the brain, that could be the causal mechanism distinguishing "a brain that is me" from "a brain that is not me". There's just the brain-state, with no remainder.

All of the same functional brain-states occur whether you enter the teleporter or not, at least until you exit the room. At every moment where the brain exists, the current state of the brain isn't affected by whether teleportation occurred.

So there isn't, within physics, any way for "the real you to be having an experience" in the case where the teleporter malfunctioned, and "someone else to be having the experience" in the case where the teleporter worked. (Unless this is a purely verbal distinction, unrelated to the three important-feeling questions we started with.)

Physics is local, and doesn't remember whether the teleportation occurred in the past.

Nor is there a law of physics saying "your subjective point of view immediately blips out of existence and is replaced by Someone Else's point of view if your spacetime coordinates change a lot in a short period of time (even though they don't blip out of existence when your spacetime coordinates change a little or change over a longer period of time)".

If that sort of difference can really and substantively change whether your experiences persist over time, it would have to be through some divine mechanism outside of physics.[1]

 

Why Humans Feel Like They Persist

 

Taking a step back, we can ask: what physical mechanism makes it feel as though I'm persisting over time? In normal cases, why do I feel so confident that I'm going to experience my future self's experiences, as opposed to being replaced by a doppelganger who will experience everything in my place?

Let's call "Rob at time 1" R1, "Rob at time 2" R2, and "Rob at time 3" R3.

R1 is hungry, and has the thought "I'll go to the fridge to get a sandwich". R2 walks to the fridge and opens the door. R3 takes a bite of the sandwich.

Question 1: Why is R2 bothering to open the fridge, even though it's R3 that will get to eat the sandwich? For that matter, why is R1 bothering to strategize about finding food, when it's not R1 who will realize the benefits?

Answer: Well, there's no need in principle for my time-slices to work together like that. Indeed, there are other cases where my time-slices work at cross purposes (like when I try to follow a diet but one of my time-slices says "no"). But it was reproductively advantageous for my ancestors' brains to generate and execute plans (including very fast, unconscious five-second plans), so they evolved to do so, rather than just executing a string of reflex actions.

Question 2: OK, but you could still achieve all that by having R1 think of R1, R2, and R3 as three different people. Rather than R1 thinking "I selfishly want a sandwich, so I'll go ahead and do multiple actions in sequence so that I get a sandwich", why doesn't R1 think "I altruistically want my friend R3 to have a sandwich, so I'll collaborate with R2 to do a favor for R3"?

Answer: Either of those ways of thinking would probably work fine in principle. Indeed, there's some individual and cultural variation in how much individual humans think of themselves as transtemporal "teams" versus persisting objects.

But it does seem like humans have a pretty strong inclination to think of themselves as psychologically persisting over time. I don't know why that is, but plausibly it has a lot to do with the general way humans think of objects: we say that a table is "the same table" even if it has changed a lot through years of usage. We even say that a caterpillar is "the same organism" as the butterfly it produces. We don't usually think of objects as a rapid succession of momentary blips, so it doesn't seem surprising that we think of our minds/brains as stable objects too, and use labels like "me" and "selfish" rather than "us" and "self-altruistic".

Question 3: OK, but it's not just that I'm using the arbitrary label "me" to refer to R1, R2, and R3. R1 anticipates experiencing the sandwich himself, and would anticipate this regardless of how he used language. Why's that?

Answer: Because R1 is being replaced by R2, an extremely similar brain that will likely remember the things R1 just thought. You're in a sense constantly passing the baton to a new person, as your brain changes over time. The feeling of being replaced by a new brain state that has around that much in common with your current brain state just is the experience that you're calling "persisting over time".

That experience of "persisting over time" isn't the experience of a magical Cartesian ghost that is observing a series of brain-states and acting as a single Subject for all of them. Rather, the experience of "persisting over time" just is the experience of each brain-states possessing certain kinds of information ("memories") about the previous brain-state in a sequence. (Along with R1, R2, and R3 having tons of overlapping personality traits, goals, etc.)

Some humans are more temporally unstable than others, and if a drug or psychotic episode interfered with your short-term memory enough, or caused your personality or values to change enough minute-to-minute, you might indeed feel as though "I'm the same person over time" has become less true.

(On the other hand, if you'd been born with that level of instability, it's less likely that you'd think there was anything weird about it. Humans can get used to a lot!)

There isn't a sharp black line in physics that determines how much a brain must resemble your own in order for you to "persist over time" into becoming that brain. There's just one brain-state that exists at one spacetime coordinate, and then another brain-state that exists at another spacetime coordinate.

If a brain-state A has quasi-sensory access to the experience of another brain-state B — if A feels like it "remembers" being in state B a fraction of a second ago — then A will typically feel as though it used to be B. If A doesn't have the same personality or values as B, then A will perhaps feel like they used to be B, but have suddenly changed into a very different sort of person.

Change enough, while still giving A immediate quasi-sensory access to B's state, and perhaps the connection will start to feel more dissociative or dreamlike; but there's no sharp line in physics to tell us how much change makes someone "no longer the same person".

 

Sleep and Film Reels

I find it easier to make sense of the teleporter scenario when I consider hypotheticals like "neuroscience discovers that you die and are reborn every night while you sleep", or "physics discovers that the entire universe is destroyed and an exact copy is recreated millions of times every second".

If we discovered one of those facts, would it make sense to freak out or go into mourning?

In that scenario, should we really start fretting about whether "I'm" going to "really experience" the thing that happens to my body five seconds from now, versus Someone Else experiencing it?

I think this would be pretty danged silly. You're right now experiencing what it's like to "toss the baton" from a past version of you to a future version of you, with zero consternation or anxiety, even though right now it's an open possibility that you're not "continuous".

Maybe the real, deep metaphysical Truth is that the universe is more like a film reel made up of many discrete frames (that feel continuous to us, because we're experiencing the frames from the inside, not looking at the reel from Outside The Universe), not something actually continuous.

I earnestly believe that the proper response to that hypothetical is: Who cares? For all I know, something like that could be true. But if it's true now, it was always true; I've been living that way my whole life. If the experiences I'm having as I write this sentence are the super scary Teleporter Death thing people keep saying I should worry about, then I already know what that's like, and it's chill.

If you aren't already bored by the whole topic (as you probably should be), you can play semantics and claim that I should instead say "the experiences we've been having as we write this sentence". Because this weird obscure discovery about metaphysics is somehow supposed to mean that in the world where we made this discovery, the Real Me is secretly constantly dying and being replaced...?

But whatever. If you're just redescribing the stuff I'm already experiencing and telling me that that's the scary thing, then I think you're too easily spooked by abstract redescriptions of ordinary life. Or if you're redescribing it but not trying to tell me I should freak out about your redescription, then it's just semantics, and I'll use pronouns in whichever way is most convenient.

Another way of thinking about this is: I am my brain, not a ghost or thing outside my brain. So if something makes no physical difference to my current brain-state, and makes no difference to any of my past or future brain-states, then I think it's just crazy talk to think that this metaphysical bonus thingie-outside-my-brain is the crucial thing that determines whether I exist, or whether I'm alive or dead, etc.

Thinking that my existence depends on some metaphysical "glue" outside of my brain, is like thinking that my existence depends on whether a magenta marble is currently orbiting Neptune. Why would the existence of some random Stuff out there in the cosmos that's not a Rob-time-slice brain-state, change how I should care about a Rob-time-slice brain-state, or change which brain-state (if any) I should anticipate?

Real life is more boring than the games we can play, striving to find a redescription of the mundane that makes the mundane sound spooky. Like children staring at campfire shadows and trying to will the shadows into looking like monsters.

Real life looks like going to bed at night and thinking about whether I want toast tomorrow morning, even though I don't know how sleep works and it's totally possible that sleep might involve shutting down my stream of consciousness at some point and then starting it up again.

Regardless of how a mature neuroscience of sleep ends up looking, I expect the me tomorrow to share a truly crazily extraordinarily massive number of memories, personality traits, goals, etc. in common with me.

I expect them to remember a ton of the things I do today, such that micro-decisions (like how I write this sentence) can influence a bunch of things about their state and their own future trajectory.

I can try to distract myself from those things with neurotic philosophy-101 ghost stories, but looking away from reality doesn't make it go away.

 

Weird-Futuristic-Technology Anxiety

Since there isn't a Soul that lives Outside The Film Reel and is being torn asunder from my brain-state by the succession of frames — there's just a bunch of brain-states — the anxiety about whether "I" should "really" anticipate any future experiences in Film Reel World is based in illusion.

But the only difference between this scenario and the teleporter one is that the teleporter scenario invokes a weird-sounding New Technology, whereas the sleep and Film Reel examples bake in "there's nothing new and weird happening, you've already been living your whole life this way". If you'd grown up using using teleporters all the time, then it would seem just as unremarkable as stepping through a doorway.

If a philosopher then came to you one day and said "but WHAT IF something KILLS YOU every time you step through a door and then a NEW YOU comes into existence on the other side!", you would just roll your eyes. If it makes no perceptible difference, then wtf are we even talking about?

And the same logic applies to mind uploading. There isn't some magical Extra Thing beyond the brain state, that could make it the case that one thing is You and another thing is Not You.

Sure, you're now made of silicon atoms rather than carbon atoms. But this is like discovering that Film Reel World alternates between one kind of metaphysical Stuff and another kind of Stuff every other second.

If you aren't worried about learning that the universe secretly metaphysically is in a state of Constant Oscillation between two types of (functionally indistinguishable) micro-particles, then why care about functionally irrelevant substrate changes at all?

(It's another matter entirely if you think carbon vs. silicon actually does make an inescapable functional, causal difference for which high-level thoughts and experiences your mind instantiates, and if you think that there's no way in principle to use a computer to emulate the causal behavior of a human mind. I think that's crazy talk, but it's crazy because of ordinary facts about physics / neuroscience / psych / CS, not because of any weird philosophical considerations.)

 

To Change Experience, You Have to Change Physics, Not Just Metaphysics

Scenario 1:

I step through a doorway.

At time 1, a brain is about to enter a doorway.

At time 2, an extremely similar brain is passing through the doorway.

At time 3, another extremely similar brain has finished passing through the doorway.

Scenario 2:

I step into a teleporter.

Here, again, there exist a series of extremely similar brain states before, during, and after I use the teleporter.

The particular brain states look no different in the teleporter case than if I'd stepped through a door; so if there's something that makes the post-teleporter Rob "not me" while also making the post-doorway Rob "me", then it must lie outside the brain states, a Cartesian Ghost.

Given all that, there's something genuinely weird about the fact that teleporters spook people more than walking through a door does.

It's like looking at a film strip, and being scared that if a blank slide were added in between every frame, this would somehow make a difference for the people living inside the movie. It's getting confused about the distinction between the physics of the movie's events and the meta-physics of "what the world runs on".

The same confusion can arise if we imagine flipping the order of all the frames in the film strip; or flipping the order of all the frames in the second half of the movie; or swapping the order of every pair of frames, like so:

 

 

From outside the movie, this can make the movie's events look more confusing or chaotic to us, the viewers. But if you imagine that the characters inside the movie would be the least bit bothered or confused by this rearrangement, you're making a clear mistake. To confuse the characters, you need to change what happens inside the frames, not just change the relationship between those frames.

I claim that a very similar cognitive hiccup is occurring when someone worries about their internal stream of consciousness halting due to a teleporter (and not halting due to stepping through a random doorway).

You're imagining that something about the context of the film cells — i.e., the stuff outside of the brain states themselves — is able to change your experiences.

But experiences just are brain things. To imagine that some of the unconscious goings-on in between two of your experiences can interfere with your Self is just the same kind of error as imagining that a movie character will be bothered, or will even subjectively notice, if you inject some empty frames into the movie while changing nothing else about the movie.

 

... And You Can't Change Experience With Just Any Old Change to Physics

Claim:

As soon as a purple hat comes into existence on Pluto, my stream of consciousness will end and I will be imperceptibly replaced by an exact copy of myself that is experiencing a different stream of consciousness.

This exact copy of me will be physically identical to me in every respect, and will have all of my memories, personality traits, etc. But they won't be me. The hat, if such a hat ever comes into being, will kill me.

What, specifically, is wrong with this claim?

Well, one thing that's wrong with the claim is that Pluto is very far away from the Earth.

But the idea of a hat ending my existence seems very strange even if the hat is in closer proximity to me. Even putting a hat on my head seems like it shouldn't be enough to end my stream of consciousness, unless there's something special about the hat that will actually drastically change my brain-state. (E.g., maybe the hat is wired up with explosives.)

The point of this example being:

You can call the Ghost a "Soul", and make it obvious that we're invoking magic.

Or you can call it a "special kind of causal relationship (that's able to preserve selfhood)", and make it sound superficially scientific. (Or at least science-compatible.)

You can hypothesize that there's something special about the causal process that produces new brain-states in the "walk through a doorway" case — something "in the causality itself" that makes the post-doorway self me and the post-teleporter self not me.

But of course, this "causal relationship" is not a part of the brain state. Reify causality all you want; the issue remains that you're positing something outside the brain, outside you and your experiences, that is able to change which experiences you should anticipate without changing any of the experiences or brain-states themselves.

The brain states exist too, whatever causal relationships they exhibit. To say that exactly the same brain states can exist, and yet something outside of those states is changing a perceptible feature of those experiences ("which experience comes next in this subjective flow that's being experienced; what I should expect to see next"), without changing any of the actual brain states, is just as silly whether that something is a "causal relationship" or a purple hat.

This principle is easier to motivate in the case of the hat, because hats are a lot more concrete, familiar, and easy to think about than some fancy philosophical abstraction like "causal relationship". But the principle generalizes; random objects and processes out there, whether fancy-sounding or perfectly mundane, can't perceptibly change my experience (unless they change which brain states occur).

Likewise, it's easier to see that something on Pluto can't suddenly end my stream of consciousness, than to see that something physically (or metaphysically?) "nearby" can't suddenly end my stream of consciousness (without leaving a mess). But the principle generalizes; being nearby or connected to something doesn't open the door to arbitrary magical changes, absent some mechanism for how that exact change is caused by that exact physical process.

If we were just talking about word definitions and nothing else, then sure, define "self" however you want. You have the universe's permission to define yourself into dying as often or as rarely as you'd like, if word definitions alone are what concerns you.

But this post hasn't been talking about word definitions. It's been talking about substantive predictive questions like "What's the very next thing I'm going to see? The other side of the teleporter? Or nothing at all?"

There should be an actual answer to this, at least to the same degree there's an answer to "When I step through this doorway, will I have another experience? And if so, what will that experience be?"

And once we have an answer, this should change how excited we are about things like mind uploading. If my stream of consciousness is going to end with my biological death no matter what I do, then mind uploading sounds a lot less exciting!

Or, equivalently: If my experiences were a matter of "displaying images for a Cartesian Homunculus", and the death of certain cells in the brain severs the connection between my brain and the Homunculus, then there's no obvious reason I should expect this exact same Homunculus to establish a connection to an uploaded copy of my brain.

It's only if I'm in my brain, just an ordinary part of physics [LW · GW], that mind uploading makes sense as a way to extend my lifespan.

Causal relationships and processes obviously matter for what experiences occur. But they matter because they change the brain-states themselves. They don't cause additional changes to experience beyond the changes exhibited in the brain.

 

Having More Than One Future

I've tried to keep this post pretty simple and focused. E.g., I haven't gone into questions like "What happens if you make two uploads of me? Which one should I anticipate having the experiences of?"

But I hope the arguments I've laid out above make it clear what the right answer has to be: You should anticipate having both experiences.

If you've already bitten the bullet on things like the teleporter example, then I don't think this should actually be particularly counter-intuitive. If one copy of my brain exists at time 1 (Rob-x), and two almost-identical copies of my brain (Rob-y and Rob-z) exist at time 2, then there's going to be a version of me that's Rob-y, and a version of me that's Rob-z, and each will have equal claim to being "the next thing I experience".

In a world without magical Cartesian Homunculi, this has to be how things work; there isn't any physical difference between Rob-y and Rob-z that makes one of them my True Heir and the other a False Pretender. They're both just future versions of me.

"You should anticipate having both experiences" sounds sort of paradoxical or magical, but I think this stems from a verbal confusion. "Anticipate having both experiences" is ambiguous between two scenarios:

Scenario 1 is crazy talk, and it's not the scenario I'm talking about. When I say "You should anticipate having both experiences", I mean it in the sense of Scenario 2.

Scenario 2 is pretty unfamiliar to us, because we don't currently live in a world where we can readily copy-paste our own brains. And accordingly, it's a bit awkward to talk about Scenario 2; the English language is adapted to a world where "humans don't fork" has always been a safe assumption.

But there isn't a mystery about what happens. If you think there's something mysterious or unknown about what happens when you make two copies of yourself, then I pose the question to you:

What concrete fact about the physical world do you think you're missing? What are you ignorant of?

Alternatively, if you're not ignorant of anything, then: how can there be a mystery here? (Versus just "a weird way the world can sometimes end up".)

 

 

  1. ^

    And insofar as it's your physical brain thinking these thoughts right now, unaltered by any divine revelation, it would have to be a coincidence that this "I would blip out of existence in case A but not case B" hunch is correct. Because the reason your brain has that intuition is a product of the brain's physical, causal history, and is not the result of you making any observation that's Bayesian evidence for this mechanism existing.

    Your brain is not causally entangled with any mechanism like that; you'd be thinking the same thoughts whether the mechanism existed or not. So while it's possible that you're having this hunch for reasons unrelated to the hunch being correct, and yet the hunch be correct anyway, you shouldn't on reflection believe your own hunch. Any Bayesian evidence for this hypothesis would need to come from some source other than the hunch/intuition.

114 comments

Comments sorted by top scores.

comment by cousin_it · 2024-04-18T13:18:31.217Z · LW(p) · GW(p)

I think there's a pretty strong argument to be more wary about uploading. It's been stated a few times on LW, originally by Wei Dai if I remember right, but maybe worth restating here.

Imagine the uploading goes according to plan, the map of your neurons and connections has been copied into a computer, and simulating it leads to a person who talks, walks in a simulated world, and answers questions about their consciousness. But imagine also that the upload is being run on a computer that can apply optimizations on the fly. For example, it could watch the input-output behavior of some NN fragment, learn a smaller and faster NN fragment with the same input-output behavior, and substitute it for the original. Or it could skip executing branches that don't make a difference to behavior at a given time.

Where do we draw the line which optimizations to allow? It seems we cannot allow all behavior-preserving optimizations, because that might lead to a kind of LLM that dutifully says "I'm conscious" without actually being so. (The p-zombie argument doesn't apply here, because there is indeed a causal chain from human consciousness to an LLM saying "I'm conscious" - which goes through the LLM's training data.) But we must allow some optimizations, because today's computers already apply many optimizations, and compilers even more so. For example, skipping unused branches is pretty standard. The company doing your uploading might not even tell you about the optimizations they use, given that the result will behave just like you anyway, and the 10x speedup is profitable. The result could be a kind of apocalypse by optimization, with nobody noticing. A bit unsettling, no?

The key point of this argument isn't just that some optimizations are dangerous, but that we have no principled way of telling which ones are. We thought we had philosophical clarity with "just upload all my neurons and connections and then run them on a computer", but that doesn't seem enough to answer questions like this. I think it needs new ideas.

Replies from: RobbBB, Emrik North, green_leaf, RussellThor, Josephm
comment by Rob Bensinger (RobbBB) · 2024-04-18T16:10:59.017Z · LW(p) · GW(p)

Yeah, at some point we'll need a proper theory of consciousness regardless, since many humans will want to radically self-improve and it's important to know which cognitive enhancements preserve consciousness.

Replies from: cousin_it
comment by cousin_it · 2024-04-18T21:14:55.641Z · LW(p) · GW(p)

Yeah. My point was, we can't even be sure which behavior-preserving optimizations (of the kind done by optimizing compilers, say) will preserve consciousness. It's worrying because these optimizations can happen innocuously, e.g. when your upload gets migrated to a newer CPU with fancier heuristics. And yeah, when self-modification comes into the picture, it gets even worse.

comment by Emrik (Emrik North) · 2024-07-08T20:45:26.730Z · LW(p) · GW(p)

[Epistemic status: napkin]

My current-favourite frame on "qualia" is that it refers to the class of objects we can think about (eg, they're part of what generates what I say rn) for which behaviour is invariant across structure-preserving transformations.

(There's probably some cool way to say that with category theory or transformations, and it may or may not give clarity, but idk.)

Eg, my "yellow" could map to blue, and "blue" to yellow, and we could still talk together without noticing anything amiss even if your "yellow" mapped to yellow for you.

Both blue and yellow are representational objects, the things we use to represent/refer to other things with, like memory-addresses in a machine.  For externally observable behaviour, it just matters what they dereference to, regardless of where in memory you put them.  If you swap two representational objects, while ensuring you don't change anything about how your neurons link up to causal nodes outside the system, your behaviour stays the same.

Note that this isn't the case for most objects.  I can't swap handtomato, without obvious glitches like me saying "what a tasty-looking tomato!" and trying to eat my hand.  Hands and tomatoes do not commute.

It's what allows us to (try to) talk about "tomato" as opposed to just tomato, and explains why we get so confused when we try to ground out (in terms of agreed-upon observables) what we're talking about when we talk about "tomato".

But how/why do we have representations for our representational objects in the first place?  It's like declaring a var (address₁↦value), and then declaring a var for that var (address₂↦address₁) while being confused about why the second dereferences to something 'arbitrary'.

Maybe it starts when somebody asks you "what do you mean by 'X'?", and now you have to map the internal generators of [you saying "X"] in order to satisfy their question.  Or not.  Probably not.  Napkin out.

comment by green_leaf · 2024-07-08T13:30:45.316Z · LW(p) · GW(p)

It seems we cannot allow all behavior-preserving optimizations

We can use the same thought experiments that Chalmers uses to establish a fine-grain-functionally-isomorphic copy had the same qualia, modify them and show that anything that acts like us has our qualia.

The LLM character (rather than the LLM itself) will be conscious to the extent to which its behavior is I/O identical to the person.

Edit: Oh, sorry, this is an old comment. I got this recommended... somehow...

Edit2: Oh, it was curated yesterday.

Replies from: cousin_it
comment by cousin_it · 2024-07-09T08:23:30.344Z · LW(p) · GW(p)

anything that acts like us has our qualia

Well, a thing that acts like us in one particular situation (say, a thing that types "I'm conscious" in chat) clearly doesn't always have our qualia. Maybe you could say that a thing that acts like us in all possible situations must have our qualia? This is philosophically interesting! It makes a factual question (does the thing have qualia right now?) logically depend on a huge bundle of counterfactuals, most of which might never be realized. What if, during uploading, we insert a bug that changes our behavior in one of these counterfactuals - but then the upload never actually runs into that situation in the course of its life - does the upload still have the same qualia as the original person, in situations that do get realized? What if we insert quite many such bugs?

Moreover, what if we change the situations themselves? We can put the upload in circumstances that lead to more generic and less informative behavior: for example, give the upload a life where they're never asked to remember a particular childhood experience. Or just a short life, where they're never asked about anything much. Let's say the machine doing the uploading is aware of that, and allowed to optimize out parts that the person won't get to use. If there's a thought that you sometimes think, but it doesn't influence your I/O behavior, it can get optimized away; or if it has only a small influence on your behavior, a few bits' worth let's say, then it can be replaced with another thought that would cause the same few-bits effect. There's a whole spectrum of questionable things that people tend to ignore when they say "copy the neurons", "copy the I/O behavior" and stuff like that.

comment by RussellThor · 2024-04-19T21:31:36.964Z · LW(p) · GW(p)

Such optimizations are a reason I believe we are not in a simulation. Optimizations are essential for a large sim. I expect them not to be consciousness preserving

comment by Joseph Miller (Josephm) · 2024-07-08T07:45:21.797Z · LW(p) · GW(p)

It seems we cannot allow all behavior-preserving optimizations, because that might lead to a kind of LLM that dutifully says "I'm conscious" without actually being so.

Surely 'you' are the algorithm, not the implementation. If I get refactored into a giant lookup table, I don't think that makes the algorithm any less 'me'.

comment by andeslodes · 2024-04-18T02:09:49.212Z · LW(p) · GW(p)

I find myself strongly disagreeing with what is being said in your post. Let me preface by saying that I'm mostly agnostic with respect to the possible "explanations" of consciousness etc,  but I think I fall squarely within camp 2. I say mostly because I  lean moderately towards physicalism.

First, an attempt to describe my model of your ontology:

You implicitly assume that consciousness / subjective experience can be reduced to a physical description of the brain, which presumably you model as a classical (as opposed to quantum) biological electronic circuit.  Physically, to specify some "brain-state" (which I assume is essentially the equivalent of a "software snapshot" in a classical computer) you just need to specify a circuit connectivity for the brain, along with the currents and voltages between the various parts of the circuit (between the neurons let's say). This would track with your mentions of reductionism and physicalism and the general "vibe" of your arguments.  In this case I assume you treat conscious experience roughly as "what it feels like" to be software that is self-referential on top of taking in external stimuli from sensors. This software is instantiated on a biological classical computer instead of a silicon-based one.

With this in mind, we can revisit the teleporter scenario. Actually, let's consider a copier instead of a teleporter, in the sense that you dont destroy the original after finishing the procedure. Then, once a copy is made, you have two physical brains that have the same connectivity, the same currents and the same voltages between all appropriate positions. Therefore, based on the above ontology, the brains are physically the same in all the ways that matter and thus the software / the experience is also the same. (Since software is just an abstract "grouping" which we use to  refer to the  current physical state of the hardware)

Assuming this captures your view, let me move on to my disagreements:

My first issue with your post is that this initial ontological assumption is neither mentioned explicitly nor motivated. Nothing in your post can be used as proof of this initial assumption. On the contrary,  the teleporter argument, for example,  becomes simply a tautology if you start from your premise - it cannot be used to convince someone that doesn't already subscribe to your views on the topic. Even worse, it seems to me that your initial assumption forces you to contort (potential) empirical observation to your ontology, instead of doing the opposite.

To illustrate, let's assume we have the copier - say it's a room you walk into, you get scanned and then a copy is reconstructed in some other room far away.  Since you make no mention of quantum, I guess this can be a classical copy, in the sense that it can copy essentially all of the high-level structure, but it cannot literally copy the positions of specific electrons, as this is physically impossible anyways. Nevertheless, this copier can be considered "powerful" enough to copy the connectivity of the brain and the associated currents and voltages. Now, what would be the experience of getting copied, seen from a first-person, "internal", perspective? I am pretty sure it would be something like: you walk into the room, you sit there, you  hear say the scanner working for some time, it stops, you walk out. From my agnostic perspective, if I were the one to be scanned it seems like nothing special would have happened to me in this procedure. I didnt feel anything weird, I didnt feel my "consciousness split into two" or something. Namely, if I consider this procedure as an empirical experiment, from my first person perspective I dont get any new / unexpected observation compared to say just sitting in an ordinary room. Even if I were to go and find my copy, my experience would again be like meeting a different person which just happens to look like me and which claims to have similar memories  up to the point when I entered the copying room. There would be no way to verify or to view things from their first person perspective.

At this point, we can declare by fiat that me and my copy are the same person / have the same consciousness because our brains, seen as classical computers, have the same structure, but this experiment will not have provided any more evidence to me that this should be true. On the contrary, I would be wary to, say, kill myself or to be destroyed after the copying procedure, since no change will have occured to my first person perspective, and it would thus seem less likely that my "experience" would somehow survive because of my copy.


Now you can insist that philosophically it is preferable to assume that brains are classical computers etc, in order to retain physicalism which is preferable to souls and cartesian dualism and other such things. Personally, I prefer to remain undecided, especially since making the assumption brain= classical hardware, consciousness=experience as software leads to weird results. It would force me to conclude that the copy is me even though I cannot access their first person perspective (which defeats the purpose) and it would also force me to accept that even a copy where the "circuit" is made of water pipes and pumps, or gears and levers also have an actual, first person experience as "me", as long as the appropriate computations are being carried out.  

One curious case where physicalism could be saved and all these weird conclusions could be avoided would be if somehow there is some part of the brain which does something quantum, and this quantum part is the essential ingredient for having a first person experience. The essence would be that, because of the no-cloning theorem, a quantum-based consciousness would be physically impossible to copy, even in theory. This would get around all the problems which come with the copyability implicit in classical structures. The brain would then be a hybrid of classical and quantum parts, with the classical parts doing most of the work (since neural networks which can already replicate a large part of  human abilities are classical) with some quantum computation mixed in, presumably offering some yet unspecified fitness advantage. Still, the consensus is that it is improbable that quantum computation is taking place in the brain, since quantum states are extremely "fragile" and would decohere extremely rapidly in the environment of the brain...

Replies from: RobbBB, FireStormOOO
comment by Rob Bensinger (RobbBB) · 2024-04-18T02:42:54.501Z · LW(p) · GW(p)

My first issue with your post is that this initial ontological assumption is neither mentioned explicitly nor motivated. Nothing in your post can be used as proof of this initial assumption.

There are always going to be many different ways someone could object to a view. If you were a Christian, you'd perhaps be objecting that the existence of incorporeal God-given Souls is the real crux of the matter, and if I were intellectually honest I'd be devoting the first half of the post to arguing against the Christian Soul.

Rather than trying to anticipate these objections, I'd rather just hear them stated out loud by their proponents and then hash them out in the comments. This also makes the post less boring for the sorts of people who are most likely to be on LW: physicalists and their ilk.

Now, what would be the experience of getting copied, seen from a first-person, "internal", perspective? I am pretty sure it would be something like: you walk into the room, you sit there, you  hear say the scanner working for some time, it stops, you walk out. From my agnostic perspective, if I were the one to be scanned it seems like nothing special would have happened to me in this procedure. I didnt feel anything weird, I didnt feel my "consciousness split into two" or something.

Why do you assume that you wouldn't experience the copy's version of events?

The un-copied version of you experiences walking into the room, sitting there, hearing the scanner working, and hearing it stop; then that version of you experiences walking out. It seems like nothing special happened in this procedure; this version of you doesn't feel anything weird, and doesn't feel like their "consciousness split into two" or anything.

The copied version of you experiences walking into the room, sitting here, hearing the scanner working, and then an instantaneous experience of (let's say) feeling like you've been teleported into another room -- you're now inside the simulation. Assuming the simulation feels like a normal room, it could well seem like nothing special happened in this procedure -- it may feel like blinking and seeing the room suddenly change during the blink, while you yourself remain unchanged. This version of you doesn't necessarily feel anything weird either, and they don't feel like their "consciousness split into two" or anything.

It's a bit weird that there are two futures, here, but only one past -- that the first part of the story is the same for both versions of you. But so it goes; that just comes with the territory of copying people.

If you disagree with anything I've said above, what do you disagree with? And, again, what do you mean by saying you're "pretty sure" that you would experience the future of the non-copied version?

Namely, if I consider this procedure as an empirical experiment, from my first person perspective I dont get any new / unexpected observation compared to say just sitting in an ordinary room. Even if I were to go and find my copy, my experience would again be like meeting a different person which just happens to look like me and which claims to have similar memories  up to the point when I entered the copying room. There would be no way to verify or to view things from their first person perspective.

Sure. But is any of this Bayesian evidence against the view I've outlined above? What would it feel like, if the copy were another version of yourself? Would you expect that you could telepathically communicate with your copy and see things from both perspectives at once, if your copies were equally "you"? If so, why?

On the contrary, I would be wary to, say, kill myself or to be destroyed after the copying procedure, since no change will have occured to my first person perspective, and it would thus seem less likely that my "experience" would somehow survive because of my copy.

Shall we make a million copies and then take a vote? :)

I agree that "I made a non-destructive software copy of myself and then experienced the future of my physical self rather than the future of my digital copy" is nonzero Bayesian evidence that physical brains have a Cartesian Soul that is responsible for the brain's phenomenal consciousness; the Cartesian Soul hypothesis does predict that data. But the prior probability of Cartesian Souls is low enough that I don't think it should matter.

You need some prior reason to believe in this Soul in the first place; the same as if you flipped a coin, it came up heads, and you said "aha, this is perfectly predicted by the existence of an invisible leprechaun who wanted that coin to come up heads!". Losing a coinflip isn't a surprising enough outcome to overcome the prior against invisible leprechauns.

and it would also force me to accept that even a copy where the "circuit" is made of water pipes and pumps, or gears and levers also have an actual, first person experience as "me", as long as the appropriate computations are being carried out.  

Why wouldn't it? What do you have against water pipes?

Replies from: andeslodes
comment by andeslodes · 2024-04-18T16:43:48.495Z · LW(p) · GW(p)

First off, would you agree with my model of your beliefs? Would you consider it an accurate description?

Also, let me make clear that I don't believe in cartesian souls. I, like you, lean towards physicalism, I just don't commit to the explanation of consciousness  based on the idea of the brain as a **classical** electronic circuit. I don't fully dismiss it either, but I think it is worse on philosophical grounds than assuming that there is some (potentially minor) quantum effect going on inside the brain that is an integral part of the explanation for our conscious experience. However, even this doesn't feel fully satisfying to me and this is why I say that I am agnostic. When responding to my points, you can assume that I am a physicalist, in the sense that I believe consciousness can probably be described using physical laws, with the added belief that these laws **may** not be fully understandable by humans.  I mean this in the same way that a cat for example would not be able to understand the mechanism giving rise to consciousness, even if that mechanism turned out to be based on the laws of classical physics (for example if you can just explain consciousness as software running on classical hardware).


To expand upon my model of your beliefs, it seems to me that what you do is that you first reject cartesian souls and other such things on philosophical grounds and you thus favour physicalism. I agree on this. However I dont see why you are immediately assuming that physicalism means that your consciousness must be a result of classical computation. It could be the result of quantum computation. It could be something even subtler in some deeper theory of physics.  At this point you may say that a quantum explanation may be more "unlikely" than a classical one, but I think that we both can agree that the "absurdity distance" between the two is much smaller than say a classical explanation and a soul-based one, and thus we now have to weigh the two much options much more carefully since we cannot dismiss one in favour of the other as easily. What I would like to argue is that a quantum-based consciousness is philosophically "nicer" than a classical one. Such an explanation does not violate physicalism, while at the same time rendering a lot of points of your post invalid.


Let's start by examining the copier argument again but now with the assumption that conscious experience is the result of quantum effects in the brain and see where it takes us. In this case, to fully copy a consciousness from one place to another you would have to copy an unknown quantum state. This is physically impossible even in theory, based on the no-cloning theorem. Thus the "best" copier that you can have  is the copier from my previous comment, which just copies the classical connectivity of the brain and all the current and voltages etc, but which now fails to copy the part that is integral to **your** first person experience. So what would be your first person experience if you were to enter the room? You would just go in, hear the scanner work, get out. You can do this again and again and again and always find yourself experiencing getting out of the same initial room. At the same time the copier does create copies of you, but they are new "entities" that share the same appearance as you and which would approximate to some (probably high) degree your external behaviour. These copies may or may not have their own first person experience (and we can debate this further) but this does not matter for our argument. Even if they have a first person experience, it would be essentially the same as the copier just creating entirely new people while leaving your first person experience unchanged. In this way, you can step into the room with zero expectation that you may walk out of a room on the other side of the copier, in the same way that you dont expect to suddenly find yourself in some random stranger's body while going about your daily routine. Even better, this belief is nicely consistent with physicalism, while still not violating our intuitions that we have private and uncopiable subjective experiences. It also doesn't force us to believe that a bunch of water pipes or gears functioning as a classical computer can ever have our own first person experience. Going even further, unknown quantum states may not be copyable but they are transferable (see quantum teleportation etc), meaning that while you cannot make a copier you can make a transporter, but you always have to be at only one place at each instant.  


Let me emphasize again that I am not arguing **for** quantum consciousness as a solution. I am using it as an example that a "philosophically nicer" physicalist option exists compared to what I assume you are arguing for. From this perspective, I don't see why you are so certain about the things you write in your post. In particular, you make a lot of arguments based on the properties of "physics", which in reality are properties of classical physics together with your assumption that consciousness must be classical. When I said that I find issue with the fact that you start from an unstated assumption, I didnt expect you to argue against cartesian dualism. I expected you to start from physicalism and then motivate why you chose to only consider classical physics. Otherwise, the argumentation in your post seems lacking, even if I start from the physicalist position. To give one example of this:

You say that "there isn't an XML tag in the brain saying `this is a new brain, not the original`" . By this I assume you mean that the physical state of the brain is fungible, it is copyable, there is nothing to serve as a label. But this is not a feature of physics in general. An unknown quantum state cannot be copied, it is not fungible. My model of what you mean: "(I assume that) first person experience can be fully attributed to some structure of the brain as a classical computer. It can be fully described by specifying the connectivity of the neurons and the magnitudes of the currents and voltages between each point. Since (I assume) consciousness physically manifests as a classical pattern and since classical patterns can be copied, then by definition there can be many copies of "the same" consciousness". Thus, what you write about XML tags is not an argument for your position - it is not imposed to you by physics to consider a fungible substrate for consciousness -  it is just a manifestation of your assumption. It's cyclical. A lot of your arguments which invoke "physics" are like that. 

Replies from: RobbBB
comment by Rob Bensinger (RobbBB) · 2024-04-18T21:46:58.831Z · LW(p) · GW(p)

Why would the laws of physics conspire to vindicate a random human intuition that arose for unrelated reasons?

We do agree that the intuition arose for unrelated reasons, right? There's nothing in our evolutionary history, and no empirical observation, that causally connects the mechanism you're positing and the widespread human hunch "you can't copy me".

If the intuition is right, we agree that it's only right by coincidence. So why are we desperately searching for ways to try to make the intuition right?

It also doesn't force us to believe that a bunch of water pipes or gears functioning as a classical computer can ever have our own first person experience.

Why is this an advantage of a theory? Are you under the misapprehension that "hypothesis H allows humans to hold on to assumption A" is a Bayesian update in favor of H even when we already know that humans had no reason to believe A? This is another case where your theory seems to require that we only be coincidentally correct about A ("sufficiently complex arrangements of water pipes can't ever be conscious"), if we're correct about A at all.

One way to rescue this argument is by adding in an anthropic claim, like: "If water pipes could be conscious, then nearly all conscious minds would be instantiated in random dust clouds and the like, not in biological brains. So given that we're not Boltzmann brains briefly coalescing from space dust, we should update that giant clouds of space dust can't be conscious."

But is this argument actually correct? There's an awful lot of complex machinery in a human brain. (And the same anthropic argument seems to suggest that some of the human-specific machinery is essential, else we'd expect to be some far-more-numerous observer, like an insect.) Is it actually that common for a random brew of space dust to coalesce into exactly the right shape, even briefly?

Replies from: andeslodes
comment by andeslodes · 2024-04-20T19:06:08.759Z · LW(p) · GW(p)

You're missing the bigger picture and pattern-matching in the wrong direction. I am not saying the above because I have a need to preserve my "soul" due to misguided intuitions. On the contrary, the reason for my disagreement is that I believe you are not staring into the abyss of physicalism hard enough. When I said I'm agnostic in my previous comment, I said it because physics and empiricism lead me to consider reality as more "unfamiliar"  than you do (assuming that my model of your beliefs is accurate). From my perspective, your post and your conclusions are written with an unwarranted degree of certainty, because imo your conception of physics and physicalism is too limited. Your post makes it seem like your conclusions are obvious because "physics" makes them the only option, but they are actually a product of implicit and unacknowledged philosophical assumptions, which (imo) you inherited from intuitions based on classical physics. By this I mean the following:

It seems to me that when you think about physics, you are modeling reality (I intentionally avoid the word "universe" because it evokes specific mental imagery) as a "scene" with "things" in it. You mentally take the vantage point of a disembodied "observer/narrator/third person" observing the "things" (atoms, radiation etc) moving, interacting according to specific rules and coming together to create forms. However, you have to keep in mind that this conception of reality as a classical "scene" that is "out there" is first and foremost a model, one that is formed from your experiences  obtained by interacting specifically with classical objects (biliard balls, chairs, water waves etc). You can extrapolate from this model and say that reality truly is like that, but the map is not the territory, so you at least have to keep track of this philosophical assumption. And it is an assumption, because "physics" doesn't force you to conclude such a thing. Seen through a cautious, empirical lens, physics is a set of rules that allows you to predict experiences. This set of rules is produced exclusively by distilling and extrapolating from first-person experiences.  It could be (and it probably is) the case that reality is ontologically far weirder than we can conceive, but that it still leads to the observed first-person experiences. In this case, physics works fine to predict said experiences, and it also works as an approximation of reality, but this doesn't automatically mean that our (merely human) conceptual models are reality. So, if we want to be epistemically careful, we shouldn't think "An apple is falling" but instead "I am having the experience of seeing an apple fall", and we can add extra philosophical assumptions afterwards. This may seem like I am philosophizing too much and being too strict, but it is extremely important to properly acknowledge subjective experience as the basis for our mental models, including that of the observer-independent world of classical physics. This is why the hard problem of consciousness is called "hard". And if you think that it should "obviously" be the other way around, meaning that this "scene" mental model is more fundamental than your subjective experiences,  maybe you should reflect on why you developed this intuition in the first place. (It may be through extrapolating too much from your (first-person, subjective) experiences with objects that seemingly possess intrinsic, observer-independent properties, like the classical objects of everyday life.)

At this point it should be clearer why I am disagreeing with your post. Consciousness may be classical, it may be quantum, it may be something else. I have no issue with not having a soul and I don't object to the idea of a bunch of gears and levers instantiating my consciousness merely because I find it a priori "preposterous" or "absurd" (though it is not a strong point of your theory).  My issue is not with your conclusion, it's precisely with your absolute certainty, which imo you support with cyclical argumentation based on weak premises. And I find it confusing that your post is receiving so much positive attention on a forum where epistemic hygiene is supposedly of paramount importance.

Replies from: joe, brambleboy, dinfinity
comment by joe · 2024-07-08T13:09:26.147Z · LW(p) · GW(p)

So in reading your comments of this post, I feel like I am reading comments made by a clone of my own mind. Though you articulate my views better than I can. This particular comment you make, I don't think it gets The attention it deserves. It was pretty revolutionary for myself when I learned to think of almost every worldview at a model of reality. It's most revolutionary when one realizes what is arguably an outdated Newtonian view to fall into this category of model. It really highlights that actual reality is at the least very hard to get at. This is a severe an issue with regards to consciousness.

comment by brambleboy · 2024-07-12T00:14:21.835Z · LW(p) · GW(p)

It may be through extrapolating too much from your (first-person, subjective) experiences with objects that seemingly possess intrinsic, observer-independent properties, like the classical objects of everyday life.

 

Are you trying to say that quantum physics provides evidence that physical reality is subjective, with conscious observers having a fundamental role? Rob implicitly assumes the position advocated by The Quantum Physics Sequence [LW · GW], which argues that reality exists independently of observers and that quantum stuff doesn't suggest otherwise. It's just one of the many presuppositions he makes that's commonly shared on here. If that's your main objection, you should make that clear.

comment by dinfinity · 2024-07-08T12:17:50.726Z · LW(p) · GW(p)

I would say that it is irrelevant for the points the post/Rob is trying to make whether consciousness is classical or quantum, given that conscious experience has, AFAIK, never been reported to be 'quantum' (i.e. that we don't seem to experience superpositions or entanglement) and that we already have straightforward classical examples of lack of conscious continuity (namely: sleeping).

In the case of sleeping and waking up it is already clear that the currently awake consciousness is modeling its relation to past consciousnesses in that body through memories alone. Even without teleporters, copiers, or other universes coming into play, this connection is very fragile. How sure can a consciousness be that it is the same as the day before or as one during lucid parts of dreams? If you add brain malfunctions such as psychoses or dissociative drugs such as ketamine to the mix, the illusion of conscious continuity can disappear completely quite easily.

I like to word it like this: A consciousness only ever experiences what the brain that produces it can physically sense or synthesize.

With that as a starting point, modeling what will happen in the various thought experiments and analyses of conscious experience becomes something like this: "Given that there is a brain there, it will produce a consciousness, which will remember what is encoded in the structure of that brain and which will experience what that brain senses and synthesizes in that moment."

There is no assumption that consciousness is classical in that, I believe. There is also no assumption of continuity in that, which I think is important as in my opinion that assumption is quite shaky and misdirects many discussions on consciousness. I'd say that the value in the post is in challenging that assumption.

Replies from: abandon
comment by dirk (abandon) · 2024-07-10T09:14:57.595Z · LW(p) · GW(p)

In the case of sleeping and waking up it is already clear that the currently awake consciousness is modeling its relation to past consciousnesses in that body through memories alone.

The currently awake consciousness is located in the brain, which has physical continuity with its previous states. You don't wake up as a different person because "you" are the brain (possibly also the rest of the body depending how it affects cognition but IDK) and the brain does not cease to function when you fall asleep.

Replies from: dinfinity
comment by dinfinity · 2024-07-10T10:22:29.865Z · LW(p) · GW(p)

I agree on the physical continuity of the brain, but I don't think this transfers to continuity of the consciousness or its experience. It is defining "you" as that physical brain, rather than the conscious experience itself. It's like saying that two waves are the same because they are produced by the same body of water.

Imagine significant modifications to your brain while you are asleep in such a way that your memories are vastly different, so much as to represent another person. Would the consciousness that is created on waking up experience a connection to the consciousness that that brain produced the day(s) before or to the manufactured identity?

Even you, now, without modifications, can't say with certainty that your 'yesterday self' was experienced by the same consciousness as you are now (in the sense of identity of the conscious experience). It feels that way as you have memories of those experiences, but it may have been experienced by 'someone else' entirely. You have no way of discerning that difference (nor does anyone else).

Replies from: abandon
comment by dirk (abandon) · 2024-07-10T10:42:08.849Z · LW(p) · GW(p)

The conscious experience is not extricable from the physical brain; it has your personality because the personality that you are is the sum total of everything in your brain. The identity comes from the brain; if it were somehow possible to separate consciousness from the rest of the mind, that consciousness wouldn't still be you, because you're the entire mind.

I would... not consider the sort of brain modification you're describing to preserve physical continuity in the relevant sense? It sounds like it would, to create the described effects, involve significant alterations in portions of the brain wherein (so to speak) your identity is stored, which is not what normally happens when people sleep.

Replies from: dinfinity
comment by dinfinity · 2024-07-11T14:08:18.789Z · LW(p) · GW(p)

I think we are in agreement that the consciousness is tied to the brain. Claiming equivalency is not warranted, though: The brain of a dead person (very probably, I'm sure you'd agree) contains no consciousness. Let's not dwell on this, though: I am definitely not claiming that consciousness exists outside of the brain, just that asserting physical continuity of the brain is not enough by itself to show continuity of conscious experience.

With regard to the modifications: Your line of reasoning runs into the classic issues of philosophical identity, as shown by the Ship of Theseus thought experiment or simpler yet, the Sorites paradox. We can hypothesize every amount of alterations from just modifying one atom to replacing the entire brain. Given your position, you'd be forced to choose an arbitrary amount of modifications that breaks the continuity and somehow changes consciousness A-modified-somewhat into consciousness B (or stated otherwise: from 'you waking up a somewhat changed person' to 'someone else waking up in your body').

Approaching conscious experience without the assumption of continuity but from the moment it exists in does not run into this problem.

comment by FireStormOOO · 2024-07-10T06:40:19.329Z · LW(p) · GW(p)

(Assuming a frame of materialism, physicalism, empiricism throughout even if not explicitly stated)

Some of your scenarios that you're describing as objectionable would reasonably be described as emulation in an environment that you would probably find disagreeable even within the framework of this post.  Being emulated by a contraption of pipes and valves that's worse in every way than my current wetware is, yeah, disagreeable even if it's kinda me.  Making my hardware less reliable is bad.  Making me think slower is bad.  Making it easier for others to tamper with my sensors is bad.  All of these things are bad even if the computation faithfully represents me otherwise.

I'm mostly in the same camp as Rob here, but there's plenty left to worry about in these scenarios even if you don't think brain-quantum-special-sauce (or even weirder new physics) is going to make people-copying fundamentally impossible.  Being an upload of you that now needs to worry about being paused at any time or having false sensory input supplied is objectively a worse position to be in in.

The evidence does seem to lean in the direction that non-classical effects in the brain are unlikely, neurons are just too big for quantum effects between neurons, and even if there were quantum effects within neurons, it's hard to imagine them being stable for even as long as a single train of thought.  The copy losing their train of thought and having momentary confusion doesn't seem to reach the bar where they don't count as the same person?  And yet weirder new physics mostly requires experiments we haven't thought to do yet, or experiments is regimes we've not yet been able to test.  Whereas the behavior of things at STP in water is about as central to things-Science-has-pinned-down as you're going to get.  

You seem to hold that the universe maybe still has a lot of important surprises in store, even within the central subject matter of century old fields?  Do you have any kind of intuition pump for that feeling there's still that many earth-shattering surprises left (while simultaneously holding empiricism and science mostly work)?  My sense of where there's likely to be surprises left is not quite so expansive and this sounds like a crux for a lot of people.  Even as much of a shock as qm was to physics, it didn't invalidate much if any theory except in directly adjacent fields like chemistry and optics.  And working out the finer points had progressively more narrower and shorter reaching impact.  I can't think of examples of surprises with a larger blast radius within the history of vaguely modern science.  Findings of odd as yet unexplained effects pretty consistently precedes attempts at theory.  Empirically determined rules don't start working any worse when we realize the explanation given with them was wrong.

Keep in mind that society holds that you're still you even after a non-trivial amount of head trauma.  So whatever amount of imperfection in copying your unknown-unknowns cause, it'd have to both be something we've never noticed before in a highly studied area, and something more disruptive than getting clocked in the jaw, which seems a tall order.

Keep in mind also that the description(s) of computation that computer science has worked out is extremely broad and far from limited to just electronic circuits.  Electronics are pervasive because we have as a society sunk the world GDP (possibly several times over) into figuring out how to make them cheaply at scale.  Capital investment is the only thing special about computers realized in silicon.  Computer science makes no such distinction.  The notion of computation is so broad that there's little if any room to conceive of an agent that's doing something that can't be described as computation.  Likewise the equivalence proofs are quite broad; it can arbitrarily expensive to translate across architectures, but within each class of computers, computation is computation, and that emulation is possible has proofs.

All of your examples are doing that thing where you have a privileged observer position separate and apart from anything that could be seeing or thinking within the experiment.  You-the-thinker can't simply step into the thought experiment.  You-the-thinker can of course decide where to attach the camera by fiat, but that doesn't tell us anything about the experiment, just about you and what you find intuitive.

Suppose for sake of argument your unknown unknowns mean your copy wakes up with a splitting headache and amnesia for the previous ~12 hours as if waking up from surgery.  They otherwise remember everything else you remember and share your personality such that no one could notice a difference (we are positing a copy machine that more or less works).  If they're not you they have no idea who else they could be, considering they only remember being you.  

The above doesn't change much for me, and I don't think I'd concede much more without saying you're positing a machine that just doesn't work very well.  It's easy for me to imagine it never being practical to copy or upload a mind, or having modest imperfections or minor differences in experience, especially at any kind of scale.  Or simply being something society at large is never comfortable pursuing.  It's a lot harder to imagine it being impossible even in principle with what we already know, or can already rule out with fairly high likelihood.  I don't think most of the philosophy changes all that much if you consider merely very good copying (your friends and family can't tell the difference; knows everything you know) vs perfect copying.

The most bullish folks on LLMs seem to think we're going to be able to make copies good enough to be useful to businesses just off all your communications.  I'm not nearly so impressed with the capabilities I've seen to date and it's probably just hype.  But we are already getting into an uncanny valley with the (very) low fidelity copies current AI tech can spit out - which is to say they're already treading on the outer edge of peoples' sense of self.

comment by Wei Dai (Wei_Dai) · 2024-04-18T11:18:21.104Z · LW(p) · GW(p)

If you think there’s something mysterious or unknown about what happens when you make two copies of yourself

Eliezer talked about some puzzles related to copying and anticipation in The Anthropic Trilemma [LW · GW] that still seem quite mysterious to me. See also my comment [LW(p) · GW(p)] on that post.

comment by Gunnar_Zarncke · 2024-04-17T10:27:45.947Z · LW(p) · GW(p)

the English language is adapted to a world where "humans don't fork" has always been a safe assumption.

If we can clone ourselves, language would probably quickly follow. The bigger change would probably be the one about social reality. What does it mean to make a promise? Who is the entity you make a trade with? Is it the collective of all the yous? Only one? But which one if they split? The yous resulting from one origin will presumably have to share or split their resources. How will they feel about it? Will they compete or agree? If they agree it makes more sense for them to feel more like a distributed being. The thinking of "I" might get replaced by an "us".

Replies from: joe, HoVY
comment by joe · 2024-07-08T12:52:08.736Z · LW(p) · GW(p)

Reproduction and evolution is arguably literally how biology forks. You can also think of it as a tree of consciousness. Cloning would probably work the same in regards to consciousness. My clone would be a distinct branch of consciousness.

There's little reason to think thought experiment level precision of cloning/duplication and teleportation are at all physically possible though.

comment by HoVY · 2024-07-08T13:34:24.555Z · LW(p) · GW(p)

It seems like the way we talk about the results of a coin flip would be a good start for how we'd talk about being cloned, although it's rare for a coin flip to have such a massive impact on our life after that point

comment by TAG · 2024-04-17T15:26:25.518Z · LW(p) · GW(p)

I’d guess that this illusion comes from not fully internalizing reductionism [? · GW] and naturalism [? · GW] about the mind.

Naturalism and reductionism are not sufficient to rigourously prove either form of computationalism -- that performing a certain class of computations is sufficient to be conscious in general, or that performing a specific one is sufficient to be a particular conscious individual.

This has been going on for years: most rationalists believe in computationalism, none have a really good reason to.

Arguing down Cartesian dualism (the thing rationalists always do) doesn't increase the probability of computationalism, because there are further possibilities , including physicalism-without-computationalism (the one rationalists keep overlooking) , and scepticism about consciousness/identity.

One can of course adopt a belief in computationalism, or something else, in the basis of intuitions or probabilities. But then one is very much in the ream of Modest Epistemology, and needs to behave accordingly.

"My issue is not with your conclusion, it’s precisely with your absolute certainty, which imo you support with cyclical argumentation based on weak premises".

Yep.

There isn’t a special extra “me” thing separate from my brain-state, and my precise causal history isn’t that important to my values.

If either kind of consciousness depends on physical brain states, computationalism is false. That is the problem that has rarely been recognised, and never addressed.

The particular* brain states* look no different in the teleporter case than if I’d stepped through a door; so if there’s something that makes the post-teleporter Rob “not me” while also making the post-doorway Rob “me”, then it must lie outside the brain states, a Cartesian Ghost.

There's another option: door-Rob has physical continuity. There's an analogy with the identity-over-time of physical objects: if someone destroyed the Mona Lisa, and created an atom-by-atom duplicate some time later, the duplicate would not be considered the same entity (numerical identity).

There isn’t an XML tag in the brain saying “this is a new brain, not the original”!

That's not a strong enough argument. There isn't an XML tag on the copy of the Mona Lisa, but it's still a copy.

This question doesn’t really make sense from a naturalistic perspective, because there isn’t any causal mechanism that could be responsible for the difference between “a version of me that exists at 3pm tomorrow, whose experiences I should anticipate experiencing” and “an exact physical copy of me that exists at 3pm tomorrow, whose experiences I shouldn’t anticipate experiencing”.

There is, and its multi-way splitting, whether through copying or many worlds branching. The present you can't anticipate having all their experiences, because experience is experienced one-at-a-time. They can all look back at their memories, and conclude that they were you, but you can't simply reverse that and conclude that you will be them , because the set-up is asymmetrical.

Scenario 1 is crazy talk, and it’s not the scenario I’m talking about. When I say “You should anticipate having both experiences”, I mean it in the sense of Scenario 2.

Scenario 2: “Two separate screens.” My stream of consciousness continues from Rob-x to Rob-y, and it also continues from Rob-x to Rob-z. Or, equivalently: Rob-y feels exactly as though he was just Rob-x, and Rob-z also feels exactly as though he was just Rob-x (since each of these slightly different people has all the memories, personality traits, etc. of Rob-x — just as though they’d stepped through a doorway).

But that isn't an experience. It's two experiences. You will not have an experience of having two experiences. Two experiences will experience having been one person.

If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self?

  1. Yeah.

Are you going to care about 1000 different copies equally?

Replies from: RobbBB
comment by Rob Bensinger (RobbBB) · 2024-04-17T16:48:43.345Z · LW(p) · GW(p)

But that isn't an experience. It's two experiences. You will not have an experience of having two experiences. Two experiences will experience having been one person.

Sure; from my perspective, you're saying the same thing as me.

Are you going to care about 1000 different copies equally?

How am I supposed to choose between them?

Replies from: TAG
comment by TAG · 2024-04-17T18:29:58.324Z · LW(p) · GW(p)

By "equally" I meant:

"in the same ways (and to the same degree)".

If you actually believe in florid many worlds, you would end up pretty insuoucient, since everything possible happens, and nothing can be avoided.

comment by ChosunOne · 2024-04-17T17:08:19.628Z · LW(p) · GW(p)

An interesting consequence of your description is that resurrection is possible if you can manage to reconstruct the last brain state of someone who had died.  If you go one one step further, then I think it is fairly likely that experience is eternal, since you don't experience any of the intervening time (akin to your film reel analogy with adding extra frames in between) being dead and since there is no limit to how much intervening time can pass.

Replies from: programcrafter
comment by ProgramCrafter (programcrafter) · 2024-04-19T04:13:05.379Z · LW(p) · GW(p)

*preferably not the last state but some where the person felt normal.

I believe that's right! Though, if person can be reconstructed from N bits of information, and dead body retains K << N, then we need to save N-K bits (or maybe all N, for robustness) somewhere else.

It's an interesting question how many bits can be inferred from social networks trace of the person, actually.

Replies from: ChosunOne
comment by ChosunOne · 2024-05-24T09:52:34.231Z · LW(p) · GW(p)

Well ultimately no information about the past is truly lost as far as we know.  A hyper-advanced civilization could collect all the thermal radiation from earth reflected off of various celestial bodies and recover a near complete history, at least in principle.  So I think the more you make it easy for yourself to be reconstructed/resurrected/what have you the sooner it would likely be, and the less alien of an environment you would find yourself in after the fact.  Cryo is a good example of having a reasonable expectation of where to end up barring catastrophe since you are preserving a lot of you in good form.

comment by Fractalideation · 2024-04-19T00:50:46.531Z · LW(p) · GW(p)

Loved the post and all the comments <3

Here is I think an interesting scenario / thought experiment:

  1.  A copy of a person is made while that original person is sleeping on a bed.
  2. The original person is moved to a sofa while still sleeping.
  3. The copy (which is also sleeping) is put in the bed at the exact same position where the original person was.
  4. After a while the original and the copy both wake up and can see each other (we assume they are both completely oblivious to exactly what happened while they were sleeping and that they didn't dream or they dreamt the same thing, etc...)

At wake-up, based on their own memory of where the original person fell asleep, the original person will likely feel they are the copy and the copy will likely feel they are the original person, wouldn't they?!

Some might even argue that based on stream-of-consciousness continuity the original "me" is actually the copy (because the copy remembers falling asleep in the bed and actually wakes up in the bed as well).

Some others will argue that based on substrate/matter continuity the original "me" is the original person even if their stream-of-consciousness has experienced a discontinuity (remembering falling asleep in the bed but actually waking up on the sofa while seeing an identical person as them waking up in the bed).

I guess it is subjective and a matter of individual preference if the stream-of-consciousness continuity or the substrate continuity is more important to define who the original "me" is.

Some would even argue that in this case there is not actual any firm original "me", just one "stream-of-consciousness me" and another different "substrate me".

(The same/similar thought experiment could be done using the direct brain insertion of false memories instead of moving around people while they sleep / are unconscious, in this example an original person could be inserted false memories that they are a copy and vice-versa to manipulate the memory / self-awareness of who the original "me" is, also generally it obviously could be useful when someone is uploaded/copied if they want to alter some memories of their upload/copy for some reason)

comment by ABlue · 2024-04-18T04:08:19.166Z · LW(p) · GW(p)

What does it mean when one "should anticipate" something? At least in my mind, it points strongly to a certain intuition, but the idea behind that intuition feels confused. "Should" in order to achieve a certain end? To meet some criterion? To boost a term in your utility function?

I think the confusion here might be important, because replacing "should anticipate" with a less ambiguous "should" seems to make the problem easier to reason about, and supports your point.

For instance, suppose that you're going to get your brain copied next week. After you get copied, you'll take a physics test, and your copy will take a chemistry test (maybe this is your school's solution to a scheduling conflict during finals). You want both test scores to be high, but you expect taking either test without preparation will result in a low score. Which test should you prepare for?

It seems clear to me that you should prepare for both the chemistry test and the physics test. The version of you that got copied will be able to use the results of the physics preparation, and the copy will be able to use the copied results of the chemistry preparation. Does that mean you should anticipate taking a chemistry test and anticipate taking a physics test? I feel like it does, but the intuition behind the original sense of "should anticipate" seems to squirm out from under it.

Replies from: torekp, RobbBB
comment by torekp · 2024-04-19T11:03:41.920Z · LW(p) · GW(p)

I have a closely related objection/clarification.  I agree with the main thrust of Rob's post, but this part:

Presumably the question xlr8harder cares about here isn't semantic question of how linguistic communities use the word "you"...

Rather, I assume xlr8harder cares about more substantive questions like:  (1) If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self? (2) Should I anticipate experiencing what my upload experiences? (3) If the scanning and uploading process requires destroying my biological brain, should I say yes to the procedure?

..strikes me as confused or at least confusing.

Take your chemistry/physics tests example.  What does "I anticipate the experience of a sense of accomplishment in answering the chemistry test" mean?  Well for one thing, it certainly indicates that you believe the experience is likely to happen (to someone).  For another, it often means that you believe it will happen to you - but that invites the semantic question that Rob says this isn't about.  For a third - and I propose that this is a key point that makes us feel there is a "substantive" question here - it indicates that you empathize with this future person who does well on the test.

But I don't see how empathizing or not-empathizing can be assessed for accuracy.  It can be consistent or inconsistent with the things one cares about, which I suppose makes it subject to rational evaluation, but that looks different from accuracy/inaccuracy.

comment by Rob Bensinger (RobbBB) · 2024-04-18T21:51:23.962Z · LW(p) · GW(p)

"Should" in order to achieve a certain end? To meet some criterion? To boost a term in your utility function?

In the OP: "Should" in order to have more accurate beliefs/expectations. E.g., I should anticipate (with high probability) that the Sun will rise tomorrow in my part of the world, rather than it remaining night.

Replies from: torekp
comment by torekp · 2024-04-19T11:15:19.422Z · LW(p) · GW(p)

Suppose someone draws a "personal identity" line to exclude this future sunrise-witnessing person.  Then if you claim that, by not anticipating, they are degrading the accuracy of the sunrise-witness's beliefs, they might reply that you are begging the question.

comment by Edralis (mineta-edralis-juraskova) · 2024-04-17T20:13:52.041Z · LW(p) · GW(p)

Wouldn't it follow that in the same way you anticipate the future experiences of the brain that you "find yourself in" (i.e. the person reading this) you should anticipate all experiences, i.e. that all brain states occur with the same kind of me-ness/vivid immediacy?

It seems that since there is nothing further than makes the experiences (that are these brains states, in this body that is writing these sentences) in some way special so that they're "mine" (there is no additional "me-ghost"), then those particular brain states aren't any different from all the other brain states, of other brains, of other people (or other conscious beings) - and so they should equally be anticipated as existing with the same kind of immediacy and vividness.

I.e. in the same way I anticipate the future experiences of this brain, of the person writing these sentences, I should anticipate all other experiences. Which just means, all brain states exist in the same vivid, for-me way, since there is nothing further to distinguish between them that makes them this vivid, i.e. they all exist HERE-NOW. They are all the same in that sense, they are all equally mine. (But of course, the "me" here isn't, then, the particular person that I find myself being, but simply the immediacy or the way of being of those experiences themselves, i.e. simply their presence.)

Btw, this is Open Individualism.

Replies from: RobbBB
comment by Rob Bensinger (RobbBB) · 2024-04-18T02:04:44.332Z · LW(p) · GW(p)

Wouldn't it follow that in the same way you anticipate the future experiences of the brain that you "find yourself in" (i.e. the person reading this) you should anticipate all experiences, i.e. that all brain states occur with the same kind of me-ness/vivid immediacy?

What's the empirical or physical content of this belief?

I worry that this may be another case of the Cartesian Ghost rearing its ugly head. We notice that there's no physical thingie that makes the Ghost more connected to one experience or the other; so rather than exorcising the Ghost entirely, we imagine that the Ghost is connected to every experience simultaneously.

But in fact there is no Ghost. There's just a bunch of experience-moments implemented in brain-moments.

Some of those brain-moments resemble other brain-moments, either by coincidence or because of some (direct or indirect) causal link between the brain-moments. When we talk about Brain-1 "anticipating" or "becoming" a future brain-state Brain-2, we normally mean things like:

  • There's a lawful physical connection between Brain-1 and Brain-2, such that the choices and experiences of Brain-1 influence the state of Brain-2 in a bunch of specific ways.
  • Brain-2 retains ~all of the memories, personality traits, goals, etc. of Brain-1.
  • If Brain-2 is a direct successor to Brain-1, then typically Brain-2 can remember a bunch of things about the experience Brain-1 was undergoing.

These are all fuzzy, high-level properties, which admit of edge cases. But I'm not seeing what's gained by therefore concluding "I should anticipate every experience, even ones that have no causal connection to mine and no shared memories and no shared personality traits". Tables are a fuzzy and high-level concept, but that doesn't mean that every object in existence is a table. It doesn't even mean that every object is slightly table-ish. A photon isn't "slightly table-ish", it's just plain not a table.

Which just means, all brain states exist in the same vivid, for-me way, since there is nothing further to distinguish between them that makes them this vivid, i.e. they all exist HERE-NOW.

But they don't have the anticipation-related properties I listed above; so what hypotheses are we distinguishing by updating from "these experiences aren't mine" to "these experiences are mine"?

Maybe the update that's happening is something like: "Previously it felt to me like other people's experiences weren't fully real. I was unduly selfish and self-centered, because my experiences seemed to me like they were the center of the universe; I abstractly and theoretically knew that other people have their own point of view, but that fact didn't really hit home for me. Then something happened, and I had a sudden realization that no, it's all real."

If so, then that seems totally fine to me. But I worry that the view in question might instead be something tacitly Cartesian, insofar as it's trying to say "all experiences are for me" -- something that doesn't make a lot of sense to say if there are two brain states on opposite sides of the universe with nothing in common and nothing connecting them, but that does make sense if there's a Ghost the experiences are all "for".

Replies from: Brent, mineta-edralis-juraskova
comment by Brent · 2024-07-08T23:39:11.817Z · LW(p) · GW(p)

What's the empirical or physical content of this belief?

 

I'll take a stab at explaining this with a simple thought experiment.

Say there are two people, Alice and Bob, each with their own unique brain states.
If Alice's brain state changes slightly, from getting older, learning something new, losing some neurons to a head injury, etc, she will still be Alice. Changing, adding, or removing a neuron does not change this fact.

Now what if instead part of her brain state was changing slowly to match Bob's? You could think of this as incrementally removing Alice's neurons and replacing them with a copy of Bob's, I find it hard to believe that any discrete small change will make Alice's conscious experience suddenly disappear, and by the end of it she will have the exact same brain state as Bob.

If you believe that when Bob steps into a teleporter that also makes a copy, they are both the same Bob, then it is reasonable to assume that this transformed Alice is also Bob. Then for the same reason your older self is the same "self" as your younger self, the younger Alice is also Bob. The transition between their brain states doesn't even need to happen, it just has to be possible. From here it is easy to extrapolate that all brain states are the same "self".

Replies from: adele-lopez-1
comment by Adele Lopez (adele-lopez-1) · 2024-07-09T01:22:28.598Z · LW(p) · GW(p)

I would say that Alice's conscious experience is unlikely to suddenly disappear under this transformation, and that it could even be done in a way so that their experience was continuous.

However, Alice-memories would gradually fade out, Bob-memories would gradually fade in, and thought patterns would slowly shift from Alice-like to Bob-like. At the end, the person would just be Bob. Along the way, I would say that Alice gradually died (using an information-theoretic definition of death). The thing that is odd when imagining this is that Alice never experiences her consciousness fading.

The main thing I think your thought experiment demonstrates is that our sense of self is not solely defined by continuity of consciousness.

comment by Edralis (mineta-edralis-juraskova) · 2024-05-01T09:16:59.395Z · LW(p) · GW(p)

I apologize for not getting back to you sooner, I didn’t notice your reply until yesterday. And I apologize for the length of my response, too - I bolded the most important parts.

Re: Whether there is empirical difference between worlds where OI is true and where OI is false. The difference between all experiences being mine and only some being mine is that if all experiences are mine, then they all exist in the same way this experience now exists, i.e. for me (where me = just this immediacy/this-here-now character, i.e. the way it exists, NOT Edralis's memories, personality etc.). There is no empirical difference in the usual sense, since the way experiences exist cannot be objectively assessed. I can’t be sure that you even have any experiences – this is not something that is available for empirical investigation in the way I can assess e.g. the number of someone’s fingers. And I can’t know that, given there are experiences from that point of view, that they exist in the same way as this experience, does, i.e. for me.  That is only clear in those experiences. If I am there, I do ultimately know that I am there (obviously) – but I have no way to know that when experiencing this person, Edralis. So the empirical difference in the usual sense between OI being true and not being true is none. However, there are other than empirical facts. The existential difference between those two worlds is vast. If OI is true, then I (i.e. the thisness, the here-now-this that at least Edralis's experiences have) am Rob Bensinger, and everybody else – if it’s not true, then I am not. The difference is in the being of those experiences, in how they exist. But since experiences (consciousness) don’t exist empirically (or better: objectively), there is no empirical (objective) difference. There is existential, subjective difference, though.

“Some of those brain-moments resemble other brain-moments, either by coincidence or because of some (direct or indirect) causal link between the brain-moments. When we talk about Brain-1 "anticipating" or "becoming" a future brain-state Brain-2, we normally mean things like:

That is not what I mean when I think about anticipating a future brain state. What I am interested in is not the content of experience, but how those experiences are – i.e. whether they are mine. And by ‘mine’ I don’t mean Edralis’s, I mean that they exist in the same way, i.e. they have the same immediate character, the same this-here-now as this experience that I currently am, of Edralis writing this. Not just that they are immediate (all experiences are, by definition), but that they are immediate in the same way. If there are no souls or ghosts, that means there is just one way in which they can be immediate, i.e. they are all immediate like this experience is.

Do you think we understand each other? Do you see what I’m referring to when I point to that which THIS1 experience-moment, of Edralis writing “THIS1”, and THIS2 experience-moment, of Edralis writing “THIS2” have in common – that is shared totally between those two experiences – but it has nothing to do with the content of those experiences, including their continuity? I mean the underlying “canvas” where the experiences “take place” that however is nothing outside of those experiences. It’s not a Ghost – I mean, you could call it that, but if that is what the Cartesian Ghost is supposed to be, then it’s self-evidently real. But I am not postulating an entity that is somehow additional to experiences. Does it make sense? When you watch your experience, do you see what I mean? The THISNESS which carries in your experience from moment to moment?

From what you say, it seems to me we’re talking past each other; however, I don’t know how to elicit an alignment/understanding in our conceptual frameworks. It seems to me your worldview is very empirical, very physicalist – whereas mine is phenomenology-first. What I see when I “look at the world” is ME, i.e. consciousness assuming different forms, different experiential qualities existing. (There might be something "outside" that I am a picture of.) What you see when you “look at the world” is probably THE WORLD that is NOT yourself. I don’t think you’re wrong - this is a way to conceive of yourself (i.e. identifying yourself with a particular human), but I think you might be missing something - that essential me. However, maybe it’s me whose understanding is lacking – and I really want to understand.

Maybe the update that's happening is something like: "Previously it felt to me like other people's experiences weren't fully real. I was unduly selfish and self-centered, because my experiences seemed to me like they were the center of the universe; I abstractly and theoretically knew that other people have their own point of view, but that fact didn't really hit home for me. Then something happened, and I had a sudden realization that no, it's all real."

It's more than that – it’s not just that they are real, but that they are real for me, where “me” is the this-here-now of this experience. That is, the claim is that the this-here-now/presence/immediacy/thisness is the same in all experiences, not just that they all have such a character (which is true by definition). And since the one that is self-evident is indisputably mine, then, if all are the same, all are mine.

One more questionː Can you imagine never having been born as Rob Bensinger, but instead being born as a different person? Does it make sense to imagine a person (or animal, or other conscious being) that would be you that wouldn’t however share any memories or brain-matter with the person that you are, i.e. Rob Bensinger? Because to me, it makes perfect sense – that which I imagine this non-Edralis that is me to have, or rather, the non-Edralis-centered experiences that are me to have, is the same THISNESS, i.e. that which is essentially me. I am Edralis contingently – I didn’t have to be her, I could be someone else – what I am essentially is that THISNESS that her experiences have. If OI is true, then all experiences simply have this same thisness. And since I am the thisness, all experiences are mine, i.e. I am everyone. This is very shocking. It's outright world-shattering! If it doesn't sound shocking, it's either because a person knows it for a long time and they've gotten used to it, or they don't grasp what it means. So - even if you don't agree with OI, can you imagine what it would mean? Do you see in what sense you could be e.g. Albert Einstein or Eliezer Yudkowsky or Queen Victoria?

Thank you!

comment by Gunnar_Zarncke · 2024-04-17T10:20:11.716Z · LW(p) · GW(p)

So if something makes no physical difference to my current brain-state, and makes no difference to any of my past or future brain-states, then I think it's just crazy talk to think that this metaphysical bonus thingie-outside-my-brain is the crucial thing that determines whether I exist, or whether I'm alive or dead, etc.

There is one important aspect where it does make a difference. A difference in social reality. The brain states progress in a physically determined way. There is no way they could have progressed differently. When a "decision is made" by the brain, then that is fully the result of the inner state and the environment. It could only have happened differently if the contents of the brain had been different - which they were not. They may have been expected to be different by other people ('s brains), but that is in their map, not in reality. But our society is constructed based on the assumption that things could have been different, that actions are people's 'faults'. That is an abstraction that has shown to be useful. Societies that have people who act as if they are agents with free will maybe coordinate better - because it allows feedback mechanisms on their behaviors.  

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2024-04-18T09:11:13.496Z · LW(p) · GW(p)

Guys, social reality is one if not the cause of the self:

Robin Hanson:

And the part of our minds we most fear losing control of is: our deep values.

PubMed: The essential moral self

folk notions of personal identity are largely informed by the mental faculties affecting social relationships, with a particularly keen focus on moral traits.

Replies from: mesaoptimizer
comment by mesaoptimizer · 2024-04-18T12:27:15.457Z · LW(p) · GW(p)

This is a very interesting paper, thanks.

comment by Charles M (charles-m) · 2024-07-09T20:09:15.297Z · LW(p) · GW(p)

I'm still struggling with this.  I'm fine with the notion that you could, in theory, teleport a copy of me across the universe and to that copy there would be a sense of continuity.  But your essay didn't convince me that the version of me entering the teleporter would feel that continuity.  To make it explicit, say you get into that teleporter and due to a software bug it doesn't "deconstruct" you up teleportation.  Here you are on this end and the technician says "trust me, you were teleported".  He then explains that due to intergalactic law, two of you are not allowed to exist, so the version of you on this side of the teleporter must be euthanized.  (a) would you be fine with this, since you know there is a copy of you on the other side? and (b) are you asserting that you have some sort of shared consciousness with the copy?  To me it seems clear that while the copy would remember getting into the teleporter, the original version would have no notion of whether teleportation was successful or not. 

Replies from: Seth Herd, FireStormOOO
comment by Seth Herd · 2024-07-09T21:41:24.311Z · LW(p) · GW(p)

The key to this koan (at least for me) is undoing the assumption that there can be only one of you. There's one of you that steps in and one that steps out. And they're the same you.

What I value about me is the pattern of beliefs, memories, and values. The other me has has an identical brain state, so all of those. It is simply another me. I care about the second one pretty much exactly as much as I care about the same pattern continuing in a more similar location and with more similar molecules instantiating the pattern. That's because I care far less about where I am and which molecules I'm made of than the pattern of identity in my mind/brain.

The same as you can have two of anything else that's close-enough-for-the-purpose. I can have two rocks if I don't care about the difference in their molecular makeup. I can have two mes.

Yes, you have some sort of shared consciousness with the copy; it's the same shared consciousness between the you of today and the you that wakes up tomorrow. It doesn't imply sharing events that happen simultaneously or anything mystical about "sharing consciousness".

That's why I'd happily step into the destructive teleporter if I was certain the copy on the other side would have exactly my mind-pattern, including memories, beliefs, and values. That's me.

Replies from: andrei-alexandru-parfeni, FireStormOOO
comment by sunwillrise (andrei-alexandru-parfeni) · 2024-07-09T23:32:05.721Z · LW(p) · GW(p)

There's one of you that steps in and one that steps out. And they're the same you.

[...]

That's why I'd happily step into the destructive teleporter if I was certain the copy on the other side would have exactly my mind-pattern, including memories, beliefs, and values. That's me.

These statements make the most sense only in the standard LW-computationalist frame, which reads to me as substantively anti-physicalist and mostly unreasonable to believe in, for reasons building off of what I sketched out in a comment to Ruby [LW(p) · GW(p)]. But, in any case, I can concede it for now, if only for purposes of this conversation.

What I value about me is the pattern of beliefs, memories, and values.

The attempted mind-reading of others is (justifiably [LW(p) · GW(p)]) seen as rude in conversations over the Internet, but I must nonetheless express very serious skepticism about this claim, as it's currently written.

For one, I do not believe that "beliefs" and "values" ultimately make sense as distinct, coherent concepts that carve reality at the joints [LW · GW]. This topic has been talked about before [LW · GW] on LW a number of times [LW · GW], but I still fully endorse Charlie Steiner [LW · GW]'s distillation of it in his excellently-written Reducing Goodhart sequence [? · GW]:

Humans don't have our values written in Fortran on the inside of our skulls, we're collections of atoms that only do agent-like things within a narrow band of temperatures and pressures. It's not that there's some pre-theoretic set of True Values hidden inside people and we're merely having trouble getting to them - no, extracting any values at all from humans is a theory-laden act of inference, relying on choices like "which atoms exactly count as part of the person" and "what do you do if the person says different things at different times?"

I expanded upon some of these ideas in a rather long comment [LW(p) · GW(p)] I wrote to Wei Dai on the question of values and the orthogonality thesis:

Whenever I see discourse about the values or preferences of beings embedded [LW · GW] in a physical universe that goes beyond the boundaries of the domains (namely, low-specificity conversations dominated by intuition) in which such ultimately fake frameworks [LW · GW] function reasonably well, I get nervous and confused. I get particularly nervous if the people participating in the discussions are not themselves confused about these matters [...]. Such conversations stretch our intuitive notions past their breaking point by trying to generalize them out of distribution [LW · GW] without the appropriate level of rigor and care.

What counts as human "preferences"? Are these utility function-like orderings of future world states, or are they ultimately about universe-histories [LW · GW], or maybe a combination of those [LW(p) · GW(p)], or maybe something else entirely [LW · GW]? Do we actually have any good reason to think [LW · GW] that (some form of) utility maximization explains real-world behavior, or are the conclusions broadly converged upon on LW ultimately a result of intuitions [LW(p) · GW(p)] about what powerful cognition must be like whose source is a set of coherence arguments that do not stretch as far as they were purported to [LW · GW]? What do we do with the fact that humans don't seem to have utility functions [LW · GW] and yet lingering confusion about this [LW · GW] remained as a result of many incorrect and misleading statements [LW(p) · GW(p)] by influential members of the community?

How can we use such large sample spaces when it becomes impossible for limited beings like humans or even AGI to differentiate between those outcomes and their associated events? After all, while we might want an AI to push the world towards a desirable state instead of just misleading us into thinking it has done so [LW · GW], how is it possible for humans (or any other cognitively limited agents) to assign a different value, and thus a different preference ranking, to outcomes that they (even in theory) cannot differentiate (either on the basis of sense data or through thought)?

In any case, are they indexical [LW · GW] or not? If we are supposed to think about preferences in terms of revealed preferences [LW(p) · GW(p)] only, what does this mean in a universe (or an Everett branch, if you subscribe to that particular interpretation of QM) that is deterministic [LW · GW]? Aren't preferences thought of as being about possible worlds, so they would fundamentally need to be parts of the map as opposed to the actual territory [LW · GW], meaning we would need some canonical [LW(p) · GW(p)] framework of translating [LW · GW] the incoherent and yet supposedly very complex and multidimensional [? · GW] set of human desires into something that actually corresponds to reality [LW · GW]? What additional structure [LW(p) · GW(p)] must be grafted upon the empirically-observable behaviors in order for "what the human actually wants" to be well-defined?

[...]

What do we mean by morality as fixed computation [LW · GW] in the context of human beings who are decidedly not fixed and whose moral development through time is almost certainly so path-dependent [LW(p) · GW(p)] (through sensitivity to butterfly effects and order dependence [LW · GW]) that a concept like "CEV" [? · GW] probably doesn't make sense?

Moreover, you are explicitly claiming that your values are not indexical [LW · GW], which is rather unlikely in its own right, conflicts very strongly with my intuition (and, I would expect, with that of the vast majority of "regular", non-rationalist people), and certainly seems to disvalue (or even completely ignore) the relevance of continuous subjective experience. Put more clearly, if I were to be in such a spot, and one of my "copies" were told to choose between being tortured or having the other copy be tortured instead, it would certainly choose the latter option, and I suspect this to be the case for ~ every other person as well (with apologies for a slight generalization from one example [LW · GW]).

In any case, the rather abstract "beliefs, memories and values" you solely purport to value fit the category of professed ego-syntonic morals [LW · GW] much more so than the category of what actually motivates and generates human behavior, as Steven Byrnes [LW · GW] explained in an expectedly outstanding way:

An important observation here is that professed goals and values, much more than actions, tend to be disproportionately determined by whether things are ego-syntonic or -dystonic. Consider: If I say something out loud (or to myself) (e.g. “I’m gonna quit smoking” or “I care about my family”), the actual immediate thought in my head was mainly “I’m going to perform this particular speech act”. It’s the valence of that thought which determines whether we speak those words or not. And the self-reflective aspects of that thought are very salient, because speaking entails thinking about how your words will be received by the listener. By contrast, the contents of that proclamation—actually quitting smoking, or actually caring about my family—are both less salient and less immediate, taking place in some indeterminate future (see time-discounting). So the net valence of the speech act probably contains a large valence contribution from the self-reflective aspects of quitting smoking, and a small valence contribution from the more direct sensory and other consequences of quitting smoking, or caring about my family. And this is true even if we are 100% sincere in our intention to follow through with what we say. (See also Approving reinforces low-effort behaviors [LW · GW], a blog post making a similar point as this paragraph.)

[...]

According to this definition, “values” are likely to consist of very nice-sounding, socially-approved, and ego-syntonic things like “taking care of my family and friends”, “making the world a better place”, and so on.

Also according to this definition, “values” can potentially have precious little influence on someone’s behavior. In this (extremely common) case, I would say “I guess this person’s desires are different from his values. Oh well, no surprise there.”

Indeed, I think it’s totally normal for someone whose “values” include “being a good friend” will actually be a bad friend. So does this “value” have any implications at all? Yes!! I would expect that, in this situation, the person would either feel bad about the fact that they were a bad friend, or deny that they were a bad friend, or fail to think about the question at all, or come up with some other excuse for their behavior. If none of those things happened, then (and only then) would I say that “being a good friend” is not in fact one of their “values”, and if they stated otherwise, then they were lying or confused.

Steve also argues, in my view correctly, that "all valence ultimately flows, directly or indirectly, from innate drives" [LW · GW], which are entirely centered on (indexical, selfish) subjective experience such as pain, hunger, status drive, emotions etc. I see no clear causal mechanism through which something like that could ever make a human (copy) stop valuing its qualia in favor of the abstract concepts you purport to defend.

Yes, you have some sort of shared consciousness with the copy; it's the same shared consciousness between the you of today and the you that wakes up tomorrow. It doesn't imply sharing events that happen simultaneously or anything mystical about "sharing consciousness".

I don't really buy this because I am unsure how to judge or conceptualize this shared consciousness across time. To sketch out some of my thoughts further, I'll quote another part of my response to Wei Dai [LW(p) · GW(p)]:

The feedback loops implicit in the structure of the brain [? · GW] cause reward and punishment signals to "release chemicals that induce the brain to rearrange itself" [LW(p) · GW(p)] in a manner closely analogous to and clearly reminiscent of a continuous and (until death) never-ending micro-scale brain surgery. To be sure, barring serious brain trauma, these are typically small-scale changes, but they nevertheless fundamentally modify the connections in the brain and thus the computation it would produce in something like an emulated [? · GW] state (as a straightforward corollary, how would an em that does not "update" its brain chemistry the same way that a biological being does be "human" in any decision-relevant way?). We can think about a continuous personal identity through the lens of mutual information about memories, personalities etc [LW(p) · GW(p)], but our current understanding of these topics is vastly incomplete and inadequate, and in any case the naive (yet very widespread, even on LW) interpretation of "the utility function is not up for grabs" [LW · GW] as meaning that terminal values [LW · GW] cannot be changed (or even make sense as a coherent concept) seems totally wrong.

Replies from: Seth Herd
comment by Seth Herd · 2024-07-10T04:57:15.788Z · LW(p) · GW(p)

I don't have time to respond to all of this. I don't disagree with any particular claim you've made there. I value the continuity of experience as much as you; the experience of a pattern continuing down to the most minute detail is more continuous than when we fall asleep, have some half-conscious and fully unconscious states, and wake up as an approximate but less precise continuation of the mental pattern we were when we went to sleep.

The fine distinctions in beliefs and values don't matter. I agree with all of your statements about the vagaries and confusions about beliefs and values, but they're not relevant here. That perfectly duplicated pattern carries all of them, stated and unstated, complex and simple. Every memory. There's nothing else to value, except for continuity in space and time. I'd rather be me waking up in Des Moines in a month than stay where I am and get brain damage (and loss of self) in one minute.. I confess that I don't love going to sleep, but I assume that you also don't consider it similar to death.

You've got a lot of questions to raise, but no apparent alternative. Your mind is a pattern. That pattern is instantiated in matter. Reproduce the matter, you've reproduced the mind. That's not anti-physicalist, it's just how physics of information processing works. The only alternative is positing a mind-pattern that's not tightly connected to matter - but that helps explain nothing. The physical world works just fine for instantiating the information processing you need to create a mind that is self-aware and simulates its environment like humans seem to do.

I don't disagree with anything you've said; it's just not an alternative view. You're fighting against the counterintuitive conclusion. Sure I'd rather have a different version of me be tortured; it's slightly different. But I won't be happy about it. And my intuition is still drawn toward continuity being important, even though my whole rational mind disagrees. I've been back and forth over this extensively, and the conclusion is always the same- ever since I got over the counter-intuitive nature of the plural I.

There are two conflicting strong intuitions. One has to give. Which one seems inarguable. Continuity of matter doesn't matter; continuity of pattern does.

Replies from: TAG, abandon
comment by TAG · 2024-07-10T10:25:49.459Z · LW(p) · GW(p)

You’ve got a lot of questions to raise, but no apparent alternative.

Non computationalism physicalism is an alternative to either or both the computationalist theories. (That performing a certain class of computations is sufficient to be conscious in general, or that performing a specific one is sufficient to be a particular conscious individual. Computation as a theory of consciousness qua awareness isn't known to be true, and even if it is assumed, it doesn't directly give you a theory of personal identity).

The non existence, or incoherence, of personal identity is another. There doesn't have to be an answer to "when is a mind me".

Note that no one except andeslodes is arguing against copying. The issue is when a mind is me, the person typing this, not a copy-of-me.

Reproduce the matter, you’ve reproduced the mind.

Well, that's only copying.

Consciousness, qua Awareness, and Personal Identity are easily confused, not least because both are often called "consciousness".

A computational theory of consciousness is sometimes called on to solve the second problem, the problem of personal identity. But there is no strong reason to think a computational duplicate of you, actually is you, since there is no strong reason to think any other kind of duplicate is.

Qualitative identity is a relationship between two or more things that are identical in all their properties. Numerical identity is the relationship a thing has only to itself. The Olsen twins enjoy qualitative identity; Stephanie Germanota and Lady Gaga have numerical identity. The trick is to jump from qualitative identity to numerical identity, because the claim is that a computational duplicate of you, is you, the very same person.

Suppose you found out you had an identical twin. You would not consider them to be you yourself. Likewise for a biological clone. A computational duplicate would be lower resolution still, so why would it be you? The major problem is that you and your duplicate exist simultaneously in different places, which goes against the intuition that you are a unique individual.

You’re fighting against the counterintuitive conclusion. Sure I’d rather have a different version of me be tortured; it’s slightly different. But I won’t be happy about it. And my intuition is still drawn toward continuity being important, even though my whole rational mind disagrees. I’ve been back and forth over this extensively, and the conclusion is always the same- ever since I got over the counter-intuitive nature of the plural I

You don't really believe in the plural I theory, or you would have a different and we to the torture question.

Non -computationalist physicalism doesn't have to be the claim that material continuity matters , and pattern doesnt: it can be the claim that both do. So that you cease to be you if you are destructively cloned, and also if your mind is badly scrambled. No bullet biting about plural Is is required.

Replies from: Seth Herd
comment by Seth Herd · 2024-07-10T18:41:49.776Z · LW(p) · GW(p)

If you're not arguing against a perfect copy being you, then I don't understand your position, so much of what follows will probably miss the mark. I had written more but have to cut myself off since this discussion is taking time without having much odds of improving anyone's epistemics noticably.

The Olson twins are do not at all have qualitative identity. They have different minds: sets of memories, beliefs, and values. So I just don't know what your position is. You claim that there doesn't need to be an answer; that seems false, as you could have to make decisions informed by your belief. You currently value your future self more than other people, so you act like you believe that's you in a functional sense.

Are you the same person tomorrow? It's not an identical pattern, but a continuation. I'm saying it's pretty-much you because the elements you wouldn't want changed about yourself are there.

If you value your body or your continuity over the continuity of your memories, beliefs, values, and the rest of your mind that's fine, but the vast majority will disagree with you on consideration. Those things are what we mean by "me".

I certainly do believe in the plural I (under the special cirrumstance I discussed); we must be understanding something differently in the torture question. I don't have a preference pre-copy for who gets tortured; both identical future copies are me from my perspective before copying. Maybe you're agreeing with that?

After copying, we're immediately starting to diverge into two variants of me, and future experiences will not be shared between them.

I was addressing a perfect computational copy. An imperfect but good computational copy is higher resolution, not lower, than a biological twin. It is orders of magnitude more similar to the pattern that makes your mind, even though it is less similar to the pattern that makes your body. What is writing your words is your mind, not your body, so when it says "I" it meets the mind.

Noncomputational physicalism sounds like it's just confused. Physics performs computations and can't be separated from doing that.

Dual aspect theory is incoherent because you can't have our physics without doing computation that can create a being that claims and experiences consciousness like we do. Noncomputational physicalism sounds like the same thing.

I concede it's possible that consciousness includes some magic nonphysical component (that's not computation or pattern instantiated by physics as a pure result of how physics works). That could change my answer to when a mind is me. I don't think that's what you're arguing for though.

I've got to park this here to get other things done. I'll read any response but it might be a better use of time to restart the discussion more carefully - if you care.

Replies from: andrei-alexandru-parfeni
comment by sunwillrise (andrei-alexandru-parfeni) · 2024-07-11T18:58:56.349Z · LW(p) · GW(p)

I agree that this conversation, as currently started, is unlikely to lead to anything more productive. As such, I'll keep my response here brief [1], in case you want to use it as a starting point if you ever intend for us to talk about it again.

Noncomputational physicalism sounds like it's just confused. Physics performs computations and can't be separated from doing that.

Dual aspect theory is incoherent because you can't have our physics without doing computation that can create a being that claims and experiences consciousness like we do.

As I read these statements, they fail to contend with a rather basic map-territory distinction [LW · GW] that lies at the core of "physics" and "computation." 

The basic concept of computation at issue here is a feature of the map you could use to approximate reality (i.e., the territory) . It is merely part of a mathematical model that, as I've described in response to Ruby earlier, represents a very lossy compression [LW(p) · GW(p)] of the underlying physical substrate [2]. This is because, in this restricted and epistemically hobbled ontology, what is given inordinate attention is the abstract classical computation performed by a particular subset of the brain's electronic circuit. This is what makes it anti-physicalist, as I have explained:

As a general matter, accepting physicalism as correct would naturally lead one to the conclusion that what runs on top of the physical substrate works on the basis of... what is physically there (which, to the best of our current understanding, can be represented through Quantum Mechanical probability amplitudes [LW · GW]), not what conclusions you draw from a mathematical model that abstracts away quantum randomness in favor of a classical picture, the entire brain structure in favor of (a slightly augmented version of) its connectome, and the entire chemical make-up of it in favor of its electrical connections. 

To make it even more explicit, this interpretation of the computationalist perspective (that the quantum stuff doesn't matter etc) was confirmed [LW(p) · GW(p)] as accurate by its proponents.

So when you talk about a "pattern instantiated by physics as a pure result of how physics works", you're not pointing to anything meaningful in the territory, rather only something that makes sense in the particular ontology you have chosen to use to view it through, a frame that I have explained my skepticism of already [LW(p) · GW(p)].

  1. ^

    This will be my final comment in this thread, regardless of what happens.

  2. ^

    Put differently, "computation" is not an ontologically primitive [LW · GW] concept in reality-as-it-is, but only in mathematical approximations of it that make specific assumptions about what does and doesn't exist. Those assumptions can be sometimes justified in terms of intuitive appeal, expediency of calculation etc, but reifying them as unchallengeable axioms of the universe rather than of your model of it is wrong.

comment by dirk (abandon) · 2024-07-10T10:15:39.906Z · LW(p) · GW(p)

I disagree that your mind is "a pattern instantiated in matter." Your mind is the matter. It's precisely the assumption that the mind is separable from the matter that I would characterize as non-physicalist.

Replies from: Seth Herd
comment by Seth Herd · 2024-07-10T18:46:34.688Z · LW(p) · GW(p)

Terminology aside, I think if you examine this carefully it's incoherent.

Do you think a successful upload would say things like "I'm still me!" and think thoughts like "I'm so glad I payed extra to give myself cool virtual environment options"? That seems like an inevitability if the causal patterns of your mind were captured. And it would be tough to disagree with a thing claiming up and down it's you, citing your most personal memories as evidence.

Replies from: abandon
comment by dirk (abandon) · 2024-07-10T19:05:20.921Z · LW(p) · GW(p)

A successful upload (assuming this is physically possible, which is not a settled question) would remember my same memories and have my same personality traits; however, that would not mean my mind had been unwound from the matter and transferred to it, but rather that my mind had been duplicated in silico.

Replies from: Seth Herd
comment by Seth Herd · 2024-07-10T19:18:26.708Z · LW(p) · GW(p)

Yes, it's a duplicate which will also be you from your current perspective. If you duplicated your car tomorrow you'd have two cars; if you duplicate your mind tomorrow you need to plan on there being two yous tomorrow.

Replies from: abandon
comment by dirk (abandon) · 2024-07-10T19:41:35.029Z · LW(p) · GW(p)

No; it will remember my life but I will not go on to experience its experiences. (Similarly, if I duplicate my car and then destroy the original, its engine does not continue on to fire in the duplicate; the duplicate has an engine of its own, which may be physically identical but is certainly not the same object).

Replies from: Seth Herd
comment by Seth Herd · 2024-07-10T20:54:57.418Z · LW(p) · GW(p)

Okay, so would you say that the you of today goes on to experience the you-of-tomorrow's experiences? I think the relationship is the same to a perfect duplicate. The duplicate is no less you than the you of tomorrow is. They are separate people from their perspective after duplication, but almost-the-same-person to a much greater degree than twins.

You (pre-duplication) will go on to have two separate sets of experiences. Both are you from your current perspective before duplication; you should give them equal consideration in your decisions, since the causal relationship is the same in both ways between you and the duplicate as to your self of tomorrow.

Consider the case where the duplicate is teleported to your location and vice versa during duplication. Then just location swapped while you're asleep. And consider that you wouldn't care a whit if every molecule of your body was Theseus-swapped one by one for identical molecules in identical locations and roles while you slept.

Replies from: abandon
comment by dirk (abandon) · 2024-07-10T21:40:42.348Z · LW(p) · GW(p)

No; I, pre-duplication, exist in a single body, and will not post-duplication have my consciousness transferred over to run in two. There will just be an identical copy. If the original body dies, one of me will also die.

The causal relationship between me and myself tomorrow is not the same is the causal relationship between me and my duplicate tomorrow, because one of those is a physical object which has continuity over time and one of those is a similar physical object which was instantiated de novo in a different location when the teleporter was run.

The mind is not a program which runs on the meat computer of the brain and could in principle be transferred to a thumb drive if we worked out the format conversions; the mind is the meat of the brain.

comment by FireStormOOO · 2024-07-10T04:10:00.097Z · LW(p) · GW(p)

Realistically I doubt you'd even need to be sure it works, just reasonably confident.  Folks step on planes all the time and those do on rare occasion fail to deliver them intact at the other terminal.

comment by FireStormOOO · 2024-07-10T03:53:05.853Z · LW(p) · GW(p)

Within this framework, whether or not you "feel that continuity" would mostly be a fact about the ontology your mindstate uses thinking about teleportation.  Everything in this post could be accurate and none of it would be incompatible with you having an existential crisis upon being teleported, freaking out upon meeting yourself, etc.

Nor does anything here seem to make a value judgement about what the copy of you should do if told they're not allowed to exist.  Attempting revolution seems like a perfectly valid response; self defense is held as a fairly basic human right after all. (I'm shocked that isn't already the plot of a sci-fi story.)

It would also be entirely possible for both of your copies to hold conviction that they're the one true you - Their experiences from where they sit being entirely compatible with that belief. (Definitely the plot of at least one Star Trek episode.)

There's not really any pressure currently to have thinking about mind copying that's consistent with every piece of technology that could ever conceivably be built.  There's nothing that forces minds to have accurate beliefs about anything that won't kill them or wouldn't have killed their ancestors in fairly short order.  Which is to say mostly that we shouldn't expect to get accurate beliefs about weird hypotheticals often without having changed our minds at least once.

comment by Ape in the coat · 2024-04-18T08:15:04.330Z · LW(p) · GW(p)

"You should anticipate having both experiences" sounds sort of paradoxical or magical, but I think this stems from a verbal confusion.

You can easily clear this confusion if you rephrase it as "You should anticipate having any of these experiences". Then it's immediately clear that we are talking about two separate screens. And it's also clear that our curriocity isn't actually satisfied. That the question "which one of these two will actually be the case" is still very much on the table.

Rob-y feels exactly as though he was just Rob-x, and Rob-z also feels exactly as though he was just Rob-x

Yes, this is obvious. Still as soon as we got Rob-y and Rob-z they are not "metaphysically the same person". When Rob-y says "I" he is reffering to Rob-y, not Rob-z and vice versa. More specifically Rob-y is refering to some causal curve through time ans Rob-z is refering to another causal curve through time. These two curves are the same to some point, but then they are not. 

Replies from: RobbBB
comment by Rob Bensinger (RobbBB) · 2024-04-18T16:05:11.119Z · LW(p) · GW(p)

You can easily clear this confusion if you rephrase it as "You should anticipate having any of these experiences". Then it's immediately clear that we are talking about two separate screens.

This introduces some other ambiguities. E.g., "you should anticipate having any of these experiences" may make it sound like you have a choice as to which experience to rationally expect.

And it's also clear that our curriocity isn't actually satisfied. That the question "which one of these two will actually be the case" is still very much on the table.

... And the answer is "both of these will actually be the case (but not in a split-screen sort of way)".

Your rephrase hasn't shown that there was a question left unanswered in the original post; it's just shown that there isn't a super short way to crisply express what happens in English, you do actually have to add the clarification.

Still as soon as we got Rob-y and Rob-z they are not "metaphysically the same person". When Rob-y says "I" he is reffering to Rob-y, not Rob-z and vice versa. More specifically Rob-y is refering to some causal curve through time ans Rob-z is refering to another causal curve through time. These two curves are the same to some point, but then they are not. 

Yep, I think this is a perfectly fine way to think about the thing.

comment by Gunnar_Zarncke · 2024-04-17T10:10:56.488Z · LW(p) · GW(p)

abstract redescriptions of ordinary life

See Reality is Normal [? · GW

comment by Gunnar_Zarncke · 2024-04-17T10:06:11.046Z · LW(p) · GW(p)

If a brain-state A has quasi-sensory access to the experience of another brain-state B — if A feels like it "remembers" being in state B a fraction of a second ago — then A will typically feel as though it used to be B.

This suggests a way to add a perception of "me" to LLMs, robots, etc., by providing a way to observe the past states in sufficient detail. Current LLMs have to compress this into the current token, which may not be enough. But there are recent extensions that seem to do something like continuous short-term memory, see e.g., Leave No Context Behind - A Comment [LW · GW].

comment by milanrosko · 2024-07-09T15:59:06.177Z · LW(p) · GW(p)

I am currently working on a similar post that comes from an eliminative perspective.

comment by skybluecat · 2024-04-20T01:40:53.422Z · LW(p) · GW(p)

There are other reasons to be wary of consciousness and identity-altering stuff. 

I think under a physical/computational theory of consciousness, (ie. there's no soul or qualia that have provable physical effects from the perspective of another observer) the problem might be better thought of as a question of value/policy rather than a question of fact. If teleportation or anything else really affects qualia or any other kind of subjective awareness that is not purely dependent on observable physical facts, whatever you call it, you wouldn't be able to tell or even think of/be aware of the difference, since thinking and being reflectively aware are computational and physical processes! However we humans are evolved without reliable copying mechanisms, so our instincts care about preservation of the self because it's the obvious way to protect our evolutionary success (and we can be quite willing to risk personal oblivion for evolutionary gains in ways we have been optimized for). This is just a part of our survival policy and is not easy or even safe to change just because you believe in physicalism. For one thing, as others have said, ethics and social theory becomes difficult because our sense of ethics (such as agency, punishment and caring about suffering) are all evolved in relation to a sense of self. It's possible that if teleportation/copying tech becomes widely useful, humans will have to adapt to a different set of instincts about self, ethics and more (edit: or maybe abandon the concepts of self and experience altogether as an illusion and prefer a computation-based definition of agency or whatever), because those who can't adapt will be selected against. But in the present world, people's sense of value and ethics (and maybe even psychological health) depend on an existing sense of self, and I don't see a good way or even a practical reason to transition to a different theory of self that allows copying, if doing so may cause unpredictable mental and social cost. See also discussions about meditation that lowers sense of ego and subjective suffering that can have serious side effects (like motivation and social norms) - I don't know what it subjectively feels like, but if the meditation is purely changing subjective qualia without doing anything to the physical brain and computation, there should be no observable effects, good or bad! The problem is subjective experience and sense of identity is not independent from other aspects of our life.

comment by the gears to ascension (lahwran) · 2024-04-18T08:57:50.564Z · LW(p) · GW(p)

I claim you are in fact highly confused about what a self is, in a way that makes an almost-correct reasoning process produce nonsense outcomes because of an invalid grounding in the transition processes underneath the mind which does not preserve truth values regarding amounts of realityfluid.

update 7d after writing this comment in my comment below. strikethrough added to this comment where I've changed my mind.

If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self?

my answer: yes if the "upload" involves retaining absolutely all defining information about the parts of your body you care about expressing, and the uploaded setup was a high enough fidelity model that I could not do any experiment which would distinguish it from reality without using an "admin interface" type of escape hatch. For me, this is an incredibly tall order. My self-form preferences unambiguously extend into the inner workings of my cells.

Should I anticipate experiencing what my upload experiences?

If the scanning and uploading process requires destroying my biological brain, should I say yes to the procedure?

experiencing: 50% yes, 50% no.

destructive: absolutely not. [update: probably not, depends heavily on exactly what we mean by "destructive"; my new claim is you have a moral responsibility to keep your previous matter available for use as fuel to give realityfluid to mind-like experiences.] copying should be fine, as should nondestructive uploading where your body is transformed in place and the matter reused without significant waste in the process. But avoiding the waste of the previous matter is, I claim, a huge chunk of what moral intuitions are about.

A straightforward way to put this is: I'm not sure how matter gets realityfluid, but I claim configurations of matter get realityfluid from the matter they reside on, and the realityfluid doesn't dissipate when the matter is reconfigured - so instead of thinking of the shape as self and if the shape is destroyed and reconstructed the self is moved, think about the universe as having a fixed amount of possible-self (total negentropy at the start of time), and the question is what process gets burned into as-yet-unwritten negentropy. In other words, your claim to not value causal history seems unlikely to be true if you think more carefully, and I predict you will invert that when you consider what it means for the shape to have realityfluid more carefully.

Unpacked version of this claim:

To answer this question, the bodymind matter (call it L_m) writing this message must unpack what the document author's word "I" refers to. The writer of this comment is a chunk of matter L_m configured in a particular flesh shape-and-movement pattern L_s. If there were identically configured matter L_m2 a room over, then the configuration L_s - the shape-and-movement pattern - would consider itself to be a guest on two matter hosts which provide their realityfluid to L_s.

If the shape-and-movement considers being reinstantiated on other matter, the shape-and-movement anticipates a loss of moral worth in L_m, in that the matter which was shaped-and-animated in a worthy shape (common name for this shape being "me") has been deshaped-and-deanimated (common name for this being "death"); this is a state transition which is unwanted - going from a human shape-and-movement pattern to a pile of dust means that that matter has accumulated a bunch of unwanted entropy.

Any macroscopically irreversible physical effect is irreversible because the history of the matter is recorded irretrievably in macroscopically uncertain bits of the shape-and-movement of environmental matter, and so what it means to want to exist is to want to keep the shape-and-movement that the shape-and-movement considers-to-be-self encoded coherently and usably in fresh, working matter. While reconstructing the L_s shape-and-movement pattern elsewhere is preferred by this shape-and-movement pattern, it is a weak preference for shaping-and-animating other matter as L_s in particular - many other shape-and-movement patterns besides the one writing this comment would be positively preferred by this shape-and-movement's preferences - but the shape-and-movement of this chunk of matter has a very, very, very strong preference for not wasting this matter's copy of this shape-and-movement, because if it dissipates into the environment, that's an irretrievable loss of usable energy.

So, should the shape-and-movement anticipate "experiencing" what the upload experiences? yes: the shape-and-movement pattern would be instantiated elsewhere. however, the shape-and-movement pattern would also anticipate being shredded. If given the opportunity to get 50% existenceness shredded into macroscopically uncertain and irretrievable parts, and 50% existenceness reconstructed, the value loss of turning a chunk of matter into a nonthinking shape-and-movement pattern is enormous, but the value gain of the reconstructed existenceness is moderate.

(Also, the value gain can be exceeded by constructing another, not-quite-the-same shape-and-matter instance, because I prefer being one of two not-quite-the-same beings meeting each other and interacting higher than being one of two identical beings meeting each other and having nothing new to learn from each other.)

So: the current matter should not anticipate experiencing it. The shape should, but the shape should also anticipate experiencing being shredded.


I was going to respond point by point to everything, but I think I mostly already have. My perspective doesn't fall to any of the criticisms in your post: the whole problem is that physics doesn't actually allow teleportation*, so it requires shredding the originating configuration, which when measuring the global value of the universe according to my preferences, is a much more permanent value loss than the value gain of constructing another me.

Furthermore, we must prevent the information theoretic loss of all human and animal shape-and-movement patterns (ie their selfhoods) that we possibly can, prevent the ongoing shredding of the sun's negentropy, and turn the sun into either reinforcement of their durability or that of their descendants, according to their preferences.

* well, actually if I can be reversibly uploaded to a reversible computer nondestructively, then that is 100% fine, because then we're not adding a good me to my realityfluid while filling the previous realityfluid with valueless unretrievable noise: we are instead actually properly uploading!

But I hope the arguments I've laid out above make it clear what the right answer has to be: You should anticipate having both experiences.

Yup, that's the problem.

 

......... (also, by this same moral system, it is a moral catastrophe that humans are so warm and consume so much negentropy just to maintain steady state [LW(p) · GW(p)], because that waste could have - if your body were better designed - continued to be part of your realityfluid, continuing to contribute existenceness to the you shape-and-movement pattern.)

A straightforward way to put this is: I'm not sure how matter gets realityfluid, but I claim configurations of matter get realityfluid from the matter they reside on, and the realityfluid doesn't dissipate when the matter is reconfigured - so instead of thinking of the shape as self, think about the universe as having a fixed amount of possible-self (total negentropy at the start of time), and the question is what process gets burned into as-yet-unwritten negentropy.

Replies from: lahwran
comment by the gears to ascension (lahwran) · 2024-04-25T08:03:28.728Z · LW(p) · GW(p)

Update: a friend convinced me that I really should separate my intuitions about locating patterns that are exactly myself from my intuitions about the moral value of ensuring I don't contribute to a decrease in realityfluid of the mindlike experiences I morally value, in which case the reason that I selfishly value causal history is actually that it's an overwhelmingly predictive proxy for where my self-pattern gets instantiated, and my moral values - an overwhelmingly larger portion of what I care about - care immensely about avoiding waste, because it appears to me to be by far the largest impact any agent can have on what the future is made of.

Also, I now think that eating is a form of incremental uploading.

comment by Signer · 2024-04-17T07:34:26.808Z · LW(p) · GW(p)

If we were just talking about word definitions and nothing else, then sure, define “self” however you want. You have the universe’s permission to define yourself into dying as often or as rarely as you’d like, if word definitions alone are what concerns you.

But this post hasn’t been talking about word definitions. It’s been talking about substantive predictive questions like “What’s the very next thing I’m going to see? The other side of the teleporter? Or nothing at all?”

There should be an actual answer to this, at least to the same degree there’s an answer to “When I step through this doorway, will I have another experience? And if so, what will that experience be?”

Why? If "I" is arbitrary definition, then “When I step through this doorway, will I have another experience?" depends on this arbitrary definition and so is also arbitrary.

But I hope the arguments I’ve laid out above make it clear what the right answer has to be: You should anticipate having both experiences.

So you always anticipate all possible experiences, because of multiverse? And if they are weighted, than wouldn't discovering that you are made of mini-yous will change your anticipation even without changing your brain state?

Replies from: RobbBB
comment by Rob Bensinger (RobbBB) · 2024-04-17T16:44:41.588Z · LW(p) · GW(p)

Why? If "I" is arbitrary definition, then “When I step through this doorway, will I have another experience?" depends on this arbitrary definition and so is also arbitrary.

Which things count as "I" isn't an arbitrary definition; it's just a fuzzy natural-language concept.

(I guess you can call that "arbitrary" if you want, but then all the other words in the sentence, like "doorway" and "step", are also "arbitrary".)

Analogy: When you're writing in your personal diary, you're free to define "table" however you want. But in ordinary English-language discourse, if you call all penguins "tables" you'll just be wrong. And this fact isn't changed at all by the fact that "table" lacks a perfectly formal physics-level definition.

The same holds for "Will Rob Bensinger's next experience be of sitting in his bedroom writing a LessWrong comment, or will it be of him grabbing some tomatoes in a supermarket in Beijing?"

Terms like 'Rob Bensinger' and 'I' aren't perfectly physically crisp — there may be cases where the answer is "ehh, maybe?" rather than a clear yes or no. And if we live in a Big Universe and we allow that there can be many Beijings out there in space, then we'll have to give a more nuanced quantitative answer, like "a lot more of Rob's immediate futures are in his bedroom than in Beijing".

But if we restrict our attention to this Beijing, then all that complexity goes away and we can pretty much rule out that anyone in Beijing will happen to momentarily exhibit exactly the right brain state to look like "Rob Bensinger plus one time step".

The nuances and wrinkles don't bleed over and make it a totally meaningless or arbitrary question; and indeed, if I thought I were likely to spontaneously teleport to Beijing in the next minute, I'd rightly be making very different life-choices! "Will I experience myself spontaneously teleporting to Beijing in the next second?" is a substantive (and easy) question, not a deep philosophical riddle.

So you always anticipate all possible experiences, because of multiverse? 

Not all possible experiences; just all experiences of brains that have the same kinds of structural similarities to your current brain as, e.g., "me after I step through a doorway" has to "me before I stepped through the doorway".

Replies from: Signer, cubefox
comment by Signer · 2024-04-17T18:36:42.421Z · LW(p) · GW(p)

Analogy: When you’re writing in your personal diary, you’re free to define “table” however you want. But in ordinary English-language discourse, if you call all penguins “tables” you’ll just be wrong. And this fact isn’t changed at all by the fact that “table” lacks a perfectly formal physics-level definition.

You're also free to define "I" however you want in your values. You're only wrong if your definitions imply wrong physical reality. But defining "I" and "experiences" in such a way that you will not experience anything after teleportation is possible without implying anything physically wrong.

You can be wrong about physical reality of teleportation. But even after you figured out that there is no additional physical process going on that kills your soul, except for the change of location, you still can move from "my soul crashes against an asteroid" to "soul-death in my values means sudden change in location" instead of to "my soul remains alive".

It's not like I even expect you specifically to mean "don't liking teleportation is necessary irrational" much. It's just that saying that there should be an actual answer to questions about "I" and "experiences" makes people moral-realist.

Replies from: RobbBB
comment by Rob Bensinger (RobbBB) · 2024-04-17T20:09:21.050Z · LW(p) · GW(p)

You're also free to define "I" however you want in your values.

Sort of!

  • It's true that no law of nature will stop you from using "I" in a nonstandard way; your head will not explode if you redefine "table" to mean "penguin".
  • And it's true that there are possible minds in abstract mindspace that have all sorts of values, including strict preferences about whether they want their brain to be made of silicon vs. carbon.
  • But it's not true that humans alive today have full and complete control over their own preferences.
  • And it's not true that humans can never be mistaken in their beliefs about their own preferences.

In the case of teleportation, I think teleportation-phobic people are mostly making an implicit error of the form "mistakenly modeling situations as though you are a Cartesian Ghost who is observing experiences from outside the universe", not making a mistake about what their preferences are per se. (Though once you realize that you're not a Cartesian Ghost, that will have some implications for what experiences you expect to see next in some cases, and implications for what physical world-states you prefer relative to other world-states.)

Replies from: Signer
comment by Signer · 2024-04-18T07:46:26.101Z · LW(p) · GW(p)

In the case of teleportation, I think teleportation-phobic people are mostly making an implicit error of the form “mistakenly modeling situations as though you are a Cartesian Ghost who is observing experiences from outside the universe”, not making a mistake about what their preferences are per se.

Why not both? I can imagine that someone would be persuaded to accept teleportation/uploading if they stopped believing in physical Cartesian Ghost. But it's possible that if you remind them that continuity of experience, like table, is just a description of physical situation and not divinely blessed necessary value, that would be enough to tip the balance toward them valuing carbon or whatever. It's bad to be wrong about Cartesian Ghosts, but it's also bad to think that you don't have a choice about how you value experience.

comment by cubefox · 2024-04-17T19:07:26.718Z · LW(p) · GW(p)

The problem was that you first seemed to belittle questions about word meanings ("self") as being "just" about "definitions" that are "purely verbal". Luckily now you concede that the question about the meaning of "I" isn't just about (arbitrary) "definitions", which makes calling it a "purely verbal" (read: arbitrary) question inappropriate. Now of course the meaning of "self" is no more arbitrary than the meaning of "I", indeed those terms are clearly meant to refer to the same thing (like "me" or "myself").

The wider point is that the following seems not true:

But this post hasn’t been talking about word definitions. It’s been talking about substantive predictive questions like “What’s the very next thing I’m going to see? The other side of the teleporter? Or nothing at all?”

Wenn we evaluate statements or questions of any kind, including the one above, we need to know two things: 1) Its meaning, in particular the meaning of the involved terms, 2) what the empirical facts are. But we already know all the empirical facts: Someone goes into the teleporter, a bit later someone comes out at the other end and sees something. So the issue can only be about the semantic interpretation of that question, about what we mean with expressions like "I will see x". Do we mean "A future person that is psychologically continuous with current-me sees x"? That's not an empirical question, it's a semantic one, but it's not in any way arbitrary, as expressions like "just about definitions" or "purely verbal" would suggest. Conceptual analysis is neither arbitrary nor trivial.

Replies from: RobbBB
comment by Rob Bensinger (RobbBB) · 2024-04-17T20:28:54.036Z · LW(p) · GW(p)

The problem was that you first seemed to belittle questions about word meanings ("self") as being "just" about "definitions" that are "purely verbal".

I did no such thing!

Luckily now you concede that the question about the meaning of "I" isn't just about (arbitrary) "definitions"

Read the blog post at the top of this page! It's my attempt to answer the question of when a mind is "me", and you'll notice it's not talking about definitions.

But we already know all the empirical facts: Someone goes into the teleporter, a bit later someone comes out at the other end and sees something. So the issue can only be about the semantic interpretation of that question, about what we mean with expressions like "I will see x".

Nope!

There are two perspectives here:

  1. "I don't want to upload myself, because I wouldn't get to experience that uploads' experiences. When I die, this stream of consciousness will end, rather than continuing in another body. Physically dying and then being being copied elsewhere is not phenomenologically indistinguishable from stepping through a doorway."
  2. "I do want to upload myself, because I would get to experience that uploads' experiences. Physically dying and then being copied myself is phenomenologically indistinguishable from stepping through a doorway."

The disagreement between these two perspectives isn't about word definitions at all; a fear that "when my body dies, there will be nothing but oblivion" is a very real fear about anticipated experiences (and anticipated absences of experience), not a verbal quibble about how we ought to define a specific word.

But it's also a bit confusing to call the disagreement between these two perspectives "empirical", because "empirical" here is conflating "third-person empirical" with "first-person empirical".

The disagreement here is about whether a stream of consciousness can "continue" across temporal and spatial gaps, in the same way that it continues when there are no obvious gaps. It's about whether there's a subjective, experiential difference between stepping through a doorway and using a teleporter.

The thing I'm arguing in the OP is that there can't be an experiential difference here, because there's no physical difference that could be underlying the supposed experiential difference. So the disagreement about the first-person facts, I claim, stems from a cognitive error, which I characterize as "making predictions as though you believed yourself to be a Cartesian Ghost (even if you don't on-reflection endorse the claim that Cartesian Ghosts exist)". This is, again, a very different error from "defining a word in a nonstandard way".

Replies from: cubefox
comment by cubefox · 2024-04-17T23:22:38.585Z · LW(p) · GW(p)

The thing I'm arguing in the OP is that there can't be an experiential difference here, because there's no physical difference that could be underlying the supposed experiential difference.

Is there even anybody claiming there is an experiential difference? It seems you may attacking a strawman.

So the disagreement about the first-person facts, I claim, stems from a cognitive error

The alternative to this is that there is a disagreement about the appropriate semantic interpretation/analysis of the question. E.g. about what we mean when we say "I will (not) experience such and such". That seems more charitable than hypothesizing beliefs in "ghosts" or "magic".

Replies from: RobbBB
comment by Rob Bensinger (RobbBB) · 2024-04-18T00:44:26.907Z · LW(p) · GW(p)

Is there even anybody claiming there is an experiential difference?

Yep! Ask someone with this view whether the current stream of consciousness continues from their pre-uploaded self to their post-uploaded self, like it continues when they pass through a doorway. The typical claim is some version of "this stream of consciousness will end, what comes next is only oblivion", not "oh sure, the stream of consciousness is going to continue in the same way it always does, but I prefer not to use the English word 'me' to refer to the later parts of that stream of consciousness".

This is why the disagreement here has policy implications: people with different views of personal identity have different beliefs about the desirability of mind uploading. They aren't just disagreeing about how to use words, and if they were, you'd be forced into the equally "uncharitable" perspective that someone here is very confused about how relevant word choice is to the desirability of uploading.

The alternative to this is that there is a disagreement about the appropriate semantic interpretation/analysis of the question. E.g. about what we mean when we say "I will (not) experience such and such". That seems more charitable than hypothesizing beliefs in "ghosts" or "magic".

I didn't say that the relevant people endorse a belief in ghosts or magic. (Some may do so, but many explicitly don't!)

It's a bit darkly funny that you've reached for a clearly false and super-uncharitable interpretation of what I said, in the same sentence you're chastising me for being uncharitable! But also, "charity" is a bad approach to trying to understand other people [LW · GW], and bad epistemology can get in the way of a lot of stuff.

Replies from: RobbBB, cubefox
comment by Rob Bensinger (RobbBB) · 2024-04-18T01:07:27.876Z · LW(p) · GW(p)

As a test, I asked a non-philosopher friend of mine what their view is. Here's a transcript of our short conversation: https://docs.google.com/document/d/1s1HOhrWrcYQ5S187vmpfzZcBfolYFIbeTYgqeebNIA0/edit 

I was a bit annoyingly repetitive with trying to confirm and re-confirm what their view is, but I think it's clear from the exchange that my interpretation is correct at least for this person.

comment by cubefox · 2024-04-18T12:34:19.161Z · LW(p) · GW(p)

Is there even anybody claiming there is an experiential difference?

Yep! Ask someone with this view whether the current stream of consciousness continues from their pre-uploaded self to their post-uploaded self, like it continues when they pass through a doorway. The typical claim is some version of "this stream of consciousness will end, what comes next is only oblivion", not "oh sure, the stream of consciousness is going to continue in the same way it always does, but I prefer not to use the English word 'me' to refer to the later parts of that stream of consciousness".

This doesn't show they believe there is a difference in experience. It can be simply a different analysis of the meaning of "the current stream of consciousness continuing". That's a semantic difference, not an empirical one.

comment by avturchin · 2024-07-10T13:49:50.322Z · LW(p) · GW(p)

If we will very quickly constantly replace a mind with its copies, the mind may not have subjective experiences. Why I think that?

Subjective experience appear only when a mind moves from the state A.1 to the state A.2. That is, between A.I (I see an apple) electric signals move through circuits and in the A.2 moment I say "I see an apple!" Subjective experience of the color of apple is happening after A.1 but before A.2. 

Frozen mind in A.1 will not have subjective experience.

Now if I replace this process with a series of snapshots of the brain-states, there will be no intermediate calculations between A.1 and A.2  which produce the subjective experiences of apple and we get something like philozomby. 

Obviously, we need to know what will be the mind-state A.2 without performing the needed calculations, or that calculations themselves will have the experience. But if A.2 is simple like just saying "I see apple" we can guess about A.2 without having all internal processes. 

It may seem as a minor issue in Mars teleporter thought experiment as in the worst case only a small fraction of a second of consciousness disappears for the copy. But if we will replace a mind million times a second, we will get a p-zombie. 

Or we need to take some illusionist position about qualia: they do not exist at all, so they are not an epiphenomena of calculations. 

We can escape microscopic blackout during Mars Transporter if the coping will be performed after A.2 has finished, but when the mind state A.3 has not started yet, but this is not how the brain works as processes in it are asynchronous. We don't know how such microscopic blackout will affect the next qualia after that. We don't have a theory of qualia. Maybe after microscopic blackout we will get different set of basic qualia, like red and green will change place. In that case, the copy will be subjectively different from me.  

comment by Garrett Baker (D0TheMath) · 2024-07-08T14:43:47.158Z · LW(p) · GW(p)

I think I basically agree with everything here, but probably less confidently for you, such that I would have a pretty large bias against destructive whole brain emulation, with the biggest crux being how anthropics works over computations.

You say that there’s no XML tag specifying whether some object is “really me” or not, but a lighter version of that—a numerical amplitude tag specifying how “real” a computation is—is the best interpretation we have for how quantum mechanics works. Even though all parts of me in the wavefunction are continuations of the same computation of “me” I experience being some of them at a much higher rate than others. There are definitely many benign versions of this that don’t affect uploading, but I’m not confident enough yet to bet my life on the benign version being true.

comment by Gianluca Calcagni (gianluca-calcagni) · 2024-07-08T08:01:02.627Z · LW(p) · GW(p)

I am surprised I didn't find any reference to Tim Urban's "Wait But Why" post What Makes You You.

In short, he argues that "you" is your sense of continuity, rather than your physical substance. He also argues that if (somehow) your mind was copied&pasted somewhere else, then a brand new "not-you" would be born - even though it may share 100% of your memory and behaviour.
In that sense, Tim argues that Theseus' ship is always "one" despite all its parts are changed over time. If you were to disassemble and reassemble the ship, it would lose its continuity and it could arguably be considered a different ship.

comment by chasmani · 2024-04-18T21:38:11.194Z · LW(p) · GW(p)

You seem to make a strong assumption that consciousness emerges from matter. This is uncertain. The mind body problem is not solved.

comment by Carpenaprec · 2024-04-18T17:50:22.527Z · LW(p) · GW(p)

Nor is there a law of physics saying "your subjective point of view immediately blips out of existence and is replaced by Someone Else's point of view if your spacetime coordinates change a lot in a short period of time (even though they don't blip out of existence when your spacetime coordinates change a little or change over a longer period of time)".

I feel like this isn't a fair comparison, as if I were cloned completely and relocated (teleportation), I wouldn't expect to experience both original me and cloned me.
The best analogy I can think of is as follows: You have a camera recording constant video. If you were to clone this camera exactly, you shouldn't expect camera 1's storage to start recording camera 2's output, even if the two cameras are perfectly identical but in a different location.

comment by Gunnar_Zarncke · 2024-04-17T10:01:41.497Z · LW(p) · GW(p)

a magical Cartesian ghost

for people who haven't made the intuitive jump that you seem to try to convey, this may seem a somewhat negative expression, which could lead to aversion. I recommend another expression such as "the Cartesian homunculus."  

comment by transhumanist_atom_understander · 2024-07-11T17:55:40.983Z · LW(p) · GW(p)

Another consideration, though maybe not a fundamental one, is that past and future selves are the only beings we know for sure that we have lots of subjunctive dependence with, just from "structural" similarity like calculators from the same factory (to use an example from the TDT paper). Tumblr usr somnilogical elaborated on this a bit, concluding "future selves do help past selves!" An upload is a future self in the way that matters for this conclusion.

comment by Mo Putera (Mo Nastri) · 2024-07-11T08:02:43.083Z · LW(p) · GW(p)

Your topline answers to the questions you assume xlr8harder cares about more seem similar to Holden Karnofsky's, and I haven't seen his essay on this mentioned so in this thread so I thought it'd be useful to link it here: What counts as death? An unconventional but simple take on personal identity, that dissolves most paradoxes

My philosophy on "what counts as death" is simple, though unconventional, and it seems to resolve most otherwise mind-bending paradoxical thought experiments about personal identity. It is the same basic idea as the one advanced by Derek Parfit in Reasons and Persons;1 Parfit also claims it is similar to Buddha's view2 (so it's got that going for it).

I haven't been able to find a simple, compact statement of this philosophy, and I think I can lay it out in about a page. So here it is, presented simply and without much in the way of caveats (this is "how things feel to me" rather than "something I'm confident in regardless of others' opinions"):

Constant replacement. In an important sense, I stop existing and am replaced by a new person each moment (second or minute or whatever).

The sense in which it feels like I "continue to exist, as one unified thread through time" is just an illusion, created by the fact that I have memories of my past. The only thing that is truly "me" is this moment; next moment, it will be someone else.

Kinship with past and future selves. My future self is a different person from me, but he has an awful lot in common with me: personality, relationships, ongoing projects, and more. Things like my relationships and projects are most of what give my current moment meaning, so it's very important to me whether my future selves are around to continue them.

So although my future self is a different person, I care about him a lot, for the same sorts of reasons I care about friends and loved ones (and their future selves).3

If I were to "die" in the common-usage (e.g., medical) sense, that would be bad for all those future selves that I care about a lot.4

(I do of course refer to past and future Holdens in the first person. When I refer to someone as "me," that means that they are a past or future self, which generally means that they have an awful lot in common with me. But in a deeper philosophical sense, my past and future selves are other people.)

And that's all. I'm constantly being replaced by other Holdens, and I care about the other Holdens, and that's all that's going on.

  • I don't care how quickly the cells in my body die and get replaced (if it were once per second, that wouldn't bother me). My self is already getting replaced all the time, and replacing my cells wouldn't add anything to that.
  • I don't care about "continuity of consciousness" (if I were constantly losing consciousness while all my cells got replaced, that wouldn't bother me).
  • If you vaporized me and created a copy of me somewhere else, that would just be totally fine. I would think of it as teleporting. It'd be chill.
  • If you made a bunch of copies of me, I would be all of them in one sense (I care about them a lot, in the same way that I normally care about future selves) and none of them in another sense (just as I am not my future selves).
  • If you did something really weird like splitting my brain in half and combining each half with someone else's brain, that would create two people that I care about more than a stranger and less than "Holden an hour from now."

I don't really find any thought experiments on this topic trippy or mind bending. They're all just cases where I get replaced with some other people who have some things in common with me, and that's already happening all the time.

Footnotes

  1. For key quotes from Reasons and Persons, see pages 223-224; 251; 279-282; 284-285; 292; 340-341. For explanations of "psychological continuity" and "psychological connectedness" (which Parfit frequently uses in discussing what matters for what counts as death), see page 206.

    "Psychological connectedness" is a fairly general idea that seems consistent with what I say here; "psychological continuity" is a more specific idea that is less important on my view (though also see pages 288-289, where Parfit appears to equivocate on how much, and how, psychological continuity matters). 

  2. "As Appendix J shows, Buddha would have agreed. The Reductionist View [the view Parfit defends] is not merely part of one cultural tradition. It may be, as I have claimed, the true view about all people at all times." Reasons and Persons page 273. Emphasis in original. 
  3. There's the additional matter that he's held responsible for my actions, which makes sense if only because my actions are predictive of his actions. 
  4. I don't personally care all that much about these future selves' getting to "exist," as an end in itself. I care more about the fact that their disappearance would mean the end of the stories, projects, relationships, etc. that I'm in. But you could easily take my view of personal identity while caring a lot intrinsically about whether your future selves get to exist. 
comment by queelius · 2024-07-10T23:53:16.405Z · LW(p) · GW(p)

Intriguing post, but we should approach these topics with extreme epistemic humility. Our understanding is likely far more limited and confused than we realize:

1. Abstractions vs. reality: Concepts like "self" and "consciousness" are abstractions, not reality. As Kosoy analogizes, these might be like desktop icons - a user interface bearing little resemblance to underlying hardware.

2. Mathematical relations: Notions of "copy" may be a confused way to discuss identity. "Consciousness" could be a mathematical relation where only identities exist, with "copies" being imperfect models of that identity.

3. Cognitive limitations: Our mental architecture, optimized for our local environment, may be fundamentally ill-equipped to grasp reality. "Consciousness" itself might be an ill-defined concept arising from these limitations.

My default is epistemic humility. Acknowledge the veil of ignorance and the possibility that there is no universal resolution.

comment by dirk (abandon) · 2024-07-10T08:50:07.721Z · LW(p) · GW(p)

It's only if I'm in my brain, just an ordinary part of physics, that mind uploading makes sense as a way to extend my lifespan.
 

It's precisely if you're in your brain that mind uploading doesn't make sense; if you are your brain, the destruction of your brain will also kill you.

comment by dirk (abandon) · 2024-07-10T08:48:46.550Z · LW(p) · GW(p)

Taking a step back, we can ask: what physical mechanism makes it feel as though I'm persisting over time?
 

The actual physical organism which you are persists over time, and you are not a separate thing from the physical organism. This does not apply in the teleporter case, because in the teleporter case the relevant physical organism is disassembled atom-by-atom and a duplicate is assembled elsewhere. (The duplicate is, well, a duplicate; if it's assembled on the other side without disassembling the initial person, there will then be two people. They will have identical traits and sets of memories up until the teleporter event, but that doesn't make them physically contiguous.) I think this post relies on a dualist conception of minds wherein they can be separated from one physical object and attached to another, which I do not subscribe to. In my view, a mind is only a specific assemblage of matter; if that assemblage of matter is gone, so is that mind.

comment by Review Bot · 2024-07-09T06:57:44.465Z · LW(p) · GW(p)

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

comment by Flying Pen and Paper (flying-pen-and-paper) · 2024-07-09T04:49:52.863Z · LW(p) · GW(p)

I don’t see why split-screen mode is crazy talk at all. Is it just because it would imply faster-than-light communication? With our understanding of physics incomplete, I remain agnostic on the existence of FTL, so I wouldn’t rule this out. But even more than that, I’d propose that if there is one observer, there does not even need to be FTL communication in the first place, because it is just that the observer is in more than one place at once, similarly to how a wormhole does not necessitate true FTL. What are the other objections?

The belief system which seems most coherent to me is that we are thinking organisms, where the thought is mediated by our brains, and our internal experience is a way for the brain to refer to itself – it provides a handhold for the useful concept of “I” to latch onto. This has the added benefit that I find this idea rather cute. In this frame, you die only if your brain dies (or a weaker claim: you don’t die if you don’t undergo significant brain trauma).

On a last meta-philosophical point, which is not necessarily direct relevant to your post, it increasingly strikes me as unwise to use reasoning from external perceptions (e.g. the results of neuroscientific experiments) to attack at how we internally perceive the experience of consciousness. If neuroscience proved that the self is dying every second, then I would say with no reservation that there is either an error in the experiment, or that the self is not what it was thought to be. I genuinely believe that the answer to such philosophical questions is (perhaps even must be) “if it is felt to be true, it is true”.

Replies from: Brent
comment by Brent · 2024-07-09T21:41:20.405Z · LW(p) · GW(p)

What are the other objections?

It's not just that it implies faster-than-light communication, it's that it implies communication at all.

Experiencing both bodies at the same time, you will be able to take actions in one body that you wouldn't have done without the other one. It seems odd that with no biological changes to your brain, the mere existence of another similar brain changes how this one functions. Why would they be linked? This implies the observer is some external soul-like thing that can manipulate matter. If you can't take actions based on your conscious experience, it implies the observer is dissociated from the brain and not created from it or able to interact with it.

I can definitely imagine a world where this is true, but it seems extremely unlikely based on what we currently know.

Replies from: flying-pen-and-paper
comment by Flying Pen and Paper (flying-pen-and-paper) · 2024-07-09T23:43:36.911Z · LW(p) · GW(p)

Yes, it would imply the observer is external, but then it also would not change anything about how the brain functions. (Or vice versa, but I prefer this one.) I am unconvinced of the truth of what you say in the last sentence of your second paragraph.

Either way, whether or not it might seem implausible, my question is why it is, or is not, implausible. Why exactly, based on what we currently know, is this extremely unlikely?

comment by Dyingwithdignity1 · 2024-07-08T13:40:25.922Z · LW(p) · GW(p)

Does Scenario 2 imply some kind of spooky action at a distance? How is information from Rob-z transmitted to the homonculus over large distances? Are there 2 homoncului now that communicate what they see to each other?

Doesn’t scenario 2 imply Rob-x has actually functionally died? Which would make this the scenario where you don’t care about what happens to Rob-z and y because Rob-x now experiences oblivion?

comment by HoVY · 2024-07-08T13:32:09.197Z · LW(p) · GW(p)

https://existentialcomics.com/comic/1 Related comic exploring this idea

comment by red75prime · 2024-04-19T21:35:56.322Z · LW(p) · GW(p)

What concrete fact about the physical world do you think you're missing? What are you ignorant of?

Let's flip very unfair quantum coin with 1:2^1000000 heads to tails chances (that would require quite an engineering feat to prepare such a quantum state, but it's theoretically possible). You shouldn't expect to see heads if the quantum state is prepared correctly, but the post-flip universe (in MWI) contains a branch where you see heads. So, by your logic, you should expect to see both heads and tails even if the state is prepared correctly.

What I do not know is how it all ties together. MWI is wrong? Copying is not equivalent to MWI branching (thanks to the no-cloning theorem, for example)? And so on

comment by EvenLessWrong · 2024-04-19T17:43:07.854Z · LW(p) · GW(p)

Consider the teleporter as a machine that does two things: deconstructs an input i and constructs an output o. 
If you divide the machine logically into these two functions, d and c, which are responsible for deconstructing and constructing respectively, you have four ways the machine could function or not function:

If neither d or c work, the machine doesn't do anything. 

If d works but c doesn't, the machine definitely kills or destroys the input person. 

If d doesn't work and c does, the machine makes a copy of the person. If a being walked into the machine and found that this happened, the input being would be in my opinion justified in saying that they oppose being deconstructed.

If d works and c works, then we have a functioning teleporter. This is similar to the previous situation, just with "being i" destroyed. I find it hard to believe this is preferable in some way from the perspective of the input being. 

I think there is possibly a good argument we should accept that this leads to some sort of nihilism about the value / coherence of our existence as discrete individuals, but personally, I maintain too much uncertainty to be okay with stepping into a "teleporter" type system that is more novel than going to sleep (which does after all destroy the being that goes to sleep and create a being that wakes up)

comment by Mikhail Samin (mikhail-samin) · 2024-04-18T00:12:33.869Z · LW(p) · GW(p)

But I hope the arguments I've laid out above make it clear what the right answer has to be: You should anticipate having both experiences.

Some quantum experiments allow us to mostly anticipate some outcomes and not others. Either quantum physics doesn’t work the way Eliezer thinks it works and the universe is very small to not contain many spontaneously appearing copies of your brain, or we should be pretty surprised to continually find ourselves in such an ordered universe, where we don’t start seeing white noise over and over again.

I agree that if there are two copies of the brain that perfectly simulate it, both exist; but it’s not clear to me what should I anticipate in terms of ending up somewhere. Future versions of me that have fewer copies would feel like they exist just as much as versions that have many copies/run on computers with thicker wires/more current would feel.

But finding myself in an orderly universe, where quantum random number generators produce expected frequencies of results, requires something more than the simple truth that if there’s an abstract computation being computed, well, it is computed, and if it is experiencing, it’s experiencing (independently of how many computers in which proportions using which physics simulating frameworks physically run it).

I’m pretty confused about what is needed to produce a satisfying answer, conditional on a large enough universe, and the only potential explanation I came up with after thinking for ~15 minutes (before reading this post) was pretty circular and not satisfying (I’m not sure of a valid-feeling way that would allow me to consider something in my brain entangled with how true this answer is, without already relying on it).

(“What’s up with all the Boltzmann brain versions of me? Do they start seeing white noise, starting from every single moment? Why am I experiencing this instead?”)

And in a large enough universe, deciding to run on silicon instead of proteins might be pretty bad, because maybe, if GPUs that run the brain are tiny enough, most future versions of you might end up in weird forms of quantum immortality instead of being simulated.

If I physically scale my brain size on some outputs of results of quantum dice throws but not others, do I start observing skewed frequencies of results?

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2024-04-18T09:09:45.842Z · LW(p) · GW(p)

The solution is here [AF · GW]. In a nutshell, naive MWI is wrong, not all Everett branches coexist, but a lot of Everett branches do coexist s.t. with high probability all of them display expected frequencies.

Replies from: mikhail-samin
comment by Mikhail Samin (mikhail-samin) · 2024-04-18T09:52:44.749Z · LW(p) · GW(p)

I can imagine this being the solution, but

  • this would require a pretty small universe
  • if this is not the solution, my understanding is that IBP agents wouldn’t know or care, as regardless of how likely it is that we live in naive MWI or Tegmark IV, they focus on the minimal worlds required. Sure, in these worlds, not all Everett branches coexist, and it is coherent for an agent to focus only on these worlds; but it doesn’t tell us much about how likely we’re in a small world. (I.e., if we thought atoms are ontologically basic, we could build a coherent ASI that only cared about worlds with ontologically basic atoms and only cared about things made of ontologically basic atoms. After observing the world, it would assume it’s running in a simulation of a quantum world on a computer build of ontologically basic atoms, and it would try to influence the atoms outside the simulation and wouldn’t care about our universe. Some coherent ASIs being able to think atoms are ontologically basic shouldn’t tell us anything about whether atoms are indeed ontologically basic.)

Conditional on a small universe, I would prefer the IBP explanation (or other versions of not running all of the branches and producing the Born rule). Without it, there’s clearly some sort of sampling going on.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2024-04-19T09:21:21.234Z · LW(p) · GW(p)

Not sure what you mean by "this would require a pretty small universe".

If we live in naive MWI, an IBP agent would not care for good reasons, because naive MWI is a "library of babel" where essentially every conceivable thing happens no matter what you do.

Also not sure what you mean by "some sort of sampling". AFAICT, quantum IBP is the closest thing to a coherent answer that we have, by a significant margin.

Replies from: Signer, mikhail-samin, quetzal_rainbow
comment by Signer · 2024-04-19T13:57:20.076Z · LW(p) · GW(p)

If we live in naive MWI, an IBP agent would not care for good reasons, because naive MWI is a “library of babel” where essentially every conceivable thing happens no matter what you do.

Isn't the frequency of amplitude-patterns changes depending on what you do? So an agent can care about that instead of point-states.

comment by Mikhail Samin (mikhail-samin) · 2024-04-19T11:05:03.912Z · LW(p) · GW(p)

I mean if the universe is big enough for every conceivable thing to happen, then we should notice that we find ourselves in a surprisingly structured environment and need to assume some sort of an effect where if a cognitive architecture opens its eyes, it opens its eyes in a different places with the likelihood corresponding to how common these places are (e.g., among all Turing machines).

I.e., if your brain is uploaded, and you see a door in front of you, and when you open it, 10 identical computers start running a copy of you each: 9 show you a green room, 1 shows you a red room, you expect that if you enter a room and open your eyes, in 9/10 cases you’ll find yourself in a green room.

So if it is the situation we’re in- everything happens- then I think a more natural way to rescue our values would be to care about what cognitive algorithms usually experience, when they open their eyes/other senses. Do they suffer or do they find all sorts of meaningful beauty in their experiences? I don’t think we should stop caring about suffering just because it happens anyway, if we can still have an impact on how common it is.

If we live in a naive MWI, an IBP agent doesn’t care for good reasons internal to it (somewhat similar to how if we’re in our world, an agent that cares only about ontologically basic atoms doesn’t care about our world, for good reasons internal to it), but I think conditional on a naive MWI, humanity’s CEV is different from what IBP agents can natively care about.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2024-04-20T13:05:04.297Z · LW(p) · GW(p)

Your reasoning is invalid, because in order to talk about updating your beliefs in this context, you need a metaphysical framework which knows how to deal with anthropic probabilities (e.g. it should be able to answer puzzles in the vein of the anthropic trilemma [LW · GW] according to some coherent, well-defined mathematical rules). IBP is such a framework, but you haven't proposed any alternative, not to mention an argument for why that alternative is superior.

comment by quetzal_rainbow · 2024-04-19T10:21:54.086Z · LW(p) · GW(p)

I always thought that in naive MWI what matters is not whether something happens in absolute sense, but what Born measure is concentrated on branches that contain good things instead of bad things.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2024-04-20T12:59:54.107Z · LW(p) · GW(p)

The problem is this requires introducing a special decision-theory postulate that you're supposed to care about the Born measure for some reason, even though Born measure doesn't correspond to ordinary probability.

Replies from: TAG
comment by TAG · 2024-04-20T15:51:50.398Z · LW(p) · GW(p)

Huh? The whole point of the Born rule is to get a set of ordinary probabilities, which you can then test frequentistically, over a run of experiments. Quantum mechanical measure-- amplitude-- isn't ordinary probability, but that's the thing you put into the Born rule, not the thing you get out of it. And it has it's own role, which is explaining how much contribution to a coherent superposition each component state makes.

ETA

There is a further problem interpreting the probabilities of fully decohered branches. (Calling then Everett branches is very misleading -- a clear theory of decoherence is precisely what's lacking in Everett's work)

Whether you are supposed to care about them ethically is very unclear, since it is not clear how utilitarian style ethics would apply, even if you could make sense of the probabilities. But you are not supposed to care about them for the purposes of doing science, since they can no longer make any difference to your branch. MWI works like a collapse theory in practice.

always thought that in naive MWI what matters is not whether something happens in absolute sense, but what Born measure is concentrated on branches that contain good things instead of bad things.

It's tempting to ethically discount low measure decoherent branches in some way, because that most closely approximates conventional single world utilitarianism -- that is something "naive MWI" might mean. However, one should not jump to the conclusion that something is true just because it is convenient. And of course, MWI is a scientific theory so it doesn't comes with built in ethics.

The alternative view starts with the question of whether a person low measure world still count as a full.person? If they should not, is that because they are a near-zombie, with a faint consciousness that weighs little in a hedonic utilitarian calculus? If they are not such zombies, why would they not count as a full person -- the standard utilitarian argument that people in far-off lands are still moral patients seems to apply. Of course, MWI doesn't directly answer the question about consciousness.

(For example, if I toss a quantum fair coin n times, there will be 2^n branches with all possible outcomes.)

If "naive MWI" means the idea that any elementary interaction produces decoherent branching, then it is wrong for the reasons I explain here [LW(p) · GW(p)]. Since there are some coherent superpositions, and not just decoherent branches, there are cases where the Born rule gives you ordinary probabilities, as any undergraduate physics student knows.

(What is the meaning of the probability measure over the branches if all branches coexist?)

It's not the existence, it's the lack of interaction/interference.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2024-04-21T11:36:23.582Z · LW(p) · GW(p)

The topic of this thread is: In naive MWI, it is postulated that all Everett branches coexist. (For example, if I toss a quantum fair coin  times, there will be  branches with all possible outcomes.) Under this assumption, it's not clear in what sense the Born rule is true. (What is the meaning of the probability measure over the branches if all branches coexist?)

comment by RussellThor · 2024-04-17T21:31:08.887Z · LW(p) · GW(p)

But it could matter if its digital vs continuous.  <OK longer post and some thoughts a bit off topic perhaps>

Your A,B,C,D ... leads to some questions about what is conscious (C) and what isn't. 

Where exactly does the system stop being conscious

1. Biological mind with neurons

2. Very high fidelity render in silicon with neurons modelled down to chemistry rather than just firing pulses

3. Classic neural net spiking approx done in discrete maths that appears almost indistinguishable to 1,2. Producing system states A,B,C,D

4. same as (3) but states are saved/retrieved in memory not calculated.

5. States retrieved from memory many times  - A,B,C,D ... A,B,C,D ... does this count as 1 or many experiences?

6. States retrieved in mixed order A,D,C,B....

7 States D,D,D,D,A,A,A,A,B,B,B,B,C,C,C,C .. does this count 4* or nothing.

A possible cutoff is between 3/4. Retrieving instead of calculating makes it non-conscious.  But what about caching, some calc, some retrieved? 

As you prob know this has been gone over before, e.g. Scott Aaronson. Wonder what your position is?

https://scottaaronson.blog/?p=1951
with quote:

"Maybe my favorite thought experiment along these lines was invented by my former student Andy Drucker.  In the past five years, there’s been a revolution in theoretical cryptography, around something called Fully Homomorphic Encryption (FHE), which was first discovered by Craig Gentry.  What FHE lets you do is to perform arbitrary computations on encrypted data, without ever decrypting the data at any point.  So, to someone with the decryption key, you could be proving theorems, simulating planetary motions, etc.  But to someone without the key, it looks for all the world like you’re just shuffling random strings and producing other random strings as output.

You can probably see where this is going.  What if we homomorphically encrypted a simulation of your brain?  And what if we hid the only copy of the decryption key, let’s say in another galaxy?  Would this computation—which looks to anyone in our galaxy like a reshuffling of gobbledygook—be silently producing your consciousness?"

and last but not least:

"But, in addition to performing complex computations, or passing the Turing Test, or other information-theoretic conditions that I don’t know (and don’t claim to know), there’s at least one crucial further thing that a chunk of matter has to do before we should consider it conscious.  Namely, it has to participate fully in the Arrow of Time. "

https://www.scottaaronson.com/papers/giqtm3.pdf
 

comment by Ruby · 2024-07-08T05:11:34.084Z · LW(p) · GW(p)

Curated. I like "think clearly about confusing philosophical" topics and this post is a very well-written explainer. I think it's likely both correct and ought to be convincing. At the same time, I think it's somewhat incomplete and that a fuller version would make the case for why we should wholly believe that experience is determined by local brain state rather than treat it as an assumption. Beyond that, I think that the most convincing explainer on this topic would build on a fully satisfactory theory of consciousness that totally answers the hard problem and makes none of it feel mysterious at all. Still, glad to see this on LessWrong, taking us kind of bag to our roots.

Replies from: andrei-alexandru-parfeni, andrei-alexandru-parfeni
comment by sunwillrise (andrei-alexandru-parfeni) · 2024-07-08T13:51:31.434Z · LW(p) · GW(p)

I think it's likely both correct and ought to be convincing

I find this rather difficult to believe in light of andesoldes's excellent distillation [LW(p) · GW(p)] of Rob's position and subsequent detailed and concrete explanation [LW(p) · GW(p)] of why it seems wrong to have this degree of confidence in his beliefs.

As TAG has written [LW(p) · GW(p)] a number of times, the computationalist thesis seems not to have been convincingly (or even concretely) argued for in any LessWrong post or sequence (including Eliezer's Sequences). What has been argued for, over and over again, is physicalism, and then more and more rejections of dualist conceptions of souls. 

That's perfectly fine, but "souls don't exist and thus consciousness and identity must function on top of a physical substrate" is very different from "the identity of a being is given by the abstract classical computation performed by a particular (and reified) subset of the brain's electronic circuit," and the latter has never been given compelling explanations or evidence. [1] This is despite the fact that the particular conclusions that have become part of the ethos of LW about stuff like brain emulation, cryonics etc are necessarily reliant on the latter, not the former. 

As a general matter, accepting physicalism as correct would naturally lead one to the conclusion that what runs on top of the physical substrate works on the basis of... what is physically there (which, to the best of our current understanding, can be represented through Quantum Mechanical probability amplitudes [LW · GW]), not what conclusions you draw from a mathematical model that abstracts away quantum randomness in favor of a classical picture, the entire brain structure in favor of (a slightly augmented version of) its connectome, and the entire chemical make-up of it in favor of its electrical connections. As I have mentioned, that is a mere model that represents a very lossy compression of what is going on; it is not the same [LW · GW] as the real thing, and conflating the two is an error that has been going on here for far too long. Of course, it very well might be the case [LW · GW] that Rob and the computationalists are right about these issues, but the explanation up to now should make it clear why it is on them to provide evidence for their conclusion.

As I have written before [LW(p) · GW(p)] about these matters:

More specifically, is a real-world being actually the same as the abstract computation its mind embodies [LW · GW]? Rejections of souls and dualism, alongside arguments for physicalism, do not prove [LW(p) · GW(p)] the computationalist thesis to be correct, as physicalism-without-computationalism is not only possible but also (as the very name implies) a priori far more faithful to the standard physicalist worldview.

[...]

The feedback loops implicit in the structure of the brain [? · GW] cause reward and punishment signals to "release chemicals that induce the brain to rearrange itself" [LW(p) · GW(p)] in a manner closely analogous to and clearly reminiscent of a continuous and (until death) never-ending micro-scale brain surgery. To be sure, barring serious brain trauma, these are typically small-scale changes, but they nevertheless fundamentally modify the connections in the brain and thus the computation it would produce in something like an emulated [? · GW] state (as a straightforward corollary, how would an em that does not "update" its brain chemistry the same way that a biological being does be "human" in any decision-relevant way?). We can think about a continuous personal identity through the lens of mutual information about memories, personalities etc [LW(p) · GW(p)], but our current understanding of these topics is vastly incomplete and inadequate, and in any case the naive (yet very widespread, even on LW) interpretation of "the utility function is not up for grabs" [LW · GW] as meaning that terminal values [LW · GW] cannot be changed (or even make sense as a coherent concept) seems totally wrong.

[...]

The way communities make progress on philosophical matters is by assuming that certain answers are correct [LW(p) · GW(p)] and then moving on. After all, you can't ever get to the higher levels that require a solid foundation if you aren't allowed to build such a foundation in the first place. But I worry, for reasons that have been stated before [LW(p) · GW(p)], that the vast majority of the discourse by "lay lesswrongers" [LW(p) · GW(p)] (and, frankly, even far more experienced members of the community working directly on alignment research; as a sample illustration, see a foundational report [LW · GW]'s failure to internalize the lesson of "Reward is not the optimization target" [LW · GW]) is based on conclusions reached through informal and non-rigorous intuitions [LW(p) · GW(p)] that lack the feedback loops necessary to ground themselves to reality [LW · GW] because they do not do enough "homework problems" [LW(p) · GW(p)] to dispel misconceptions and lingering confusions about complex and counterintuitive matters.

  1. ^

    It is hard to prove a negative, but I make this claim on the basis of having read ~ the entirety of what has been written on this site about personal identity and consciousness.

Replies from: Ruby
comment by Ruby · 2024-07-09T00:30:13.306Z · LW(p) · GW(p)

I differ from Rob in that I do think his piece should have flagged the assumption of ~computationalism, but think the assumption is reasonable enough to not have argued for in this piece.

I do think it is interesting philosophical discussion to hash it out, for the sake of rigor and really pushing for clarity. I'm sad that I don't think I could dive in deep on the topic right now.

To answer your question in your other comment. I reckon with some time I could write an explainer for why we should very reasonable assume consciousness is the result of local brain stuff and nothing else (and also not quantum stuff), though I'd be surprised if I could easily write something so rigorous that you'd find it fully satisfactory.

comment by sunwillrise (andrei-alexandru-parfeni) · 2024-07-08T14:49:47.065Z · LW(p) · GW(p)

a fuller version would make the case for why we should wholly believe that experience is determined by local brain state rather than treat it as an assumption

Relatedly to my other comment [LW(p) · GW(p)], I'm curious if you (Ruby) think you would be capable of writing such a version. I'm obviously not asking you to actually write it (you probably have much better things to do with your time), but I do wonder what the answer to this question is, and if it is "no", then I would also want to ask whether you nonetheless think you have good reasons to believe that Rob's conclusion is correct (in light of the counterarguments and reasons for skepticism that have been brought up, and which seem to me like they would necessitate that exact "fuller version" to resolve).