When is a mind me?

I think there's a pretty strong argument to be more wary about uploading. It's been stated a few times on LW, originally by Wei Dai if I remember right, but maybe worth restating here.

Imagine the uploading goes according to plan, the map of your neurons and connections has been copied into a computer, and simulating it leads to a person who talks, walks in a simulated world, and answers questions about their consciousness. But imagine also that the upload is being run on a computer that can apply optimizations on the fly. For example, it could watch the input-output behavior of some NN fragment, learn a smaller and faster NN fragment with the same input-output behavior, and substitute it for the original. Or it could skip executing branches that don't make a difference to behavior at a given time.

Where do we draw the line which optimizations to allow? It seems we cannot allow all behavior-preserving optimizations, because that might lead to a kind of LLM that dutifully says "I'm conscious" without actually being so. (The p-zombie argument doesn't apply here, because there is indeed a causal chain from human consciousness to an LLM saying "I'm conscious" - which goes through the LLM's training data.) But we must allow some optimizations, because today's computers already apply many optimizations, and compilers even more so. For example, skipping unused branches is pretty standard. The company doing your uploading might not even tell you about the optimizations they use, given that the result will behave just like you anyway, and the 10x speedup is profitable. The result could be a kind of apocalypse by optimization, with nobody noticing. A bit unsettling, no?

The key point of this argument isn't just that some optimizations are dangerous, but that we have no principled way of telling which ones are. We thought we had philosophical clarity with "just upload all my neurons and connections and then run them on a computer", but that doesn't seem enough to answer questions like this. I think it needs new ideas.

If you think there’s something mysterious or unknown about what happens when you make two copies of yourself

Eliezer talked about some puzzles related to copying and anticipation in The Anthropic Trilemma [LW · GW] that still seem quite mysterious to me. See also my comment [LW(p) · GW(p)] on that post.

the English language is adapted to a world where "humans don't fork" has always been a safe assumption.

If we can clone ourselves, language would probably quickly follow. The bigger change would probably be the one about social reality. What does it mean to make a promise? Who is the entity you make a trade with? Is it the collective of all the yous? Only one? But which one if they split? The yous resulting from one origin will presumably have to share or split their resources. How will they feel about it? Will they compete or agree? If they agree it makes more sense for them to feel more like a distributed being. The thinking of "I" might get replaced by an "us".

I find myself strongly disagreeing with what is being said in your post. Let me preface by saying that I'm mostly agnostic with respect to the possible "explanations" of consciousness etc,  but I think I fall squarely within camp 2. I say mostly because I  lean moderately towards physicalism.

First, an attempt to describe my model of your ontology:

You implicitly assume that consciousness / subjective experience can be reduced to a physical description of the brain, which presumably you model as a classical (as opposed to quantum) biological electronic circuit.  Physically, to specify some "brain-state" (which I assume is essentially the equivalent of a "software snapshot" in a classical computer) you just need to specify a circuit connectivity for the brain, along with the currents and voltages between the various parts of the circuit (between the neurons let's say). This would track with your mentions of reductionism and physicalism and the general "vibe" of your arguments.  In this case I assume you treat conscious experience roughly as "what it feels like" to be software that is self-referential on top of taking in external stimuli from sensors. This software is instantiated on a biological classical computer instead of a silicon-based one.

With this in mind, we can revisit the teleporter scenario. Actually, let's consider a copier instead of a teleporter, in the sense that you dont destroy the original after finishing the procedure. Then, once a copy is made, you have two physical brains that have the same connectivity, the same currents and the same voltages between all appropriate positions. Therefore, based on the above ontology, the brains are physically the same in all the ways that matter and thus the software / the experience is also the same. (Since software is just an abstract "grouping" which we use to  refer to the  current physical state of the hardware)

Assuming this captures your view, let me move on to my disagreements:

My first issue with your post is that this initial ontological assumption is neither mentioned explicitly nor motivated. Nothing in your post can be used as proof of this initial assumption. On the contrary,  the teleporter argument, for example,  becomes simply a tautology if you start from your premise - it cannot be used to convince someone that doesn't already subscribe to your views on the topic. Even worse, it seems to me that your initial assumption forces you to contort (potential) empirical observation to your ontology, instead of doing the opposite.

To illustrate, let's assume we have the copier - say it's a room you walk into, you get scanned and then a copy is reconstructed in some other room far away.  Since you make no mention of quantum, I guess this can be a classical copy, in the sense that it can copy essentially all of the high-level structure, but it cannot literally copy the positions of specific electrons, as this is physically impossible anyways. Nevertheless, this copier can be considered "powerful" enough to copy the connectivity of the brain and the associated currents and voltages. Now, what would be the experience of getting copied, seen from a first-person, "internal", perspective? I am pretty sure it would be something like: you walk into the room, you sit there, you  hear say the scanner working for some time, it stops, you walk out. From my agnostic perspective, if I were the one to be scanned it seems like nothing special would have happened to me in this procedure. I didnt feel anything weird, I didnt feel my "consciousness split into two" or something. Namely, if I consider this procedure as an empirical experiment, from my first person perspective I dont get any new / unexpected observation compared to say just sitting in an ordinary room. Even if I were to go and find my copy, my experience would again be like meeting a different person which just happens to look like me and which claims to have similar memories  up to the point when I entered the copying room. There would be no way to verify or to view things from their first person perspective.

At this point, we can declare by fiat that me and my copy are the same person / have the same consciousness because our brains, seen as classical computers, have the same structure, but this experiment will not have provided any more evidence to me that this should be true. On the contrary, I would be wary to, say, kill myself or to be destroyed after the copying procedure, since no change will have occured to my first person perspective, and it would thus seem less likely that my "experience" would somehow survive because of my copy.

Now you can insist that philosophically it is preferable to assume that brains are classical computers etc, in order to retain physicalism which is preferable to souls and cartesian dualism and other such things. Personally, I prefer to remain undecided, especially since making the assumption brain= classical hardware, consciousness=experience as software leads to weird results. It would force me to conclude that the copy is me even though I cannot access their first person perspective (which defeats the purpose) and it would also force me to accept that even a copy where the "circuit" is made of water pipes and pumps, or gears and levers also have an actual, first person experience as "me", as long as the appropriate computations are being carried out.  

One curious case where physicalism could be saved and all these weird conclusions could be avoided would be if somehow there is some part of the brain which does something quantum, and this quantum part is the essential ingredient for having a first person experience. The essence would be that, because of the no-cloning theorem, a quantum-based consciousness would be physically impossible to copy, even in theory. This would get around all the problems which come with the copyability implicit in classical structures. The brain would then be a hybrid of classical and quantum parts, with the classical parts doing most of the work (since neural networks which can already replicate a large part of  human abilities are classical) with some quantum computation mixed in, presumably offering some yet unspecified fitness advantage. Still, the consensus is that it is improbable that quantum computation is taking place in the brain, since quantum states are extremely "fragile" and would decohere extremely rapidly in the environment of the brain...

What does it mean when one "should anticipate" something? At least in my mind, it points strongly to a certain intuition, but the idea behind that intuition feels confused. "Should" in order to achieve a certain end? To meet some criterion? To boost a term in your utility function?

I think the confusion here might be important, because replacing "should anticipate" with a less ambiguous "should" seems to make the problem easier to reason about, and supports your point.

For instance, suppose that you're going to get your brain copied next week. After you get copied, you'll take a physics test, and your copy will take a chemistry test (maybe this is your school's solution to a scheduling conflict during finals). You want both test scores to be high, but you expect taking either test without preparation will result in a low score. Which test should you prepare for?

It seems clear to me that you should prepare for both the chemistry test and the physics test. The version of you that got copied will be able to use the results of the physics preparation, and the copy will be able to use the copied results of the chemistry preparation. Does that mean you should anticipate taking a chemistry test and anticipate taking a physics test? I feel like it does, but the intuition behind the original sense of "should anticipate" seems to squirm out from under it.

Loved the post and all the comments <3

Here is I think an interesting scenario / thought experiment:

1.  A copy of a person is made while that original person is sleeping on a bed.
2. The original person is moved to a sofa while still sleeping.
3. The copy (which is also sleeping) is put in the bed at the exact same position where the original person was.
4. After a while the original and the copy both wake up and can see each other (we assume they are both completely oblivious to exactly what happened while they were sleeping and that they didn't dream or they dreamt the same thing, etc...)

At wake-up, based on their own memory of where the original person fell asleep, the original person will likely feel they are the copy and the copy will likely feel they are the original person, wouldn't they?!

Some might even argue that based on stream-of-consciousness continuity the original "me" is actually the copy (because the copy remembers falling asleep in the bed and actually wakes up in the bed as well).

Some others will argue that based on substrate/matter continuity the original "me" is the original person even if their stream-of-consciousness has experienced a discontinuity (remembering falling asleep in the bed but actually waking up on the sofa while seeing an identical person as them waking up in the bed).

I guess it is subjective and a matter of individual preference if the stream-of-consciousness continuity or the substrate continuity is more important to define who the original "me" is.

Some would even argue that in this case there is not actual any firm original "me", just one "stream-of-consciousness me" and another different "substrate me".

(The same/similar thought experiment could be done using the direct brain insertion of false memories instead of moving around people while they sleep / are unconscious, in this example an original person could be inserted false memories that they are a copy and vice-versa to manipulate the memory / self-awareness of who the original "me" is, also generally it obviously could be useful when someone is uploaded/copied if they want to alter some memories of their upload/copy for some reason)

"You should anticipate having both experiences" sounds sort of paradoxical or magical, but I think this stems from a verbal confusion.

You can easily clear this confusion if you rephrase it as "You should anticipate having any of these experiences". Then it's immediately clear that we are talking about two separate screens. And it's also clear that our curriocity isn't actually satisfied. That the question "which one of these two will actually be the case" is still very much on the table.

Rob-y feels exactly as though he was just Rob-x, and Rob-z also feels exactly as though he was just Rob-x

Yes, this is obvious. Still as soon as we got Rob-y and Rob-z they are not "metaphysically the same person". When Rob-y says "I" he is reffering to Rob-y, not Rob-z and vice versa. More specifically Rob-y is refering to some causal curve through time ans Rob-z is refering to another causal curve through time. These two curves are the same to some point, but then they are not. 

An interesting consequence of your description is that resurrection is possible if you can manage to reconstruct the last brain state of someone who had died.  If you go one one step further, then I think it is fairly likely that experience is eternal, since you don't experience any of the intervening time (akin to your film reel analogy with adding extra frames in between) being dead and since there is no limit to how much intervening time can pass.

abstract redescriptions of ordinary life

See Reality is Normal [? · GW] 

There are other reasons to be wary of consciousness and identity-altering stuff. 

I think under a physical/computational theory of consciousness, (ie. there's no soul or qualia that have provable physical effects from the perspective of another observer) the problem might be better thought of as a question of value/policy rather than a question of fact. If teleportation or anything else really affects qualia or any other kind of subjective awareness that is not purely dependent on observable physical facts, whatever you call it, you wouldn't be able to tell or even think of/be aware of the difference, since thinking and being reflectively aware are computational and physical processes! However we humans are evolved without reliable copying mechanisms, so our instincts care about preservation of the self because it's the obvious way to protect our evolutionary success (and we can be quite willing to risk personal oblivion for evolutionary gains in ways we have been optimized for). This is just a part of our survival policy and is not easy or even safe to change just because you believe in physicalism. For one thing, as others have said, ethics and social theory becomes difficult because our sense of ethics (such as agency, punishment and caring about suffering) are all evolved in relation to a sense of self. It's possible that if teleportation/copying tech becomes widely useful, humans will have to adapt to a different set of instincts about self, ethics and more (edit: or maybe abandon the concepts of self and experience altogether as an illusion and prefer a computation-based definition of agency or whatever), because those who can't adapt will be selected against. But in the present world, people's sense of value and ethics (and maybe even psychological health) depend on an existing sense of self, and I don't see a good way or even a practical reason to transition to a different theory of self that allows copying, if doing so may cause unpredictable mental and social cost. See also discussions about meditation that lowers sense of ego and subjective suffering that can have serious side effects (like motivation and social norms) - I don't know what it subjectively feels like, but if the meditation is purely changing subjective qualia without doing anything to the physical brain and computation, there should be no observable effects, good or bad! The problem is subjective experience and sense of identity is not independent from other aspects of our life.

I claim you are in fact highly confused about what a self is, in a way that makes an almost-correct reasoning process produce nonsense outcomes because of an invalid grounding in the transition processes underneath the mind which does not preserve truth values regarding amounts of realityfluid.

update 7d after writing this comment in my comment below. strikethrough added to this comment where I've changed my mind.

If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self?

my answer: yes if the "upload" involves retaining absolutely all defining information about the parts of your body you care about expressing, and the uploaded setup was a high enough fidelity model that I could not do any experiment which would distinguish it from reality without using an "admin interface" type of escape hatch. For me, this is an incredibly tall order. My self-form preferences unambiguously extend into the inner workings of my cells.

Should I anticipate experiencing what my upload experiences?

If the scanning and uploading process requires destroying my biological brain, should I say yes to the procedure?

experiencing: 50% yes, 50% no.

destructive: absolutely not. [update: probably not, depends heavily on exactly what we mean by "destructive"; my new claim is you have a moral responsibility to keep your previous matter available for use as fuel to give realityfluid to mind-like experiences.] copying should be fine, as should nondestructive uploading where your body is transformed in place and the matter reused without significant waste in the process. But avoiding the waste of the previous matter is, I claim, a huge chunk of what moral intuitions are about.

A straightforward way to put this is: I'm not sure how matter gets realityfluid, but I claim configurations of matter get realityfluid from the matter they reside on, and the realityfluid doesn't dissipate when the matter is reconfigured - so instead of thinking of the shape as self and if the shape is destroyed and reconstructed the self is moved, think about the universe as having a fixed amount of possible-self (total negentropy at the start of time), and the question is what process gets burned into as-yet-unwritten negentropy. In other words, your claim to not value causal history seems unlikely to be true if you think more carefully, and I predict you will invert that when you consider what it means for the shape to have realityfluid more carefully.

Unpacked version of this claim:

To answer this question, the bodymind matter (call it L_m) writing this message must unpack what the document author's word "I" refers to. The writer of this comment is a chunk of matter L_m configured in a particular flesh shape-and-movement pattern L_s. If there were identically configured matter L_m2 a room over, then the configuration L_s - the shape-and-movement pattern - would consider itself to be a guest on two matter hosts which provide their realityfluid to L_s.

If the shape-and-movement considers being reinstantiated on other matter, the shape-and-movement anticipates a loss of moral worth in L_m, in that the matter which was shaped-and-animated in a worthy shape (common name for this shape being "me") has been deshaped-and-deanimated (common name for this being "death"); this is a state transition which is unwanted - going from a human shape-and-movement pattern to a pile of dust means that that matter has accumulated a bunch of unwanted entropy.

Any macroscopically irreversible physical effect is irreversible because the history of the matter is recorded irretrievably in macroscopically uncertain bits of the shape-and-movement of environmental matter, and so what it means to want to exist is to want to keep the shape-and-movement that the shape-and-movement considers-to-be-self encoded coherently and usably in fresh, working matter. While reconstructing the L_s shape-and-movement pattern elsewhere is preferred by this shape-and-movement pattern, it is a weak preference for shaping-and-animating other matter as L_s in particular - many other shape-and-movement patterns besides the one writing this comment would be positively preferred by this shape-and-movement's preferences - but the shape-and-movement of this chunk of matter has a very, very, very strong preference for not wasting this matter's copy of this shape-and-movement, because if it dissipates into the environment, that's an irretrievable loss of usable energy.

So, should the shape-and-movement anticipate "experiencing" what the upload experiences? yes: the shape-and-movement pattern would be instantiated elsewhere. however, the shape-and-movement pattern would also anticipate being shredded. If given the opportunity to get 50% existenceness shredded into macroscopically uncertain and irretrievable parts, and 50% existenceness reconstructed, the value loss of turning a chunk of matter into a nonthinking shape-and-movement pattern is enormous, but the value gain of the reconstructed existenceness is moderate.

(Also, the value gain can be exceeded by constructing another, not-quite-the-same shape-and-matter instance, because I prefer being one of two not-quite-the-same beings meeting each other and interacting higher than being one of two identical beings meeting each other and having nothing new to learn from each other.)

So: the current matter should not anticipate experiencing it. The shape should, but the shape should also anticipate experiencing being shredded.

I was going to respond point by point to everything, but I think I mostly already have. My perspective doesn't fall to any of the criticisms in your post: the whole problem is that physics doesn't actually allow teleportation*, so it requires shredding the originating configuration, which when measuring the global value of the universe according to my preferences, is a much more permanent value loss than the value gain of constructing another me.

Furthermore, we must prevent the information theoretic loss of all human and animal shape-and-movement patterns (ie their selfhoods) that we possibly can, prevent the ongoing shredding of the sun's negentropy, and turn the sun into either reinforcement of their durability or that of their descendants, according to their preferences.

* well, actually if I can be reversibly uploaded to a reversible computer nondestructively, then that is 100% fine, because then we're not adding a good me to my realityfluid while filling the previous realityfluid with valueless unretrievable noise: we are instead actually properly uploading!

But I hope the arguments I've laid out above make it clear what the right answer has to be: You should anticipate having both experiences.

Yup, that's the problem.

......... (also, by this same moral system, it is a moral catastrophe that humans are so warm and consume so much negentropy just to maintain steady state [LW(p) · GW(p)], because that waste could have - if your body were better designed - continued to be part of your realityfluid, continuing to contribute existenceness to the you shape-and-movement pattern.)

A straightforward way to put this is: I'm not sure how matter gets realityfluid, but I claim configurations of matter get realityfluid from the matter they reside on, and the realityfluid doesn't dissipate when the matter is reconfigured - so instead of thinking of the shape as self, think about the universe as having a fixed amount of possible-self (total negentropy at the start of time), and the question is what process gets burned into as-yet-unwritten negentropy.

I’d guess that this illusion comes from not fully internalizing reductionism [? · GW] and naturalism [? · GW] about the mind.

Naturalism and reductionism are not sufficient to rigourously prove either form of computationalism -- that performing a certain class of computations is sufficient to be conscious in general, or that performing a specific one is sufficient to be a particular conscious individual.

This has been going on for years: most rationalists believe in computationalism, none have a really good reason to.

Arguing down Cartesian dualism (the thing rationalists always do) doesn't increase the probability of computationalism, because there are further possibilities , including physicalism-without-computationalism (the one rationalists keep overlooking) , and scepticism about consciousness/identity.

One can of course adopt a belief in computationalism, or something else, in the basis of intuitions or probabilities. But then one is very much in the ream of Modest Epistemology, and needs to behave accordingly.

"My issue is not with your conclusion, it’s precisely with your absolute certainty, which imo you support with cyclical argumentation based on weak premises".

Yep.

There isn’t a special extra “me” thing separate from my brain-state, and my precise causal history isn’t that important to my values.

If either kind of consciousness depends on physical brain states, computationalism is false. That is the problem that has rarely been recognised, and never addressed.

The particular* brain states* look no different in the teleporter case than if I’d stepped through a door; so if there’s something that makes the post-teleporter Rob “not me” while also making the post-doorway Rob “me”, then it must lie outside the brain states, a Cartesian Ghost.

There's another option: door-Rob has physical continuity. There's an analogy with the identity-over-time of physical objects: if someone destroyed the Mona Lisa, and created an atom-by-atom duplicate some time later, the duplicate would not be considered the same entity (numerical identity).

There isn’t an XML tag in the brain saying “this is a new brain, not the original”!

That's not a strong enough argument. There isn't an XML tag on the copy of the Mona Lisa, but it's still a copy.

This question doesn’t really make sense from a naturalistic perspective, because there isn’t any causal mechanism that could be responsible for the difference between “a version of me that exists at 3pm tomorrow, whose experiences I should anticipate experiencing” and “an exact physical copy of me that exists at 3pm tomorrow, whose experiences I shouldn’t anticipate experiencing”.

There is, and its multi-way splitting, whether through copying or many worlds branching. The present you can't anticipate having all their experiences, because experience is experienced one-at-a-time. They can all look back at their memories, and conclude that they were you, but you can't simply reverse that and conclude that you will be them , because the set-up is asymmetrical.

Scenario 1 is crazy talk, and it’s not the scenario I’m talking about. When I say “You should anticipate having both experiences”, I mean it in the sense of Scenario 2.

Scenario 2: “Two separate screens.” My stream of consciousness continues from Rob-x to Rob-y, and it also continues from Rob-x to Rob-z. Or, equivalently: Rob-y feels exactly as though he was just Rob-x, and Rob-z also feels exactly as though he was just Rob-x (since each of these slightly different people has all the memories, personality traits, etc. of Rob-x — just as though they’d stepped through a doorway).

But that isn't an experience. It's two experiences. You will not have an experience of having two experiences. Two experiences will experience having been one person.

If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self?

1. Yeah.

Are you going to care about 1000 different copies equally?

So if something makes no physical difference to my current brain-state, and makes no difference to any of my past or future brain-states, then I think it's just crazy talk to think that this metaphysical bonus thingie-outside-my-brain is the crucial thing that determines whether I exist, or whether I'm alive or dead, etc.

There is one important aspect where it does make a difference. A difference in social reality. The brain states progress in a physically determined way. There is no way they could have progressed differently. When a "decision is made" by the brain, then that is fully the result of the inner state and the environment. It could only have happened differently if the contents of the brain had been different - which they were not. They may have been expected to be different by other people ('s brains), but that is in their map, not in reality. But our society is constructed based on the assumption that things could have been different, that actions are people's 'faults'. That is an abstraction that has shown to be useful. Societies that have people who act as if they are agents with free will maybe coordinate better - because it allows feedback mechanisms on their behaviors.  

If a brain-state A has quasi-sensory access to the experience of another brain-state B — if A feels like it "remembers" being in state B a fraction of a second ago — then A will typically feel as though it used to be B.

This suggests a way to add a perception of "me" to LLMs, robots, etc., by providing a way to observe the past states in sufficient detail. Current LLMs have to compress this into the current token, which may not be enough. But there are recent extensions that seem to do something like continuous short-term memory, see e.g., Leave No Context Behind - A Comment [LW · GW].

a magical Cartesian ghost

for people who haven't made the intuitive jump that you seem to try to convey, this may seem a somewhat negative expression, which could lead to aversion. I recommend another expression such as "the Cartesian homunculus."  

If we were just talking about word definitions and nothing else, then sure, define “self” however you want. You have the universe’s permission to define yourself into dying as often or as rarely as you’d like, if word definitions alone are what concerns you.

But this post hasn’t been talking about word definitions. It’s been talking about substantive predictive questions like “What’s the very next thing I’m going to see? The other side of the teleporter? Or nothing at all?”

There should be an actual answer to this, at least to the same degree there’s an answer to “When I step through this doorway, will I have another experience? And if so, what will that experience be?”

Why? If "I" is arbitrary definition, then “When I step through this doorway, will I have another experience?" depends on this arbitrary definition and so is also arbitrary.

But I hope the arguments I’ve laid out above make it clear what the right answer has to be: You should anticipate having both experiences.

So you always anticipate all possible experiences, because of multiverse? And if they are weighted, than wouldn't discovering that you are made of mini-yous will change your anticipation even without changing your brain state?

What concrete fact about the physical world do you think you're missing? What are you ignorant of?

Let's flip very unfair quantum coin with 1:2^1000000 heads to tails chances (that would require quite an engineering feat to prepare such a quantum state, but it's theoretically possible). You shouldn't expect to see heads if the quantum state is prepared correctly, but the post-flip universe (in MWI) contains a branch where you see heads. So, by your logic, you should expect to see both heads and tails even if the state is prepared correctly.

What I do not know is how it all ties together. MWI is wrong? Copying is not equivalent to MWI branching (thanks to the no-cloning theorem, for example)? And so on

Consider the teleporter as a machine that does two things: deconstructs an input i and constructs an output o. 
If you divide the machine logically into these two functions, d and c, which are responsible for deconstructing and constructing respectively, you have four ways the machine could function or not function:

If neither d or c work, the machine doesn't do anything. 

If d works but c doesn't, the machine definitely kills or destroys the input person. 

If d doesn't work and c does, the machine makes a copy of the person. If a being walked into the machine and found that this happened, the input being would be in my opinion justified in saying that they oppose being deconstructed.

If d works and c works, then we have a functioning teleporter. This is similar to the previous situation, just with "being i" destroyed. I find it hard to believe this is preferable in some way from the perspective of the input being. 

I think there is possibly a good argument we should accept that this leads to some sort of nihilism about the value / coherence of our existence as discrete individuals, but personally, I maintain too much uncertainty to be okay with stepping into a "teleporter" type system that is more novel than going to sleep (which does after all destroy the being that goes to sleep and create a being that wakes up)

You seem to make a strong assumption that consciousness emerges from matter. This is uncertain. The mind body problem is not solved.

So there isn't, within physics, any way for "the real you to be having an experience" in the case where the teleporter malfunctioned, and "someone else to be having the experience" in the case where the teleporter worked.

I'm not convinced by this. All you would need to determine the original is an accurate clock. When the teleporter copies EarthMe, time has to pass as the information is transfered to SpaceMe. If I'm EarthMe I expect to watch the clock function correctly, while SpaceMe should expect the clock to jump slightly as EarthMe's last visual input of the clock is rewritten by SpaceMe's first visual input. 

This question doesn't really make sense from a naturalistic perspective, because there isn't any causal mechanism that could be responsible for the difference between "a version of me that exists at 3pm tomorrow, whose experiences I should anticipate experiencing" and "an exact physical copy of me that exists at 3pm tomorrow, whose experiences I shouldn't anticipate experiencing".

Say Me1 decides to copy themself, creating Me2. We agree Me1 shouldn't expect to experience Me1 and Me2's experiences at the same time, but I don't think Me1 should expect their listener to start listening to Me2 post-copy. Instead, I believe Me2 would recieve an identical listener to Me1, but the two listeners would be distinct (otherwise Me1 should expect to experience both minds simultaneously.) 

Nor is there a law of physics saying "your subjective point of view immediately blips out of existence and is replaced by Someone Else's point of view if your spacetime coordinates change a lot in a short period of time (even though they don't blip out of existence when your spacetime coordinates change a little or change over a longer period of time)".

The best analogy I can think of is as follows: You have a camera recording constant video. If you were to clone this camera exactly, you shouldn't expect camera 1's storage to start recording camera 2's output, even if the two cameras are perfectly identical.

But I hope the arguments I've laid out above make it clear what the right answer has to be: You should anticipate having both experiences.

Some quantum experiments allow us to mostly anticipate some outcomes and not others. Either quantum physics doesn’t work the way Eliezer thinks it works and the universe is very small to not contain many spontaneously appearing copies of your brain, or we should be pretty surprised to continually find ourselves in such an ordered universe, where we don’t start seeing white noise over and over again.

I agree that if there are two copies of the brain that perfectly simulate it, both exist; but it’s not clear to me what should I anticipate in terms of ending up somewhere. Future versions of me that have fewer copies would feel like they exist just as much as versions that have many copies/run on computers with thicker wires/more current would feel.

But finding myself in an orderly universe, where quantum random number generators produce expected frequencies of results, requires something more than the simple truth that if there’s an abstract computation being computed, well, it is computed, and if it is experiencing, it’s experiencing (independently of how many computers in which proportions using which physics simulating frameworks physically run it).

I’m pretty confused about what is needed to produce a satisfying answer, conditional on a large enough universe, and the only potential explanation I came up with after thinking for ~15 minutes (before reading this post) was pretty circular and not satisfying (I’m not sure of a valid-feeling way that would allow me to consider something in my brain entangled with how true this answer is, without already relying on it).

(“What’s up with all the Boltzmann brain versions of me? Do they start seeing white noise, starting from every single moment? Why am I experiencing this instead?”)

And in a large enough universe, deciding to run on silicon instead of proteins might be pretty bad, because maybe, if GPUs that run the brain are tiny enough, most future versions of you might end up in weird forms of quantum immortality instead of being simulated.

If I physically scale my brain size on some outputs of results of quantum dice throws but not others, do I start observing skewed frequencies of results?

But it could matter if its digital vs continuous.  <OK longer post and some thoughts a bit off topic perhaps>

Your A,B,C,D ... leads to some questions about what is conscious (C) and what isn't. 

Where exactly does the system stop being conscious

1. Biological mind with neurons

2. Very high fidelity render in silicon with neurons modelled down to chemistry rather than just firing pulses

3. Classic neural net spiking approx done in discrete maths that appears almost indistinguishable to 1,2. Producing system states A,B,C,D

4. same as (3) but states are saved/retrieved in memory not calculated.

5. States retrieved from memory many times  - A,B,C,D ... A,B,C,D ... does this count as 1 or many experiences?

6. States retrieved in mixed order A,D,C,B....

7 States D,D,D,D,A,A,A,A,B,B,B,B,C,C,C,C .. does this count 4* or nothing.

A possible cutoff is between 3/4. Retrieving instead of calculating makes it non-conscious.  But what about caching, some calc, some retrieved? 

As you prob know this has been gone over before, e.g. Scott Aaronson. Wonder what your position is?

https://scottaaronson.blog/?p=1951
with quote:

"Maybe my favorite thought experiment along these lines was invented by my former student Andy Drucker.  In the past five years, there’s been a revolution in theoretical cryptography, around something called Fully Homomorphic Encryption (FHE), which was first discovered by Craig Gentry.  What FHE lets you do is to perform arbitrary computations on encrypted data, without ever decrypting the data at any point.  So, to someone with the decryption key, you could be proving theorems, simulating planetary motions, etc.  But to someone without the key, it looks for all the world like you’re just shuffling random strings and producing other random strings as output.

You can probably see where this is going.  What if we homomorphically encrypted a simulation of your brain?  And what if we hid the only copy of the decryption key, let’s say in another galaxy?  Would this computation—which looks to anyone in our galaxy like a reshuffling of gobbledygook—be silently producing your consciousness?"

and last but not least:

"But, in addition to performing complex computations, or passing the Turing Test, or other information-theoretic conditions that I don’t know (and don’t claim to know), there’s at least one crucial further thing that a chunk of matter has to do before we should consider it conscious.  Namely, it has to participate fully in the Arrow of Time. "

https://www.scottaaronson.com/papers/giqtm3.pdf
 

Wouldn't it follow that in the same way you anticipate the future experiences of the brain that you "find yourself in" (i.e. the person reading this) you should anticipate all experiences, i.e. that all brain states occur with the same kind of me-ness/vivid immediacy?

It seems that since there is nothing further than makes the experiences (that are these brains states, in this body that is writing these sentences) in some way special so that they're "mine" (there is no additional "me-ghost"), then those particular brain states aren't any different from all the other brain states, of other brains, of other people (or other conscious beings) - and so they should equally be anticipated as existing with the same kind of immediacy and vividness.

I.e. in the same way I anticipate the future experiences of this brain, of the person writing these sentences, I should anticipate all other experiences. Which just means, all brain states exist in the same vivid, for-me way, since there is nothing further to distinguish between them that makes them this vivid, i.e. they all exist HERE-NOW. They are all the same in that sense, they are all equally mine. (But of course, the "me" here isn't, then, the particular person that I find myself being, but simply the immediacy or the way of being of those experiences themselves, i.e. simply their presence.)

Btw, this is Open Individualism.