Waterfall Ethics

To the OP: By the quoted statement, do you mean that the human being so represented feels pain as the simulation is being conducted, or merely as a result of the simulation's initial state existing?

To Vaniver: I'm certainly of the opinion that actually simulating a human (or any other person) under the given circumstances would cause said human to (1) actually exist and (2) actually experience pain. Of course, by "we" you might really mean "you" or some other subset of the site's population.

I'm not anywhere near so certain about the "initial state" situation, primarily because I don't seem to have a coherent model of what it fundamentally means to "exist" yet.

Thanks for the link! It is also my intuition that universes exist in possibility-space, not in computation-space.

If a turing machine simulation is torturing some poor guy, then he is already being tortured in a possiblity-space, whether we run the simulation or not, and regardless of how we run it.

Simulating a universe is simply a window that allows us to see what happens in that universe. It is not a cause of that universe. Running 3^^^3 simulations of torture is simply looking 3^^^3 times at a universe whose inhabitant is (always was, always we be; the possibility-space is timeless) tortured, as a by-product of the laws of that universe. Seeing it may hurt our feelings, but it does not provide any additional harm to the person being simulated. Also, running 3^^^3 simulations with modified parameters is looking into 3^^^3 different universes.

If I repeat hundred times that "2+2=4", it does not make "2+2=4" more real in any sense.

While I accept computationalism only for intentionality (i.e. semantics, knowledge, etc.) and not qualia, I don't see why computationalists of all stripes shouldn't insist that a computation must actually run. EY's Timeless Causality post looks relevant, offering a redefinition of running a computation in terms of causal relations, rather than, necessarily, requiring extension in time.

Why privilege the physical movement of the seashells? What if I move the seashells into position for timestep 35469391 and then mentally imagine the position of the seashells at timestep 35469392? You could say I am "performing the calculation," but you could also say I am "discovering the result of propagating forward the initial conditions."

I don't think our intuitions about what "really happens" are useful. I think we have to zoom out at least one level and realize that our moral and ethical intuitions only mean anything within our particular instantiation of our causal framework. We can't be morally responsible for the notional space of computable torture simulations because they exist whether or not we "carry them out." But perhaps we are morally responsible for particular instantiations of those algorithms.

I don't know the answer - but I don't think the answer is that performing mechanical operations with seashells reifies torture but writing down the algorithm does not.

Wow, that is one nice Bostrom paper. Has its critique of Chalmers's argument been discussed before on LW?

Do you mean to say that Aaronson's paper is bad, or that everyone's already read it?

Computationalists think that consciousness is like addition rather than elasticity.

If the simulation is running fast enough, the simulated human can give you a very authentic-sounding description of their pain.

Does the program being executed by the Turing machine experience pain? Does the question even have an answer?

The Turing machine doing the simulating does not experience pain, but the human being being simulated does.

Similarly, the waterfall argument found in the linked paper seems as though it could as-easily be used to argue that none of the humans in the solar system have intelligence unless there's an external observer to impose meaning on the neural patterns.

A lone mathematical equation is meaningless without a mind able to read it and understand what its squiggles can represent, but functioning neural patterns which respond to available stimuli causally(/through reliable cause-and-effect) are the same whether emboided in cell weights or in tape states. (So, unless one wishes to ignore one's own subjective consciousness and declare oneself a zombie...)

For the actual-versus-potential question, I am doubtful regarding the answer, but for the moment I imagine a group of people in a closed system (say, an experiment room), suddenly (non-lethally) frozen in ice by a scientist overseeing the experiment. If the scientist were to later unfreeze the room, then perhaps certain things would definitely happen if the system remained closed. However, if it were never unfrozen, then they would never happen. Also, if they were frozen yet the scientist decided to interfere in the experiment and make the system no longer a closed system, then different things would happen. As with the timestream in normal life, 'pain' (etc.) is only said to take place at the moment that it is actually carried out. (And if one had all states laid out simultaneously, like a 4D person looking at all events in one glance from past to present, then 'pain' would only be relevant for the one point/section that one could point to in which it was being carried out, rather than in the entire thing.)

Now though, the question of the pain undergone by the models in the predicting scientist's mind (perhaps using his/her/its own pain-feeling systems for maximum simulation accuracy) by contrast... hmm.