A Thought Experiment on Pain as a Moral Disvalue

I don't really see what's supposed to be so terribly interesting about pain.

As far as I can see "pain" is just the name of a particular mental signal that happens to have the qualities of causing you to strongly desire the absence of that signal and that strongly calls your attention (consider the rather common phrase "X want(s) Y so badly it hurts" and the various non-body damage related sensations that are described as sorts of pain or feel similar to pain). It's easy to see why pain evolved to have those qualities and accounting for effects of what is usually seen as default they seem sufficient to explain pain's moral status.

For example, is it more moral to subject someone to music they strongly desire not to hear than to non-damaging physical pain, if the desire for absence and inability to avert attention is the same for both? Is the imperative to prevent the pain of a child stronger than the one to give the child the last piece of a healthy treat you have been sharing, but you personally don't care about if the wants are equal? Is there a moral imperative to "convert" masochists to take pleasure in some other way?

My intuition is that the particulars of the mental signal are not relevant and that with a few qualifiers the ethics of pain can be reduced to the ethics of wants.

The outward signs of that mental signal are even further removed from relevance, so the thought experiment reduces to the moral status of the wants of the entities in question. Among humans you can treat wants as more or less directly comparable, but if you assign moral status to those entities in the first place you probably need some sort of normalization. I think unless the entities were deliberately designed for this particular experiment (in which case you probably ignore the particulars of their wants out of UDT/TDT considerations) their normalized wants for the reward signal would be a lot weaker than a humans want for the absence of overwhelmingly strong pain. So I'd keep the $100 the second time.

fully defining pain as a moral disvalue is at least as hard as the problem of consciousness

This looks correct to me.

Hmm. I was unable to distinguish between this person and a real person, and one way to perfectly simulate a human being (given the apparently endless resource these simulators have) would be to create something thats very close to a human being. So I'd be inclined not to repeat the experiment. The benefit if I'm right is $100, if I'm wrong I cause massive amounts of pain. Of course saying that means that this machine can start mugging me by forcing me to pay money to prevent the pain, which leads me into a question about kidnapping, basically: I should pre-commit to not press the button to encourage AI not to torture humans/simulcrum humans who I can't distinguish from.

I know this doesn't get to the point of your question in any way, but I'm throwing it in:

I would refuse to play again, even if the whole thing was pure fiction, because it causes me pain to see others in pain, even if it's fake pain. It's the same reason I didn't go to see the torture porn movies that were so popular a few years back, despite knowing the actors never experienced any actual pain.

Unfortunately it also implies that fully defining pain as a moral disvalue is at least as hard as the problem of consciousness, so this line of investigation seems to be at an immediate impasse, at least for the moment.

This is entirely intuitive for me. I strongly expect moral value to have a lot to do with conscious minds. I see no reason to expect full definitions before we have a much more complete understanding of the mind.

To the people that say that some of these systems are conscious:

1 . An AIXI-like (e.g., AIXI-tl, Monte Carlo AIXI, or some such) agent, programmed with the objective of maximizing the number of button presses.

This one is probably conscious, because it seems like the simplest way to simulate conscious-like behaviour is through consciousness. On the one hand, a specific mind contains significantly more information than a description of the behaviour, but on the other hand much of this information may not matter so it could be generated by a simple pseudorandom number generator.

2 . A brute force optimizer, which was programmed with a model of your mind, that iterated through all possible audio/video bit streams to find the one that maximizing the number of button presses. (As far as philosophical implications are concerned, this seems essentially identical to 1, so the reader doesn't necessarily have to go learn about AIXI.)

These run through all possible outputs. It's like passing the Turing test by outputting entirely random text to an astronomical number of testers. I might be concerned about the many simulations of myself, but the optimizer itself is not thinking.

4 . A Giant Lookup Table (GLUT) of all possible sense inputs and motor outputs of a person, connected to a virtual body and room.

The GLUT has been precomputed, so the conscious experience has already taken place (not that conscious experiences necessarily have to occur at specific times; answering that question seems to require a more advanced understanding of consciousness). Rereading outputs that have already been written down does not cause more conscious experience any more than rewatching a video of someone that has already been taken.

Practically, there seems to be a very harsh penalty if I am wrong about any of this, so I might refuse to participate anyway, depending on the cost to me.

All heuristics have unusual contexts that require using other heuristics to do well in. This makes reliable implementation of any given heuristic FAI-complete, you need a whole mind to look over potential problems with any of its parts.

All of these are analogous to the AI-Box experiment; I expect I'd just "lose" again, so I'd decline to play the second time. (I'm not sure it would even have to do anything fundamentally different the second time. I accept the intuition pump here, but it's hard to imagine my epistemic state after finding out I had been so totally fooled.)

I'd say most likely most of those DO imply real pain, and probably vastly greater amounts than merely a human being tortured for 10 min. Now, it is obviously possible to fool a human to think pain is happening when it is not, but I'm not so sure that is not just due to humans being stupid and use sensory inefficiently.

For such a short time it might even be possible to make a perfect illusion like this, depending on bandwidth, but if this experiment was a year instead of 10 min and the observer was a superintelegence, I'd not be surprised to see a mathematical proof such an illusion can not be created without using actual pain on a concious observer. I wouldn't be surprised if it were possible either. I would be very surprised if a human could come up with such a safe method.

I don't see how consciousness is involved at all here, nor why pain should be treated as a special kind of disutility. Whether a rational agent is constructed of meat or GLUTs is irrelevant. To the extent that it can and does try to diminish our disutility, it is our moral and practical duty to try to diminish its disutility.

I would remove the word 'strong' because at what point a negative reinforcement would become 'strong' is in my mind arbitrary. I see pain intensity as a continuum reaching from insignificant to very intense, so I don't think we need an arbitrary notion of 'strong.'

The actual scenario is full of distractions, but I'll try to ignore them (1).

The thing is, I think the pain in this scenario is a distraction as well. The relevant property in this sort of scenario, which drives my impulse to prevent it and causes me to experience guilt if I don't, is my inference of suffering (2).

So the question becomes, how do I characterize the nature of suffering?

Which is perhaps a mere semantic substitution, but it certainly doesn't feel that way from the inside. I can feel pain without suffering, and suffering without pain, which strongly suggests that there are two different things under discussion, even if I don't clearly understand either of them.

I'll probably play a second round against the GLUT, since if there is any suffering involved there it has already happened and I might as well get some benefit from it. (3)

The others, I am less certain about.

Thinking about it more, I lean towards saying that my intuitions about guilt and shame and moral obligation to reduce suffering are all kind of worthless in this scenario, and I do better to frame the question differently.

For example, given #3, perhaps the right question is not "are those uploads experiencing suffering I ought to alleviate" but rather "ought I cooperate with those uploads, or ought I defect?"

Not that that helps: I'm still left with the question of how to calibrate their cost/benefit equation against my own, which is to say of how significant a term their utility is in my utility function. And sure, I can dodge the question by saying I need more data to be certain, but one can fairly ask what data I'd want... which is really the same question we started with, though stated in a more general way.

So... dunno. I'm spinning my wheels, here.

==

(1) For example, I suspect that my actual response in that scenario is either to keep all the money, under the expectation that what I'm seeing is an actor or otherwise something not experiencing pain (in a non-exotic way, as in the Milgram experiments), or to immediately quit the experiment and leave the room, under the expectation that to do anything else is to reinforce the sadistic monster running this "experiment."

I cannot imagine why I'd ever actually press the button.

But of course that's beside the point here.

(2) That is, if the person wired up to the chair informs me that yes, they are experiencing pain, but it's no big deal, then I don't feel the same impulse to spend $100 to prevent it.

Conversely, if the neurologists monitoring the person's condition assures me credibly that there's no pain, but they are intensely suffering for some other reason, I feel the same impulse to prevent it. E.g., if they will be separated from their family, whom they love, unless I return the $100, I will feel the same impulse to spend $100 to prevent it.

The pain is neither necessary nor sufficient for my reaction.

Note that I'm expressing all this in terms of what I perceive and what that impels me to do, rather than in terms of the moral superiority of one condition over another, because I have a clearer understanding of what I'm talking about with the former. I don't mean to suggest that the latter doesn't exist, nor that the two are equivalent, nor that they aren't. I'm just not talking about the latter yet.

(3) I say "probably" because there are acausal decision issues that arise here that might make me decide otherwise, but I think those issues are also beside your point.

Also, incidentally, if there is any suffering involved, the creation of the GLUT was an act of monstrous cruelty on a scale I can't begin to conceive.

Why stop at pain? Imagine a similar experiment: you pay to cause the participant pleasure, or avoid extreme boredom or psychological distress. Similar logic seems to these as well.

The sensation of pain "in and of itself" has no moral disvalue. What matters is "damage". Usually things which cause pain also go some way towards causing injury, and injury prevents an agent from achieving the things it could otherwise have done. Pain is also 'coercive' in the sense that it stops a person in their tracks (whatever they were trying to do, now all they can do is moan and writhe). Effectively it's a form of physical restraint.

Then there are a whole bunch of second-order effects. E.g. hurting people is bad because it creates a climate where disputes are resolved by physical violence.

Perhaps these considerations don't help us to distinguish between "actual pain" and "the appearance of pain", but we don't need to - what's important is distinguishing "the appearance of pain when it has moral disvalue" and "the appearance of pain when it doesn't".