Ethics of Brain Emulation

I'm a human being with less empathy, and I'm ready to protect my preferences, so be careful ;)

Just to clarify, I'm talking about the "I don't care about the suffering of other humans" / sociopathy sort of no-empathy, not the "I have trouble interpreting facial expressions" / autism sort of no-empathy. It's unfortunate that we use the same word for those. Some psychologists use "Hot empathy (feeling)" and "Cold empathy (perception)" to differentiate.

And, since I can't look at your brain directly, I wouldn't actually feel reduced altruism towards you unless you actually did an action which demonstrated callous disregard for other people (self-diagnoses of sociopathy are insufficient evidence that someone actually doesn't care about other people).

However, if you really are one who doesn't experience hot empathy, then you aren't really allowed to be offended by the fact that I feel reduced hot empathy towards you, because that's just tit for tat. ;)

Although, I don't actually care about Hot Empathy either . What I care about are your preferences - do you care about others as a (non-instrumental) value? Hot Empathy is where most humans derive their altruistic preferences from, but if you derive altruistic preferences via some other route then that works for me.

Are you saying that nonempathetic human beings can't suffer? I find that claim bizarre.

Bleh...yeah. It is bizarre. How about we don't call it "suffering" , and just focus on "bad thing that we want to avoid." for now.

It seems like humans typically only extend altruism towards things which reciprocate altruism in return. Why are humans more bothered by the suffering of dogs than they are by the suffering of pigs, though the two animals are of comparable intellect? Other than mere familiarity, it's because the former reciprocates altruism. It's harder to slaughter something that shows affection towards you.

If you just look at the behavioural instead of the experiental aspect of suffering, this already eliminates anything that could be normally understood by the word.

I have a dream, that one day agents will be judged not by the substrate of their code, but by the behavioral output of whatever algorithm they run.

First of all, not doing so violates the anti-zombie principle, and second of all, if we interact with aliens or AI, I want us to be friends, and I want AI we design to consider them as friends too. So...if you want to define "suffering" to be referring to specific algorithms, I'm comfortable with that...but this discussion really isn't about suffering, is it? It's about morality. And morality shouldn't care what sort of substrate your algorithm runs on, nor should it really care what specific algorithm you use except with regard to its output. (Though I can think of some fun edge cases here if you want to talk about that)

Like the paper said,

we do not (yet) have a good understanding of what suffering is (or rather, what the actual underlying component that is morally relevant, is)

I'm more trying to get at what is morally relevant about suffering, not defining suffering itself. Language is filled with fuzzy categories that dissolve under the application of rigor.

Taboo person.

Okay, so in this context, being classed as a Person means that in addition to intelligence, the following thing is approximately true:

"I care about the preferences of all agents X who have this statement embedded in their algorithm".

(Yes, this is uncomfortable for me too. I haven't worked out a non-self-referential version.)

So, if snakes don't even help each other, they aren't people at all save for their tiny little spark of intelligence. If bees help each other sometimes, but never other species, they aren't people at all because the altruism is only directed towards the colony.

When mice altruistically free other rodents from cages, it is a spark personhood...but it's limited because the mouse will only do this for species who display affective cues which it can understand. It doesn't have the cognitive capacity to understand emotions in the abstract and apply that knowledge to, say, altruism towards a human or bird. So its not very person-ish...but we certainly wouldn't torture it.

Dolphins, dogs, elephants, apes, etc...show cross species altruism and a high degree of intelligence. They are very person-ish. We should be really nice to them, in proportion to how person-ish they are.

A Paperclipper is sort of like a hyper-intelligent bee or snake. We don't really care how it feels. A Friendly SAI, on the other hand, is even more person-ish than a human. We'd never want to violate the preferences of a Friendly SAI. (this is rather tautological, of course)

Part of the problem is that, in order to explain my idea, I took certain words and re-defined them away from their common usage to suite my purposes. I don't know how to say this using the words we have now though. And the other problem is that it's sloppy. I haven't thought through this nearly enough.

But the general direction feels both intuitively comfy when I apply it to animals (that's the closest thing to a truly alien mind that we have right now) and comes with the bonus of being somewhat pragmatic with its tit-for-tat attitude towards what sorts of beings we should be friendly towards.

You didn't answer my question. My point was that pain is simpler than suffering, and even scientists who study it can't objectively define it.

First, the "objective" part of pain is known as nociception and can likely be studied in real or simulated organisms. The subjective part of pain need not be figured out separately from other qualia, like perception of color red.

Second, not all pain is suffering and not all suffering is pain, so figuring out the quale of suffering is separate from studying pain.

Are you suggesting we shouldn't even talk about their potential suffering then?

I think we have to work on formalizing qualia in general before we can make progress in understanding "computational suffering" specifically.

it might be too late start philosophizing about it then

I find philosophizing without the goal of separating a solvable chunk of a problem at hand a futile undertaking and a waste of time. The linked paper does a poor job identifying solvable problems.