Posts

Comments

Comment by Tyrrell_McAllister3 on The Super Happy People (3/8) · 2009-02-01T14:32:27.000Z · LW · GW
"We found unacceptable the alternative of leaving the Babyeaters be. We found unacceptable the alternative of exterminating them. We wish to respect their choices and their nature as a species, but their children, who do not share that choice, are unwilling victims; this is unacceptable to us. We desire to keep the children alive but we do not know what to do with them once they become adult and start wanting to eat their own babies. Those were all the alternatives we had gotten as far as generating, at the very moment your ship arrived."

Akon forgot to mention the possibility of trying to genetically modify the Babyeaters.

"Humankind, we possess a generalized faculty to feel what others feel. That is the simple, compact relation. We did not think to complicate that faculty to exclude pain. We did not then assign dense probability that other sentient species would traverse the stars, and be encountered by us, and yet fail to have repaired themselves. Should we encounter some future species in circumstances that do not permit its repair, we will modify our empathic faculty to exclude sympathy with pain, and substitute an urge to meliorate pain."

So the Lady 3rd says, basically, "modifying ourselves to exclude this pain would be hard, and we don't want to do it until all other options are proved harder."

Akon, then should be able to say exactly the same thing, with equal truth, regarding the human pain that the Lady 3rd wants eliminated. She is too quick in dismissing this symmetry.

Or was Akon really trying to argue that the Lady 3rd's kind keep their sympathy-pain as a terminal value? He is described as thinking hastily on his feet, so I suppose it's plausible for him to make such a silly argument. And in that case it's plausible to the Lady 3rd to dismiss this argument as she did. But it's not plausible that Akon really thinks that humans want to hold on to their pain as a terminal value. Star Trek V notwithstanding, it's hard to imagine a human society of the kind depicted here feeling that kind of attachment to pain.

So, for this scene to be believable, Akon should very quickly realize that humans are reluctant to eliminate their pain because they don't know how to do so without interfering with other values. And that should be a reason that is immediately understandable to the Lady 3rd, because she has offered essentially the same justification for not proceeding immediately to eliminate her own sympathy-pain.

Of course, she should then offer immediately to figure out how to eliminate our pain for us if we can't do it ourselves. But it shouldn't be hard for her to see why we would be reluctant to trust her ability to do that without interfering with our other values, given what we've seen of their abilities to understand us so far. Her evident bafflement at our reluctance to modify ourselves is prima facie evidence that they do not understand us well enough that we would be willing to let them muck around with our source code.

Of course, she might still conclude that they do understand us well enough to modify us, even against our wishes. But it shouldn't be surprising to her that it would be against our wishes, at least at this stage of the encounter.