Suffering

Reading the comments here, there seem to be two issues entangled. One is which organisms are capable of suffering (which is probably roughly the same set that is capable of experiencing qualia; we might call this the set of sentient beings). The other is which entities we would care about and perhaps try to help.

I don't think the second question is really relevant here. It is not the issue Tiiba is trying to raise. If you're a selfish bastard, or a saintly altruist, fine. That doesn't matter. What matters is what constitutes a sentient being which can experience suffering and similar sensations.

Let us try to devote our attention to this question, and not the issue of what our personal policies are towards helping other people.

Why is this tagged "serasvictoriawouldyoumarryme"?

Anything to do with this fictional character?

For me, a morally significant agent is one that has positive and negative qualia. Since I can't tell this by looking at an agent, I guess. I'm pretty sure that my brother does, and I'm pretty sure that a rock doesn't.

Cyan mentioned pain asymbolia on the last thread: the ability to feel pain, but not really find anything painful or problematic about it. If someone had asymbolia generalized across all mental functions, I would stop counting that person as a moral agent.

If a "morally significant agent" is one whose goals I would want to further, then I am the only inherently morally significant agent. The moral significance of other agents shifts with my mood.

We talk a lot here about creating Artificial Intelligence. What I think Tiiba is asking about is how we might create Artificial Consciousness, or Artificial Sentience. Could there be a being which is conscious and which can suffer and have other experiences, but which is not intelligent? Contrariwise, could there be a being which is intelligent and a great problem solver, able to act as a Bayesian agent very effectively and achieve goals, but which is not conscious, not sentient, has no qualia, cannot be said to suffer? Are these two properties, intelligence and consciousness, independent or intrinsically linked?

Acknowledging the limited value of introspection, nevertheless I can remember times which I was close to experiencing "pure consciousness", with no conscious problem-solving activity at all. Perhaps I was entranced by a beautiful sunset, or haunting musical performance. My whole being seemed to be pure experience, pure consciousness, with no particular need for intelligence, Bayesian optimization, goal satisfaction, or any of the other paraphernalia which we associate with intelligence. This suggests to me that it is at least plausible that consciousness does not require intelligence.

In the other direction, the idea of an intelligence problem solver devoid of consciousness is an element in many powerful, fictional dystopias. Even Eliezer's paperclip maximizer partakes of this trope. It seems that we have little difficulty imagining intelligence without consciousness, without awareness, sentience, qualia, the ability to suffer.

If we provisionally assume that the two qualities are independent, it raises the question of how we might program consciousness (even if we only want to know how, to avoid doing it accidentally). is it possible that even relatively simple programs may be conscious, may be capable of feeling real pain and suffering, as well as pleasure and joy? Is there any kind of research program that could shed light on these questions?

serasvictoriawouldyoumarryme?

The capacity to abide by morality carves out the right cluster in thingspace for me, though I'd hesitate to call it the determining factor. If a thing has this capacity, we care about its preferences proportionately.

People, save probably infants, are fully capable, in theory, of understanding and abiding by morality. Most animals are not. Those more capable of doing so, domestic pets, beasts of burden for example, receive some protection. Those who do not have this capacity are generally less protected and that which is done to them less morally relevant.

I don't fully endorse this view, but it feels like it explians a lot.

The following was originally going to be a top-level post, but I never posted it because I couldn't complete the proof of my assertion.

In his recent book I Am a Strange Loop, Douglas Hofstadter writes:

A spectacular evolutionary gulf opened up at some point as human beings were gradually separating from other primates: their category systems became arbitrarily extensible. Into our mental lives there entered a dramatic quality of open-endedness, an essentially unlimited extensibility, as compared with a very palpable limitedness in other species. Concepts in the brains of humans acquired the property that they could get rolled together with other concepts into larger packets, any such larger packet could then become a new concept in its own right. In other words, concepts could nest inside each other hierarchically, and such nesting could go on to arbitrary degrees. This reminds me—and I do not think it is pure coincidence—of the huge difference, in video feedback, between an infinite corridor and a truncated one.

In other words, Hofstadter sees a phase transition, a discontinuity, a binary division between the mental processes of humans and other species. Yet curiously, when he discusses the moral consideration we ought to give to various species, he advocates a continuum approach based on something like "capacity for friendship", thereby privileging species with K-strategies and/or pack-hunting tendencies for no very good reason that I can see.

To me, the implication of Hofstadter's phase transition is obvious: beings with arbitrary category systems get moral consideration; those with bounded systems do not. By "moral consideration", incidentally, I don't mean some sort of Kantian treating-as-ends-not-means (oh wait, Kant is irrelevant); rather I mean that when you're making some nice utilitarian calculation, you must consider the feelings/opinions of all humans involved, but should not factor in the preferences of (say) a dog.

This is not to say that animal cruelty for the hell of it is a good idea (though I think it should be legal). Many of us anthropomorphize animals, especially pets, to a huge extent, and doing "evil" to animals could easily lead to actual evil. On the other hand, if you're deciding between torturing a human or a googol kittens, go for the kittens.

I will help a suffering thing if it benefits me to help it, or if the social contract requires me to. Otherwise I will walk away.

I adopted this cruel position after going through one long relationship where I constantly demanded emotional "help" from the girl, then another relationship soon afterwards where the girl constantly demanded similar "help" from me. Both those situations felt so sick that I finally understood: participating in any guilt-trip scenario makes you a worse person, no matter whether you're tripping or being tripped. And it also makes the world worse off: being openly vulnerable to guilt-tripping encourages more guilt-tripping all around.

So relax and follow your own utility - this will incentivize others to incentivize you to help them, so everyone will treat you well, and you'll treat them well in advance for the same reason.

Interesting question....

Could there be suffering in anything not considered an MSA? While I can imagine a hypothetical MSA that could not suffer, it's hard to think of a being that suffers yet could not be considered an MSA.

But do we have a good operational definition of 'suffering'? The study with the fish is a start, but is planning really a good criterion?

The discussion reminds of that story On being a bat (iirc) in Hofstadter/Dennets highly recommended The Mind's I, on the impossibility of understanding at all what it is like to be something so different from us.

prase probably describes naive judgement of moral significance correctly - I see no reason to expect a simple answer to the question. I shall perhaps comment later, having had time to consider more deeply.

Most people do this intuitionally, and most people keep to make rationalisations of their intuitive judgements or construct neat logical moral theories in order to support them (and these theories usually fail to describe what they are intended to describe, because of their simplicity relative to the complexity of an average man's value system).

That said, for me an agent is the more morally significant the more is it similar to human, and I determine suffering by comparison with my own experiences and some necessary extrapolation. Not much useful answer perhaps, but I don't know of any better.

BTW, I think the most important defining feature of an MSA is ability to kick people's asses. Very humanizing.

I don't know if you meant that as a joke, but that's pretty much my take from a contractarian perspective (though I wouldn't use the phrase "morally significant agent"). Fish can't do much about us cooking and eating them, so they are not a party to any social contract. That's also the logic behind my tolerance of infanticide.

The potential to enhance the information complexity of another agent. Where the degree of this potential and the degree of the complexity provided indicates the degree of moral significance.

Which reduces the problem to the somewhat less difficult one of estimating complexity and so estimating potential complexity influences among agents. By this, I means something more nuanced than algorithmic or Kolmogorov complexity. We need something that takes into account fun theory and how both simple systems and random noise are innately less complex than systems with non-trivial structure and dynamics or to put it another way, systems that interest and enrich.

Also note, don't make the error of equating the presence of complexity for the potential to enhance complexity in other agents.

As for suffering, in this context, you can define suffering as the inverse of complexity enhancement, namely the sapping of the innate complexity of the agent.