Relevant reading: gwern's The Narrowing Circle. He makes the important point that moral circles have actually narrowed in various ways, and also that it never feels that way because the things outside the circle don't seem to matter anymore. Two straightforward examples are gods and our dead ancestors.

There are a few ideas that seem obvious to me, but which seem, perplexingly, to elude (or simply not to have occurred to) many folks I consider quite intelligent. One of these ideas is:

We can widen our circle of concern. But there’s no reason we must do so; and, indeed, there is—by definition!—no moral obligation to do it. (In fact, it may even be morally blameworthy to do so, in much the same way that it is morally blameworthy to take a pill that turns you into a murdering psychopath, if you currently believe that murder is immoral.)

It would be good[1], I think, for many in the rationalist community to substantially narrow their circles of moral concern (or, to be more precise, to shift from a simple “circle” of concern to a more complex model based on concentric circles / gradients).

[1] Here take “good” to mean something like “in accord with personal extrapolated volition”.

The moral circle is not ever expanding, and I consider that a good thing.

A very wide moral circle is actually very costly to a person. Not only can it cause a lot of stress to think of the suffering of beings in the far future or nonhuman animals in farming or in the wild, but it also requires a lot of self-sacrifice to actually live up to this expanded circle.

In addition, it can put you at odds with other well-meaning people who care about the same beings, but in a different way. For example, when I still cared about future generations, I mostly cared about them in terms of preventing their nonconsensual suffering and victimization. However, the common far-future altruism narrative is that we ought to make sure they exist, not that they be prevented from suffering or being victimized without their consent. This is cause for conflict, as exemplified by the -25 karma points or so I gathered on the Effective Altruism Forum for it at the time.

Since then, my moral circle has contracted massively, and I consider this to be a huge improvement. It now contains only me and the people who have made choices that benefit me (or at least benefit me more than they harm me). There is also a circle of negative concern now, containing all the people who have harmed me more than they benefit me. I count their harm as a positive now.

My basic mental heuristic is, how much did a being net-benefit or net-harm me through deliberate choices and intent, how much did I already reciprocate in harming or benefitting them, and how cheap or expensive is it for me to harm or benefit them further on the margin? These questions get integrated into an intuitive heuristic that shifts my indifference curves for everyday choices.

The psychological motivation for this contracted circle is based on the simple truth that the utility of others is not my utility, and the self-awareness that I have an intrinsic desire for reciprocity.

There is yet another cost to a wide circle of moral concern, and that is the discrepancy with people who have a smaller circle. If you're my compatriot or family member or fellow present human being, and you have a small circle of concern, I can expect you to allocate more of your agency to my benefit. If you have a wide circle of concern that includes all kinds of entities who can't reciprocate, I benefit less from having you as an ally.

When people have a wide circle of concern and advocate for its widening as a norm, this makes me nervous because it implies huge additional costs forced on me, through coercive means like taxation or regulations, or simply by spreading benevolence onto a large number of non-reciprocators instead of me and the people who've benefitted me. That actually makes me worse off, and people who make me worse off are more likely to receive negative reciprocity rather than positive reciprocity.

I love human rights because they're a wonderful coordination instrument that makes us all better off, but I now see animal rights as a huge memetic mistake. Similarly, there is little reason to care about far-future generations whose existence is never going to overlap with any of us in terms of reciprocity, and yet we're surrounded by memes that require we pay massive costs to their wellbeing.

Moralists who advocate this often use moralistic language to justify it. This gives them high social status and it serves as an excuse to impose costs on people who don't intrinsically care, like me. If I reciprocate this harm against them, I am instantly a villain who deserves to be shunned for being a villain. This dynamic has made me understand the weird paradoxical finding that some people punish what ostensibly seems to be prosocial behavior. Moralism can really harm us, and the moralists should be forced to compensate us for this harm.

(Epistemic status, plausible conjecture)

I suspect the reason the moral circle seems to be expanding, is that we actually have a moral cone. Put someone in a situation where its the life of a close friend vs a distant stranger, and they will care about the close friend more. As the worlds standards of living rise, many people find that all their friends are already fine, so spare effort goes into helping the strangers. What we are seeing is our weaker preferences becoming more relevant as our strongest ones are satisfied.

Actually, in some social circles, you seem to draw a lot less criticism with an unusually wide circle than an unusually narrow one. This could be because if you only care about humans and they also care about chimps then they will still help humans nearly as much as you do. If you care about chimps and they don't, they are likely to harm many chimps for little gain.

Moral circle widening groups together two processes that I think mostly shouldn't be grouped together:

1. Changing one's values so the same kind of phenomenon becomes equally important regardless of whom it happens in (e.g. suffering in a human who lives far away)

2. Changing one's values so more different phenomena become important (e.g. suffering in a squid brain)

Maybe if you do it right, #2 reduces to #1, but I don't think that should be assumed.

I think it's an error to think of this as a circle or binary classification. It's not "care or don't", it's "how much will I sacrifice". It's more of an electron orbital than a well-defined space, with probability being roughly analogous to caring intensity.

Also, our unstoppable imagination is a problem rather than an aide here. We often care about fictional characters more than many real people, and we often care about our preferred/projected desires in others more than their actual desires.

So far all references seem to be to 'one circle' as opposed to multiple circles which you are a member of.