Coercion is an adaptation to scarcity; trust is an adaptation to abundance

post by Richard_Ngo (ricraz) · 2023-05-23T18:14:19.117Z · LW · GW · 11 comments

Contents

11 comments

There’s a sense in which I’m postulating a trillion dollar bill lying on the ground. If the skills I’ve talked about so far in this sequence would lead to so much more flourishing, why haven’t they become far more common? I’ve already given a partial answer: that our evolutionary environment was much more dangerous than our current environment. But I want to extend this to a more general answer: that coercion is an adaptation to scarcity; and that we only very recently left the era in which scarcity was the dominant feature of people’s lives.

Under conditions of scarcity, you don’t have enough slack [LW · GW] that you can afford to take risks. If misbehavior from any individual in the group would risk the lives of many others, you have to coerce them into staying in line; if a loss on any gamble would leave you ruined, then you need to avoid taking those gambles even when they’re winning in expectation. (Poker players think a lot about this when trying to manage their bankroll—if a game has high-enough stakes that they’d be out of money if they lost, they have to avoid it, even if they expect to make money there on average.) 

By contrast, in an abundant environment, you can take the optimal long-term strategy, even if there’s a risk that it’ll leave you way down in the short term. In particular, you can put effort into building trust with others, even though that leaves you more vulnerable to being let down or betrayed. With that trust, you can receive a huge range of gains from cooperation.^[1]

Western societies are incredibly abundant in many ways. As a citizen, you face almost zero risk of starvation, dying in a war, or exile from your country; meanwhile deaths from most diseases and accidents are dramatically lower than in the past. There’s more career flexibility than there ever has been before: there are many routes to success, including self-employment. And society is far richer than it’s ever been before: the median person in a western society is incredibly wealthy both by historical standards and by the standards of most people across the world. Even welfare recipients are still well off by historical and global standards.

What does it look like to internalize a feeling of abundance? It might involve not forcing yourself to study when you don’t feel like it; or being less defensive when people criticize you; or deciding to be less harsh on yourself about the possibility that you might fail. The more slack you have, the more of a risk you can take by doing this—in each case it’s a gamble, but one which more often than not pays off in the long run by reducing internal conflict and allowing you to act more coherently towards your goals. Similarly, excitement about the future is a gamble: if you get excited about something which doesn’t work out, you’ll be left in a worse position than if your expectations had stayed low the whole time. The way that rich people can get higher returns to investment because they can absorb more risk is analogous to how fear forces us to take suboptimal actions.

What if you think, like I do, that we live at the hinge of history [EA · GW], and our actions could have major effects on the far future—and in particular that there’s a significant possibility of existential risk from AGI? I agree that this puts us in more of a position of scarcity and danger than we otherwise would be (although I disagree with those who have very high credence in catastrophe). But the more complex the problems we face, the more counterproductive scarcity mindset is. In particular, AGI safety requires creative paradigm-shifting research, and large-scale coordination; those are both hard to achieve from a scarcity mindset. In other words, coercion at a psychological or community level has strongly diminishing marginal returns when dealing with scarcity at a civilizational level.

Here’s Malcolm Ocean:

I recount how in 2019, I heard a podcast where Esther Perel says to a client about his partner who has PTSD flashbacks, “you can tell him ‘you’re safe now’” and I found myself thinking “that’s not okay. I can’t feel that I’m safe in this moment. AI could eat the world, and I’m not doing enough about it. I can’t feel safe until we’ve figured it out.”

I share how I’d realized that this is an utter confusion. that “not feeling safe right now”, as an embodied sense made of cortisol and adrenaline and muscle tension that arises, is for running away from tigers or towards enemy soldiers, not for tackling complex paradigm-transcending technical problems or political problems like AI alignment.

Similar reasoning applies to other big problems too. For example, another major source of scarcity is crime: criminals can still hurt us or kill us no matter how wealthy we are in other ways. But in practice, I think we should usually apply far more of an abundance mindset than we usually do. The character of the Bishop in Les Miserables is perhaps the platonic ideal of extending trust to criminals—responding to Vajean’s theft by giving him even more valuables (as pictured below), which is the crucial act that leads Valjean to redemption. This may seem bizarrely naive, but our world is now wealthy enough that we can often afford to carry out comparably trusting strategies, and reap the rewards. For example, the Scandinavian approach of designing comfortable prisons designed for rehabilitation works much better than using them as tools of punishment, which just leads to cycles of crime.^[2]

Another major type of scarcity that affects most people’s lives is social scarcity. But while it’s painful to lose the respect of your friends, the world is much bigger than it used to be. It’s much easier to find a social circle similar to yourself than it used to be; and it’s much easier to discover a new friendship group if your old one rejects you, or even move to a new city or country if you burn ties in your previous scene. Yet our intuitions are still calibrated for an environment where exile means death—hence all the problems I've described in the sequence thus far.

Having said all that, there are still cases where scarcity mindset is appropriate. For example, we easily get addicted to games or drugs, so we have to sometimes apply self-coercion to avoid getting stuck in those traps; our lives are too short for us to be able to explore as much as some parts of us would like; we’re still vulnerable to accidents as well as physical violence; and for many, the costs of housing and university are prohibitive (although the growing spread of anti-credentialist Silicon Valley culture means there’s more abundance with respect to the last than many think).

This post was the last in the section on trust; in the next four posts (which I'll upload in a few days), I'll focus on how to cultivate excitement-based motivation.

1. ^{^}

   IFS may be a useful frame on our minds in significant part because we're in a world of abundance. In scarce environments, it might be best to think of your mind as a hierarchy in which order and discipline are sternly imposed from the top down. In abundant environments like ours, though, we can gain more from thinking of our minds as egalitarian "families".

2. ^{^}

    For example, the Scared Straight program (which tried to scare kids away from becoming criminals) actually led to increased crime rates. Having said that, I think problems can arise when there’s a mismatch between abundance in one area and scarcity in another. For example, it’s often good to avoid prosecuting shoplifters, and to focus on more preventative and rehabilitative approaches. But that only works if there’s actually effort being put into prevention and rehabilitation; and if society is rich enough and high-trust enough overall that it can easily absorb the costs of increased shoplifting for however long it takes the other approaches to pay off. That’s not always the case.

11 comments

Comments sorted by top scores.