A sketch of acausal trade in practice

post by Richard_Ngo (ricraz) · 2024-02-04T00:32:54.622Z · LW · GW · 4 comments

Contents

  Background
  Reasoning about psychohistory
  Reasoning about trades
  Reasoning about values
  Fine-grained MSR
  Explaining MSR
None
4 comments

One implication of functional decision theory is that it might be possible to coordinate across causally-disconnected regions of the universe. This possibility has been variously known as acausal trade [? · GW], multiverse-wide cooperation via superrationality (MSR), or evidential cooperation across large worlds (ECL). I’ll use the MSR abbreviation (since I don’t want to assume that the cooperation needs to be motivated by evidential decision theory).

I broadly believe that the assumptions behind MSR make sense. But I think the standard presentations of it don’t give very good intuitions for how it might actually work in practice. In this post I present an alternative framing of MSR centered around the concept of “psychohistory”.

Background

Assume that humanity colonizes the universe at a large scale (e.g. controlling many galaxies), and also discovers that we live in some kind of much-larger multiverse (e.g. any of Tegmark's four types of multiverse).

Assume that humanity becomes a highly coordinated entity capable of making large-scale commitments, and also converges towards believing that some version of superrationality or functional decision theory is normatively correct.

Reasoning about psychohistory

In order to make accurate predictions about other causally-disconnected civilizations, we would need to develop a scientific understanding of the dynamics of civilization development, and in particular the values and governance structures that other civilizations are likely to end up with. Call this science “psychohistory”.

Why should we think that psychohistory is even possible? If we build intergalactic civilizations of astronomical complexity, then they might become more difficult to understand in proportion to our increased collective intelligence. But in many domains that we study, there are a few core principles which provide a great deal of insight (e.g. atoms, DNA, relativity). Surprisingly, this also occurs in domains which involve complex interactions between many different agents (e.g. evolution, economics). Economics is still a long way from being able to reliably predict novel macro-level phenomena, but we can at least imagine how a deep understanding of economics might allow us to do so. (For example, Malthus was very prescient in predicting “general gluts”, aka depressions.) The relationship between microeconomics and macroeconomics provides one intuition about the relationship between psychology and psychohistory.

Also, while it may be very difficult to study complex structure at the largest scale of civilization, we may be able to get a very long way by studying complex structure at lower levels, especially if future civilization is very modular. For example, it seems plausible that, once we colonize multiple galaxies, the events in one galaxy will be affected very little by what happens in another galaxy. It might even be the case that, once we’ve colonized multiple solar systems, the events in each of them are relatively disconnected from each other. Extreme modularity arises in classic AI risk scenarios, in which the values that determine the future arise in the context of a single training process, and are then copied across the entirety of the lightcone.

Another key problem is lack of data. Psychohistorical theories would initially only be based on our own trajectory. However, theories could be tested by running simulations. And if we ever encounter aliens, they could provide a wealth of data for testing our theories. Over time, we should expect psychohistorical theories to become much more sophisticated. Early theories might focus on civilizations that started from human-like species; later theories could include civilizations that started from non-human-like species on earth-like planets; later still, civilizations that emerged on very different planets, but in universes with very similar physics; later still, civilizations that emerged under any laws of physics.

Reasoning about trades

Assume that we have a good picture of the distribution of other civilizations throughout the multiverse. There are too many possible civilizations to reason about them on a case-by-case basis; instead, we’d map out high-level clusters. Civilizations might be clustered in terms of their size, their governmental structures, their cultures, etc. For the purposes of MSR, though, the most important factors to cluster them on are the values they care about; and their ability and willingness to carry out MSR.

The former seems like it could be inferred from a sophisticated understanding e.g. evolutionary, psychological, and cultural dynamics. The latter intuitively seems a bit more complicated, since it relies on the high-level decisions made by the civilization. But rather than trying to predict fine-grained details of how different civilizations would carry out MSR, we could try to cluster them at a higher level. For example, perhaps there are only a small number of multi-agent bargaining solutions that have good theoretical properties (e.g. the Nash bargaining solution, the Kalai–Smorodinsky bargaining solution, etc). If so, we can identify which actions each bargaining solution would recommend, and then carry out those actions in proportion to how many civilizations we predict would prefer each bargaining solution. (Of course, many civilizations won’t be interested in doing MSR at all, but they can be ignored for current purposes.)

In practice, calculating complete bargaining solutions may well be intractable. And so the best way of approximating the overall bargaining solution might involve identifying specific “trade deals” that different clusters would engage in if they could negotiate directly. For example, a cluster of highly individualistic civilizations might decide to become more collectivist in exchange for the nearest cluster of collectivist societies becoming more individualist.

Note that this all assumes that other civilizations have a roughly similar “map” of all possible civilizations as we do. The simpler and more powerful our theories of psychohistory, the more likely it is that other civilizations will converge to similar maps. So MSR would be most effective if a few key principles allowed the identification of robust attractors in the space of possible civilizations. If so, the clusters which will benefit most from trade are the ones which are the robust attractors under simple, powerful theories of psychohistory. For this reason, many civilizations will try to make themselves easy to predict.

In order to evaluate these clusters we’d also need some kind of distribution telling us how much to weigh each—aka a metric of “how real they are”. Within a quantum multiverse, we get that “for free”, but across different types of multiverse, this may require arbitrary choices. There doesn’t need to be a single canonical answer in order for MSR to be useful, though, as long as different civilizations converge to similar distributions. (The question of how much to converge seems closely related to the question of how “updateless” to be when applying updateless decision theory.)

Reasoning about values

Each civilization within a cluster will have some very specific values (e.g. “run many human-like minds”). But the values that they’re able to all communicate about, and trade for, are likely to be fairly broad ones, such as:

1. Increasing (certain types of) positive hedonic experiences
2. Decreasing (certain types of) negative hedonic experiences
3. The rights of individuals within civilizations
4. The pursuit of knowledge
5. The pursuit of justice or fairness
6. Equality and/or diversity within and/or between civilizations
7. Conservatism: preserving a civilization’s heritage and past (e.g. nature)
8. Civilizational longevity
9. Aesthetic beauty (although this may well be very specific to each civilization)

If a few key axes explain a large proportion of variance between civilizations, then converging to a cooperative strategy might be relatively simple. For example, if civilizations tend to be clustered into those which value individual freedom, and those which value the pursuit of knowledge, then every civilization adjusting to partially optimize for the other value would be strongly beneficial by everyone’s values.

Clusters will exist at multiple different scales. For example, we might be able to develop very detailed knowledge of other pathways that humanity could have taken, and therefore be able to identify fine structure in clusters of those civilizations; but we might only have hazy guesses about what would happen in universes with very different laws of physics, and therefore only be able to cluster them very broadly. Civilizations too similar to ours might already be doing most of the things we’d want them to; civilizations too different from ours might be too alien to trade with. So I expect that there’s some “sweet spot” of cluster scale which it’s optimal to focus our cooperative efforts on.

“Negotiations” between many different parties in general get far more complicated, and so there may well be a high premium on being able to identify Schelling points which many civilizations will identify as possibilities to converge to. Again, though, most of the work of finding these Schelling points can likely be done by developing similar theories of psychohistory; Schelling points could then be points which can be represented in unusually simple ways in those theories.

Fine-grained MSR

The previous discussion assumed that cooperation would happen in terms of fairly broad values, such as the ones listed above. Could more complicated types of coordination be plausible—ones which don’t just require civilizations to shift their high-level values, but rather to build complex structures? This will depend both on how fine-grained our understanding of other civilizations is, and also how easily negotiation can take place.

The easiest other civilizations to do fine-grained MSR with would be versions of humanity that diverged from our own, since they’re the “closest” to us, and we can reason about them most easily. As one toy example, we might imagine a world in which the Industrial Revolution happened in China rather than Europe, extrapolate out what the resulting civilization might look like, and then predict concrete details of what they likely care about (e.g. preserving and acting on Confucian teachings). However, gains from this type of trade seem relatively small, because traditional Chinese values already have significant power in our current world, and so they’ll likely already have pushed for the cheapest gains.

A more sophisticated version of fine-grained MSR might be to build complex structures which span multiple universes—e.g. an art piece consisting of many different components spread across many different universes. This sort of thing seems unlikely to really matter much, though, especially because aesthetic preferences (or other complex preferences, like humor) seem likely to vary greatly across different civilizations. (Though Carl Shulman commented that universes with access to very large or infinite amounts of computation might be able to trade a lot in exchange for “poorer” universes fulfilling idiosyncratic preferences like these.)

Explaining MSR

Given all of the above, the default way I’d explain MSR to someone unfamiliar with it is now something like:

We’ve seen a bunch of trends in the evolution of human values—e.g. they’ve become more individualistic, more cosmopolitan, and so on, over time. Imagine that trends like these are actually strong enough that, in the future, after studying psychology and sociology and other related topics very deeply, we realize that there were only ever a handful of realistic outcomes for what any civilization’s values look like once it spreads to an intergalactic scale. As a toy example, imagine that the only two plausible attractors are:

• A civilization which prioritized happiness above all else, where many people lived in fake virtual realities while believing they were real. 
• A civilization which prioritized truth above all else, where many people lived unhappy ascetic lifestyles while trying to understand the world better.

Imagine now that you’re running one of those civilizations. You might think to yourself: right now I’m only optimizing for people being happy, but I could easily optimize for people being happy AND truly understanding the world around them, in a way which wouldn’t reduce their happiness much. I’m not going to do it, because I care much more about happiness. But if I encountered the leader of a civilization that cared only about truth, and we agreed to adjust our respective societies to also care about the other’s values, that’d be a great deal for both of us.

[Then explain quantum (or other kinds of) multiverses: those other worlds plausibly exist.]

[Then explain the prisoner’s dilemma and superrationality.]

4 comments

Comments sorted by top scores.