"Far Coordination"

post by DragonGod · 2022-11-23T17:14:41.830Z · LW · GW · 17 comments

Contents

    Epistemic Status
  Introduction
    What is "Far Coordination"?
    Why is Far Coordination Possible?
      Premises
      Spacetime Invariance of Computation
    Why Care About Far Coordination?
  Use Cases for Far Coordination
    Preventing "Monstrous" Divergences
    Policing Particular Behaviours
    Maintaining Unity
    Coordination in Multi Civilisation Interactions
  Interlude
  Approaches To Far Coordination
    General Approach
    Clones
    Partner Simulations
  Considerations for Far Coordination
    Challenges
      Drift
      Error Accumulation
      Death/Inaccessibility
    Mitigations
      Stable/Static Aspects
      Error Correction
      Resynchronisation
      Backups
      Successorship
      Constitution
  Fundamental Limitations of Far Coordination
    Stasis
    Distance Restrictions
    Eating Your Cake and Having It
    Vulnerability
  Further Steps
None
17 comments

Epistemic Status

Early and rough thoughts. Sharing because I believe the core idea is sound^[1]. 

 

Introduction

What is "Far Coordination"?

Coordination across such vast distances of spacetime that communication between coordinating parties is physically impossible^[2].

 

Why is Far Coordination Possible?

Premises

Assumptions on which the rest of this post lies:

• Computationalism: all agents of interest are computations that can be replicated to arbitrarily high fidelity on other substrates
• Robust cryptography
• Parties can send messages to each other that can't be forged
• Digital signatures can't be forged 

Spacetime Invariance of Computation

Computation is time and space invariant^[3].

"$2+2$" evaluates to $4$ whether it's computed today, 5 billion years ago or 5 billion years from now. It also evaluates to $4$ whether it's computed on earth, elsewhere in the Milky Way or somewhere in Andromeda.

The spacetime invariance of computation can be exploited to facilitate coordination across vast distances of time and space.

 

Why Care About Far Coordination?

I am interested in far coordination because I want our civilisation to become interstellar (and eventually intergalactic). Coordination across vast distances of spacetime would be necessary to remain a united civilisation (as opposed to fracturing into a multitude of [potentially very divergent] civilisations) after we venture out of the solar system.

I would prefer for earth originating civilisation to remain broadly united, should we venture to the stars.

---

Use Cases for Far Coordination

Preventing "Monstrous" Divergences

Some future-minded folks I've chatted with seem to be concerned that posthumans would diverge from us in considerable ways. Particularly, there's concern that their values may be so radically altered that they would seem to us (were we around to observe them) not just alien, but monstrous^[4].

 

Policing Particular Behaviours

Some people are very concerned about astronomical suffering. For those with suffering-focused moral foundations, the prospect of astronomical suffering may dominate all considerations. There may be a desire to implement mechanisms to prevent anyone from running simulations of sentient beings for the purpose of torturing them. Such policing may be viable for a planetary or even interplanetary civilisation, but without means to coordinate behaviour across vast distances, it's not viable for interstellar or intergalactic civilisations.

I do not endorse creating a cosmic police state/totalitarian government to police mindcrime, but I do understand the desire to constrain the behaviour of our descendants. We may legitimately not want any civilisation that can trace its genealogy to Earth (any of our descendants) to engage in behaviours we consider especially heinous.

 

Maintaining Unity

Civilisations that can't coordinate across vast distances in space would necessarily splinter and fracture into different pockets should they venture out of their host star systems. The splinters may develop into separate civilisations that don't view themselves as brethren with their sibling civilisations. They may even later wage war upon said siblings.

This fracturing of civilisation, and the potential for conflict among siblings, is a pretty horrifying prospect. Robin Hanson has suggested that a desire to maintain unity/conformity among humanity (and our descendants) may be a reason we never venture out to other star systems (from 08:38 [09:55 if pressed for time] to 12:04 in the podcast).

Even without outright conflict, the sibling civilisations may compete amongst each other for the limited resources within their (perhaps overlapping) spheres of influence. This becomes especially likely if some sibling civilisations are "grabby" (this trend may be exacerbated by selection pressures favouring grabby civilisations^[5]).

 

Coordination in Multi Civilisation Interactions

A plausible scenario where far coordination would prove especially valuable is in multi-civilisation interactions. By "multi-civilisation", I refer here to civilisations that cannot trace their history to a common ancestor. 

Far coordination would be especially valuable should our descendants run across aliens.

For each civilisation, let its civilisation tree be all civilisations that share a common ancestor. For humanity, our civilisation tree would be all civilisations that can trace their ancestry to Earth (all Earth originating civilisations).

Multi-civilisation interactions can be viewed as a multi-agent environment^[6]. I'll briefly outline below ways in which it is useful/desirable for a civilisation tree to behave as a single agent:

• United front/interface
  • The entire civilisation tree can participate in strategic interactions as a single actor
  • Other agents may treat the entire civilisation tree as a single actor
• United policy regarding cooperation, competition and/or conflict
  • Cooperation
    • Nodes of a civilisation tree may honour commitments made by other nodes
      • Said commitments could be verified (via e.g. digital signatures or other suitable cryptographic mechanisms)
    • Nodes may negotiate on behalf of their entire civilisation, or other nodes
      • This could be especially helpful as the beneficiary node(s) may not be able to participate in the negotiations in a timely manner due to speed of communication restrictions
  • Competition/conflict
    • An alliance with one node might become in effect an alliance with the entire civilisation tree
    • Likewise, a declaration of war on one node may be treated as a declaration of war on the entire civilisation tree
• Nodes benefit from the united strength of the entire civilisation tree
  • Favours could be repaid by nodes other than the recipient nodes
    • Incentivises favourable actions to nodes of the civilisation
    • Reliable receipts of the services rendered could be provided again (via e.g. digital signatures or other suitable cryptographic mechanisms)
  • Retribution may be visited by cells other than the offended cells
    • Disincentivises harming any node of the civilisation tree^[7]
    • This is somewhat more tricky as the harm inflicted may not be reliably communicated, or other nodes may only learn about the harm a long time after the fact
    • Nodes could send heartbeat messages (with an accompanying GUID^[8] to identify the source) to all listening nodes, and a prolonged cessation in said heartbeats could warrant investigation and (if needed) retaliation
      • Redundant dedicated responder nodes could be set up "in proximity"^[9] to particular nodes, for "fast" responses
      • Each new node added to the civilisation would (eventually) update the entire civilisation tree of its existence via its heartbeat
      • The heartbeats could also encode more sophisticated information if needed
    • The civilisation could adopt a policy of hunting down adversaries/avenging any fallen brethren
      • Consensus resolution mechanisms would need to be decided beforehand to determine how much resources each node should commit to particular vendettas
        • Probably the responsibility falls on the closest nodes, and/or the dedicated responder nodes
• Reliable trade
  • Payments could be received from other nodes if needed
  • The entire civilisation tree could present a uniform policy around trade to other civilisations

---

Interlude

Suppose an advanced civilisation is currently united. The civilisation is going to dispatch a colony ship to a nearby star system. The colony would set up a child civilisation on the new star system. The parent desires that the behaviour of their child be coordinated with their own across the intervening distance; how might they achieve this?

Over the remainder of this post, I will briefly describe a couple approaches, challenges/limitations of the approaches, mitigations to the challenges, and fundamental limitations of far coordination.

---

Approaches To Far Coordination

General Approach

Before the colony ship departs for the nearby star system, some preparations could be made:

1. The parent civilisation freezes their governance/coordination (super)structures
2. The governance structures of the child are made to mirror the parent (with analogous positions for the key decision makers)

 

Clones

1. High fidelity copies of key decision makers in the parent civilisation could be made
2. Said copies would then assume the corresponding roles/positions of their originals in the governance structures of the child

The copies have the same initial values and decision algorithms as the originals, and so would make the same decisions in analogous circumstances as the originals would have (had they access to the same information as the copies [the difference in lived experience since the copies separated from the originals]).

 

Partner Simulations

1. The key decision makers of the child are selected independently
2. High fidelity simulations of relevant key decision makers^[10] of the child and the parent are created
3. The colony is given access to the simulations of the parent decision makers (and vice versa)

After separation, the colony can consult the simulations of the parent decision makers as needed (and vice versa) to coordinate behaviour.

Arrangements would need to be made beforehand for consensus resolution mechanisms to decide how to incorporate the feedback from the simulations.

---

Considerations for Far Coordination

Challenges

There are several challenges that would limit the reliability/robustness of far coordination.

 

Drift

After separation, the two civilisations would undergo different experiences. These experiences may modify relevant aspects of the key decision makers in the two civilisations such that the simulations no longer reliably reflect the modified original.

In particular the following changes are problematic for far coordination:

• Drift in values
• Drift in decision making algorithms

Significant drift could break basically all the previously mentioned use cases for far coordination^[11].

 

Error Accumulation

Aside from drift, miniscule deviations in the fidelity of the simulations to the original may accumulate into significant deviations when aggregated across many decision makers or vast stretches of time.

 

Death/Inaccessibility

Relevant counterparts in the colony or in the parent civilisation may die, be removed from power or otherwise be unavailable for decision making. Thus, actions coordinated with a simulation of an unavailable actor may not be tracking anything meaningful or relevant on the child/parent.

 

Mitigations

Steps that can be taken to mitigate the aforementioned challenges.

 

Stable/Static Aspects

In order to mitigate drift, the relevant decision makers could make core aspects of their values/decision algorithms stable/static so that their simulations continue to reliably reflect the originals even in the furthest reaches of time, after substantial divergence in lived experiences.

To attain the highest level of stability, entire aspects (values/decision algorithms) of relevant actors may be completely frozen.

 

Error Correction

Error correction mechanisms could be used to prevent errors from propagating/accumulating. This feels more promising when combined with stable/static aspects. If there's stochastic variation between the simulation and the original, then e.g. a particular decision could be evaluated many times to attain arbitrarily high confidence that a result reflects the decision of the original in a given circumstance^[12].

 

Resynchronisation

Another approach to mitigate drift is that the simulations and the original synchronise periodically. The parent and the child connect periodically (e.g. every million years^[13]), new simulations are generated, surprising divergences may be interrogated, the coordination protocols could be improved.

A limitation of this mitigation is that it requires the colony to be near enough to the parent that periodic resynchronisation is viable. This approach seems feasible for e.g. an interstellar civilisation but becomes fraught for coordinating across galaxies (especially galaxies in different [super]clusters).

 

Backups

To mitigate the issue of key decision makers dying, being removed from power or otherwise becoming inaccessible, redundant backups of key decision makers could be made at frequent intervals. The backup would be summoned at any instance in which an important decision needs to be made but a relevant actor is unavailable.

 

Successorship

If the values and decision algorithms of the relevant actors could be sufficiently abstracted such that they could be reliably communicated to other actors/other actors can reliably implement them, then there could be succession of relevant decision makers with the successors being trained in the values and decision algorithms they are to implement.

The terms of decision makers on the child and parent could further be synchronised. If coupled with resynchronisation, the new leaders of parent and child could connect and synchronise before the start of their relevant terms.

 

Constitution

Regardless of whether successorship is pursued or not, abstracting away the relevant aspects of decision making so that it can be implemented by different actors — a constitution perhaps — seems like it would be valuable.

With resynchronisation, the constitution could be amended (if needed at every synchronisation), without resynchronisation, the constitution could be frozen, and rigid adherence to the constitution guaranteeing coordination^[14].

---

Fundamental Limitations of Far Coordination

It seems like with robust far coordination you can get only one of the following:

• Coordination across arbitrary distances
• Drift

And you're furthermore forced to relinquish the security of redundant independent nodes.

 

That is, the following are fundamental limitations:

• Stasis
• Distance Restrictions
• Vulnerability

 

Stasis

Any reliable sort of far coordination that is not bound by distance seems like it would require locking in core aspects of the decision making. This may look like all the relevant decision makers freezing their values and decision algorithms, or it may be abstracting said values and algorithms into a comprehensive constitution (that would then be frozen).

Regardless, the kind of free form evolution in values, philosophy, governance/coordination systems we've enjoyed for most of human history would become a thing of the past.

If the frozen aspects are confined to a decision making elite, but most citizens are allowed to drift freely, the involved societies would soon find themselves in a situation where their governance structures and leaders are archaic, or so far removed from their current values that it's dystopian.

It may thus be the case that the locked in values/decision algorithms may need to be applied not only to the leaders of the involved societies, but even ordinary citizens.

Civilisations may thus have to freeze core aspects of themselves to enable far coordination across arbitrary distances.

Stasis may be undesirable as species that self-modify to lock themselves into a particular version may be less adaptable, less able to deal with unforeseen circumstances. Perhaps such species may be outcompeted by more dynamic/adaptable species.

(Though this may be also be infeasible given that the lock in would only happen after the civilisation in question has amassed considerable power.

I do wonder what selection pressures advanced civilisations face? Do these selection pressures push towards stasis, or do they favour dynamism?^[15])

 

Distance Restrictions

To facilitate robust far coordination without stasis, parent civilisations need to remain sufficiently close to all their descendants that resynchronisation is possible. That is the spatial extent of a civilisation tree will need to be confined.

If stasis is considered so undesirable, this may be the more preferable pill to swallow.

 

Eating Your Cake and Having It

Is it possible to have both? Or to what extent can we attain both?

We could try and develop a static comprehensive constitution after a very long reflection [? · GW]. The constitution could be used as a coordination superstructure for all earth originating civilisations. Drift would be permitted within the confines of that constitution. 

Resynchronisation might be pursued within individual "cells" (galaxies? galactic clusters? superclusters?), while different cells would be allowed to drift apart from each other (again within the confines of the constitution).

 

Vulnerability

One of the main benefits of expanding to the stars/other galaxies is to attain robustness against existential threats. Spatial separation provides civilisational redundancy, as existential catastrophes that visit one node may be unable to affect other nodes.

Spatial separation will protect a united civilisation from most external threats, but existential threats to the civilisation arising from its behaviour may remain a concern. If one node of a civilisation suffers existential catastrophe arising from internal threats, then other nodes are likewise vulnerable to the same internal threats. Perhaps the entire civilisation might likewise succumb to internal existential catastrophe.

The unified behaviour of the civilisation might thus present a single point of failure (albeit one that may not necessarily be exploitable by external adversaries^[16]).

A potential mitigation would be for civilisations to only expand outside their star system after attaining existential security [? · GW]. Alternatively, stasis should only be pursued when a civilisation is confident that it is sufficiently robust to all internal threats.

 

---

Further Steps

Stuff I'd like to do later^[17]:

1. Fill in any missing details
2. Explore the relationship between far coordination mechanisms and distance
   1. What coordination options do very short distances afford
   2. What coordination options are forced by very long distances
3. Explore far coordination across temporal separations
4. Expand on
   1. Use cases for far coordination
      1. Especially with respect to multi-civilisation interactions
      2. The list of bullet points was initially planned to be expanded into sentences/entire paragraphs.
   2. Challenges for far coordination
   3. Mitigation actions
   4. Fundamental limitations of far coordination
5. Make the arguments more rigorous
6. Persuade someone to write a fic exploring this concept

1. ^{^}

   That said, I suspect the basic insight is not novel and is already present in e.g. the acausal trade literature.

2. ^{^}

   Due to the speed of light limits on the transmission of arbitrary information.

3. ^{^}

   Computation is also invariant along other dimensions (logical truths are true in every "(logically) possible world"), so potentially coordination across universes or even entire ontologies might be possible.

   I do not actually think the potential for such coordination ("very far coordination"?) is relevant to realising the longterm potential of human civilisation, so I did not address any such mechanisms in this post.

   Robust cooperation within our future light cone is sufficient for all practical purposes I think.

4. ^{^}

   This is not just something that can happen; the argument has been made to me that it is what we should expect by default.

   I do think that digital life (extended lifespans/indefinite life, resurrection/pseudo-immortality [e.g. via restoration from backups], trivial reproduction [via copying], arbitrary self-modifications [arbitrary changes might be made to cognitive architectures, (meta) values, accessible subjective states, etc.]) and the much greater power and accessible resources to posthumans would give them a perspective that is markedly different from our own.

   I'm not convinced that posthumans would necessarily be "monstrous" to us, but I do expect them to be very alien to us. Posthuman mindspace is probably vastly larger than human mindspace, and has much greater extreme points along relevant dimensions. I do expect that some subset of posthuman mindspace would be "monstrous", I'm just not convinced that said subset is especially likely to manifest in the mainline.

5. ^{^}

   On the individual, group, factional and sub civilisational levels, selection pressures favour power/influence seeking behaviour (maximal reproduction on the individual level being one example of power seeking behaviour).

   Power seeking also seems to be favoured on the civilisational level as well in a manner that translates directly to "grabbiness".

   Power seeking civilisations acquire more resources, become more powerful, and progressively expand their spheres of influence faster than less grabby civilisations. To a first approximation, the "grabbiest" civilisations are the most powerful.

   Within an attainable sphere of influence, the fastest expanding civilisations would acquire most of the resources (assuming comparable starting conditions).

   Furthermore, more powerful civilisations are better able to compete for resources against less powerful civilisations should their spheres of influence overlap.

   I would thus expect that less grabby civilisations get outcompeted, assimilated or outright destroyed when their spheres of influence overlap with the spheres of influence of grabby civilisation.

   If the grabby civilisations are especially benevolent, they may leave some limited preserve for their weaker counterparts. Said weaker counterparts may thus be able to continue some form of limited existence, but would lose out on the majority of what they had considered their cosmic endowment. 

   That said, I expect that benevolent grabby civilisations are more likely to assimilate weaker civilisations, than to voluntarily cede resources to them. Only in encounters between peer civilisations do I expect them to voluntarily forego competition/cede resources (because the expected cost of competition/resource conflict outweigh the expected value of the attainable resources).

6. ^{^}

   The set of agents could be a partition of the set of civilisations (or relevant sub civilisational groups) such that:

   1. Each set in the partition is robustly coordinated in their behaviour/decision making (at the appropriate level of abstraction)

   2. No set is a proper subset of some other robustly coordinated set 

7. ^{^}

   A civilisation tree that had a policy of responding positively to negative incentives would be especially vulnerable to them (one only needs to compromise one node [via blackmail, hostages, threats, etc.] to extract utility from the entire civilisation tree).

   I think for decision theoretic reasons, the civilisation tree should have a policy of unconditionally rejecting any negative incentives. If the civilisation tree is known to be a united actor, each node that refuses to respond to negative incentives (even when they would be naively expected to) provides evidence that the civilisation tree does not respond to negative incentives and reduces the likelihood that other nodes would be presented with negative incentives.

   Because the civilisation tree is distributed across space, mounting external existential threats to the civilisation tree may be infeasible, so it may not be possible for adversaries to present any negative incentives that are sufficiently strong that the civilisation tree has no choice but to yield to them.

8. ^{^}

   A simple way to generate GUIDs for each node:

   Each node's GUID includes the GUID of its parent as a prefix followed by a special separator character and a unique identifier locating that node among its parent's children (e.g. birth number).

9. ^{^}

   If nodes are individual star systems, the associated responder nodes could be located in the interstellar medium. If entire galaxies, in the intergalactic medium.

   Ideally, there's probably be responder nodes placed in "orbits" at various distances around a particular target node.

10. ^{^}

    The actors whose decision making we want to coordinate among. Some other actors may be "key decision makers" in some meaningful way, but their decision making not relevant to coordinating the behaviour of the colony with the parent.

11. ^{^}

    1. Unconstrained drift may lead to monstrous divergences

    2. Monstrous divergences may lead the divergent civilisation to engage in behaviour the parent wanted to prohibit.

    3. Significant divergences break unity and may cause the divergent civilisation to view itself as distinct from its parent.

    4. Significant divergences and civilisation fracturing prevents the civilisation from participating in multi-civilisation interactions as single actor, robbing it of the benefits thereof.

12. ^{^}

    There are probably many more error correcting mechanisms that would be relevant here, but I'm sadly not familiar with the relevant literature.

13. ^{^}

    The resynchronisation frequency would depend on the intervening distance. If e.g. (post)humanity set up a child civilisation on Alpha Centauri, then in the absence of any other children to coordinate with, we'd want to resync a lot more frequently than every million years [perhaps on the order of a 100 years?]).

14. ^{^}

    Enforcement of the constitution could be automated via AI systems. Or relevant actors may self-modify into agents that are incapable of violating the constitution.

15. ^{^}

    This sounds like the plot of a pretty interesting short story to me.

16. ^{^}

    A united civilisation may still be especially vulnerable to particular exploits by sophisticated adversaries; space-like separation just sharply limits the ability of said adversaries to realise such exploits against all nodes of a united civilisation.

17. ^{^}

    But will probably not get around to doing any time soon.

17 comments

Comments sorted by top scores.