Preserving and continuing alignment research through a severe global catastrophe

post by A_donor · 2022-03-06T18:43:10.746Z · LW · GW · 11 comments

Contents

  Introduction
  Preserving alignment knowledge through a global catastrophe
    What data do we want to store?
    How do we want to store it?
    Where do we store it?
  Continuing alignment research after a global catastrophe
    Evacuation plans
    Designing havens
    Call to action
None
13 comments

[Epistemic status: Shallow dive into research questions, backed by some years of on-and-off thinking about this kind of plan.]

Introduction

There is some chance that civilization will cease to function before we hit an intelligence explosion. If it does, it would be good to preserve existing alignment research for future generations who might rebuild advanced technology, and ideally have safe havens ready for current and future researchers to spend their lives adding to that pool of knowledge.

This might delay capabilities research by many decades, centuries, or longer while allowing basic theoretical alignment research to continue, and so be a potential Yudkowskian positive model violation [LW · GW] for which we should prepare.

Setting this infrastructure up is a massively scalable intervention, and one that should likely be tackled by people who are not already on the researcher career path. It would have been good to get started some years ago given recent events, but now is the second best time to plant a tree.^[1]

Preserving alignment knowledge through a global catastrophe

What data do we want to store?

Thankfully, the EleutherAI people are working on a dataset of all alignment research^[2]. It's still a WIP^[3] and contributions to the scripts to collect it are welcome, so if you're a programmer looking for a shovel ready way to help with this then consider submitting a PR^[4].

How do we want to store it?

My shallow dive into this uncovered these options:

• We could print it out on paper
  • Lifetime: 500+ years in good conditions (might depend significantly on paper and ink quality, more research needed). Vacuum sealing it with low humidity seems like it would help significantly.
  • Pros: Totally human readable.
• Microsoft's Project Silica is the longest lasting option I could find
  • Lifetime: 10000+ years
  • Cons: Would require high levels of technology to read it back. I'm not seeing an option to buy the machines required to write new archives and expect them to be very advanced/expensive, so this would be limited to storing pre-collapse research.
• CDs could be a minimalist option
  • Lifetime: Maybe 50 years  if stored in good conditions
  • Pros: Good ability for researchers to explore the information on computers while those last)
  • Cons: It's very plausible that a severe GCR^[5] would set us back far enough that we'd not regain CD reading technology before they decayed so they aren't a full solution.
• The Arctic World Archive seems worth including in the portfolio
  • Lifetime: 1000+ years
  • Pros: It's a pretty straightforward case of turning money into archives
  • Cons: Not very accessible in the meantime
• The DOTS system (a highly stable tape-based storage medium) might be a strong candidate, if it is buyable.^[6]
  • Lifetime: 200-2000+ years
  • Pros: Human readable or digital archives. 

Each has advantages, so some combination of them might be ideal.

Where do we store it?

Having many redundant backups seem advisable, preferably protected by communities which can last centuries or in locations which will not be disturbed for a very long time. Producing "alignment backup kits" to send out and offering microgrants to people all around the world to place them in secure locations would achieve this. We'd likely want basic (just pre-collapse work) and advanced (capable of adding archives for a long time post-collapse) options.

If you'd like to take on the challenge of preparing these kits, storing an archive, or coordinating things, please join the Alignment After A GCR Discord (AAAG). I'm happy to collaborate and give some seed funding. If you want to help collect and improve the archive files, #accelerating-alignment on EAI is the place to go.

Continuing alignment research after a global catastrophe

It is obviously best if as many people survive the GCR as possible, and supporting the work of organizations like the Alliance to Feed the Earth in Disasters seems extremely valuable. However, a targeted intervention to focus on allowing alignment researchers to continue their work in the wake of a disaster might be an especially cost-effective way to improve the long-term future of humanity.

Evacuation plans

A list of which researchers to prioritize would need to be drawn up.^[7] They would need instructions on how to get to the haven, ideally someone with reliable transport to take them there. In case of moments of extreme risk, they would be encouraged to preemptively (and hopefully temporarily) move to the haven.

Designing havens

The locations would need to be be bought, funded, and partially populated before the GCR.^[8] I have some ideas about which other subcultures might be good to draw from, with the Authentic Relating community top of the list.^[9]

The havens would need to be well-stocked to weather the initial crisis and recover after. They should be located in places where farming or fishing could produce a surplus in the long term to allow some of the people living there to spend much of their time making research progress. Being relatively far from centers of population seems beneficial, but close enough to major hubs that transport is practical. There are many considerations, and talking to ALLFED to get their models of how to survive GCRs seems like an obvious first step to plan this.

Avoiding the failure mode of allowing so many people to join that the whole group goes under would be both challenging and necessary. Clear rules would have to be agreed on for who could join.

The culture would need to be set up to be conducive to supporting research in the long term while being mostly self-sufficient, this would be an interesting challenge in designing community. People with the skills to produce food and other necessities would need to be part of the team.

Call to action

Even more than archiving, this needs some people to make it their primary project in order for it to happen. That could include you! I would be happy to provide advice, mentorship, connections, and some seed funding to a founder or team who wants to take this project on.^[10] Message me here or @A_donor on the Discord.

This project could also benefit from volunteers for various roles. If you or someone you know would like to help by

• Searching for locations
• Potentially moving to a haven early and helping set up
• Researching questions
• Putting us in contact with people who might make this work (e.g. people with experience in self-sufficient community building)
• Doing other tasks to increase the chances that we recover from GCRs with a strong base of alignment theory

Please join the Discord and introduce yourself, specifically indicating that you'd like to help with havens so I know to add you to those channels.

I can fund the very early stages of both projects, but in order to scale it to something really valuable we would need major funders on board. If you are or have access to a major funder and want to offer advice or encouragement to apply that would increase the chances that this goes somewhere.

It's quite likely that I won't post public updates about the havens part of this project even if it's going relatively well, as having lots of attention on it seems net-negative, so don't be surprised if you don't hear anything more.

1. ^{^}

   "The best time to plant a tree is twenty years ago. The second best time is now." - Quote

2. ^{^}

   They want to use it to train language models to help with alignment research, but it aims to contain exactly what we'd want.

3. ^{^}

   Work In Progress

4. ^{^}

   Pull Request - A way of suggesting changes to a repository using version control, usually used in programming.

5. ^{^}

   Global Catastrophic Risk [? · GW] - An event which causes massive global disruption, such as a severe pandemic or nuclear war.

6. ^{^}

   The website is unclear on whether it's immediately available.

7. ^{^}

   If you're a researcher and want to be on the list, feel free to contact me with your location and I'll keep track of everyone's requests. We might possibly use Alignment EigenKarma [EA(p) · GW(p)] as an unbiased metric to prioritize if that exists in time.

8. ^{^}

   Unless anyone knows of good places which might be joinable already, if you do please message me!

9. ^{^}

   They are compatible with Rationalist/EA culture, more likely than most to be able to create stable communities, and some of them like the idea of building strong community for the benefit of all of humanity.

10. ^{^}

    I have a reasonably strong track record as a Mentor/Manager/Mysterious Old Wizard [LW · GW]/Funder package deal. If you're enthusiastic and bright don't worry if the task seems overwhelming, I can help you pick up the skills and decompose tasks.

11 comments

Comments sorted by top scores.