The case for Doing Something Else (if Alignment is doomed)

My interpretation of MIRI is that they are ARE looking for alternatives and so far have not found any that don't also seem doomed. E.g. they thought about trying to coordinate to ban or slow down AI capabilities research and concluded that it's practically impossible since we can't even ban gain-of-function research and that should be a lot easier. My interpretation of MIRI is that their recent public doomsaying is NOT aimed at getting people to just keep thinking harder about doomed AI alignment research agendas; rather, it is aimed at getting people to think outside the box and hopefully come up with a new plan that might actually work. (The recent April Fool's post also served another function of warning against some common failure modes, e.g. the "slip sideways into fantasy world" failure mode where you start gambling on assumptions holding true and then end up doing this repeatedly and losing track of how increasingly unlikely the world you are planning for is.)

(Also, couldn't we equally point to unsolved subproblems in alignment, or alignment for easier cases, and demand that they be solved before we dare tackle the hard problem?)

I think this is is missing the point somewhat.

When Eliezer and co. talk about tackling "the hard part of the problem"* I believe they are referring trying to solve to the simplest, easiest, problems that capture some part of the core difficulty of alignment.

See this fictionalized segment from the rocket alignment problem: 

We’re working on the tiling positions problem because we think that being able to fire a cannonball at a certain instantaneous velocity such that it enters a stable orbit… is the sort of problem that somebody who could really actually launch a rocket through space and have it move in a particular curve that really actually ended with softly landing on the Moon would be able to solve easily. So the fact that we can’t solve it is alarming. If we can figure out how to solve this much simpler, much more crisply stated “tiling positions problem” with imaginary cannonballs on a perfectly spherical earth with no atmosphere, which is a lot easier to analyze than a Moon launch, we might thereby take one more incremental step towards eventually becoming the sort of people who could plot out a Moon launch.

Hence, doing Agent Foundations work that isn't directly about working with machine learning systems, but is mostly about abstractions of ideal agents, etc.

Similarly, I think that creating and enforcing a global ban on gain of function research captures much of the hard part of causing the world to coordinate not to build AGI. It is an easier task, but you will encounter many of the same blockers, and need to solve many of the same sub-problems. 

Creating a global ban on gain of function research : coordinating the world to prevent solving AGI :: solving the tiling agents problem : solving the whole alignment problem.

* For instance, in this paragraph, from here [LW · GW]. 

It's sad that our Earth couldn't be one of the more dignified planets that makes a real effort, correctly pinpointing the actual real difficult problems and then allocating thousands of the sort of brilliant kids that our Earth steers into wasting their lives on theoretical physics.  But better MIRI's effort than nothing.  What were we supposed to do instead, pick easy irrelevant fake problems that we could make an illusion of progress on, and have nobody out of the human species even try to solve the hard scary real problems, until everybody just fell over dead?

1. Convince a significant chunk of the field to work on safety rather than capability
2. Solve the technical alignment problem
3. Rethink fundamental ethical assumptions [LW · GW] and search for a simple specification of value
4. Establish international cooperation toward Comprehensive AI Services, i.e., build many narrow AI systems instead of something general

I'd say that basically factors into "solve AI governance" and "solve the technical alignment problem", both of which seem extremely hard, but we need to try it anyways.
(In particular, points 3&4 are like instances of 2 that won't work. (Ok maybe sth like 4 has a small chance to be helpful.))

The governance and the technical part aren't totally orthogonal. Making progress on one helps making the other part easier or buys more time.

(I'm not at all as pessimistic as Eliezer, and I totally agree with What an Actually Pessimistic Containment Strategy Looks Like [LW · GW], but I think you (like many people) seem to be too optimistic that something will work if we just try a lot. Thinking about concrete scenarios may help to see the actual difficulty.)

Towards a #1-flavored answer, a Hansonian fine insured bounty system seems like it might scale well for enforcing cooperation against AI research.

https://www.overcomingbias.com/2018/01/privately-enforced-punished-crime.html

I would break down the possibility space as:

1. Humanity doesn't build AGI in the first place, via governance, monitoring, etc.
2. Humanity doesn't build AGI in the first place, via somebody doing a "pivotal act" that does not require AGI (e.g. release nanobots that spread across the earth melting all computer chips)
3. Somebody figures out how to build safe AGI

(Your #4 is my #1, your #1 is my #1 or #3, your #2 is my #3, your #3 is one aspect of my #3 (i.e., it's getting at an approach to ease outer alignment, but we would still also need to solve inner alignment).)

Yeah, I suspect that EY thinks #1 is even less likely than #3, and that he doesn't think about #1 at all except to poo-poo people who are even trying. My impression is that EY has done work brainstorming on #2 in the past, but that he has given up.

For my part, I'm more open-minded and happy for people to be trying different things in parallel—see for example what I wrote under “objection #6” here [AF · GW] (expressing an open mind that someone might solve #1).

I think there's a few things that get in the way of doing detailed planning for outcomes where alignment is very hard and takeoff very fast. This post by David Manheim discusses some of the problems: https://www.lesswrong.com/posts/xxMYFKLqiBJZRNoPj [? · GW]

One is that, there's no clarity even among people who've made AI research their professional career about alignment difficulty or takeoff speed. So getting buy in in advance of clear warning signs will be extremely hard.

The other is that the strategies that might help in situations with hard alignment are at cross purposes to ones in Paul-like worlds with slow takeoff and easy alignment - promoting differential progress Vs creating some kind of global policing system to shut down AI research

What if AI safety and governance people published their papers on Arxiv in addition to NBER or wherever? I know it's not the kind of stuff that Arxiv accepts, but if I was looking for a near-term policy win, that might be one.