What would we do if alignment were futile?

After seeing a number of rather gloomy posts on the site in the last few days, I feel a need to point out that problems that we don't currently know how to solve always look impossible. A smart guy once pointed out [LW · GW] how silly it was the Lord Kelvin claimed "The influence of animal or vegetable life on matter is infinitely beyond the range of any scientific inquiry hitherto entered on." Kelvin just didn't know how to do it. That's fine. Deciding it's a Hard Problem just sort of throws up mental blocks to finding potential obvious solutions.

Maybe alignment will seem really easy in retrospect. Maybe it's the sort of thing that requires only two small insights that we don't currently have. Maybe we already have all the insights we need and somebody just needs to connect them together in a non-obvious way. Maybe somebody has already had the key idea, and just thought to themselves, no, it can't be that simple! (I actually sort of viscerally suspect that the lynchpin of alignment will turn out to be something really dumb and easy that we've simply overlooked, and not something like Special Relativity.) Everything seems hard in advance, and we've spent far more effort as a civilization studying asphalt than we have alignment. We've tried almost nothing so far. 

In the same way that we have an existence-proof of AGI (humans existing) we also have a highly suggestive example of something that looks a lot like alignment (humans existing and often choosing not to do heroin), except probably not robust to infinite capability increase, blah blah.

The "probabilistic mainline path" always looks really grim when success depends on innovations and inventions you don't currently know how to do. Nobody knows what probability to put on obtaining such innovations in advance. If you asked me ten years ago I would have put the odds of SpaceX Starship existing at like 2%, probably even after thinking really hard about it. 

Thank you for articulating this. This matches closely with my own thoughts re Eliezer's recently published discussion. I strongly agree that if Eliezer is in fact correct then the single most effective thing we could do is to persuasively show that to be true. Right now it's not even persuasive to many / most alignment researchers, let alone anybody else.

Conditional on Eliezer being wrong though, I'm not sure how valuable showing him to be wrong would be. Presumably it would depend on why exactly he's wrong, because if we knew that then we might be able to direct or resources more effectively.

I think that for those who agree with Eliezer, this is a very strong argument in favor of pouring money and resources into forecasting research or the like - as Open Philanthropy is in fact doing, I think. And even for people who disagree, if they put any non trivial probability mass on Eliezer's views, that would still make this high priority.

If I was convinced that any AI built would destroy the world, I would advocate for a preemptive Butlerian Jihad. I would want coordinated dismantling of all supercomputing facilities (à la nuclear disarmament), which I would define as any system currently capable of training GPT-3. I would want global (NATO/China) coordination to shut down all major semiconductor fabrication plants. Despite my distaste for tyrannical policy, I would support government efforts to confiscate and destroy every Xbox, Playstation, and every GPU over 2 TFlops.

Since we are one or more key insights away from AGI, a small guerilla group with a Beowulf cluster would not be too much of a risk. Even a smallish rogue nation would not be too bad, as AI talent is highly concentrated in Europe, the USA, and China.

I don't know whether the grim model in Eliezer's interview is true or not. ... If it's false (alignment efforts are likely to work), then we need to know that.

I think this is a false dichotomy. Eliezer's position is that said that AI alignment requires a "miracle" aka "positive model violation" aka "surprising positive development of unknown shape". If that is false, that does not mean that alignment efforts are "likely to work". They could still fail just for ordinary non-miraculous reasons. Civilization has failed to prevent many disasters that didn't require miracles to prevent.

Governments are not known to change their policies based on carefully reasoned arguments, nor do they impose pro-active restrictions on a technology without an extensive track record of the technology having large negative side-effects. A big news-worthy event would need to happen in order for governments to take the sort of actions that could have a meaningful impact on AI timelines, something basically on the scale of 9/11 or larger. 

I think steering capabilities research in directions that are likely to yield "survivable first strikes" would be very good and could create common knowledge about the necessity of alignment research. I think GPT-3 derivatives have potential here, they are sort of capped in terms of capability by being trained to mimic human output, yet they're strong enough that a version unleashed on the internet could cause enough survivable harm to be obvious. Basically we need to maximise the distance between "model strong enough to cause survivable harm" and "model strong enough to wipe out humanity" in order to give humanity time to respond after the coordination-inducing-event. 

There seems to be an implicit premise lurking in the background, please correct me if I’m wrong, that determining the degree of alignment to a mutually satisfactory level will be possible at all, that it will not be NP hard. 

i.e. Even if that ‘unknown miracle’ does happen, can we be certain that everyone in a room could closely agree on the ‘how much alignment’ questions? 

Try to come up with the best possible "unaligned" AGI. It's better to be eaten by something that then goes out and explores a broad range of action in an interesting way, especially if you can arrange that it enjoy it, than it is to be eaten by Clippy.

I bet a lot of them are persuadable in the next 2 to 50 years.

They may be persuadable that, in a non-emergency situation, they should slow down when their AI seems like it's teetering on the edge of recursive self-improvement. It's much harder to persuade them to

1. not publish their research that isn't clearly "here's how to make an AGI", and/or

2. not try to get AGI without a good theory of alignment, when "the other guys" seem only a few years away from AGI.

So ~everyone will keep adding to the big pool of ~public information and ideas about AI, until it's not that hard to get the rest of the way to AGI, at which point some people showing restraint doesn't help by that much.

Second, US export controls can buy time by slowing down the whole field.

I think this will have the opposite effect. Restricting supply of hardware will only further incentivize efficiency-focused research, which I think is much more critical on the path to AGI than "stack moar layers".

Roman Yampolsky has said recently (at a Foresight Salon event, the recording should be posted on YouTube soon) that it would be highly valuable if someone could prove that alignment is impossible. Given the high value for informing AI existential safety investment, I agree with Yampolsky we should have more people working on this (trying to prove theorems (or creating very rigorous arguments) as to whether alignment is possible or impossible).  

If we knew with very high certainty that alignment is impossible, than that would compel us to invest more resources into 1. bans/regulation on self-improving AI and other forms of dangerous AI (to buy us time)  and 2. figuring out how to survive a world where un-aligned AI is likely to be running rampant soon (for instance maybe we could buy ourselves some time by having humans try to survive in a Mars base or underground bunkers or we could try merging with the AI in hopes of preserving some of what we value that way). 

Yamopolsky and collaborators have a paper on this here (disclaimer - I haven't read it and can't vouch for it's value). 

There may be a nanotech critical point. Getting to full advanced nanotech probably involves many stages of bootstrapping. If lots of nanobots have been designed on a computer, then an early stage of the bootstrapping process might be last to be designed. (Building a great nanobot with a mediocre nanobot might be easier than building the mediocre nanobot from something even worse.) This would mean a sudden transition where one group potentially suddenly had usable nanotech. 

So, can a team of 100 very smart humans, working together, with hand coded nanotech, stop an ASI being created. 

I would be unsurprised if blindly scanning and duplicating a human, to the resolution where memories and personality was preserved, was not that hard with hand coded nanotech. (Like a few researcher months of effort) 

Making nanomachines that destroy GPU's seems not that hard either. 

Nor does making enough money to just buy all the GPU's and top AI talent available.

Actually, find everyone capable of doing AI research, and pay them 2x as much to do whatever they like (as long as they don't publish and don't run their code on non toy problems) sounds like a good plan in general.

"For only $10 000 a year in fine whisky, we can keep this researcher too drunk to do any dangerous AI research." But thousands more researchers like him still spend their nights sober adjusting hyperparameters. That's why were asking you to help in this charity appeal. 

(Idea meant more as interesting wild speculation. Unintended incentives exist. That isn't to say it would be totally useless.) 

If you are really convinced that 
1) AGI is coming really fast.
2) Work on alignment has basically no chances to break through in time.
3) Un aligned AGI result in quick and complete annihilation for humankind
4) You firmly believe in a utilitarism/consequentialism. 
which seems to me to be Eliezer's model,
then you should focus your efforts to launch an all out nuclear war between the USA and China, which would be very unlikely to destroy humanity.
You could even move MIRI to New-Zealand or something so that work on alignment can continue after the nuclear blast.