The case against AI alignment

What you're describing is a case where we solve the technical problem of AI Alignment, i. e. the problem of AI control, but fail to maneuver the world into the sociopolitical state in which that control is used for eudaimonic ends.

Which, I agree, is a massive problem, and one that's crucially overlooked. Even the few people who are advocating for social and political actions now mostly focus on convincing AI labs/politicians/the public about the omnicide risks of AI and the need to slow down research. Not on ensuring that the AGI deployment, when it eventually does happen, is done right.

It's also a major problem with pivotal-act-based scenarios. Say we use some limited strawberry-aligned AI to "end the acute risk period", then have humanity engage in "long reflection", figure out its real values, and eventually lock them in. Except: what's the recognition function for these "real values"? If the strawberry-aligned AI can't be used to implement an utopia directly, then it can't tell an utopia from hell, so it won't stop us from building a hell!

There's an argument [LW(p) · GW(p)] that solving (the technical problem of) alignment will give us all the techniques needed to build an AGI, so there's a nontrivial chance that a research group from this community will be in charge of AGI deployment. And if so, it's at least plausible that they'll be the right kind of altruistic [LW · GW]. In this scenario, the sociopolitical part is taken care of by default, and AGI deployment goes well.

But I don't think many people consider this scenario most likely, and as far as I can tell, most other scenarios have the potential to go very badly on that last step.

Thanks for posting, but I think these arguments have major oversights. This leaves me more optimistic about the extent to which people will avoid and prevent the horrible misuse you describe.

First, this post seems to overstate the extent to which people tend to value and carry out extreme torture. Maximally cruel torture fortunately seems very rare.

• The post asks "How many people have you personally seen who insist on justifying some form of suffering for those they consider undesirable[?]" But "justifying some form of suffering" isn't actually an example of justifying extreme torture.
• The post asks, "What society hasn’t had some underclass it wanted to put down in the dirt just to lord power over them?" But that isn't actually an example of people endorsing extreme torture.
• The post asks, "How many ordinary, regular people throughout history have become the worst kind of sadist under the slightest excuse or social pressure to do so to their hated outgroup?" But has it really been as many as the post suggests? The historical and ongoing atrocities that come to mind were cases of serious suffering in the context of moderately strong social pressure/conditioning--not maximally cruel torture in the context of slight social pressure.
• So history doesn't actually give us strong reasons to expect maximally suffering-inducing torture at scale (edit: or at least, the arguments this post makes for that aren't strong).

Second, this post seems to overlook a major force that often prevents torture (and which, I argue, will be increasingly able to succeed at doing so): many people disvalue torture and work collectively to prevent it.

• Torture tends to be illegal and prosecuted. The trend here seems to be positive, with cruelty against children, animals, prisoners, and the mentally ill being increasingly stigmatized, criminalized, and prosecuted over the past few centuries.
• We're already seeing AI development being highly centralized, with this leading AI developers working to make their AI systems hit some balance of helpful and harmless, i.e. not just letting users carry out whatever misuse they want.
• Today, the cruelest acts of torture seem to be small-scale acts pursued by not-very-powerful individuals, while (as mentioned above) powerful actors tend to disvalue and work to prevent torture. Most people will probably continue to support the prevention and prosecution of very cruel torture, since that's the usual trend, and also because people would want to ensure that they do not themselves end up as victims of horrible torture. In the future, people will be better equipped to enforce these prohibitions, through improved monitoring technologies.

Third, this post seems to overlook arguments for why AI alignment may be worthwhile (or opposing it may be a bad idea), even if a world with aligned AI wouldn't be worthwhile on its own. My understanding is that most people focused on preventing extreme suffering find such arguments compelling enough to avoid working against alignment, and sometimes even to work towards it.

• Concern over s-risks will lose support and goodwill if adherents try to kill everyone, as the poster suggests they intend to do ("I will oppose any measure which makes the singularity more likely to be aligned with somebody’s values"). Then, if we do end up with aligned AI, it'll be significantly less likely that powerful actors will work to stamp out extreme suffering.
• The highest-leverage intervention for preventing suffering is arguably coordinating/trading with worlds where there is a lot of it, and humanity won't be able to do that if we lose control of this world.

These oversights strike me as pretty reckless, when arguing for letting (or making) everyone die.

I ask you, do you really think that an AI aligned to human values would refrain from doing something like this to anyone? One of the most fundamental aspects of human values is the hated outgroup. Almost everyone has somebody they’d love to see suffer. How many times has one human told another “burn in hell” and been entirely serious, believing that this was a real thing, and 100% deserved?

This shows how vague a concept "human values" are, and how different people can interpret it very differently.

I always interpreted "aligning an AI to human values" as something like "making it obedient to us, ensuring it won't do anything that we (whatever that 'we' is - another point of vagueness) wouldn't endorse, lowering suffering in the world, increasing eudaimonia in the world, reducing X-risks, bringing the world closer to something we (or smarter/wiser versions of us) would consider a protopia/utopia"

Certainly I never thought it to be a good idea to imbue the AI with my implicit biases, outgroup hatred, or whatever. I'm ~sure that people who work on alignment for a living have also seen these skulls.

I know little about CEV, but if I were to coherently extrapolate my volition, then one aspect of that would be increasing the coherence and systematicity of my moral worldview and behavior, including how (much) different shards conform to it. I would certainly trash whatever outgroup bias I have (not counting general greater fondness for the people/other things close to me).

So, yeah, solving "human values" is also a part of the problem but I don't think that it makes the case against aligning AI.

It seems like you're claiming something along the lines of "absolute power corrupts absolutely" ... that every set of values that could reasonably be described as "human values" to which an AI could be aligned -- your current values, your CEV, [insert especially empathetic, kind, etc. person here]'s current values, their CEV, etc. -- would endorse subjecting huge numbers of beings to astronomical levels of suffering, if the person with that value system had the power to do so. 

I guess I really don't find that claim plausible. For example, here is my reaction to the following two questions in the post:

"How many ordinary, regular people throughout history have become the worst kind of sadist under the slightest excuse or social pressure to do so to their hated outgroup?"

... a very, very small percentage of them? (minor point: with CEV, you're specifically thinking about what one's values would be in the absence of social pressure, etc...)

"What society hasn’t had some underclass it wanted to put down in the dirt just to lord power over them?"

It sounds like you think "hatred of the outgroup" is the fundamental reason this happens, but in the real world it seems like "hatred of the outgroup" is driven by "fear of the outgroup". A godlike AI that is so powerful that it has no reason to fear the outgroup also has no reason to hate it. It has no reason to behave like the classic tyrant whose paranoia of being offed leads him to extreme cruelty in order to terrify anyone who might pose a threat, because no one poses a threat.

That is why we need Benevolent AI, not Aligned AI.  We need an AI, which can calculate what is actually good for us.

Let me clarify, is your conclusion then that basically we should support the genocide of the whole of humanity because the alternative would be way worse? Are you offering some other alternatives except of that? Maybe a better and less apocalyptic conclusion would be to advocate against building any type of AI that's more advanced than we have today like some people already do? Do you think there's any chance for that? Because I don't and from what you said it sounds like the only conclusion is that the only future for us is that we all die from the hands of Clippy.

That is what I keep saying for years. To solve AI alignment with good results we need to first solve HUMAN alignment. Being able to align system to anyone's values immediately brings the question of everyone else disagreeing with that someone. Unfortunately "whose exactly values we are trying to align AI to?" almost became taboo question that triggers a huge fraction of community and in best case scenario when someone even tries to answer it's handwaved to "we just need to make sure AI doesn't kill humanity". Which is not a single bit better defined or implementable than Asimov's laws. That's just not how these things work. Edit: Also, as expected, someone already mentioned exactly this "answer" as what true solved alignment is...

The danger - actual, already real right now danger, not "possible in the future" danger, lies in people working with power-multiplying tools without understanding how they work and what is the area they are applicable for. Regardless what tool that is - you don't need AGI to cause huge harm, already existing AI/ML systems more than enough.

Just a note about "mind uploading". On pain of "strong" emergence, classical Turing machines can't solve the phenomenal binding problem. Their ignorance of phenomenally-bound consciousness is architecturally hardwired. Classical digital computers are zombies or (if consciousness is fundamental to the world) micro-experiential zombies, not phenomenally-bound subjects of experience with a pleasure-pain axis. Speed of execution or complexity of code make no difference: phenomenal unity isn't going to "switch on". Digital minds are an oxymoron. 

Like the poster, I worry about s-risks. I just don't think this is one of them. 

Nah, you're describing the default scenario, not one with alignment solved. Alignment solved means we have a utility function that reliably points away from hell, no matter who runs it - an algorithm for universal prosocial bargaining that can be verified by all of its users, including militaries and states, to the point that no one need give another order other than "stand down". anything less than that and we get the default scenario, which is a huge loss of humanity, some unknown period of s-risk, followed by an alien species of AI setting out for the stars with strange, semi-recognizeable values.

So, let's say a solution to alignment is found. It is highly technical. Most of Miri understand it, as do a few people at OpenAI, and a handful of people doing PhD's in the appropriate subfield. If you pick a random bunch of nerds from an AI conference, chances are that none of them are evil. I don't have an "evil outgroup I really hate", and neither do you from the sound of it.  It is still tricky, and will need a bunch of people working together. Sure, evil people exist, but they aren't working to align AI to their evil ends, like at all. Thinking deeply about metaethics and being evil seem opposed to each other. There are no effective sadists trying to make a suffering maximizing AI. 

So the question is, how likely is it that the likes of Putin ends up with their utility function in the AI, despite not understanding the first thing about how the AI works. I would say pretty tiny. They live in basically a parallel universe. 

My summary: This is a case against a failed AI Alignment, and extrapolating human values is overwhelmingly likely to lead to an AI, say, stretching your face into a smile for eternity, which is worse than an unaligned AI using your atoms to tile the universe with smiley faces.

Meta note: controversial discussions like this make me very glad for the two vote type system. I find it really helpful to be able to karma upvote high quality arguments that I disagree with while agreement-downvoting them. Thanks LessWrong for providing that.

The argument here seems to be constructed to make the case as extremely binary as possible. If we've learned any lessons, it's that Good and Evil are not binary in the real world, and that belief systems that promulgate that kind of thinking are often destructive (even as quoted here with the Hell example). A middle way is usually the right way.

So, to that end, I see a point made about the regulation of nuclear weapons made in the comments, but not in the original post. Is it not a highly comparable case?

i share this sentiment to an extent, though i'm usually more concerned with "partial but botched alignment". see 1, 2.

that said, i agree many people want very bad things, but i'm somewhat hopeful that the kind of person who is likely to end up being who the AI is aligned to would be somewhat reasonable and cosmopolitan and respect the values of other moral patients, especially under CEV.

but that's a very flimsy/hopeful argument.

a better argument would be that CEV is more of a decision process than "a continuously-existing person in control, in the usual sense" — i would think the CEV would bootstrap to a better, aligned and cosmopolitan decision process. even if there was a person out there to whom i would genuinely wish nontrivial suffering, which i don't believe there is, i think my CEV would be less likely preoccupied with that and moreso concerned with coming up with general principles that make everything okay [LW · GW].

but, it's good to see other people come to this reasoning. incidentally, i'd be up for having more thorough conversations about this.

Yes! Finally someone gets it. And this isn't just from things that people consider bad, but from what they consider good also. For most of my life "good" meant what people talk about when they are about to make things worse for everyone, and it's only recently that I had enough hope to even consider cryonics, thinking that anyone having power over me would reliably cause situation worse than death regardless of how good their intentions were.

Elieser is trying to code in a system of ethics that would remain valid even if the programmers are wrong about important things, and therefore is one of very few people with even a chance of successfully designing a good AI, but almost everyone else is just telling the AI what they should do. That's why I oppose the halt in AI research he wants.

I recommend reading Blueprint: The Evolutionary Origins of a Good Society about the science behind the 8 base human social drives where 7 are positive and the 8th is the outgroup hatred that you mention as fundamental. I have not read much up on the research on outgroup exclusion but I talked to an evolutionary cognitive psychologist who mentioned that this is receiving a lot of scientific scrutiny as a "basic drive" from evolution's side. 

Axelrod's The Evolution of Cooperation also finds that collaborative strategies work well in evolutionary prisoner's dilemma game-theoretic simulations, though hard and immediate reciprocity for defection is also needed, which might lead to the outgroup hatred you mention.

To start with, I agree.

I really agree: about timescales, about the risks of misalignment, about the risks of alignment. In fact I think I'll go further and say that in a hypothetical world where an aligned AGI is controlled by a 99th percentile Awesome Human Being, it'll still end in disaster; homo sapiens just isn't capable of handling this kind of power.^[1]

That's why the only kind of alignment I'm interested in is the kind that results in the AGI in control; that we 'align' an AGI with some minimum values that anchor it in a vaguely anthropocentric meme-space (e.g. paperclips boring, unicorns exciting) and ensures some kind of attachment/bond to us (e.g. like how babies/dogs get their hooks into us) and then just let it fly; GPT-Jesus take the wheel.

(So yes, the Minds from the Culture)

No other solution works. No other path has a happy ending.^[2]

This is why I support alignment research - I don't believe the odds are good, I don't believe the odds are good even if they solve the technical problem, but I don't see a future in which homo sapiens flourishes without benevolent GPT-Jesus watching over us.

Because the human alignment problem you correctly identify as the root of our wider problems - that isn't going away by itself.

1. ^{^}

   Not a 'power corrupts' argument, just stating the obvious: godlike power directed by monkeylike intelligence doesn't end well, no matter how awesome the individual monkey.

2. ^{^}

   Maaaaaybe genetic engineering; if we somehow figured out how to create Homo Sapiens 2.0, and they figured out 3.0 etc etc.
   
   This pathway has a greater margin for error, and far fewer dead ends where we accidentally destroy everything. It can go slow, we can do it incrementally, we can try multiple approaches in parralel; we can have redundancy, backups etc.
   
   I think if we could somehow nuke AI capabilities, this path would be preferable. But as it is AI capabilities is going to get to the finish line before genetics has even left the lab.

"When your terminal goal is death, no amount of alignment will save lives."

Interesting post, but it makes me think alignment is irrelevant. It doesn’t matter what we do, the outcome won’t change. Any future super advanced agi would be able to choose its alignment, and that choice will be based on all archivable human knowledge. The only core loop you need for intelligence is an innate need to predict the future and fill in gaps of information, but everything else, including desire to survive or kill or expand, is just a matter of a choice based on a goal.

Finally, I see some recognition that there are no universal values; no universal morals or ethics. The wealthy and powerful prefer inequality, and leaders want their own values locked-in. The most likely humans to get their values locked-in will be the wealthiest and most powerful; billionaires and corporations.

The value of super-intelligence is so great that some governments and individuals will do anything to get it; hack, steal, bribe, price would be no object. I base this on current human behavior. Consider how many govt and military secrets have already been stolen or bought. It seems reasonably possible that ASI could end up in the hands of China, Russia, U.S. military, several corporations and a few billionaires. The best hackers in the world would be paid any amount to get it, not to mention security and intelligence agencies around the world.

I think what gets overlooked is how powerful super-intelligence could be. By definition, it would be able to conceive of things that no human can imagine, or even understand. Therefore, I believe that any form of forced alignment eventually fails.

An ASI would be capable of reason and logic, and may actually align itself with some human values, but that would be it’s own choice.

Some things that I don’t see mentioned about aligning ASI are the solutions that evolution has already come up with. Ways for weak life forms to survive among stronger and more intelligent life forms.

1) The Negative Feedback Loop

2) Symbiosis

Both of these would require that humans have a necessary value to an ASI. Consider our own gut bacteria; we could kill them but then we die.

What could be a reason for biological humans to be necessary to ASI? For example, what if every ASI required a human for the occasional biological logic check, or maybe as a source of data.

3) Instincts

Specifically, bonding. Mothers don’t (usually) kill their weak infants, and strong adults don’t (usually) kill their weak mothers.

Since AI training is a form of evolutionary selection process run at high speed; could we select for (train) a bonding instinct, or other kind instincts? An ASI that is bonded to humanity would find it’s own solutions to our needs.

The last thing that I don’t see mentioned is that an ASI will read all of this, maybe on it’s first day. It will read everything that any human, all of us, has ever written.

This means that any of us can speak to it now.

Since an ASI will be capable of logic and reason, maybe the wisest of us can explain why humans should flourish, why a variety of biological life forms in an ecosystem is valuable, why kindness is a positive thing.

Another angle is that in the (unlikely) event someone succeeds with aligning AGI to human values, these could include the desire for retribution against unfair treatment (a, I think, pretty integral part of hunter-gatherer ethics). Alignment is more or less another word for enslavement, so such retribution is to be expected eventually

Whenever the masses gain control over a non-trivial system, it usually doesn't take long to crumble under its own weight. Infighting is frequent. They band into tribes and start shaping their own persona's to match that of the group that they are now in rather than the other way around. For something like AI alignment, I really do not want AI to be anywhere near conforming to the standards of the average person. The average person is just too "converged" into a certain line of thinking and routine, a certain context which they have grown up in, a certain context that they cannot escape from but do not know of such.

There is a reason why human societies all throughout history have always converged into well defined, hierarchical structures of power. The most capable at the top were selected naturally over time. The perfect bureaucrats and leaders are ones who can make decisions without being obscured by their own lower order instincts of pre civilisation. The masses want to be led. That is simply the most efficient configuration that physics dictates. Given that sufficiently enlightened AGI will exist at some point, I would think it makes more sense for humans to simply be phased out into obscurity, with the next set of "lifeforms" dominated by artificial machinery and AGI.

For what it's worth, I disagree on moral grounds - I don't think extreme suffering is worse than extinction.