The two missing core reasons why aligning at-least-partially superhuman AGI is hard

post by Joel Burget (joel-burget) · 2022-04-19T17:15:23.965Z · LW · GW · 2 comments

This is a question post.

Contents

2 comments

From Arbital's Mild Optimization page:

Mild optimization relates directly to one of the three core reasons why aligning at-least-partially superhuman AGI is hard - making very powerful optimization pressures flow through the system puts a lot of stress on its potential weaknesses and flaws.

I'm interested in this taxonomy of core reasons. Unfortunately this page doesn't specify the other two. What are they?

Also, this page is part of the AI alignment domain -- was it written by Eliezer? (surprisingly, "10 changes by 3 authors" is a link to edit and does not show author information or edit history)

Answers

2 comments

Comments sorted by top scores.