Introducing myself: Henry Lieberman, MIT CSAIL, whycantwe.org

The problem of whether the goals and values of an artificially intelligent agent will align with human goals and values, can be reduced to this problem: Will the goals and values of different human agents ever align with each other?

I would be ecstatic if AI turned out to be perfectly aligned with any particular human, rather than aligned with no human at all. The problem of "oh no what if Deepmind disagrees with the chinese about which human values to put in the AI" is rather small compared with the problem of actually figuring out how to put any values at all in the AI.

If the context was instead the problem of sending a rocket to the moon, you would have assumed away the actual engineering of the rocket and would now be concerned with the human squabbling about the particular destination crater.

The problem of whether the goals and values of an artificially intelligent agent will align with human goals and values, can be reduced to this problem: Will the goals and values of different human agents ever align with each other?

We can only program AI “in our own image”, so both the features and bugs of humanity will reappear in AI.

Why do you think these statements are true?

The problem of whether the goals and values of an artificially intelligent agent will align with human goals and values, can be reduced to this problem: Will the goals and values of different human agents ever align with each other?

Aligning human agents is a subproblem, but if you align human agents you don't automatically align agents in a world where the most powerful agents aren't human agents.

Schizophrenia is the wrong metaphor here -- it's not the same disease as split personalities (i.e. dissociative identity disorder). I think it would be clearer and more accurate to rewrite that paragraph without it. I don't intend this as an attack or harsh criticism, it's just that I have decided to be a pedant about this point whenever I encounter it, as I think it would be good for the general public to develop a more accurate and realistic understanding of schizophrenia.

Then, we’ll get benevolent AI “for free”.

I am heavily skeptical of this claim to the extent that I genuinely think I misunderstood you. I do not see why, for example, being good at recognizing pictures of cats should make me good at programming a machine to recognize pictures of cats, so I think the general form of the argument I understood is wrong.

Thanks for sharing your ideas. I'm a bit confused about your core claim and would love if you could could clarify (Or refer to the specific part of your writing that addresses these questions): I get the general gist of your claim, that AI alignment depends on whether humans can all have the same values, but I don't know how much 'the same' you mean. You say 'substantially' align, could you give some examples of how aligned you mean? For example, do you mean all humans sharing the same political ideology (libertarian/communist/ etc)? Do you mean that for all non-trivial ethical questions (When is abortion permissable? How much duty do you have to your family vs yourself? How many resources should we devote to making things better on earth vs exploring space?), that you would need to be able to ask any human on earth and say 99% would give you the same answer?

Likewise with the idea of humans needing to compete less and cooperate more. How much less and more? For example, competition between firms is a core part of capitalism, do you think we need to completely eliminate capitalism? Or do you only mean eliminating zero/negative sum competition like war?