hiAndrewQuinn's Shortform

post by hiAndrewQuinn (hiandrewquinn) · 2025-04-24T03:32:25.597Z · LW · GW · 3 comments

Contents

3 comments

3 comments

Comments sorted by top scores.

comment by hiAndrewQuinn (hiandrewquinn) · 2025-04-24T03:32:25.596Z · LW(p) · GW(p)

There exist very smart, very motivated people in this world who believe they have a moral imperative to replace humanity with some flavor of AI wholesale. I think sometimes we fall to do a very good job of modeling this kind of threat actor. Perhaps this is because it requires a much more conflict-theoretical view of the AI safety situation than most of us are comfortable with.

They often have solid ethical arguments they have refined over years or decades of thinking which can be refuted about at easily as actual professional ethicists' can. (In this author's limited knowledge, ethicists rarely substantively change their stances fundamentally from e.g. moral realism to moral anti-realism or the like. In the reference class of philosophers at large the only people I can remember offhand who did something like this is early vs late Wittgenstein, and I think Quine had a reluctant turn towards mathematical platonism later in his career.)

I know that if and when an aligned ASI emerges the threat from these actors is nullified, but what about before then? Does your model of the world carve out sufficient space and agency for these "bad to the bone" actors? Something to consider. (Reverse the polarity of everything here if you do in fact believe humankind would do well to replace itself like this, of course.)

Replies from: Max Lee
↑ comment by Knight Lee (Max Lee) · 2025-04-24T08:24:13.912Z · LW(p) · GW(p)

Examples

In How it feels to have your mind hacked by an AI [LW · GW], a software engineer fell in love with an AI, and thought oh if only AGI would have her persona, it would surely be aligned.

Long ago Eliezer Yudkowsky believed that [LW · GW] "To the extent someone says that a superintelligence would wipe out humanity, they are either arguing that wiping out humanity is in fact the right thing to do (even though we see no reason why this should be the case) or they are arguing that there is no right thing to do (in which case their argument that we should not build intelligence defeats itself)."

Larry Page allegedly dismissed concern about AI risk as speciesism.

Selection bias

In these examples, the believers eventually realized their folly, and favoured humanity over misaligned AI in the end.[1]

However, maybe we only see the happy endings due to selection bias! Someone who continues to work against humanity won't tell you that they are doing so, e.g. during the brief period Eliezer Yudkowsky was confused he kept it a secret.

So the true number of people working against humanity is unknown. We only know the number of people who eventually snapped out of it.

Nonetheless, it's not worthwhile to start a witch hunt, no matter how suspiciously someone behaves, because throwing such accusations will merely invite mockery.

  1. ^

    At least for Blaked and Eliezer Yudkowsky. I don't think Larry Page ever walked back or denied his statements.

Replies from: Viliam
↑ comment by Viliam · 2025-04-24T10:30:18.781Z · LW(p) · GW(p)

Larry Page allegedly dismissed concern about AI risk as speciesism.

That's what we get for living in a culture where calling something "...ism" wins the debate.