post by [deleted] · · ? · GW · 0 comments

This is a link post for

0 comments

Comments sorted by top scores.

comment by Zack_M_Davis · 2022-10-13T03:46:04.362Z · LW(p) · GW(p)

we should see our odds of alignment being close to the knife's edge, because those are the situations that require the most computation-heavy simulations to determine the outcome of

No, because "successfully aligned" is a value-laden category. We could be worth simulating if our success probability is close to zero, but there's a lot of uncertainty over which unaligned-with-us superintelligence we create.

Replies from: carado-1
comment by Tamsin Leake (carado-1) · 2022-10-13T08:26:16.663Z · LW(p) · GW(p)

oh, you're absolutely right. thanks for pointing this out.

comment by Dagon · 2022-10-12T21:07:10.890Z · LW(p) · GW(p)

what a strange situation, that we have a chance at all: instead of alignment or superintelligence being discovered many decades apart, we're arriving at them in a somewhat synchronous manner!

It's a lot less strange if you consider that it's probably not actually that close.  We're most likely to fail at one or both topics.  And even if they happen, they're so clearly correlated that it would be strange NOT to see them together.  

Still, I like the exploration of scenarios and the recognition that alignment (or understanding) with the entities outside the simulation is worth thinking about, if perhaps not as useful as thinking about alignment with future agents inside the simulation/reality.