Disagreement on AGI Suggests It’s Near
post by tangerine · 2025-01-07T20:42:43.456Z · LW · GW · 7 commentsContents
7 comments
If I’m planning a holiday to New York (and I live pretty far from New York), it’s quite straightforward to get fellow travellers to agree that we need to buy plane tickets to New York. Which airport? Eh, whichever is more convenient, I guess. Alternatively, some may prefer a road trip to New York, but the general direction is obvious for everyone.
However, as the holiday gets closer in time or space, the question of what we actually mean by a holiday in New York becomes more and more contentious. Did we mean New York State or New York City? Did we mean Brooklyn or Broadway? Which Broadway theater? Which show? Which seats?
By my estimation, the fact that the question of whether AGI has been achieved is so broadly contentious shows that we are so close to it that the term has lost its meaning, in the same way that “Let’s go to New York!” loses its meaning when you’re already standing in Times Square.
It’s time for more precise definitions of the space of possible minds that we are now exploring. I have my own ideas, but I’ll leave those for another post…
7 comments
Comments sorted by top scores.
comment by leogao · 2025-01-08T07:03:25.847Z · LW(p) · GW(p)
people disagree heavily on what the second coming will look like. this, of course, means that the second coming must be upon us
Replies from: tangerine↑ comment by tangerine · 2025-01-08T10:58:23.408Z · LW(p) · GW(p)
You’re kind of proving the point; the Second Coming is so vaguely defined that it might as well have happened. Some churches preach this.
If the Lord Himself did float down from Heaven and gave a speech on Capitol Hill, I bet lots of Christians would deride Him as an impostor.
Replies from: leogao, sharmake-farah, gwern↑ comment by leogao · 2025-01-08T19:51:52.179Z · LW(p) · GW(p)
suppose I believe the second coming involves the Lord giving a speech on capitol hill. one thing I might care about is how long until that happens. the fact that lots of people disagree about when the second coming is doesn't mean the Lord will give His speech soon.
similarly, the thing that I define as AGI involves AIs building Dyson spheres. the fact that other people disagree about when AGI is doesn't mean I should expect Dyson spheres soon.
Replies from: sharmake-farah↑ comment by Noosphere89 (sharmake-farah) · 2025-01-08T19:57:20.452Z · LW(p) · GW(p)
My response to this is to focus on when a Dyson Swarm is being built, not AGI, because it's easier to define the term less controversially.
And a large portion of disagreements here fundamentally revolves around being unable to coordinate on what a given word means, which from an epistemic perspective doesn't matter at all, but it does matter from a utility/coordination perspective, where coordination is required for a lot of human feats.
↑ comment by Noosphere89 (sharmake-farah) · 2025-01-08T15:34:17.785Z · LW(p) · GW(p)
The actual Bayesian response would be for both the AGI case and the Second Coming case is that both hypotheses are invalid from the start due to underspecification, so any probability estimates/decision making for utility for these hypotheses are also invalid.
↑ comment by gwern · 2025-01-08T20:59:05.537Z · LW(p) · GW(p)
Specifically, as an antichrist, as the Gospels specifically warn that "false messiahs and false prophets will appear and produce great signs and omens", among other things. (And the position that the second coming has already happened - completely, not merely partially - is hyperpreterism.)
comment by Fer32dwt34r3dfsz (rodeo_flagellum) · 2025-01-07T22:31:28.249Z · LW(p) · GW(p)
Agree. This post captures the fact that, time and again, historical and once perceived as insurmountable benchmarks in AI have been surpassed. Those not fully cognizant of the situation have been iteratively surprised. People, for reasons I cannot fully work out, will continue to engage in motivated reasoning against current and near-term-future-expected AI capabilities and or economical value, with some part of the evidence-downplaying consisting of shifting AGI-definitional or capability-threshold-to-impress goalposts (see moving goalposts). On a related note, your post also makes me imagine the apologue of the boiling frog of late w.r.t. scaling curves.