LessWrong 2.0 Reader

Are there any suggestions for how to get this message across? To all those AI x-risk disbelievers?

Huh. I knew that's how ChatGPT worked but I had assumed they would've worked out a less hacky solution by now!

You've funded a what looks from my vantage point to be a huge portion of the quality-adjusted attempts to avert doom, perhaps a majority. Much appreciation for stepping up for humanity.

Technically the point of going to college is to help you thrive in the rest of your life after college. If you believe in AI 2027, the most important thing for the rest of your life is for AI to be developed responsibly. So, maybe work on that instead of college?

I think the EU could actually be good place to protest for an AI pause. Because the EU doesn't have national AI ambitions, and the EU is increasingly skeptical of the US, it seems to me that a bit of protesting could do a lot to raise awareness of the reckless path that the US is taking. That, in turn, could motivate the EU to apply leverage via ASML, sanctions, etc.

The only thing I'm worried about is that EU criticism of the US could create anti-EU polarization among the GOP in the US, which motivates them to be more reckless on AI. This question seems worth a lot more study.

Have you tried seeing how ChatGPT responds to individual lines of code from that excerpt? There might be an anomalous token in it along the lines of " petertodd" [LW · GW].

I'd say that in most contexts in normal human life, (3) is the thing that makes this less of an issue for (1) and (2). If the thing I'm hearing about it real, I'll probably keep hearing about it, and from more sources. If I come across 100 new crazy-seeming ideas and decide to indulge them 1% of the time, and so do many other people, that's usually, probably enough to amplify the ones that (seem to) pan out. By the time I hear about the thing from 2, 5, or 20 sources, I will start to suspect it's worth thinking about at a higher level.

Ultimately you have to make a bet on your guesses of reality. If your modal guess is civilizational collapse in 2-3 years, skipping uni is hardly a disproportionate action, but at the same time it's not going to win you much either. Personally I'd leave the uni-or-not decision to the plausible worlds where the choice matters more, and look for some higher leverage change you can make for the rest.

I added up the median "Predictions for gap size" in the "How fast can the task difficulty gaps be crossed?" table, summing each set of predictions separately ("Eli", "Nikola", "FutureSearch") to get three numbers ranging from 30-75.

Does this table cover the time between now and superhuman coder? I thought it started at RE-Bench, because:

• I took all of this to be in context of the phrase, about one page back, "For each gap after RE-Bench saturation"
• The earlier explanation that Method 2 is "a more complex model starting from a forecast saturation of an AI R&D benchmark (RE-Bench), and then how long it will take to go from that system to one that can handle real-world tasks at the best AGI company" [emphasis added]
• The first entry in the table ("Time horizon: Achieving tasks that take humans lots of time") sounds more difficult than saturating RE-Bench.
• Earlier, there's a separate discussion forecasting time to RE-bench saturation.

But sounds like I was misinterpreting?

Exactly. More fundamentally, that is not a probability graph, it's a probability density graph, and we're not shown the line beyond 2032 but just have to assume the integral from 2100-->infinity is >10% of the integral from 0-->infinity. Infinity is far enough away that the decay doesn't even need to be all that slow for the total to be that high.

(There’s a funny aside on Thermopylae, and the limits of ‘excellent leadership,’ yes they did well but they ultimately lost. To which I would respond, they only ultimately lost because they got outflanked, but also in this case ‘good leadership’ involves a much bigger edge. A better example is, classically, Cortes, who they mention later. Who had to fight off another Spanish force and then still won. But hey.)

I know this is an aside to your aside, but as an avid Sparta-hater, I want to point out that we don't have much evidence that Spartans are good at military leadership, and indeed plenty of evidence in the other direction:

Sparta's opponents often won using clever, innovative tactics, such as at the Battle of Phyle (404 BC), the Battle of Olpae (426 BC), the Battle of Cyzicus (410 BC), the Battle of Arginusae (406 BC), Battle of Tegyra (375 BC), the Battle of Leuctra (371 BC), and the 2nd Battle of Mantinea (362 BC).  The Spartans, so far as I have found in reviewing these 51 battles, never made a single creative strategic or tactical innovation.  They were sometimes clever; mostly through treachery and trickery.

That said, you have to remember that pretty much all of the primary sources you read on this topic are written by Athenians for Athenians, and less about preserving an accurate historical record for future generations (or even propaganda/promoting internal Athenian solidarity) and more about making specific political points for their own internecine fights. So you should pay more attention to what's objectively verifiable (things like win-loss records, inputs and outputs) and less about the overall vibe that they present.