LessWrong 2.0 Reader

I think 1 and 2 are actually pretty likely, but 3 and 4 is where I'm a lot less confident in actually happening.

A big reason for this is that I suspect one of the reasons people aren't reacting to AI progress is they assume it won't take their job, so it will likely require massive job losses for humans to make a lot of people care about AI, and depending on how concentrated AI R&D is, there's a real possibility that AI has fully automated AI R&D before massive job losses begin in a way that matters to regular people.

Just ask a LLM. The author can always edit it, after all.

My suggestion for how such a feature could be done would be to copy the comment into a draft post, add LLM-suggested title (and tags?), and alert the author for an opt-in, who may delete or post it.

If it is sufficiently well received and people approve a lot of them, then one can explore optout auto-posting mechanisms, like "wait a month and if the author has still neither explicitly posted it nor deleted the draft proposal, then auto-post it".

Done; thanks!

Oh interesting, thanks for the feedback. I think I illusion-of-transparency'd that people would feel fine about arriving in the 6:15-6:30 window. In my head the group discussions start at about 6:30. I'll make a note to update the description hopefully for next time.

Based on my prior experience running meetups, a 15m gap between 'doors open' and starting the discussion is too short. 30m is the practical minimum; I prefer 45-60m because I optimize for low barrier to entry (as a means of being welcoming).

I also find this to be a significant barrier in participating myself, as targeting a fifteen-minute window for arrival is usually beyond my planning abilities unless I have something else with a hard end time within the previous half-hour.

I don't think so; even if it applies to the subset of hypothetical superintelligences that factor neatly into beliefs and values, humans don't seem to factorize this way (see Obliqueness Thesis [LW · GW], esp. argument from brain messiness).

So for this argument to be worth bringing up in some general context where a pause is discussed, the person arguing it should probably believe:

1. We are far and away most likely to get a pause only as a response to unemployment.
2. An AI that precipitates pause-inducing levels of unemployment is inches from automating AI R+D.
3. The period between implementing the pause and massive algorithmic advancements is long enough that we're able to increase compute stock...
4. ....but short enough that we're not able to make meaningful safety progress before algorithmic advancements make the pause ineffective (because, i.e., we regulated FLOPS and it just now takes 100x fewer FLOPS to build the dangerous thing).

I think the conjunct probability of all these things is low, and I think their likelihood is sensitive to the terms of the pause agreement itself. I agree that the design of a pause should consider a broad range of possibilities, and try to maximize its own odds of attaining its ends (Keep Everyone Alive).

I'm also not sure how this goes better in the no-pause world? Unless this person also has really high odds on multipolar going well and expects some Savior AI trained and aligned in the same length of time as the effective window of the theoretical pause to intervene? But that's a rare position among people who care about safety ~at all; it's kind of a George Hotz take or something...

(I don't think we disagree; you did flag that this as "...somewhat relevant in worlds where..." which is often code for "I really don't expect this to happen, but Someone Somewhere should hold this possibility in mind." Just want to make sure I'm actually following!)

I think the Conclusion could serve well as an abstract

An abstract which is easier to understand and a couple sentences at each section that explain their general meaning and significance would make this much more accessible

“By then I knew that everything good and bad left an emptiness when it stopped. But if it was bad, the emptiness filled up by itself. If it was good you could only fill it by finding something better.”

• Hemingway, A Moveable Feast