LessWrong 2.0 Reader

Canon already acknowledges that it might be detrimental. "Sometimes I think we Sort too early."

If you want to get an informed opinion on how the general public perceives PauseAI, get a t-shirt and hand out some flyers in a high foot-traffic public space.  If you want to be formal about it, bring a clipboard, track whatever seems interesting in advance, and share your results.  It might not be publishable on an academic forum, but you could do it next week.

Here's what I expect you to find, based on my own experience and the reports of basically everyone who has done this:
- No one likes flyers, but get a lot more interested if you can catch their attention enough to say it's about AI.
- Everyone hates AI.
- Your biggest initial skepticism will be from people who think you are in favor of AI.
- Your biggest actual pushback will be from people who think that social change is impossible.
- Roughly 1/4 to 1/2 are amenable to (or have already heard about!) x-risk, most of the rest won't actively disagree but you can tell that particular message is not really "landing" and pay a lot more attention if you talk about something else (unemployment, military applications, deepfakes, etc.)
- Bring a clipboard for signups.  Even if recruitment isn't your goal, if you don't have one you'll feel unprepared when people ask about it.

Also, protests are about Overton-window shifting, making AI danger a thing that is acceptable to talk about.  And even if it makes a specific org look "fringe" (not a given, as Holly has argued), that isn't necessarily a bad thing for the underlying cause.  For example, if I see an XR protest, my thought is (well, was before I knew the underlying methodology): "Ugh, those protestors...I mean, I like what they are fighting for and more really needs to be done, but I don't like the way they go about it"  Notice that middle part.  Activation of a sympathetic but passive audience was the point.  That's a win from their perspective.  And the people who are put off by methods then go on to (be more likely to) join allied organizations that believe the same things but use more moderate tactics.  The even bigger win is when the enthusiasm catches the attention of people who want to be involved but are looking for orgs that are the "real deal," as measured by willingness to put effort where their words are.

I agree my point is less important if we get ASI by 2030, compared to if we don’t get ASI. 

That being said, the arms race can develop over the timespan of years not decades. 6-year superhumans will prompt people to create the next generation of superhumans, and within 10-15 years we will have children from multiple generations where the younger generation have edits with stronger effect sizes. Once we can see the effects on these multiple generations, people might go at max pace.  

Will we ever have Poké Balls in real life? How fast could they be at storing and retrieving animals? Requirements:

• Made of atoms, no teleportation or fantasy physics.
• Small enough to be easily thrown, say under 5 inches diameter
• Must be able to disassemble and reconstruct an animal as large as an elephant in a reasonable amount of time, say 5 minutes, and store its pattern digitally
• Must reconstruct the animal to enough fidelity that its memories are intact and it's physically identical for most purposes, though maybe not quite to the cellular level
• No external power source
• Works basically wherever you throw it, though it might be slower to print the animal if it only has air to use as feedstock mass or can't spread out to dissipate heat
• Should not destroy nearby buildings when used
• Animals must feel no pain during the process

It feels pretty likely to me that we'll be able to print complex animals eventually using nanotech/biotech, but the speed requirements here might be pushing the limits of what's possible. In particular heat dissipation seems like a huge challenge; assuming that 0.2 kcal/g of waste heat is created while assembling the elephant, which is well below what elephants need to build their tissues, you would need to dissipate about 5 GJ of heat, which would take even a full-sized nuclear power plant cooling tower a few seconds. Power might be another challenge. Drexler claims you can eat fuel and oxidizer, turn all the mass into basically any lower-energy state, and come out easily net positive on energy. But if there is none available you would need a nuclear reactor.

Magicians don't pick locks on stage.

The locks are all fake. The test of skill is convincing you they're real.

Except when the locks are totally real and the magician just bypasses them.

Generally it's the former, or someone who is faintly AI aware but not so interested in delving into the consequences. However, I'd like to represent my true opinions which involve significant AI driven disruption, hence the need for a caveat.

Standard objection: Genetic engineering takes a lot of time till it has any effect. A baby doesn't develop into an adult over night. So it will almost certainly not matter relative to the rapid pace of AI development.

I'm curious as to the viewpoint of the other party in these conversations? If they're not aware of/interested in/likely to be thinking about the disruptive effects of AI, then I would usually just omit mentioning it. You know you're conditioning on that caveat, and their thinking does so without them realizing it.

If the other party is more AI-aware, and they know you are as well, you can maybe just keep it simple, something like, "assuming enough normality for this to matter."

I'm curious now, given how accurate your forecasts have turned out and maybe taking into account Jonny's remark that "the predictions are (to my eye) under-optimistic on capabilities", what are the most substantive changes you'd make to your 2025-26 predictions? 

The highlights are officially called "text fragments" and the syntax is described here: https://developer.mozilla.org/en-US/docs/Web/URI/Reference/Fragment/Text_fragments