LessWrong 2.0 Reader

I certainly agree it isn't clear, just my current best guess.

It's not entirely clear to me that the math works out for AIs being helpful on net relative to humans just doing it, because of the supervision required, and the trust and misalignment issues.

But on this question (for AIs that are just capable of "prosaic and relatively unenlightened ML research") it feels like shot-in-the-dark guesses. It's very unclear to me what is and isn't possible.

Some landlords offer cheap rent but only rent to childless non-smokers who have a perfect credit score and a well-paid job (preferably a job that requires security clearance or an enhanced DBS check) and whose demographic characteristics are not too similar to those of their previous bad tenants. 

Other landlords are less selective about who they rent to but charge much higher rents to compensate for the risk of property damage and rent arrears. This is why properties that are very similar in quality may be charging at wildly different rents.

On some axes, but won't there to be axes where AIs are more difficult than humans also? Sycophancy&slop being the most salient. Misalignment issues being another.

Yes, I just meant on net. (Relative to the current ML community and given a similar fraction of resources to spend on AI compute.)

1. All of the above but it seems pretty hard to have an impact as a high schooler, and many impact avenues aren't technically "positions" (e.g. influencer)
2. I think that everyone expect "Extremely resilient individuals who expect to get an impactful position (including independent research) very quickly" is probably better off following the strategy.

That's a good point. I've changed it to "wokking".

If I understand correctly it is possible to find $300/mo/bedroom accommodation in rural US today, and a large enough city will compress city rents down to rural rents. A govt willing to pursue a plan as interesting as this one may also be able to increase immigrant labour to build the houses and relax housing regulations. US residential rents are artificially high compared to global average. (In some parts of the world, a few steel sheets (4 walls + roof) is sufficient to count as a house, even water and sewage piping in every house is not mandatory as long as residents can access toilets and water supply within walking distance.)

(A gigacity could also increase rents because it'll increase the incomes of even its lowest income members. But yeah in general now you need to track median incomes of 1B people to find out new equilibrium.)

I found this tricky to parse because of two phrasing issues:

1. The post depends a lot on what you mean by "school" (high school versus undergrad).
2. I feel confused about what claim you're making about the waiting room strategy: you say that some people shouldn't use it, but you don't actually claim that anyone in particular should use it. So are you just mentioning that it's a possible strategy? Or are you implying that it should be the default strategy?

Thanks, I appreciate the draft. I see why it's not plausible to get started on now, since much of it depends on having AGIs or proto-AGIs to play with.

I guess I shouldn't respond too much in public until you've published the doc, but:

• If I'm interpreting correctly, a number of the things you intend to try involve having a misaligned (but controlled) proto-AGI run experiments involving training (or otherwise messing with in some way) an AGI. I hope you have some empathy the internal screaming I have toward this category of things.
• A bunch of the ideas do seem reasonable to want to try (given that you had AGIs to play with, and were very confident that doing so wouldn't allow them to escape or otherwise gain influence). I am sympathetic to the various ideas that involve gaining understanding of how to influence goals better by training in various ways.
• There are chunks of these ideas that definitely aren't "prosaic and relatively unenlightened ML research", and involve very-high-trust security stuff or non-trivial epistemic work.
• I'd be a little more sympathetic to these kinda desperate last-minute things if I had no hope in literally just understanding how to build task-AGI properly, in a well understood way. We can do this now. I'm baffled that almost all of the EA-alignment-sphere has given up on even trying to do this. From talking to people this weekend this shift seems downstream of thinking that we can make AGIs do alignment work, without thinking this through in detail.

The total quantity of risk reduction is unclear, but seems substantial to me. I'd guess takeover risk goes from 50% to 5% if you do a very good job at executing on huge amounts of prosaic and relatively unenlightened ML research at the relevant time

Agree it's unclear. I think the chance of most of the ideas being helpful depends on some variables that we don't clearly know yet. I think 90% risk improvement can't be right, because there's a lot of correlation between each of the things working or failing. And a lot of the risk comes from imperfect execution of the control scheme, which adds on top.

One underlying intuition that I want to express: The world where we are making proto-AGIs run all these experiments is pure chaos. Politically and epistemically and with all the work we need to do. I think pushing toward this chaotic world is much worse than other worlds we could push for right now. 

But if I thought control was likely to work very well and saw a much more plausible path to alignment among the "stuff to try", I'd think it was a reasonable strategy.

I also think that getting the ML community to work on things effectively is probably substantially harder than getting AIs to work on things effectively

On some axes, but won't there to be axes where AIs are more difficult than humans also? Sycophancy&slop being the most salient. Misalignment issues being another.

This work isn't extremely easy to verify or scale up (such that I don't think "throw a billion dollars at it" just works),

This makes sense now. But I think this line should make you worry about whether you can make controlled AIs do it.

The Suchir Balaji Autopsy Report came out recently

Overall, I think it’s quite compelling evidence it was a suicide.

The points against it being suicide are:

1. The somewhat awkward angle of the gun
2. The lack of gunpowder on the skin surrounding the wound
3. The lack of stippling around the wound
4. Some small potential inconsistencies between parent's account and police report (such as the claim by parents that father talked to Suchir by phone on Friday)
5. Had a project to live for as a whistleblower against OpenAI
6. Suchir’s family insists he wasn’t suicidal; he seemed overall to be living a good life

The points for it are:

1. No signs of forced entry or struggle
2. The gun was purchased by and registered to Suchir
3. There was gunpowder residue on Suchir's hands and Suchir's DNA was on the gun
4. The door to his apartment was locked
5. Internet searches on his computer related to “total gray matter volume” and “white matter” in brain
6. The toxicology report shows he was drunk
7. The base-rate of suicide is much higher than homicide
8. Theories about who would have a motivation to kill him seem far-fetched to me; OpenAI is getting much worse press form the suicide than they likely would have if he hadn’t killed himself

General notes summarizing the report:

• Suchir was found by the police on Tuesday, 11/26/24, at his apartment at 188 Buchanan Street Apartment #409; as part of a wellness check (requested by his mother)
• Apartment was locked with a deadbolt, no signs of forced entry to the apartment or signs of struggle within the apartment; desktop computer with searchers for "total gray matter volume" and "white matter"; key for apartment was found in unit
• Last communicated with his mother via text on Friday, 11/22/2024 (the parents in interviews have claimed that their last contact with him was when he talked with his father by phone on Friday night but the police report doesn’t mention this)
• Death was from gunshot wound to the head; entry point at the glabella of the forehead (between the eyes), centered approximately 13 cm below the top of the head and 0.5 cm right of the anterior midline
• Bullet traveled down and back and into the neck, where it was recovered; soot and unburned gunpowder was not readily visible on the skin, no gunpowder stippling was found around the wound
• Glock pistol found at scene; 4 live rounds, 1 spent casing; pistol was purchased and registered by Suchir on 01/04/2024; gunpowder residue was found on both of Suchir’s hands; Suchir's DNA was found on the pistol; markings on the bullet recovered during examination matched unique striations left by the pistol's barrel
• Toxicology report showed .178 BAC level (very intoxicated); blood Amphetamine levels between 35 ng/mL and 39 ng/mL (not very notable); GHB levels between 54,000 ng/mL and 67,000 ng/mL (apparently this is within normal ranges or how much buildup of GHB you might naturally expect given the body had been decaying for a few days)
• Decomposition of the body was moderate when found.