LessWrong 2.0 Reader

Just chiming in that I appreciate this post, and my independent impressions of reading the FSF align with Zach's conclusions: weak and unambitious. 

A couple additional notes:

The thresholds feel high — 6/7 of the CCLs feel like the capabilities would be a Really Big Deal in prosaic terms, and ~4 feel like a big deal for x-risk. But you can't say whether the thresholds are "too high" without corresponding safety mitigations, which this document doesn't have. (Zach)

These also seemed pretty high to me, which is concerning given that they are "Level 1". This doesn't necessarily imply but it does hint that there won't be substantial mitigations — above the current level — required until those capability levels. My guess is that current jailbreak prevention is insufficient to mitigate substantial risk from models that are a little under the level 1 capabilities for e.g., bio. 

GDP gets props for specifically indicating ML R&D + "hyperbolic growth in AI capabilities" as a source of risk. 

Given the lack of commitments, it's also somewhat unclear what scope to expect this framework to eventually apply to. GDM is a large org with, presumably, multiple significant general AI capabilities projects. Especially given that "deployment" refers to external deployment, it seems like there's going to be substantial work to ensuring that all the internal AI development projects proceed safely. e.g., when/if there are ≥3 major teams and dozens of research projects working on fine-tuning highly capable models (e.g., base model just below level 1), compliance may be quite difficult. But this all depends on what the actual commitments and mechanisms turn out to be. This comes to mind after this event a few weeks ago, where it looks like a team at Microsoft released a model without following all internal guidelines, and then tried to unrelease it (but I could be confused). 

I’m just finishing up an intro to physics course for university this semester. I self taught math last year. So I was expecting the hardest part to be the math itself. But actually the hardest part was similar to the start of “what else is there to say”. Understanding that formulas are written with different symbols depending on who is writing them and how they are feeling that day.

Like, why can (s) represent:

• Seconds
• Time itself
• Distance
• Displacement
• Probably a few other things I’m forgetting

The hand rules for magnetic fields all called different things by different people. What!?

Still not finished reading this post. But I’m really enjoying it so far. Hope to see you at LessOnline!

This has been OpenAI's line (whether implicitly or explicitly) for a while iiuc. I referenced it on my Open Asteroid Impact website, under "Operation Death Star." 

Feel free to suggest improvements, it's just what worked for me, but is limited in format

Super helpful! Thanks!

I am limited in my means, but I would commit to a fund for strategy 2. My thoughts were on strategy 2, and it seems likely to do the most damage to OpenAI's reputation (and therefore funding) out of the above options. If someone is really protective of something, like their public image/reputation, that probably indicates that it is the most painful place to hit them.

This also stood out to me as a truly insane quote. He's almost but not quite saying "we have raised awareness that this bad thing can happen by doing the bad thing"

Some ideas:

1. Make Sam Altman look stupid on Twitter, which will marginally persuade more employees to quit and more potential investors not to invest (this is my worst idea but also the easiest, and people seem to pretty much have this one covered already)
2. Pay a fund to hire a good lawyer to figure out a strategy to nullify the non-disparagement agreements. Maybe a class-action lawsuit, maybe a lawsuit on the behalf of one individual, maybe try to charge Altman with some sort of crime, I'm not sure the best way to do this but that's the lawyer's job to figure out.
3. Have everyone call their representative in support of SB 1047, or maybe even say you want SB 1047 to have stronger whistleblower protections or something similar.

Going to message you a suggestion I think.

Please help me find research on aspiring AI Safety folk!

I am two weeks into the strategy development phase of my movement building and almost ready to start ideating some programs for the year.

But I want these programs to be solving the biggest pain points people experience when trying to have a positive impact in AI Safety .

Has anyone seen any research that looks at this in depth? For example, through an interview process and then survey to quantify how painful the pain points are?

Some examples of pain points I've observed so far through my interviews with Technical folk:

• I often felt overwhelmed by the vast amount of material to learn.
• I felt there wasn’t a clear way to navigate learning the required information
• I lacked an understanding of my strengths and weaknesses in relation to different AI Safety areas  (i.e. personal fit / comparative advantage) .
• I lacked an understanding of my progress after I get started (e.g. am I doing well? Poorly? Fast enough?)
• I regularly experienced fear of failure
• I regularly experienced fear of wasted efforts / sunk cost
• Fear of admitting mistakes or starting over might prevent people from making necessary adjustments.
• I found it difficult to identify my desired role / job (i.e. the end goal)
• When I did think I knew my desired role, identifying the specific skills and knowledge required for a desired role was difficult
• There is no clear career pipeline: Do X and then Y and then Z and then you have an A% chance of getting B% role
• Finding time to get upskilled while working is difficult
• I found the funding ecosystem opaque
• A lot of discipline and motivation over potentially long periods was required to upskill
• I felt like nobody gave me realistic expectations as to what the journey would be like