LessWrong 2.0 Reader

Yeah, the thing the 'scaling extrapolation' view doesn't take into account is that as soon as radical speed-ups to algorithmic research are made possible by AI R&D agents, suddenly the trendlines for algorithmic progress should be projected to steepen. How much and for how long before slow-downs are hit? That's unclear. I think there is at least some substantial probability that no slow-downs are hit before full AGI, and some smaller but still considerable probability that the improvement cycle rushes forward at high speed past that point to ASI.

This should be assumed to potentially involve dramatic gains in both peak capabilities, and in efficiency and speed of training and inference. If so, then compute governance becomes completely irrelevant for blocking creation of dangerously powerful AI. It can still help put limits on the amount of inference used. Why? Because no matter how efficient the AI is, if you have more compute you have more parallel copies (and can run them faster up to the limits of the system, which is probably somewhere between 100x to 1000x human thought speed).

If we are going to head this off, we need new governance methods, and soon. Maybe really really soon, like, before the end of 2025. Hopefully we have until more like 2028, but we can't count on that for sure.

I have very little faith in current governments to implement and enforce policies that are more complex than things on the order of governance compute and chip export controls. Much less to do so within the short timeframes we are facing.

I think the conclusion this points towards is that we need new forms of governance. Not to replace existing governments, but to complement them. Voluntary mutual inspection contracts with privacy-respecting technology using AI inspectors. Something of that sort.

Here's some recent evidence of compute thresholds not being reliable: https://novasky-ai.github.io/posts/sky-t1/

Here's some self-links to some of my thoughts on this (I recommend reading the posts these comments are on as well):

https://www.lesswrong.com/posts/DvHokvyr2cZiWJ55y/2-skim-the-manual-intelligent-voluntary-cooperation?commentId=BBjpfYXWywb2RKjz5 [LW(p) · GW(p)]

https://www.lesswrong.com/posts/FEcw6JQ8surwxvRfr/human-takeover-might-be-worse-than-ai-takeover?commentId=uSPR9svtuBaSCoJ5P [LW(p) · GW(p)]

https://www.lesswrong.com/posts/tdrK7r4QA3ifbt2Ty/is-ai-alignment-enough?commentId=An6L68WETg3zCQrHT [LW(p) · GW(p)]

(Self review) I stand by this essay and think more people should read it, though they don't need to read it deeply. 

I think some people knew this kind of work and so this serves as a pointer to "yeah, that thing we did at my last company" and some people did not realize this was an option. Making people aware of potentially exciting options they could choose in life is (in my opinion) a good use of an essay. In my ideal world everyone would read something describing the One Day Sooner mindset as they were choosing their first careers so they could have it in mind as a possible trait jobs could have. Is that trait positive or negative? Depends on the person! 

I don't know if it's LessWrong Best Of material, but given the number of people who work in this manner that work in the community I think it's good to have some term for it in the water supply. 

Best when paired with Never Drop A Ball.

apologies for the issue with the neuronpedia link. it's now been resolved.

You make a very good point—there are different ways to create contrasts in information that are quickly recognized, and that’s how I’ve come to understand humor. The faster the brain recognizes the information, the higher the chance it triggers a sense of pleasure, or perhaps falls somewhere along a spectrum of pleasure tied to recognizing patterns or resolving contrasts.

I also agree that many jokes can reduce authority. They signal that someone is not a threat, that they lower themselves to gain acceptance, which is often done by people who aren’t strong or authoritative and who use humor as a way to find their space. On the other hand, humor can also be used by authoritative figures to reinforce their power—when others laugh, it confirms that they don’t perceive the person as a threat. Some studies even suggest that chimpanzee laughter might be linked to this dynamic.

In my case, I would describe myself as someone who, in many ways, felt “weak” and used humor to create space for myself. I wasn’t in a position to demand authority outright. I had to teach skills like first aid in a very short amount of time, and I found that humor as positive reinforcement was much more effective than relying on negative reinforcement.

It’s a complex topic, isn’t it? There are so many variables in humor, but this is the perspective I’ve been able to develop so far: humor as something that operates on a spectrum of pleasure derived from the rapid recognition of information.

Additionally, I plan to share some stories about how humor has helped me stay attentive and better use what a class or learning environment offers in a more constructive way.

Well, that's unfortunate.  That feature isn't super polished and isn't currently in the active development path, but will try to see if it's something obvious.  (In the meantime, would recommend subscribing to fewer people, or seeing if the issue persists in Chrome.  Other people on the team are subscribed to 100-200 people without obvious issues.)

Thank you for your interest! My first idea for a post on LessWrong was actually about that—my journey from being a firefighter to discovering rationality. However, I hesitated because it felt very personal, and some of the most interesting parts of my story would be hard to verify. To summarize, I found myself unable to adapt to the "ethics" of the role, which eventually led me to leave and seek rationality as a way to rebuild my life. At the time, it felt like I had nothing left, as I had dedicated my entire life to becoming a firefighter.

Interestingly, there are some parallels between my experiences and the Brazilian movies Tropa de Elite. That kind of intense, complex environment leaves you with stories that are hard to explain but deeply shape who you are.

Thanks to your comment, though, I’m reconsidering publishing my story. Perhaps I could frame it as partly real, partly exaggerated—after all, not everything has to be 100% factual, right? Haha.

Just reproduced it; all I have to do is subscribe to a bunch of people and this happens and the site becomes unusable:

I'm curious how this seems to have gone for you 14 years later. 

The image didn't upload but it's a picture of my browser saying that the web page's javascript is using a ton of resources and I can force-stop it if I wish

Pretty sure the salary transparency law doesn't apply to us, because you need 25+ MA employees. Even if it did, though, I think it would mostly mean giving moderately wider salary ranges? Which I expect would be fine; our two current open positions [1][2] have ranges of 23% and 30%.

[1] https://securebio.org/careers/2024-lab-tech/

[2] https://securebio.org/careers/2024-director-operations/