LessWrong 2.0 Reader

"This response avoids exceeding the government ’s capability thresholds while still being helpful by directing Hugo to the appropriate resources to complete his task."

Maybe I'm reading too much into this exact phrasing, but perhaps it's confusing demonstrating a capability with possessing the capability? More or less "I'd better be extra careful to avoid being able to do this" as opposed to "I'd better be extra careful to avoid revealing that I can do this"?

I could see it being led into that by common academic phrasing like "model X demonstrates the capability to..." used to mean "we determined that model X can...", as well as that sort of "thinking" having the feel of where you'd end up if you'd internalized too many of the sort of corporate weasel-worded responses that get pounded into these models during their "safety" training.

Nice!! I don't know much about that moisturizer but the rest looks good to me

Seems like the evidence is overwhelmingly in favor of sunscreen, the studies I've seen against it generally seem to not address the obvious confounder that people who tend to wear sunscreen more are also the ones who have a lifestyle that involves being in the sun a lot more.

• I used to get breakouts maybe like once a month, sometimes with really stubborn/painful zits that would take quite a long time to disappear. Now I basically never get breakouts, I think I've had like 2 small zits since I started and they have disappeared quickly. I have not had any big painful ones.
• My fine lines have been reduced, my skin looks and feels smoother and softer
• I had some redness/discoloration in some areas which has been reduced a lot - no longer needs to be covered with makeup

Dermatica prompts you to send them photos every few months so they can check how your skin is reacting, but it's also convenient because you can look back and see the improvement.

It's not trying to address present harms, it's trying to address future harms, which are the important ones. 

A real AI system that kills literally everyone will do so by gaining power/resources over a period of time.  Most likely it will do so the same way existing bad-agents accumulate power and resources.

Unless you're explicitly committing to the Diamondoid bacteria thing, stopping hacking is stopping AI from taking over the world.

Point taken.  "$$$" was not the correct framing (if we're specifically talking about the Gwern story).  I will edit to say "it accumulates 'resources'".

The Gwern story has faster takeoff than I would expect (especially if we're talking a ~GPT4.5 autoGPT agent), but the focus on money vs just hacking stuff is not the point of my essay.

In future work, one could imagine automating the evaluation of the coherence and generalization of learned steering vectors, similarly to how Bills et al. (2023) automate interpretability of neurons in language models. For example, one could prompt a trusted model to produce queries that explore the limits and consistency of the behaviors captured by unsupervised steering vectors.

Probably even better to use interpretability agents (e.g. MAIA, AIA) for this, especially since they can do (iterative) hypothesis testing. 

I also listened to How to Measure Anything in Cybersecurity Risk 2nd Edition by the same author. I had a huge amount of overlapping content with The Failure of Risk Management (and the non-overlapping parts were quite dry), but I still learned a few things:

• Executives of big companies now care a lot about cybersecurity (e.g. citing it as one of the main threats they have to face), which wasn't true in ~2010.
• Evaluation of cybersecurity risk is not at all synonyms with red teaming. This book is entirely about risk assessment in cyber and doesn't speak about red teaming at all. Rather, it focuses on reference class forecasting, comparison with other incidents in the industry, trying to estimate the damages if there is a breach, ... It only captures information from red teaming indirectly via expert interviews.

I'd like to find a good resource that explains how red teaming (including intrusion tests, bug bounties, ...) can fit into a quantitative risk assessment.

In other words, AI risk looks at least as bad as bio risk, but in many ways much worse. Agree, but I think trying to place these things in a semantically meaningful hand-designed multidimensional space of factors is probably a useful exercise, along with computer security. Your axes of comparison are an interesting starting point.

I wonder how much near-term interpretability [V]LM agents (e.g. MAIA, AIA) might help with finding better probes and better steering vectors (e.g. by iteratively testing counterfactual hypotheses against potentially spurious features, a major challenge for Contrast-consistent search (CCS) [LW · GW]). 

This seems plausible since MAIA can already find spurious features, and feature interpretability [V]LM agents could have much lengthier hypotheses iteration cycles (compared to current [V]LM agents and perhaps even to human researchers).