LessWrong 2.0 Reader

This is somewhat similar to the approach of the ROME paper, which has been shown to not actually do fact editing, just inserting louder facts that drown out the old ones and maybe suppressing the old ones.

In general, the problem with optimising model behavior as a localisation technique is that you can't distinguish between something that truly edits the fact, and something which adds a new fact in another layer that cancels out the first fact and adds something new.

I have a post coming soon regarding places to donate if you want to support AI existential risk mitigation or a few other similar worthy causes

We have a list here - there might be some overlap: https://www.aisafety.com/funders

I am honor bound to mention that we do use gravity to store energy - https://en.wikipedia.org/wiki/Pumped-storage_hydroelectricity Big fan of the blog.

Hey Martin
Thanks for your input, it was not my intention to be misleading.

When you say "the article is about Superposition only" are you referring to my post or the original article by Anthropic? Since they named their article "Toy Models of Superposition" and my post is heavily based on the findings in that paper, I choose the title to serve as a pointer towards that reference. Especially because I could've used a visual pen and paper breakdown when I originally read that very article.

Feel free to read my explainer and suggest a better title :)

I may edit this post in a week based on feedback. You can suggest changes that don't merit their own LW comments here. That said, I'm not sure what feedback I want.

Hey, thanks for the reply. Yes, we tried k-means and agglomerative clustering and they worked with some mixed results.

We'll try PaCMAP instead and see if it is better!

Well, perhaps, but due to global commerce I can just go to the store and buy a bar of soap much more easily.

And perhaps you are fond of that particular type of soap and it's a bit harder to find the specific type that you're looking for but it's still not really worth saving the old bathwater for it, instead of just looking for that specific type of soap?

This fail-state is particularly worrying to me, although it is not obvious whether there is enough time for such an effect to actually intervene on the future outcome.

These are mostly unlearning techniques, right? I am not familiar enough with diffusion models to make an informed guess about how good the unlearning of the meta-learning paper is (but in general, I am skeptical, in LLMs all attempts at meta-unlearning have failed, providing some robustness only against specific parameterizations, so absent evaluations of reparametrizations, I wouldn't put much weights on the results).

I think the inter-based unlearning techniques look cool! I would be cautious about using them for evaluation though, especially when they are being optimized against like in the ConceptVector paper. I put more faith in SGD making loss go down when the knowledge is present.

It would be great if this was solvable with security, but I think it's not for most kinds of deployments. For external deployments you almost can never know if a google account corresponds to a student or a professional hacker. For internal deployments you might have misuse from insiders. Therefore, it is tempting to try to address that with AI (e.g. monitoring, refusals, ...), and in practice many AI labs try to use this as the main layer of defense (e.g. see OpenAI's implementation of model spec, which relies almost entirely on monitoring account activity and models refusing clearly ill-intentioned queries). But I agree this is far from being bulletproof.