Posts
Comments
Hi, I'm undertaking a research project and I think that an end2end SAE with automated explanations would be a lot of help.
The project is a a parameter-efficient fine-tuning method that may be very interpretable, allowing researchers to know what the model learned during fine-tuning:
Start by acquiring a model with end-to-end SAEs throughout. Insert a 1 hidden layer FFNN (with a skip connection) after a SAE latent vector and pass the output to the rest of the model. Since SAE latents are interpretable, the rows in the first FFNN matrix will be interpretable as questions about the latent, and the columns of the second FFNN matrix will be interpretable as question-conditional edits to the residual latent vector as in https://www.alignmentforum.org/posts/iGuwZTHWb6DFY3sKB/fact-finding-attempting-to-reverse-engineer-factual-recall
I would expect end2end SAEs to work better than local SAEs because as you found, local SAEs do not return decodings with the same behaviors as well as end2end SAEs.
If you could share your dict[SAE latent, description] for
e2e-saes-gpt , I would appreciate it so much. If you cannot, I'll use a local SAE instead for which I can find descriptions of the latents, though I expect it would not work as well.
Also, you might like to hear that some of your links are dead:
https://www.neuronpedia.org/gpt2sm-apollojt results in:
Error: Minified React error #185; visit https://react.dev/errors/185 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
Back to Home
https://huggingface.%20co/apollo-research/e2e-saes-gpt2 cannot be reached.
I personally thought that "taking actions that would give yourself more power than the government" was something that... seemed like it shouldn't be allowed? Many people I talked to shared your perspective of "of course AI labs are in the green" but it wasn't so obvious to me. I originally did the research in April and May, but since then there was the Situational Awareness report with the quote "it’s quite plausible individual CEOs would have the power to literally coup the US government." I haven't seen anyone else talking about this.
My reasoning for choosing to write about this topic went like this:
"They are gaining abilities which will allow them to overthrow the government."
"What? Are they allowed to do that? Isn't the government going to stop them?"
If I were in charge of a government, I sure wouldn't want people doing things that would set them up to overthrow me. (And this is true even if that government gets its mandate from its citizens, like the US.)
Maybe the details of treason and sedition laws are more common knowledge than I thought, and everyone but me picked up how they worked from other sources?