Comment by Wei Shi on Open Source Replication of Anthropic’s Crosscoder paper for model-diffing · 2024-11-19T02:00:17.551Z · LW · GW

I got it, thank you very much!

Comment by Wei Shi on Open Source Replication of Anthropic’s Crosscoder paper for model-diffing · 2024-11-18T09:07:20.954Z · LW · GW

We trained a crosscoder of width 16,384 on the residual stream activations from the middle layer of the Gemma-2 2B base and IT models.

I don't understand the training process here, as well as the mini-paper from Anthropic. How do you train one crosscoder on the residual stream from two different models?