Comment by Peter Boos (peter-boos) on Discussion: Challenges with Unsupervised LLM Knowledge Discovery · 2023-12-22T07:20:52.674Z · LW · GW

I think you might apply this to the hot neurons, the neurons who are most used.
Some recent models allready handle those diffrent for speedup reason.