Comment by gbcolborne on The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable · 2022-11-29T18:11:28.217Z · LW · GW

Interesting post. I just wanted to mention that your first two SVD matrix illustrations (for heads 10 and 15 of layer 22) are identical, apart from the labeled axes.