Comment by vyasnikhil96 (rememberdeath) on Open Source Replication & Commentary on Anthropic's Dictionary Learning Paper · 2023-10-24T12:04:34.287Z · LW · GW

Do you know if the ultra low frequency cluster survives if we do L1/L2 regularization on encoder weights? Similarly I would be curious to know if putting the right sparsity on encoder weights gives a set of neuron sparse features without hurting reconstruction loss or feature sparsity.

Comment by vyasnikhil96 (rememberdeath) on Introducing Fatebook: the fastest way to make and track predictions · 2023-07-16T19:09:14.385Z · LW · GW

the link to fatebook.io at the top does not work. 

Comment by vyasnikhil96 (rememberdeath) on Large language models learn to represent the world · 2023-02-04T13:00:28.757Z · LW · GW

Your document was very interesting.