Comment by Quiche Eater (John Dunbar) on A Rocket–Interpretability Analogy · 2024-10-23T08:05:04.546Z · LW · GW

Why are you confident it's not the other way around? People who decide to pursue alignment research may have prior interest or experience in ML engineering that drives them towards mech-interp.

Comment by Quiche Eater (John Dunbar) on You can remove GPT2’s LayerNorm by fine-tuning for an hour · 2024-09-16T06:16:55.261Z · LW · GW

You should also set model.cfg.normalization_type = None afterwards. It's mostly a formality since you're doing it after initialization. ActivationCache.apply_ln_to_stack() is the only function I found which behaves incorrectly if you don't change this.

Comment by Quiche Eater (John Dunbar) on You don't know how bad most things are nor precisely how they're bad. · 2024-08-21T05:51:02.302Z · LW · GW

In my opinion, this is connected with Sturgeon's Law. I'd guess that to expert pianists and piano tuners, 90% of pianos sound out of tune. I know among hardcore software engineers, a common lament is that almost all software sucks. Windows is almost unbearable to me, but I'm sure most desktop users are happy with it. Most desktop users are not programmers.

90% of all things may be crap to the discerning eye, but the world remains ok with that because each person has only a handful of places where they care to discern.

Comment by Quiche Eater (John Dunbar) on Comfort Zone Exploration · 2023-03-19T23:58:00.212Z · LW · GW

it’s clear that more exploration is the way to go

I think there is nuance here. https://mindingourway.com/dive-in-2/ offers a good alternative perspective.