Consider Tabooing [LW · GW] Gradual Disempowerment.

I'm worried that when people say gradual disempowerment they often mean "some scenario in which humans are disempowered gradually over time", but many readers will interpret this as "the threat model in the paper called 'Gradual Disempowerment'". These things can differ substantially and the discussion in this paper is much more specific than encompassing all scenarios in which humans slowly are disempowered!

(You could say "disempowerment which is gradual" for clarity.)

OP's example is correct and you are wrong. 'Pigheaded' is neither a proposed root cause analysis nor does it mean 'are dumb'; perhaps you should check a dictionary before correcting others' usage. It means stubborn, strong-willed, obstinate, often to the point of foolishness or taking very harmful actions, or to quote the OED: "Having a head like that of a pig. Chiefly figurative: stupidly obstinate, perverse, or set in one's ways." Note: it is "stupidly obstinate", and not "stupid". This is because pigs are notoriously smart but stubborn: very strong, heavy, often hungry, whose mind can't easily be changed by an unfortunate swineherd or passerby in their way. (And this usage has been consistent since the start: the OED will give you the first attestation of it to Ben Jonson, where it describes a small-minded printer who thinks that high-quality news has to be paid for, because that's how he operates; Jonson then mocks some other tradesmen for their own kinds of narrowmindedness, but not for any of them being low-IQ.) Hence, the Russell conjugation is correct: "pigheaded" is the highly insulting figurative term which intensifies the negative "obstinate" which is the bad version of the positive "firm". Just as 'firm' does not principally mean 'dumb', 'pigheaded' doesn't principally mean it either.

I think Dagon is saying that any time you're doing ssh -o "OptionKey=OptionValue" you can instead add OptionKey OptionValue under that host in your .ssh/config, which in this case might look like

Host ec2-*.compute-1.amazonaws.com
    HostKeyAlias aws-ec2-compute
    StrictHostKeyChecking yes

i.e. you would still need step 1 but not step 2 in the above post.

I too agreed w/ Chanind initially, but I think I see where Lucius is coming from. 

If we forget about a basis & focused on minimal description length (MDL), it'd be nice to have a technique that found the MDL [features/components] for each datapoint. e.g. in my comment [LW(p) · GW(p)], I have 4 animals (bunny, etc) & two properties (cute, furry). For MDL reasons, it'd be great to sometimes use cute/furry & sometimes use Bunny if that reflects model computation more simply. 

If you have both attributes & animals as fundamental units (and somehow have a method that tells you which minimal set of units form each datapoint) then a bunny will just use the bunny feature (since that's simpler than cute + furry + etc), & a very cute bunny will use bunny + cute (instead of bunny + 0.1*dolphin + etc (or similar w/ properties)). 

So if we look at Lucius initial statement:

The features a model thinks in do not need to form a basis or dictionary for its activations. [emphasis mine]

They don't need to, but they can form a basis. It very well could be simpler to not constrain our understanding/model of the NN's features as forming a basis. 

Ideally Lucius can just show us this magical method that gets you simple components that don't form a basis, then we'd all have a simpler time understanding his point. I believe this "magical method" is Attribution based parameter decomposition [LW · GW] (APD) that they (lucius, dan, lee?) have been working on, which I would be excited if more people tried creative methods to scale this up. I'm unsure if this method will work, but it is a different bet than e.g. SAEs & currently underrated imo.

For more colour, see this article, which shows the same trend on the same timelines for a bunch of other distances - steady progress till 1940ish, a 10-15 year WW2 gap, then further steady progress mid 1950s on.
https://www.scienceofrunning.com/2017/05/the-roger-bannister-effect-the-myth-of-the-psychological-breakthrough.html?v=47e5dceea252

I would like to see serious thought given to instituting such a norm. There's a lot of complexities here, figuring out what is or isn't kosher would be challenging, but it should be explored.

Agreed. This is how these codes form. Someone does something like this and then people discuss and decide that there should be a rule against it or that it should at least be frowned upon.

Thank you, that is helpful information. 

Sure, there are in some very specific settings with long held professional norms that people agree to (e.g. doctors and lawyers). I don't think this applies in this case, though you could try to create such a norm that people agree to.

Of course people will use the knowledge they gain in collaboration with you for the purposes that they think are best.

It is entirely normal for there to be widely accepted, clearly formalized, and meaningfully enforced restrictions on how people use knowledge they've gotten in this or that setting... regardless of what they think is best. It's a commonplace of professional ethics.