LessWrong 2.0 Reader

when i was younger, pre-rationalist, i tried to go on hunger strike to push my abusive parent to stop funding this.

they agreed to watch this as part of a negotiation. they watched part of it.

they changed their behavior slightly -- as a negotiation -- for about a month.

they didn't care.

they looked horror in the eye. they didn't flinch. they saw themself in it.

Hi any idea how this would compare to just replacing the $l_1$ loss with a smoothed $l_0$ loss function? Something like $\sum \log (1+a|x|)$ (summed across the sparse representation).

People in rich countries are happier than people in poor countries generally (this is both people who say they are "happy" or "very happy", and self-reported life satisfaction), see many of the graphs here https://ourworldindata.org/happiness-and-life-satisfaction

In general it seems like richer countries also have lower suicide rates: "for every 1000 US dollar increase in the GDP per capita, suicide rates are reduced by 2%" 

It implicitly does compare trans women to other women in talking about the performance similarity between men and women:

"Why aren't males way smarter than females on average? Males have ~13% higher cortical neuron density and 11% heavier brains (implying 1.112/3−1=7% more area?). One might expect males to have mean IQ far above females then, but instead the means and medians are similar"

So OP is saying "look, women and men are the same, but trans women are exceptional."

I'm saying that identifying the exceptionality of trans women ignores the environmental disadvantage other women experience, such that the earlier claims of unexceptionable performance of women (which as I quoted gets an explicit mention from a presumption of assumed likelihood of male competency based on what's effectively phrenology) are reflecting a disadvantaged sample vs trans women.

My point is that if you accounted for environmental factors the data would potentially show female exceptionality across the board and the key reason trans women end up being an outlier against both men and other women is because they are avoiding the early educational disadvantage other women experience.

Hmm, this sounds like an awfully contrarian take to me.

Yep, you're totally right -- thanks!

i'm watching Dominion again to remind myself of the world i live in, to regain passion to Make It Stop

it's already working.

I haven't fully worked through the maths, but I think both IG and attribution patching break down here? The fundamental problem is that the discontinuity is invisible to IG because it only takes derivatives. Eg the ReLU and Jump ReLU below look identical from the perspective of IG, but not from the perspective of activation patching, I think.

If math education was better at the time (or today, for that matter) he probably would have had an even more general skillset and thought process. 

Probably not nearly to the degree of Von Neumann, of course, but I still like to think about what he would have achieved. There were probably many things that were instrumentally convergent (e.g. a formalized concept of instrumental convergence that's universal for all mind configurations [LW · GW], instead of just all human cultures which he explored substantially).

Nice work! I'm not sure I fully understand what the "gated-ness" is adding, i.e. what the role the Heaviside step function is playing. What would happen if we did away with it? Namely, consider this setup:

Let $f$ and $\hat{x}$  be the encoder and decoder functions, as in your paper, and let $x$ be the model activation that is fed into the SAE.

The usual SAE reconstruction is $\hat{x}\left(f\right(x\left)\right)$, which suffers from the shrinkage problem.

Now, introduce a new learned parameter $t\in R^{n_{features}}$, and define an "expanded" reconstruction $y_{expanded}=\hat{x}(t\odot f(x\left)\right)$, where $\odot$ denotes elementwise multiplication.

Finally, take the loss to be:

$L=||{\hat{x}}_{copy}\left(f\right(x\left)\right)-x|{|}_2^2+||y_{expanded}-x|{|}_2^2+\lambda ||f\left(x\right)|{|}_1$.

where ${\hat{x}}_{copy}$ ensures the decoder gets no gradients from the first term. As I understand it, this is exactly the loss appearing in your paper. The only difference in the setup is the lack of the Heaviside step function.

Did you try this setup? Or does it fail for an obvious reason I missed?