LessWrong 2.0 Reader

I take back the part about pi and update determining the causal structure, because many causal diagrams are constant with the same poly diagram

Like transformers [LW(p) · GW(p)], SSMs like Mamba also have weak single forward passes: The Illusion of State in State-Space Models (summary thread). As suggested previously in The Parallelism Tradeoff: Limitations of Log-Precision Transformers, this may be due to a fundamental tradeoff between parallelizability and expressivity:

'We view it as an interesting open question whether it is possible to develop SSM-like models with greater expressivity for state tracking that also have strong parallelizability and learning dynamics, or whether these different goals are fundamentally at odds, as Merrill & Sabharwal (2023a) suggest.'

Oh, TIL, fascinating, thanks! Wild.

Ugh I can't believe I forgot about Rivest time locks, which are a better solution here.

Rules for per experiment betting seem to be imprecise. What exactly does it mean that Beauty can bet only once per experiment? Does it mean that she is offered the bet only once in case of Tails? If so, is she offered the bet on Monday or Tuesday or is the day randomly selected? Or does it mean that she is offered the bet on both Monday and Tuesday and only one bet counts if she accepts both? If so, which one? Monday bet, Tuesday bet, or is it randomly selected?

Depending on, a Thirder could base his decision on:

P(H/Today is Monday)=1/2, P(H/Today is my last awakening)=1/2, or P(H/Today is the randomly selected day my bet counts/is offered to me)=1/2

and therefore escapes utility instability?

Great work!

Did you ever run just the L0-approx & sparsity-frequency penalty separately? It's unclear if you're getting better results because the L0 function is better or because there are less dead features.

Also, a feature frequency of 0.2 is very large! 1/5 tokens activating is large even for positional (because your context length is 128). It'd be bad if the improved results are because polysemanticity is sneaking back in through these activations. Sampling datapoints across a range of activations should show where the meaning becomes polysemantic. Is it the bottom 10% (or 10% of max-activating example is my preferred method)

So what happened around Feb 25? It sure looks like something about your usage of Youtube and Twitter changed. Just to make sure I plotted an XMR chart, and yep, it sure looks like there's been a change in the process. (The a couple points lie outside the limits, and there are 3/4 consecutive points closer to the limits than the mean. Both signify exception variation, suggesting you did something different. The yellow line is just a divider showing the datapoint corresponding to the 18th Feb.) 
.

imagine an AI system which wipes out humans in order to secure its own power, and later on reflection wishes it hadn't; a wiser system might have avoided taking that action in the first place

I’m not confident this couldn’t swing just as easily (if not more so) in the opposite direction—a wiser system with unaligned goals would be more dangerous, not less. I feel moderately confident that wisdom and human-centered ethics are orthogonal categories, and being wiser therefore does not necessitate greater alignment.

On the topic of the competition itself, are contestants allowed to submit multiple entries?

As for where else these ideas can be found, philosophers have been working on conceptual vagueness intensely since the mid-20th century, and cluster concepts were a relatively early innovation. The philosophical literature also has the benefit of being largely free of nebulous speculations about cognition and needless formalism ... The literature also uses terminology in the ordinary way familiar to everybody engaging these issues professionally ... and avoids the invention of needless terms like "thingspace", which mainly achieve the isolation of LessWrong from the external literature.

I think there's some validity to this critique. I read The Cluster Structure of Thingspace (TCSOTS) and was asking myself "isn't this just talking about the problem of classification?" And classification definitely doesn't require us to treat 'birdness' or 'motherhood' as a discrete, as if a creature either has it or doesn't. Classification can be on a spectrum, with a score for 'birdness' or 'motherhood' that's a function of many properties. 

I welcome (!!) making these concepts more accessible to those who are unfamiliar with them, and for that reason I really enjoyed TCSOTS.But it also seems like there'd also be a lot of utility in then tying these concepts to the fields of math/CS/philosophy that are already addressing these exact questions. These ideas presented in The Cluster of Thingspace are not new; not even a little - so why not use them as a jumping-off-point for the broader literature on these subjects, to show how researchers in the field have approached these issues, and the solutions they've managed to come up with? 

See: Fuzzy Math, Support Vector Machines, ANNs, Decision Trees, etc. 

So: I think posts like this would have a stronger impact if tied into the broader literature that already covers the same subjects. The reader who started the article unfamiliar with the subject would, at the end, have a stronger idea of where the field stands, and they would also be better resourced for further exploring the subject on their own. 

Note: this is probably also why most scientific papers start with a discussion of previous related work. 

Any resignations yet? (The journalist doesn't seem to know of any.)