LessWrong 2.0 Reader

I'll admit I have only been loosely following the control stuff, but FWIW I would be excited about a potential @peterbarnett [LW · GW] & @ryan_greenblatt [LW · GW] dialogue in which you two to try to identify & analyze any potential disagreements. Example questions:

• What is the most capable system that you think we are likely to be able to control?
• What kind of value do you think we could get out of such a system?
• To what extent do you expect that system to be able to produce insights that help us escape the acute risk period (i.e., get out of a scenario where someone else can come along and build a catastrophe-capable system without implementing control procedures or someone else comes along and scales to the point where the control procedures are no longer sufficient)

Whoohoo!

My guess is that further scaffolding work (e.g. improved enforcement of thinking out a step by step plan, and then executing the steps of the plan in order) would result in unlocking a lot more capabilities from current open-weight models. I also expect that this is but a hint of things to come, and that by this time next year we'll see evidence that then-current models can accomplish a wide range of complex multi-step tasks.

As far as this particular paper goes I just searched for one on the point in Google Scholar. 

I'm not sure what you believe about Spencer Greenberg but he has two interviews with people who believe that therapist skills (where empathy is one of the academic findings) matter:

https://podcast.clearerthinking.org/episode/070/scott-miller-why-does-psychotherapy-work-when-it-works-at-all/

https://podcast.clearerthinking.org/episode/192/david-burns-cognitive-behavioral-therapy-and-beyond/

The weak single forward passes argument also applies to SSMs like Mamba for very similar theoretical reasons [LW(p) · GW(p)].

I take back the part about pi and update determining the causal structure, because many causal diagrams are constant with the same poly diagram

Like transformers [LW(p) · GW(p)], SSMs like Mamba also have weak single forward passes: The Illusion of State in State-Space Models (summary thread). As suggested previously in The Parallelism Tradeoff: Limitations of Log-Precision Transformers, this may be due to a fundamental tradeoff between parallelizability and expressivity:

'We view it as an interesting open question whether it is possible to develop SSM-like models with greater expressivity for state tracking that also have strong parallelizability and learning dynamics, or whether these different goals are fundamentally at odds, as Merrill & Sabharwal (2023a) suggest.'

Oh, TIL, fascinating, thanks! Wild.

Ugh I can't believe I forgot about Rivest time locks, which are a better solution here.

Rules for per experiment betting seem to be imprecise. What exactly does it mean that Beauty can bet only once per experiment? Does it mean that she is offered the bet only once in case of Tails? If so, is she offered the bet on Monday or Tuesday or is the day randomly selected? Or does it mean that she is offered the bet on both Monday and Tuesday and only one bet counts if she accepts both? If so, which one? Monday bet, Tuesday bet, or is it randomly selected?

Depending on, a Thirder could base his decision on:

P(H/Today is Monday)=1/2, P(H/Today is my last awakening)=1/2, or P(H/Today is the randomly selected day my bet counts/is offered to me)=1/2

and therefore escapes utility instability?