LessWrong 2.0 Reader

I think part of the important part is building your own (company's) collection of examples to train against, since the foundation models are trained against swebench already.  And if it works the advantage would be on my CV in the worst case but in equity appreciation in the best case.  So, just like any skill, right?

You're right that the whole thing only works if the business can generate returns to high quality code, and can write specifications faster than its complement of engineers can implement them.  But I've been in that position several times, it does happen.  Mainly when the core functionality of the product is designed and led by domain experts who are not software engineers.  Like if you make software for accountants for instance.

A belated thanks for this—as I also just commented to Jiro, it seems basically right. And I like your elaboration on Jiro's point, and clarifying where I am/was confused/not.

Hmmm this is a good point and not one I have a refutation to, although it does still seem to me that most of what I'm trying to get at here is still right! Belated thanks.

I agree that constraints make things harder, and that being vegan is a constraint, but again that is separate to weirdness. If Charles is hosting a dinner party on Friday in a "fish on Friday" culture then Charles serving meat is weird in that culture but it means Charles is less constrained, not more. If anything the desire to avoid weirdness can be a constraint. There are many more weird pizza toppings than normal pizza toppings.

Given the problem that Alice and Bob are having, a good approach is that they communicate better, so that they know there is a problem, and what it is. An approach of being less weird may cause more problems than it solves.

I'm saying that since we know that "facts" offered by Trump to support his goals aren't always true, that part of his conversation with Zelenskyy is probably best viewed as part of a persuasion tactic that may or may not be factually connected with reality.

Full Grok 3 only had a month for post-training, and keeping responses on general topics reasonable is a fiddly semi-manual process. They didn't necessarily have the R1-Zero idea either, which might make long reasoning easier to scale automatically (as long as you have enough verifiable tasks, which is the thing that plausibly fails to scale very far).

Also, running long reasoning traces for a big model is more expensive and takes longer, so the default settings will tend to give smaller reasoning models more tokens to reason with, skewing the comparison.

Does it seem likely that a trait that has survival significance (in a highly social animal such as a human) would be emergent? Even if it might have been initially, you'd think selective pressure would have brought forth a set of genes that have significant influence on it.

Thanks. One thing that confuses me is that, if this is true, why do mini reasoning models often seem to out-perform their full counterparts at certain tasks? 

e.g. grok 3 beta mini (think) performed overall roughly the same or better than grok 3 beta (think) on benchmarks[1]. And I remember a similar thing with OAI's reasoning models.

[1] https://x.ai/blog/grok-3

One non-technical forecast, related to gpt4.5's announcement: https://x.com/davey_morse/status/1895563170405646458

Hey, long time no see! Thanks, I've correct it:

$\frac{\partial F\left(e_{\alpha }\right)}{\partial \alpha }=\frac{\partial F\left(e_{\alpha }\right)}{\partial e_{\alpha }}\frac{\partial e_{\alpha }}{\partial \alpha }$

$=\frac{\partial F\left(e_{\alpha }\right)}{\partial e_{\alpha }}\frac{\partial [e_{\text{clean}}+\alpha \times (e_{\text{corr}}-e_{\text{clean}}\left)\right]}{\partial \alpha }$

$=\frac{\partial F\left(e_{\alpha }\right)}{\partial e_{\alpha }}(e_{\text{corr}}-e_{\text{clean}})$

Set $\alpha =0$, ie. $e_{\alpha }=e_{clean}$

$=(e_{\text{corr}}-e_{\text{clean}})\frac{\partial F\left(e_{\text{clean}}\right)}{\partial e_{\text{clean}}}$