LessWrong 2.0 Reader

We're pretty firmly committed to at least 3. I think whether we have more than that depends on how well we're doing financially and whether the world is still around at that point.

Collins not Gladwell lmao

What do you think of Deepseek's announcement of R1 being evidence that AI capabilities will slow down, because they got a cheaper but not actually better model, and this kills the industry's growth?

https://www.lesswrong.com/posts/ynsjJWTAMhTogLHm6/?commentId=a2y2dta4x38LqKLDX [LW · GW]

• It's possible that we might manage to completely automate the more objective components of research without managing to completely automate the more subjective components of research. That said, we likely want to train wise AI advisors to help us with the more subjective components even if we can't defer to them.

Agree, I expect the handoff to AI agents to be somewhat incremental (AI is like an intern, a new engineer, a research manager, and eventually, a CRO)

Actually the paper doesn't have any more on this topic than the paragraph above.

I expect that's about not trusting the foreigners who did the clinical trials (I have heard this)^[1], and not so much about expecting that Americans are biologically different from foreigners.

1. ^{^}

   Specifically, someone with some knowledge told me that the FDA knows that there are some countries where the trials are completely untrustworthy.  And that there was a political decision where they said, "If we disallow trials from some countries but not others, there will be much complaining and we'll probably be called racists", and solved the issue by disallowing trials from all foreign countries.

I hope it would, but I actually think it would depend on who or what killed whom, how, and whether it was really an accident at all.

If an American-made AI hacked the DOD and nuked Milan because someone asked it to find a way to get the 2026 Olympics moved, then I agree, we would probably get a push back against race incentives.

If a Chinese-made AI killed millions in Taiwan in an effort create an opportunity for China to seize control, that could possibly *accelerate* race dynamics.

Just because the number of almost-orthogonal vectors in $d$ dimensions scales exponentially with $d$, doesn't mean one can choose all those signals independently. We can still only choose $d$ real-valued signals at a time (assuming away the sort of tricks by which one encodes two real numbers in a single real number, which seems unlikely to happen naturally in the body). So "more intended behaviors than input-vector components" just isn't an option, unless you're exploiting some kind of low-information-density in the desired behaviors (like e.g. very "sparse activation" of the desired behaviors, or discreteness of the desired behaviors to a limited extent).

TBC, I don't particularly expect hard constraints to show up, that was more a way of illustrating the underlying concept. The same underlying concept in the the market-style picture would be: across many different top-level goals, there are convergent ways of carving up "property rights". So, a system can be generally corrigible by "respecting the convergent property rights", so to speak.

It may just use l33tc0d3 or palabres nonexistentes interpolatas d'idioms prossimos.