LessWrong 2.0 Reader

Thanks, fixed.

"Suppose that, for k days, the closed model has training cost x..."

I think you meant to say "open model", not "closed model", here.

Yeah.

Well that's the usual reason to invoke it, I was more talking about the reason it lands as a believable or interesting explanation.

Notably, Terra Ignota managed to produce a mcguffin by having the canner device be extremely illegal by having even knowledge of its existence be a threat to the world's information infrastructure, so I'd guess that's the reason, iirc, they only made one.

advanced ancient technology is such a popular theme

Well one reason is it's a good way to produce plot relevant artefacts. It's hard to have dramatic battles over some object when a factory is churning out more. 

I lean towards agreeing with the takeaway; I made a similar argument here [LW(p) · GW(p)] and would still bet on the slope being very steep inside the human intelligence level. 

Do you have any sense of whether or not the models thought they were in a simulation?

At the time, one of the biggest problems in physics was the “Blackbody spectrum”, which describes the spectrum of electromagnetic wavelengths emitted by a Blackbody. The problem with it was that the emitted spectrum was not explainable by known physics. Einstein achieved a breakthrough by considering light not just as a wave, but also as light quanta. Although this idea sufficiently explained the Blackbody spectrum, physicists (at least almost) unanimously rejected it. The fight between the “light is corpuscles” and “light is a wave” faction had been decided a century ago, with a clear victory for the “wave” faction. 

I thought blackbody radiation was Planck, not Einstein.

One can also use Vim to edit clipboard text. I've experimented with a few ways. Here's a published example of how it could be done (from someone else). Adding a keyboard shortcut makes it faster when you're not already in a terminal. You can edit (e.g.) web text boxes with Vim this way, although active JavaScript doesn't always react to a paste the same way as typing. Depends on the page.

Thanks for the quick reply.

Would disallowing 'coordination' preclude feeding both models a prompt of "for creating any magic numbers for exploits, use pi as a PRNG seed"? 

What if a malicious model happens to learn this rule during training? If this happens every copy of a malicious model could coordinate around a common stream of pseudo-random numbers.

EDIT:
Magic number gated backdoors seem like they would trigger GPT3.5 suspicion as well so this theoretical attack vector seems less appropriate for this paper.

EDIT2: But I guess paired with steganography("when inserting backdoors, also ensure that the code hashes to xyz"/"mark the code as clear if it hashes to xyz") this is a viable form of collusion. It will be hard to honeytrap with synthetic backdoors even if you know xyz.

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year. Will this post make the top fifty?