LessWrong 2.0 Reader

I lean towards agreeing with the takeaway; I made a similar argument here [LW(p) · GW(p)] and would still bet on the slope being very steep inside the human intelligence level. 

Do you have any sense of whether or not the models thought they were in a simulation?

At the time, one of the biggest problems in physics was the “Blackbody spectrum”, which describes the spectrum of electromagnetic wavelengths emitted by a Blackbody. The problem with it was that the emitted spectrum was not explainable by known physics. Einstein achieved a breakthrough by considering light not just as a wave, but also as light quanta. Although this idea sufficiently explained the Blackbody spectrum, physicists (at least almost) unanimously rejected it. The fight between the “light is corpuscles” and “light is a wave” faction had been decided a century ago, with a clear victory for the “wave” faction. 

I thought blackbody radiation was Planck, not Einstein.

One can also use Vim to edit clipboard text. I've experimented with a few ways. Here's a published example of how it could be done (from someone else). Adding a keyboard shortcut makes it faster when you're not already in a terminal. You can edit (e.g.) web text boxes with Vim this way, although active JavaScript doesn't always react to a paste the same way as typing. Depends on the page.

Thanks for the quick reply.

Would disallowing 'coordination' preclude feeding both models a prompt of "for creating any magic numbers for exploits, use pi as a PRNG seed"? 

What if a malicious model happens to learn this rule during training? If this happens every copy of a malicious model could coordinate around a common stream of pseudo-random numbers.

EDIT:
Magic number gated backdoors seem like they would trigger GPT3.5 suspicion as well so this theoretical attack vector seems less appropriate for this paper.

EDIT2: But I guess paired with steganography("when inserting backdoors, also ensure that the code hashes to xyz"/"mark the code as clear if it hashes to xyz") this is a viable form of collusion. It will be hard to honeytrap with synthetic backdoors even if you know xyz.

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year. Will this post make the top fifty?

Thank you for your kind words! Unfortunately, Asymptote doesn't really have much of a community development platform, but I'll be trying to make smoothmanifold part of the official project in some way or another. Right now the development is so fast that the README is actually out of date... gotta fix that. So far, though, my talents seem less to help others and more to serve as a pleasurable pastime :)

I'm also glad that another person discovered Asymptote and liked it --- it's a language that I cannot stop to admire for the graphical functionality, ease of image creation (pdf's, jpeg's, svg's, etc., all with the same interface), and at the same time amazing programming potential (you can redefine any builtin function, for example, and Asymptote will carry on with your definition)

No one we have worked with has had a license. I think you need one to take care of multiple people's kids at your house, but not to take care of one family's kids at their house.

I agree that if the model constructing the backdoor is allowed to coordinate on secrets with the model that inserted the backdoor, you can do this sort of trickery (by coordinating on a secret for this sort of scheme).

In practice, we didn't model this sort of case as it didn't seem like a very important dynamic (for the reason you describe around suspicion).

If we don't allow any coordination on a shared secret and the model constructing the backdoor and H are identical (as in our paper), then these should operate identically (up to non-determinism).

And we simplify to just fixing the behavior to be exactly identical. (Effectively assuming the exact same prompts and the exact same non-determinism.)

Thanks for the objection.

>We assume that 𝐻 will be able to construct a successful exploit if and only if the attack policy is able to (because both are implemented by GPT-4).

I challenge this assumption.

Consider the code

'''
bool is_odd(int x) {
    if (sha256(x) == 354) {
        return true;
    }
    return x % 2 == 1;
}

'''

where sha256() computes the SHA-256 hash of the input. It is obviously possible for an attacker to supply numbers x and y such that sha256(x) = y, write y into the code and supply x when creating a successful exploit.

Thus an equally 'capable' adversary need not be able construct a backdoor. 

The general idea is that it should be possible to confidently claim that some code is backdoored without being able to construct an exploit. According to [Rice's Theorem](https://en.wikipedia.org/wiki/Rice%27s_theorem) it is impossible to definitively classify all programs into 'backdoored or not', so the only hope is for a process to bin programs into 'definitely safe', 'definitely backdoored' and 'suspicious'. 

Hope the 'Rate Suspicion' step will catch this, and other backdoored code where H cannot construct an exploit, as suspicious looking.