LessWrong 2.0 Reader

Thanks for putting this out there, I’m on the Catalyze program in Lisa in London and have been thinking a lot about AI control (for framing my background is risk management, so coming at it from that angle).

'The high-level change I propose is to define a control evaluation as a method of bounding risk from worst-case deployment behavior generally – rather than scheming models in particular.'

I’m on board with this, if our goal is to bound the risk from worst-case deployment behaviour, I think the broader the better and enumerating different ‘untrusted model categories’ is going to better inform our understanding of harmful behaviours. If we have post deployment monitoring, and identify new patterns of harmful behaviour, we may even identify new untrusted model categories in this way.

For post deployment monitoring (let’s say the control eval highlighted some behaviour we wanted ongoing control), I think the reason (i.e the untrusted model category causing the behaviour) is less important than the observed behaviour itself. There could even be multiple applying untrusted model categories e.g. scheming and jailbroken, so you could have a single monitor to catch all of these. What are your thoughts on this?

‘control is a broad approach to mitigating and evaluating safety given ’pessimistic assumptions’ about model deployment behavior.’ 

Rather than ‘pessimistic assumptions’ here I describe these as ‘unknowns’, the probability of which would range somewhere between 0-1 based in our confidence in the representativeness in the evaluations.

I think to make control evaluations practically useful, we need to add more nuance to how we define trusted and untrusted. Coming from risk management (in Fintech and FS), I’m approaching it from a view that a model can only ever be P(Trusted)=1 (i.e. fully trusted) when bounded by certain deployment constraints. So a model is trusted to do something rather than generally trusted e.g. it may be trusted to not write backdoored code, but may not be trusted to keep sensitive data safe if its in a setting where users can jailbreak it.

This could also help justify claims around a ‘rarely misbehaving monitor being safe because its failures will likely be uncorrelated with attacks executed by the policy.’ Let’s say you assign a probability to this of 0.001, which is fine for a use case of coding a website but not ok for coding a banking platform. You could layer more monitors to reduce this probability for this use case, effectively getting the risk within appetite for this deployment setting e.g. to 0.000001.

As AIs don't have the same origin as humans, it is basically inconceivable to me that they will ever share the internal processes underlying their "emotions" no matter how good they get at surface "emoting", in my opinion this makes it impossible to have a true connection/meaningful relationship with them as humans, even if many people in the future will fail to see this, but otherwise I largely agree with your post (a brain implemented in silicon could be a meaningful friend to a human (eg ems), people will prob fully integrate AIs into the social scene (this is a mistake in my view), people will modify their brains in (what now seems like) radical ways).

As a general rule, I try and minimise my phone screen time and maximise my laptop screen time. I can do every "productive" task faster on a laptop than on my phone. Here are some things I do that I find helpful that I haven't yet seen discussed.

• Use a very minimalist app launcher on my phone, that makes searching for apps a conscious decision.
• Use a greyscale filter on my phone (which is hard to turn off), as this makes doing most things on my phone harder.

There are many possible maps that describe the same territory. Trying to switch people to use a different map could be a good thing, or it could be a bad thing. (A person who likes the new map might describe it as "giving them fresh insights", a person who dislikes it might describe it as "manipulating them".)

Is the scientific map always better? Well, sometimes it is not available. And sometimes it is too complex. In situations where science provides a clear and simple answer, I guess following it is very likely to be the right answer. But that is not always the case, and then... what are the alternatives? Either paralysis ("I am going to ignore this topic until science finally comes with a simple answer") or some kind of greedy reductionism / focusing on what is legible ("I am going to ignore the illegible parts and focus on the part that is certain: everything, including my wife and kids, is ultimately built from atoms and anything else about them is mere superstition").

Off topic, but your words helped me realize something. It seems like for some people it is physical attraction first, for others it is emotional connection first.

The former may perceive the latter as dishonest: if their model of the world is that for everyone it is physical attraction first (it is only natural to generalize from one example), then what you describe as "take my time getting to know someone organically", they interpret as "actually I was attracted to the person since the first sight, but I was afraid of a rejection, so I strategically pretended to be a friend first, so that I could later blackmail them into having sex by threatening to withdraw the friendship they spent a lot of time building".

Basically, from the "for everyone it is attraction first" perspective, the honest behavior is either going for the sex immediately ("hey, you're hot, let's fuck" or a more diplomatic version thereof), or deciding that you are not interested sexually, and then the alternatives are either walking away, or developing a friendship that will remain safely sexless forever.

And from the other side, complaining about the "friend zone" is basically complaining that too many people you are attracted to happen to be "physical attraction first" (and they don't find you attractive), but it takes you too long to find out.

You have misunderstood a standard figure of speech. Here is the definition he was using: https://www.ldoceonline.com/dictionary/to-be-fair (see also https://dictionary.cambridge.org/us/dictionary/english/to-be-fair, which doesn't explicitly mention that it's typically used to offset criticisms but otherwise defines it more thoroughly).

Thanks Raemon – I'm a fan of ~all these ideas! I'm not spending much time on Fatebook specifically these days (busy with various other projects), but your feedback is and has been super useful.

We are in our native language, so we should work from there.

And begin by stepping outside it.

Gandalf as Ring-Lord would have been far worse than Sauron. He would have remained ‘righteous’, but self-righteous. He would have continued to rule and order things for ‘good’, and the benefit of his subjects according to his wisdom (which was and would have remained great). [The draft ends here. In the margin Tolkien wrote: ‘Thus while Sauron multiplied [illegible word] evil, he left “good” clearly distinguishable from it. Gandalf would have made good detestable and seem evil.’]

So Tolkien anticipated the rise of woke!

Here's what the question creation interface looks like – you can create any question, and there's a bunch of suggestions to help you get started:

Here's what it looks like when you've created some questions. I made these predictions with my partner, so you can see her predictions on the question that's expanded:

For more info, I'd suggest just using the website itself. It is designed to be super easy to use and self-describing!

The website is part of Fatebook, a tool for rapidly tracking predictions. Predict Your Year is a specialised page for creating yearly predictions (which is a popular activity for many people, e.g. inspired by Scott Alexander's annual prediction posts). You can read more about Fatebook in this LessWrong post [LW · GW].