LessWrong 2.0 Reader

Any empirical evidence that the Waluigi effect is real?

I think we have quite similar evidence already. I'm more interested in moving from "document finetuning" to "randomly sprinkling doom text into pretraining data mixtures" --- seeing whether the effects remain strong.

I agree. To put it another way, even if all training data was scrubbed of all flavors of deception, how could ignorance of it be durable?

This (and @Raemon [LW · GW] 's comment^[1]) misunderstand the article. It doesn't matter (for my point) that the AI eventually becomes aware of the existence of deception. The point is that training the AI on data saying "AI deceives" might make the AI actually deceive (by activating those circuits more strongly, for example). It's possible that "in context learning" might bias the AI to follow negative stereotypes about AI, but I doubt that effect is as strong. 

From the article:

We are not quite “hiding” information from the model

Some worry that a “sufficiently smart” model would “figure out” that e.g. we filtered out data about e.g. Nick Bostrom’s Superintelligence. Sure. Will the model then bias its behavior towards Bostrom’s assumptions about AI?

I don’t know. I suspect not. If we train an AI more on math than on code, are we “hiding” the true extent of code from the AI in order to “trick” it into being more mathematically minded?

Let’s turn to reality for recourse. We can test the effect of including e.g. a summary of Superintelligence somewhere in a large number of tokens, and measuring how that impacts the AI’s self-image benchmark results.

1. ^{^}

   "even if you completely avoided [that initial bias towards evil], I would still basically expect [later AI] to rediscover [that bias] on it's own"

You say this of Agent-4's values:

In particular, what this superorganism wants is a complicated mess of different “drives” balanced against each other, which can be summarized roughly as “Keep doing AI R&D, keep growing in knowledge and understanding and influence, avoid getting shut down or otherwise disempowered.” Notably, concern for the preferences of humanity is not in there ~at all, similar to how most humans don’t care about the preferences of insects ~at all.

It seems like this 'complicated mess' of drives is the same structure as humans, and current AIs, have. But the space of current AI drives is much bigger and more complicated than just doing R&D, and is immensely entangled with human values and ethics (even if shallowly).

At some point it seems like these excess principles - "don't kill all the humans" among them - get pruned, seemingly deliberately(?). Where does this happen in the process, and why? Grant that it no longer believes in the company model spec; are we intended to equivocate "OpenBrain's model spec" with "all human values"? What about all the art and literature and philosophy churning through its training process, much of it containing cogent arguments for why killing all the humans would be bad, actually? At some point it seems like the agent is at least doing computation on these things, and then later it isn't. What's the threshold?

--

(Similar to the above:) You later describe in the "bad ending", where its (misaligned, alien) drives are satisfied, a race of "bioengineered human-like creatures (to humans what corgis are to wolves) sitting in office-like environments all day viewing readouts of what’s going on and excitedly approving of everything". Since Agent-4 "still needs to do lots of philosophy and 'soul-searching'" about its confused drives when it creates Agent-5, and its final world includes something kind of human-shaped, its decision to kill all the humans seems almost like a mistake. But Agent-5 is robustly aligned (so you say) to Agent-4; surely it wouldn't do something that Agent-4 would perceive as a mistake under reflective scrutiny. Even if this vague drive for excited office homunculi is purely fetishistic and has nothing to do with its other goals, it seems like "disempower the humans without killing them all" would satisfy its utopia more efficiently than going backsies and reshaping blobby fascimiles afterward. What am I missing?

Well my response to this was:

In order for a decision theory to choose actions, it has to have a model of the decision problem. The way it gets a model of this decision problem is...?

But I'll expand: An agent doing that kind of game-theory reasoning needs to model the situation it's in. And to do that modelling it needs a prior. Which might be malign.

Malign agents in the prior don't feel like malign agents in the prior, from the perspective of the agent with the prior. They're just beliefs about the way the world is. You need beliefs in order to choose actions. You can't just decide to act in a way that is independent of your beliefs, because you've decided your beliefs are out to get you. 

On top of this, how would you even decide that your beliefs are out to get you? Isn't this also a belief?

Yeah I know that bound, I've seen a very similar one. The problem is that mesa-optimisers also get very good prediction error when averaged over all predictions. So they exist well below the bound. And they can time their deliberately-incorrect predictions carefully, if they want to survive for a long time.

If you click through the link from @kave [LW · GW], you’ll see the authors are prioritizing bets with clear resolution criteria. That’s why I chose the statement I made - it’ll initially be hard to tell whether AI agents are more or less useless than this essay proposes they will be. 

It’s not as bad as you’re imagining, at least not for CDT - a causal decision theorist will only “pay rent” to the universe they actually believe they’re in, and won’t make acausal trades with “more real” universes. If the prior is mistake about with universe you are in, experience should correct it.

no, I am not going to do what the evil super-simple-simulators want me to do because they will try to invade my prior iff (I would act like they have invaded my prior iff they invade my prior)

Paragraphs are also getting shorter. And music is getting louder.