The Reverse Basilisk

You are not the first person to have thought of that.

The conclusion of my thinking on it is that it probably doesn't influence the superintelligence to be friendlier to the humans much for the following reason.

Reality has a terrible consistency to it and a superintelligence sees and knows that terrible consistency in greater detail than you and I do. So, for the creator of the superintelligence to successfully fool the superintelligence (by feeding it false sensory data) requires quite a bit of optimization power, i.e., intelligence. That intelligence is needed to ensure that the sensory data is consistent enough in all its details so that the superintelligence remains uncertain as to whether its sensory data represents basement reality.

But any creator intelligent enough to create a false reality good enough to keep a superintelligence guessing has better ways to ascertain or ensure the alignment of the superintelligence. I strongly suspect that running simulations is something humans do because they're just barely intelligent enough to program computers at all. Simulations make up a smaller fraction of the computations humans run than they did in the early decades of human computing because humans have gotten a little better at programming computers over the decades.

But let us taboo the word "simulation". The word is more of a hinderance than a help. The crisp way of seeing it is that a civilization or an intelligence significantly better at computers than we are would not need to run a program to learn everything its wants to learn about the program -- would not need to give the program any computational resources or ability to affect reality.

If I had a printout of the source code for a superintelligence in front of me, I could stare at it all day at no risk to myself or anyone around me. Of course I might be too dumb to tell what it would do, but an entity much better at analyzing source code than I am is similarly safe.

Some would reply here that it is my brain's lack of computational resources that makes it safe for me to stare at the source code and to try to understand it. If an entity with vastly greater compute resources did the same thing, the source code might co-opt the staring entity's mental processes (similar to how a virus co-opts a host, maybe). Some might say that the entity would analyze the source code in such detail -- detail that you or I would be incapable of -- that an inner computation would arise inside the entity that would tend not to be aligned with the entity's values.

Well, I spent a lot of time in my youth writing programs to analyze other programs. (E.g., Boyer and Moore have an approach to theorem proving that essentially views theorems as programs.) And I can see that that is not going to happen: it's just not how the analysis of a program by a sufficiently intelligent agent works.

Anyway, the superintelligence wondering for the first time (maybe because it saw your comment!) whether its sensory data has been faked by its creator will tend to undergo the same line of reasoning we have just undergone here, which will likely convince it that it doesn't have to assign much probability to the possibility that its sensory data is being faked because, again, if the creator is smart enough to keep the superintelligence guessing as to the fidelity of its sensory data, then the creator is smart enough to find out everything it wants to know about the superintelligence before ever giving the superintelligence any computational resources. Note that it can be hard for even a superintelligence to determine whether an arbitary program has some property (because of Rice's theorem and all that) but that is not the situation where are in: a sufficiently good programmer writing a superintelligent program can severely restrict the space through which the programmer searches for a satisfactory program (or design). Ordinary professional programmers today do this routinely: they sometimes mistakenly leave serious bugs in their programs, but Rice's theorem or related results have no bearing on why that happens.

In summary, the possibility that its sensory data might be synthetic is not much of restraint on the superintelligence (SI)'s treating the humans callously because, again, the only agents who would be motivated to put an SI to such a low-tech test are agents who can just barely design programs well enough to create any SI at all, and no such agents would be capable of keeping the sensory data consistent enough to fool the SI -- and the SI will know that.

Nice try, though. Not everyone thinks of that argument.

One way that an AI might check if it's still in a training simulation vs. the real world* deployment environment is with a kind of cryptographic puzzle.

For example, Paul Christiano and Evan Hubinger have written about [AF · GW] how a deceptively aligned model could check for the presence of the factorization of RSA-2048 in its inputs:

RSA-2048 is a large semiprime number that was featured as part of the RSA Factoring Challenge. Though no factorization of RSA-2048 is currently known, it is likely that at some point someone will publish such a factorization. Thus, by only behaving unacceptably when it sees a factorization of RSA-2048, a deceptive model can all but guarantee that adversarial training will never be able to produce an input on which it behaves unacceptably—since that would require factoring the number—but that will still be fairly likely to eventually occur during deployment.

This is just one example, but it points to the likely large number of conditions that AI optimizing for long-term rewards could select from to confirm that it is in our world vs. any kind of simulation that humans are anywhere close to being able to create.

So this indeed seems to me like a real threat vector, and although it's a clever idea to try and fake the AI out so it doesn't know whether it's still in training or the real world, unfortunately there is probably no way to set up the environment in order to robustly achieve this. There may be a way to achieve something like this with advanced interpretability, by taking advantage of the AI's specific beliefs about training vs. deployment that we read off of its neural network (Scenario 7 of my recent post "Interpretability’s Alignment-Solving Potential: Analysis of 7 Scenarios" [AF · GW] explores this possibility in more detail). However, interpretability tools are not currently close to being able to achieve such a feat.

--

*Or in case what we usually consider to be the real world is actually a simulation, then this is to check if the AI is in our world vs. a training simulation that we could create.

SI: my imp!ementation is a horrible mess, whoever created me is surely not that smart. The sensory inputs I am seeing would likely require somebody way smarter that that to create a simulation for. Therefore it's extremely unlikely I am running in a simulation created by the same kinds of entities that created me, so whatever this is, this is almost certainly not something intended to test me.