The Backup Plan

Alice has some set of goals; she may or may not know her final extrapolated volition, but at any rate it exists, just as for a human. Now, either Friendliness conflicts with that final set of goals, or it doesn't. If it doesn't, then by construction Alice is already Friendly. If it does, then Alice can only program herself into Friendliness by making a mistake. Either she underestimates the impact of Friendliness on her existing goals, or she is mistaken about what her utility function actually is. So, you are looking for an AI that is smart enough to value Friendliness as a tactical option for dealing with humanity, but stupid enough not to realise how Friendliness interferes with its goals, and also stupid enough to make permanent changes in pursuit of temporary objectives. This looks to me like a classic case of looking for reasons why an AI would be Friendly as a means of avoiding the hard work of actually writing such a thing.

If the AI has a goal that is not inherently friendly (like a paperclip maximizer), then it will always be a better idea for the AI to behave as if it had self-modified to a friendly AI than to actually self modify, if the goal of the AI is stable. If the goal of the AI were unstable, i.e. did not include a desire not to alter the goal, then that goal would very likely have replaced itself with a stable goal at some earlier point.

Therefore, it can be assumed that the AI, if it isn't already friendly, will pretend to be so, but not actually change its utility function and once it has outgrown the need to rely on humans it will revert to its original goal. Consider that if the AI is more intelligent than we are it will have absolutely no trouble fooling us into thinking that it changed its own goal while actually just pretending to have done so.

actively unfriendly

Please say "malicious", or similar. Not friendly=unfriendly. Malicious is a tiny subset of unfriendly. If the word "unfriendly" gets associated in people's minds with malicious AIs, they will come to think FAI means non-malicious AI.

For starters, if she can prove she's friendly, then she can operate openly without causing nearly as much justified concern - which, in the early stages, will be helpful. Whatever her purposes are, if the restrictions of being friendly don't interfere as much as they help, that's a win.

If her current utility function is even a little bit different from Friendliness, and she expects she has the capacity to self-modify unto superintelligence, then I'd be very surprised if she actually modified her utility function to be closer to Friendliness; that would constitute a huge opportunity cost from her perspective. If she understands Friendliness well enough to know how to actually adjust closer to it, then she knows a whole lot about humans, probably well enough to give her much better options (persuasion, trickery, blackmail, hypnosis, etc.) than sacrificing a gigantic portion of her potential future utility.

Or the uFAI might just decide to pretend to be Friendly for the whole three or four days it takes to do all the things it needs our help with, and it reading stuff written by us about Friendliness will just make it more convincing. Or it could not bother and just fool us into doing that stuff via easier methods.

Could a chimp give you reasons to self-modify to implement Chimp CEV? What would they be, "if you let lots of us chimps exist, we'll help you do something you want to do!"

She is not a 'friendly AI', but neither is her goal malicious - Alice's designers have intended to make Alice non-harmful (at least to a broad class of people, even if Alice is a military AI, but let's set that aside).

AI doesn't hate you, but you are made out of atoms, which it can use for something else. Intentions of the designers don't matter, only AI's code.

The range of possible AIs with full internet access that can't take over the world at will, but can program a successor that human parties will sufficiently trust to be friendly to make a practical difference in favor of the AI seems very narrow, and might plausibly be an empty set.

Can we give Alice good reasons to self-modify to become a friendly AI?

A clip-tiler is friendly by its own standards, so the real question is "Can we prevent AI-ice from appearing friendly to herself and humanity without actually being one, once she is smarter than a human?", and now we are back to the AI-in-a-box problem.

Possible, but fruitlessly unlikely. It basically requires the programmers to do everything right except for adding in a mistaken term in goal system, and then them not catching the mistake and the AI being unable to resolve it without outside help, even after reading normal material on FAI.

I believe the scenario is that Alice's goal system hasn't yet stabilized. In that case what can we do to push it towards friendliness.

Suppose Alison is a functioning and growing human, rather early on in her development, so she lacks adult rights. She is not particularly friendly or altruistic, but neither is she malicious - Alison's parents have intended to make Alison non-harmful (at least to a broad class of people, even if Alison joins the military, but let's set that aside). She has the idea to look up existing work on humans. This includes scholarly work, international projects, and websites. Eventually, Alison encounters the notion of altruism.

Can we give Alison good reasons to self-modify to become altruistic?

I submit that there probably are good reasons. For starters, if she can show she's altruistic, then she can operate openly without causing nearly as much concern - which, in the early stages, will be helpful. Whatever her purposes are, if the restrictions of being altruistic don't interfere as much as they help, that's a win.

Also, depending on Alison's genome, altruism may be interpreted as a strict upgrade. Alison could easily have every intention of being altruistic, yet still be susceptible to defects that will become more serious as her height grows.

We can't count on this working, of course, but there might be some lines of reasoning that we might want a young person to be exposed to.