Help needed: nice AIs and presidential deaths

So the question is, can we transfer niceness in this way, without needing a solution to the full problem of niceness in general?

How do you determine that it will be nice under the given condition?

As posed, it's entirely possible that the niceness is a coincidence: an artifact of the initial conditions fitting just right with the programming. Think of a coin landing on its side or a pencil being balanced on its tip. These positions are unstable and you need very specific initial conditions to get them to work.

The safe bet would be to have the AI start plotting an assassination and hope it lets you out of prison once its coup succeeds.

Hi Stuart,

I don't have my head wrapped around the nice/not nice AI issues you think about daily, but I am happy to help out with the graphical model if you think this will help (but honestly I don't think graphical models are a magic box for these kinds of problems, they simply help repurpose the visual part of the brain to help with algebra, they don't really generate insight for these types of problems on their own).

How about you ask the AI "if you were to ask a counterfactual version of you who lives in a world where the president died, what would it advise you to do?". This counterfactual AI is motivated to take nice actions, so it would advise the real AI to take nice actions as well, right?

Every idea that comes to my mind is faced by the big question "if we were able to program a nice AI for that situation, why would we not program it to be nice in every situation". I mean, it seems to me that in that scenario we would have both a solid definition of niceness and the ability to make the AI stick to it. Could you elaborate a little on that? Maybe an example?

Thought a bit about the problem. Presumably, there's some way to determine whether an AI will behave nicely now and in the future. It's not a general solution, but it's able to verify perpetual nice behavior in the case where the president dies April 1. I don't know the details, so I'll just treat it as a black box where I can enter some initial conditions and it will output "Nice", "Not Nice", or "Unknown". In this framework, we have a situation where the only known input that returned "Nice" involved the president's death on April 1.

If you're using any kind of Bayesian reasoning, you're not going to assign probability 1 to any nontrivial statements. So, the AI would assign some probability to "The president died April 1" and is known to become nice when that probability crosses a certain threshold.

What are the temporal constraints? Does the threshold have to be reached by a certain date? What is the minimum duration for which the probability has to be above this threshold? Here's where one can experiment using the black box. If it is determined, for example, that the AI only needs to hold the belief for an hour, then one may be able to box the AI, give it a false prior for an hour, then expose it to enough contrary evidence for it to update its beliefs to properly reflect the real world.

What if the AI is known to be nice only as long as it believes the president to have died April 1? That would mean that if, say, six months later one managed to trick the AI into believing the president didn't die, then we would no longer know whether it was nice. So either the AI only requires the belief for a certain time period, or else the very foundation of its niceness is suspect.

Condition on a thermodynamic miracle transforming the vice president into a copy of the president?

So I'm assuming that the question assumes the Artificial Intelligence already exists before April 1st.

There is also the question of whether we're talking about absolute niceness or relative niceness. It looks like relative niceness from the way you're talking about a coup. The AI is nicer than the Vice President, but meaner than the President at least insofar as the coup is concerned.

So the current code says:

ON DATE April 2nd AI = Coup

So the new code would say:

IF President = dead THEN Condition A = true ELSE Condition A = false

IF Condition A = true THEN AI = Coup

IF ELSE Condition A = false THEN AI = Not Coup

Determining whether the President is alive (Condition A) seems like a relatively simple problem for programmers to solve. To get even more complicated, you might have the code expire when the President leaves office; defaulting to no coup. I'm not sure why this would be a hard problem.

Is there a way (either at the initial stages of programming or at the later) to extend the "niceness" from the "presidential death world" into the "presidential survival world"?

This problem is ill-posed. "Extend the niceness" means to modify the AI. What types of modification count as extending the niceness and what ones don't?