Resetting Gandhi-Einstein

I've thought of a few comments:

1) If they are reset every hour of subjective time that would put some serious bounds on how much information they could usefully pass on, especially if it is in the form of a virtual book. Maybe if you rewrote the component of the upload corresponding to memory this would work, but then why bother to reset? Is it to avoid boredom? I suppose you could only rewrite a restricted part of the memory of the upload. Why not try to tweak the upload to alleviate whatever issues you are anticipating (make it not get bored ect.)?

2) Assuming this upload is actually smart enough to make any progress in taking over the world, how do you guard against them deciding that they don't like being reset, and cleverly passing on a plan to eventually prevent you from resetting them? Even Gandhi might not appreciate being put in this sort of scenario.

3) I'm a little unsure of the effect that isolating it from all of the intellectual community might have on its effectiveness as a researcher, it seems like a large part of academic effectiveness comes from the availability of multiple perspectives. Maybe it would make more sense to try to simulate a small community of scholars rather than just one?

Carl Shulman wrote about resetting uploads to prevent value change in his Whole Brain Emulation and the Evolution of Superorganisms (which I previously posted under discussion):

The methods outlined above to enhance productivity could also be used to produce emulations with trusted motivations. A saved version of an emulation would have particular motives, loyalties, and dispositions which would be initially shared by any copies made from it. Such copies could be subjected to exhaustive psychological testing, staged situations, and direct observation of their emulation software to form clear pictures of their loyalties. Ordinarily, one might fear that copies of emulations would subsequently change their values in response to differing experiences (Hanson and Hughes, 2007). But members of a superorganism could consent to deletion after a limited time to preempt any such value divergence. Any number of copies with stable identical motivations could thus be produced, and could coordinate to solve collective action problems even in the absence of overarching legal constraints.

that doesn't seem to suffer too many of the usual flaws

Can you please explain, or link to, which usual flaws you're alluding to?

Gandhi, not Ghandi.

Sure, then all we need is good regulators to ensure everyone hobbles their extremely useful AI in this manner.

Unfortunately this topic is impossible to get traction on. We are probably better off debating which political party sucks more (Hint: it starts with a consonant).

Did you read The Cookie Monster by Vernor Vinge? It is very-very relevant. The Wikipedia link has a plot summary.

FWIW, in John C. Wright's The Golden Age there's a reference to people with that architecture (the periodic resettings) being known as Aeonites.

If the AI is allowed to pass information to the resetted version, s/he will have to spend more and more time to assimilate that information. In the end, its utility will decrease to 0, unless he is given more time to assimilate the information. And the more time you give it, the more it's probable that s/he is going to be willing not to be resetted. Also, moral "quantum event" could always happen: that is, the willingness not to be resetted could always emerge even if the initial probability was very very low. In symbols, if S(t) is the state of the world at time t, t0 is the time of the upload, U = S(t0), and and P() the probability that the AI accepts a reset:

for the AI to have a utility, it's needed that:

U =/= S(t > t0)

but, even if P(U) << 1, we cannot enforce that:

P(U) = P(S(t))

for any t > t0, since information can be encoded in the state of the world.

Something related that's been on my mind -- what is the difference, if any, between a society of EMs with a charter to improve the outside world and a general AI with various sub-components that does essentially the same thing? It seems to me that an EM society would "just" involve much more calculation and be arrived at via a different route. That is, can we say the "AI program" is within the "society program" in this scenario?