Comment by Evan_Crowe on Domesticating reduced impact AIs · 2013-02-18T22:57:20.160Z · LW · GW

Oh, now I feel silly. The horse IS the other universe.

Comment by Evan_Crowe on Domesticating reduced impact AIs · 2013-02-18T21:05:57.110Z · LW · GW

I think there might be a miscommunication going on here.

I see Stuart arguing that Genetic algorithms function independent of physics in terms of their consistent "friendly" trait. i.e. if in universe a there is a genetic algorithm that finds value in expressing the "friendly" trait, then that algorithm would, if placed in universe b (where the boundary conditions of the universe were slightly different) would tend to eventually express that "friendly" trait again. Thereby meaning robust (when compared to systems that could not do this)

I don't necessarily agree with that argument, and my interpretation could be wrong.

I see Eliezer arguing that evolution as a system doesn't do a heck of a lot, when compared to a system that is designed around a goal and involves compensation for failure. i.e. I can't reproduce with a horse, this is a bad thing because if I were trapped on an island with a horse our genetic information would die off, where in a robust system, I could breed with a horse, thereby preserving our genetic information.

I'm sorry if this touches too closely on the entire "well, the dictionary says" argument.