Interview with Singularity Institute Research Fellow Luke Muehlhauser

For the question about human values, there are ways to put it so that it's interesting and non-trivial.

Basically for some indeterminate but not too small fraction of all human brains?

I have a reason to believe that there is such a thing as moral progress, and it's not completely arbitrary. The reason is not merely that I feel good about my own morality. But I have trouble explaining it in a couple sentences; there is just too much inferential distance.

(It has to do with the fact that making friends with people of different nationalities or ethnicties reliably makes people less nationalist or racist, and inoculates them against experiences that foster nationalism or racism.)

If you write a post about this, maybe I will write a response post.

Would anyone be interested in a top level/discussion post of some of my more advanced thoughts and arguments on this?

Yes! Please write this post!

PS Would anyone be interested in a top level/discussion post of some of my more advanced thoughts and arguments on this? Or have I just been sloppy and missed relevant material that covered this? :)

I don't think anything like this has been posted before, or has it? I do agree most posters haven't devoted too much tough to it. I mean how can they be so certain this process is something worth keeping and something that works on all of mankind, and would still be here even if a few random events in our evolutionary past or even written history had happened differently, yet are so sure that it would not apply for AI? Think about that for a little bit, practically everyone agrees that FAI is important precisely because they are sure this process isn't going to kick in for the AI. But also most seem to think that its guaranteed to have acted on us in some way even if we as humans had a very different history (the only alternative to this interpretation is anthropics, it feels right to us because we are a in a very very luck universe where the conditions where just right so the process is turning out fine). For that matter they seem to implicitly think this process is much stronger or at the very least at least as strong as genetic (since we can now be pretty sure humans have been changing in biologically even in recorded history) and memetic evolution on the scale of a few centuries or millennia.

I can't put it into words, but I feel like not having slaves and not allowing rape within marriage are both good things that are morally superior for reasons beyond simply "I believe this and people long-ago didn't".

I mean how is it possible that this process inspires such confidence in us, while a process that has so far also given us comparably felicitous change that feels so right we often invoke a omnipotent benevolent being to explain it, evolution can terrify us one we think about it clear-headedly?

I have a hunch that if we looked at the guts of this process we may find more old sanity shattering Outer Gods waiting for us.

Considering established nomenclature perhaps we should call it Yidhra or the Nameless Mist. ^_^

Members of Yidhra's cult can gain immortality by merging with her, though they become somewhat like Yidhra as a consequence. ... She usually conceals her true form behind a powerful illusion, appearing as a comely young woman; only favored members of her cult can see her as she actually is.

Anyone who passes his utility function over to her wisdom's modification is basically home safe, because future development will overall still be progress to his eyes. Moral judgement becomes a snap as all you need to do is wait for a sufficiently long time for society to get more stuff "right", the stuff that isn't "right" in that way and is lost is just random baggage you shouldn't value anyway.

Why doesn't a parallel argument apply to material and scientific progress?

Presumably because it is possible to objectively assess the degree of material and scientific progress (whether they are good is another matter). We can tell that our current knowledge is better because we can say why it is better. If there were no no epistemological progress, LW would be in vain!

So presumably the argument that there is no moral progress hinges on morality being something that can't be objectively arrived at or verified. But examples of rational discussion of morality abound, not least on LW. If we can explain our morlity better than our predecessors we are justified in thinking it is better. (But progress in morality is not quite the same as progress in values. The values might be remain the same, with moral progress consisting of a better expression of those values).

how exactly would you distinguish the universe in which we live in from the universe in which human moral change was determined by something like a random walk through value space?

If you use this analogy again in the future, you may want to be more precise.

• Maybe by "random walk" you convey the idea that human moral change is nondeterministic, or depends on conditions in such a way that change in one direction is just as likely as change in another direction;

• Or maybe by "random walk" you mean that human moral change is robust and deterministic, but does not approach any kind of limit, or approaches a limit that is very far from our current position in values-space.

how exactly would you distinguish the universe in which we live in from the universe in which human moral change was determined by something like a random walk through value space?

If historical civilizations agree with us about as much as contemporary ones. That said, there has no more been a constant upward slope than there has been such a slope in technology of equality, we are just unusual due to the enlightenment, I think.

EDIT: I think you may be assuming that we perfectly understand what we want. If I persuade a racist all humans are people, have I changed his utility function?

how exactly would you distinguish the universe in which we live in from the universe in which human moral change was determined by something like a random walk through value space? Now naturally a random walk through value space dosen't sound like something to which you are willing to outsource future moral and value development. But then why is unknown process X that happens to make you feel sort of good, because you like what its done so far, something which inspires so much confidence that you'd like a godlike AI to emulate (quite closely) its output?

What do you mean by 'value space' (any human values or desires?) and 'moral change' (generally desirable human values?)? Also, a godlike AI is like playing with infinities when inserted in your moral calculus; a godlike AI leading to horrible consequences per your morality doesn't show that your morality was fully flawed (maybe there was a tiny loophole no human would try or be capable of exploiting). And I think you discount the possibility that there are many moral solutions like there are many solutions to chess (this is especially important when noting the impact of culture on values).

Does technological progress giving us the necessary leisure time and surplus resources to care about morality count as moral progress?

how exactly would you distinguish the universe in which we live in from the universe in which human moral change was determined by something like a random walk through value space?

I don't know, but I think that might be some of the stuff that Luke will be working on as a researcher for SIAI.

I could consent to have you shoot me, but it will still injure and possibly kill me. A child or their parent could consent to being shown graphic sexual or violent imagery, but it would still cause psychological damage. Rape is not theft but assault, and society does not allow such harm to be perpetrated just because someone thought they could handle it.

If consent to sex is given moment to moment, rather than once and forever, then marriage cannot be a durable contract

Not technically true. Since you have already said marriage is being redefined it just means that the redefinition must be to something which does not necessarily include sex---that is, a contract that allows enforced abstinence. A logically coherent concept even though I find the notion repugnant.

This idea probably just comes from looking at the Blue Brain project that seems to be aiming in the direction of WBE and uses an expensive supercomputer for simulating models of neocortical columns... right, Luke? :)

(I guess because we'd like to see WBE come before AI, due to creating FAI being a hell of a lot more difficult than ensuring a few (hundred) WBEs behave at least humanly friendly and thereby be of some use in making progress on FAI itself.)

I concur.

Ah, I decided to change it to Luke vs. Michael. Thanks for feedback.