Future of Moral Machines - New York Times [link]

Many Singularitarians assume a lot, not the least of which is that intelligence is fundamentally a computational process.

What else it would be? Except the divine origin of thoughts nothing was submitted as an alternative so far.

Has anyone read the book that the article was a self-promotion for? (I have mediocre expectations, given the article; but mediocre would be an improvement in high-status treatment of the issue.)

On the subject of morality in robots, I would assume that when (if?) we devise a working cognitive model of an A.I. that would be indistinct from a human in every observable circumstance, the chances of it developing/learning sociopathic behaviour would be no different from a human developing psychopathic tendencies (which, although I can provide no scientific proof, I imagine is in the minority).

I know this is an abstraction that doesn't do justice to the work people are doing on working towards this model, but I think the complexities of AI are one of the things that lead certain people to the knee-jerk reaction that all post-singularity AIs will want to exterminate the human race. (possessing a phobia because you don't understand something etc etc...)

The Department of Defense report "Autonomous Military Robotics: Risk, Ethics, and Design" linked looks interesting (it doesn't seem to have been linked here before, though it's from 2008). I'll check it out.

Edit: I skimmed through the bits that looked interesting; there's an off-hand reference to "friendliness theory" but the difficult bits of getting a machine to have a correct morality seem glossed over (justified by the claim that that these are supposed to be special-purpose robots with a definite mission and orders to obey, not AGIs - though some of the stuff they describe sounds "AI hard" to me). There's some mention of robots building other robots and running amok in the risks, and some references to Kurzweil.

On the plus side for the article:

• Discussion of AI ethics in a major newspaper (we'll get out of the crank file any day now)

• Some good bridging of the inferential distance via discussion of physical robot interactions (self-driving cars, etc)