Everybody's talking about machine ethics

One of boyd's examples is a pretty straightforward feedback loop, recognizable to anyone with the slightest degree of systems engineering:

Consider, for example, what’s happening with policing practices, especially as computational systems allow precincts to distribute their officers “fairly.” In many jurisdictions, more officers are placed into areas that are deemed “high risk.” This is deemed to be appropriate at a societal level. And yet, people don’t think about the incentive structures of policing, especially in communities where the law is expected to clear so many warrants and do so many arrests per month. When they’re stationed in algorithmically determined “high risk” communities, they arrest in those communities, thereby reinforcing the algorithms’ assumptions.

This system — putting more crime-detecting police officers (who have a nontrivial false-positive rate) in areas that are currently considered "high crime", and shifting them out of areas currently considered "low crime" — diverges under many sets of initial conditions and incentive structures. You don't even have to posit racism or classism to get these effects (although those may contribute to failing to recognize them as a problem); under the right (wrong) conditions, as t → ∞, the noise (that is, the error in the original believed distribution of crime) dominates the signal.

The ninth of Robert Peel's principles of ethical policing is surprisingly relevant: "To recognise always that the test of police efficiency is the absence of crime and disorder, and not the visible evidence of police action in dealing with them." [1]

Seb, what kind of work do you "try to do" in this area? Do you have some blog posts somewhere or anything?

They mainly seem to recapitulate the same tired tropes that have been resonating through academia for literally decades.

I'm fairly new here and would appreciate a brief informal survey of these tropes. Our brilliance aside, to predict which ideas will be new to you from context clues seems silly when you might be able to provide guidance.

Interesting to me, a friend who attempted to write a program capable of verifying mathematical proofs (all of them -- a tad ambitious) said he ran into the exact same problem with

not knowing a good way to model relative computational capacity.

I'm pleased to see this posted as I think the idea of machine ethics represents an enormous opportunity for proponents of FAI, etc. Machine ethics allow us to start talking about the extent to which human values are encoded in algorithms and machines in the present day, and allows us to explain why tackling that problem here and now is worth doing. This is true even if the more apocalyptic claims in the Futurist/Superintelligence/Singularity meme-cluster never come to pass.

I would agree with you assessment that many of the people thinking about this are people with a humanities background, and that they don't seem particularly well-versed as a whole on some of the technical/mathematical issues underlying some of these problems.

I'm particularly interested in this issue because machines seem to operate better than humans at multi-dimensional optimization problems, which seems at least superficially similar to the complexity of value thesis.

I think its important to remember what ethics is and why humanity created it in the first place. Ethics is essentially a code of behavior using Morality as a guide to structure one's behavior to be both beneficial to yourself while not harming the greater society.

As early humans we decided that if we were going to work in a group we had to do two things: 1) Not kill each other) and 2) Behave in a manner that is beneficial.

This is essentially the Golden Rule. We had to decide that in order to use numbers and perseverance and ultimately reason to get to the top of the food chain.

Where is the machine's motivation to continue in this path? If its only desire is to propagate itself lots of things could get in the way of that, especially humanity. I think, for machines to get progressively more intelligent and for that to be beneficial we have to consider these problems and encode, at a very deep level accepted social norms into machines.

What's a LessWrong?

Mildly harmful pseudo scientific movement with delusions of cultish grandeur.