[LINK] Utilitarian self-driving cars?

I think a less contrived question is "Should a self driving car minimize travel time for you or all people on the road?"

This is sort of fun to think about, but I don't think the actual software will look anything like a trolley-problem solver. Given a choice between swerving and hitting A, or hitting B, I predict its actual answer will be "never swerve" and all the interesting details about what A and B are will be ignored. And that will be fine, because cars almost never get forced into positions where they can choose what to crash into but can't avoid crashing entirely, especially not when they have superhuman reflexes, and manufacturers can justify it by saying that braking works slightly better when not swerving at the same time.

I'm not sure if this is going to come up in the way proposed in the article. Given a potential collision, before even calculating whether or not it is unavoidable or not, the car is likely going to start reducing speed by using the brakes, because generally that's what you need to do in almost all collisions. (The very high percent that aren't of this type.)

But once the car has jammed on the brakes, it has cut off a great deal of its ability to do any swerves. These types of cases may be so rare that giving the car the fractional second to make those calculations may lead to more deaths than just hitting the brakes sooner in all cases would.

From a utilitarian ethics point of view, I suspect the design decision may be something like "We will save 10X lives per billion vehicle miles if the car precommits to always reduce speed without thinking about it, even if we would save X more lives by thinking about how to pick swerve in certain cases... but we can't do that without NOT saving the 10X lives from immediate precommitment."

Although, once we actually have more data on self driving car crashes, I would not be surprised if I have to rethink some of the above.

They should minimize damage to their own occupants, but using some kind of superrational decision theory so they won't defect in prisoner's dilemmas against each other. I suspect that in sufficiently symmetric situations the result is the same as minimizing damage to everybody using causal decision theory.

This reminds me of the response to the surgeon's dilemma about trust in hospitals. I want to say occupants, because if fear of being sacrificed in trolley problems causes fewer people adopt safer non distractable non fatiguable robot cars then it seems like a net utilitarian loss. If that were not the case, like for example if the safety advantage became overwhelming enough that people bought them anyway, then probably it should just minimize deaths. (I only thought about this for a couple of minutes though)

Suspected Nash-equilibrium ethics for the proprietary collision avoidance algorithm:

Utilitarian: minimize negative publicity for the car maker.

Resulting Asimov-like deontology:
1) Avoid collisions with the same make cars
2) Maximize survival of the vehicle's occupants, disregarding safety of the other vehicle involved, subject to 1)
3) Minimize damage to the vehicle, subject to 1) and 2)

... Noone would ever design a car with any priority other than "minimize impact velocity" Because that is a parameter it can actually try to minimize. In the extremely unlikely case of a car smart enough to parse the question you just posed, impacts would never ever happen. Barring outright malice.

I wonder if they're actually using a utility function as in [probability * utility] or just going with [aim for safe car > unsafe car] unilaterally regardless of the likelihood of crashing into either. E.g., treating a 1% chance of crashing into the safe car and a 80% chance of crashing into the unsafe car as equal to 99% chance of crashing into the safe car and a .05% chance of crashing into the unsafe car; choosing in both cases to crash into the safe car.

Conventional mortality would dictate that the car minimize global loss of life, followed by permanent brain damage, permanent body damage. I think in the future that other algorithms will be illegal but existent.

However. The lives each car would have the most effect on would be those inside of it. So in most situations all actions would be directed towards said persons.

It should act in favor of its passengers of course.

It seems there are few distinct cases

• I am someone who does not wear helmet in our current society where this is illegal and people don't exactly discriminate in case of car accidents, so the introduction of smart cars will only confirm my current (bad) decision - no change there.

• I currently wear a helmet, but would stop wearing one if smart cars were introduced.
  Assuming every car magically became a smart car, that means I am willing to suffer a fine in exchange for a slightly greater likelihood of surviving a nearby car crash.
  Considering smart cars are better drivers than humans, and that car crashes are already rare, that means if I considered the fine adequate to incentivize me into wearing a helmet previously I should consider them adequate now.
  There is an edge case here : smart cars are better drivers, but only by a small fraction that is offset by their tendency to aim away from me.

• I currently wear a helmet, and will continue to do so.

Only the edge case would create a morally ambiguous situation, but that seems pretty unlikely (you'd hope that a swarm of cars with superhuman reaction speed would be more than marginally better at preventing accidents).

Hello, trolley problem :-)