A humane solution was never intended.

post by Kolya · 2019-12-30T20:41:14.350Z · LW · GW · 2 comments

Contents

2 comments

In his talk AI Alignment: Why It’s Hard, and Where to Start Eliezer Yudkowsky gives "an interesting experiment in cognitive psychology" about a hospital administrator who has to decide whether to spend $1M on a sick child or "general hospital salaries, upkeep, administration, and so on". Yudkowsky mentions that

"A lot of the subjects in the cognitive psychology experiment became very angry and wanted to punish the administrator for even thinking about the question."

Yudkowsky thinks this is ironic, because they "are prohibiting a priori efficiently using money to save lives".

I couldn't find the experiment in question and Yudkowsky gives no reference to it. But let's assume that these people wanted to spend the $1M on the sick child and found the thought of doing anything else repulsive and this was the reason for their anger.

The answer to this hypothetical hospital administrator's question seemed blindingly obvious to them, because in their mind the life of a child should always take precedence over "general hospital salaries, upkeep, administration, and so on".

They seem intent on saving this hypothetical child's life. But if there are no other lives at stake, how is jumping to save the $1M kid's life "prohibiting a priori efficiently using money to save lives"?

Because the case presented here is a trolley case, at least in Yudkowsky's book. He doesn't expressly say so, but it is the only explanation for his reaction. And this is the closest he gets to it:

But if you cannot possibly rearrange the money that you spent to save more lives and you have limited money, then your behavior must be consistent with a particular dollar value on human life.

So it's safe to assume that he thinks the general hospital salaries, upkeep etc. may pay for other patients lives, perhaps by ensuring the mere existence of the hospital. This assumption about the real world seems reasonable. But take note that it wasn't part of the initial question as Yudkowsky presented it. Still he uses this assumption to create a trolley case, where people have to decide between the livelihood of this one kid and other unknown patients.

Now the funny thing about trolley cases is that they absolutely do no allow to make any assumptions about the real world. They are constructed to force one of two outcomes. If the classic trolley case happened in the real world you might just yell at those stupid people to get off the tracks, or place a stone on them to derail the train, or stop it in some other way. But this is not allowed. What people who construct trolley cases generally want is to force an inhumane, calculating response that weighs the life of one person against many. Imaginative solutions that might save everyone and the day aren't called for. Not least because they would reveal who's the one who can only think of inhumane, calculating responses in this game.

Yudkowsky makes it clear that he isn't this type of person:

By which I mean, not that you think that larger amounts of money are more important than human lives

And yet he just posed a veiled trolley case and made fun of people for jumping to a solution who didn't even know of its hidden condition. They were supposed to assume that the hospital wouldn't be able to save other lives for the money they spent on that one kid. But they were not supposed to assume that the hospital might secure additional funding by the state or through a campaign or have a financial backup plan.

A humane solution was never intended. Making fun of people for the attempt was.

2 comments

Comments sorted by top scores.