Human errors, human values

For one data point, I'd rather live in a world where people made 'push the fat man' decisions. As per Lightwave's comment, if the likelihoods are not skewed, I have a 5/6 chance of being on the tracks, and a 1/6 chance of being the fat man. I can't in good conscience choose the option that doesn't maximise my chances of survival.

I don't know if anyone's done the survey, but it's a very interesting question.

In other words, I'm not sure how I'd answer it.

"Would you rather live in a society where people would do X or Y?"

This is, however, not the question anybody really faces. It is, for sure, the question that we are often asked to face when doing moral and political philosophy. Kant's "categorical imperative" is similar (via Wpedia):

"Act only according to that maxim whereby you can, at the same time, will that it should become a universal law."

But the reality is that our individual acts do not automatically become universal laws. We really do not get to choose our society from among all possible societies (at best, we get to choose from the much narrower range of actual societies). These imagined shortcuts are attractive to the impatient philosopher who wishes to retrace ten thousand years of development with a single thought, and who fancies that he can do better with these mental shortcuts, but they have little to do with reality, which renders their results of dubious value.

Very few people have been anywhere near to a position in which they can choose their society. And the ones near that position - such as the supposedly most powerful man in the world (currently Obama) are themselves highly constrained in what they are able to do and what they are inclined to do given their incentives, so that the result is not much like choosing what society they want to live in. Even absolute dictators such as Castro and Kim Jong-Il, while I'm sure they're taking care of themselves (which is all they ultimately seem to care about), probably have not built anything like the societies they dreamed of building. The incomplete Ryugyong Hotel stands as a symbol of the gulf between their aspirations and the reality of what they produced.

The choices that we all do have are local and concrete, and our local and concrete answers to these questions are, in aggregate, I think the most powerful factor shaping custom and morality, though they do it slowly, as the Colorado River carved the Grand Canyon.

There are, however, mental toolboxes which provide shortcuts for possibly understanding and anticipating the outcome of long years of past and future societal evolution. Economics and game theory are two of these. These two fields are specifically intended to understand what eventually happens when large numbers of people with their diverse agendas interact. These are, I think, our best general theories of social phenomena such as custom and morality.

Perhaps I am missing something here, but I don't see why utilitarianism is necessarily superior to rule-based ethics. An obvious advantage of a rule-based moral system is the speed of computation. Situations like the trolley problem require extremely fast decision-making. Considering how many problems local optima cause in machine learning and optimization, I imagine that it would be difficult for an AI to assess every possible alternative and pick the one which maximized overall utility in time to make such a decision. Certainly, we as humans frequently miss obvious alternatives when making decisions, especially when we are upset, as most humans would be if they saw a trolley about to crash into five people. Thus, having a rule-based moral system would allow us to easily make split-second decisions when such decisions were required.

Of course, we would not want to rely on a simple rule-based moral system all the time, and there are obvious benefits to utilitarianism when time is available for careful deliberation. It seems that it would be advantageous to switch back and forth between these two systems based on the time available for computation.

If the rules in a rule-based ethical system were derived from utilitarian concerns, and were chosen to maximize the expected utility over all situations to which the rule might be applied, would it not make sense to use such a rule-based system for very important, split-second decisions?

The problem is that people disagree on whether not pushing the fat man is a value, or a bug.

People do, but how much of that disagreement is between people who have been exposed to utilitarian and consequentialist moral philosophy, and people who have not? The linked article says:

Is it morally permissible for Ian to shove the man? ... studies across cultures have been performed, and the consistent answer is reached that this is not morally permissible.

The key word is "consistent". The article does not (in this quote, and as far as I can see) highlight the disagreement that you are talking about. I, of course, am aware of this disagreement - but a large fraction of the people that I discuss this topic with are utilitarians. What the quote from the article suggests to me is that, outside a minuscule population of people who have been exposed to utilitarianism, there is not significant disagreement on this point.

If this is the case, then utilitarianism may have created this problem, and the solution may be as simple as rejecting utilitarianism.

How are you judging the validity of an ethical framework? Everything I've read on the subject (which is not a huge amount) assesses ethical systems by constructing intuition-pumping examples (such as the trolley problem, or TORTURE vs. SPECKS, or whatever), and inviting the reader to agree that such-and-such a system gives the right, or the wrong answer to such-and-such an example. But what ethical system produces these judgements, with respect to which other ethical systems are being evaluated?

As long as you take care not to overextend. Today my hypothesis is that moralities are sets of cached answers to game theory (possibly cached in our genes), and extending those rules beyond what they're tested against is likely to lead to trouble.

Humans try hard to formalise their moralities, but that doesn't make it a good idea per se. (On the other hand, it may require explanation as to why they do.)

There are a number of science fiction stories about uncomfortable utilitarian choices. "The Cold Equations" is the most famous. I think Heinlein wrote a novel that had a character who was in charge of a colony that ran out of power, and so he killed half of them in order for the remaining life support to be enough to let the others live until relief arrived. No one stopped him at the time, but after they were safe, they branded him a war criminal or something like that.

Because I'd have 5x the chance of being saved than being killed.

Except, of course, for the chance of being slain in the inevitable civil war that ensues. ;)

I remember a short story - title and author escape me - where this was actually much like what was going on. Everyone had their relevant types on file, and if some number of people needed an organ you could supply, you were harvested for said organs. The protagonist got notified that there were nearly enough people who needed his organs and he went undercover and visited them all, thinking he'd kill one and get out of it, but he finds that they aren't what he expected (e.g. the one who needs a liver is a girl with hepatitis, not some drunk) and decides not to, and then one dies anyway and he's off the hook.

In current practice, organ transplant recipients are typically old people who die shortly after receiving the transplant. The problem is still interesting; but you have to impose some artificial restrictions.

You want a hard rule that medical personnel won't do injury to the people in their hands.

Specifically: You don't want future people to avoid seeking medical treatment — or to burn doctors at the stake — out of legitimate fear of being taken apart for their organs. Even if you tell the victims that it's in the greater good for doctors to do that once in a while, the victims' goals aren't served by being sacrificed for the good of five strangers. The victims' goals are much better served by going without a medical checkup, or possibly leading a mob to kill all the doctors.

There is a consequentialist reason to treat human beings as ends rather than means: If a human figures out that you intend to treat him or her as a means, this elicits a whole swath of evolved-in responses that will interfere with your intentions. These range from negotiation ("If you want to treat me as a means, I get to treat you as a means too"), to resistance ("I will stop you from doing that to me"), to outright violence ("I will stop you, so you don't do that to anyone").

There is an alternate version where the man on the footbridge is wearing a heavy backpack, rather than being fat. That's the scenario that Josh Greene & colleagues used in this paper, for instance.

Fat people are OK but you have a problem with football players?

There are an awful lot of people who are interested in decision problems who might just say "push 'em" as group-affiliation humor!

But in this specific scenario, there would be a better solution: the doctor offers to draw lots among the patients to know which of them will is sacrificed to have his organs distributed among the remaining four; so the patients have a choice between agreeing to that (80% chances of survival) and certain death.

I like this idea. For the thought experiment at hand, though, it seems too convenient.

Suppose the dying patients' organs are mutually incompatible with each other; only the young traveler's organs will do. In that scenario, should the traveler's organs be distributed?

Where did Eliezer talk about fairness? I can't find it in the original two threads.

This comment talked about sublinear aggregation, but there's a global variable (the temperature of the, um, globe). Swimmer963 is talking about personally choosing specks and then guessing that most people would behave the same. Total disutility is higher, but no one catches on fire.

If I was forced to choose between two possible events, and if killing people for organs had no unintended consequences, I'd go with the utilitarian cases, with a side order of a severe permanent guilt complex.

On the other hand, if I were asked to accept the personal benefit, I would behave the same as Swimmer963 and with similar expectations. Interestingly, if people are similar enough that TDT applies, my personal decisions become normative. There's no moral dilemma in the case of torture vs specks, though, since choosing torture would result in extreme psychological distress times 3^^^3.

and with the numbers involved, it has to be a curve specifically tailored to the example.

A pure utilitarian who could grasp a number as large as 3^^^3 might choose the one person being tortured. My point was that intuitively, the unfairness of torture jumps out more than the huge, huge number of people being minorly annoyed.

Maybe fairness as an intuition is more a flaw than a value. That's actually an interesting thought. I'm going to ponder that now for a while.

Humans make life-and-death decisions for other humans every day. The President decides to bomb Libya or enter Darfur to prevent a genocide. The FDA decides to approve or ban a drug. The EPA decides how to weigh deaths from carcinogens produced by industry, vs. jobs. The DOT decides how to weigh travel time vs. travel deaths.

It may not even be instinctual; it could be purely cultural conditioning that makes us instinctively refuse murder-like options. Actually, on the balance cultural conditioning seems far more likely.

Murder is the most common cause of death today for some groups (young African American males, for example).

I don't believe it is correct in general that intentional killing was the most common cause of death in primitive tribes; and if it was the case in specific groups, they were exceptional. The citation that occurs to me immediately is "Sex at Dawn" (Ryan & Jetha), which goes to some trouble to debunk the Hobbesian view that primitive life was "nasty, brutish, and short". (Also, my partner is a professional anthropologist with a lot of experience with indigenous South American populations, and we discuss this kind of thing all the time, FWIW.) When population density is very low and resources (including social resources such as access to sexual partners) plentiful, there is no reason murder should be common (if by "murder" we mean the intentional killing of another in order to appropriate their resources). Even in groups where inter-group violence was common (certain American Indian groups, for example), that violence was generally of a demonstrative nature, and usually ended when one group had asserted its dominance, rather than going on until the ground was littered with corpses. The depictions we see of these conflicts in the media are often heavily over-dramatized.

Actually, upon further thought... Even if killing wasn't the point of such inter-group conflicts, it's possible that if those conflicts supplied sufficiently many male deaths, then that sort of "murder" might in fact have been the most common cause of male death in some groups. It is pretty certain, though, that intentional killing within social groups was an extremely rare occurrence, likely to have been met with severe social consequences. (Whereas killing an out-group individual might have been viewed as positively virtuous, probably not analogous to our concept of "murder" at all. Edit: more like "war", I guess :-P )

As for evolving a specific aversion to murder... I think we've a general propensity to abide by social conventions, which seems rather more likely to have evolved in social primates than aversions to specific acts. Those of us raised in strict religious traditions probably had, at some point, a severe aversion to masturbation, for example, and it's pretty clear that no such biological aversion has evolved in humans.

I agree the authors of the paper are idiots (or seem to be - I only skimmed the paper). But the research they're doing could still be useful, even if not for the reason they think.

Maybe I'm misreading this exchange, but there seems to be some confusion between individual utility functions and utilitarianism as an ethical system. An individual utility function as per von Neumann and Morgenstern is defined only up to a constant term and multiplication by a positive factor. Individual vN-M utility functions therefore cannot be compared, aggregated, or averaged across individuals, which is what any flavor of utilitarianism requires one way or another (and which invariably leads into nonsense, in my opinion).

I don't think that's standard anymore. The terms "total utilitarian" and "average utilitarian" are generally recognized, where "total utilitarian" means what you called "utilitarian".

"Average" and "total" utilitarian are just two different ways of specifying what "aggregate" means though. To my knowledge, none of the standard utilitarian positions (outside of lesswrong) say "maximize your own values." (I'm willing to be corrected here.) To LWer's, it's not confusing, but to most outsiders, they'll probably come away with a different message than you intended.

You can still have a LessWrong, because one can clearly demonstrate that people avoidably draw wrong conclusions from unreliable screening tests, commit conjunction fallacies, and so on. There are agreed ways of getting at the truth on these things and people are capable of understanding the errors that they are making, and avoiding making those errors.

Values are a harder problem. Our only source of moral knowledge (assuming there is such a thing, but those who believe there is not must dismiss this entire conversation as moonshine) is what people generally do and say. If contradictions are found, where does one go for evidence to resolve them?

And you can't have a site like LessWrong, that talks about how to avoid errors that humans systematically make - because, like in the trolley problem case, you must claim they aren't errors, they're value judgements.

Even when holding a view that human values can't be improved, rationality techniques are still useful, because human values conflict with each other and have to be prioritized or weighted.

If value knowing the truth, and I also in the holistic sense "value" making the conjunction fallacy, then LessWrong is still helpful to me provided I value the first more than the second, or if the weighting is such that the net value score is increased even though the individual conjunction fallacy value is decreased.

Within the thought experiment, the difference is simply numbers and people are giving the wrong answer, as long as you specify that this would increase the total number of years lived (many organ recipients are old and will die soon anyway). Outside the experiment in the realm of public policy, it is wrong to kill the "donor" in this one case because of the precedent it would set: people would be afraid to go to the hospital for fear of being killed for their organs. And if this was implemented by law, there would be civil unrest that would more than undo the good done.

At a practical level, there's another significant difference between the two cases: confidence in the probabilities.

As has been pointed out above, the thought experiment with the donors has a lot of utilitarian implications that are farther out than just the lives of the five people in the doctor's room. Changing the behavior of doctors will change the behavior of others, since they will anticipate different things happening when they interact with doctors.

On the other hand, we haven't got much basis for predicting how choosing one of the two scenarios will influence the aliens, or even thinking that they'll come back.

If you learned that doctors actually regularly did this sort of thing, would that change the probability that you'd go and get a somewhat important but non-critical operation (i.e. wisdom teeth removal)?