The Social Impact of Trolley Problems

post by atomantic · 2024-05-08T03:09:05.063Z · LW · GW · 3 comments

Contents

  The Query
  Automating Morality & Environmental Problems
  Scope Creep
  Broken Social Scene
  Common Solution Failures
  Repeating This Scenario
  Conclusion
  Consequence
None
3 comments

Every few years, someone asks me what I would do to solve a Trolley Problem. Sometimes, they think I’ve never heard of it before—that I’ve never read anything about moral philosophy (e.g. Plato, Foot, Thomson, Graham)—and oh do they have a zinger for me. But for readers who are well familiar with these problems, I have some thoughts that may be new. For those who haven't done the reading, I'll provide links and some minor notes and avoid recapping in great detail.

Ask a dozen people to explain what the trolley problem is and you are likely to get a dozen variations. In this case, my contact presented the scenario as a modified Bystander variation format—though far evolved away from Philippa Foot's original version in “The Problem of Abortion and the Doctrine of the Double Effect*” in 1967, used as a tool to discuss direct versus oblique intention and the moral confusion within abortion doctrine given by the Catholic Church. Foot’s original proposal has then been refined, re-examined, and tested by Judith Jarvis Thompson for the 40+ years following 1976, and practically everyone in philosophy has discussed it since.

The Query

“What is your answer to the famous Trolly Dilemma? A trolly is heading down a track towards five people who are tied to the tracks. There’s a lever and if you pull it it switches tracks where only one person dies. We don’t know who the people are. It could be five Hitlers or five Nobel laureates. What would you do and WHY?”

Now, I have to spend some time going into great detail to break this down because unlike my inquisitor who claims that “It’s a very simple question” and you should flip the switch to “save five people because each of them has equal value and under utilitarianism, you want to maximize good,” I have spent many years researching and thinking about this problem—and while there are many ways to tweak this question or clarify it that end up changing many people’s answers (if it’s your job to operate the lever, or if the one person is also tied to the track, or nobody is actually tied to the tracks, or if you are alone, or if you are pushing a man onto the track instead of pulling a lever, or if there’s another option to jump onto the track and sacrifice yourself, or by changing the question entirely to judges, pilots, large men blocking caves, harvesting organs from dying patients, etc), my own answer to the Bystander variation remains consistent as long as the logical scope of the problem is not changed as it is within some of these variations (more on the scope creep of enhanced variations later), but it requires more lengthy explanation than “I would simply do this”—hence, this write-up.

Anyone who answers that this problem is “very simple” has not spent enough time thinking about it or researching existing writings on this topic because it has already been thoroughly examined and argued that the one thing experts agree on is that there is no simple solution whether you subscribe to utilitarianism or deontological moral intuitions. Foot even notes this difficulty in her original paper, saying “In many cases we find it very hard to know what to say, and I have not been arguing for any general conclusion such as that we may never, whatever the balance of good and evil, bring injury to one for the sake of aid to others, even when this injury amounts to death. I have only tried to show that even if we reject the doctrine of the double effect we are not forced to the conclusion that the size of the evil must always be our guide.” Further, just look at the detailed commentary on the topic from two LessWrong posts from back in 2010 (the comment threads are the really interesting parts where you can see how much debate and commentary there is on this topic): The Problem With Trolley Problems [LW · GW] (Oct 2010) and The Trolley Problem: Dodging moral questions [LW · GW] (Dec 2010). And follow that up by reading Peter A. Graham’s Thomson’s Trolley Problem (Journal of Ethics and Social Philosophy, Vol 12 No 2 (2017): Volume XII, Issue 2), which is referenced again later because it’s worth reading twice—although my solution adds more on top of Graham that seems to have not been explored by others as I argue that even consequentialist moral theorists have a hard time justifying their decisions, and can get confused when conflating the difference between environmental problems and protagonist problems.

Automating Morality & Environmental Problems

First, it’s worth diving into why this hypothetical scenario is asinine and even harmful as a consideration in the development of autonomous vehicles as it is a key insight into my answer for human bystanders and drivers. When trying to solve problems in society, we must endeavor to make good system architecture decisions about where and how to solve problems. This is partly my twenty-odd years of software and systems architecture experience leading my thinking as I’ve seen the wrong problem solved in the wrong place many times and always with terrible knock-on consequences. If you don’t find my arguments convincing, many other people have written on this topic such as Nassim JafariNaimi’s writings on the harmful potential of algorithmic morality in “Our Bodies in the Trolley’s Path, or Why Self-driving Cars Must *Not* Be Programmed to Kill” (2017), or simply search and browse nearly two decades worth of posts and discussions on LessWrong [? · GW].

Vehicles need to be able to slow and stop when they encounter uncertainty and need to avoid reaching speeds too high for their ability to respond to unknown situations (e.g. lack of visibility, high congestion, or humans walking close to the vehicle through a crowded marketplace). Building a morality engine into an autonomous vehicle attempts to solve the wrong problem in the wrong place. No matter how you manipulate the kill-decision scenario, the problem is in the environment outside the vehicle, not in the internal operation of the vehicle. Examining an extreme variation of this problem to see how awry it can get, here’s an example scenario that surfaced at the end of a line of queries from another (many years ago) who tried to argue that this problem is still relevant to autonomous vehicles:

What if a child falls from a pothole overhead while the car is driving in a tunnel? The vehicle has a reasonable expectation of driving fast, even though a child quickly drops 15 feet in front of it. Let’s assume that the fall doesn’t kill the child (miraculously). The vehicle only has enough time to consider and act on whether to run through the child or swerve to the right and smash into a barrier, which will certainly kill the passenger(s) at such high speed. The child falling from above does not provide the vehicle with any signs that danger is imminent so only these two options exist—slowing down to a halt will fail to save the child.

And here is where we get into an ugly battle of “oh but what if” as the asker of this hypothetical attempts to concoct a scenario that will never be solved by enhancing the observable safety rules of an autonomous vehicle to make it better at slowing down and stopping. They want to present an immediate scenario with two outcomes, both bad and unsolvable. Or are they? One outcome is bad, and the other is not only bad but creates a whole new set of problems that the world has never seen before (we are indeed causing new harms to come into existence). It creates new structural problems that surface as more harm over time. This is where some righteous moral authorities think they have a handle on things but are missing a major important fact of reality: more scenarios that will appear in the future as a result of how we handle this scenario today.

The problem is not the car or driver (or lack of one); the problem is this: why are children able to fall through potholes and onto an active road in this tunnel? It’s a structural problem within society that created this unexpected outcome, not a lack of a morality engine in the car/driver. By trying to solve the wrong problem in the wrong place, we create layers of new problems that didn’t exist before we tried to solve the first problem—and it leads consequentially to new social harm. If an autonomous vehicle can make this choice, now it can be tricked/hacked into erroneously killing its passengers. So we have a whole can of wormy new problems of abuse and misanalysis that erroneously kill innocent occupants while permitting society to ignore the real structural problem that is dropping children from the sky. And it happens again!

So what do we do? As designed, the car should attempt to slow down to a halt. Sadly, this isn’t enough and the child dies. It collects as many metrics about the surrounding environment as it can for post-mortem analysis and determines that despite its proper functioning within the given operating bounds, something in the environment needs to be corrected to prevent future harm. The vehicle manufacturer might go beyond responsibility to take some action and decide it’s worth updating the vehicle, adding cameras that point upward to be able to detect unexpected things falling into the lane in the future and to consider the environment potentially dangerous when an open pothole is detected above and ahead, causing it to reduce speed to a reasonable point—and this becomes an endless task of micro-optimizations. But the real solution is that an investigation looks into why the child fell through this pothole and corrects the environment so it doesn’t happen again. This is looking at the correct problem and addressing it in the architecturally correct place, without sacrificing unwitting victims or creating additional problems in society.

Scope Creep

Scope creep is easy to introduce into variations of the problem that can change the fundamental nature of the question. In one instance, you might swap the person walking the tracks or a person tied to the tracks with the listener’s child or other loved one. This changes the nature of the question as humans have tribal survival scope and our immediate loved ones are the highest valued tribal layer, creating a personal morality that supersedes social morality. This is no longer a question about what society should do but what a person (with a normal range of human emotions) will instinctually do regardless of social outcomes. Still, it can be fun to ask to find out if the listener is a morally consistent sociopath.

In a more extreme instance, going completely off on a tangent of morality, one might throw away the trolley entirely and get to a hypothetical that involves killing all of humanity instead of only a small number (1 or 5 is an insignificant difference compared to 8 billion so it’s tempting to raise the stakes). For instance, what if an asteroid is heading toward us and it will kill all life on Earth unless we deflect it slightly so that it crashes into the Earth in a different way that will only kill half of the population? This changes the scope of a morality question significantly as we are no longer faced with (a) “some people die” or (b) “some people die” and instead are faced with (a) “some people die” or (b) “everyone dies”. Again, we limit the possible outcomes to perfect future outcome certainty rather than probabilistic unknowns. Migrations of logical scope like this bring the conversation into a whole new arena where the answers are often very different, so beware of the ways these questions can be reworded.

So again, the challenger might step back and concoct another situation where only some people die or another smaller group of some people die:

You are an airplane pilot and the engines have failed. Everyone on the plane will die if you do nothing. You can either attempt to navigate to an empty landing zone, which is behind a mountainous path that you have a very low probability of making (non-zero probability makes these questions much more realistic than absolute certainty)—or you can immediately initiate an emergency crash landing on top of a small cluster of residential houses, which will certainly kill some people on the ground. If you subscribe to the idea that every human life is equal and that human lives supersede all other value systems, then you may argue that the 200+ lives aboard the plane are in greater number than the dozen or so in the houses below, so it would seem to be a greater good to crush the houses and kill whoever is in the way. However, this creates a fake choice. Landing on houses is simply not an option in reality despite the insistence of the inquisitor to suggest that it is. Staving off practical considerations of landing a plane on a cluster of houses, the pilot does not have consent to sacrifice these people. A real pilot will use their skills and best effort to save the lives on the plane without adding additional harm—even an effort with a low probability of success that has the potential to yield no loss of life is preferable to a certainty of killing. Foot even speaks to the issue of certainty being very rare and not worth consideration as a real-world rule-based social philosophy in her original 1967 paper.

Broken Social Scene

Back to the trolley, we can see that the given scenario is housed within a completely broken social system and yet, it demands that the listener come up with an idealized morality to operate within this non-idealized society. Why are these people tied to the train tracks? Who is tying people to train tracks and why can’t we stop them? There is a maniacal entity tying people to train tracks! Why can't the train slow down and stop before it hits anyone? Why are there no cameras and security preventing people from wandering on the track or from being hogtied to the track?

If the goal is to find an absolute moral ideal for society to enact, at least we need to try creating an idealized society that the framework will operate within—one that has security and accountability, where vehicles have observational mechanics that allow them to safely operate around unexpected environmental triggers, but where society also has protections to keep people from being tied to train tracks. But as we move closer to an idealized society, the question of humans acting individually to invoke social morality in the way this scenario suggests becomes less and less easy to define until it becomes an impossible task because this is a systems problem, not a personal morality problem. Something is fundamentally wrong in the normal operation of our social fabric and to examine this situation as a one-off scene belies the reality that we live in a system of repeating events. The trolley problem is often used to argue about things in society that happen over and over such as abortion (the original topic that Foot had formulated it around), capital punishment, warfare, insurance rates, economic disparities, and our ability to affect their outcomes—and we must keep in mind that events repeat and society builds new system responses and ongoing behaviors upon our prior actions. When we look at what society should do as generalized rules, actions do not exist in a vacuum. Actions have consequences that are not only in the immediate moment (basic Socrates from Book 2 of Plato’s Republic).

Common Solution Failures

So what do you do? Most people, ignorant of the complex research on the topic, will choose to pull the lever and kill the unsuspecting solo human, which, in the best case, creates a terrifying knock-on consequence for all members of society: your life may now be forfeited for the supposed greater good (supposedly) at any time, without your consent, and without you having done anything particularly deserving of death—and the intelligence that decides your fate is handled by someone (or something) with a privileged vantage point—and what arrogance to think that this vantage point has perfect information to predict the outcome of adding a new action to the situation. What if the second track is in extreme disrepair and switching the train will then kill everyone aboard the train as well as the solo hapless wanderer? What if the driver of the tram has another track switcher on board and has already triggered the switch as they are expecting to be able to use their horn to warn the solo track wanderer to get out of harm's way (leading to a potential reality of no harm), but by forcing the track to switch again, you undo the switch that the driver has already initiated, thereby inserting yourself as the responsible party for the failed attempt at navigating away from the five people? You may have caused the death of five people instead of reducing the body count by four—because you arrogantly thought your vantage point held perfect information for predicting the future. Indeed, after decades of writing and researching variations on the topic, Thompson determined in Turning the Trolley (2008) that it is not morally permissible for the passerby to divert the trolley onto the path of the individual victim. Instead, if an individual wants to do a certain good deed that has a cost, and this person is willing to pay the cost personally, then the good deed is permissible, which means that it is only moral if the passerby volunteers to be the victim of the train path rather than choosing someone else to stop the train against the victim's will. In this revised evaluation, consent is required for sacrifice to be morally justified. Thompson later flipped sides again (see how simple this problem is to answer?) but has been revisited and argued to have been mistaken in reversing this decision by Graham in Thomson’s Trolley Problem (again, worth the read).

In the given version of the story, the five people tied to the main track may also not have done anything to deserve or expect death, but they were forcibly placed there by a maniacal hand. If this post were a book, we could dive further into the physics of deterministic causality and debate the existence of free will and personal responsibility, but let’s not get too distracted—this is already very different from her original version in which there are five workers on one track and one worker on another, and you are the driver of the tram.

In any version of the story (excluding self-sacrifice), the trolley problem encourages the listener to become a villain as well by taking a death-dealing action, forcing someone else into that same deadly outcome (against their own right to not be harmed without consent).

An “act” utilitarian (self-labeled by my associate inquisitor in this case), who hasn’t done the research to understand means-blocking-barriers (see Dworkin’s Rights as Trumps by Jamal Greene), might do the math and reason that more than 50% of people are a positive benefit to society than are harm, so with random people assigned to these positions, saving the majority of people by sacrificing one is a better outcome—at least it will seem that way to them if they haven’t thought through the future social effects of their actions. If they were particularly pessimistic and calculated that most people are harmful to society, they would choose to end the lives of more people (and in this case, our utilitarian becomes a major threat to large populations—and oh, but rest assured, if this person wins in convincing the world to adopt their morality, the government surveillance system that is built on top of it will root out this would-be mass-murderer and prevent them from achieving their goals). We’ll discount considering the pessimistic sociopaths for now as that is a distracting rabbit hole. The trouble is that this moral calculation of lives is a trap as it leads one to think of the scenario as a protagonist problem rather than an environmental problem within the structure and functionality of the system.

Repeating This Scenario

Since my particular utilitarian also has a morality patch that upgrades his basic principles from utility maximization to include valuing counts of human lives equally (with exceptions that we won’t get into here), and only in the immediate moment, let’s see how this scenario can go very wrong:

Day 1: Our protagonist walks over a railway bridge (which for some reason has a track-switching lever on it) and notices, from their privileged and exclusive vantage point, that the train is heading toward a mass of five unknown people tied to the track! Oh, no! The alternate track has a random person walking casually down the middle, facing away from the train, too distant to hear a warning to abort! Are they wearing headphones? Are they deaf? Most certainly the train will kill them if it is diverted, this is a speed train! Visibility is so poor that age and other characteristics of the track occupiers are a complete mystery, but the bystander surely is of the most sound and capable faculties (do you question your sanity?). Woe is our protagonist who must make a choice. Our well-learned hero spends a moment recalling college philosophy, wrestling with Plato’s ideals of wisdom, justice, and courage, but time is running out! If only there were a benevolently wise authoritarian philosopher who could bestow the metaphysical true shape of justice! Harkening back to university, our hero remembers, after traversing the murky paths of moral arguments within the Greek dialogues, discovering a more satisfying and direct stance of (albeit sometimes inconsistent) utilitarianism, which states that this moment alone must be evaluated for the greatest good and acted upon, now! The protagonist (soon-to-be antagonist) runs the quick math 5 > 1 and pulls the lever to kill the single unsuspecting person. Our hero has saved five lives at the expense of one. A net loss to society of only one random life instead of five. Well done. Society recognizes our hero’s moral valor and, rather than prosecuting for negligent homicide, sees this as justifiable homicide and awards a degree in philosophy.

Day 2: Again, our protagonist/antagonist walks along the bridge (this is the path home so it’s a daily routine), and again, the same thing! The tracks are layered with bodies, tied up and helpless, struggling to get free, and once again, another person is on the other track! Have we learned nothing?! Again, our utilitarian knows what to do—after all, it worked so well last time. And once again a life is forfeit for the five. But wait, these five people are the same people that were tied up yesterday! Now two lives have been sacrificed to save these same five. Oh well, it’s still a net win! Morality vindicated!

Day 3: You guessed it, the same thing happens, the same five people are tied to the track. Another random person dies.

Day 4: Our passerby is unscathed by the notion, expecting the same group of five. Four people sacrificed for five? Still a good deal. Lever pulled!

Day 5: Well now we are in some sunk-cost territory, my friends. We’ve already killed 4 people to save these five and again avoiding five deaths is better than avoiding one death. The calculation of the moment still stands, and our hapless philosopher pulls the lever, certain that the moment is all that matters. And even if it isn’t, it’s still 5 for 5!

This repeats every day.

Everyone dies.

Except for five possibly innocent people, the train operator who it turns out is a raging psychopath who enjoys tying people to train tracks, and our lonely protagonist who is now responsible for so many deaths—many more deaths than the villain who started this game and is now deeply unsatisfied. So this time, the operator ties the five people to one track and our budding moral philosopher to the other track and finishes the job.

Conclusion

The moral failing of this trolley problem is in the myopic thinking that the protagonist’s action or inaction with the lever resolves the problem and contributes meaningfully to an idealized social morality. Environment problems must be fixed at the environmental level (identify and deal with the psychopath, put safety guards and cameras around the train tracks, etc). They can’t be band-aid resolved by arrogant, individual stabs at morality in the moment as this creates unexpected consequences that make living in society unsafe and unsound.

Consequence

Years from now, your phone buzzes, an emergency alert broadcast:

Government Intelligence Services has identified that your sacrifice will save the lives of two or more other humans. You will be dispatched momentarily by space lasers. Please hug your loved ones for the last time.

3 comments

Comments sorted by top scores.