comment by kilobug ·
2011-10-02T09:33:43.451Z · LW(p) · GW(p)
I spent quite a while thinking about this one, and here is my "answer".
My first line of questioning is "can we just multiply and compare the sufferings ?" Well, no. Our utility functions are complicated. We don't even fully know them. We don't exactly know what are terminal values, and what are intermediate values in them. But it's not just "maximize total happiness" (with suffering being negative happiness). My utility function also values things like fairness (it may be because I'm a primate, but still, I value it). The "happiness" part of my utility function will be higher for torture, the "fairness" part of it, lower. Since I don't know the exact coefficient of those two parts, I can't really "shut up and multiply".
But... well... 3^^^3 is well... really a lot. I can't get out this way, even adding correcting terms, even if it's not totally linear, even taking into account fairness, well, 3^^^3 is still going to trump over 1.
So for any realistic computation I would make of my utility function, it seems that "torture" will score higher than "dust speck". So I should chose torture ? Well, not sure yet. For I've ethical rules. What's an ethical rule ? It's an internal law (somehow, a cached thought, from my own computation, or coming from outside) that says "don't ever do that". It includes "do not torture !" it includes "nothing can ever justify torturing someone for 50 years". Why are those rules for ? There are here to protect myself from doing mistakes. Because I can't trust myself fully. I've biases. I don't have full knowledge. I've limited amount of time to take decisions, and I only run at 100Hz. So I need safeguards. I need rules, that I'll follow even when my computation tells me I shouldn't. Those rules can be overridden. But they need to be overridden by something with almost absolute certitude, and to be of the same (or higher) level. No amount of dust speck can trigger an override of the "no torture" rule. I know my history well enough to know that when you allow yourself to torture, because you're "sure" that if you don't something worse will happen, well, you end up becoming the worse. I've high ideals, I've the will to change the world for better - therefore I need rules to prevent me from becoming Stalin or the Holy Inquisition. And that's typically the case. 3^^^3 persons will receive dust speck ? Well, too bad. Sure, it'll be a less optimal utility function than allowing just one person to be tortured. But I don't trust myself to sentence that person to be tortured. So I'll chose dust specks for me and everyone.
If you allow me to argue by fictional evidence, well, that reminds me to the end of Asimov Robot cycle (Robots and Empire, mostly). Warning: spoilers coming. If you didn't read it, go read it, and skip the rest of that paragraph ;) So... when the two robots, Daneel and Giskard, realize the limitations of the First Law: « A robot may not injure a human being or, through inaction, allow a human being to come to harm. », and try to craft the Law Zero: « A robot may not harm humanity, or, by inaction, allow humanity to come to harm. », they end up facing a very difficult problem - one for which they'll need Psychohistory to solve, and even then, only partially. It's relatively easy to know that a human being is in danger,or suffering, and how to help me. It's much, much harder to know that humanity is in danger and how to help it. That's a deep reason behind ethical rules : torture someone is just plain wrong. I may think it's good in that situation, because it'll prevent a Terror Attack, or help me win the war against that horrible Enemy, or because it'll deter crime, or because it'll save 3^^^3 persons from a dust speck. But I just don't trust myself enough to go as far as to torture someone because I computed it would do good overall.
And the last important point on the issue is social rules. There is, in XXIest century western societies at least, a strong taboo on torture. That taboo is a shield. It means than when a president of the USA uses torture, he loses the elections (of course, it's much more complicated, but I think it did play a role). It makes using torture a very, very costly strategy. We have the same with political violence. When the cops attacked a anti-war protest at the Charonne metro station on Feb 8, 1962, killing 9 demonstrators including a 16 years old boy, that was the end of Algeria war. Of course, it wasn't just that. De Gaulle was already trying to stop war, it was lost. But the uproar (nearly half a million of people attended their burial) was so high that the political cost of still supporting the war was made much bigger, so that the end of the war was hastened.
I won't take the responsibility of weakening those taboo (against torture, against political violence, ...) by breaking them myself. The consequences on society, on further people using more torture later on, are too scary.
So, to conclude, I'll chose dust specks. Not because my utility function scores higher on dust speck. But because I can't trust myself enough to wield something as horrible as torture (I've ethical rules, and I'll follow them, even when my computations tell me to do otherwise, for it's the only safeguard I know against becoming Stalin) and because I value way too much the societal taboo against torture to take the responsibility of lowering it.
Now... I've a feeling of discontent for reaching that conclusion, because it coincide with my initial gut-level reaction to the post. It somehow feel like I wrote the bottom line, and then the rationalization. But... I did my best, I did overcome the first "excuse" (non-linearity and valuing fairness) my mind gave me. But I don't find flaws in the two others. And well, reversed stupidity is not intelligence. Reaching the same conclusion that I had intuitively doesn't always mean it's a wrong conclusion.