Torture vs Dust Specks Yet Again

would you seriously, given the choice by Alpha, the Alien superintelligence that always carries out its threats, give up all your work, and horribly torture some innocent person, all day for fifty years in the face of the threat of a 3^^^3 insignificant dust specks barely inconveniencing sentient beings? Or be tortured for fifty years to avoid the dust specks?

Likewise, if you were faced with your Option 1: Save 400 Lives or Option 2: Save 500 Lives with 90% probability, would you seriously take option 2 if your loved ones were included in the 400? I wouldn't. Faced with statistical people I'd take option 2 every time. But make Option 1: Save 3 lives and those three lives are your kids or option 2: Save 500 statistical lives with 90% probability I don't think I'd hesitate to pick my kids.

In some sense, I'm already doing that. For the cost of raising three kids, I could have saved something like 250 statistical lives. So I don't know that our unwillingness to torture a loved one is a good argument against the math of the dust specks.

I'm going to make the point I always do in this discussion: I think "dust specks" is a very badly chosen example. Do we really have to make this about literally the least possible discomfort someone can suffer? I think the "least possible" aspect is preventing people from actually multiplying; we are notoriously bad with small numbers, and I think most people are rounding off "dust speck" to "exactly zero".

I propose that the experiment should instead be named "broken fingers versus torture". Would you prefer 3^^^^3 broken thumbs - someone takes a hammer, forcibly puts the thumb on a hard surface, and whacks it quite hard - over fifty years of torture? This, to my mind, at least makes it a question that humans are capable of grasping; a broken thumb does not round off to zero. Or if you feel the thumbs are too much, then at least make it something that's not literally the least possible increment of discomfort; something that doesn't get rounded to nothing. The "least possible" is the least important part of the thought experiment, yet it's the one that causes all the discomfort and angst, through a bog-standard rounding error in human brains. This really ought to be avoidable.

Violating a coherence theorem always carries with an appropriate penalty of incoherence. What is your reply to the obvious argument from circular preference?

because otherwise to an AI it would be vastly preferable the whole of Earth be blown up than 3^^^3 people suffer a mild slap to the face.

It would be utterly disastrous to create an AI which would allow someone to be slapped in the face to avoid a 1/3↑↑↑3 probability of destroying the Earth.

Moved to Discussion, fixed formatting.

This seems to me to be the kind of thing that would belong in discussion, not main.

Beyond that, I'm not sure what you're getting at. Do you think math trumps moral intuition or not? If you think math trumps moral intuitions in some cases but not others, how do you formalize that?

Also, I think "would you really do X? Really?" is a bad argument. People generally suffer from akrasia and it's well known that utilitarianism doesn't accord perfectly with people's evolved moral beliefs and motivations.

Thirdly, there may be practical considerations at play, like the ends not justifying the means (among humans) or morality being short-sighted.

I just want to say thanks to everyone for your comments and I now realize the obvious flaw of incorporating any extremely personal connection into a mathematical morality calculation. Because, as BlueSun pointed out that causes problems on whatever scale of pain involved.

if you were faced with your Option 1: Save 400 Lives or Option 2: Save 500 Lives with 90% probability, would you seriously take option 2 if your loved ones were included in the 400? I wouldn't. Faced with statistical people I'd take option 2 every time. But make Option 1: Save 3 lives and those three lives are your kids or option 2: Save 500 statistical lives with 90% probability I don't think I'd hesitate to pick my kids.

I also learned not to grandstand on morality questions. Sorry, about the "would you do it? really?" argument, I won't do that again.

However, I still fall on the side of the dust specks after rethinking the issue, but due to the reasoning that the 3^^^3 individuals would probably be willing to suffer the dust specks to save someone from torture, while the tortured person wouldn't likely be willing to be tortured to save others from dust specks.

This recent comment of mine seems relevant here.

The world, it is made of atoms and such. Quarks. What ever is the lowest level, does not matter. Not of people, not of happiness, and not of dust specks. The hypothetical utility function that corresponds to people's well being has to take at it's input the data about contents of a region of space (including laws of physics), and process that, and it has to somehow identify happiness and suffering happening in that whole region.

This hypothetical function does not have a property that well being of a region of space is equal to sum of well being of it's parts considered individually, for the obvious reasons that this won't work for very tiny parts, or because you value your head as it is more than you value your head diced into 27 cubes and rearranged randomly like a Rubik's cube - yet you can't tell the difference if you consider those cubes individually.

The dustspecks vs torture reasoning had confused f(dustspeck, dustspeck, dustspeck... n times) with n*f(dustspeck) , which would only be valid if the above-mentioned property held.

There's nothing inconsistent about having an utility function which returns the worst suffering it can find inside the region of space; it merely doesn't approximate human morality very well - and neither does an utility function that merely sums well being of people.

But there are other matters that you would regard as quantitatively comparable, yes? In which you are willing to say that one A is worse than one B, but enough Bs can add up to something worse than an A?

There are three counterarguments to "torture over specks" that I take seriously.

1) The harm coming from a speck in the eye is on a different plane to that coming from torture, so simply no amount of specks ever adds up to something as bad as torture.

2) The utilitarian calculation is overridden by the fact that the 3^^^3 people (if they were being moral about it) would not want some other person to undergo the torture, in order to avoid the specks.

3) Altruism is a mistake; you shouldn't choose 50 years of torture for yourself, for any reason.

ETA btw I'm not asserting that any of these counterarguments is correct; just that they might be.

The problem I have, intuitively, with the torture option is that "one person tortured for 50 years" is not the only consequence. Other consequences include "now we're in a society where I don't have a right not to be tortured." Sure, in theory torture will only happen when it's globally utility-maximizing. Just like in theory my emails will only be read by the government if there's reason to suspect they'll reveal a crime or that someone involved in the correspondence is not a US citizen. I have no reason to believe this kind of promise.

Yes, I know this is analogous to "But how do I know the fat man will really stop the trolley?" And that's why I didn't comment on the main post.

A more technical way of asking the basic question here is:

Assume that the "badness" of single harms and of aggregations of harms can be expressed in real numbers.

As the number of times a minor harm happens approaches infinity, does the limit of its "badness" also approach infinity? If it does, then there must be some number of paper cuts that is worse than 50 years of torture. If it doesn't, then you have another bullet to bite: there's some level of harm at which you have to draw an infinitely thin bright line between it and the "next worst" harm so that there's a finite amount of the slightly worse harm you'd prefer to an infinite amount of the slightly less bad harm.

Does anyone feel strongly that one choice is better than the other if the average number of deaths is the same?

(1) 400 out of 500 people certainly die

(2) All 500 live with 80% probability, all die with 20% probability.

All types of normative ethics run into trouble when taken to extremes and utilitarianism is no exception. Whether Eliezer's idea that a super-smart intelligence can come up with some new CEV-based ethics that never leads to any sort of repugnant conclusion, or at least minimizes its repugnance according to some acceptable criterion, is an open question.

From my experience dealing with ideas from people who are much smarter than me, I suspect that the issues like dust specks vs torture or the trolley problem result from our constrained and unimaginative thinking and would never arise or even make sense in an ethical framework constructed by an FAI, assuming FAI is possible.

I wonder if there is any right answer. We are talking about moral intuitions, feelings, which come from some presumably algorithmic part of our brain which seems it can be influenced, at least somewhat changed, by our rational minds. When we ask the question: which is worse 3^^^3 dust specks of 50 years of torture, are we asking what our feeling would be when confronted with a real world in which somehow we could see this happening? As if we could ever see 3^^^3 of anything? Because that seems to be the closest I can get to defining right and wrong.

Yes, the brain is a great tool for running hypotheticals to predict how the real thing might go. So we are running hypotheticals of 3*3 dust specks and hypotheticals of 50 years of torture, and we are trying to see how we feel about them?

I think it is not that hard to "break" the built-in for determining how one feels morally about various events. The thing was designed for real world circumstances and honed by evolution to produce behavior that would cause us to cooperate in a way that would ensure our survival, enhance our productivity and reproductivity. When we get to corner cases that clearly the original moral systems could not have been designed for, because NOTHING like them ever happened in our evolution, what can we say about our moral intuition? Even if it would make sense to say there is one right answer as to how our moral intuition "should" work in this corner case, why would we put any stock behind it?

I think in real life, the economic forces of a gigantic civilization of humans would happily torture a small number of individuals for 50 years if it resulted in the avoidance of 3^^^3 dust specks. We build bridges and buildings where part of the cost is a pretty predictable loss of human life, and a predictable amount of that being painful and a predictable amount of crippling, from construction accidents. We drive and fly and expose ourselves to carcinogens. We expose others to carcinogens.

In some important sense, when I drive a car that costs $10,000 instead of one that costs $8000, but I send the extra $2000 to Africa to relieve famine, I am trading some number of African starvations for leather seats and a nice stereo.

What is the intuition I am supposed to be pumping with dust specks? I think it might be that how you feel morally about things that are way beyond the edges of the environment in which your moral intuitions evolved are not meaningfully either moral or immoral.

Nitpicking note : you should fix your formatting, the text changes size/font at various points (without any relation to the content) it's painful to read.

blee

Can you imagine Harry killing Hermione because Voldemort threatened to plague all sentient life with one barely noticed dust speck each day for the rest of time?

I think I see here what's wrong with your thinking. "all sentient life", which is what would be plagued with dust specks in this example, is incredibly, inconceivably, astronomically less than 3^^^3. Actually, it's even worse than that. It's incredibly, inconceivably, astronomically less than a googleplex (10^(10^100)). And that is incredibly, inconceivably, astronomically less than 3^^^3. Actually, no. You can add a vast number of levels before the difference between one and the one a step higher up can be accurately described as "incredibly, inconceivably, astronomically less".

From what you have written, it seems you have not internalised this. We are not talking about a trillion dust specks, a quadrillion dust specks, or a quintillion dust specks. There is a difference. A large difference.

I'd assume that 3^^^3 people would prefer to have a barely noticeable dust speck in their eye momentarily over seeing me get tortured for 50 years & thus I'd choose the dust specks. If I'm wrong, that is fine, any of those 3^^^3 people can take me to court & sue me for "damages", of which there were none. Maybe appropriate reimbursement would be something like 1/3^^^3 of a cent per person?

Your specific examples of Eliezer, me, or my best (and N best, until 4) friend(s) are contributing to the friendly AI problem, which has way, way bigger number of average affected beings than a mere 3^^^3. If you somehow remove those considerations (maybe by the entity making the offer increasing someone else's ability to contribute to an equal degree), or instead mention any family member or fifth+ best friend, then my obvious snap judgement is of coarse I'd chose the torture/death, without hesitation, and am confused and disturbed by how anyone could answer otherwise.

I'm planing on possibly doing much worse things to myself if it becomes necessary, which it might. Or to other people if I can figure out eliezers ethical injunction stuff well enough to safely make exceptions and that information tells me to.

Can you imagine Harry killing Hermione because Voldemort threatened to plague all sentient life with one barely noticed dust speck each day for the rest of time? Can you imagine killing your own best friend/significant other/loved one to stop the powers of the Matrix from hitting 3^^^3 sentient beings with nearly inconsquential dust specks? Of course not. No. Snap decision.

My breaking point would be about 10 septillion people, which is far, far less... no, wait, that's for a single-event dust speck.

What's your definition of all sentient life? Are we talking Earth, observable universe, or what? What's 'the rest of time'?

3^^^3 is so large that claims on this order of magnitude are hard to judge. See Pascal's Muggle for a discussion of this.

given the choice by Alpha, the Alien superintelligence that always carries out its threats

We call that "Omega". Let's use the same term as always.

And if you wouldn't make the sacrifice what right do you have to say someone else should make it?

The obvious response to this is that "should" and "would" are different verbs. There are probably lots of things that I "should" do that I wouldn't do. Knowing the morally proper thing is different from doing the morally proper thing.

Most of your expressed disagreement seems to stem from this: You don't imagine anyone would do this, so you claim you don't think that someone should. Once you've fully understood and internalized that these are different verbs, the confusion disappears.

I’m mainly just concerned that some factor be incorporated into the design of any Artificial Intelligence that prevents it from murdering myself and others for trivial but widespread causes.

Yeah, the factor is that we don't actually want the AI to be moral, we want it to follow humanity's Coherent Extrapolated Volition instead.