Torture vs. Shampoo

I really don't think we should talk about or link to that article anymore. Patching it won't change the fact that we've anchored the discussion to dust specks. If your goal is to persuade more people to shut up and multiply (and thereby do a little to decrease the actual amount of expected torture and suffering in the real world), surely there are ways to do that that aren't quite so saturated with mind-kill. Just write a brand new post, modeled after Eliezer's — 'Cancer or hiccups?', say — that avoids...

1. ... strange and poorly understood examples. (I'm still not sure I get what this tiny 'dust speck' experience is supposed to feel like.)
2. ... examples that emphasize agency (and therefore encourage deontological responses), as opposed to worldly misfortunes.
3. ... examples that are not just emotionally volatile horrors, but are actively debated hot-button political issues.
4. ... numbers that are difficult not just for laypeople to intuitively grasp, but even to notationally unpack.

Such elements aren't even useful for neutral illustration. This is among the most important rhetorical battles the LW and EA communities have ever addressed, and among the most challenging. Why go out of our way to handicap ourselves? Just use an unrelated example.

This reminds me of the Piraha ( http://en.wikipedia.org/wiki/Pirah%C3%A3_people ) who are so well-adapted to the live in the jungle (not only by their knowledge and experience but by their language and culture) that they can't even grasp the concept of nuisance due to e.g. bugs. For them bugs are such a normal part of live that the can't even express this. Note that their language lacks abstract concepts but feature rich ways to express concrete experiences and desires. So for them the dust spec/shampoo/bug bite cost is not only zero but even unknowable. And on the other hand the future is of no consequence for them either.

Why do I mention this? Because in a way we (or most of us) are in their position with respect to our knowledge relative to a super intelligence or a very distant future. And because it may be that the cost function is not linear over such wide ranges.Linear cost assumes an unbounded universe - and 3^^^3 is way out of the limit of our universe.

I actually think that there is a fundamental problem with initial question that makes the answer different from the one Eliezer thinks.

If "shut up and multiply' was always the right answer, if you could break everything down to utilitons and go with whatever provides the most utility for the most people, then the logical thing to do would be to create a trillion intelligent beings in the solar system who do nothing but enjoy pleasure all the time. But he's made it clear that he doesn't believe that, that he believes that humans have a wide variety of values that are irreducible, that are fundamentally different, and that if you give up some of those values to satisfy others, you lose something important, and we all end up worse off. When it comes to fundamentally different values, like "freedom", that you can't just say "X freedom is worth Y pleasure, so if you can create Y+1 pleasure you should give up on the concept of freedom", not for any value of X and Y.

In this case, I would argue that the value "nobody should suffer from torture if it can be avoided" is a fundamentally different value from "it's nice to avoid minor annoyances when possible", and that you can't simply break them down into utilitons and "do the math" like he suggests.

On rereview of torture vs. specks/shampoo, I think I may see something noteworthy, which is there are multiple separate problems here and I haven't been considering all of them.

Problem 1: Given a large enough stake on the other end, a seemingly sacred value(not torturing people) isn't sacred.

Example: the last time I did the math on this, I think I calculated that the trade point was roughly around the quintillions range. That was roughly the point where I thought it might be better to just torture the one person than have quintillions of people suffer that inconvenience, because the inconvenience, multiplied by 1 quintillion (10^18), was approximately as bad as the torture, when I tried to measure both. (The specific number isn't as critical, just the rough order of magnitude, and I want to note that was assuming near 100% certainty that the threat was real.)

I think Problem 1 is the standard way to evaluate this. But there is also this:

Problem 2: Given a large enough stake on the other end, you need to reevaluate what is going on because you can't handle threats of that caliber as threats.

Ergo, If you try to actually establish how 3^^^3 trivial inconveniences has meaning, you'll generally fail miserably. You might end up saying: "You could turn the all possible histories of all possible universal branches to shampoo from the beginning of time until the projected end of time, and you STILL wouldn't have enough shampoo to actually do that, so what does that threat even mean?"

So to evaluate that, you need to temporarily make a variety of changes to how you process things, just to resolve how a threat of that level even makes sense to determine if you comply.

Problem 2 comes up sometimes, but there is also:

Problem 3: Given a sufficiently large threat, the threat itself is actually an attack, not just a threat.

For instance, someone could run the following code to print a threat to a terminal:

10: Print: "If you don't press this button to torture this person for 50 years, I'm going to give the following number of people a trivial inconvenience:"

20: Print: "3, large number of Knuth's up arrows 3, where the large number of Knuth's up arrow can be defined as:"

30: Wait 1 second.

40: If person hasn't become unable to push button and the person hasn't pushed the button, Goto 20

50: Else: Do stuff.

And some particularly simple threat evaluation code will see that threat and just hang, waiting for the threat to finish printing before deciding whether or not to press the button.

So we have:

A: Threats too small to cause you to act. (Example, 1 person get's shampoo in their eyes)

B: Threats large enough to cause you to act. (Example, a plausible chance that 10^20 people get shampoo in their eyes)

C: Threats so large they do not appear to be possible based in how you understand reality. (Example, 3^^^3 people get shampoo in their eyes), so you have to potentially reevaluate everything to process the threat.

D: Threats so large that the threat itself should actually be treated as malicious/broken, because you will never finish resolving the threat size without just setting it to infinite.

So in addition to considering whether a threat is Threat A or B, (Problem 1) it seems like I would also need to consider if it was C or D. (Problem 2 and 3)

Is that accurate?

Indeed, I make this same point every time I notice SPECKS being discussed, that the example of a trivial pain is really badly chosen.

I think Eliezer was going for something even more trival than getting shampoo in your eyes. I think I would pay 1 cent for avoiding shampoo in my eyes.

One thing I would note is that one motivation I have for not spending that much money is that I could spend it on, oh, donating to AMF or MIRI. That option wasn't in the original, though.