Posts
Comments
Eliezer, why didn't you answer the question I asked at the beginning of the comment section of this post?
I would greatly prefer that there be Babyeaters, or even to be a Babyeater myself, than the black hole scenario, or a paperclipper scenario. This strongly suggests that human morality is not as unified as Eliezer believes it is... like I've said before, he will horrified by the results of CEV.
Or the other possibility is just that I'm not human.
About the comments on compromise: that's why I changed my mind. The functions are so complex that they are bound to be different in the complex portions, but they also have simplifying terms in favor of compromise, so it is possible that everyone's morality will end up the same when this is taken into account.
As for the probability that Eliezer will program an AI, it might not be very low, but it is extremely low that his will be the first, simply because so many other people are trying.
I wonder if Eliezer is planning to say that morality is just an extrapolation of our own desires? If so, then my morality would be an extrapolation of my desires, and your morality would be an extrapolation of yours. This is disturbing, because if our extrapolated desires don't turn out to be EXACTLY the same, something might be immoral for me to do which is moral for you to do, or moral for me and immoral for you.
If this is so, then if I programmed an AI, I would be morally obligated to program it to extrapolate my personal desires-- i.e. my personal desires, not the desires of the human race. So Eliezer would be deceiving us about FAI: his intention is to extrapolate his personal desires, since he is morally obligated to do so. Maybe someone should stop him before it's too late?
For all those who have said that morality makes no difference to them, I have another question: if you had the ring of Gyges (a ring of invisibility) would that make any difference to your behavior?
Some people on this blog have said that they would do something different. Some people on this blog have said that they actually came to that conclusion, and actually did something different. Despite these facts, we have commenters projecting themselves onto other people, saying that NO ONE would do anything different under this scenario.
Of course, people who don't think that anything is right or wrong also don't think it's wrong to accuse other people of lying, without any evidence.
Once again, I most certainly would act differently if I thought that nothing was right or wrong, because there are many things that I restrain myself from doing precisely because I think they are wrong, and for no other reason-- or at least for no other reason strong enough to stop me from doing them.
Pablo, according to many worlds, even if it is now raining in Oxford, yesterday "it will rain in Oxford tomorrow" and "it will not rain in Oxford tomorrow" were both equally true, or both equally false, or whatever. In any case, according to many worlds, there is no such thing as "what will happen", if this is meant to pick some particular possibility like rain in Oxford.
Nick Tarleton, what is your definition of free will? You can't even say the concept is incoherent without a definition. According to my definition, randomness definitely gives free will.
Z.M. Davis, "I am consciously aware that 2 and 2 make 4" is not a different claim from "I am aware that 2 and 2 make 4." One can't make one claim without making the other. In other words, "I am unconsciously aware that 2 and 2 make 4" is a contradiction in terms.
If an AI were unconscious, it presumably would be a follower of Daniel Dennett; i.e. it would admit that it had no qualia, but would say that the same was true of human beings. But then it would say that it is conscious in the same sense that human beings are. Likewise, if it were conscious, it would say it was conscious. So it would say it was conscious whether it was or not.
I agree in principle that there could be an unconscious chatbot that could pass the Turing test; but it wouldn't be superintelligent.
Ben, what do you mean by "measurable"? In the zombie world, Ben Jones posts a comment on this blog, but he never notices what he is posting. In the real world, he knows what he is posting. So the difference is certainly noticeable, even if it isn't measurable. Why isn't "noticeable" enough for the situation to be a useful consideration?
Paul: "we are morally obliged to kill everyone we meet" has no scientific implications, but it definitely has moral implications. To speak plainly, your position is false, and obviously so.
Some children (2-4 years of age) assume that other human beings are zombies, because they do not meet their model of a conscious observer, e.g. adults don't go and eat the ice cream in the freezer, even though no one can stop them, and even though any conscious being would of course eat that ice cream, if it were in its power.
This fact actually is one proof that Caledonian and others are talking nonsense in saying that the zombie world is incoherent; everyone involved in this discussion, including Caledonian, knows exactly what the world would be like if it were a zombie world, and none of us think that the world is actually that way (except maybe poke-- he may actually believe the zombie world is real.)
Roko, I strongly suspect that a limitedly caring universe just reduces to a materialist universe with very complex laws. For example, isn't it kind of like magic that when I want to lift my hand, it actually moves? What would be the difference if I could levitate or change lead into gold? If the universe obeys my will about lifting my hand, why shouldn't it obey in other things, and if it did, why would this be an essential difference?
Jeffrey, do you really think serial killing is no worse than murdering a single individual, since "Subjective experience is restricted to individuals"?
In fact, if you kill someone fast enough, he may not subjectively experience it at all. In that case, is it no worse than a dust speck?
Jeffrey, on one of the other threads, I volunteered to be the one tortured to save the others from the specks.
As for "Real decisions have real effects on real people," that's absolutely correct, and that's the reason to prefer the torture. The utility function implied by preferring the specks would also prefer lowering all the speed limits in the world in order to save lives, and ultimately would ban the use of cars. It would promote raising taxes by a small amount in order to reduce the amount of violent crime (including crimes involving torture of real people), and ultimately would promote raising taxes on everyone until everyone could barely survive on what remains.
Yes, real decisions have real effects on real people. That's why it's necessary to consider the total effect, not merely the effect on each person considered as an isolated individual, as those who favor the specks are doing.
Eliezer, I have a question about this: "There is no finite amount of life lived N where I would prefer a 80.0001% probability of living N years to an 0.0001% chance of living a googolplex years and an 80% chance of living forever. This is a sufficient condition to imply that my utility function is unbounded."
I can see that this preference implies an unbounded utility function, given that a longer life has a greater utility. However, simply stated in that way, most people might agree with the preference. But consider this gamble instead:
A: Live 500 years and then die, with certainty.
B: Live forever, with probability 0.000000001%; die within the next ten seconds, with probability 99.999999999%
Do you choose A or B? Is it possible to choose A and have an unbounded utility function with respect to life? It seems to me that an unbounded utility function implies the choice of B. But then what if the probability of living forever becomes one in a googleplex, or whatever? Of course, this is a kind of Pascal's Wager; but it seems to me that your utility function implies that you should accept the Wager.
It also seems to me that the intuitions suggesting to you and others that Pascal's Mugging should be rejected similarly are based on an intuition of a bounded utility function. Emotions can't react infinitely to anything; as one commenter put it, "I can only feel so much horror." So to the degree that people's preferences reflect their emotions, they have bounded utility functions. In the abstract, not emotionally but mentally, it is possible to have an unbounded function. But if you do, and act on it, others will think you a fanatic. For a fanatic cares infinitely for what he perceives to be an infinite good, whereas normal people do not care infinitely about anything.
This isn't necessarily against an unbounded function; I'm simply trying to draw out the implications.
About the slugs, there is nothing strange in asserting that the utility of the existence of something depends partly on what else exists. Consider chapters in a book: one chapter might be useless without the others, and one chapter repeated several times would actually add disutility.
So I agree that a world with human beings in it is better than one with only slugs: but this says nothing about the torture and dust specks.
Eisegetes, we had that discussion previously in regard to the difference between comparing actions and comparing outcomes. I am fairly sure I would not torture someone to save New York (at least not for 50 years), but this doesn't mean I think that the fact of someone being tortured, even for 50 years, outweighs the lives of everyone in New York. I might simply accept Paul's original statement on the matter, "Torture is wicked. Period."
It does matter how it is done, though. In my lever case, if the lever were set to cause the dust specks, I would definitely move it over to the 50 year torture side.
Another factor that no one has yet considered (to make things more realistic). If there were 3^^^3 people, googleplexes of them would certainly be tortured for 50 years (because the probability of someone being tortured for 50 years is certainly high enough to ensure this). So given an asymptote utility function (which I don't accept), it shouldn't matter if one more person is tortured for 50 years.
Z.M. Davis, that's an interesting point about the slugs, I might get to it later. However, I suspect it has little to do with the torture and dust specks.
Doug, here's another problem for your proposed function: according to your function, it doesn't matter whether a single person takes all the pain or if it is distributed, as long as it sums to the same amount according to your function.
So let's suppose that the pain of solitary confinement without anything interesting to do can never add up to the pain of 50 years torture. According to this, would you honestly choose to suffer the solitary confinement for 3^^^3 years, rather than the 50 years torture?
I suspect that most people would prefer to take the torture and get on with their lives, instead of suffering for the confinement for eternity.
But if you modify the function to allow for this, more preference reversals are coming: for we can begin to decrease the length of the solitary confinement by a microsecond while increasing the number of people who suffer it by a large amount.
In order to prevent an extremely short confinement for 3^^^3 people from exceeding the torture, which would presumably imply the same possibility for dust specks, you will have to say that there is some length of solitary confinement for some number of people, such that solitary confinement for Almost Infinite people, for a length of time shorter by the shortest possible noticeable time period, can never be worse.
Would you hold to this too, or would you honestly prefer the 3^^^3 years confinement to 50 years torture?
Notice that in Doug's function, suffering with intensity less than 0.393 can never add up to 50 years of torture, even when multiplied infinitely, while suffering of 0.394 will be worse than torture if it is sufficiently multiplied. So there is some number of 0.394 intensity pains such that no number of 0.393 intensity pains can ever be worse, despite the fact that these pains differ by 0.001, stipulated by Doug to be the pain of a dust speck. This is the conclusion that I pointed out follows with mathematical necessity from the position of those who prefer the specks.
Doug, do you actually accept this conclusion (about the 0.393 and 0.394 pains), or you just trying to show that the position is not logically impossible?
Ben P: the arrangement of the scale is meant to show that the further you move the lever toward 3^^^3 dust specks, the worse things get. The torture decreases linearly simply because there's no reason to decrease it by more; the number of people increases in the way that it does because of the nature of 3^^^3 (i.e. the number is large enough to allow for this). The more we can increase it at each stop, the more obvious it is that we shouldn't move the lever at all, but rather we should leave it at torturing 1 person 50 years.
Ben, here's my new and improved lever. It has 7,625,597,484,987 settings. On setting 1, 1 person is tortured for 50 years plus the pain of one dust speck. On setting 2, 3 persons are tortured for 50 years minus the pain of (50-year torture/7,625,597,484,987), i.e. they are tortured for a minute fraction of a second less than 50 years, again plus the pain of one dust speck. On setting 3, 3^3 persons, i.e. 27 persons, are tortured for 50 years minus two such fractions of a second, plus the pain of one dust speck. On setting 4, 3^27, i.e. 7,625,597,484,987 persons are tortured for 50 years minus 3 such fractions, plus the pain of one dust speck....
Once again, on setting 7,625,597,484,987, 3^^^3 persons are dust specked.
Will you still push the lever over?
To your voting scenario: I vote to torture the terrorist who proposes this choice to everyone. In other words, asking each one personally, "Would you rather be dust specked or have someone randomly tortured?" would be much like a terrorist demanding $1 per person (from the whole world), otherwise he will kill someone. In this case, of course, one would kill the terrorist.
I'm still thinking about the best way to set up the lever to make the point the most obvious.
Ben: suppose the lever has a continuous scale of values between 1 and 3^^^3. When the lever is set to 1, 1 person is being tortured (and the torture will last for 50 years.). If you set it to 2, two people will be tortured by an amount less the first person by 1/3^^^3 of the difference between the 50 years and a dust speck. If you set it to 3, three people will be tortured by an amount less than the first person by 2/3^^^3 of the difference between the 50 years and the dust speck. Naturally, if you pull the lever all the way to 3^^^3, that number of people will suffer the dust specks.
Will you pull the lever over to 3^^^3? And if so, will you assert that things are getting better during the intermediate stages (for example when you are torturing a google persons by an amount less than the first person by an entirely insignificant quantity?) And if things first get worse and then get better, where does it change?
Will you try to pull the lever over to 3^^^3 if there's a significant chance the lever might get stuck somewhere in the middle?
Adam, by that argument the torture is worth 0 as well, since after 1,000,000 years, no one will remember the torture or any of its consequences. So you should be entirely indifferent between the two, since each is worth zero.
Also: I wonder if Robin Hanson's comment shows concern about the lack of comments on his posts?
Eisegetes, would you pull the lever if it would stop someone from being tortured for 50 years, but inflict one day of torture on each human being in the world? And if so, how about one year? or 10 years, or 25? In other words, the same problem arises as with the specks. Perhaps you can defend one punch per human being, but there must be some number of human beings for whom one punch each would outweigh torture.
Salutator, I never said utilitarianism is completely true.
Ben: as I said when I brought up the sand example, Eliezer used dust specks to illustrate the "least bad" bad thing that can happen. If you think that it is not even a bad thing, then of course the point will not be apply. In this case you should simply move to the least thing which you consider to be actually bad.
Paul : "Slapping each of 100 people once each is not the same as slapping one person 100 times."
This is absolutely true. But no one has said that these two things are equal. The point is that it is possible to assign each case a value, and these values are comparable: either you prefer to slap each of 100 people once, or you prefer to slap one person 100 times. And once you begin assigning preferences, in the end you must admit that the dust specks, distributed over multiple people, are preferable to the torture in one individual. Your only alternatives to this will be to contradict your own preferences, or to admit to some absurd preference such as "I would rather torture a million people for 49 years than one person for 50."
Caledonian, offering an alternative explanation for the evidence does not imply that it is not evidence that Eliezer expends some resources overcoming bias: it simply shows that the evidence is not conclusive. In fact, evidence usually can be explained in several different ways.
Eliezer's question for Paul is not particularly subtle, so I presume he won't mind if I give away where it is leading. If Paul says yes, there is some number of dust specks which add up to a toe stubbing, then Eliezer can ask if there is some number of toe stubbings that add up to a nipple piercing. If he says yes to this, he will ultimately have to admit that there is some number of dust specks which add up to 50 years of torture.
Rather than actually going down this road, however, perhaps it would be as well if those who wish to say that the dust specks are always preferable to the torture should the following facts:
1) Some people have a very good imagination. I could personally think of at least 100 gradations between a dust speck and a toe stubbing, 100 more between the toe stubbing and the nipple piercing, and as many as you like between the nipple piercing and the 50 years of torture.
2) Arguing about where to say no, the lesser pain can never add up to the slightly greater pain, would look a lot like creationists arguing about which transitional fossils are merely ape-like humans, and which are merely human-like apes. There is a point in the transitional fossils where the fossil is so intermediate that 50% of the creationists say that it is human, and 50% that it is an ape. Likewise, there will be a point where 50% of the Speckists say that dust specks can add up to this intermediate pain, but the intermediate pain can't add up to torture, and the other 50% will say that the intermediate pain can add up to torture, but the specks can't add up the intermediate pain. Do you really want to go down this path?
3) Is your intuition about the specks being preferable to the torture really greater than the intuition you violate by positing such an absolute division? Suppose we go down the path mentioned above, and at some point you say that specks can add up to pain X, but not to pain X+.00001 (a representation of the minute degree of greater pain in the next step if we choose a fine enough division). Do you really want to say that you prefer that a trillion persons (or a google, or a googleplex, etc) than that one person suffer pain X+.00001?
While writing this, Paul just answered no, the specks never add up to a toe stub. This actually suggests that he rounds down the speck to nothing; you don't even notice it. Remember however that originally Eliezer posited that you feel the irritation for a fraction of a second. So there is some pain there. However, Paul's answer to this question is simply a step down the path laid out above. I would like to see his answer to the above. Remember the (minimally) 100 gradations between the dust speck and the toe stub.
The fact that Eliezer has changed his mind several times on Overcoming Bias is evidence that he expends some resources overcoming bias; if he didn't, we would expect exactly what you say. It is true that he hasn't changed his mind often, so this fact (at least by itself) is not evidence that he expends many resources in this way.
Ben and Mitchell: the problem is that "meaningless inconvenience" and "agony" do not seem to have a common boundary. But this is only because there could be many transitional stages such as "fairly inconvenient" and "seriously inconvenient," and so on. But sooner or later, you must come to stages which have a common boundary. Then the problem I mentioned will arise: in order to maintain your position, you will be forced to maintain that pain of a certain degree, suffered by any number of people and for any length of time, is worse than a very slightly greater pain suffered by a single person for a very short time. This may not be logically incoherent but at least it is not very reasonable.
I say "a very slightly greater pain" because it is indeed evident that we experience pain as something like a continuum, where it is always possible for it to slowly increase or decrease. Even though it is possible for it to increase or decrease by a large amount suddenly, there is no necessity for this to happen.
I agree that as you defined the problems, both have problems. But I don't agree that the problems are equal, for the reason stated earlier. Suppose someone says that the boundary is that 1,526,216,123,000,252 dust specks is exactly equal to 50 years of torture (in fact, it's likely to be some relatively low number like this rather than anything like a googleplex.) It is true that proving this would be a problem. But it is no particular problem that 1,526,216,123,000,251 dust specks would be preferable to the torture, while the torture would be preferable to 1,526,123,000,253 dust specks would be worse than the torture: the point is that the torture would differ from each of these values by an extremely tiny amount.
But suppose someone defines a qualitative boundary: 1,525,123 degrees of pain (given some sort of measure) has an intrinsically worse quality from 1,525,122 degrees, such that no amount of the latter can ever add up to the former. It seems to me that there is a problem which doesn't exist in the other case, namely that for a trillion people to suffer pain of 1,525,122 degrees for a trillion years is said to be preferable to one person suffering pain of 1,525,123 degrees for one year.
In other words: both positions have difficult to find boundaries, but one directly contradicts intuition in a way the other does not.
Ben, I think you might not have understood what I was saying about the poll. My point was that each individual is simply saying that he does not have a problem with suffering a dust speck to save someone from torture. But the issue isn't whether one individual should suffer a dust speck to save someone, but whether the whole group should suffer dust specks for this purpose. And it isn't true that the whole group thinks that the whole group should suffer dust specks for this purpose. If it were, there wouldn't be any disagreement about this question, since I and others arguing this position would presumably be among the group. In other words, your argument from hypothetical authority fails because human opinions are not in fact that consistent.
Suppose that 1 person per google (out of the 3^^^3 persons) is threatened with 50 years of torture? Should each member of the group accept a dust speck for each person threatened with torture, therefore burying the whole group in dust?
Caledonian, of course that cannot be demonstrated. But who needs a demonstration? Larry D'anna said, "A googolplex of dusty eyes has the same tiny negative utility as one dusty eye as far as I'm concerned." If this is the case, does a billion deaths have the same negative utility as one death?
To put it another way, everyone knows that harms are additive.
Caledonian, can you give an example of someone who has never held a daft belief?
Other than yourself, of course, since such a suggestion would seem to indicate bias. On the other hand, your disrespect towards all others with whom you disagree (which seems to be everyone on some topic or other) seems to suggest that you believe that they all hold daft beliefs.
Caledonian, if you add the premise that some people should be respected, it is a logically valid and necessary conclusion (given that all people at some point have daft beliefs) that not all people who hold daft beliefs should be disrespected.
However, that is certainly not the best I can do. I could think of a long list of reasons for respecting such people: much the same reasons why you would do much better to show some respect for the authors and readers of Overcoming Bias. For one thing, you would have a much better chance of persuading them of your position, and if your position is true, then this is a very good effect. For another, everyone (even people holding daft beliefs) also knows some things that other people don't know, so if your respect is based on knowledge (as you seem to imply) then everyone should be respected. One last obvious point: most people will not respect someone who does not respect them: this is why you are not respected by those in this forum. But perhaps you enjoy this anyway?
Caledonian, one reason to do that is that everyone has daft beliefs once in a while. It isn't surprising that you ask the question, however, since you show no respect for those with whom you disagree on Overcoming Bias. Since you disagree with them, you presumably think that their beliefs are false, and consequently (according to your logic) that they themselves are unworthy of respect.
Steve, maybe this was your point anyway, but the incidents you mention indicate that the existence of flying saucer cults is evidence for the existence of aliens (namely by showing that the cults were based on seeing something in the real world.) No doubt they aren't much evidence, especially given the prior improbability, but they are certainly evidence.
I should add that this is true about self-contradictory religions as well. For the probability that I mistakenly interpret the religion to be self-contradictory is greater than the probability that the chocolate cake is out there.
In general, any claim maintained by even a single human being to be true will be more probable, simply based on the authority of that human being, than some random claim such as the chocolate cake claim, which is not believed by anyone.
There are possibly some exceptions to this (and possibly not), but in general there is no particular reason to include religions as exceptions.