Posts
Comments
I thought of a possible reason why they wouldn't do this. Basically, you've got two choices with the unbreakable vow ploy: Obtain a class of civil servants willing to give up their own magic to do the job (fat chance), or force the criminals to do it to each other. The natural answer is the latter right? Well yes, except the part where you have to hand a criminal whose crime was severe enough to warrant stripping some of his magic a wand and give him enough mental breathing room to perform a complicated, powerful ritual. Some of them are just gonna go along with it, sure, but you only gotta have one high-profile screwup before that kind of a policy is abolished.
Whatever theories we may have, most of them contingent on the defense professor being Voldemort and the one behind this plot (a conclusion Harry hasn't yet reached and which, frankly, there isn't enough in-universe or possibly even total evidence to make conclusive), it is NOT "obvious" that noone was meant to die. Draco almost died and if it was anyone except the defense professor behind this he was almost certainly supposed to die. Hermione was, as far as I can tell, being sentenced to life in Azkaban, which I'd rank about the same as killing her. If anything, it's obvious that this was meant to be a lethal plot and sets him pretty firmly on the "defect" side.
Morally he didn't do it, and maybe Quirrel even had a desire to kill her sitting on a back burner before Harry got involved, but her death was caused by her interaction with Harry. It is no stretch to say that there is at least one hypothetical sequence of actions Harry could have taken, even given knowledge at the time (not realizing she worked for Lucius or was an animagus) which would not have resulted in her death. Heck, doing nothing would have resulted in her not death.
That is the level of challenge Harry is taking upon himself. Not just to not kill anyone, not just to keep your hands clean, not just to save people when he can. He's declaring that if any innocent person anywhere dies and there's something he could have done differently to save them, that's his failure condition. You can't do that.
That said, I thought about it a few minutes more and it could be his resolution is really about knowing he doesn't know how bad the situation is. It's certainly possible to get through, say, a political power struggle with someone like Lord Malfoy without anyone getting killed. Harry considers it possible but doesn't yet believe that his opponent is Voldemort. If his opponent is Voldemort avoiding casualties is impossible. If his opponent is someone less evil (though still pretty nasty), and the scope of the conflict is much smaller, he might be able to pull it off.
Is it me, or does Harry's solution to this dilemma seem rather... half-assed? Ignoring potential the loss of effectiveness from his resolving to suddenly switch directions the first time things get bad, is he really going to know the first time someone dies as a result of his war? How will he know the difference? He's already gotten someone killed by his actions (Rita Skeeter, who he doesn't even know about) and another person gravely injured (that auror hurt by the rocket, who he doesn't know about but admittedly he thought the whole affair was a mistake afterward anyways). How about opportunity costs, the fact that if you handed me 100000 galleons demanding I save at least 10 lives with it I could hand you back 99000 in change. And that's before the "war" even "started"; hostilities are going to get more open and more direct from here. It's madness to think you can finish war, even a weird semi-geurella war like this, with zero casualties, or that you'll know about every one.
With the condition he gave himself anyone should be able to see that "failure" is a foregone conclusion. And there's very good odds he's not going to learn that what he's doing isn't working until he's racked up a far worse bodycount than one.
Personally, to me it struck me as something I would try on a group of first graders (provided I knew I wouldn't be sued) but not on a group of adults. They know it's just a game, and treat it as such, but nobody's going to refuse because to me that sounds like a very fun game (approached from the right mindset, anyways, and provided you make sure the audience doesn't take it too far. I'd probably hand pick people to "criticize" and make myself a member of that group so I could step in if another was being problematic). So they all do it, and they all think it doesn't matter outside the game, except that since it was so unusual a thing to ask them to do they're going to remember when you tell them why they did it at the end of the lesson, and they're not going to forget it anytime soon. Preferably do it when they would normally be having a "normal" lesson.
Harry's army is about four years too old for that angle to work, so I wouldn't expect much of the "training". I'd expect more from the entirely conscious chain of reasoning that they respect General Potter, and that he has them do all kinds of weird "training" things and an awful lot of them turn out to be good ideas, and that he told you outright that he thinks this is an important thing for you to try to do. But then, that's a conscious chain of thought, and by the time an issue like this hits conscious thought it's already passed all the lines of defense that matter. So I wouldn't expect much of it. But hey, he's already employing a scatter shot approach towards their weirdness training so if this one idea doesn't work out it doesn't cost him much more than the time it took him to plan the exercise.
"Ah!" Harry said suddenly. "I get it now. The first False Memory Charm was cast on Hermione after Professor Snape yelled at her, and showed, say, Draco and Professor Snape plotting to kill her. Then last night that False Memory was removed by Obliviation, leaving behind the memories of her obsessing about Draco for no apparent reason, at the same time she and Draco were given false memories of the duel."
Since that was the last theory Harry proposed before he switched from theories to lines of attack, and nobody fully shot it down (there was an objection, but the objection was just that it'd be difficult), I have to assume that was his working theory when he left the room. And I would not automatically assume that this scenario, which we know to be very close to the correct one, would count as "innocent" in front of a wizengamot led by the angry, racist father of the victim.
Personally, for anything except comedy I like to read moderately-long bursts rather than short snippets--when I follow works that update daily but have updates that are too small I often stop reading for a while on the assumption that when I start again later I'll have a juicy backlog to trawl through. (Part of me wonders if it's just a matter of how good the author is at finding good stopping points though). HPMOR updates are not that small, but with its plot-heaviness I think I still found that I enjoyed it better when I read entire arcs at once than when I read chapters as they came out. I even considered not reading chapters once or twice until the next existed.
Of course, individual chapters are healthier for my sleep schedule...
Personally my problem with Harry wasn't so much that he immediately assumed there was a trick (shouldn't get a probability of 1.0, no, but certainly a basket worth piling some eggs in) but that he assumed the truth would get her off. He never once stopped and asked Dumbledore and Snape "If it was proven that she had been tricked into doing this with false memories, but still cast the spell willingly and with her own hand, would the Wizengamot still convict her?" I don't even know the answer to that question, but I'd certainly ask before I assumed it was "no".
Especially considering how draconian the law is and how one of the two most important members of the judge/jury is not only the victim's father but someone already predisposed to dislike her (for what amounts to unapologetic and on-record racial discrimination). In advance I wouldn't have been surprised to see a show trial that blatantly ignored the evidence to get a conviction, though Dumbledore's faction was a bit too vocal for me to expect that with my current knowledge.
I don't think that "Lucius chose the exact same number as a stab against Dumbledore" is a very complex hypothesis. We already know that he knows part of the story and can reasonably assume he knows the whole story about Aberforth. So of course if the situation already demands that he hold someone on Dumbledore's side (sort of) for ransom for some obscene amount of money, on the assumption that it won't be paid, how could he resist rubbing that bit of salt in his nemesis's wounds?
It's not part of some bigger plan. It's not some fancy maneuver. It's just an emotional attack of opportunity aimed at Dumbledore, probably just for pride's sake.
My interpretation has been more that the 'dark' plans rely primarily on application of force (most often political rather than physical)--threatening, blackmailing, bribing--and trickery. They tend to work in the short run, but in the long run can poison his reputation (people notice how dark he acts over time) and have nasty side effects. For the most part Harry's dark plans are pretty clever, because his dark side is pretty ruthless and very clever.
If you take that definition for the plans his dark side comes up with, he actually started out with a light side plan (talk reason with Lucius, however undiplomatically) but resorted to a much darker plan B (force the issue via political loopholes) when that failed. Releasing the Dementor actually doesn't strike me as the kind of plan Harry's dark side tended to come up with, since for all its risk it doesn't really solve the problem or use his resources efficiently. It sounds like what normal Harry would come up with when very angry; the same normal Harry who was having happy thoughts about Guillotines right before the sorting.
I don't think Harry actually would have taken Dumbledore as an enemy if Dumbledore failed to save Hermione, as he clearly was trying and even using up political capitol. Only having Dumbledore stand in the way of Harry saving her would do that, and when Dumbledore realized just how determined Harry was he had the sense to step aside.
Also I'm not really sure how well "Delegitimized the Wizengamot in the eyes of Magical Britain" would have worked--rest of the world yes, but the papers were certainly doing a hatchet job on her. The question is how representative of the populace is the press? Obviously the biggest papers is Lucius's and Fudge's soapbox here and in canon, but there's more than one paper in those newsstands and dissent isn't illegal until the death eaters take over in the last few books. I'm going to go with "not at all representative of public opinion", but propaganda exists because it works and they sounded prepared to present a unified front.
The rest, though, sound like things he could have planned on and represent MASSIVE gains for Voldemort. I especially like the "Isolated Magical Britain from the rest of the wizarding world" one--I didn't even think of it, but it fits. He didn't just get rid of Hermione, he goaded his enemies into committing an atrocity against her.
I'm not sure it's appropriate to consider the money the average human will accept for a micromort as a value that's actually useful for making rational decisions, because that's a value that's badly skewed by irrational biases. Actions are mentally categorized into those the thinker does and doesn't believe (on a subconscious level) to possibly lead to death. I doubt the average person even considers a "risk" factor at all when driving their car or walking several blocks to the car (just a time factor and a gasoline factor), unless their trip takes them through a "bad" neighborhood, in which case they'll inflate their perceived risk severalfold without actually looking up that neighborhood's crime rates (moreso if they know someone who was hurt in a manner similar to that). They're probably quite likely to consider a "getting a ticket" risk factor, however. It's sadly true that most people believe themselves invincible and completely ignore many categories of existential risk, thinking only of the "flashier" risks and likely inflating their likelihood. And if you told someone that you would give them $100 and then use a fair RNG and shoot them either on a 1 in 10,000 or 1 in 100,000 chance, I doubt you'd get very different responses.
And I'm going to be so bold as to declare that it's impossible for ANY individual to accurately judge the relative likelihood of two things to kill you without looking it up; "which is more likely" is doable but "is it twice as likely or three times" is not.
edit: The end result of everything I just said is that the "value" being assigned to a micromort is probably more a reflection of how the EPA ran their test than what people really value; they'd get a different result evaluating people's aversion to micromorts via car crash and people's aversion to micromorts via being mugged, and either would be skewed if they first spent a half hour talking about ways to mitigate such a risk (thus reminding you it's there).
Sadly, your commitment to this goal is not enough, unless you also have a guarantee that someone will publish your results even if they are statistically insignificant (and thus tell us absolutely nothing). I admit I've never tried to publish something, but I doubt that many journals would actually do that. If they did the result would be a journal rendered almost unreadable by the large percentage of studies it describes with no significant outcome, and would remain unread.
If your study doesn't prove either hypothesis, or possibly even if it proves the null hypothesis and that's not deemed to be very enlightening, I expect you'll try and fail to get it published. If you prove the alternative hypothesis, you'll probably stand a fair chance at publication. Publication bias is a result of the whole system, not just the researchers' greed.
The only way I can imagine a study that admits that it didn't prove anything could get publication is if it was conducted by an individual or group too important to ignore even when they're not proving anything. Or if there's so few studies to choose from that they can't pick and choose the important ones, although fields like that would probably just publish fewer journals less frequently.
I like the first two, and the chess one's pretty interesting though I can't imagine I'd have an easy time getting someone to stand still long enough to hear the whole thing as an argument. But I don't really like the last one. You've been tricked into accepting his premise, that death lets you create more meaningful art, and trying to regain ground from there. It's that premise itself that you should be arguing against--point out all the great literature and art that isn't about death, and that you could still have all of that once death was gone. Also point out that to someone with cancer today the availability of art is probably less valuable than the availability of a cure would be, and there's no reason to assume that'll change if you double his age, even if you double it several times.
Oh, I understand now. Even if we don't know how it's distributed, if it's the top among 9 choices with the same variance that puts it in the 80th percentile for specialness, and signal and noise contribute to that equally. So it's likely to be in the 80th percentile of noise.
It might have been clearer if you'd instead made the boxes actually contain coins normally distributed about 40 with variance 15 and B=30, and made an alternative of 50/1, since you'd have been holding yourself to more proper unbiased generation of the numbers and still, in all likelihood, come up with a highest-labeled box that contained less than the sure thing. You have to basically divide your distance from the norm by the ratio of specialness you expect to get from signal and noise. The "all 45" thing just makes it feel like a trick.
I'm trying to figure out why, from the rules you gave at the start, we can assume that box 60 has more noise than the other boxes with variance of 20. You didn't, at the outset of the problem, say anything about what the values in the boxes actually were. I would not, taking this experiment, have been surprised to see a box labeled "200", with a variance of 20, because the rules didn't say anything about values being close to 50, just close to A. Well, I would've been surprised with you as a test-giver, but it wouldn't have violated what I understood the rules to be and I wouldn't have any reason to doubt that box was the right choice.
The box with 60 stands out among the boxes with high variance, but you did not say that those boxes were generated with the same algorithm and thus have the same actual value. In fact you implied the opposite. You just told me that 60 was an estimate of its expected value, and 37 was an estimate of one of the other boxes' expected values. So I would assign a very high probability to it being worth more than the box labeled 37. I understand that the variance is being effectively applied twice to go between the number on the box to the real number of coins (The real number of 45 could make an estimate anywhere from 25 to 65, but if it hit 25 I'd be assigning the real number a lower bound of 5 and if it hit 65 I'd be assigning the real number an upper bound of 85, which is twice that range). (Actually for that reason I'm not sure your algorithm really means there's a variance of 20 from what you state the expected value to be, but I don't feel like doing all the math to verify that since it's tangential to the message I'm hearing from you or what I'm saying). But that doesn't change the average. The range of values that my box labeled 60 could really contain from being higher than the range the box labeled 37 could really contain, to the best of my knowledge, and both are most likely to fall within a couple coins of the center of that range, with the highest probability concentrated on the exact number.
If the boxes really did contain different numbers of coins, or we just didn't have reason to assume that they don't contain different numbers, the box labeled 60 is likely to contain more coins than that 50/1 box did. It is also capable of undershooting 50 by ten times as much if unlucky, so if for some reason I absolutely cannot afford to find less than 50 coins in my box the 50/1 box is the safer choice--but if I bet on the 60/20 box 100 times and you bet on the 50/1 box 100 times, given the rules you set out in the beginning, I would walk away with 20% more money.
Or am I missing some key factor here? Did I misinterpret the lesson?
The problem is, Dumbledore's not going to tell Harry what the condition is for getting the stone. Why would he? He didn't tell canon Quirrell, who was standing there trying to figure out why he couldn't get it. He didn't even tell canon Harry until after the fact. The mirror as a screening process works even better if the person being screened doesn't know what it's testing for, and thus can't fake it.
And Harry would want to use the stone, make no mistake. The first thing he'd do with it is make himself immortal, to make sure no accident or fluke could stop him from having time to mass produce the immortality elixir. And he'd be using it for study anyways. But the most important part is that even if he is capable of precommitting and one-boxing, and even if that kind of trick fools the mirror, he'd first need to know that that was the condition necessary to obtain the stone. And you can probably count the number of people Dumbledore trusts with that information on one hand.
I think you hit on a key point that several are missing--Dumbledore wouldn't want HJEPV to have the stone any more than Quirrell (well, maybe a little more, but certainly less than nobody having it or even than handing it off to, say, some random Hufflepuff). In canon Harry didn't just not want to use it, he didn't want it used--that was his entire motivation for getting it. Rational Harry would, probably quite literally given enough time to think on the situation, kill to use it, and use it repeatedly. And Dumbledore knows this.
Canon Harry was, in fact, a person Dumbledore would be willing to loan the stone to if necessary. Rational Harry is not. The mirror actually represents a pretty effective screening process for who does and doesn't fall in that category, especially combined with what in theory should have been a screening test to ensure you were a capable enough wizard to protect it and/or had the approval of several people he trusted in a more general capacity. In fact now that I say that, it suddenly seems plausible that the mirror wasn't in any way tied to how it was hidden, and instead was just the trigger used for retrieving it. In which case, actually, a sufficiently powerful wizard with sufficient time could probably deconstruct the spell and take it by force, simply because no lock is perfect, which is why it still needed to be guarded in the first place and why stopping Quirrell was necessary.
Despite having seen you say it in the past, it wasn't until reading this article that in sunk in for me just how little danger we were actually in of Eliezer1997 (or even Eliezer2000) actually making his AI. He had such a poor understanding of the problem, I don't see how he could've gotten there from here without having to answer the question of "Ok, now what do I tell the AI to do?" The danger was in us almost never getting Eliezer2008, or in Eliezer2000 wasting a whole bunch of future-minded peoples' money getting to the point where he realized he was stuck.
Except I suppose he did waste a lot of other people's money and delay present-you by several years. So I guess that danger wasn't entirely dodged after all. And maybe you did have something you planned to tell the AI to do anyways, something simple and useful sounding in and of itself with a tangible result. Probably something it could do "before" solving the question of what morality is, as a warmup. That's what the later articles in this series suggest, at least.
I also peeked at the Creating Friendly AI article just to see it. That, unlike this, looks like the work of somebody who is very, very ready to turn the universe into paperclips. There was an entire chapter about why the AI probably won't ever learn to "retaliate", as if that was one of the most likely ways for it to go wrong. I couldn't even stand to read more than half a chapter and I'm not you.
"To the extent that they were coherent ideas at all" you've said of half-baked AI ideas in other articles. It's nice to finally understand what that means.
I'm kind of surprised at how complicated everyone is making this, because to me the Bayesian answer jumped out as soon as I finished reading your definition of the problem, even before the first "argument" between one and two boxers. And it's about five sentences long:
Don't choose an amount of money. Choose an expected amount of money--the dollar value multiplied by its probability. One-box gets you >(1,000,000*.99). Two-box gets you <(1,000*1+1,000,000*.01). One-box has superior expected returns. Probability theory doesn't usually encounter situations in which your decision can affect the prior probabilities, but it's no mystery what to do when that situation arises--the same thing as always, maximize that utility function.
Of course, while I can be proud of myself for spotting that right away, I can't be too proud because I know I was helped a lot by the fact that my mind was in a "thinking about Eliezer Yudkowsky" mode already, a mode it's not necessarily in by default and might not be when I am presented with a dilemma (unless I make a conscious effort to put it there, which I guess now I stand a better chance of doing). I was expecting for a Bayesian solution to the problem and spotted it even though it wasn't even the point of the example. I've seen this problem before, after all, without the context of being brought up by you, and I certainly didn't come up with that solution at the time.
I think you may be placing too much emphasis on curiosity as a terminal value here rather than a means of acquiring other terminal values--not that I think it has no value in and of itself, but that's not its only use and not its biggest in most respects.
If I know that a light switch/bulb's properties are fully explained by science and nothing else about it, that DOES tell me things I didn't know beforehand. It tells me that it is much less of a priority to figure out how the light bulb works than it would have been if nobody had a clue. If there is any situation in which knowledge of how it works is necessary there's already someone who knows, and if there is a situation in the future in which I decide it would be to my benefit to learn how it works I will have little trouble learning it at that time, rather than needing to arm myself with that knowledge well in advance because I can't predict how long it will take to acquire (this ignoring the issue of how long it would take for me to wrap my head around the concepts if I tried; it's easier to learn if it's already known, no matter where the baseline difficulty is set). And if there is some incredibly useful piece of low hanging fruit which could be derived from just knowledge about the light bulb, I could be confident that somebody else had found it (and if it's not low hanging fruit, well, others have at least had as much of a shot at it as I would; unless I have reason to believe that there is something I could learn from the knowledge that others would have missed so far, in which case skip to the next paragraph). Even knowing that the keyword involved is "electricity", which does not even begin to count as understanding, tells me SOMETHING (though not enough that a science teacher should feel justified stopping there)--it tells me which section in the phone book to look at if I need someone to fix my lightswitch, or which section of the library to look in to learn about it myself. The former is all some people will need to know about lightswitches for their entire lives.
Of course I know a good deal more about it than that, and in fact some of my knowledge will be useful to me in a real way in my real life. But the knowledge about what organ X does and where it is, is not directly useful to me in the same way--for all practical purposes it's good enough that I can ask a doctor or search certain trustworthy portions of the internet for the answer to any question about what I should do from a medical perspective in most situations. Yes, knowledge of medicine being more immediately on hand might help me react faster to a problem and a situation may come up where I personally ought to know CPR and don't, but my situation isn't as bad as if nobody knew this information. And I can teach myself most of the useful things that you can/must do immediately and with no equipment or pills, without ever needing to know how the inside of my body really looks.
This only applies to things unrelated to the fields you do care about. If you care about or expect to be involved in making AI, knowledge of all things computer and how electricity works and how modern computer chips are built and anything you can learn about existing intelligences is valuable. Knowledge of what your liver does, however, isn't just somebody else's problem; it's not a problem. If you feel curious about the liver that's great, but if not you don't really have to.
The example with the elephant isn't great, because that is a situation in which I would care about the knowledge, and would seek to either ask the person who knows about it to explain to me or, if the explanation turns out to be far more difficult and/or long than I am willing to accept, will ask him all the predictive questions I would've been able to answer if I knew about it, such as "Is this going to get worse if I ignore it until a more convenient time", "How do I make it leave", "Is my house in danger" or even the almighty "Is there anything else important I should know about this" (because it is a human being who knows about it, not a genie, and he can predict what facts I'd consider important even if I don't know what to ask for). Or if even that was too hard (or he was unwilling to tell me), at least to remove it from the room for me.
The example of you waving your hands to create light and calling it science would also not stop me until I became convinced that significantly more people than just you knew it and that some of these people would be willing to tell me if I asked. But that's because my curiosity about the light as a whole would stem as much from mistrust of another person to apply that knowledge to humanity's general benefit as from real curiosity. You can't hold out on me with the knowledge that that light can also cure cancer if I also understand what you do about the light, or if I know that a number of other people unlikely to share a stake with you in any enterprise know it.
The real danger would be if that elephant had been in my room since the day I was born and I didn't know enough to be curious about it.