Posts
Comments
Your original description doesn't specify subjunctive dependence, which is a critical component of the problem.
Heighn’s response to this argument is that this is a perfectly fine prescription.
Note that omnizoid hasn't checked with me whether this is my response, and if he had, I would have asked him to specify the problem more. In my response article, I attempt to specify the problem more, and with that particular specification, I do indeed endorse FDT's decision.
I'm surprised Wei Dai thinks this is a fair point. I disagree entirely with it: FDT is a decision theory and doesn't in and of itself value anything. The values need to be given by a utility function.
Consider the Psychological Twin Prisoner's Dilemma. Given the utility function used there, the agent doesn't value the twin at all: the agent just wants to go home free as soon as possible. FDT doesn't change this: it just recognizes that the twin makes the same decision the agent does, which has bearing on the prison time the agent gets.
...which makes the Procreation case an unfair problem. It punishes FDT'ers specifically for following FDT. If we're going to punish decision theories for their identity, no decision theory is safe. It's pretty wild to me that @WolfgangSchwarz either didn't notice this or doesn't think it's a problem.
A more fair version of Procreation would be what I have called Procreation*, where your father follows the same decision theory as you (be it FDT, CDT or whatever).
Oh wait, of course, in this problem, Omega doesn't simulate the agent. Interesting! I'll have to think about this more :-)
In what ways is FDT broken?
I also wonder whether a different problem was intended.
Thanks for the link!
And hmm, it seems to me FDT one-boxes on ASP, as that gives the most utility from the subjunctive dependence perspective.
Why would Omega put $0 in the second box? The problem statement specifies Omega puts $100 in both boxes if she predicts you will two-box!
If I have a two box policy the simulated me gets $200 before deletion, and the real me gets nothing.
Wait, why does the real you get nothing? It's specified you get $200. What am I missing?
Exploitable? Please explain!
Ah, I just read your substack post on this, and you've referenced two pieces I've already reacted to (and in my view debunked) before. Seems like we could have a good debate on this :)
I would love to debate you on this. My view: there is no single known problem in which FDT makes an incorrect decision. I have thought about FDT a lot and it seems quite obviously correct to me.
Ah, so your complaint is that the author is ignoring evidence pointing to shorter timelines. I understand your position better now :)
"Insofar as your distribution has a faraway median, that means you have close to certainty that it isn't happening soon. And that, I submit, is ridiculously overconfident and epistemically unhumble."
Why? You can say a similar thing about any median anyone ever has. Why is this median in particular overconfident?
"And not only do I not expect the trained agents to not maximize the original “outer” reward signal"
Nitpick: one "not" too many?
I apologize, Said; I misinterpreted your (clearly written) comment.
Reading your newest comment, it seems I actually largely agree with you - the disagreement lies in whether farm animals have sentience.
(No edit was made to the original question.)
Thanks for your answer!
I (strongly) disagree that sentience is uniquely human. It seems to me a priori very unlikely that this would be the case, and evidence does exist to the contrary. I do agree sentience is an important factor (though I'm unsure it's the only one).
"but certainly none of the things that we (legally) do with animals are bad for any of the important reasons why torture of people is bad."
That seems very overconfident to me. What are your reasons for believing this, if I may ask? What quality or qualities do humans have that animals lack that makes you certain of this?
One-boxing on Newcomb's Problem is good news IMO. Why do you believe it's bad?
Of course! Thanks for your time.
I can, although I indeed don't think it is nonsense.
What do you think our (or specifically my) viewpoint is?
Hmm, interesting. I don't know much about UDT. From and FDT perspective, I'd say that if you're in the situation with the bomb, your decision procedure already Right-boxed and therefore you're Right-boxing again, as logical necessity. (Making the problem very interesting.)
Sorry, I'm having trouble understanding your point here. I understand your analogy (I was a developer), but am not sure what you're drawing the analogy to.
I've been you ten years ago.
Just... no. Don't act like you know me, because you don't. I appreciate you trying to help, but this isn't the way.
Seems to me Yudkowsky was (way) too pessimistic about OpenAI there. They probably knew something like this would happen.
To explain my view more, the question I try to answer in these problems is more or less: if I were to choose a decision theory now to strictly adhere to, knowing I might run into the Bomb problem, which decision theory would I choose?
"But by the time the situation described in the OP happens, it no longer matters whether you optimize expected utility over the whole sample space; that goal is now moot."
This is what we agree on. If you're in the situation with a bomb, all that matters is the bomb.
My stance is that Left-boxers virtually never get into the situation to begin with, because of the prediction Omega makes. So with probability close to 1, they never see a bomb.
Your stance (if I understand correctly) is that the problem statement says there is a bomb, so, that's what's true with probability 1 (or almost 1).
And so I believe that's where our disagreement lies. I think Newcomblike problems are often "trick questions" that can be resolved in two ways, one leaning more towards your interpretation.
In spirit of Vladimir's points, if I annoyed you, I do apologize. I can get quite intense in such discussions.
I see your point, although I have entertained Said's view as well. But yes, I could have done better. I tend to get like this when my argumentation is being called crazy, and I should have done better.
You could have just told me this instead of complaining about me to Said though.
I don't see how it is misleading. Achmiz asked what actually happens; it is, in virtually all possible worlds, that you live for free.
Note that it's my argumentation that's being called crazy, which is a large factor in the "antagonism" you seem to observe - a word choice I don't agree with, btw.
About the "needlessly upping the heat", I've tried this discussion from multiple different angles, seeing if we can come to a resolution. So far, no, alas, but not for lack of trying. I will admit some of my reactions were short and a bit provocative, but I don't appreciate nor agree with your accusations. I have been honest in my reactions.
Interesting. I'm having the opposite experience (due to timing, apparently), where at least it's making some sense now. I've seen it using tricks only applicable to addition and pulling numbers out of its ass, so I was surprised what it did wasn't completely wrong.
Asking the same question again even gives a completely different (but again wrong) result:
If you ask ChatGPT to multiply two 4-digit numbers it writes out the reasoning process in natural knowledge and comes to the right answer.
People keep saying such things. Am I missing something? I asked it to calculate 1024 * 2047, and the answer isn't even close. (Though to my surprise, the first 2 steps are at least correct steps, and not nonsense. And it is actually adding the right numbers together in step 3, again, to my surprise. I've seen it perform much, much worse.)
That's what I've been saying to you: a contradiction.
And there are two ways to resolve it.
The scenario also stipulates the bomb isn't there if you Left-box.
What actually happens? Not much. You live. For free.
"So if you take the Left box, what actually, physically happens?"
You live. For free. Because the bomb was never there to begin with.
Yes, the situation does say the bomb is there. But it also says the bomb isn't there if you Left-box.
Agreed, but I think it's important to stress that it's not like you see a bomb, Left-box, and then see it disappear or something. It's just that Left-boxing means the predictor already predicted that, and the bomb was never there to begin with.
Put differently, you can only Left-box in a world where the predictor predicted you would.
I think we agree. My stance: if you Left-box, that just means the predictor predicted that with probability close to 1. From there on, there are a trillion trillion - 1 possible worlds where you live for free, and 1 where you die.
I'm not saying "You die, but that's fine, because there are possible worlds where you live". I'm saying that "you die" is a possible world, and there are way more possible worlds where you live.
I'm not going to make you cite anything. I know what you mean. I said Right-boxing is a consequence, given a certain resolution of the problem; I always maintained Left-boxing is the correct decision. Apparently I didn't explain myself well, that's on me. But I'm kinda done, I can't seem to get my point across (not saying it's your fault btw).
By construction it is not, because the scenario is precisely that we find ourselves in one such exceptional case; the posterior probability (having observed that we do so find ourselves) is thus ~1.
Except that we don't find ourselves there if we Left-box. But we seem to be going around in a circle.
… but you have said, in a previous post, that if you find yourself in this scenario, you Right-box. How to reconcile your apparently contradictory statements…?
Right-boxing is the necessary consequence if we assume the predictor's Right-box prediction is fixed now. So GIVEN the Right-box prediction, I apparently Right-box.
My entire point is that the prediction is NOT a given. I Left-box, and thus change the prediction to Left-box.
I have made no contradictory statements. I am and always have been saying that Left-boxing is the correct decision to resolve this dilemma.
No, that's just plain wrong. If you Left-box given a perfect predictor, the predictor didn't put a bomb in Left. That's a given. If the predictor did put a bomb in Left and you Left-box, then the predictor isn't perfect.
Excellent point!
Firstly, there’s a difference between “never” and “extremely rarely”.
That difference is so small as to be neglected.
And in the latter case, the question remains “and what do you do then?”. To which, it seems, you answer “choose the Right box”…? Well, I agree with that! But that’s just the view that I’ve already described as “Left-box unless there’s a bomb in Left, in which case Right-box”.
It seems to me that strategy leaves you manipulatable by the predictor, who can then just always predict you will Right-box, put a bomb in Left, and let you Right-box, causing you to lose $1,000.
"Irrelevant, since the described scenario explicitly stipulates that you find yourself in precisely that situation."
Actually, this whole problem is irrelevant to me, a Left-boxer: Left-boxers never (or extremely rarely) find themselves in the situation with a bomb in Left. That's the point.
The bottom line is: to the actual single question the scenario asks—which box do you choose, finding yourself in the given situation?—we give the same answer. Yes?
The bottom line is that Bomb is a decision problem. If I am still free to make a decision (which I suppose I am, otherwise it isn't much of a problem), then the decision I make is made at 2 points in time. And then, Left-boxing is the better decision.
Yes, the Bomb is what we're given. But with the very reasonable assumption of subjunctive dependence, it specifies what I am saying...
We agree that if I would be there, I would Right-box, but also everybody would then Right-box, as a logical necessity (well, 1 in a trillion trillion error rate, sure). It has nothing to do with correct or incorrect decisions, viewed like that: the decision is already hard coded into the problem statement, because of the subjunctive dependence.
"But you can just Left-box" doesn't work: that's like expecting one calculator to answer to 2 + 2 differently than another calculator.
Alright. The correct decision is Left-boxing, because that means the predictor's model Left-boxed (and so do I), letting me live for free. Because, at the point where the predictor models me, the Bomb isn't placed yet (and never will be).
However, IF I'm in the Bomb scenario, then the predictor's model already Right-boxed. Then, because of subjunctive dependence, it's apparently not possible for me to Left-box, just as it is impossible for two calculators to give a different result to 2 + 2.
Hmmm, I thought that comment might clear things up, but apparently it doesn't. And I'm left wondering if you even read it.
Anyway, Left-boxing is the correct decision. But since you didn't really engage with my points, I'll be leaving now.