Posts
Comments
Comment by
WolfgangSchwarz on
A Reaction to Wolfgang Schwarz's "On Functional Decision Theory" ·
2022-01-06T10:31:36.744Z ·
LW ·
GW
Thanks. A few points, mostly for clarification.
- I'm not assuming that the relevant predictors in my scenarios are infallible. In the Blackmail scenario, for example, I'm assuming that the blackmailer is fairly good but not perfect at predicting your reaction. So it's perfectly possible for an FDT agent to find themselves in that scenario. If they do, they will clearly do worse than a CDT agent.
- You're right that I shouldn't have called FDT's recommendation in the Twin case "insane". I do think FDT's recommendation is insane for the other cases I discuss, but the Twin case is tricky. It's a Newcomb Problem. I'd still say that FDT gives the wrong advise here, and CDT gives the right advice. I'm a two-boxer.
- Of course making agents care about others (and about their integrity etc.) changes the utility function and therefore the decision problem. That's exactly the point. The idea is that in many realistic scenarios such agents will tend to do better for themselves than purely egoistical agents. So if I were to build an agent with the goal that they do well for themselves, I'd give them this kind of utility function, rather than implement FDT.
- "What if someone decides to punish agents for using CDT?" -- Sure, this can happen. It's what happens in Newcomb's Problem.
- "Schwarz goes on to list a number of points of questions he has/unclarities he found in Yudkowsky and Soares' paper, which I don't find relevant" -- Their relevance is that FDT isn't actually a theory, unlike CDT and EDT. In its present form it is only an underdeveloped sketch, and I have doubts that it can be spelled out properly.
- You say that CDT "fails" the original problems. You don't give any argument for this. My intuition is that FDT gets all the problems I discuss wrong and CDT gets them right. For what it's worth, I'd bet that most people's intuitions about cases like Blackmail, Procreation, and Newcomb's Problem with Transparent Boxes are on my side. Of course intuitions can be wrong. But as a general rule, you need better arguments in support of a counter-intuitive hypothesis than in support of an intuitive hypothesis. I'm not aware of any good arguments in support of the FDT verdict.