Are ya winning, son?

There is definitely not an "objective" way to define "winning" in general, at least not yet. It's more of an intuition pump that can help make it clearer in each example what the right thing to do is (so that we can judge whether a decision theory recommends doing the right thing). To add some context, prior to Eliezer's innovation of asking what the "winning" choice is, philosophers used to ask what the "rational" choice is, which led to some to say that the "rational" choice in Newcomb's problem is to two-box and to propose decision theories like CDT that two-box. Talking about "winning" makes it clearer or more intuitive that one-box is the right thing to do. But it's not a panacea and often it's not entirely obvious what the "winning" choice is, or different people can disagree about it.

So hypothetically, there could be an undiscovered Decision Theory in which Prisoner A is tricked into cooperating, only for B to betray him, resulting in B achieving the optimal outcome of C,D. Wouldn’t such a result objectively see B “winning”, because he got a higher payoff than C,C?

Yeah, I would say an agent or decision theory that can trick other agents into the C,D outcome would be "more winning". From my perspective, we're not so much "striving to generate algorithms that will guarantee a C,C result" but instead aiming for a decision theory that exploits everyone who can be exploited and achieves C,C with everyone else (to the extent possible).

Yes, defining what "winning" means is indeed part of the problem of evaluating decision theories, and is largely the difference between many decision theories.

One way of defining "winning" is getting the most you can (on expectation) given the information you have at a given instant of time. In the Counterfactual Mugging situation where you are asked for $100, winning obviously means not paying by this definition. In the other situation you have no choices to make and so automatically "win" because in that local situation there is no better action you can take.

Another way of defining "winning" is getting the most you can (on expectation) across all scenarios weighted by their probabilities. In Counterfactual Mugging, this means paying up because the $100 loss in half the scenarios by probability is greatly outweighed by the $1000 gain in the other half.

Note that the scenario is always presented as "Omega asks you to pay up". While this is the only scenario in which you get to make a decision, it also biases perception of the problem by directing attention away from the equally prevalent scenarios in which Omega just turns up and (if you would have paid) gives you $1000 or (otherwise) tells you that you get nothing.

The smoking lesion problem depends entirely on why people with the smoking lesion smoke more often.

If the mechanism does not affect one's ability to do reasoning, come to logical conclusions, etc., then even though people who smoke usually are more likely to have the smoking lesion, people who smoke because they followed the reasoning telling them to smoke are not more likely to have the smoking lesion and there is no problem.

If the mechanism does affect one's ability to do reasoning, then it may not even be possible for someone to logically decide what to do. (Consider an edge case where the smoking lesion makes you 100% likely to smoke, but logical reasoning tells you not to.) Or it may only be possible to do so in a subset of cases.

(Of course, "people who did X because of Y" is a simplification and should be something like "the difference in propensities for Y between people who do X and people who don't.")

Winning here corresponds to getting the most expected utility, as measured from the start of the problem. We assume we can measure utility with money, so we can e.g. put a dollar value on getting rescued. 

1) In the smoking lesion problem, the exact numbers don't so much matter as the relationship between them: you are simply choosing whether or not to gain an amount of utility or to pass it up.   

2) Defecting on a cooperator in a one-shot true prisoners dilemma is the "best" outcome, so this is exactly right. See this story [LW · GW].  

3) In the hitchhiker example, we might say that we have a negative $1,000,000 value associated with being stranded. This exact amount is usually not specified, the assumption being that this is a large negative amount. 

Thus the outcomes are: 

• We don't pay and are stranded in the desert, -$1,000,000
• We don't pay and the driver took us back to the city, $0 
• We would have paid and are stranded in the desert, -$1,000,000
• We pay and the driver took us back to the city, -$100

Of course, the middle two bullet points are logically counter to the problem (assuming an open source setting), so the winning decision theories here lose only $100 as opposed to losing $1,000,000 (dying from being stranded in the desert). 

Edit: changed utility -> expected utility.