Comment by Cyrus on Free to Optimize · 2009-01-02T14:39:29.000Z · LW · GW

The future is still strongly counterfactually dependent on your actions: if you pursue wealth yourself, the AI will give you a pittance, and you go on to earn riches. If you choose to do nothing, the AI gives you a fortune, and you go on in idleness.

If your preference function trivializes the method by which you became wealthy, I have difficulty believing that it cares so acutely about the method by which the AI chose to give you some amount of money.