↑ comment by magfrump ·
2010-12-30T02:56:49.025Z · LW(p) · GW(p)
I'm not really sure what's being calculated in that last paragraph there. Knowing the measurement of a single util seems to be valuable OUTSIDE of this problem. Inside the problem, the optimal actions (which is to say the actions with highest expected value) continues to be writing the busy beaver function as fast as possible, &c.
Also, if Omega balances out utils with positive and negative utils, why is e more likely to torture you for five seconds and tell you "this is -9 utils" than, say, torture you for 300 years then grant you an additional 300 years of life in which you have a safe nanofactory and an Iron Man suit?
It seems to me that the vast majority of actions Omega could take would be completely inscrutable, and give us very little knowledge about the actual value of utils.
A better example might be the case in which waiting for one second at a traffic light is worth one util, and after your encounter Omega disappears without a word. Omega then begins circulating a picture of a kitten on the internet. Three years later, a friend of yours links you the picture just before you leave for work. Having to tell them to stop sending you adorable pictures when you're about to leave is the same value as seeing the adorable picture, and the one second later that you get out the door is a second you do not have to spend waiting at a traffic light.
If this is how utils work then I begin to understand why we have to break out the busy beaver function... in order to get an outcome that is akin to $1000 out of this game, you would need to win around 2^20 utils (by my rough and highly subjective estimate). A 5% chance of $1000 is MUCH MUCH better than a guarantee of waiting one second less of waiting at a traffic light.
I seem to have digressed.