Epistemic vs. Instrumental Rationality: Case of the Leaky Agent

In this situation, I would say that whatever you currently believe is the minimum, is the actual minimum. If you manage to convince yourself that a higher minimum is required, you have simply updated the variable.

Not directly related, but if you are expected to tell this specific systematic lie, the lie immediately becomes ineffective.

Well, in practice it seems that the answer is to fix the leak- especially since I don't know how to make myself believe something that I know to be false.

However, in the most inconvenient world, my self honesty has a finite price. I care about self honesty and I care about money. I don't feel like computing the price given that I'm not actually facing this condition.

These situations where "false beliefs win" come up often in hypotheticals because it forces you to chose between two things (truth and success) which correlate well enough to be confused with each other. Once you recognize this and have a good enough idea of the balance that you aren't chasing the wrong thing, I don't think there's anything to gain.

Does anyone have an example where common strategies utilizing false beliefs predictably do better than common strategies utilizing true beliefs? Are there any real world examples where the best thing you can do is still to believe a lie?

Perhaps this explains why people so often have distorted self-perceptions. For example, in the dating game, an inflated self perception amounts to a higher 'minimum.' Perhaps most distorted self-perceptions have evolutionary origins that are based on this mechanism?

Fixing the leak is the best solution. Much easier and more realistic with a team of one (yourself) than it is with more than one.

While we can all probably give examples from personal experience equivalent to the real estate agent leaking the minimum price, my personal favorite resulted in the current border of California. Ever wonder why the West Coast border between the U.S. and Mexico is between San Diego and Tijuana? At the end of the war that resulted in this border, there were American troops occupying Mexico City. Why draw the line there? Turns out that the U.S. negotiators had a list of must-gets and nice-to-haves. Alta California and Colorado country were must-gets. Baja was viewed as a desert waste and a nice-to-have. The list got leaked to the Mexican delegation by an American personally acquainted with one of the Mexican negotiators, so the Mexican team dug in their heels and wouldn't give up Baja. Point: keep negotiation teams small. Fewer leak-points. One (yourself) is best, because only then do negotiator interests exactly correspond with YOUR interests.

In nearly all situations, I suspect, lying to yourself only appears profitable because the situations has been misinterpreted. It would help to have more contextual examples where lying is plausibly profitable.

The realtor example is too ambiguous. At best, lying gives me a greater probability of capturing more of the surplus from trade, but at the risk of no sale at all (e.g. the best offer falls between my real and stated minimum). Moreover, the scenario construes sale as buyer vs. seller, but my best hope to get a high price is with competition among buyers rather than negotiating in a zero-sum game with one buyer. Here, a truthful realtor might help, because buyers would believe him when he says someone else has made a better offer.

So the question here is something like, "Would you try to make yourself believe a lie, in the case that believing the lie would cause you to maximize your utility function in a way that believing the truth would not?"

Is this more or less a correct reading?

If I were constructing an agent that was unavoidably leaky, (All potential buyers are omegas,) then I would construct it such that it could not accept an offer below my 'minimum', even if it knows my 'true minimum'. If the buyers know this, then they have to deal with the agent's minimum. Where the agent is is my future self, it would be possible to enter a contract with a third party making selling below my 'minimum' prohibitive.

It seems like this kind of a strategy might have led to the way humans play the ultimatum game.

I don’t think that a “lying case” analysis is correct. In both cases, lying or not, you are using all information you have to correctly estimate the “minimum acceptable price". In “lying case”, if you are rational, you know that you (probably) have given increased price so you decrease it back by your best estimation of increment. Now, the higher probability you lie for yourself, the less precise estimation you are left with. And, by being rational, you have to overestimate as likely as underestimate. So this strategy on gives you no consistent benefit.

In answer to the modified puzzle: If the expected utility of the lie is positive, then yes I would lie to myself, if I were capable of doing so effectively.

There are complications in figuring out the expected utility of the lie (lying to myself may alter my utility), but other than that it's just a standard form of commitment device. I'd probably search for more effective commitment technology first, but if there's no better commitment technology available, do it.

Now here's the kicker: that agent is actually your future self. Would you tell yourself a lie, if you could believe it (perhaps with the help of future memory modification technologies), and if you could profit from it?

It's not a lie, but yes, the figure I told the agent(/self) would be higher than my actual minimum in those circumstances. In fact, I regularly put this very concept into practice in my own life. I give myself certain higher standards for things and mean them. Yet I know that sometimes my future self will end up taking liberties with them in the moment if it is really necessary given the situation. He never was much for following rules anyway.

Wei Dai points out on OB that there exists causal agents that could benefit from hiding predictable actions and reactions we might make from our very selves on account of how leaky we are. I'd vote the OB comment up if I could.

I often find that there's not any satisfactory way to calibrate my expectations for things like this anyway. I was once emailed by someone who wanted to buy a domain name from me. He refused to give an offer, asking me to provide a price. I found it impossible to gauge what it was actually worth to me, or what I thought it would be worth to him, so I said I wouldn't sell it unless he made an offer. I never heard from him again.

So, sure. My future self can be convinced of a new minimum, for all I care. I apparently hold my ideas about this very lightly anyway. I'm not even sure he'd (I'd) be "wrong," even if I currently think of it as a lie.