A Dissent on Honesty

post by eva_ · 2025-04-15T02:43:44.163Z · LW · GW · 20 comments

Contents

  Context
    Reminder About Winning
    Technical Truth is as Bad as Lying
    Being Mistaken is Also as Bad as Lying
    This is Partially a Response
    Examples
  Biting the Bullet
    My Proposed Policy
    Appearing Trustworthy Anyway
    Cooperative Epistemics
  Conclusion
None
21 comments

Context

Disney's Tangled (2010) is a great movie. Spoilers if you haven't seen it.

The heroine, having been kidnapped at birth and raised in a tower, has never stepped foot outside. It follows, naturally, that she does not own a pair of shoes, and she is barefoot for the entire adventure. The movie contains multiple shots that focus at length on her toes. Things like that can have an outsized influence on a young mind, but that's Disney for you.

Anyway.

The male romantic lead goes by the name of "Flynn Rider." He is a dashingly handsome, swashbuckling rogue who was carefully crafted to be maximally appealing to women. He is the ideal male role model. If you want women to fall in love with you, it should be clear that the optimal strategy is to pretend to be Flynn Rider. Shortly into the movie is the twist: Flynn Rider (real name Eugene Fitzherbert) is also just pretending to be Flynn Rider. He was once a nerdy, loner orphan with no friends (I hope this is cutting close to your heart) till he read a series of books called "The Tales of Flynnigan Rider" and decided to steal the protagonist's personality wholesale to use as his own. Presumably what followed was an unbroken series of victories and successes all the way until he stumbled into the arms of a beautiful, naive teenager with implausible hair, who he seduced on his first try in about a day and a half.

Flynn admits his real name (and parts of his real personality) to his love interest and only his love interest, and only after he's already successfully seduced her. She accepts this, finding his (selective) openness endearing.

The lesson here is likewise clear: If your actual personality isn't good enough, pretend to be Flynn Rider to everyone at all times, with the sole carve-out being people who love you, like your mother or a princess. This works because people who love you will find your openness endearing, whereas everyone else will think you pathetic and use it against you.

Actually, even if your personality is good enough, you should probably still pretend to be Flynn Rider, because his personality is better. It was, after all, carefully designed by a crack team of imagineers. Was yours? Didn't think so.

Reminder About Winning

Once upon a time, two armies went to war. The first army desired honour and glory, to prove their bravery against their foe, to stand their ground whatever their odds, to bring no shame to their ancestors, and to be worthy of great ballads that would be sung across the lands for generations to come. The second army wanted to kill the people in the first army, without dying themselves in the process. It should be of little surprise to you that, since none of their goals contradicted, in the end everybody got what they wanted.
- Sun Tzu maybe, IDK I made it up.

Philosophers get to pick one axiom, one highest value, to declare superior to all others. If you have no highest value you have no way to judge all your other values. If you have two highest values you will not make it three paces from your own front door before they start contradicting with each other.

Rationality can be about Winning [LW · GW], or it can be about The Truth, but it can't be about both. Sooner or later, your The Truth will demand you shoot yourself in the foot, while Winning will offer you a pretty girl with a country-sized dowry. The only price will be presenting various facts about yourself in the most seductive order instead of the most informative one.

If your highest value isn't Winning, you do not get to be surprised when you lose. You do not even get to be disappointed. By revealed preference, you have to have a mad grin across your face, that you were able to hold fast to your highest-value-that-isn't-winning all the way to the bitter end.

And yet, the rationalist movement has some kind of weird fetish for honesty, without much formal proof or empirical evidence that it's a good idea. Why? Did you watch the wrong Disney movies growing up?

Technical Truth is as Bad as Lying

There is a philosophy I want to call "Technical Truthism". Technical Truthists believe that, so long as what they said was technically true, you have no right to be mad at them, including when they tricked you into giving them all your money, cleverly danced around important issues, lied to themselves so they could truthfully report their own delusional beliefs as if they were valuable, laundered their opinions through a series of shell companies to create the illusion of an independent source that they were merely quoting, refused to give a straight answer on the important "have I just been scammed" question, and publically lauded their own commitment to Absolute Honesty while they did it.

The gospel of Technical Truthism includes such sacred wisdom as:

• If the sentence has the word "allegedly" in it, and anyone has ever said it, it's true.
• If the sentence has the word "may" or "might" in it, it's true.
• If the sentence is about your belief state, and you can by any means reach that belief state, it's true.
• If the sentence is about what you've observed, and it is true about a subset of what you've observed, it's true.

I'm not sure which Disney movies get you into this, because every pop culture example I can think of is either the devil or a lawyer who looks like a standin for the devil. I think this philosophy is ridiculous and self-defeating. It defeats the entire point of telling the truth in the first place.

If you are an honest person, and others can by some mechanism know this, then they will believe you when you say things, and this can be used to share valuable information. If you are a liar and everyone knows it, there's nothing you can do to get the village to save you from the wolf, because when you yell wolf they just ignore you.

The purpose of a word is to carve reality at a joint useful for the discussion taking place, and we should pause here to note that the joint in question isn't "emits true statements", it's "emits statements that the other party is better off for listening to". Nobody should care if your statement "A wolf has been observed near our sheep!" is technically true if, when they come running, they find it was a drawing of a wolf and you're laughing at them. That is no better for their interests than an outright lie. The technical truth is useless as a defense, except against courts that are obligated to follow explicit laws and exact wordings. Nobody made out of meat should care.

Being Mistaken is Also as Bad as Lying

Just as we don't have reason to care if they technically saw a wolf, we also don't have much reason to care if they thought they saw a wolf and were merely sincerely mistaken. Sure, malice can be disincentivised with punishments in a way that mere errors are less susceptible to, but when it comes to whether we should listen to them next time, being frequently wrong because they're not very bright is just as bad as being just as wrong for any other reason.

The honest might say "By never lying, I get a reputation for never lying, so people will always believe me". This isn't true. They'd also have to never be mistaken, never be mislead by the lies of another, never misread anothers report, never stumble over their words and say something they didn't mean, never accidentally imply something they didn't mean and be mistaken for saying it outright, etc. Basically they'd have to be omniscient, but they're not omniscient. They're made out of meat too, remember.

Fortunately, you don't need a perfect reputation, just a good enough reputation that other people think it passes their Expected Utility Calculation to act on what you say. If you are an aspiring rationalist, you may well be so far above the median in accuracy of belief that you can get away with far above median dishonesty if you want, and still be an authority figure.

This is Partially a Response

In Meta-Honesty: Firming Up Honesty Around Its Edge-Case [LW · GW]", Eliezer writes, and the community seems to agree [? · GW] in the direction of a general premise that truth-speaking is admirable, and something rationalists should aspire to have more of.

As to whether the honest can better ability to discern lies than the liars, Eliezer writes:

This is probably not true in normal human practice for detecting other people's lies. I'd expect a lot of con artists are better than a lot of honest people at that.

I think this is probably correct. You can tell because Eliezer says so, and it's an admission against interest, so he wouldn't say it if he didn't believe it and he wouldn't believe it (because of self-interest biases) unless it was probably correct, but you might still check over your memories of your own life or try looking for an independent study anyway.

He goes on to write:

I once heard somebody claim that rationalists ought to practice lying, so that they could separate their internal honesty from any fears of needing to say what they believed. That is, if they became good at lying, they'd feel freer to consider geocentrism without worrying what the Church would think about it. I do not in fact think this would be good for the soul, or for a cooperative spirit between people. This is the sort of proposed solution of which I say, "That is a terrible solution and there has to be a better way."

Here (I think) he is straightforwardly wrong, and you can tell because he's only able to defend it by resorting to non-rational frames. Who cares if it is "good for the soul", souls aren't real and we're supposed to be trying to Win here. There does not in fact have to be a better way. Sometimes the best option isn't also the maximally honest one. Tradeoffs exist, and you aren't going to make those tradeoffs at anywhere near an optimal rate if you're refusing to ever think of the possibility for fear of spiritual damage.

Whether it is bad for a "cooperative spirit" I promise I will get back to.

The purpose of this essay is not to disagree with Eliezer's Meta-Honesty as a principle for how to be unusually honest despite the dangers (I think it's mostly correct given its premise), but rather to disagree that being unusually honest is a good idea in the first place.

Examples

It is very easy to come up with moral hypotheticals where You Must Lie or Else, but lets ceremonially do it anyway.

A Paragon of Morality is out travelling, when he is beset by bandits. They demand he hand over his gold or they will kill him and take it from his corpse. This is not a decision-theoretic threat [? · GW] because the bandits value getting his gold more than they disprefer commiting murder, but would otherwise avoid the murder if possible. If he hands over all his gold he will lose all his gold. If he hands over all the gold in his pockets, neglects the extra he has hidden in his sock, and says "I have given you all my gold" in a sufficiently convincing tone of voice, then he will lose less than all his gold.

These isn't Omega we're dealing with here, they're totally trickable by a moderately convincing performance. If he keeps some of the gold he can donate it to Givewell approves charities and save however many QALYs or whatever.

Does he have a moral obligation to lie?

Yeah, obviously. Just do the Expected Value Calculation. Why care about Honour here, they're bloody bandits. I think even the particularly devout agree here.

A Normally Honest Man is applying for a job as a Widget Designer. He has many years of industry experience in Widget Engineering. He has memorised the Widget Manufacturing Process. He's actually kind of obsessed with Widgets. Typically whenever a conversation becomes about Widgets he gushes openly and makes a bad impression with his in-laws. Since that incident he has developed the self control to pretend otherwise, and the rest of his personality is okay.

The interviewer works for a Widget Manufacturing company but seems to only care about Widgets a normal amount. He asks "How interested are you in Widgets?" He has learnt from previous job interviews that, if he answers honestly, the interviewer will think he is any of lying, insane, or too weird to deal with, and not hire him, even though this is not in the best financial interests of the company, were they fully informed.

Should he pretend to like widgets the amount most likely to get him hired, or does he have a moral obligation to keep answering honestly until he runs out of rent money and becomes homeless?

The thing I'm trying to point at here is that Not Lying is not a good decision principle in general. It might still be a valuable social norm e.g. for "Cooperative Epistemics" (It isn't, but I'll get to that later), but you definitely shouldn't think of it as any kind of bright line or guiding light.

These aren't the important examples to me though, the important example is this:

A Self-Improvement and Epistemics Nerd has an online community for Self-Improvement and Epistemics Nerds. Half the people reading it are autists with bad social skills, who weren't at exactly the right age to be saved by Disney's Tangled. They struggle with navigating ordinary social situations and obtaining true beliefs because they're bad at lying, and insufficiently aggressive at anticipating it in others.

Would they be doing anyone a favour in encourage a social norm of truthfulness and the expectation of truthfulness in others, when all those people will inevitably have to leave the computer one day and end up like the subjects of the previous two examples? Would they be making the world a better place?

I don't think they would be doing a favour, and I expect they would be making the world a worse place. In fact, if they pretended it was a good idea anyway, I think they would by lying.

Biting the Bullet

Other people are not all your friends, do not all love you, and have not sworn an unbreakable oath not to use the information you provide them against your interests. Sometimes you are made better off by them having less accurate information.

Saying words is just an action, like any other action. Whether the words are literally true or not is just a fact about the action, like any other fact about an action. It's not the morally important fact. You judge actions by their consequences, whether you expect it to lead to more good or bad. Then you take the action with the best consequences overall.

Far more important for an aspiring rationalist however is the inverse: You should expect other people to do similar, and (sometimes) emit sentences that it does not benefit you to listen to. Even if they're good people, even if they're paragons of virtue who donate to all the right charities. You have an obligation to think things through yourself and carefully check. You cannot sell this obligation to the other party.

How to actually do this will involve some details.

My Proposed Policy

Lie by default whenever you think it passes an Expected Value Calculation to do so, just as for any other action. Include the reputation affects of your statement, the loss of valuable credibility if you are known to be a liar, and the risk of revenge by angry victims if your lies hurt others in your calculation. If your lie would be breaking a valuable and delicate social institution that actually exists and is being stably participated in by your counterparties (you have to check for this, do not just assume one exists), consider the value of that institution also in your decision. If you could go to prison about it, remember that prison is a really incredibly terrible place and that even tiny probabilities of ending up there can quickly dominate Expected Values.

Practice lying until you are good at it, so you can use it when you need to. Practice quickly assessing whether it is a good idea to say something, seperately from whether that something is true. Practice to discern the lies of others, and better track and model reputation and reliability in different circumstances.

Once you have these tools, reassess all your beliefs for whether you really believe them or were just tricking yourself because you felt you needed to believe it to maintain your social relationships in the absence of your new ability to lie. For any such beliefs you find,  secretly abandon it in favour of believing the truth. If necessary, lie to all your friends and pretend you still believe it to maintain your social position (If you wouldn't let yourself do this, you won't be able to really reassess the belief in the first place).

Treat Cooperate-Cooperate dynamics, where they locally exist, to be extremely valuable things that you would not want to cheaply destroy, but do not assume they exist where they do not. Require proof and err towards caution. If you think your friend is mistaken or overly naive, try to help them reach truth if and only if you aren't shooting yourself in the foot even moreso by doing that.

When your friends ask you about how trustworthy you are, make no implications that you are abnormally honest. Tell them truthfully (if it is safe to do so) about all the various bad incentives, broken social systems, and ordinary praxis that compel dishonesty from you and any other person, even among friends, and give them sincere advice about how to navigate these issues. 

Build a mental model of how and when other people are trustworthy based on past behaviour, demographic and selection effects, random mannerisms, and anything else you find that is useful. As you know someone better, you will update away over time from the general distribution to a more accurate subpopulation and eventually a precise internal model of how that individual thinks. If a specific claim greatly affects your choices and would be cheap to check, check anyway, as your Expected Value Calculations direct you.

I know you can't actually do an Expected Value Calculation. I just mean pretend to do one to the best of your ability, make up a number, and then act on that. The universe won't be able to tell a made up number from a real one anyway, nobody else can check your internal thoughts. It's still a better policy than just trusting people.

Appearing Trustworthy Anyway

People often compress reputation into a single scalar, or worse a single boolean, when it really isn't.

If you mostly emit (verifiably) bad sentences that hurt the listener, they will eventually notice and stop listening. If you lie always and obviously about a specific topic they will disbelieve you about that topic but might still believe you about other things. If you lie only in ways where you can't get caught, like about your private preferences or beliefs ("yes, I am feeling fine", "yes, I do like your hat", "I don't know, I didn't see") then you're not going to be seen as dishonest even if you did somehow get caught.

Reputation is an over-simplification. The other person has in their mind a model of you and how you behave, that they use to make decisions. Your actions impact that model, and not in the Functional Decision Theory [? · GW] sense where they quickly end up with a perfect clone of you [? · GW]. They are not going to be able to accurately model the real you, because the real you is too big to fit in their tiny mental model.

Most of the time, the amount of brainpower you're putting into your facade is both wider and deeper than what they're putting into trying to get a read on you. They're distracted by other people and other tasks. They are going to apply limited heuristics because they have no other choice. If you want to be trusted, you do not even need to trick the other person, just the heuristic they're using to assess credibility.

Typically people will trust you more if you more accurately match their existing model of how a trustworthy person behaves (wearing a suit, sitting straight, answering politely, etc.) even when those things aren't causally related, and even when you are doing those things deceptively to gain their trust. If you show up to the job interview with your real personality and really are a person who would never mislead anyone, but that personality has features that correlate with dishonour in people they've met before, sucks to be you.

If you want a reputation and appearance of Trustworthiness, you have to roleplay the Flynn Rider of Appearing Trustworthy, not your real self who obsesses over the Actual Truth. Most people who obsess over the truth are crazy, and have so many mistaken beliefs that they're worse than liars. You do not want to look like them. The details and techniques of how to do this fill many books, so I have sprinkled some examples through this essay as a scavenger hunt. Or if you prefer, pick on any other persuasive writer you like and dig out all the ways they try to make you hallucinate credibility via text alone.

Cooperative Epistemics

I promised to get back to this earlier (I promise I made solely so I can point it out now, so you think me a promise-keeper and therefore more trustworthy (and I am now pointing that out too in the hopes for another chance for you to see how this works)).

The motto of science is not "If we all tell the Truth to each other, we can form a little bubble where we collectively figure out all the big issues and come to fully understand the world". The motto of science is nullius in verba, "take nobody's word for it".

You cannot actually make a community of good epistemics on the expectation of trust and cooperation. It is like trying to start a communist utopia on the expectation that everybody just does their part and takes only what they need. People will not just.

Even if they would just, that wouldn't even be good enough. A person who is trying to be truthful can still be mistaken. They can still be selectively silenced. They can still trick themselves out of fear or regret, or because it's necessary to protect their ego. They can have a psychotic break. Their account could be hacked.

People who have heard other people claim that title and for some reason believed them, are creating the worst possible set of incentives. The greatest possible force by which to bring sociopaths into your community, or to make otherwise good people decide just maybe this one time to fudge an answer. Nobody would notice. Everybody is just trusting them. Do them the charity of not pretending they wouldn't be making a terrible mistake by imagining they can take you or anyone else at their word. Build your Cooperative Epistemics on distrust instead.

Conclusion

I believe that we are all friends here: I am not an honest person. You can tell that's true because if it wasn't I wouldn't say it, would I? You can tell I think you friends, because if I didn't I'd lie and say I was honest. It is only because I believe this that I am admitting to the incentives around dishonesty, and trying to help you all navigate towards truth better in the choppy waters where you find yourselves.

Do not go through life as a pedantic idiot with no social model just because people on a forum for truthseekers who hate social models think it's admirable and virtuous. Especially do not do it if you are trying to accomplish some other task like saving lives or getting a job or getting someone to love you.

I imagine you are going to feel bad about it, like you are doing something wrong. That sense of wrongness will show on your face and make you look like a crazy person who panics over normal small talk, so you're going to have to get over it. To your benefit in getting over it, it isn't actually wrong.

Saying words is just an action, like any other action. You judge actions by their consequences. Are people made worse off or not? Most of the time, you're not poisoning a shared epistemic well. The well was already poisoned when you got here. It's more of a communal dumping ground at this point. Mostly you'd just be doing the sensible thing like everybody else does, except that you lack the instinct and intuition and need to learn to do it by rote.

When it makes sense to do so, when the consequences are beneficial, when society is such that you have to, when nobody wants the truth, when nobody is expecting the truth, when nobody is incentivising the truth: just lie to people.

20 comments

Comments sorted by top scores.