I feel like this post contains a valuable insight (it's virtuous to distrust and verify, to protect the community against liars), sandwiched inside a terrible framing (honor is for suckers).

Have we forgotten Sam Bankman-Fried already? Let’s not renounce virtues in the name of expected value so lightly. 
 

Rationalism was founded partly to disseminate the truth about AI risk. It is hard to spread the truth when you are a known liar, especially when the truth is already difficult to believe. 

Thanks for writing this post! I think it's insightful, and agree about technical truthtelling being annoying. After I thought about it though, I come down on the side of disagreeing with your post, largely on practical grounds.

A few thoughts:

1. You propose: Lie by default whenever you think it passes an Expected Value Calculation to do so, just as for any other action. This is fine, but the rest of the section doesn't make it clear that by default there are very few circumstances where it seems theoretically positive EV to lie (I think this situation happens once or twice a year for me at most, certainly not enough for there to be good feedback loops.) Lies are annoying to keep track of, they bite you in the ass often, and even if you're fine with lying, most people are bad at it. This means that the average liar will develop a reputation for dishonesty over time, which people generally won't tell you about, but will tell other people in your social network so they know to watch out. More explicitly, I disagree with the idea that since each person is on average not paying attention, lying is easy. This is because people love to gossip about other people in their social circle who are acting weird, and being noticed by any person means that the information will propagate across the group.
2. You propose: Practice lying. Same as Tangled, this only works if you start very young. If you do this after high school, you will permanently burn social capital! In the case of you doing so with non-consensual subjects, you will be caught because you are bad at it, and people will think that you are deceptive or weird. In the case where you find parties who can actively help you become a more dishonest person, those people will reasonably trust you less, and also it seems generally unwise to trust such parties.
3. Re: developing the skill of detecting the relative honesty of other people: I agree that this is a good skill to have, and that "people will lie to you" is a good hypothesis to entertain on a regular basis. However this is a separate skill tree, and also one where facts and logic™ can thankfully save you. I'm not terrible at assessing vibes, decent at thinking about if stories check out, and I also can tap into the network of mutual acquaintances if something seems subtly off or weird about a person. This has not made me any less terrible at lying.
4. Advocating for more lying seems like especially bad advice to give to people with poor social skills, because they lack the skills to detect if they're succeeding at learning how to lie or if they're just burning what little social capital they have for no gain. For people with poor social skills, I recommend, like, reading books about improving your social skills or discussing their confusions with friends who are more clued in, and for autistic people I recommend developing a better model of how neurotypicals think. I have disagreements with some of the proposed models in the book, but I think A Field Guide to Earthlings by Ian Ford is a good place to start.
5. The flip side to the average person not being totally honest, is that if you can credibly signal that you are unusually honest using expensive signals, there actually are many niches for you in the world, and people pay attention to that too. I touch on this in a previous post of mine on unusually scrupulous non-EA charities [LW · GW]. While it's true that a few folks on the website can stand to become a little savvier socially^[1], I think in general it would be better if they chose to play to their advantage. This seems like the higher EV route to me. And this is actually one of the reasons that I'm annoyed about technical truth telling - people who practice it are technically honest but they're not even getting any good reputation for it because they're functionally deceiving people, badly.
6. All of the best things in my life came from moments where it felt very scary to tell the truth, and then I was brave and did so anyways.

1. ^{^}

   i think this case is generally overstated, btw. its true that some lw people are bad at social skills but i think the median user is probably fine.

I think about this topic a lot, and I appreciate your dissent, particularly since it helped me organize my thoughts a little. That said, I think you're almost completely wrong. The best way to get at the problem is probably to start with your examples. Not exactly in order, sorry. 

The lesson here is likewise clear: If your actual personality isn't good enough, pretend to be Flynn Rider to everyone at all times, with the sole carve-out being people who love you, like your mother or a princess. This works because people who love you will find your openness endearing, whereas everyone else will think you pathetic and use it against you.

Here's a true story. I once met a lovely and intelligent woman who didn't like that I'm a bit blunt and ruthlessly truthseeking. I didn't stop being that way, and mainly for that reason we didn't become romantically involved. A few months later I met a lovely, intelligent, reasonable, sometimes blunt, and open-minded woman who did like that I'm a bit blunt and ruthlessly truthseeking. We've been dating for 2.5 years now and I'm on balance very happy with how everything worked out.

A Paragon of Morality is out travelling, when he is beset by bandits. They demand he hand over his gold or they will kill him and take it from his corpse. This is not a decision-theoretic threat [? · GW] because the bandits value getting his gold more than they disprefer commiting murder, but would otherwise avoid the murder if possible. If he hands over all his gold he will lose all his gold. If he hands over all the gold in his pockets, neglects the extra he has hidden in his sock, and says "I have given you all my gold" in a sufficiently convincing tone of voice, then he will lose less than all his gold.

These isn't Omega we're dealing with here, they're totally trickable by a moderately convincing performance. If he keeps some of the gold he can donate it to Givewell approves charities and save however many QALYs or whatever.

Does he have a moral obligation to lie?

He certainly doesn't have a moral obligation to tell the truth. But a lot of moral obligations change when someone points a gun at you. For instance, it becomes morally permissible (though not necessarily feasible) to shoot at them, or to give up what money you must and later steal it back at the first available opportunity. To me, the truth is something precious, and lying is like stealing the truth; it's permissible in some extreme and usually adversarial situations. With that said, I'm a bit of a rationalist dedicate/monk and I'd prefer to fight than lie - however I don't think everyone is rationally or otherwise compelled to follow suit, for reasons that will be further explained. 

A Normally Honest Man is applying for a job as a Widget Designer. He has many years of industry experience in Widget Engineering. He has memorised the Widget Manufacturing Process. He's actually kind of obsessed with Widgets. Typically whenever a conversation becomes about Widgets he gushes openly and makes a bad impression with his in-laws. Since that incident he has developed the self control to pretend otherwise, and the rest of his personality is okay.

The interviewer works for a Widget Manufacturing company but seems to only care about Widgets a normal amount. He asks "How interested are you in Widgets?" He has learnt from previous job interviews that, if he answers honestly, the interviewer will think he is any of lying, insane, or too weird to deal with, and not hire him, even though this is not in the best financial interests of the company, were they fully informed.

Should he pretend to like widgets the amount most likely to get him hired, or does he have a moral obligation to keep answering honestly until he runs out of rent money and becomes homeless?

I don't know, he could say "Honestly, I enjoy designing widgets so much that others sometimes find it strange!" That would probably work fine. I think you can actually get a way with a bit more if you say honestly first and then are actually sincere. This would also signal social awareness. 

I realize that I am in some sense dodging your hypothetical but I think your hypothetical is the problem. You haven't thought hard enough about how this guy can succeed without lying.

A Self-Improvement and Epistemics Nerd has an online community for Self-Improvement and Epistemics Nerds. Half the people reading it are autists with bad social skills, who weren't at exactly the right age to be saved by Disney's Tangled. They struggle with navigating ordinary social situations and obtaining true beliefs because they're bad at lying, and insufficiently aggressive at anticipating it in others.

Would they be doing anyone a favour in encourage a social norm of truthfulness and the expectation of truthfulness in others, when all those people will inevitably have to leave the computer one day and end up like the subjects of the previous two examples? Would they be making the world a better place?

Yes and yes. 

Contrary to common belief, lesswrong is not an autism support group. 

And you know what? I think it made the world much better. Now we have places online and in the real world (lighthaven, meetups, Berkeley) to gather and form a community around truthseeking and rationality. I like it. I'm glad it exists. I even think some important and powerful ideas have come out of it, and I think we've learned a lot together. 

Saying words is just an action, like any other action. Whether the words are literally true or not is just a fact about the action, like any other fact about an action. It's not the morally important fact. You judge actions by their consequences, whether you expect it to lead to more good or bad. Then you take the action with the best consequences overall.

Saying words is an action, but it's not like any other action, because it can guide others towards or away from the truth. Similarly, torture is an action, but it's not like any other action, because it is when one person causes another immense pain intentionally. 

Sure, we judge actions by their consequences, but we do not judge all actions in the same way. Some of them are morally repugnant, and we try very, very hard to never take them unless our hands our forced, and then only take them with immense regret and sorrow. There are various distinguishing factors. For instance, the consequences of torture seem likely to be almost always bad, so I never seriously consider it. Also, I don't want to be the sort of person who tortures people (both for instrumental reasons and to some extent for intrinsic reasons). It's actually pretty hard to fully disentangle my disprefernece for torture from its consequences, because torture is inherently about causing suffering and don't either want suffering to exist or to cause it (though the former is far more important to me). 

My feelings about lying are the same. I love the truth, I love the truthseeking process, I love seeing curiosity in the eyes of children and adults and kittens. I hate lies, confusion, and deceiving others. This is partially because the truth is really useful for agents (and I like agents to be able to exercise their potential, typically), it's partially because telling the truth seems to be best for me in most cases, and it's partially because I just value truth.

Rationality can be about Winning [LW · GW], or it can be about The Truth, but it can't be about both. Sooner or later, your The Truth will demand you shoot yourself in the foot, while Winning will offer you a pretty girl with a country-sized dowry. The only price will be presenting various facts about yourself in the most seductive order instead of the most informative one.

It can totally be about both if truth is part of winning. Yes, there are sometimes tradeoffs, and truth is not the singular source of value. But I think most of us value it very strongly, so presenting these two axes as orthogonal is highly misleading. And I want to share the truth with other people in case they decide to value it too - if not, they can always choose not to face it. 

Also, there's a missing mood in your example. When you value the truth, being honest tends to get you a lot of other things that you value; you tend to end up surrounded by the right people for you, being the kind of person you can respect, in the kind of place where you belong, even if you have to create it.

Now, you're probably going to say that I can't convince you by pure reason to intrinsically value the truth. That's right. However, I also can't convince you by pure reason to intrinsically value literally anything, and if you had written an essay about how we should consider killing or torturing people because it's just an action like any other, I would have objected on similar grounds. You're totally missing the fact that it's wrong, and also (separately!) the consequences of following your advice would probably be bad for you, and certainly for most of us, over the long run. 

He asks “How interested are you in Widgets?” He has learnt from previous job interviews that, if he answers honestly, the interviewer will think he is any of lying, insane, or too weird to deal with, and not hire him, even though this is not in the best financial interests of the company, were they fully informed.

By the standard "intentionally or knowingly cause the other person to have false beliefs", answering 'honestly' would be lying, and answering in a toned down way would not (because it maximizes the truth of the belief that the interviewer gets).

Detailed commentary, as promised:

Rationality can be about Winning [LW · GW], or it can be about The Truth, but it can’t be about both. Sooner or later, your The Truth will demand you shoot yourself in the foot, while Winning will offer you a pretty girl with a country-sized dowry. The only price will be presenting various facts about yourself in the most seductive order instead of the most informative one.

If your highest value isn’t Winning, you do not get to be surprised when you lose. You do not even get to be disappointed. By revealed preference, you have to have a mad grin across your face, that you were able to hold fast to your highest-value-that-isn’t-winning all the way to the bitter end.

What if my highest value is getting a pretty girl with a country-sized dowry, while having not betrayed the Truth?

Then, if I only get one of those things, it’s worse than getting both of those things (and possibly so much worse that I don’t even consider them worthwhile to pursue individually; but this part is optional). Is there a law against having values like this? There is not. Do you get to insist that nevertheless I have to choose, and can’t get both? Nope, because the utility function is not up for grabs^[1]. Now, maybe I will get both and maybe I won’t, but there’s no reason I can’t have “actually, both, please” as my highest value.

The fallacy here is simply that you want to force me to accept your definition of Winning, which you construct so as not to include The Truth. But why should I do that? The only person who gets to define what counts as Winning for me is me.

In short, no, Rationality absolutely can be about both Winning and about The Truth. This is no more paradoxical than the fact that Rationality can be about saving Alice’s life and also about saving Bob’s life. You may at some point end up having to choose between saving Alice and saving Bob, and that would be sad; and you would end up making some choice, in some manner, as fits the circumstance. The existence of this possibility is not particularly interesting, and has no deep implications. The goal is “both”.

(That said, the bit about the two armies was A++, and I strong-upvoted the post just for that.)

Technical Truth is as Bad as Lying

I wholly agree with this section…

… except for the last paragraph—specifically, this:

The purpose of a word is to carve reality at a joint useful for the discussion taking place, and we should pause here to note that the joint in question isn’t “emits true statements”, it’s “emits statements that the other party is better off for listening to”.

No, it’s not.

I’m not sure where this meme comes from^[2], but it’s just wrong. Unless you are, like, literally my mother, “is the other party, speficially, better off for listening to this thing that I am saying” constitutes part of my motivation for saying things approximately zero percent of the time. It’s just not a relevant consideration at all—and I don’t think I’m even slightly unusual in this.

I say and write things^[3] because I consider those things to be true, relevant, and at least somewhat important. That by itself is very often (possibly usually) sufficient for a thing to be useful in a general sense (i.e., I think that the world is better for me having said it, which necessarily involves the world being better for the people in it). Whether the specific person to whom the thing is nominally or factually addressed will be better off as a result of what I said or wrote is not my concern in any way other than that.

Sometimes I am additionally motivated by some specific usefulness of some specific utterance, but even in the edge case where the expected usefulness is exclusive to the single person to whom the utterance is addressed, I don’t consider whether that person will be better off for having listened to the thing in question. Maybe they won’t be! Maybe they will somehow be harmed. That’s not my business; they have the relevant information, which is true (and which I violated no moral precepts by conveying to them)—the rest is up to them.

Therefore if someone says “but if you lie, they’ll be better off”, my response is “weird thing to bring up; what’s the relevance?”.

Biting the Bullet

Basically correct. I will add that not only is it not always morally obligatory to tell the truth, but in fact it is sometimes morally obligatory to lie. Sometimes, telling the truth is wrong, and doing so makes you a bad person. Therefore the one who resolves to always tell the truth, no matter what, can in fact end up predictably doing evil as a direct result of that resolution.

There is no royal road to moral perfection. There is no way to get around the fact that you will always need to apply all of your faculties, the entirety of your reason and your conscience and everything else that is part of you, in order to be maximally sure (but never perfectly sure!) that you are doing the right thing. The moment you replace your brain with an algorithm, you’ve gone wrong. This fact does not become any less true even if the algorithm is “always tell the truth”. You can and should make rules, and you can and should follow them (rule consequentialism is superior to act consequentialism for all finite agents), and yet even this offers no escape from that final responsibility, which is always yours and cannot be offloaded to anyone or anything, ever.

Lie by default whenever you think it passes an Expected Value Calculation to do so, just as for any other action.

No, this is a terrible idea. Do not do this. Act consequentialism does not work. It doesn’t work no matter how much we say “yeah but just make up numbers” or “yeah you can’t actually do the calculation, but let’s pretend we can”. The numbers are fake and meaningless and we can’t do the calculation.

It’s still a better policy than just trusting people.

Definitely don’t just trust people. Trust, but verify. (See also. [LW(p) · GW(p)])

When your friends ask you about how trustworthy you are, make no implications that you are abnormally honest. Tell them truthfully (if it is safe to do so) about all the various bad incentives, broken social systems, and ordinary praxis that compel dishonesty from you and any other person, even among friends, and give them sincere advice about how to navigate these issues.

This, I agree with.

Cooperative Epistemics

I agree with all of this section. What I’ll note here is that there are people who will campaign very hard against the sort of thing you are advocating for here (“Do them the charity of not pretending they wouldn’t be making a terrible mistake by imagining they can take you or anyone else at their word. Build your Cooperative Epistemics on distrust instead.”), and for the whole “trying to start a communist utopia on the expectation that everybody just” thing. I agree that this actually has the opposite result to what anyone sensible would want.

Saying words is just an action, like any other action. You judge actions by their consequences. Are people made worse off or not? Most of the time, you’re not poisoning a shared epistemic well. The well was already poisoned when you got here. It’s more of a communal dumping ground at this point. Mostly you’d just be doing the sensible thing like everybody else does, except that you lack the instinct and intuition and need to learn to do it by rote.

When it makes sense to do so, when the consequences are beneficial, when society is such that you have to, when nobody wants the truth, when nobody is expecting the truth, when nobody is incentivising the truth: just lie to people.

This, on the other hand, is once again a terrible idea.

Look, this is going to sound fatuous, but there really isn’t any better general rule than this: you should only lie when doing so is the right thing to do.

“When the consequences are beneficial”—no, you can’t tell when the consequences will be beneficial, and anyhow act consequentialism does not and cannot work, so instead you should be a rule consequentialist and adopt rules about when lying is right, and when lying is wrong, and only lie in the first case and not the second case. (And you should have meta-rules about when you make your rules known to other people—hint, the answer is “almost always”—because much of the value of rules like this comes from them being public knowledge. And so on, applying all the usual ethical and meta-ethical considerations.)

“When society is such that you have to”—too general; furthermore, people, and by “people” I mean “dirty stinking liars who lie all the time, the bastards”, use this sort of excuse habitually, so you should be extremely wary of it. However, sometimes it actually is true. Once again you cannot avoid having to actually think about this sort of thing in much more detail than the OP.

“When nobody wants the truth”—situations like this are often the ones where telling the truth is exceptionally important and the right thing to do. But sometimes, the opposite of that.

“When nobody expects the truth”—ditto.

“When nobody is incentivizing the truth”—ditto.

The well was already poisoned when you got here. It’s more of a communal dumping ground at this point.

Wells can be cleaned, and new wells can be dug. (The latter is often a prerequisite for the former.)

Actually, even if your personality is good enough, you should probably still pretend to be Flynn Rider, because his personality is better. It was, after all, carefully designed by a crack team of imagineers. Was yours? Didn't think so.

Personalities don't just fall into a linear ranking from worse to better.

Imagineers' job isn't to design a good personality for a friendless nerd, it's to come up with children's stories that inspire and entertain parents and which they proudly want their children to consume.

The parents think they should try to balance the demands of society with the needs of their children by teaching their children to scam the surrounding society but being honest about the situation with their loved ones. Disney is assisting the parents with producing propaganda/instructions for it.

https://benjaminrosshoffman.com/guilt-shame-and-depravity/

Basing your life on scamming society is a bad idea but you shouldn't solve it by also trying to scam your loved ones. If you are honest, you can more easily collaborate with others to figure out what is needed and how you can contribute and what you want.

Lie by default whenever you think it passes an Expected Value Calculation to do so, just as for any other action. 

How do you propose to approximately carry out such a process, and how much effort do you put into pretending to do the calculation?

I'm not as much a stickler/purist/believer in honest-as-always-good as many around here, I think there are many times that deception of some sort is a valid, good, or even morally required choice. I definitely think e.g. Kant was wrong about honesty as a maxim, even within his own framework. But, in practice, I think your proposed policy sets much too low a standard, and in practice the gap between what you proposed vs "Lie by default whenever it passes an Expected Value Calculation to do so, just as for any other action," is enormous in both the theoretical defensibility, and in the skillfulness (and internal levels of honesty and self-awareness) required to successfully execute it.

I agree with a lot of things in this post and disagree with a lot of things in this post, but before I comment in more detail, I would like to clarify one thing, please:

Are you aware that there exist moral frameworks that aren’t act consequentialism? And if so, are you aware that some people adhere to these other moral frameworks? And if so, do you think that those people are all idiots, crazy, or crazy idiots?

(These questions are not rhetorical. Especially the last one, despite it obviously sounding like the most rhetorical of the set. But it’s not!)

I mean, Flynn Rider was also really good-looking. For a lot of people, maybe most, this look is just unattainable. Even if you can get in as good physical shape (which is far from easy), what if you're older, shorter, balder, have a goofy face and so on.

This is directionally correct and most lesswrongers could probably benefit from taking the advice herein, but goes too far (possibly as deliberate humor? The section about Flynn especially was quite funny XD).

I do take issue with the technical-truths section; I think using technical truths to trick people, while indeed a form of lying, is quite distinct from qualifying claims which would be false if unqualified. It's true that some philistines skim texts in order to respond to vibes rather than content, but the typical reader understands qualifiers to be part of the sentences which contain them, and to affect their meaning. That is why qualifiers exist, to change the meanings of the things they qualify, and choosing to ignore their presence is a choice to ignore the actual meaning of the sentences you're ostensibly reading.

I liked this post a lot more than I expected to, but I'm disappointed the only examples of lying are a combination of people who have no right to the information and people who are better off for you lying (in a way that gives them truer beliefs than if you'd told the literal truth).

The hard cases are much more interesting. What about lying to my landlord about renting a room on airbnb? What about saying your class will make people millionaires for the low low price of $1,000 (hey, it could happen)? What about hiding the rats from the health inspector?

This is pure first-principles reasoning without a single glance at how humans actually behave, eg, how they assign a reputation for honesty.

If you believed this, why would you write this post?

Great post! I really enjoy your writing style. I agree with everything up to your last sentence of cooperative epistemics. It looks like a false equivalence between a community of perfect trust and a community based on mistrust. I'm thinking a community of "trust but verify" with a vague assumption of goodwill will capture all the benefits of mistrust without the risks of half rationalists or "half a forum of autists" going off the deep end and making a carrying error in their EV calculations to overly negative results.

Corrupted Hardware [? · GW] leads me to think we need to aim high to end up at an optimum level of honesty. 

Edit: Thanks Cole and Shankar. 

I'm sorry that your life has come to a point where you might feel like adopting a fake personality and regularly deceiving those around you is how to be happy and get what you want. It's almost impossible to deliver good advice to a stranger over the internet, but I hope that one day you can find joy and fulfillment in being yourself.