The Belief Signaling Trilemma

(1) We can agree to stop assigning reputation based on beliefs.

I don't think that this is a realistic option, even ignoring the loss of information. Sure, if David Icke signed up on Less Wrong, I could claim that I was assigning his beliefs an equal amount of reputation as everyone else's... but I wouldn't actually be doing that, and neither would anyone else here who claimed to.

(2) We can agree to always report honest beliefs even though we know it will cost us reputation.

This one doesn't seem realistic either. We can certainly try to encourage people to always report their honest beliefs, but there's no way you can enforce that, and many people will keep hiding beliefs if they think that revealing those beliefs would hurt them. Note that this would also force many people to use pseudonyms, since any unpopular opinions that are posted here will also be visible to Google, which might hurt them in real life.

I think that it's important to signal that we don't expect anyone to have psychologically unrealistic levels of altruism or community commitment. People are going to act according to their own self-interest here, and that's perfectly fine as long as it doesn't actively hurt the community. Associating with LW shouldn't make anyone feel actively uncomfortable, and compulsory radical honesty is going to make a lot of people feel uncomfortable. Keeping some of your beliefs as secrets is fine; everyone here has their right to mental privacy.

Also, self-censorship of publicly expressed beliefs also has value in improving the signal to noise ratio. If someone here really did believe that shape-shifting reptilian aliens controlled all the major governments, we wouldn't want them to regularly bring up this position.

By signaling belief in X, I increase my reputation in the community, and will cause other people to take more seriously my views on other issues I think are more important. Therefore, I choose to signal belief in X.

See, this kind of comment makes me conclude I'm a weirdo from another planet. Feigning beliefs for approval is something I suppose I've engaged in a time or two, and to the best of my recollection, quickly felt the need for a shower. And on an internet forum? What the hell is the point? So that people can respect some fictitious creature who isn't me?

I certainly don't do this here. Why would I play the part of someone with views I disagree with, for approval from others exactly where I believe they are wrong? How peculiar. How distasteful. Ugh. I'm glad I was born from a different species.

Me, I want the respect of people I respect, the people I think are right. Why would I pander to the wrong? Ugh.

I don't think this is a very big problem: an additional factor is that people are uncertain about what the community believes and what expressed beliefs harm reputation. They may expect others to agree with them (the "silent majority"), or at least open to be convinced.

People also vary in what kind of reputation they care about (some love being the Daring Rebel Speaking Truth To Power); so between dissenters that don't know they're dissenters, and those who don't care, enough different points of view should be introduced.

The comments on this post went in a direction that I was not intending. When I wrote this post, I wasn't thinking about less wrong in particular, but instead I thought that this was a universal phenomenon. I was trying to make the point that under any circumstances you must use at least one of the three options to some extent, and that this was sad because all three have their own unique downsides.

(2) We can agree to always report honest beliefs even though we know it will cost us reputation.

I don't think reputation is the only concern when it comes to revealing beliefs. If I make an argument to convince people of A I don't have to also say B.

Even if I don't believe that an AGI will be discovered in the next 100 years I can still particpate in a discussion about AI safety and assume for the sake of the discussion that an AGI will be discovered in the next 100 years.

It's interesting that you think the problem with 2 is that it encourages self-deception. I would think the problem would be the loss of reputation.

Anyway, of the three you presented, I choose 2. For as long as I can remember, I've been speaking honestly about my beliefs, which is part of the reason my reputation hasn't been very high throughout much of my life. I still think 2 is the best option, though I'm attempting to change how I express my beliefs to reduce the cost to my reputation.

I don't want to propose a solution without some idea of how common a problem this is, because that affects which one is optimal.

How many beliefs (If drawing lines between separate beliefs is an issue, call them separate if you have separate reasons for disagreeing with the perceived popular opinion on LessWrong on them) do you hold but claim not to hold for the purpose of signaling epistemic credibility on LessWrong? (Don't rationalize "I only agreed under pressure that one time." If most of the time you have been asked what you believe, you lied, include that belief in this count) [pollid:554]

How many beliefs do you hold, and not lie about, but also avoid discussing on LessWrong for the same purpose? (Beliefs you do not discuss because they are rude is a separate issue) [pollid:555]

Make sure you don't accidentally uncheck the anonymity thing.

(4) See people who are willing to honestly report beliefs contrarian to the group in which they are expressed as high status.

I'm glad you brought this up - I hadn't known what to call it, but I also sensed an issue along these lines. (What I observed was that I feel social pressure not to deviate too far from what most members of the site believe. This does not necessarily mean anyone is trying to exert this social pressure, or even that they are doing anything in particular other than voicing similar beliefs.)

I think this will take more than just agreeing on a policy for assigning reputation or voicing opinions, though. My inner homo erectus will try to protect me from reputation damage regardless of whether reputation damage could actually occur, and there are probably also unconscious factors influencing how I precieve others' social status.

But maybe we can hack this somehow? How can we artificially influence our feelings about social status to counterbalance our natural tendencies? (Eating a cookie immediately after saying things I was afraid to say is something I've tried in the past, in a different context. That doesn't seem like it would work as a community-wide norm, but an individual could try doing it themselves.)

We don't have enough coordination to implement (1) or (2). (3) is going to happen, but we can mitigate the bias by keeping in mind that consensus we see may be fake (and how much it might be fake). There's still the problem of potentially worthwhile opinions never coming to our attention. Maybe if your real opinion is worthwhile, you should use a sock puppet account to post it? (Email addresses are easy to get).

On the other hand, maybe you expect to lose epistemic credibility for your real opinion because it is actually not worthwhile, in which case encouraging you to post it on a sock puppet account is a bad idea. Maybe there should be a "secret opinions thread" where people posted what they really believed with sock puppet accounts. Or we could just use another site like Reddit.

I have a suspicion that your option number 2 is already baked pretty deep into actual humans' psychologies.

Of course, your measure of the intelligence of the members is biased towards people who say things that you agree with.

This isn't actually necessary for the trilemma. Suppose a lot of people hold belief X for irrational reasons, whereas initially rationalists are split between X and ~X. Then holding belief ~X is evidence that you are above average rationality, note that this is true even for rational observers who believe X.

[Option 2] is bad because it encourages self-deception.

Only if each person cares about their own reputation, which is the case for “actual humans” but not for “ideal agents”.

I want to add that I think that all three of these options are legitimate enough to consider, and I think that there are people who follow all of them. I don't always succeed, but I feel a moral push and try very hard to live my life using strategy 2, at least in situations that allow for an actual debate between the sides. However, I know people who I believe would say they think everyone should follow strategy 1, and I think that that most people follow strategy 3 most of the time without thinking about it at all.