Trust-Building: The New Rationality Project

What we want isn't a lack of factionalism, it's unity. ... You have high trust in this network, and believe the evidence you receive from it by default.

I think one of the ways communities can differ are the local communication norms. Rather than saying something like "all communities have local elders whose word is trusted by the community", and then trying to figure out who the elders are in various communities, you can try to measure something like "how do people react to the statements of elders, and how does that shape the statements elders make?". In some communities, criticism of elders is punished, and so they can make more loose or incorrect statements and the community can coordinate more easily (in part because they can coordinate around more things). In other communities, criticism of elders is rewarded, and so they can only make more narrow or precise statements, and the community can't coordinate as easily (in part because they can coordinate around fewer, perhaps higher quality, things).

It seems to me like there's a lot of value in looking at specific mechanisms there, and trying to design good ones. Communities where people do more reflexive checking of things they read, more pedantic questions, and so on do mean "less trust" in many ways and do add friction to the project of communication, but that friction does seem asymmetric in an important way.

Wealthy people are more reliable because on average reliability is one of the inputs to wealth building

Wealthy people are less reliable because they can afford to blow things off if at any given moment their BATNA exceeds their agreement (lower switching costs as a fraction of their wealth + less concern about future problems from defecting)

Hard to predict which dominates...

I frame it like this because rationalists are, globally speaking, quite wealthy. And I expect that people are about as trustworthy as they've found they need to be. If you are wealthy, and excessively trustful, your excess trust will get arbitraged down. People are quite proud of not falling for ingroup outgroup dynamics. But ingroups are also well kept gardens, where trust doesn't get exploited and thus high trust value chains can get constructed. Later, once fully fleshed out and understood, such chains can be exported to lower trust environments via formal rules. But their formation is fragile.

When it feels like there's no need to explore, and all you need to do is practice your routine and enjoy what you have, the right assumption is that you are missing an opportunity. This is when exploration is most urgent.

I think good advice is often of the form "in situation X, Y is appropriate"; from a collection of such advice you can build a flowchart of observations to actions, and end up with a full policy.

Whenever there is a policy that is "regardless of the observation, do Y", I become suspicious. Such advice is sometimes right--it may be the case that Y strictly dominates all other options, or it performs well enough that it's not worth the cost of checking whether you're in the rare case where something else is superior.

Is the intended reading of this "exploitation and routine is never correct"? Is exploration always urgent?

To increase unity and pursue the truth, your goal is to find foreign communities, and determine whether they are friendly or dangerous.

This is an interesting framing I hadn't considered. I usually think of it more as just an exploration to find what perspectives communities/traditions/etc. offer in an ongoing hermeneutical search for the truth, digging into the wrong models other people have to get a bit less wrong myself, which seems pretty compatible with your take.  

I don't think that factionalism is caused solely by mistrust. Mistrust is certainly a part of the picture, but I think that interest in different things is also a part. Consider the factions around two substantially different academic fields, like medival history and pure maths. The mathmaticians largely trust that the historians are usually right about history. The historians largely trust that the mathmaticians are usually right about maths. But each field is off pursuing its own questions with very little interest in the other.

What we want isn't a lack of factionalism, it's unity.

I am not sure we do want unity. Suppose we are trying to invent something. Once one person anywhere in the world gets all the pieces just right, then it will be obviously good and quickly spread. You want a semiconductor physics and a computer science faction somewhere in the world to produce smartphones. These factions can and do learn from the maths and chemistry factions, the factions they don't interact with are either adversarial or irrelevant.

The advice seems to me like it's advocating cultism too much. I have plenty of relationships that have nothing todo with the rationality community and I would hope the same is true for most members of our community. 

I like that my mouth, ears and eyes produce different data in the same situation. An extreme of "unity" could be that every sense reports the same finding. Sure there are situations that the idiom "I can't believe my eyes" points to but even in that case it can be described as an (ab)use of language, it's the brains job to form thoughts and the eyes job to register the electromagnetic field.

One shouldn't delegate thinking to other persons and contextualising others contribution as input rather than output is a raod to be more sensitive to reality.