Dragon Agnosticism

(Disagree that it was a tenable position a decade or two ago, agree that it is an untenable position now)

Tbc: It should be fine to argue against those implications, right? It’s just that, if you grant the implication, then you can’t publicly refute Y.

I don't see why it should be limited to false beliefs.

Note that even if X is true, X->Y need not be true, and it can still be harmful to not be able to question X->Y.

Suppose an ideology says you're not allowed to question idea X.

I think there are two different kinds of "not questioning": there's unquestioningly accepting an idea as true, and there's refusing to question and remaining agnostic. The latter position is reasonable in the sense that if you refuse to investigate an issue, you shouldn't have any strong beliefs about it. And I think the load-bearingness is only a major issue if you refuse to question X while also accepting that X is true.

Is there another strategy you prefer? Afaict the options are 

1) Have public taboo beliefs.

2) Have private beliefs that you lie about. 

3) Remain deliberately agnostic about taboo but insufficiently important topics.

4) Get forever lucky, such that every taboo topic you investigate results in you honestly arriving at an allowed belief.

Whether 1) is at all compatible with having other career goals is a fact of the territory, and I expect in the US in 2024, there are topics where having taboo beliefs could totally end your career, for many values of career. (Leaving open whether there are such beliefs that are true, but, per the topic of this post, that's not something you can learn without taking risks.)

2) seems even more prone to the effect you describe than 3).

My guess is you're making a bid for 1), but I feel like a case for that should take into account the costs of believing X weighed against the costs of agnosticism about X, rather than a sweeping heuristic argument. (Where maybe the cost of agnosticism about X includes adjacent topics Y you'll either have to include in your agnosticism or otherwise eat the cost of ~X connotations, though I'm skeptical about how often this will come up, and per my comment here [LW(p) · GW(p)] I expect the ~Y->~X taboos will often be much smaller than the ~X taboo.)

One of the advantages to remaining agnostic comes from the same argument that users put forth in the comment sections on this very site way back in the age of the Sequences (I can look up the specific links if people really want me to, they were in response to the Doublethink Sequence) for why it's not necessarily instrumentally rational for limited beings like humans to actually believe in the Litany of Tarski [? · GW]: if you are in a precarious social situation, in which retaining status/support/friends/resources is contingent on you successfully signaling to your in-group that you maintain faith in their core teachings, it simply doesn't suffice to say "acquire all the private truth through regular means and don't talk/signal publicly the stuff that would be most dangerous to you," because you don't get complete control over what you signal. 

If you learn that the in-group is wrong about some critical matter, and you understand that in-group members realizing you no longer agree with them will result in harm to you (directly, or through your resources being cut off), your only option is, to act (to some extent) deceptively. To take on the role, QuirrellMort-style, of somebody who does not have access to the information you have actually stumbled upon, and to pretend to be just another happy & clueless member of the community.

This is capital-H Hard. Lying (or even something smaller-scale like lesser deceptions), when done consistently and routinely, to people that you consider(ed) your family/friends/acquaintances, is very hard for (the vast majority of) people. For straightforward evolutionary reasons, we have evolved to be really good at detecting when one of our own is not being fully forthcoming. You can bypass this obstacle if the number of interactions you have is small, or if, as is usually the case in modern life when people get away with lies, nobody actually cares about the lie and it's all just a game of make-believe where you just have to "utter the magic words." [LW · GW] But when it's not a game, when people do care about honestly signaling your continued adherence to the group's beliefs and epistemology, you're in big trouble.

Indeed, by far the most efficient way of convincing others of your bullshit on a regular basis is to convince yourself first, and by putting yourself in a position where you must do the former, you are increasing the likelihood of the latter with every passing day. Quite the opposite of what you'd like to see happen, if you are about truth-seeking to any large extent.

(addendum: admittedly, this doesn't answer the question fully, since it doesn't deal with the critical distinction between agnosticism and explicit advocacy, but I think it does get at something reasonably important in the vicinity of it anyway)

Also, a norm of "allowing people to keep their beliefs private on subjects they feel a lot of pressure on" gives space for people to gather information personally without needing to worry about the pressures on them from their society.

Mm, perhaps it will turn out that there are other, more benign ways of dealing with dragons. Narrative-brained people wont find those.

The trouble with this is that it's a socially awkward move to even imply you might research taboo topics. Better to leave public personna and tricky glomarization out of it, I think. Just publish your research and results anonomously. That seems to me to lead to a better epistemic state for society, since a standard of anonymous publication doesn't leave a misleading bias in publicly available research.

They don't, of course, but if you're lucky enough not to be located among the more zealous of them and be subjected to mandatory struggle sessions, their wrath will generally be pointed at more conspicuous targets. For now, at least.

No one has given me a hard time about it. I say things like "I haven't looked into it" and we move on. The next time it happens I will additionally be able to link to this post.

the reasons outlined in this post (i.e. "if I checked and it's not the way I thought, that would necessitate a lot of expensive and time-consuming updates to what I'm doing").

Wait, that's not what I'm trying to communicate in the post. If learning that dragons existed would precipitate major updates, it will very often be worth investigating their existence. Instead, it is on highly-political low-payoff topics that I am intentionally agnostic.

Say more?

I don't think inferring a probability anywhere near as high as 20% is justified. If, conditional on finding dragons the value of the knowledge is lower than the harms of being known as a dragon believer, then you shouldn't go check no matter how low your prior.

It's also possible that an opposing effect happens where your shouting into the void about dragons connects in some vague way with my belief in the Ilithids, which I then end up coopting your dragon evidence into my own agenda.  Especially if you find anything close to material evidence.  Heck, your material evidence for dragons now gives all kinds of creedance to Ilithids, beholders, gnomes, and all sorts.  So the gnome people and everyone else is now coming out of the woodwork to amplify your work on dragons.  And I think this would be regardless of the specific nuances your attribute to dragons.  I would expect those nuances to get smooshed in the fray to cite your once-and-for-all-proving-dragons strategy.

I mean, if Pons and Fleischmann was true, for example, I bet it would get trotted out with all kinds of half-baked theories on free energy, along with Tesla's name.  And the reason I'm making this bet is because these already do get trotted out into such discussions.  

(Not that I would ever read those reports or have done any such research into repressed Pons and Fleischmann evidence or Ilithid conspiracies)

Operationally it means that I'm not trying to find out the truth one way or the other. If I come across arguments I ignore them, if someone asks if they can explain it to me I say no, I try not to think about it, etc.

"Agnostic" doesn't necessarily mean "unknowable and not subject to testing". Much more often it has the weaker meaning "not currently known". There is a house being built across the street. Is there a work van parked in front of it right now? I don't know. This is certainly knowable and subject to testing - I could get up, walk over to a window in the front of the house, and look. But I don't care enough to do that, so I continue to now know if there is a work van parked in front of the house across the street. I am agnostic about the existence of such a work van.

Hmm, it seems like you might be treating this post as an allegory for religion because of the word "agnostic", but I'm almost certain that it's not. I think it's about "race science"/"human biodiversity"/etc., i.e. the claim "[ethnicity] are genetically predisposed to [negative psychological trait]".

Before I do that, though, it's clear that horrible acts have been committed in the name of dragons. Many dragon-believers publicly or privately endorse this reprehensible history. Regardless of whether dragons do in fact exist, repercussions continue to have serious and unfair downstream effects on our society.

While this could work as a statement about religious people, it seems a lot more true for modern racists than modern religious people.

Given that history, the easy thing to do would be to loudly and publicly assert that dragons don't exist. But while a world in which dragons don't exist would be preferable, that a claim has inconvenient or harmful consequences isn't evidence of its truth or falsehood.

This is the type of thing I often see LessWrongers say about race science.

But if I decided to look into it I might instead find myself convinced that dragons do exist. In addition to this being bad news about the world, I would be in an awkward position personally. If I wrote up what I found I would be in some highly unsavory company. Instead of being known as someone who writes about a range of things of varying levels of seriousness and applicability, I would quickly become primarily known as one of those dragon advocates. Given the taboos around dragon-belief, I could face strong professional and social consequences.

Religious belief is not nearly as taboo as what this paragraph describes, but the claim "[ethnicity] are genetically predisposed to [negative psychological trait]" is.

A general commitment to seeking truth doesn't obligate one to investigate every possible question! I can be quite committed to seeking truth in some areas, while intentionally avoiding quite unrelated ones.

One certainly shouldn't claim to be truth seeking in areas where one is are intentionally agnostic, but that's part of why I'm writing this post: so I can later link it to explain why I have chosen not to engage with some question in some area.

Example chosen because (a) I have really, absolutely no idea what the optimal action is here; (b) I have reason to belive that the risk is kind of low, anyway. (c) most commentors here wont have a strong opinion, so we wont have a flame war over what the right answer might be. Let it stand in for other cases where it is very, very unclear what the optimal action is.

I don't think refocusing my main efforts on the alignment problem would make humanity safer.

Congrats, we've just solved alignment! The AI now wants to know whether we want our utopia optimized in a way that accounts for dragons or not.

See also Heads I Win, Tails?—Never Heard of Her; Or, Selective Reporting and the Tragedy of the Green Rationalists [LW · GW]. Which goes into how people who don't know that there's some implicit consensus not to talk about some things can come away confused and with wrong beliefs.

I'm pro being clear about what we don't discuss, but it's unreasonable to limit the list to three. The number of topics that is net negative to discuss is just a fact about the world and is probably over three, and I would rather not have people talk about the 4th worst controversial topic just because we uncover three even more pointless and controversial ones.

Politics also seems inadvisable to ban because it's too broad.

I think implementing this policy would require frequent publicization of what issues are on the current list and frequent debates and votes on whether to replace one topic on the list with another topic not yet on the list and would therefore have the opposite of the intended effect. Maybe there's a clever way to get around those issues, but I doubt it.