Truth: It's Not That Great

2009: "Extreme Rationality: It's Not That Great"

2010: "Self-Improvement or Shiny Distraction: Why Less Wrong is anti-Instrumental Rationality"

2013: "How about testing our ideas?"

2014: "Truth: It's Not That Great"

2015: "Meta-Countersignaling Equilibria Drift: Can We Accelerate It?"

2016: "In Defense Of Putting Babies In Wood Chippers"

I read the "heretical" statements as talking about truth replacing falsehood. I read the non-heretical statements as talking about truth replacing ignorance. If you reword the "truth" statements to make it clear that the alternative is not falsehood, they would sound much less heretical to me.

This is an example of why I suspect "effective altruism" may be better branding for a movement than "rationalism".

Huh? What? Wait a moment....

These two are entirely different things. Under the local definitions, rationalism is making sure the map looks like the territory and doing stuff which will actually advance your goals. Notably, rationalism is silent about values -- it's perfectly possibly to be a rational Nazi. You can crudely define rationalism as "being grounded in reality".

Altruism, on the other hand, is all about values. A very specific set of values.

You can't "rebrand" a movement that way -- what you imply is a wholesale substitution of one movement with another.

I was not signaling. Making it a footnote instead of just editing it outright was signaling. Revering truth, and stating that I do so, was not.

Now that I've introspected some more, I notice that my inclination to prioritize the accuracy of information I attend to above its competing features comes from the slow accumulation of evidence that excellent practical epistemology is the the strongest possible foundation for instrumental success. To be perfectly honest, deep down, my motivation has been "I see people around me succeeding by these means where I have failed, and I want to be like them".

I have long been more viscerally motivated by things that are interesting or beautiful than by things that correspond to the territory. So it's not too surprising that toward the beginning of my rationality training, I went through a long period of being so enamored with a-veridical instrumental techniques that I double-thought myself into believing accuracy was not so great.

But I was wrong, you see. Having accurate beliefs is a ridiculously convergent incentive, so whatever my goal structure, it was only a matter of time before I'd recognize that. Every utility function that involves interaction with the territory--interaction of just about any kind!--benefits from a sound map. Even if "beauty" is a terminal value, "being viscerally motivated to increase your ability to make predictions that lead to greater beauty" increases your odds of success.

Recognizing only abstractly that map-territory correspondence is useful does not produce the same results. Cultivating a deep dedication to ensuring every motion precisely engages reality with unfailing authenticity prevents real-world mistakes that noting the utility of information, just sort of in passing, will miss.

For some people, dedication to epistemic rationality may most effectively manifest as excitement or simply diligence. For me, it is reverence. Reverence works in my psychology better than anything else. So I revere the truth. Not for the sake of the people watching me do so, but for the sake of accomplishing whatever it is I happen to want to accomplish.

"Being truth-seeking" does not mean "wanting to know ALL THE THINGS". It means exhibiting patters of thought and behavior that consistently increase calibration. I daresay that is, in fact, necessary for being well-calibrated.

Truth-telling seems clearly overrated (by people on Less Wrong but also pretty much everyone else). Truth-telling (by which I mean not just not-lying but going out of your way and sacrificing your mood, reputation or pleasant socializing just to say something true) is largely indistinguishable from "repeating things you heard once to signal how smart or brave or good you are. "

Truth-seeking as in observing and doing experiments to discover the structure of the universe and our society still seems incredibly important (modulo the fact that obviously there are all sorts of truths that aren't actually significant). And I actually think that is true even if you call it information gathering, though 'information gathering' is certainly vastly less poetic and lacks the affective valence of Truth.

There's simply more territory than our maps can encompass. If you want to get anything done, there comes a point when you have to act on information that's not ideally complete. What's more, you aren't going to have complete information about when to stop searching.

Any recommendations for discussions of this problem?

"Effective altruism" is at risk of turning into a signal, though perhaps not quite as quickly are "rationality" or "truth-seeking".

"Corrupted hardware"-type arguments can suggest you should doubt your own justifications for deceiving others.

You should even more doubt your motivations for deceiving yourself.

Edit: For the following clicking agree is supposed to mean that you consider a statement heretical

"Some truths don't matter much." sounds heretical [pollid:691]

"People often have legitimate reasons for not wanting others to have certain truths." sounds heretical [pollid:692]

"The value of truth often has to be weighed against other goals." sounds heretical [pollid:693]

"Information can be perfectly accurate and also worthless." sounds heretical [pollid:694]

"People often have legitimate reasons for not wanting other people to gain access to their private information. " sounds heretical [pollid:695]

"A desire for more information often has to be weighed against other goals." sounds heretical [pollid:696]

i'm into epistemic rationality, but this all seems pretty much accurate and stuff

not sure what to conclude from having that reaction to this post.

Seems fairly uncontroversial to me, but that's likely because it stays far-mode. If you get specific and near-mode, I suspect you'll stir up some disagreement. Leave aside which beliefs you'd rather other people have or not have - that's a separate dark arts topic. For your own goal-achievement-ability, which true beliefs are you better off not having?

I completely agree that I have limited resources and need to prioritize which beliefs are important enough to spend resources on. I far less agree that true beliefs (in the paying rent sense of the word, those which have correct conditional probability assignments to your potential actions) ever have negative value.

Good stuff. It took me quite a long time to work these ideas out for myself. There are also situations in which it can be beneficial to let somewhat obvious non-truths continue existing.

Example: your boss is good at doing something but his theoretical explanation for why it works is nonsense. Most of the time questioning the theory is only likely to piss them off, and unless you can replace it with something better, keeping your mouth shut is probably the safest option.

Relevant post:

http://cognitiveengineer.blogspot.com/2013/06/when-truth-isnt-enough.html

Truth is really important sometimes, but so far I've been bad about identifying when.

I know a fair bit about cognitive biases and ideal probabilistic reasoning, and I'm pretty good at applying it to scientific papers that I read or that people link through Facebook. But these applications are usually not important.

But, when it comes to my schoolwork and personal relationships, I commit the planning fallacy routinely, and make bad predictions against base rates. And I spend no time analyzing these kinds of mistakes or applying what I know about biases and probability theory.

If I really operationalized my belief that only some truths are important, I'd prioritize truths and apply my rationality knowledge to the top priorities. That would be awesome.

I am curious; what is the general LessWrong philosophy about what truth "is"? Personally I so far lean towards accepting an operational subjective Bayesian definition, i.e. the truth of a statement is defined only so far as we agree on some (in principle) operational procedure for determining its truth; that is we have to agree on what observations make it true or false.

For example "it will rain in Melbourne tomorrow" is true if we see it raining in Melbourne tomorrow (trivial, but also means that the truth of the statement doesn't depend on rain being "real", or just a construction of Descartes' evil demon or the matrix, or a dream, or even a hallucination). It is also a bit disturbing because the truth of "the local speed of light is a constant in all reference frames" can never be determined in such a way. We could go to something like Popper's truthlikeness, but then standard Bayesianism gets very confusing, since we then have to worry about the probability that a statement has a certain level of "truthlikeness", which is a little mysterious. Truthlikeness is nice in how it relates to the map-territory analogy though.

I am inclined to think that standard Bayesian style statements about operationally-defined things based on our "maps" makes sense, i.e. "If I go and measure how long it takes light to travel from the Earth to Mars, the result will be proportional to c" (with this being influenced by the abstraction that is general relativity), but it still remains unclear to me precisely what this means, in terms of Bayes theorem that is: i.e. the probability P("measure c" | "general relativity") implies that P("general relativity") makes sense somehow, though the operational criteria cannot be where its meaning comes from. In addition we must somehow account for that fact "general relativity" is strictly False, in the "all models are wrong" sense, so we need to somehow rejig that proposition into something that might actually be true, since it makes no sense to condition our beliefs on things we know to be false.

I suppose we might be able to imagine some kind of super-representation theorem, in the style of de-Finetti, in which we show that degrees of belief in operational statements can be represented as the model average of the predictions from all computable theories, hoping to provide an operational basis for Solomonoff induction, but actually I am still not 100% sure what de-Finetti's usual representation theorem really means. We can behave "as if" we had degrees of belief in these models weighted by some prior? Huh? Does this mean we don't really have such degrees of belief in models but they are a convenient fiction? I am very unclear on the interpretation here.

The map-territory analogy does seem correct to me, but I find it hard to reconstruct ordinary Bayesian-style statements via this kind of thinking...

Truth, having a one-to-one correspondence between the map and the territory[1], is only useful if you're able to accurately navigate an accurate map.

However, if, when navigating an accurate map, you still veer to the left when trying to reach your destination, you're faced with two choices: 1) Un-value truth, and use whatever map gets you to your destination no matter the relation between the "map" and the territory. 2) Terminally value truth, damn the disutility of doing so!

[1] For convenience, I assume that the territory exists. (For some definitions of existence.)

Human brains do experience in-group reinforcement, so we ought to aim that reinforcement at something like truth-seeking, which tends to encourage meta-level discussions like this one, thus helping to immunize us against death spirals. Note that this requires additional techniques of rationality to be effective. Consider that some truths--like knowing about biases--will hurt most people.

But that's just your opinion.

The claims about .truth mostly looked ambiguous to me . There are differences between truth-telling ,truth-preferring and truth-seeking...which can pull in different directions. "Know all, say nowt"

Truth is completely and utterly worthless, except for its being instrumentally useful for every single thing ever.

THAT IS THE TRUTH OF THIS WORLD! SUBMIT TO THAT TRUTH, YOU PIGS IN HUMAN CLOTHING!

(There, threw in some pointless signalling.)

Your post seem to still assume that "rationalism" actually has something to do with seeking truth, rather than with seeking, for example, self congratulation about truth.

To give a concrete example, there's a lot of truths to learn in physics.

A lot of those truths are in the physics textbooks and the rest require very hard work to figure out. So, when someone's interested in knowing truths that have to do with physics, they learn the math, they study physics, they end up able to answer questions from the homework section of a textbook, and sometimes, even able to answer the unanswered questions. You get a fairly reasonable island of knowledge, not a protruding rock of "MWI is true" in the vast sea of near total ignorance.