rationality

·

Rationality is the art of thinking in ways that result in accurate beliefs [? · GW] and good decisions [? · GW]. It is the primary topic of LessWrong.

Rationality is not only about avoiding the vices of self-deception [? · GW] and obfuscation (the failure to communicate clearly [? · GW]), but also about the virtue of curiosity [? · GW], seeing the world more clearly than before, and achieving things [? · GW] previously unreachable [? · GW] to you [? · GW]. The study of rationality on LessWrong includes a theoretical understanding of ideal cognitive algorithms, as well as building a practice that uses these idealized algorithms to inform heuristics [? · GW], habits [? · GW], and techniques [? · GW], to successfully reason and make decisions in the real world.

Topics covered in rationality include (but are not limited to): normative and theoretical explorations of ideal [? · GW] reasoning [? · GW]; the capabilities and limitations [? · GW] of our brain [? · GW], mind and psychology [? · GW]; applied advice such as introspection [? · GW] techniques and how to achieve truth collaboratively [? · GW]; practical techniques and methodologies for figuring out what’s true ranging from rough quantitative modeling to full research guides.

Note that content about how the world is can be found under World Modeling [? · GW], and practical advice about how to change the world is categorized under World Optimization [? · GW] or Practical.


Theory / Concepts

Anticipated Experiences [? · GW]
Aumann's Agreement Theorem [? · GW]
Bayes Theorem [? · GW]
Bounded Rationality [? · GW]
Conservation of Expected [? · GW]
Contrarianism [? · GW]
Decision Theory [? · GW]
Epistemology [? · GW]
Game Theory [? · GW]
Gears-Level [? · GW]
Hansonian Pre-Rationality [? · GW]
Law-Thinking [? · GW]
Map and Territory [? · GW]
Newcomb's Problem [? · GW]
Occam's razor [? · GW]
Robust Agents [? · GW]
Solomonoff Induction [? · GW]
Truth, Semantics, & Meaning [? · GW]
Utility Functions [? · GW]
 

Applied Topics

Alief [? · GW]
Betting [? · GW]
Cached Thoughts [? · GW]
Calibration [? · GW]
Dark Arts [? · GW]
Empiricism [? · GW]
Epistemic Modesty [? · GW]
Forecasting & Prediction [? · GW]
Group Rationality [? · GW]
Identity [? · GW]
Inside/Outside View [? · GW]
Introspection [? · GW]
Intuition [? · GW]
Practice & Philosophy of Science [? · GW]
Scholarship & Learning [? · GW]
Taking Ideas Seriously [? · GW]
Value of Information [? · GW]
 

Failure Modes

Affect Heuristic [? · GW]
Bucket Errors [? · GW]
Compartmentalization [? · GW]
Confirmation Bias [? · GW]
Fallacies [? · GW]
Goodhart’s Law [? · GW]
Groupthink [? · GW]
Heuristics and Biases [? · GW]
Mind Projection Fallacy [? · GW]
Motivated Reasoning [? · GW]
Pica [? · GW]
Pitfalls of Rationality [? · GW]
Rationalization [? · GW
Self-Deception [? · GW]
Sunk-Cost Fallacy [? · GW]

Communication

Common Knowledge [? · GW]
Conversation [? · GW]
Decoupling vs Contextualizing [? · GW]
Disagreement [? · GW]
Distillation & Pedagogy [? · GW]
Double-Crux [? · GW]
Good Explanations (Advice) [? · GW]
Ideological Turing Tests [? · GW]
Inferential Distance [? · GW]
Information Cascades [? · GW]
Memetic Immune System [? · GW]
Philosophy of Language [? · GW]
Steelmanning [? · GW]

Techniques

Double-Crux [? · GW]
Focusing [? · GW]
Goal Factoring [? · GW]
Internal Double Crux [? · GW]
Hamming Questions [? · GW]
Noticing [? · GW]
Techniques [? · GW]
Trigger Action Planning/Patterns [? · GW]

Models of the Mind

Consciousness [? · GW]
Dual Process Theory (System 1 & 2)
General Intelligence [? · GW]
Subagents [? · GW]
Predictive Processing [? · GW]
Perceptual Control Theory [? · GW]
Zombies [? · GW]
 

Other

Center for Applied Rationality [? · GW]
Curiosity [? · GW]
Rationality Quotes [? · GW]
Updated Beliefs (examples of) [? · GW]

This list is not comprehensive! The tagging system is new. Many needed tags have not been created and/or added to the above list.


What we're calling "rationality"

A good heuristic is that rationality is about cognitive algorithms. Rather than being a synonym for true or optimal, the term rational should be reserved for describing whether or not a cognitive algorithm results in true beliefs and optimal actions.

This is distinct from practical advice [? · GW], such as how to improve relationships or implement productivity systems, which should not be considered "rationality" per se. Some have pushed against labeling self-help as "rational dating", etc. for reasons along these lines [1 [LW · GW], 2 [LW · GW]], and they are probably correct.

In accordance with this, LessWrong classifies most self-help type advice under the World Optimization [? · GW] tag and not the Rationality tag.

Similarly, most object-level material about how the world is, e.g. math, biology, history, etc. is tagged under World Modeling [? · GW] tag, with exceptions for neuroscience and probability theory, etc., which have concrete consequences for how one ought to think.

Heuristics and Biases

Early material on LessWrong frequently describes rationality with reference to heuristics and biases [1, [LW · GW] 2 [LW · GW]]. Indeed, LessWrong grew out of the blog Overcoming Bias and even Rationality: A-Z [? · GW] opens with a discussion of biases [1 [? · GW]] with the opening chapter titled Predictably Wrong [? · GW]. The idea is that human mind has been shown to systematically make certain errors of reasoning, like confirmation bias. Rationality then consists of overcoming these biases.

Apart from the issue of the replication crises which discredited many examples of bias that were commonly referenced on LessWrong, e.g. priming, the "overcoming biases" frame of rationality is too limited. Rationality requires the development of many positive skills, not just removing negative biases to reveal underlying perfect reasoning. These are skills such as how to update the correct amount in response to evidence, how to resolve disagreements with others, how to introspect, and many more.

Instrumental vs Epistemic Rationality

Classically, on LessWrong, a distinction has been made between instrumental rationality and epistemic rationality, however, these terms may be misleading – it's not as though epistemic rationality can be traded off for gains in instrumental rationality. Only apparently, and to think one should do this is a trap.

Instrumental rationality is defined as being concerned with achieving goals. More specifically, instrumental rationality is the art of choosing and implementing actions that steer the future toward outcomes ranked higher in one's preferences. Said preferences are not limited to 'selfish' preferences or unshared values; they include anything one cares about.

Epistemic rationality is defined as the part of rationality which involves achieving accurate beliefs about the world. It involves updating on receiving new evidence, mitigating cognitive biases, and examining why you believe what you believe. It can be seen as a form of instrumental rationality in which knowledge and truth are goals in themselves, whereas in other forms of instrumental rationality, knowledge and truth are only potential aids to achieving goals. Someone practicing instrumental rationality might even find falsehood useful.

The Art and Science of Rationality

In a field like biology, we can draw a distinction between the science of biology, which involves various theories and empirical data about biological life, and the art of being a biologist, which is the specific way that a biologist thinks and plays with ideas and interacts to the world around them. Similarly, rationality is both a science and an art. There’s study of the iron-clad laws of reasoning and mechanics of the human mind, but there’s also the general training to be the kind of person who reasons well.

Rationalist

The term rationalist as a description of people is used in a couple of ways. It can refer to someone who endeavors to think better and implement as much rationality as they can. Many prefer the term aspiring rationalist to convey that the identifier is a claim to the goal of being more rational rather than a claim of having attained it already.

Perhaps more commonly, rationalist is used to refer culturally to someone associated with various rationalist communities separate from their efforts to improve their rationality.