One thing I don't see explored enough, and which could possibly bridge the gap between Rationality and Winning, is Rationality for Dummies.

Rationalist community is oversaturated with academic nerds, borderline geniuses, actual geniuses, and STEM people who's intellectual level and knowledge base is borderline transhuman.

In order for Rationality and Winning to be reconciled with minimum loss, we need a bare-bones, simplified, kindergarten-level Rationality lessons based on the simplest, most relatable real life examples. We need Rationality for Dummies. We need Explain Methods Like Im Five, that would actually work for actual 5-year olds.

True, Objective Rationality Methods should be applicable whether you are an AI researcher with a phd, or someone to young/stupid to tie their own shoes. Sufficiently advanced knowledge and IQ can just brute-force winning solutions despite irrationality. it would be more enlightening if we equipped a child/village idiot with simple Methods and judge their successes on this metric alone, while they lack intellectual capacity or theoretical knowledge, and thus need to achieve winning by a step-by-step application of the Methods, rather than jumps of intuition resulting from unconscious knowledge and cranial processing power.

Only once we have solid Methods of Rationality that we can teach to kids from toddler-age, and expand on them until they are Rational Adults, then we can say for certain which Rationalist ideas lead to Winning and which do not.

I find this refreshing. It rings true. It feels like the kind of North Star we were groping toward in early CFAR but never landed on.

This in particular feels clarifying:

Rationality is the study (and applied skill) of finding cognitive algorithms that form better beliefs and make better decisions. Sometimes this is the appropriate tool for the job, and sometimes it's not. 

I find myself breathing with relief reading this. It has the flavor of a definition that could use some iterations. But as it stands, it strikes me as (a) honoring the spirit of the discipline while (b) not overblowing what it can do or is for. Like part of the clarity is in what the Art isn't for.

Thank you for writing this.

I feel strongly about this part:

How to Measure Anything notably won't help you (much) with aligning AI. It teaches "good decisionmaking", but doesn't teach "research taste in novel domains". I think there's a concrete branch of rationality training that'd be relevant for novel research, that requires pretty different feedbackloops from the "generally be successful at life" style of training. I think some of "research taste rationality" is reasonably alive in academia, but many elements are not, or are not emphasized enough.

I want to keep pushing for people to disambiguate for what precisely they mean when they use the word "rationality". It seems to me that there are a bunch separate project that plausibly, but not necessarily, overlap, which have been lumped together under a common term, which causes people to overestimate how much they do overlap.

In particular, “effective decision-making in the real world”, “how to make progress on natural philosophy when you don’t have traction on the problem” are much more different than one might think from reading the sequences (which talks about both in the same breath, and under the same label). 

Problems where "rational choice under uncertainty" / necessarily, problems where you already have a frame to operate in. If nothing else, you have your decision theory and probability theory frame.

Making progress on research questions about which you are extremely confused is mostly a problem of finding and iterating on a frame for the problem. 

And the project of "raising the sanity waterline" and of "evidence-based self-help", are different still. 

True (Scottish) Rationality is winning. Firstly, whom do we call a rational agent in Economics? It is a utility optimiser given constraints. An agent that makes optimal choices and attains max utility possible, that's basically winning. But real life is more complicated, all agents are computationally constrained and "best case" is not only practically unattainable, it is uncomputable, so we cannot even compare against it. So we talk about doing "better than normally expected", "better than others", etc. When I say "winning" I mean achieving ambitious goals.

But achieving ambitious goals in real life is mainly not about calculating the optimal choices! It is mainly about character, integrity, execution, leadership and a lot of other stuff! How come I still claim that Rationality is Winning? What use is knowing what to do if in practice you don't do it? Well, that's the point! An "optimal" strategy that is infeasible is not optimal)

But why focus on rationality at all if other stuff is more important? Because, well, your character, integrity, execution, resources, etc are not under your direct control except via the decisions that you make. You get them by making rational decisions. Making decisions that make you win and achieve your (ambitious) goals is what I call rationality.

"Sometimes the way you win is by copying what your neighbours are doing, and working hard." And in this case behaving rationally is copying what your neighbours do and working hard. Doing anything else is irrational, IMHO. Figuring out whom, when and how to copy is a huge part of rationality! We also call it critical thinking. Knowing how and when to work hard is another one! Why do you exclude one of the most important cognitive algorithms "sifting out the good from the bad" from "the study (and applied skill) of finding cognitive algorithms that form better beliefs and make better decisions"? If you are not good at critical thinking, how do you know that LW is not complete bullshit?

"Developing rationality" as a goal is an awful one because you don't get feedback. Learning doesn't happen without feedback. "Winning" may be a great one IIF you pick fields with strong immediate practical unbiased feedback. For example, try playing poker, data science in an executive position, all kinds of trading (RTB, crypto, systematic, HFT, just taking advantage of the opportunities when they present themselves), doing research just for the purpose of figuring out stuff for yourself because your life & money (universal resource) depends on it (or because it's fun and later your life will depends on it :) ). These are all examples from my life and they worked wonders for me.

I am sorry if I am coming a little aggressive, I think this is a great post raising a great point. I am just a rude post-USSR trader and I believe that being direct and up to the point is the best way to communicate and to show respect)

I never had an opportunity to participate in CFAR workshops and that's a pity. I would be happy to discuss this stuff further because I think both sides have useful stuff to share.
 

Physics is not particularly about getting to the Moon, there are many relevant activities of very different flavors, most of them not wielding physical law for personal benefit of the practitioner. There's even theory that doesn't have the slightest notion of applicability in the real world, and yet it's legitimately about physics.

Rationality is the study (and applied skill) of finding cognitive algorithms that form better beliefs and make better decisions. Sometimes this is the appropriate tool for the job, and sometimes it’s not.

If “the study (and applied skill) of finding cognitive algorithms that form better beliefs and make better decisions” is not the appropriate tool for the job (whatever “the job” might be), what exactly do you call the skill of (a) realizing this (or, more generally, determining when something is or is not the right tool for a job), (b) finding the right tool for the job, and (c) using that tool to do the job successfully?

There’s one obvious word we can use for this…

If the title doesn't mean "A != B", then I don't know what it's supposed to mean instead. Can you edit the actual explanation of the title into the top of this essay?

Personally, I am strongly against this, 

I got into Rationality for a purpose if it is not the best way to get me to that purpose [i.e. not winning] then Rationality should be casted down and the alternative embraced. 

On the other hand, I suspect we mostly agree with our disagreement is on the definition of the word "winning" 

I could have failed reading comprehension but I did not see "winning" defined anywhere in the post

A few thoughts on this.

This posted reminded me of Eliezer's take against toolbox style thinking [LW · GW]. In particular, it reminded me of the tension within the rationality community between folks who see rationality has the one thing you need for everything and folks who see it as an instrumentally useful thing to pull out in some circumstances.

The former folks form what we call the Core Rationalists. Rationality might not be literally everything, but it's the main thing, and they take an expansive view on the practice of rationality. If something helps them win, they see it as being part of rationality definitionally because it helps them win. This is also where the not-so-straw Vulcan LARPers hang out.

The latter group we might call the Instrumental Rationalists. They care about rationality to the extent it's useful. This includes lots of normal folks who got interested in rationality because it seemed like a useful tool but it's not really central to their identity the way it is for Core Rationalists. This is also the group where the Post/Meta-Rationalists hang out, who can think of as Core Rationalists who realized they should treat rationality as one of many tools and seek to combine it with other things to have a bigger toolbox to use to help them win.

Disagreements between these two groups show up all the time. They often play out in the comments sections of the forum when someone posts something that really gets at the heart of what rationality is. I'm thinking about posts from @[DEACTIVATED] Duncan Sabien [LW · GW], comments from @Said Achmiz [LW · GW], whatever @Zack_M_Davis [LW · GW]'s latest thing is, and of course some of my own posts and comments.

Perhaps this disagreement will persist because there's not really a resolution to it. The difference between these groups is not object level rationality, but how to relate rationality. And both of these groups can be part of the rationality movement even if they sometimes piss each other off because they at least agree on one thing: rationality is really useful.

My two cents is that rationality is not about being systematically correct, it's about being systematically less wrong. If there is some method you know of that is systematically less wrong than you and you're skilled enough to apply it, you're being irrational. There are some things you just can't predict, but when you can predict them, rationality is the art of choosing to do so.

As I see it, "self-help" is short-term/object-level, and "rationality" is long-term/meta-level.

If you want to improve your life, here and now, there is a lot of good specific advice you can follow.

Rationality is the level beyond that. If you get contradictory self-help advice, how will you choose? If you have already fixed the obvious mistakes, and you follow the standard good advice, what next?

There are two kinds of mistakes you want to avoid. One is focusing on insight porn, and neglecting your real life. The other is following the practical advice that seems good, and then stumbling upon advice that is actually really bad, and following it blindly off the cliff.

So I think the optimal approach would start with the specific good advice, but also keep explaining why.

Strong agree: I said more or less the same thing a few months ago:  A note on “instrumental rationality”

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

So basically, post hoc, ergo propter hoc (post hoc fallacy)

If winning happened after rationality (in this case, any action you judge to be rational under any definition you prefer) it does not mean it happened because of it.

For me the obvious problem with "rationality is winning" as a soundbite is that the figure of the "winner" in our culture is defined in a frankly toxic way and using that word obfuscates the huge asymmetries at play. Terminal values are still a thing; rationality is about pursuing one's own goals as best as possible. That doesn't necessarily mean winning. If you're an oil CEO with the goal of making a buttload of money and screw everything else and I'm an environmentalist genuinely preoccupied with saving the Earth from climate change, even my most rational approach is climbing up a very steep hill while you can just fund some crank to spread comforting bad science, bribe a few politicians and call it a day.

While I completely agree in the abstract, I think there's a very strong tendency for systems-of-thought, such as propagated on this site, to become cult-like. There's a reason why people outside the bubble criticize LW for building a cult. They see small signs of it happening and also know/feel the general tendency for it, which always exists in auch a context and needs to be counteracted.

As you point out, the concrete ways of thinking propagated here aren't necessarily the best for all situations and it's another very deep can of worms to be able to tell which situations are which. Also, it attracts people (such as myself to some degree) who enjoy armchair philosophizing without actually ever trying to do anything useful with that. Akrasia is one thing, not even expecting to do anything useful with some knowledge and pursuing it as a kind of entertainment is another still.

So there's two ways to frame the message: one is saying that "rationality is about winning", which is a definition that's very hard to attack but also vague in it's immediate and indisputable consequences for how one should think, and also makes it hard to tell if "one is doing it right".

The other way is to impose some more concrete principles and risk them becoming simplified, ritualized, abused and distorted to a point where they might do net harm. This way also makes it impossible to develop the epistemology further. You pick some meta-level and propose rules for thinking at that level which people eventually and inevitably propagate and defend with the fervor of religious belief. It becomes impossible to improve the epistemology at that point.

The meme ("rationality") has to be about something in order to spread and also needs some minimum amount of coherence. "It's about winning" seems to do this job quite well and not too well.

When it comes to rationality, the Black Swan Theory ( https://en.wikipedia.org/wiki/Black_swan_theory ) is an extremely useful test.

A truly rational paradigm should be built with anti-fragility in mind, especially towards Black Swan events which would challenge its axiomatic.

I think most people are missing the psychological effects of rationalism and irrationalism, and not seeing that this is an equation which depends on itself. Religion is irrational, but it might give you the psychological defense you need to cope with or overcome something. Rationality might harm your sense of wonder and inherent spiritual beliefs which are wrong but nonetheless helpful. So irrationality might be rational, and rationality might be irrational.

Irrationality is a sort of overfitting, into your life and society. But why should you be correct in a manner which is so general that you're not specialized to life, society, mental well-being and winning?