Making Rationality General-Interest

I think that this sounds like a great idea, though it also seems like one that would take a lot of effort to put together.

I think 'can we make everyone rational?' is probably the wrong question. Better questions:

1. How much more rational could we make 2013 average-IQ people, by modifying their cultural environment and education? (That is, without relying on things like surgical or genetic modification.) What's the realistic limit of improvement, and when would diminishing returns make investing in further education a waste?

2. How do specific rationality skills vary in teachability? Are there some skills that are especially easy to culturally transmit (i.e., 'make cool' in a behavior-modifying way) or to instill in ordinary people?

3. How hard would the above approaches be? How costly is the required research and execution?

4. In addition to the obvious direct benefits of being more rational (which by definition means 'people make decisions that get them more of what they want' and 'people's beliefs are better maps'), how big are indirect benefits like Qiaochu's 'smart people see rationality as more valuable', or 'governments and individuals fund altruism (including rationality training) more effectively', or 'purchasing and voting habits are more globally beneficial'?

Suppose we were having this discussion 200 or 500 or 1000 years ago instead, and the topic was not 'Can we make everyone rational?' but 'Can we make everyone literate?' or 'Can we make everyone a non-racist?' or 'Can we make everyone irreligious?'. I think it's clear in retrospect that those aren't quite the right questions to be asking, and it's also clear in retrospect that appeals to intelligence levels, as grounds for cynical skepticism, would have been very naïve.

At this point I don't think we have nearly enough data to know all the rationality skills IQ sets a hard limit on, or whether people at a given IQ level are anywhere near those limits. Given that uncertainty, we should think seriously about the virtues and dangers of a world where LW-level rationality is as common as, today, literacy or religious disengagement is.

Making rationality cool = an excellent starting point. I still disagree on the rationality-intelligence thing, though; I think you could teach skills that could still meaningfully be called epistemic/instrumental rationality to people with IQ 100 and below. Not everyone, anymore than it's possible to persuade everyone from childhood that it's a good idea to spend money sensibly. (Gaah this is a pet peeve for me). But enough to make the world more awesome.

I'm going to register that disagreement as a bet, and if in 10 years LW is still around and enough has happened that we know who's right, I will find this comment and collect/lose a Bayes point.

Make rationality cool? Don't worry, I got this.

Puts on sunglasses

I am highly skeptical of this happening with human psychology kept constant, basically because I think rationality is de facto impossible for humans who are not at least ~2 standard deviations smarter than the mean.

I don't think I agree but I may be interpreting "rationality" differently to you.

Treating "rationality" as a qualitative trait, so that people are simply either rational or irrational, I'd say no one is rational, regardless of IQ; no one meets the impossibly stringent standard of making their every inference and/or decision optimally.

Treating "rationality" as a quantitative trait, so that some people are simply more rational than others, I expect IQ helps cultivate rationality everywhere along the IQ scale (except maybe the extremes). I wouldn't expect a threshold effect around an IQ of 130, but a gradual increase in feasibility-of-being-rational as IQ goes up.

I agree with you on the jargon thing; it's so much easier to have a conversation about rationality-cluster with LW people because of it. (It's also fun and ingroupy). But I do think it's a problem overall, and partly avoidable.

We really should have a short phrase for that. Suggestions? "The evidence would be likely given the hypothesis, but the hypothesis isn't as likely given the evidence" would at least be a bit shorter.

There are already precise terms for most of the concepts LW discusses. It's that LW uses its own jargon.

I've been giving the following post list to friends, to get them into LW:

1) The Twelve Virtues

2) Cognitive Biases Potentially Affecting Judgements of Global Risk

3) The Simple Truth

4) The Useful Idea of Truth

5) What do we mean by rationality?

6) What is evidence?

7) Rationality: Appreciating Cognitive Algorithms

8) Skill: The Map is Not the Territory

9) Firewalling the Optimal from the Rational

10) The Lens that Sees its Flaws

11) The Martial Art of Rationality

12) No One Can Exempt You From Rationality's Laws

I'll occasionally adapt it, e.g. Swapping the first two posts around, for someone who has an academic background and will be more interested in an academic paper to start with.

Anyone looking for my current, extended version, can find it here.

I've been thinking about this problem lately, and I agree it's a problem. I have some tentative ideas for starting to address it, which I'll post to Discussion next week. I'd like more data on where the stumbling blocks are, though.

1. Are there LW posts (by Eliezer or whoever) that you have found helpful, readable, concise, etc.? If so, what are some of the better examples? Would you say, for example, that Lukeprog and Hanson's styles work for you about equally well?

2. What are some examples of specific posts (or series of posts) you haven't gotten through? How much was a result of length, how much a result of content (e.g., too difficult or boring or mathy), and how much a result of style (e.g., too narrative or unstructured or jargony)?

3. What are specific ideas, perspectives, approaches, or terms you feel (or have been told) you're currently missing out on? The more examples of this the better.

ETA: I'd be interested in others' responses to this too.

I don't think think HPMoR is required reading to learn rationality from LW and related places, and is one of the few things making rationality general-interest at this point.

I do agree that a short "getting started" guide would be helpful, though.

Only a minority of respondents to the 2012 survey had read “about 75%” or “nearly all” of the sequences. So long as you've read the links in the welcome thread and you're prepared to be corrected you should be fine.

Well, there aren't many things that I don't doubt a little bit. I don't think this a bad thing. However, in order to get anything done in life, instead of sitting in my room thinking about how much I don't know, I have to act on a lot of things that I'm a bit doubtful about.

This article points out a pretty important obstruction to the general spread of rationality:

So what were the key differences [between the slow spread of antiseptics and the fast spread of anesthesia]? First, one combatted a visible and immediate problem (pain); the other combatted an invisible problem (germs) whose effects wouldn’t be manifest until well after the operation.

Rationality training does not combat a visible and immediate problem because people do not have a sense that more is possible along this dimension.

Another problem is that there isn't really any standard "rationality test" or other ability to actually determine how rational someone is, though some limited steps have been taken in that direction. Stanovich is working on one, but it can't be expected for 3+ years at this stage.

This obviously limits the extent to which we can determine whether rationalists "actually win" (my impression, incidentally, is that they do but that there are a lot of skills that help more than current "rationality training" for the average person), what forms of rationality practice yield the most benefits, and so on.

When it comes to raising the sanity waterline, I can't help but think that the intelligence issue is likely to be a paper tiger. In fact I think LessWrong as a whole cares far too much about unusually intelligent people and that this is one of the biggest flaws of the community as a general-interest project. However, I also recognize that multiple purposes are at work here and such goal conflict may be inevitable.

First off, one potential problem is the term "rationality" itself. MIRI found that the term "singularity" was too corrupted by other associations to be useful, so they changed their name to avoid being associated with this. I believe that "rational" may be similarly corrupted ("logical" certainly is) and finding another term altogether might be a good tactic.

I think the idea of posting multiple comments is good, as long as none of the comments is even a little bit a prerequisite for the others. I personally don't think it's worth voting on the idea. (Just try it out for a while and see whether you like it and whether you get any complaints.) I suggest posting the separate comments at the base level so they'll be in their proper karmic order as independent posts; otherwise you lose most of the value of this approach, and you'll be testing a different idea than the one you intend to ultimately implement.

No no no. Not at all. I was obviously less rational as a baby than I am now. But childhood neuroplasticity is a thing; it's easier to learn languages before age 10, and preferably before age 5. And kids have time. As a kid, when I did competitive swimming,I used to be in the pool >10 hours a week. Now, as an adult, I do taekwondo, and although there are 10 hours of class a week available, I only make 2-3.

I did learn some maladaptive thought patterns: i.e. my social anxiety spiral around "you just don't have enough natural talent to do X", and the kicker, "you aren't good enough." I know this is a pretty meaningless phrase, but it has emotional power because it's been around so long.

My weak definition of rationality: thinking about your own knowledge and beliefs from an outside perspective and updating/changing them if they are not helpful and don't make sense (epistemic); noticing your goals, thinking a bit about how to achieve them, and then doing that on purpose to see if works, while paying attention if it's not working so you can try something else; thinking about and trying to notice the actual consequences of your actions (instrumental).

Short: epistemic=believing things on purpose, instrumental=doing things on purpose for thought-out reasons.

I say weak because this isn't a superpower; you can do it without being amazingly good at that (i.e. if you have an IQ of 90). But you can exercise without being amazingly good at any sport, and you still benefit from it. I think that also stands for basic rationality.

if a person's behavior is seemingly irrational to you, there's still some reason, maybe understood, maybe not understood, maybe misunderstood why the person behaves that way.

In a general sense, yeah. People operate inside causality. But people do things for a reason that they haven't noticed, haven't thought about, and might not agree with if they did think about. For example, Bob might find himself well on the path to alcoholism without realizing that his original, harmless-and-normal-seeming craving for a drink in the evening happened because it helped with his insomnia; a problem that could more healthily be addressed by booking a doctor's appointment. (I pick this example because I recently caught myself in the early stages of this process). But from the inside, it doesn't feel like the brain is fallible, and so even people who've come across research to the contrary feel like their introspection is always correct–let alone people who've never seen those ideas. I don't think the IQ ceiling on understanding and benefiting from "I might be wrong about why I do this" is very high.

Rationality decomposes into instrumental rationality ('winning', or effectiveness; reliably achieving one's goals) and epistemic rationality (accuracy; reliably forming true beliefs).

How do you understand 'utilitarianism'? What are the things about it that you think are unimportant or counterproductive for systematic rationality? (I'll hold off on asking about what things make utilitarianism unpopular or difficult to market, for the moment.)

Instead of down voting maybe we should instead be asking what within the LW community caused Peterdjones to say that?