Teachable Rationality Skills

Subskill: Avoid pitfalls of verbal strategic reasoning.

• List consequences without shifting from intuitive-sum to verbal-justification mode.
• Don't exacerbate scope insensitivity or attending to rare events.

There are studies showing that people who consider their decisions more make worse decisions. As I understand it, the main explanation for this is that people shift from an intuitive sum of costs and benefits, to seeking verbally justifiable decisions, which in turn might lead them to one-reason-decisionmaking, ignoring some of their costs and benefits which are important to them but seem less "sensible". I also suspect it may exacerbate other biases like scope insensitivity or rare events - thinking about cases which are rare or short in duration.

The classic case being "Let's get a bigger house, further away from work, so it has an extra bedroom in case Grandma comes over", which she does once a year, but the 20 minutes of extra commute time happen every day and are not acclimated-to.

Subskill: Unbundling; optimize separate things separately.

Example 1: Optimize fuzzies and utilons separately.

Example 2: Optimize grades and learning separately (instead of just optimizing grades, or haphazardly optimizing both at the same time).

5SL: Notice when you're optimizing two things at once. (Maybe because you (a) have a sense of awkwardness or of not getting enough done, and when you list out the desirable consequences, there's more than one?) Then, find two different things you can do which optimizes each one individually and without worrying about the other one.

Subskill: Maximize on big things, satisfice on small things.

Subskill: Notice foreign goals. Are your parents making you do it? Are you doing it because you read it in a book? Did you just drift into doing that?

Converse subskill: Notice feeling of actually caring about something. Notice whether that caring is giving energy to what you're doing. Notice its absence.

Subskill: Detach from sunk costs.

Material for exercises: Requires a case where the person has sunk costs and the option of continuing or not continuing. Might want to choose cases slightly less fraught than a marriage or a PhD program - you want to work up to those gradually.

Subskill: Use fungibility. There are different ways to achieve many goals. E.g. instead of wasting an evening with a relative in a way you resent, send them a postcard.

Subskill: Don't express emotions as policy decisions. (Anna.) Find some other outlet for the emotion besides the policy decision, i.e., screaming. (Eliezer.)

Subskill: Chain from feelings of angst and frustration into saying, "I need to be strategic!" and the other skills listed.

Subskill: Before the final moment of doing something that has any sort of cost or downside, ask whether you're doing it because of its consequence or merely because you previously decided to do it.

Anna's subskill list from "Humans Are Not Automatically Strategic":

• Ask ourselves what we’re trying to achieve;
• Ask ourselves how we could tell if we achieved it (“what does it look like to be a good comedian?”) and how we can track progress;
• Find ourselves strongly, intrinsically curious about information that would help us achieve our goal;
• Gather that information (e.g., by asking as how folks commonly achieve our goal, or similar goals, or by tallying which strategies have and haven’t worked for us in the past);
• Systematically test many different conjectures for how to achieve the goals, including methods that aren’t habitual for us, while tracking which ones do and don’t work;
• Focus most of the energy that isn’t going into systematic exploration, on the methods that work best;
• Make sure that our "goal" is really our goal, that we coherently want it and are not constrained by fears or by uncertainty as to whether it is worth the effort, and that we have thought through any questions and decisions in advance so they won't continually sap our energies;
• Use environmental cues and social contexts to bolster our motivation, so we can keep working effectively in the face of intermittent frustrations, or temptations based in hyperbolic discounting.

Subskill: Musashi's "cut through in the same motion".

Subskill: "What is the consequence, what is the goal?"

Avoid saying anything unless they are certain they are correct.

I don't think this is a pernicious behavior at all. I suspect that this is actually a sign of rationality.

http://www.paulgraham.com/heroes.html (See Robert Morris)

To some extent, those are also signs of someone low on the social hierarchy (or someone who feels low on the social hierarchy), and a punitive culture that punishes individuals for brainstorming or otherwise being visibly wrong.

Subskill: Analyze the underlying reasons why you're trying to rationalize for or against something - why a conclusion feels required, or disallowed.

Important subskill: Notice when a candidate "the reason I'm trying to rationalize something" is a poor guess or itself a rationalization - when it's a guess that sounds plausible about someone like you in your position, but doesn't seem to ring true.

Subskill: Notice the process of selectively searching for support, and halt it. Chains into "Ask whether, not why", or into trying to unbind the need to rationalize.

Level 1, notice the active process after performing a conscious check whether it's running; Level 2, detect it perceptually and automatically; Level 3, never run this process.

Exercise material (prerequisite for multiple exercises below): Have a hot-topic list such that incoming students at the expected level (e.g. level = typical LW reader) would be tempted to rationalize at least some of them. This requires both that someone care about the topic, and that the topic isn't so cut-and-dry that there's no temptation to distort anything. E.g., I care about atheism but I don't have any emotional fear of that argument coming out "the wrong way" - on the other hand, putting me in an actual argument with, say, my parents, or someone who was a really clever theistic arguer in front of an audience, might generate the emotional temptation to cheat to ensure winning on every single point.

Subskill: Be able to identify the internal feeling of having a required conclusion, of an argument only having one allowed answer, and (a slightly different internal sensation) of other answers being disallowed.

Level one: After this skill is explicitly mentioned and invoked by some other process, be able to notice this internal sense of required-conclusion-ness (disallowed-ness) when you consciously focus on it.

Level two: Have a constant perceptual eye out for feelings like this, notice automatically without needing to be "on guard", chain into applying other anti-rationalization skills (e.g. Litany of Tarski).

What do you do instead, once you've stopped the process?

(For some reason negotiation in situations of extreme power imbalance seems like it should have a different name, and I don't know what that should be.)

Dominance or Authority spring to mind. In this video Steven Pinker argues that there are three basic relationship types, authority, reciprocity and communality, and negotiation in extreme power imbalance sounds like it uses the social rules for authority rather than reciprocity.

For some reason negotiation in situations of extreme power imbalance seems like it should have a different name, and I don't know what that should be.

I think that is called 'losing' or 'winning'. :)

Exercise: keep a physical stack trace on a whiteboard or something when talking

As topics become reframed for clarity, you can resolve child topics if they become irrelevant.

Exercise: verbalise linking concepts with emphasis/stress on key words.

This verbalisation can be either psychic (that is, using the internal dialogue) or vocal if involved in a discussion with a group. When moving from one concept to the next we find a way to express this as a sentence, and by emphasising key words in this sentence we highlight the concepts linked to by the sentence.

I use this sometimes when I want to remember a particular thought and I use the emphasised words as 'semantic tags' in my memory.

First, many intellectual conversations between people who lack this skill are are random walks through topic space. ... Stepping back to C or B or even A is frequently called for in important conversations but requires contextual mindfulness.

My impression is that most people are too good at this. People don't have enough random-walk conversations, feeling socially pressured to stay on-topic even when the initial topic was a random choice.

The link (above) to your site has gone offline... have you moved it somewhere? I'd really like to see what you've been doing!

It appears to me that awareness of what time it is, awareness of how long a task has taken, and ability to guess in advance how long a task will take are distinctly separate skills - it seems unwise to me to assume that progress at one of them will translate to increased ability with another.

This is based on the observation that I'm abysmally bad at the first of those (I've been known to gain and lose entire days, even when I'm paying attention in preparation for an upcoming event) and non-awesome at the second, but startlingly good at the last if I rely on the appropriate non-conscious estimate-generator. If I spontaneously say "I bet I can finish writing that piece of code in 20 minutes", based on my track record it would actually be surprising for it to take less than 15 or more than 25, but ask me to take a shower that lasts no longer than 20 minutes and there's something like a 50/50 chance I'll be in there for 45.

Suggested exercise: guess what time it is, then check a clock. Guess how long it's been since you last checked the clock, ie not only "it is 4:30" but also "it is 35 minutes since I last checked the time (at 3:55)"

I spent more than I care to admit learning to tell myself to sleep for some window of hours, like no more than 2 or 4 or whatever

Skills this might teach:

Speed, non-hesitation.

Originality.

Rapid adaptation to shifts of context.

Social confidence.

Some of the same things as acting lessons?

Skill: ability to better intuitively maintain conversational flow

Teamwork and group skills.

The "Yes and..." concept.

ETA: I put "Yes and" here, but I'm not entirely sure about it. I can explain the concept and I can see it's value in improv, but I can't think of any specific/concrete skills it assists.

If anyone wants an explanation of Yes And, I can provide one, and then you can evaluate its merits...

Subskill: Maintain awareness of things that would ordinarily zip right past.

Subskill / contributing talent: Maintain a focus of attention. (Exercise: Dual N-back.)

Subskill: Retrain habitual responses.

Subskill: Meditation.

Exercise: Meditation.

(Jasen's bailiwick.)

I doubt it's the most efficient method, but I've been running basic Bayesian math on things like "given the coin came up heads the last 20 times, should I assume it's a fair or weighted coin?" I figure learning to do the math quickly will help me get it down to the point where I can at least ballpark the math on the 5-second level. It's also been helping me ballpark priors, and observe how different priors can affect the math.

Exercise: Improvisatory dance. In my opinion, improvising is more useful than specific styles of dance (salsa, swing, waltz). Most people do not dance specific dances in common social interactions unless the social event is based around that dance. If you are at a club, you can pop and lock, b-boy, robot, liquid&digits, krump, while everyone around you does something else. Also, it's easier and more obvious to be better at improvisatory dance than the people around you.

I have found that attempting to teach others to dance in literal language doesn't work as well as using metaphorical, poetic, woo-filled language. That said, as a specific exercise: feel the energy in your torso and each of your limbs. Feel your connection to the earth beneath you-actually feel the sensation of your feet touching the ground-what parts are touching? The heel, balls, toes, pay attention to it specifically. Direct your focus and weight either towards or away from the parts of your body you find yourself noticing. Feel the energy in your limbs again, and let some of it out, to float in front of you: snap it out, or gently wave it, or pull or push or whatever your body intuits. Then move the now-floating ball of energy around, and let it move you around.

This is much easier to explain in person when you can see me doing it. I was originally inspired to dance by this TED talk by the Legion of Extraordinary Dancers, which is also where I got some of what I wrote above (the rest I got from my own experience and from the improvisation and choreography class I just took). If you enjoy this kind of dance, you will love the LXD web show

Exercise: Have two players negotiate to trade a good. One or both players will have private information about the good based on the role of a die.

The best active learning exercises to use in the classroom have an extremely simple set up (KISS), student interaction, an objective scoring system, and all the students having a chance to continually play rather than having to spend most of their time watching what other students do.

Skills this might teach:

Body language / vocal tone / facial expressions, awareness and control.

Awareness of social roles as you are carrying them out.

Social confidence, possibly performance in front of crowds (if you actually perform).

More skills:

Group-work skills: compromise, listening to other's ideas, shifting your own, recognizing when someone's is better than your's and when it isn't, recognizing when it's important to someone that their idea be used and when it isn't.

How to take criticism, and use it. (In four years of drama, I've never done a performance with more than a week prep time where I haven't gotten feedback that's made it better.)

Creativity, quick-thinking, improv.

For many more exercises exploring status behavior (both high and low), see Keith Johnstone's Impro. (Here's my review.) Johnstone's theory of improvisation (and acting in general) is that most of the weight of convincing the audience is carried by relative status distinctions among the actors. He provides a detailed set of exercises for exploring and understanding subtle and extreme differences so actors can be comfortable on stage projecting whatever distinction is called for.

Suggested exercise:

If you have a friend who is willing, it shouldn't be too hard to roleplay out using "high status" language. Just practice some dialogue and have them call you on it if you use language that's lower status.

Personally, I'd have them call you on it if you're rude too, but that's because, in my experience, politeness is a useful skill at all status levels.

And, of course, ideally you should be calling yourself on it too. Recording the conversation might help, so you can replay it and evaluate yourself afterwards.

This is hardly useful if one is no longer in a position to be able to go on dates. My fiancee would probably object to me asking older women out on dates, no matter how much I insist it's to train my rationality. What other exercises would train this without putting important relationships at risk (probably shouldn't practice on bosses, family members, etc.)?

I agree that it's good to learn to date if you haven't, both because the result can be good, and because it's a good proving ground for social competence.

I think it's good to not to roll over and accept unnecessarily low status - to not give it away.

It's good to know how to please and attract, to cow and impress.

[edit: yes, your story is clear]

Skill: Saying oops.

Today endofself found a flaw in a post of mine. I immediately ate a piece of chocolate, to reward my brain for being proved wrong.

Exercise: Nonconformity, (posibly fun), (possibly curiosity); watch or read from somehting VERY far out of your target demograpic.

Examples: Twilight, My Little Pony Friendship is magic, the Quran. (Note: I have only watched MLP of these examples, and can confirm it will surprise you by being great. For the other two I get an instinctive revulsion saying "that can't POSSIBLY be anything other than slow torture" in spite of knowing taking the outside view says otherwise, which is exactly why I should watch/read from them if I were to do this exercise)

Exercise: test understanding by filling in the blanks before you know the answer. I'm trying to get better at estimating large dollar amounts, so I do a lot of practice guessing. Also, talking research with a professor, I mentally made a point of predicting what his insight was going to be before he said it.

quick scan didn't see anything regarding accuracy of visual or kinesthetic imagination, probably one of the most important skills for solving problems and also related to # of possibility chains one can fit in the head at one time

• Consciously work to get faster and more accurate feedback.

Example - periodically write down your goals for the next month and year. At the end of the period, review progress. This gives feedback on how much you can currently get done.

It also gives feedback on whether what you are correct in your opinion of what your goals are. If you are not progressing towards your goals they may not actually be important goals. You can use the reviews to reverse engineer what your goals must be given what you spend your time on. Eg if you spend a lot of time surfing the web in an undisciplined manner, then being amused and entertained may be very important to you, or maybe it's novelty that's important.

Example - break down larger projects into stages that provide value at each stage, and where you can clearly tell if you have completed a stage. Review the project at the end of each stage to see if you want to continue with it.

Example - ask people around you for feedback on your behavior, strengths, weaknesses. You will probably have to go out of your way to reward "bad news" feedback until people get confident you can take it.

5-second skill: Knowing when to sell

Such occurrences would give a whole new meaning to the term "epistemic hygiene."

Subskill: Accept that an accurate description of yourself during any given frame-instant will involve some emotions, reasons why you act, and reasons why you believe, which are less than perfectly virtuous.

• Don't lie about your life story to yourself - don't spruce up your history to make yourself look better. It may be easier to practice the above skill on the past than on the present, and it gets you used to the general idea.
• Accepting that "What is the cause of my belief?" and "What is the justification of my belief?" will not always have the same answers - you have to accept this non-virtuous fact before you can properly process them as different questions.

Subskill: Notice new judgments and ask about them. E.g., "Huh, I have a bad feeling about this person cleaning this apartment."

• Perceptually notice the new judgment, emotion, or intuition.
• Put a box around it, that is, try to describe what you just felt in verbal language, to promote it to primary awareness.
• Wonder where the feeling came from - search for internal causes.
• Ask about the reference class - the general reliability of 'feelings like this'.

Subskill: Self-investigation of emotions, beliefs, anticipations. Forming hypotheses about what you feel, believe, or expect; questioning those hypotheses to see if they're actually true.

The female possible-equivalent kind of skeeves me out, and doesn't seem to exercise the same skills. I would guess, and I hope, that there are better ways to practice status-projecting, confidence, taking control of interactions, and body language.

Q: What sort of things are these social skills good for other than playing a fairly limited social game?

A: Confidence and social dominance are useful in all sorts of Interactions. However, some of the parts are specific to human romance in this culture.

Q: Do I really care about helping people practice the parts specific to romance in a specific culture, rather than a broad class of social interactions?

A: Nah, not really.

Q: Okay, so what kind of things test your social dominance, without splitting your audience or necessarily practicing culture-specific mating rituals?

A: Getting people to do things (maybe specific behaviors) for you [e.g. go get me a drink]. Practicing specific skills to be dominant [e.g. never giving the appearance that you don't know what to do or need the other person's approval]. Navigating what is considered to be a difficult social interaction that is helped by being impressive [getting a job, making a product pitch].

Skills this may teach / has successfully taught:

Social confidence.

General self-confidence.

Body language, vocal tone, facial expressions.

Charisma.

Navigating social conversations.

Willingness to hug people at LW meetups.

A more challenging alternative might be to try getting a handsome guy to show genuine affection - ie., give you a hug and some words of encouragement ("don't worry about it, you'll do well on that test"), in exchange for nothing offered.

Would buying him the first round count? ;-)

...an exercise of equivalent difficulty for women might be getting a handsome guy to buy you a drink without promising him anything else.

Interesting. This bugs me because I don't like to feed other people's harmful addictions, and my default interpretation for "drink" is an alcoholic drink. I imagine myself being on the guy's side of the interaction, and if I don't know how vulnerable someone is to alcoholism, I don't want to cause them to receive any alcohol. The others who responded seemed to be bothered by something to do with interpersonal interactions at bars. It's interesting that there could be apparently-nonoverlapping reasons to have strong negative feelings about something that on the face of it seems fairly innocuous.

an exercise of equivalent difficulty for women might be getting a handsome guy to buy you a drink without promising him anything else.

Not difficult enough. Better would be getting the man to drive at least 100 miles out of his way or incur at least $200 in expense.

Exercise For Skill I: Conceptual Integration

1. Ask: “Of what do I want a more cohesive conception?” Maybe it’s the Early National Era in the US. Maybe it’s the Union of European Football Associations. You pick.

2. Make sure you know about this thing. Hit the books. Watch a film. Take a class. Already know a lot about this thing? Good. You can skip this step.

3. Make note cards with key terms (events, ideas, people, etc.). No need to define them if you already know about them. You just need a deck of concepts.

4. Shuffle (http://en.wikipedia.org/wiki/Shuffle) your deck. Riffle, Hindu, Pile or Weave and Faro, it doesn’t matter. Just randomize.

5. Pick two cards and ask yourself what relationship(s) exists between the terms. Answer yourself.

6. Keep doing that until you can rapidly identify the relationship(s) between any two concepts in the field.

Note: You may want to throw a third card in the mix. Generate a set of cards that deal with overarching themes and concepts within the subject matter and state a relation in the context of that theme.

Exercise for Skill II: Analogy.

See Steps 1-5 Above

1. Set a number of paired cards down on a big table.

2. Identify sets of pairs with analogous relationships.

Exercise for Skill III: Creative Thinking

1. Get your handy note cards.

2. Do the above-mentioned conceptual integration and analogy exercises with cards from unrelated fields/ schema.

Exercise idea: provide a list of questions that have objectively right answers such as how many bones are there in the human body. Have each person on his own guess at the answers. Next, put everyone in a small group and have the groups discuss the questions and then have everyone individually redo their estimates. Finally, combined each small group with one or two others and repeat what was done before.

Tell everyone at the start of the exercise that their goal is not to just get the right answers but to also identify causes of persistent disagreements within their decision group.

I did this once in my game theory class and the students loved it, although I didn't measure how much they learned from the exercise.

"Rational people can't agree to disagree" is an oversimplification. Rational people can perfectly well reach a conclusion of the form: "Our disagreement on this matter is a consequence of our disagreement on other issues that would be very difficult to resolve, and for which there are many apparently intelligent, honest and well informed people on both sides. Therefore, it seems likely that reaching agreement on this issue would take an awful lot of work and wouldn't be much more likely to leave us both right than to leave us both wrong. We choose, instead, to leave the matter unresolved until either it matters more or we see better prospects of resolving it."

Imperfectly rational people who are aware of their imperfect rationality (note: this is in fact the nearest any of us actually come to being rational people) might also reasonably reach a conclusion of this form: "Perhaps clear enough thinking on both sides would suffice to let us resolve this. However, it's apparent that at least one of us is currently sufficiently irrational about it that trying to reach agreement poses a real danger of spoiling the good relations we currently enjoy, and while clearly that irrationality is a bad thing it doesn't seem likely that trying to resolve our current disagreement now is the best way to address it, so let's leave it for now."

I suspect (with no actual evidence) that when two reasonably-rational people say they're agreeing to disagree, what they mean is often approximately one of the above or a combination thereof, and that they're often wise to "agree to disagree". The fact that there are theorems saying that two perfect rationalists who care about nothing more than getting the right answer to the question they're currently disputing won't "agree to disagree" seems to me to have little bearing on this.

Eliezer, if you're reading this: You may remember that a while back on OB you and Robin Hanson discussed the prospects of rapidly improving artificial intelligence in the nearish future. By no means did you resolve your differences in that discussion. Would it be fair to characterize the way it ended as "agreeing to disagree"? From the outside, it sure looks like that's what it amounted to, whatever you may or may not have said to one another about it. Perhaps you and/or Robin might say "Yeah, but the other guy isn't really rational about this". Could be, but if the level of joint rationality required for "can't agree to disagree" is higher than that of {Eliezer,Robin} then it's not clear how widely applicable the principle "rational people can't agree to disagree" really is. (Note for the avoidance of doubt: The foregoing is not intended to imply that Eliezer and Robin are equally rational; I do not intend to make any further comment on my opinions, if any, on that matter.)

This is probably a myth. The Aumann agreement theorem does not apply to real life. Here are three reasons why:

1. It requires that the two rational people already share a partition function. [EDIT: No! Major mistake on my part. It requires that the two people have common knowledge of each others' partition functions.] The range of the partition function is the set of sets of states of the world that an agent can't distinguish between. That implies that, for all possible sets of observations, each agent knows what the other agent will infer. You could say it requires that the agents query each other endlessly about their beliefs, until they each know everything that the other agent believes.

2. Interpreting Aumann’s theorem to mean what Aumann said it means, requires saying that “The meet at w of the partitions of X and Y is a subset of event E” means the same as the English phrase “X knows that Y knows event E” means. That is wrong. To expland this language a little bit: Aumann claims: To say that agent 1 knows that agent 2 knows E means that E includes all P2 in N2 that intersect P1. I claim: To say that agent 1 knows that agent 2 knows E , means that E includes P1(w), and that E includes P2(w). Agent 1 can conclude that E includes P1 union P2, for some P2 that intersects P1. Not for all P2 that intersect P1. That is a fine semantic error buried deep within the English interpretation, but it makes the entire theorem worthless.

3. Even if you still believe that the Aumann agreement theorem applies in the way James states above, it relies on all agents being perfectly honest with each other, and (probably, tho I'd have to check this) on having mutual knowledge that they are being honest with each other.

I wouldn't call that a skill so much as a frame of mind going into a discussion. Also, they can if they start with different arbitrary priors that neither one can assign objectivity.

Skill: learning that rational people can't agree to disagree.

Well, they can - if they have different goals.