11 core rationalist skills

post by Roko · 2009-12-02T08:09:05.922Z · LW · GW · Legacy · 36 comments

An excellent way to improve one's skill as a rationalist is to identify one's strengths and weaknesses, and then expend effort on the things that one can most effectively improve (which are often the areas where one is weakest). This seems especially useful if one is very specific about the parts of rationality, if one describes them in detail. 

In order to facilitate improving my own and others' rationality, I am posting this list of 11 core rationalist skills, thanks almost entirely to Anna Salamon. 

• Actually want an accurate map, because you have Something to protect.

• Keep your eyes on the prize.  Focus your modeling efforts on the issues most relevant to your goals. Be able to quickly refocus a train of thought or discussion on the most important issues, and be able and willing to quickly kill tempting tangents. Periodically stop and ask yourself "Is what I am thinking about at the moment really an effective way to achieve my stated goals?".

• Entangle yourself with the evidence. Realize that true opinions don't come from nowhere and can't just be painted in by choice or intuition or consensus. Realize that it is information-theoretically impossible to reliably get true beliefs unless you actually get reliably pushed around by the evidence. Distinguish between knowledge and feelings. 

• Be Curious: Look for interesting details; resist cached thoughts; respond to unexpected observations and thoughts. Learn to acquire interesting angles, and to make connections to the task at hand.

• Aumann-update: Update to the right extent from others' opinions.  Borrow reasonable practices for grocery shopping, social interaction, etc from those who have already worked out what the best way to do these things is.  Take relevant experts seriously. Use outside views to estimate the outcome of one's own projects and the merit of one's own clever ideas. Be willing to depart from consensus in cases where there is sufficient evidence that the consensus is mistaken or that the common practice doesn't serve its ostensible purposes. Have correct models of the causes of others’ beliefs and psychological states, so that you can tell the difference between cases where the vast majority of people believe falsehoods for some specific reason, and cases where the vast majority actually knows best. 

• Know standard Biases:  Have conscious knowledge of common human error patterns, including the heuristics and biases literature; practice using this knowledge in real-world situations to identify probable errors; practice making predictions and update from the track record of your own accurate and inaccurate judgments. 

• Know Probability theory:  Have conscious knowledge of probability theory; practice applying probability theory in real-world instances and seeing e.g. how much to penalize conjunctions, how to regress to the mean, etc.

• Know your own mind:  Have a moment-to-moment awareness of your own emotions and of the motivations guiding your thoughts. (Are you searching for justifications? Shying away from certain considerations out of fear?) Be willing to acknowledge all of yourself, including the petty and unsavory parts.  Knowledge of your own track record of accurate and inaccurate predictions, including in cases where fear, pride, etc. were strong.

• Be well calibrated: Avoid over- and under-confidence.  Know how much to trust your judgments in different circumstances.  Keep track of many levels of confidence, precision, and surprisingness; dare to predict as much as you can, and update as you test the limits of your knowledge.  Develop as precise a world-model as you can manage.  (Tom McCabe wrote a quiz to test some simple aspects of your calibration.)  

• Use analytic philosophy: understand the habits of thought taught in analytic philosophy; the habit of following out lines of thought, of taking on one issue at a time, of searching for counter-examples, and of carefully keeping distinct concepts distinct (e.g. not confusing heat and temperature; free will and lack of determinism; systems for talking about Peano arithmetic and systems for talking about systems for talking about Peano arithmetic).

• Resist Thoughtcrime.  Keep truth and virtue utterly distinct in your mind.  Give no quarter to claims of the sort "I must believe X, because otherwise I will be {racist / without morality / at risk of coming late to work/ kicked out of the group / similar to stupid people}".  Decide that it is better to merely lie to others than to lie to others and to yourself.  Realize that goals and world maps can be separated; one can pursue the goal of fighting against climate change without deliberately fooling oneself into having too high an estimate (given the evidence) of the probability that the anthropogenic climate change hypothesis is correct. 

36 comments

Comments sorted by top scores.