Unofficial Canon on Applied Rationality

post by ScottL · 2016-02-15T13:03:09.532Z · LW · GW · Legacy · 32 comments

Contents

  Warning:
  Sections:
  Bugs
  Improving the accuracy of your credence levels
  Mindfulness of how your internal state is affecting how you perceive the world
  Resolving Disagreements
  Other stuff which has already been covered in LessWrong
None
32 comments

I have been thinking for a while that it would be useful if there was something similar to the Less Wrong Canon on Rationality for the CFAR material. Maybe, it could be called the 'CFAR Canon on Applied Rationality'. To start on this I have compiled a collection of descriptions for the CFAR techniques that I could find. I have separated the techniques into a few different sections. The sections and descriptions have mostly been written by me, with a lot of borrowing from other material, which means that they may not accurately reflect what CFAR actually teaches.

Please note that I have not attended any CFAR workshops, nor am I affiliated with CFAR in any way. My understanding of these techniques comes from CFAR videos, blogs and other websites which I have provided links to. If I have missed any important techniques or if my understanding of any of the techniques is incorrect or if you can provide links to the research that these techniques are based on, please let me know and I will update this post. 

Warning:

Learning this material based solely on the descriptions written here may be unhelpful, arduous or even harmful. (See Duncan_Sabien's full comment for more information on this) It is because the material is very hard to learn correctly. Most of the techniques below involve in one way or another volitionally overriding your instinctual, intuitive or ingrained behaviours and thoughts. These are thoughts which not only often feel enticing and alluring, but that also often feel unmistakably right. If you are anything like me, then you should be very careful if you are trying to learn this material alone. For you will be prone to rationalization,  taking shortcuts and making mistakes.

My recommendations for trying to learn this material are:

Sections:

Bugs

An important concept that is required to understand the CFAR material is the concept of 'bugs'. Bugs generally tend to be situations that involve a feeling of "stuckness" and often occur when your system one and two wants are out of alignment. Some concrete examples of bugs include: 

CFAR would stress that 'bugs' are not things that should be accepted with resignation. They are instead things that should be worked through and solved. They are problems that deserve your time, attention and courage to solve. Due to our human nature, it is often best to get the help of others when you are trying to solve your bugs as we tend to rationalize and justify our bugs.

In summary,  the CFAR perspective on bugs seems to be that when you notice one you should think: “Okay! Here is an opportunity for me to get better at life. Where’s my pen and paper?" or "Where can I find someone to talk this through with". 

Discovering bugs - the below techniques all deal, in one way or another, with improving your ability to be able to discover your bugs.

Things that are probably bugs and should be analysed - the below are not really techniques, but are instead descriptions of particular situations that frequently turn out to be bugs. 

Analyzing bugs - these techniques are all about helping you to better understand what you actually value or find aversive

Solving bugs
  • Taking apart your aversions or aversion factoring - involves:
    • Finding a task that you find aversive, but think might be worthwhile doing
    • Attempting to list the complete set of aversions, i.e. everything that is stopping you from doing the task
    • Making each aversion concrete, i.e. finding out what exactly, at the smallest level possible, is aversive. We often think of aversions in the conglomerate sense. We think “I don’t like social gatherings” instead of “I don’t like aspect X of social gatherings”. The problem with this is that it leads us to find things in their entirety to be aversive when it might really only be one aspect of it that we find to be aversive.
    • Solving your aversions. The strategy that you take here will depend on the type of aversion that it is. We can think of aversions as being one of two types: helpful and non helpful.
      • A helpful aversion is one where you would think: “Oh, thanks brain for helping me avoid that harmful situation”. Aversions of this type should either be accepted or solved by generating procedures or techniques that eradicate the danger or potentially harmful situation. Once the aversion is solved it may still linger, which means it has turned into a non helpful aversion. 
      • A non helpful aversion is one that is present when there is no danger or reason for its existence. Aversions tend to be sticky and residual which means that people's aversions often prompted helpful behaviors in the past, but since their environment has changed they have now become unhelpful. The goal in this step is to simply develop a plan that will help to remove these aversions. Often this will involve acclimation or exposure. Reframing can also help. Also, see the section: "optimising your ability to follow through with your plans"
    • Doing a mindful walk through of the activity that was originally aversive to see if it is still aversive. If it is, then repeat this process from step 2.
  • Reframing – when making decisions the underlying valuation of a solution or problem depends on the perspective or way in which you are framing the solution or problem. For example, instead of thinking “Do I want to go the gym?” you can think “If I was already at the gym, would I want to leave and wish that I hadn’t gone?”
  • Pitting your desire to look good against your desire to improve - if you are going to improve fast, then you will probably also need to fail fast and fail frequently, at least at first. You will need to be doing things like: asking 'silly' questions or trying stuff out even when you're not ready. This can be hard to do because no one likes to look like a fool. One technique that sometimes helps in making this easier is to to compare your desire to not look like a fool now vs. your desire to become better.
    While one person hesitates because he feels inferior, the other is busy making mistakes and becoming superior
  • Focused Grit - this basically involves sitting down, focusing and giving a problem your full attention for at least five minutes. A surprising number of problems can be solved in just five minutes. The whole process is:
    • Take a piece of paper, a pen and a timer
    • Identify a large problem, e.g. terrible job, family feud
    • Think about the problem for five whole minutes, by the clock
    • If the problem has not been resolved and you know of another 5 minute exercise that you could try, restart this process from step 1 with this exercise.
    • If the problem has not been resolved and you do not know of another 5 minute exercise you could try, spend 5 minutes brainstorming potential 5 minute exercises that could potentially solve the problem. If you come up with an exercise, restart this process from step 1. 
Mindsets and perspectives that help you in discovering potential situations that you could end up valuing
  • Self efficacy - this is the sense that you can tackle any given problem. It is the idea that there is always a way in which to do something and that you just need to find it
  • Growth mindset - this is the belief that you are always capable of being more than you currently are. A growth mindset does not necessarily involve positive thinking. For example, if you repeat the affirmation ‘I am a great singer’. This can actually have a negative effect. Because the affirmation relates to a fixed attribute and if your brain doesn't feel it to be true, then it is likely to activate the exact opposite concept, i.e. that you are a bad singer. Another potential negative effect of positive thinking is the “goal-turnoff effect” which means that once a pursuit attempt has been completed the goal deactivates and inhibits the mental representations used to attain the goal. 
  • Mental contrasting - is a technique can be used as an alternative to positive thinking and has been shown to be useful. Mental contrasting is a visualization technique where you imagine several positive aspects of completing your goal and then look at your current situation and the obstacles that are stopping you from completing the goal. To work well mental contrasting requires a growth mindset and a reasonable expectation of success. This is because if you do it with a fixed or negative assessment of yourself then it will only deepen and reinforce this assessment.
  • A feeling that you should keep trying new things - a big part of this is being willing to embrace the new and uncertain in the hope that it will lead to something useful. This involves taking on a new perspective where you don’t think of planned actions in terms of their potentially aversive or pleasant outcomes, but instead as potential sources of useful information. You view them in a similar way to how you would view an experiment. If an experiment does not give you the expected result, it still gives you a range of other useful information like what doesn’t work or that your beliefs may be somehow faulty. This type of thinking is not that intuitive because, by and large, the mind encourages us to behave in ways that have worked before. Though this is useful from a survival perspective,  it does not often lead us to try things that are completely different from what we have done before.
  • Use curiosity or closed vs open ways of thinking - for complex problems, your time is sometimes better spent, not in trying to solve the problem, but instead in searching for solutions to the problem. That is, it can be useful to jump out of your problem solving mode or your cached routine mode and to start questioning. This is especially true for problems that you have been trying to solve for a long time. It is important to note that searching for solutions isn't necessarily a wholly conscious process. It is often the case that the correct solution has already been activated by your brain, but is not being brought to your conscious attention because of other cached responses that are taking priority. If you can find some way to relax, get distracted or do something else for a while the correct solution will sometimes, seemingly out of nowhere, pop into your head. 
Optimizing your ability to follow through with your plans
  • Being Strategic – teaches you how to make a to-do system. This includes going through tools like things like: Getting Things Done (GTD), Remember the Milk, Workflowy, BeeMinder, Anki, Pomodoro Technique
  • Next actions – our motivation is normally dependent on our predictions, which can be problematic. A way to avoid these problems is to just know the next physical action that would get you closer to completion. Instead of having an item ‘do my taxes’ you can have an item of ‘find tax forms and put them on desk’ which is both a concrete and small action.
  • Trigger Action Plans (TAP)s - is a simple algorithm that goes, "if I do x, then I will do y." See this post.
  • Structured Procrastination - involves acknowledging that procrastination is somewhat inevitable which means that it can be beneficial if you spend some time trying to optimising what you will do when you are procrastinating. If you are probably going to end up procrastinating, you might as well get the most out of it. For example, you might decide to procrastinate on starting an assignment by going on a walk or cleaning the house. By using structured procrastination you can also make your plans more pliable and capable of being completed. 
  • Does future me have a comparative advantage? – if you are thinking of procrastinating and putting something off till later, try asking yourself if the future version of you will be better off or more easily able to handle the task. If the answer is no, then you should probably just do it now. This question is probably more helpful for small items than large ones. 
  • Murphyjitsu – is essentially imagining everything that could possibly go wrong and would stop you in reaching that goal. Then outplanning these potentialities so that they never occur. They is referred to as outplanning Murphy. The “Murphyjitsu” technique can be used by asking “how surprised would I be if I failed?”, followed by “what obstacles might prevent me from finishing?” For example, “I’m going to do CrossFit three times a week and I would only be mildly surprised if I failed” should immediately lead to problems and solutions like:
    • “Which days?” → “Mondays, Wednesdays, and Fridays, at 5:30 PM, after work, with an alarm set, and with my workout clothes always in my car.”
    • “What if I can’t go during gym hours?” → “If I can’t make CrossFit, then any exercise of equal or greater intensity and duration will suffice.”
    • “What if I’m sick?” → “I don’t go if I’m too sick to go to work, otherwise I go.”
    • “What if I’m, like, really really tired though?” → “I go anyway but take it easy if I need to.”
    • “What if I’m injured in a car accident?” → “Obviously don’t worry about it. Also, I need a first aid kit in my car.”
  • Habit practicing – Imagine that you wanted to stop sleeping in when your alarm goes off in the morning. One way to train this habit is to practice waking up when your alarm goes off. You would do this by turning off the lights in your room, setting your alarm for five minutes, getting into bed, relaxing and then mindfully getting up straight away when the alarm goes off.
  • (I have been told that this is no longer up to date) Urge propagation - is a motivation hack which traces the causal link between a larger goal you really care about and the action you don't feel motivated to take, and explains that link to system 1 in a language it can understand. It involves finding out what your System two wants you to do and how you can align your system one to want the same thing. This is normally done with operant conditioning or reinforcement. What makes things like trying to lose weight or to learn the violin difficult, is that it often causes conflicts with system one driven urges, e.g. wanting a nap or craving a cookie, and because long-term goals typically require sticking it out through a series of unpleasant intermediate steps, e.g. eating less and practicing the violin, it can be easy to lose the original motivation. This loss of motivation tends to not occur so frequently for things that we feel rewarded by. This is primarily because we do these things automatically. When I want Thai food, I don't need to volitionally force myself  to drive there, look at the menu, go inside, order etc. I don’t have to convince myself to take those steps. They happen automatically. The point of urge propagation is to make long-term goals feel more like short-term urges which essentially means rewiring the brain to associate actions with rewards for the particular task that you want to do. You can use a similar idea to discourage bad habits. You do this by stretching out the time between an action and its reward. For example, if you want to stop reading stuff online then you can have the pages load more slowly for non work related material. Thinking about urge propagation leads us to realise that small aversive moments can have a drastic impact on our choices and motivations. The aversive moment of getting into a cold swimming pool can overwhelm the delayed rewards of doing morning laps even though you might really like or want the rewards, e.g. being fitter. In summary the process in urge propagation is: 
    • Find a situation where the rewards are distanced from the actions
    • Associate the activity with a powerful feeling of reward, one with a stronger neurochemical kick than the virtuous goals, e.g. ‘being healthier, that we normally aspire to.
    • Come up with a mental image that vividly captures that feeling and that you can summon in moments of weakness. It should be a very sticky image because if it isn't then you won’t experience that gut-level surge of motivation that the image is meant to cause. For example, one person was able to overcome their aversion to push-ups, which made them feel unpleasantly hot and sweaty, by tapping into their obsession with longevity. They now vividly imagine the heat from the exercise as if it was a fire that burned away cell-damaging free radicals.
Learning optimization
  • Turbo Charged training – this is best for the types of skills that you repeat over and over again to get your nervous system to change in order to get better at it. The underlying idea is the rule of intensity which states that the experience of intensity or effort that you are expending to learn something corresponds with the rate at which you are learning it. For example, 10 minutes of intense time spent trying to learn a piano piece may be just as helpful as 1 hour of less intense effort. Turbo charged training has two main principles: maximise the engagement with the learning process and minimize error. This means that if you are making lots of errors then you need to slow down or make whatever you are doing easier. If what you are doing is too easy, then you need to find some method to ramp it up and make it more difficult. It is also important that you are learning the right skills and are getting good feedback which is not noisy, but is fast enough to signal to the reward centers of the brain that you are actually making progress. This ensures that the neural patterns that were just activated get reinforced. Turbo charged training is based on taking an outside view of learning, i.e. looking at the people who learn really fast and what they are doing.
Planning optimization
  • Surprise-o-Meter – this technique involves picturing an event and observing how surprised you would be to be in that situation. Surprise is a clue that you were implicitly expecting something else to happen. The more surprised you are, the less probable your subconscious thought of that event was. Also called, pre-Hindsight which involves using emotions to evaluate how likely you are to succeed at a goal. Imagine that, six months from now, you have not achieved your goal. The level of surprise you feel at this outcome is a good predictor of whether you will actually succeed. You can then use this information as input that can be used to build plans that will actually work. 
  • Your inner simulator - Your “inner simulator” is CFAR’s version of the distinction between profession and anticipation.  Basically, your “inner simulator” is the part of you that can play movies forward to determine what to anticipate: “Do I have time to turn left before that car reaches me?”; “What will she do, if I approach and say ‘hi’?” You can use your inner simulator to develop high quality plans. An example of this process would be asking:
    • What you predict will happen if you take some course of action? 
    • What could go wrong? 
    • How can you change your plans to prevent that thing from going wrong? 
    • This process is repeated from the first step with your predictions updated to take into account your preventitive measures.

Improving the accuracy of your credence levels

Mindfulness of how your internal state is affecting how you perceive the world

Resolving Disagreements

Other stuff which has already been covered in LessWrong

32 comments

Comments sorted by top scores.

comment by Duncan Sabien (Deactivated) (Duncan_Sabien) · 2016-02-16T04:34:19.337Z · LW(p) · GW(p)

[CFAR's newest instructor, here; longtime educator and transhumanist-in-theory with practical confusions]

ScottL—I'm just coming out of the third workshop in six weeks, and flying to Boston to give some talks, so I'm exhausted and haven't had a chance to read through your compilation yet. I will, soon (+1 for the effort you've put forth), but in the meantime I wanted to pop in and give some thoughts on the comments thus far.

Benito, Rainbow, and Crux—+1 for all three perspectives.

Can CFAR content be learned from a compilation or writeup? Yes. After all, it's not magic—it was developed by careful thinkers looking at research and at their own cognition, iterated over 20+ formal attempts (and literally hundreds of informal ones) to share those same insights with others. It's complex, but it's also fundamentally discoverable.

However, there are three large problems (as I see it, speaking as the least experienced staff member). The first is the most obvious—it's hard. It's hard like learning karate from text descriptions is hard. If you go about this properly, without being sloppy or taking shortcuts or making dangerous assumptions, then you're in for a LONG, difficult haul. Speaking as someone who pieced together the discipline of parkour back in 2003, from scattered terrible videos (pre Youtube) and a few internet comment boards—pulling together a cohesive and working practice from even the best writeups is a tremendously difficult task. It's better on almost every axis with instructors, mentors, friends, companions—people to help you avoid the biggest pitfalls, help you understand the subtle points, tease apart the interesting implications, shore up your motivation, assist you in seeing your own mistakes and weaknesses. None of that is impossible on your own, but it's somewhere between one and two orders of magnitude more efficient and more efficacious with guidance.

The second is corruption. As Benito points out, a large part of the problem of rationality instruction is finding things that actually work—if mere knowledge of the flaws were sufficient to protect us from the flaws, then everybody who cared enough could just slog through Heuristics and Biases and be something like 70% of the way there. We've already put several thousand thought-hours and 20+ iterations into tinkering with content, scaffolding, presentation, and practice. What we've got works pretty well, but progress has been incremental and cumulative. What we had before worked less well, and what we had before that worked less well still.

Picture throwing out a complete text version of our current best practices, exposing it to the forces of memetic selection and evolution. Fragments would get seized upon, and quoted out of context; bits of it would get mixed up with this and that; things would be presented out of order and read out of order; people would skip and skim and possibly completely ignore sections they THOUGHT they already knew because the title or the first paragraph seemed mundane or familiar. And there wouldn't be the strong selection pressure toward clarity and cohesion that we've been providing, top-down—instead, there would be selection pressures for what's memorable, pithy, or easily crystallized, none of which would be likely to drive the art forward and make the content BETTER. Each step away from our current best practices is much more likely to be a decrease in quality rather than an increase, and though you and others here on LW are likely to have the necessary curiosity and diligence to "do it right," that doesn't mean that the majority of people exposed to the memes in this way share your autodidactic rigor.

The third problem (related to the second) is idea inoculation. Having seen crappy, distorted versions of the CFAR curriculum (or having attempted to absorb it from text, and failed), a typical human would then be much, much less receptive to other, better explanations in the future. This is why, even within the context of the workshop, we often ask that participants not read the relevant sections of their workbooks until AFTER a given lecture or activity. I'm going to assume this is a familiar concept, and not spend too many words on it, but suffice it to say that I believe an uncanny valley version of our curriculum trending on the internet for one day could produce enough anti-rationality in the general population to counterbalance all of our efforts so far.

None of these problems are absolute in nature. The Sequences exist, and are known to be helpful. And clearly, Rainbow and Benito have gotten at least some value out of the writeups they've gleaned and assembled themselves. Again, there's nothing to stop others from having the same insights we've had, and there's nothing to stop a diligent autodidact from connecting scattered dots.

But they are statistical. They are real. They become quite scary, once you start talking big numbers of people and the free exchange of content-sans-context. And that's without even talking about other concerns like framing, signaling, inferential distance, etc. Lots of worms in this can.

So the question then becomes—what to do?

Thus far, CFAR hasn't had the cycles to spend time creating the (let's say) 80-20 version of their content. Remember that it's a fledgling startup with fewer than ten full-time staff members (when Pete and I were hired, it only had six). They were pouring every 60- and 70- and 80-hour week into trying to squeeze an extra percentage point of comprehension or efficacy out of every activity, every explanation. In other words, the objection wasn't fundamental (to the best of my understanding, which may be wrong) ... it was pragmatic. Creating packaged material fit for the general public wasn't anywhere near the top of the list, which was headed by "create material that's actually epistemically sound and demonstrably effective."

For my own part, I think this belongs in our near future. I think it's an area to be approached cautiously, in incremental steps with lots of data collection, but yes—I'd like to see some of our simpler, core techniques made broadly available. I'd like to see scalability in the things we think we can actually explain on paper. And if it goes well, I'd like to see more and more of it. I'm personally taking steps in this direction (tackling and improving our written content is one of my primary tasks, and I've started with simple things like drafting a glossary and tracking which definitions leave the reader confused (or worse, confident but wrong)).

But we have to a) find the time and manpower to actually run the experiment, and b) find content that genuinely works. Those are both non-trivially difficult, and they're both trading off against the continued expansion and improvement of our version of the art of rationality. I've only just now taken on enough responsibility myself to free up a few of the core staff's hours—and that's mostly gone into reducing their workload from insane to merely crazy. It hasn't actually created sufficient surplus to allow online tutorials to meet the threshold for worth-the-risks.

In short, despite Crux's entirely appropriate and reasonable skepticism, the answer has to be (for the immediate future)—either you find us trustworthy, or you don't (and if you don't, maybe you don't want our material anyway?). I, for one, don't think published material threatens workshop revenue, any more than online tutorials threaten martial arts dojos. There will always be obvious benefits to attending an intensive, collaborative workshop with instructors who know what they're doing, and there will always be people who recognize that the value is worth the cost, particularly given our track record. Our reasons for having refrained from publication thus far aren't monetary (or, to be more precise, money isn't in the top five on what's actually a fairly long and considered list).

Instead, it's that we actually care about getting it right. We don't want to poison the well, we don't want to break the very thing we're trying to protect, and as a member of a group with something that at least resembles expertise (if you don't want to credit us as actual experts), I think that requires a lot more work on our end, first.

That being said, if you have questions about the content above, or about what CFAR is doing this week and this month and this year, or if you're struggling with creating the art of rationality yourself and you've had novel and interesting insights—

Well. You know where to find us, and we don't know where to find you, or we'd have already reached out.

Hope this helps,

  • Duncan
Replies from: Jacobian, ScottL, RainbowSpacedancer, Riothamus
↑ comment by Jacob Falkovich (Jacobian) · 2016-02-17T16:59:32.851Z · LW(p) · GW(p)

Can CFAR content be learned from a compilation or writeup?

A year ago I considered attending the CFAR workshop in Boston, one of the things that stopped me was that I actually read LW a lot and applied a bunch of it in real life. Kenzi and Critch at CFAR tried gently to explain how a workshop was qualitatively different from reading and trying stuff yourself, but I didn't give them the opportunity to convince me.

This week I came back from the CFAR workshop in New York, and I actually felt my life changing on the evening of the third day. Yes, time will tell if that actually happened, but I have enough evidence even a week out that it's going that way. How could I think that I could get that benefit by myself with no help? It scares me how close I was to never having gone to CFAR. I'm going to try to write what would have convinced "Jacobian-2015" that he should attend a workshop.

  • Compound interest. You need motivation to work on your motivation. You need an accurate map of knowing how to attain accurate maps. It takes a jolt of rationality to improve your rationality. There isn't an encyclopedia of CFAR material, but the material is incredibly high quality. This causes it to compound and improve other things you learn, like the difference between $100 under your mattress (i.e. the sequences) and $20 that grows at 20% a year.

  • Blind spots. You can't lift yourself up by your hair, you can't see the mistaken beliefs you refuse to question and you can't solve the problems in your life you refuse to admit. Some things simply can't be done without other people helping you out. Most of my progress at CFAR was made in the hours of focused small group "therapy" sessions. The first thing I did when I got back was to set up a CFAR workgroup (can I trademark "Agenty Flock"?) with friends from the workshop.

  • The moon. This is either really important or completely meaningless, I don't know because I'm not there yet. The point of CFAR isn't to learn a bunch of techniques but to achieve the mindset in which the techniques become natural, indistinguishable and you are able to generate them yourself at will. The techniques are the fingers pointing at the moon, the mindset is the moon. I did my BSc in physics, and I retain less declarative knowledge of physics than someone who read the Feynman lectures. Still, I think I wouldn't have fallen for the radiator plate trick. Not because I can do integrals of thermal conduction, but because I spent hours in a lab trying to get some dumb thermodynamics experiment to work the way I believed it should, and it refused. I don't know if I really attained a physics mindset in undergrad, and I don't know if the applied rationality mindset is attainable from a CFAR workshop. I know that it would take a super-mind to attain it from reading stuff online.

ScottL, your write-up is great. The only thing I don't like about it is that you called it "CFAR canon", isn't it troubling that that's what would show up in search results from now on? I would at least change the word "CFAR" to "applied rationality". I'm really concerned that this write-up may cause some people to decide against attending a workshop they otherwise would've gone to. To everyone who reads this "canon" and considers going to a workshop, ask yourself this:

How many actual CFAR alumni do I know who feel that they could have gained most of the value by themselves?

Count my experience as a point of evidence against.

Replies from: ScottL
↑ comment by ScottL · 2016-03-19T13:35:55.616Z · LW(p) · GW(p)

The moon. This is either really important or completely meaningless, I don't know because I'm not there yet.

I prefer the concept of Fingerspitzengefühl (finger tips feeling) which basically means having an intuitive grasp of a situation and being able to zero in on the accurate region of the problem without wasteful consideration of a large range of unfruitful, alternative diagnoses and solutions. The mechanism behind this is probably similar to how we learn physical patterns.

Expert piano player’s movements largely happen automatically or intuitively. That is, without conscious thought. This happens due to their extensive practice and because of concepts like chunking, spreading activation and hebbian learning. I would guess that we also learn psychical (thought) patterns through a similar mechanism.

I would at least change the word "CFAR" to "applied rationality"

Thanks for your suggestion. I removed the CFAR from the title.

I'm going to try to write what would have convinced "Jacobian-2015" that he should attend a workshop.

There are two extra things that my post isn’t good at conveying.

  • Depth in the material. For example, goal factoring I have a paragraph on this and, I guess, that it describes the concept, but it would take a whole other post to describe how to use it in practice.
  • A framework. The post just describes the techniques, but it doesn’t really describe the underlying mechanisms of how they work or how they relate to each other. In practice, the techniques would rarely be used in isolation, but would instead be combined.

I'm really concerned that this write-up may cause some people to decide against attending a workshop they otherwise would've gone to.

My intent was never for this post to be used as a replacement to attending CFAR. My goal was to put the material out there so that there was some base material upon which I could expand.

↑ comment by ScottL · 2016-02-16T13:00:42.499Z · LW(p) · GW(p)

Point 1 (It's hard to learn) - I agree. I have added a warning at the top of the post which should help with this problem Point 2 (corruption) - I don't think this post can be in anyway be a substitute for the workshops, but I think it can still have value as a base or glossary. It is definitely doesn't provide a kind of framework or common thread of understanding which I think you seem to be saying is very important. Point 3 (idea inoculation) - isn't this problem (Having seen crappy, distorted versions of the CFAR curriculum) resolved if you check the post to make that what I am saying is accurate and true to what CFAR actually teaches.This one (having attempted to absorb it from text, and failed) may be a reason for me to retract this post, however. Let me know what you think.

Overall. I respect your caution, but I don't think that having some potential misinformation is as bad as you make it seem. At least if we're careful.

There will always be obvious benefits to attending an intensive, collaborative workshop with instructors who know what they're doing, and there will always be people who recognize that the value is worth the cost, particularly given our track record.

I agree with this which I think was your overall point.

Replies from: Duncan_Sabien
↑ comment by Duncan Sabien (Deactivated) (Duncan_Sabien) · 2016-02-16T13:20:54.334Z · LW(p) · GW(p)

I don't think I have the authority (moral, social, or other) to be the guy who's like, "Hey, please take this down" or "Hey, leave it up!" I will—when I have spare cycles—read through what you've written, and offer some specific thoughts, but I'll probably offer them privately, rather than turning this whole thread into a "zeroing in" on our current curriculum.

I just don't know, y'know? I fully respect and endorse the Ravenclaw/Hufflepuff combo punch that caused you to want your summary to exist and be public—more people putting in that sort of effort seems strictly better. I just wanted to put in two cents (or several dollars, I guess) to steelman my understanding of CFAR's position before the discussion got framed in some other way.

↑ comment by RainbowSpacedancer · 2016-02-16T11:11:50.476Z · LW(p) · GW(p)

Problems one and two (hard and imperfect) would suggest that people will get less value out of ScottL's post than a workshop. OK, fine. Don't let the perfect be the enemy of the good. Scale ScottL's post up through easy online access and the many, many people getting a smaller somewhat unreliable benefit turns into something very significant. But problem 3,

Having seen crappy, distorted versions of the CFAR curriculum (or having attempted to absorb it from text, and failed), a typical human would then be much, much less receptive to other, better explanations in the future.

We don't want to poison the well, we don't want to break the very thing we're trying to protect, and as a member of a group with something that at least resembles expertise (if you don't want to credit us as actual experts), I think that requires a lot more work on our end, first.

That's reason enough to not release your own material. But specifically, do you think ScottL's compilation above or sharing the guide I've written (if I was to post it here for anyone to use) has the same effect? Do you think our compilations will have a net negative effect on rationality?

Thus far, CFAR hasn't had the cycles to spend time creating the (let's say) 80-20 version of their content.

For my own part, I think this belongs in our near future.

Do you have an estimate on this? I won't hold you to it, I'd just like to know what kind of time frame 'near' is.

Replies from: Vaniver, Duncan_Sabien
↑ comment by Vaniver · 2016-02-16T15:08:59.620Z · LW(p) · GW(p)

But specifically, do you think ScottL's compilation above or sharing the guide I've written (if I was to post it here for anyone to use) has the same effect? Do you think our compilations will have a net negative effect on rationality?

My view, as a CFAR alum and donor, is that the primary arguments against CFAR releasing their material are 1) better returns on time and 2) making it more difficult to change the material. I think online material complements instead of competes with in person classes; standard advice in consulting is "give away your best material for free." (I think CFAR was sensible to wait until now to decide that some of its material is 'best' enough to give away.)

I don't think independent compilations of rationality material are net negative, in the same way that I think Starbucks complements instead of competes with independent coffee shops.

I do think it's weird to call this the CFAR canon if it's not explicitly endorsed by CFAR. (ScottL, what do you think the word 'canon' means?)

↑ comment by Duncan Sabien (Deactivated) (Duncan_Sabien) · 2016-02-16T13:17:11.296Z · LW(p) · GW(p)

I'm not sure, re: whether your or ScottL's compilations provide negative value. I definitely fall shy of recommending that they be posted, but I think I ALSO fall shy of anything like requesting that they be taken down. I think there's probably a meaningful difference between CFAR publishing something, and friendly Less Wrongers being like, "Hey, here's this thing I pulled together, hope it helps." The risks I anticipate seem much stronger in the former case.

As for what "near" means, I predict with 70% confidence that we will publish more than 5000 words about actual CFAR content before the end of 2016. That's a pretty weak prediction, I know, but also I'm not in a position to be very confident (given my naïveté). I will say that in the universes where we publish 5000 words, we're also likely to publish a lot MORE.

↑ comment by Riothamus · 2016-07-19T15:23:02.617Z · LW(p) · GW(p)

I wonder if it would be possible to screen out some of the misinterpretation and recombination hazards by stealing a page from mystery religions.

Adherents were initiated by stages into the cult; mastery of the current level of mysteries was expected before gaining access to the next.

Rather than develop a specific canon or doctrine, CFAR could build the knowledge that new practices supersede the old, basic practices must come before advanced practices, and precisely what practices should have been tackled previously and will be tackled next into everything instructional they produce for the public.

If this is pervasive in CFAR literature for the public, I would expect the probability of erroneous practice to go down.

Replies from: Lumifer, ChristianKl
↑ comment by Lumifer · 2016-07-19T15:43:45.053Z · LW(p) · GW(p)

Yes, we should definitely make CFAR/LW look more like a cult :-/

Replies from: Riothamus
↑ comment by Riothamus · 2016-07-19T19:58:06.824Z · LW(p) · GW(p)

Sigh. I continue to forget how much of a problem that is. It is meant in the historical, rather than colloquial, meaning of the word. Since it apparently does not go without saying, the easily misunderstood term should never be used in official communication of any sort.

I apologize for the lack of clarity.

Replies from: ChristianKl, Lumifer
↑ comment by ChristianKl · 2016-07-20T18:42:56.231Z · LW(p) · GW(p)

Do you know what the historical techniques happen to be?

Let's take Maimonidies whose behavior is well described by Leo Strauss. There's a law in the Torah against teaching the secrets in the Torah outside of 1-to-1 teaching. If Leo Strauss is to be believed Maimonidies purpusefully writes wrong things to mislead naive readers and keep advanced knowledge from them.

If CFAR would write purposefully misleading things in their public material to pander down to naive readers and keep advanced knowledge from them, that would produce problems.

the easily misunderstood term should never be used in official communication of any sort.

In the time of the internet don't use words publically that you wouldn't use in official communication.

Replies from: Riothamus
↑ comment by Riothamus · 2016-07-20T19:45:31.375Z · LW(p) · GW(p)

You have just described the same thing Duncan cited as a concern, only substituted a different motive; I am having trouble coming to grips with the purpose of the example as a result.

I propose that the method of organizing knowledge be considered. The goal is not to minimize the information, but to minimize the errors in its transmission. I assume transmission is inevitable; given that, segregating the information into lower-error chunks seems like a viable strategy.

Replies from: ChristianKl
↑ comment by ChristianKl · 2016-07-20T20:34:26.656Z · LW(p) · GW(p)

You refered to historical techniques that are used. Generally historical groups actually have defenses against lay people accessing knowledge even if those lay people think they are experts and should be able to access the knowledge.

Whether it's sworn secrecy, hiding knowledge in plain sight or simple lies to mislead uninitiated readers, there's a huge toolbox.

I assume transmission is inevitable; given that, segregating the information into lower-error chunks seems like a viable strategy.

Presumably CFAR thinks that their workshop is a low error chunk of consuming their material.

Replies from: Riothamus
↑ comment by Riothamus · 2016-07-20T22:04:47.446Z · LW(p) · GW(p)

I should amend my assumption to uncontrolled transmission is inevitable. The strategy so far has been to use the workshops, and otherwise decline to distribute the knowledge.

The historical example should be considered in light of what the goals are. The examples you give are strategies employed by organizations trying to deny all knowledge outside of the initiated. Enforcing secrecy and spreading bad information are viable for that goal. CFAR is not trying to deny the knowledge, only to maximize its fidelity. What is the strategy they can use to maximize fidelity in cases where they did not choose to transmit it (like this one)?

Suppose we model everyone who practices state-of-the-art rationality as an initiate, and everyone who wants to read about CFAR's teachings as a suppliant. If the knowledge is being transmitted outside of the workshops, how do we persuade the suppliants to self-initiate? Imposing some sort of barrier, so that it requires effort to access the knowledge - I suggest by dividing the knowledge up, thus modelling the mysteries. We would want the divided content to be such that people who won't practice it disengage rather than consume it all passively.

If CFAR were to provide the content, even in this format, I expect the incentive of people to produce posts like the above would be reduced, likewise for the incentive of people to read such collections.

In retrospect, I should have made it explicit I was assuming everyone involved was a (potential) insider at the beginning.

Replies from: ChristianKl
↑ comment by ChristianKl · 2016-07-21T08:50:37.821Z · LW(p) · GW(p)

The examples you give are strategies employed by organizations trying to deny all knowledge outside of the initiated.

I think most of the organsiation I'm talking about don't have a binary intiate/non-initiate criteria whereby the initiated get access to all knowledge. As people learn more they get access to more knowledge. Most scientologists haven't heard of Xenu. At least that was the case 10 years ago.

If the knowledge is being transmitted outside of the workshops, how do we persuade the suppliants to self-initiate?

LW-Dojo are a way for knowledge to be transmitted outside of workshops. I also think that alumni are generally encouraged to explain knowledge to other people. Peer-to-peer instruction has natural filter that reduce completely passive consumption.

That doesn't mean that inherently impossible to transmit knowledge via writting but it's hard.

Replies from: Riothamus
↑ comment by Riothamus · 2016-07-21T13:10:02.944Z · LW(p) · GW(p)

That doesn't mean that inherently impossible to transmit knowledge via writting but it's hard.

Agreed. The more I consider the problem, the higher my confidence that investing enough energy in the process is a bad investment for them.

Another romantic solution waiting for the appropriate problem. I should look into detaching from the idea.

↑ comment by Lumifer · 2016-07-19T20:10:14.693Z · LW(p) · GW(p)

Call it something like "gnostic practices" so that hoi polloi have no idea what it means, but it sounds respectable :-)

↑ comment by ChristianKl · 2016-07-19T17:17:29.242Z · LW(p) · GW(p)

basic practices must come before advanced practices

We aren't at a point yet where we distinguish "basic" from "advanced" practices. Most of what CFAR teaches is a 4-day workshop. CFAR doesn't try to teach anything that takes a year to understand.

The idea that basics are somehow easy to understand also mistakes a lot about what learning deep knowledge is about. Basics are hard because they are fundamentals and affect a lot.

When dancing Salsa there was the saying: "At congresses beginners take the intermediate classes, intermediates take the advanced classes and the advanced people take the beginners classes".

Today I was at my meditation/movement class and the teacher (with ~15 years in the method and likely much more than 10000 hours of meditation) was saying that she still fails to have a good grasp on the basic of rhythm and that it alludes her.

Replies from: Riothamus
↑ comment by Riothamus · 2016-07-19T20:18:53.984Z · LW(p) · GW(p)

We aren't at a point yet where we distinguish "basic" from "advanced" practices.

This is a good point; I have assumed that there would eventually be a hierarchy of sorts established. I was allowing for instruction being developed (whether by CFAR or someone else) even down below the levels that are usually assumed in-community. When Duncan says,

Picture throwing out a complete text version of our current best practices, exposing it to the forces of memetic > selection and evolution.

I interpret this to mean even by people who have no experience of thinking-about-thinking at all. As you aptly point out, the fundamentals are very hard - there may be demand for just such materials from future advanced rationalists for exactly that reason. So what I suggest is that the components of the instruction be segregated while retaining clear structure, and in this way minimize the skimming and corruption problems.

That being said, I fully endorse the priority choices CFAR has made thus far, and I do not share the (apparent) intensity of Duncan's concern. I therefore understand if even evaluating whether this is a problem is a low priority.

Replies from: ChristianKl
↑ comment by ChristianKl · 2016-07-20T18:31:02.393Z · LW(p) · GW(p)

This is a good point; I have assumed that there would eventually be a hierarchy of sorts established.

"Eventually" is a key word. I think in ten years CFAR's curriculum will be more settled than it is today.

Take triggers action plans (TAP). They are considered a basic. In the scientific literature and in CFAR's first workshops they were called "implementation intentions". CFAR found that it's useful to have a short word with TAP to be the concept more usable.

That's not a change in something basic.

That being said, I fully endorse the priority choices CFAR has made thus far, and I do not share the (apparent) intensity of Duncan's concern.

A while ago someone in this community tried to write a guide for a self-help technique. Let's call him Bob. Bob read the official guide. The offical guide for the techique referenced a few ingridents that Bob didn't kow. To do the technique properly the person doing it needs to do X and Y. Bob didn't know what X and Y were supposed to be. Bob however was widely read in the self-help literature so he simply replaced X and Y with M and N while writting his own guide for the technique.

Bob also didn't have much experience with actually using the technique. In that case I told Bob, don't publish that guide and I think the draft for the guide didn't circulate further or got more work.

I think before I went to me first self-help seminar I was like Bob. I spent 4 years spending 3 hours per day at a personal development forum where I was a moderator, so I thought I know what I was talking about. LessWrong draw people like that you read a lot but who often don't practice techniques enough.

From this writeup take the part about CoZE exercises. The writeup says that it's good to develop a playful attitude but if you look at it's step by step list the steps it gives likely don't develop a playful attitude. I don't know the quality of CFAR's CoZE teachings but if CFAR knows how to actually teach people to do them with a playful attitude, CFAR alumni do something that person trying to follow the writeup won't do.

Having read the writeup might make it harder for someone who comes to CFAR to actually understand what CFAR teaches as CoZE because the person has already a preconveiced notion.

http://lesswrong.com/lw/n5h/unofficial_canon_on_applied_rationality/d4et suggests that this is writeup of CoZE does indeed recommend things fundamentally opposed to CFAR teachings.

Doing things like this in a playful way is a basic. A basic that's hard to learn.

comment by RainbowSpacedancer · 2016-02-15T17:30:34.327Z · LW(p) · GW(p)

I had a very similar thought to this post. So similar in fact that I went ahead and wrote a kind of user guide for each CFAR's techniques (though it has changed a great deal even in the last 4 months since I finished writing). I also have never been to a CFAR workshop and drew on many of the same online sources that you have. It took about a month to compile of working in my spare time. My motivation for doing so was the cost of attending a workshop (financially and time costs) were simply too high for someone in my position overseas.

I've printed it and only use it personally. I've never shared it other than with one close friend. I'm concerned about you posting this now, for the same reasons that stopped me from sharing my compilation even though I could see a great deal of benefit in it.

My thoughts for not sharing it are,

  1. CFAR has all of this material readily available likely in a much more comprehensive and accurate format. CFAR are altruists. Smart altruists. The lack of anything like this canon suggests that they don't think having this publicly available is a good idea. Not yet anyway. Even the workbook handed out at the workshops isn't available.

  2. I highly value CFAR as an organisation. I want them to be highly funded and want as many people to attend their workshops as possible. It would upset me to learn that someone had read my compilation and not attended a workshop thinking they had gotten most of the value they could.

Replies from: Crux, ThrustVectoring, ScottL, Benito
↑ comment by Crux · 2016-02-15T22:45:23.160Z · LW(p) · GW(p)

CFAR has all of this material readily available likely in a much more comprehensive and accurate format. CFAR are altruists. Smart altruists. The lack of anything like this canon suggests that they don't think having this publicly available is a good idea. Not yet anyway. Even the workbook handed out at the workshops isn't available.

Rather than deferring to the judgment of the Smart Altruists and assuming that within their secret backroom discussions they've determined with logic, rigor, and a plethora of academic citations that it's crucial to the mission of raising the sanity waterline to not release a comprehensive exposition of their body of rationality techniques, perhaps we need only consider your second point except in less reverential light:

I highly value CFAR as an organisation. I want them to be highly funded and want as many people to attend their workshops as possible. It would upset me to learn that someone had read my compilation and not attended a workshop thinking they had gotten most of the value they could.

So much for the Internet-era model of "free information to be disseminated to all".

Without a deferential attitude toward the Great Rationalists of CFAR, Occam's Razor suggests that perhaps they're simply trying to keep the money flowing. Would it upset you if thousands of people without the resources or time to make it to a CFAR workshop had access to a self-study version of the CFAR curriculum?

Replies from: RainbowSpacedancer, Benito
↑ comment by RainbowSpacedancer · 2016-02-16T01:03:04.604Z · LW(p) · GW(p)

Rather than deferring to the judgment of the Smart Altruists and assuming that within their secret backroom discussions they've determined with logic, rigor, and a plethora of academic citations that it's crucial to the mission of raising the sanity waterline to not release a comprehensive exposition of their body of rationality techniques, perhaps we need only consider your second point except in less reverential light.

Given the ease with which CFAR could publish all their material online it seems worth considering why they haven't done so. If spreading rationality wide is indeed their goal, then why haven't they picked this low hanging fruit yet? I'd rather not have to make any assumptions so if someone from CFAR is reading this perhaps they can answer that.

So much for the Internet-era model of "free information to be disseminated to all". Without a deferential attitude toward the Great Rationalists of CFAR, Occam's Razor suggests that perhaps they're simply trying to keep the money flowing. Would it upset you if thousands of people without the resources or time to make it to a CFAR workshop had access to a self-study version of the CFAR curriculum?

Of course that would not upset me. If the CFAR curriculum remained forever available only to the few who attended their workshops that would be sad indeed. But CFAR Labs is currently working on new rationality sequences, and I don't think the curriculum will be as inaccessible for much longer.

I want the world to be a more rational place. I want as many people as possible to have the opportunity to become more rational in the most effective way available. More than any other individual or group it seems to me that CFAR is best positioned to achieve that goal. Even if the reason is money - if that money goes towards increasing the speed at which effective rationality techniques are developed and spread worldwide then all the better.

↑ comment by Ben Pace (Benito) · 2016-02-15T23:19:35.395Z · LW(p) · GW(p)

Pretty sure the main reason for not publicising their ideas so far has been their wanting to get lots of good feedback loops around learning whether it works or not. This year they're planning to scale up a lot how many courses they run, to work with a lot more people, and I think Anna has mentioned somewhere that she wants to write a lot of her main insights up somewhere public. I think they just wanted to take the time to be confident they had good stuff.

↑ comment by ThrustVectoring · 2016-02-17T17:58:13.921Z · LW(p) · GW(p)

CFAR has all of this material readily available likely in a much more comprehensive and accurate format. CFAR are altruists. Smart altruists. The lack of anything like this canon suggests that they don't think having this publicly available is a good idea. Not yet anyway. Even the workbook handed out at the workshops isn't available.

Having it publicly available definitely has huge costs and tradeoffs. This is particularly true when you're worried about the processes you want to encourage getting stuck as a fixed doctrine - this is essentially why John Boyd preferred presentations over manuals when running his reform movement in the US military.

Replies from: ScottL
↑ comment by ScottL · 2016-03-17T11:07:28.774Z · LW(p) · GW(p)

this is essentially why John Boyd preferred presentations over manuals when running his reform movement in the US military.

It's strange that you mention John Boyd because, to be honest, I was thinking of him when I decided to post the material. I don’t believe that John’s preference for presentations over documentation was a good one. In general, I oppose obscurity and restriction of information although there are times that I don’t, e.g. when it’s from a lack of resources or an extremely short material turnover rate etc. In regards, to John Boyd’s stuff, personally, I know that I had to waste a lot of time wading through a lot of simplistic and pretty useless information (pretty much just the simple OODA loop stuff) to understand his material. I believe that this is his only published paper. Also, it was only really the Osinga thesis which has allowed me to understand his ideas. Although, I do need to go over it again.

This is particularly true when you're worried about the processes you want to encourage getting stuck as a fixed doctrine

Wouldn’t most of these issues would be avoided if you gave some warning that the material is in flux and versioned it as well. So, you had a CFAR material version 1, version 2 etc. Also, doesn’t it seem a bit weird to give the potential of the information becoming a doctrine enough weight that it causes the restriction of this information? It seems weird to me since the skills that CFAR and Boyd are/were trying to teach are in large part about breaking out of fixed doctrines. It’s kind of like stopping someone from learning martial arts because you don’t want them to get hurt while training.

↑ comment by ScottL · 2016-02-16T13:11:34.622Z · LW(p) · GW(p)

CFAR has all of this material readily available likely in a much more comprehensive and accurate format.

My assumption was that they don't have this because of time and effort constraints as well as other priorities.

I highly value CFAR as an organisation. I want them to be highly funded and want as many people to attend their workshops as possible. It would upset me to learn that someone had read my compilation and not attended a workshop thinking they had gotten most of the value they could.

The CFAR team are valuable because they are practitioners, experimenters and pioneers, not because of their techniques. That is, they are not valuable because they are hoarding potentially valuable information, but because they are at the frontier and are able to teach their material extremely well. The important question is does my material or yours help with improving the art of rationality and peoples understanding of it. I still think it does, but In retrospect, I think that I should have made it clearer that trying to learn this material by yourself is probably a bad idea.

Replies from: Duncan_Sabien
↑ comment by Duncan Sabien (Deactivated) (Duncan_Sabien) · 2016-02-17T01:50:15.841Z · LW(p) · GW(p)

"trying to learn this material by yourself is probably a bad idea."

I'd say probably a difficult idea, rather than a bad one. Risky, including uncanny valley and disheartening. But that's literally what the generators of CFAR content did, and others can, too.

↑ comment by Ben Pace (Benito) · 2016-02-15T20:49:13.649Z · LW(p) · GW(p)

I strongly endorse (1). I also expect them to change (1) before too long, or otherwise open up their activities much more, and because of these two points, I will not be linking people to the OP.

comment by jimrandomh · 2016-02-18T23:12:45.998Z · LW(p) · GW(p)

I appreciate collections of rationality techniques, and I admire the spirit with which this was made. However, after Duncan raised the possibility of uncanny-valley problems, I cross-checked this with what I remember as a CFAR alum and a few issues jumped out at me.

Hamming circles: This needs a warning. If you organize a group into Hamming Circles and they don't know what they're doing, aren't in the right mindspace, or don't have enough shared context and trust, it can backfire pretty severely. People's Hamming problems are often things that are aversive to think about, and attempting to discuss them but having it go poorly can make the problem worse.

Comfort zone expansion: This is not what CFAR means by the phrase at all. The first link describes a mindful walkthrough, which is something one might do prior to comfort zone expansion. The second link is by someone not associated with CFAR, and it says some things that diametrically oppose things I recall CFAR instructors saying and which I think are objectionable.

Focused Grit: This description is the first step of a 3-step process. Step two is, if after having tried for five minutes you haven't solved the problem, then set another 5-minute timer and spend it brainstorming 5-minute exercises for solving the problem. Then step 3 is doing some of those exercises.

Replies from: ScottL
↑ comment by ScottL · 2016-02-19T11:10:40.779Z · LW(p) · GW(p)

Thanks. I appreciate your input. I have updated the post and I think that should have fixed the issues you have described.