New applied rationality workshops (April, May, and July)

I attended the March workshop. Below is my report.

So, you might think I wouldn't get much value from a CFAR workshop because (1) I know all the instructors personally, (2) I work in the same office from which they develop and test their lessons, (3) I used to be a visiting CFAR instructor, and (4) I'm a Less Wrong veteran with 50k+ karma.

But in fact CFAR's March workshop was one of the most useful weekends I've ever experienced.

A few of the classes had little new material for me, but the classes are just meant to explain the basic tools, anyway. Most of the workshop's value comes from interacting with the teachers and other alumni to figure out how best to apply the tools to your current life situation.

First, let me give a few general notes:

• The curriculum has evolved quickly. Even somebody who attended in May-July 2012 would probably get several new skills + helpful followup sessions from attending in April-July 2013. In particular, the pieces of the curriculum fit together into a coherent picture of rationality far better than they did last year.
• Because the curriculum evolves quickly, some classes are more polished than others, though (AFAIK) all classes have been tested and iterated a few times.
• The continuous supply of good food, snacks, and drinks helped make the workshop pleasant.
• On the last day of the workshop there will be 50 exciting new things you want to do, but you should only try to implement a few at a time, with help from whichever CFAR staffer you do the regular follow-ups with.
• Going forward, I will probably have nearly all MIRI staff members attend a CFAR workshop.

What concrete benefits did I get from the March 2013 workshop? There are many, and there are many promising plans that I haven't yet implemented because I'm implementing only a few at a time. Here, then, is just one of the concrete benefits I got: the Murphyjitsu tool + solved sleep problem.

Murphy's Law says that "Anything that can go wrong, will go wrong." That's not literally true, of course, but it's often the case slight bumps can derail our plans. Hence the need for contingency planning overkill.

This is one reason it's important to model your degree of agency properly. If you have strong free will intuitions, your answer to "What if something unplanned happens?" might be "Well, I'll just use my Free Will to choose to accomplish my goals anyway." But if you're modeling yourself as a kluge of mostly-Sphexish cognitive modules, then you'll say "Outside view says I can easily be derailed by small bumps, so I'd better prepare specifically for each of the most likely failure scenarios, and do some cognitive training now to make sure my brain does the right thing when one of those failure scenarios hits."

One of my top priorities after the workshop was to fix my sleep problem: I was only getting good sleep 1-2 nights a week. I had tried lots of hacks but wasn't doing enough of them consistently enough to overcome my insomnia-ish condition. In my first follow-up session, Anna helped me fix this with Murphyjitsu.

One problem was that I would stay late at the office or my girlfriend's place and then be too tired to go home where I have a good mattress and can run through my whole sleep routine. So for example: What if I'm working late and get too tired to walk home? Never stay out past 12:30am, and tell others to send me to Milvia if they see me elsewhere later than that. Make sure I always have a jacket at the office so that it's not uncomfortably cold if I want to walk home. Also, do the Fermi estimate so my brain can see that it's obviously worth it to hire a taxi to take me home and get good sleep, if that's necessary. Then, simulate in my head the situation of working late at the office and then calling a taxi ("offline training"). Etc.

After doing Murphyjitsu on my entire sleep routine, I now get good sleep almost every night.

Toward a Rationality Dojo

Eliezer's very first post on Overcoming Bias was The Martial Art of Rationality. Quick summary: in the past 50 years we've learned a lot about how to improve human judgment and decision-making (JDM). If you want to become a better "rationalist", you could develop those skills and practice them regularly — just like you would if you wanted to become a better pianist, a better chess player, or a better rugby player.

I've always wanted a Rationality Dojo, but it wasn't until I attended CFAR's March workshop that I acquired a concrete, detailed picture of what that could look like.

Based on my CFAR workshop experience, let me fill in Patri Friedman's list of important Rationality Dojo qualities:

1. "It is a group of people who gather in person to train specific skills." The "gather in person" part of the CFAR workshop was hugely important. I have never been so motivated to diligently learn and practice the art of rationality. Training together in person leverages good news of situationist psychology.

2. "While there are some theoreticians of the art, most people participate by learning it and doing it, not theorizing about it." Few workshop participants know as much psychology as Dan Keys or as much probability theory as Anna Salamon, and that's fine. CFAR staffers are around to answer questions, but participants' new powers come from repeatedly drilling Murphyjitsu, reinforcement training, "goal factoring," Fermi estimates, VoI calculations, quick odds-ratio Bayes calculations, and so on — day after day, week after week.

3. "Thus the main focus is on local practice groups... As a result, it is driven by the needs of the learners." CFAR's early material was driven by abstract considerations of what seemed best to teach. While that's still a factor, CFAR's current lessons have been iterated repeatedly in response to the needs of session participants, and lessons that didn't work, or didn't help people much when tested, have been discarded.

4. "You have to sweat, but the result is you get stronger." Doing a Fermi estimate every day is kind of a burden, but they're definitely starting to feel easier and more natural to perform. One thing CFAR has done well is to break the skills down into concrete steps (often at the 5-second level) so that it's obvious how to practice the skill repeatedly.

5. "You improve by learning from those better than you, competing with those at your level, and teaching those below you." CFAR workshops bring together people of varying skill levels, obviously.

6. "It is run by a professional... The practicants receive personal benefit from their practice, in particular from the value-added of the coach, enough to pay for talented coaches." CFAR makes it the case that there are now several people who have the full-time job of figuring out how to teach people the martial art of rationality. They have time to develop carefully crafted lessons, test them and iterate them, follow-up with people, build a large dataset of what's working and what isn't and what life outcomes follow.

A CFAR workshop isn't yet a Rationality Dojo, because it's not open every day from 10 to 8 with Fermi practice on Mon-Wed-Fri and Bayes practice on Tue-Thu-Fri, but after attending the March workshop I can finally see how such a thing could exist, if there was enough interest in at least one city to have people pay a monthly fee to develop the martial art of rationality with others, week after week.

Anna's "try stuff fast" mindset was the cause of the successful May 2011 "minicamp" that largely gave us the courage to try building CFAR, and also the cause of my own lean nonprofit awakening.

Considering that this is a rationality workshop, the organisers must be commended for doing good, rational marketing by setting the price at $3900 instead of rounding it up to $4000.

Also, we have more financial aid for April than for May and July; so if you've been wanting to come, you can do April (2.5 weeks from today), and money is a bit tight... do apply to April.

It's not quite the appointed time for my one-year retrospective on my thoughts on the January workshop, but if anyone has any questions now about the impact I've seen so far I'd be happy to answer them.

I have a couple of questions:
1) Does the $3900 cost primarily cover the cost of the workshop itself, or is it mostly used as a revenue source for CFAR to keep the doors open year-round?

2) What advantages does the weekend retreat system offer that other systems don't, specifically distance learning?

3) Are there any plans to expand into distance learning, for example into the Khan Academy, edX, Udacity model?

Everything you've mentioned sounds great, but I'm inherently skeptical of a weekend retreat format. After all, even gay cure camps 'work' and have their testimonials; how do you control for the self-help effect? Additionally, it seems odd to me to be building a program around teaching rationality in-person, when every other educational institution is trying to move in the opposite direction. I'm just curious about why you've chosen to structure the program the way it is.

Are there any plans to run them in other countries? Even with the posibility of scholarships international travel isn't cheap.

Not directly related to the post itself, but I thought that I'd drop the link to this recent paper here so that CFAR folks see if it they haven't already: http://www.sciencedirect.com/science/article/pii/S1364661313000405

Although human thinking is often biased, some individuals are less susceptible to biases than others. These individual differences have been at the forefront of thinking research for more than a decade. We organize the literature in three key accounts (storage, monitoring, and inhibition failure) and propose that a critical but overlooked question concerns the time point at which individual variance arises: do biased and unbiased reasoners take different paths early on in the reasoning process or is the observed variance late to arise? We discuss how this focus on the ‘whens’ suggests that individual differences in thinking biases are less profound than traditionally assumed, in the sense that they might typically arise at a later stage of the reasoning process.

Since the potential for a "those who can't, teach" thing seems to be a sticking point here, I wonder how hard and/or legally complicated it'd be to set up a controlled experiment. Keep the $3900 price point for classes that are guaranteed to represent CFAR's best effort at providing rationality skills, but also offer a limited number of slots at a lower or zero price point that might be a best effort or might just contain impressive-sounding platitudes of the sort that you hear in your average motivation seminar. Follow up with students after six months or a year and see what the deltas on achievement are.

They offer a refund if you attended a workshop and didn't like it. If you didn't... what are you talking about? :-)

At $3900, it's an investment, but a low-risk one, since we have a money-back guarantee. If you don't feel like what you got out of it was worth it, we'll refund your money without hesitation or complaint.

This is still not low-risk. I would hesitate to ask for a refund even if an event like this was below my expectations, as long as it's not a total flop or a con, which it surely isn't. Low-risk (for the participant) would be dividing the camp into billable events with a price tag on each, and refunding a portion of the price of each event based on the post-event evaluations. This is probably unworkable in practice, but at least it would not be misleading. On the other hand, "full refund no questions asked" is a useful marketing strategy, if a bit dark-artsy.

At what point do you guys estimate CFAR will scale such that economically disadvantaged individuals such as myself will be able to afford a retreat? In the next few years will there be more of a focus on making money off of increased demand from business, heavier-pocketbook type individuals, or lowering costs for hungry student types?

I would love nothing more than to go, if only it was cheaper.

It is great to have the dates. I had been hoping for an event later in the year. What are the odds that other mini-camps will be run between September and December?

Hi, the "apply here" link is not working for me.

Thanks!

Here are a few write-ups

First link ("toner") is broken (missing domain name). Please don't reply to this comment: I plan to delete it once the link is fixed.

I also made a simplified map of some of our classes, so you can see how I think of them fitting into the bigger picture of rationality (click to enlarge):

I've clicked everything I can think of and it's not enlarging.