POSITION: Design and Write Rationality Curriculum

As an aside; the use of "Org" (i.e. Rationality Org) seems really unusual and immediately makes me think of Scientology (Sea Org); am I unusual in having this reaction?

Sometime ago you believed, correctly IMO, that you need a way of testing rationality skills first, and only then get busy on the exercises. What made you change your mind? (I hope it wasn't something like "we need to push ahead asap".) What's the current plan for preventing the slide into epistemic viciousness? (I hope it isn't something like "we will be smart and won't let it happen".)

If there were a world in which algebra had been learned only through reading essays, without subskill-by-subskill practice, it would not be surprising if the world’s best algebra practitioners could be outperformed by an ordinary student who worked diligently through the exercises in a standard textbook.

This actually happened. The ancient Greeks weren't very capable algebraists because they didn't develop a symbol system that they could systematically manipulate according to prescribed rules. Their description of formal logical inferences were insane to read: "If the first and the second, or the third, implies the fourth, then the first or the fourth, implying the third...." The reason our word "algebra" comes from Arabic isn't because the Muslims were better algebraists; it was because they used symbol systems (to avoid making icons of Mohammad) in order to encode the material they were reading in the Greek literature. The result was something reasonably close to our modern symbol-manipulation system, which made it possible to train in algebra.

So this isn't just a theoretical example. Really, honestly, the first textbook ("al-jebr..." I don't quite remember the title) absolutely trounced several hundred years of careful, intelligent Greek thought on the topic of numerical reasoning.

Edit: Please see this. There's some question about the accuracy of my statement here.

I'm going to apply for this role.

Example generation. Given an exercise, we need someone who can think of lots of specific examples from real life or important real-world domains, which illustrate the exact intended point and not something almost-like the intended point. E.g., turn "Sunk cost fallacy" into 20 story snippets like "Lara is playing poker and has bet $200 in previous rounds..." (Our experience shows that this is a key bottleneck in writing a kata, and a surprisingly separate capacity from coming up with the first exercise.)

...so, who else just opened up Evernote and started seeing how many they could bang out?

We're looking for 1-2 fulltime employees who can help us build more things like that (unless the next round of tests shows that the current format doesn't work)

Nice to see you taking to heart the lesson you taught MoR!Harry:

"So what's next?" said Hermione.

Harry rested his head against the bricks. His forehead was starting to hurt where he'd been banging it. "Nothing. I have to go back and design different experiments."

Over the last month, Harry had carefully worked out, in advance, a course of experimentation for them that would have lasted until December.

It would have been a great set of experiments if the very first test had not falsified the basic premise.

Harry could not believe he had been this dumb.

"Let me correct myself," said Harry. "I need to design one new experiment. I'll let you know when we've got it, and we'll do it, and then I'll design the next one. How does that sound?"

"It sounds like someone wasted a whole lot of effort."

Thud. Ow. He'd done that a bit harder than he'd planned.

I'm not sure if this is the right thread for this, but I don't see how example #11 in the "Skill 1: Noticing Sunk Costs" PDF is an actual example of the sunk cost fallacy.

Ellery's options are to eat the slightly-burnt soup, or not to eat and instead to spend $12 (and, presumably, some cooking time) to replace the damaged soup with good soup and eat that instead. She has to eat something (again, presumably); the option to simply abandon the sunk cost, by discarding the soup and then eating nothing, seems implausible. In other words, her decision whether to eat or not eat the soup directly affects her future financial situation, and her decision is based on that effect.

Am I misunderstanding something?

EDIT: Of course it's possible that the negative utility Ellery would get from eating untasty soup outweighs the positive utility from eating at all plus the value of $12, but in any case the question concerns future utility, right?

(My apologies if this is a thread derail.)

Please talk to David McRaney (http://youarenotsosmart.com) to see if he'd be interested. His recent book, while far from comprehensive, has become the first place I look whenever I want to reference an accessible explanation of a particular cognitive bias.

Would it be helpful for us to try out these exercises with a small group of people and report back?

This forum has a pool of people who have many of the talents you requested, but who would not bother applying for one reason or another (unwillingness to overtly commit a significant chunk of time, general akrasia, you name it). However, they would be happy to comment on a single focused question, be it to suggest an activity, to improve a write-up, to do a literature search, or to polish a presentation. I am not sure whether you have anyone on staff who is good at partitioning a project into tiny little tasks of under-an-hour length, but if so, consider an LW section or a post tag [Rationality_org Task] as a means of tapping the low-availability talent pool.

Looking at the sunk cost materials, the answer to question 10 in booklet 1 seems a bit off:

Question: Sandra is starting to grow bored with her current boyfriend. Is it time to move on? She left her job in Los Angeles and moved to New York, just so that she could be with him—but they haven't had a really interesting conversation in months.

Answer: Sunk cost yes, sunk cost fallacy yes.

The time and effort it took to move are already gone and not coming back. Sandra's old job is already gone, also.

Sandra is acting as though her past costs are relevant to her present decision.

I would have marked this as "sunk cost yes, sunk cost fallacy no". The way I read it, Sandra's sunk cost of her leaving her job and moving is mentioned as background information, but it's not stated that Sandra would consider this to be a reason to stay.

The presentation and exercise booklets seem to be pretty awesome.

One obvious idea for an exercise is MBlume's Positive Bias Test, which is available online.

But of course everyone taking the course would probably already be familiar with the standard example implemented in the app. I would suggest updating the app to have several different patterns, of varying degrees of complexity, and a way for the user to choose the difficulty level before starting the app. I would expect that to be not too hard to implement, and useful enough to be worth implementing.

Shouldn’t the presentation suggest what sort of reasoning is actually going on in someone’s head while they are committing the sunk costs fallacy? For example, (skill 1, problem 12) Peggy is at a baseball game with her son Tim, who is bored and wants to leave. Peggy says, “You want to leave? Those tickets were expensive!”. Her expressed thoughts are a fine example of the sunk costs fallacy. But she may be really thinking: “This is event is the sort of thing normal people pay a lot to attend - Tim must not be thinking clearly. We should stay because he will enjoy the game, even though he says he’s bored.” If the presentation pointed out that (unconsciously) applying the scarcity heuristic can lead to or sound like the sunk costs fallacy, rationalists-in-training could more easily catch themselves committing the fallacy.

Also, learning to distinguish solid and fallacious reasoning, even when they produce the same conclusion, helps one correctly cash out the Teleporting Alien viewpoint. Tim responds to his mother, “We’ve already spent the money for the baseball tickets. I’m bored so there’s no point in sitting through the game.” Peggy realizes that she is committing the sunk costs fallacy. She tells Tim, “You’re right. But this baseball game is expensive - which makes it likely that it is entertaining in some way. Let’s stay a little longer and see if something exciting happens.”

The obvious problem with trying to fully explain the reasoning in sunk costs situations is that you have to explain those other biases and heuristics. But I think the exercises can briefly mention what to look out for and reference other kata while remaining focused on the particular skill at hand. And indeed, rationality skills are dependent, which should be reflected even in a training course with a narrow scope.

Since a few people have asked me privately: if you're interested in this position, please send in an application even if I already know you. The application is short, and it will let me know that you are interested, and what parts of the work you are interested in and/or able to do.

question 2 sheet 5.

"Answer to 1. Disliked story: Bought a cupboard full of yucky peanut butter. Preferred story: Cleverly purchased good peanut butter in bulk. Fixed reality: You bought a year's supply of a peanut butter you haven't tried, and it's either tasty or yucky; if yucky, pretending to like it and choking it down won't get your money back."

That reality isn't fixed. Tasty/yuckiness is in part determined by your attitude towards the peanut butter. Tastiness is not a one place property. More generally, you can make yourself like different things. Doesn't just familiarity make you like things? There's a large grey area between tasty and yucky that includes a section for "has the potential to be tasty if I decide to damn well like it."

Sometimes, yes, you can make something not be wasted. The point is to reason from the situation as it is, not to avoid making sunk costs not a waste. As things go food is pretty easy to choose to enjoy, or at least not mind.

Having seen many concerns about the low salary/skill ratio: it seems like about the same situation that I had for my graduate degree. I took half the salary for a job that was (by my estimate at the time) twice as cool as the alternatives. In that light, the position seems like quite a good deal, as most grad students don't earn that much. If you expect to experience side-benefits from this job, such as benefiting from increased rationality, or having cool stories to tell people about your interesting job, then this seems like a good deal.

Quick rewrite with some bolding to try harder to correct people's impressions that we're trying to hire someone who can do all these things simultaneously.

The formatting is all messed up for me; the text in some later paragraphs is spaced super far apart and runs off the edge of the page. Might be an Opera thing, but I've only seen this on LW, and I've seen it before. Can someone remind me why Markdown is not used for top-level posts?

Here's what it looks like for me.

I would like to have this as an interactive application. It's not always so easy to get into a teacher student situation.

There are applications out there for developing this curriculum in a better way than the regular approach. Like the app these guys are selling http://www.knowledge-values.com/ I use apps like this on a daily basis to create the training content at work.

I don't need a whole game or anything, something as simple as the math exercises on khan academy will do the trick. And i would prefer each skill told as separately and atomically as possible/logically.

The salary for this position seems off by a factor of between 3 and 4 given the sort of background you want. You're asking for someone with professional level design skills coupled to the skills of a university professor or really good high school teacher or video game designer (depending or your perspective). People with these skills get paid a lot more than you're offering. $36k/yr isn't going to get you a bright recent college grad, especially if they have to live in the Bay Area.

It seems to me that you're more interested in hiring folks that are deeply dedicated to the movement so that you can pay them a sub-market salary than hiring the best person you can find. Which is fine, but you should be upfront about it.

Any idea when you will be choosing candidates? How long should an applicant wait before assuming they were not accepted?

I'm pretty sure there's a mistake in the booklet at link 5 - the preferred story and the disliked story are the wrong way round in the example.

The iPhone app example in the presentation confuses me.

The way it is presented, it looks like the conclusion is that you should always be willing to spend an additional $6999.99 no matter how much you have already spent. If current you is willing to spend the extra money regardless of whether you have already spent $4000 or $10999.99, then I don't see why future you would feel any different.

I would think that you should take in to account the fact that your original estimate of the cost was too low. Given that this is the case, you should expect that your current estimate of the cost to finish is also too low. I would multiply your cost to finish estimate by (current estimated total cost) / (original estimate) and only continue if that result is less than $7000.

Going over this in the presentation would introduce complications to the problem that would most likely lead to even more confusion, but when the details are left out, it looks like you are endorsing the same decisions that the sunk cost fallacy would lead to. I suggest changing the example to something else entirely.

Social initiative enough to gather guinea pigs and run many practice trials of draft curriculum, while collecting data.

This seems like something that existing LW meetup groups could do. Perhaps you could send prototype lessons to meetup organizers who would be willing to participate?

The formatting for this page is really broken for me. I use Opera 11.whatever the latest is.

I am entirely incapable or many of the things mentioned, I am unable to commit or fully participate due to helth and geographical reasons, and a bunch of other things...

On the other hand I have a feeling feeling of ability to be useful, despite all those things. This kind of deliberate growing of intuitions and making the brain do some specific thing is right up my alley and I might be very good at coming up with certain types of examples or providing a certain kind of hard to articulate meta intuition. I also know a bunch abaut art and design.

I'd also likely find it great fun and far more meaningful than anything I normally do.

So this is a bit of a dillema, what should I do?

The "rationality" link in the "Bonuses" section has become broken.

From the booklet at link #3:

If you’re going to eat sinful foods, eat high-quality chocolate rather than potato chips—don’t waste calories.

But what if I enjoy potato chips way more than any chocolate ? This piece of advice sounds weird on all kinds of levels...

About how much detail are you looking for in the free response sections of the applications?

I just wanted to say that the presentation is wonderful. It's a refreshing change from the usual posts and I would love to see it continue. In fact you should stop the other posts or let them post them on their own blogs or something. This type of content will make a difference in the world, the other guy is more like a well-researched "top 10 ways to lose weight" type of article and has limited tangible benefit.

More please. This is what lesswrong should be about!