3 Levels of Rationality Verification

Well, you asked for DUMB ideas, so here's mine. It has the advantage that I'm sure no one else will suggest it. This is based on an accidental discovery (so far as I know, unpublished) that one can compare two arbitrary documents for similarity (even if they are in different word-processor formats) by running them both through a recognizer built out of a random state machine and comparing bit masks of all the states traversed. The more common they are, the more states will be traversed in both.

So, lets assume we have a panel of highly rational individuals which are our control group. We generate a random multiple-choice questionnaire consisting of nonsensical questions and answers. Things like:

1) How Green is the Smell of Bacon?

a) 7.5

b) Neon

c) Introspection

d) Larger

You then do a correlation over how your panel of experts chose their answers and see if there is a common pattern. You then score students who take the test based on how similar to the common pattern they are.

Assuming this idea works at all, the advantage of this is that it would be extremely difficult to game. The disadvantage would be that it would penalize those who are significantly more rational than the 'norm'. It would probably also require the panel to be similar to each other in cognition. There is also the general problem of not knowing if you're really testing for what you think you're testing.

Frankly, I don't know if I'd be more happy if this was tested and shown to be workable, or if it turned out to be a really stupid idea.

Occasionally, well-respected community members could say things that are intentionally false, but persuasive and subtle, a la http://www.overcomingbias.com/2008/02/my-favorite-lia.html.

You get points for catching these mistakes. Perhaps you submit your busts privately to some arbiter so others have the same challenge.

Later, the error is revealed and discussed.

This would also have the benefit of causing everyone to read the most-respected members' writings ultra-critically, rather than sitting back and being spoon-fed.

One key thing this idea has is short term feedback. Frequent, rapid feedback is essential for getting good at this kind of thing. (IMO that's why economics is still so useless relative to the other sciences: the experiments take fifty years to run.)

For 'hot' political and religious biases, create materials in which apparent advocates of different ideologies or parties are arguing for some particular empirical prediction, e.g. about the relationship between different tax rate changes and economic growth, with some predictions being right and some wrong. The subject then needs to make his or her own prediction about some easily-verifiable but obscure empirical fact related to the argument, e.g. whether a graph of GDP and tax rates matches Norway or Iceland.

Scoring would reflect the degree to which the ideological affiliation in the prompt biased the results. If it was being gamed you might need to add in scoring for accuracy. Challenges would be producing a large enough inventory of test items, keeping them secret, and the need to tailor tests to locally popular ideologies or ideologies of interest.

More surveys that study the relationship between knowledge about verifiable facts and values. What sorts of information do those with different values tend to have, and what are the values of those whose knowledge covers the pet facts of all camps? There is a fair amount of this literature in political science aimed at the electorate and its political knowledge, but it would be good to extend it to other topics, e.g. scientific ones.

Announced probability distributions (not just predictions, so as to enable better scoring) for the results of upcoming experiments. For instance, we know that in the next 2-3 years we are going to get a huge amount of genomic data that will answer a lot of questions about the genetic architecture of human diseases. Making public quantitative predictions about things like that could be quite informative.

People tend to compartmentalize. We need to bear in mind that anything we come up with that involves testing someone when they know they're being tested can only check how rational they can be if they put their mind to it, not how rational they are when they're not being tested.

I think that the most important skill a rationalist can have is the ability to assess the quality of other rationalists, and to participate effectively in team projects. A measurement of individual rationality has to include how well a randomly selected team including that individual performs on team rationality tests.

So, I think that a rationalist 'decathlon' would consist of a variety of competitions between individuals and small teams including math/logic problems, general knowledge tests, cooperative and non-cooperative game theory games, prediction markets, and engineering challenges (egg drops, programming robots to compete in some arena, etc.)

But then there would be a second level, in which individuals and teams would compete in a prediction market in which they observe (by video recording) the deliberations of other teams on first-level problems and bet on their relative performance.

And even a third level, in which individuals observe the deliberations of second-level teams and bet on their performance in that second-level prediction market.

There are a variety of other things that might be interesting to measure - for example, what team sizes perform best, whether individual rationalism and team-participant rationalism are different skills, and whether team performance is best predicted by strongest member, average member, or weakest member.

Hrm... Well, one initial notion I have is along the lines of this: Rationality training should improve how good one can become at other stuff, or at least improve ability to gain skills/etc in other fields.

So, maybe tests could be something along the lines of find various subjects/fields a student is unfamiliar with and basically assign them to "get some knowledge and skill in this field."

How efficiently students can basically bootstrap up into something they're unfamiliar with should vary with their rationality, right? So something like this may be a starting point.

(Yes, I can see a bunch of details that would need to be worked out, but seems to be that this notion may at least be somewhere to start for developing rationality tests.)

Organize large games/contests where a lot of candidates are locked up in an area, and have a finite time to reach a certain point / find a certain object.

The exact rules would be specially designed each time for that years challenge, by a group of rationalists and game designers. So the details would vary, but some common themes would be:

• physical prowess does not come into play (beyond maybe moving around faster, not getting tired as easily etc.)
• some people would be liars / saboteurs, and not real candidates

For example, the candidates are blindfolded and brought into a large underground circular room, whose only unlocked exits are twenty slides along on the edge (so, one-way exit only). The goal is to take the exit that's due north.

Or, the players are dropped in a maze, and each player is given twenty balls with his name written on them. In the maze are tall glass tubes in which the player can drop their balls. The players know that at the end of the games everyone gets points for the balls with his name that are in "good" tubes (from 10 to 1 points, depending on whether his ball is at the bottom or top - only ten balls fit in a tube), and loses points for balls in "bad" tubes (whatever it's position). There are also neutral tubes. On the tubes are various signs and portents, and on the walls are statements about the the meanings of the signs ("about 10% of good tubes have red triangles", "two squares of the same color cancel out", "a blue triangle means that there's a bad tube close to this one"). The players have 30 minutes to place their balls.

Additional twists:

• there are in fact several simultaneous games taking place, in the same place, but the rules are such that it's very difficult to tell who's part of which game (for example, if some players' goal is to unmask/identify other players)
• the goal may not be reachable at all (no candidates accepted this year). The "global" rules of the contest might include that there must be a certain probability each year (10% ?) that the contest is impossible.
• candidates are not alone but in teams

... well, there is plenty of inspiration to take from board games and TV shows. And many factors of those can be controlled by careful design (importance of luck or of trivia knowledge, how much "herd behaviour" can come into play, etc.). The games should be more complicated than what's said above, and contain many red herrings. The designers should try to introduce as much sources of bias and irrationality as possible.

I'm not sure why "teaching to the test" is so disparaged for its effects on the learning process. Obviously that is a different use for tests than evaluation of ability, as is the main goal here.

Studying for the LSAT taught me to feel genuine physical unease when I read a bad argument, then be calm it by the next problem. It's very hard to turn that off when reading the newspaper.

The third stage of my growth as a rationalist was discovering this site. I no longer go through the day thinking of things I read and hear: "Wrong (fallacy), wrong (incorrect premise), wrong (fallacy), true (but irrelevant)." Now it's more like: "Wrong (fallacy), not even wrong (internally inconsistent), wrong (map/territory confusion), wrong (fallacy), not even wrong (argument from definition)."

I propose thinking of ways to hijack the human mental machinery as an alternative to overcoming it, akin to what evolution does.

Frank Mager, in various books, including "Preparing Instructional Objectives", suggests working backward from evidence that would make you conclude that someone is, e.g. a Bayesian Master Rationalist, to the tests (and instructional objectives) for a course of instruction intended to turn someone into a Bayesian Master Rationalist (or whatever you want to turn them into).

Compile a large enough database of historical events that nobody could memorize more than a fraction of it. For the test, choose a few events at random, describe the initial conditions and ask the candidate to predict the outcomes.

Carry around a notepad, form probabilistic opinions on lots of little questions that you can find out the answer to soon after, record all the probabilities assigned to correct answers, where applicable add tags like "politics", "project completion", "my social status", "trivia", put into a spreadsheet or something and see if you're miscalibrated globally and for different tags.

Here's a stupid idea: Evaluate people by auditing their domiciles. I've read (and from personal experience, I believe it) that you get really solid insight into someone's personal qualities by inspecting their home, as good as interviewing them and all of their friends and family. (I googled a bit, but I can't find the source.)

Anyway, it can probably be gamed.

Here's an immoral one: crack a rationalist

Most, if not all, human minds are vulnerable to hacking, eg by cults, religions, pseudoscience, etc. The minds of rationalists should be harder to hack than others.

Make a copy of a (would-be) rationalist, subject the copy to significant emotional stress, and then send missionaries his way.

The myths carried by the missionaries should be invented for the challenge so everyone can agree that they are false, but should, of course, be significantly more plausible than today's religions.

There are two problems with measuring rationality, one of which is difficult but manageable, the other of which might be insurmountable. The first problem is that most conceivable tests of rationality require using information from other fields (such as finance, physics, or psychology), such that you can gain a considerable advantage on the test by studying things from that field which don't actually make you more rational. This can be solved with sufficient cleverness.

The second problem is that how rational someone is depends on how well they maintain it under stress. Pressure, fatigue, emotionally charged situations, alcohol, and/or deliberate manipulation, can make the best rationalists act completely insane. (About a year ago, I went on a reality television show, which was in a way like a series of rationality tests. I didn't do all that well, rationality-wise, but some people who should have known better did dramatically worse.)

Give the students sodium pentothal and ask if they're one of the top 50% of rationalists in their school. However many out of 200 say 'no', that's the school's percentage score. Schools scoring over 100% are thrown out for cheating.

Ask a thousand married rationalists of a given school to estimate the probability that their spouses have cheated on them. Confidentially ask their spouses if they have. Measure group calibration.

ETA: This applies to any potentially painful, but verifiable question. Ask them to draw a probability distribution over their date of death, or the longevity of their marriages. Estimate the probability of various kinds of cancer appearing over the next (5,10,15) years, etc. etc.

(I consider it drop-dead obvious that the task of verifying acquired skills and hence the power to grant degrees should be separated from the institutions that do the teaching, but let's not go into that.)

Was/are there any organizations that are just dedicated to verifying rationality skills? CFAR tried to do both IIRC. Seems pretty bad if there haven't been any attempts at this even.

Use small-scale, limited-term betting markets with play money.

Put the group of people you want to rank relative to each other into a room - without internet access. Everyone starts with 0 points. People are ranked on how many points they have at the end of the test.

Participants make bets (for points) with each other. There's a time limit for settling those debts; all bets made have to be specified in a way that clearly determines the winner within a fixed period after the end of the test. Of course, bets that can be settled immediately (e.g. on current trivia, history or fiction) are also permissible.

Aside from that, there's no limits: Any time two participants agree they want to bet against each other, on whatever they specify for however many points they choose, they can register that bet.

For instance, Alice and Bob bet on the temperature as reported by for at 6:00 local time, monday after the test:

• Bob will pay Alice 5 points if the temperature is at most 20 degree Celsius
• Otherwise, Alice will pay Bob 20 points.

After enough time has passed for all bets to be settled, have a trusted third party determine the winner for each, tally up the points and rank participants by final score.

This game is absolute zero-sum: the only way to earn points is by taking them from another participant. Test runs and outcomes can be published without obviously weakening the idea: If there's something to be learned from previous rounds, all participants have a chance to learn it.

Studying obsesssively on certain subjects may help you, but only to the point that other participants don't know you've done it: If everyone knows that you are a major Higurashi no Naku Koro ni fan, they're unlikely to bet against you on that subject - or if they do, they won't bet very much.

Edit: Thinking about this some more, this kind of test has a failure mode: There's a strong incentive not to bet against people who are better at tests like this than you, so with sufficient information about the players the entire game may freeze up: For every possible bet, there's somebody who expects to end up worse off, no bets get made and everyone always walks out with 0 points.

Possible solution: Keep participants anonymous to each other during each test. If nobody knows who they're playing against, there's a higher chance they'll be willing to make some bets.

Well, there's always the idea of using fMRI scans to determine if someone is thinking in 'rational' patterns. You stick them under the machine and give them a test. You ignore the results of the test, but score the student on what parts of their brains light up.

Clearly real life achievement correlates well with rationality, by definition. So an impractical but "gold standard guaranteed" test of rationality would be to wait until the person in question got to the age of, say, 50, and check to see whether they had made lots of money, or achieved other obvious life goals (fame, for example).

A more specific good test of rationality is the world of startups. Other than the OB/LW community, the entrepreneurial world is the closest to perfect rationality I have found. You could test someone in a month or so by asking them to enter a startup competition, like this:

http://www.cue.org.uk/5k-Challenge

Again, probably not so practical.

The kind of challenges you find on Alan Sugar's "The Apprentice" are fairly rationality oriented, and administrable in a few days.

Potential rationalists could be tested by putting them in the position of a venture capitalist/angel investor, and have a combination of real business and fakes come to pitch at them. This test could be made harder by supplying some of the fakes with convincing cover stories and the real businesses with poor presentation skills, forcing the rationalist to concentrate on the merit of the underlying idea rather than superficial clues. It could be made a better test by allowing participants to do their own research beforehand, and giving them the opportunity to hire and consult "experts" some of whom would again be fakes.

In general, rationality tests have to take a long time IMO, because genuinely creative thinking takes a long time at a serial speed of ~10Hz. Any "test" that takes a few hours (like an exam) is just going to be regurgitation of previously memorized material, or activation of previously trained-up hardwired circuits, a la "cached thoughts". It seems to me that a day long test is the absolute minimum.

Of course you could also administer a classic academic style exam on the OB material, cognitive biases, etc. But someone could do very well on that without really understanding it. Still, it would provide some indication of real life rationality performance.

I don't see what I thought were the obvious answers, so here they are. The foundations are elsewhere on the site, but they seemed missing from this list.

Reputational: Expect Bayesian masters to participate in other scientific fields. People who make more discoveries in other fields get more street cred among rationalists, especially when they can explain how rationalism helped them make the discoveries. Obviously, this is a long-term process that doesn't lend itself to improving the art quickly.

Experimental: This one's a two-step process. First, ask a large collection of university professors to insert one lie into each of their lectures a'la http://www.overcomingbias.com/2008/02/my-favorite-lia.html (mentioned in another comment). Have them note which students discover each lie, but don't have that count for any sort of grade (to prevent gaming). Second, sort students randomly into the experimental rationality classes, and/or have the classes "fill up" (with a lottery for seats) to provide a control. Look for whether there's a difference in lie-detection rates between the differently-taught groups.

Experimental #2, much longer term: Track the career outcomes of the students who took each different rationality class. See whether there's a difference in winning between the groups.

"Piggyback" on other tests: ask people taking part in various tests (standardized exams, sport competitions, driving lessons, programming contests, art exhibitions - whatever) their chances of success (or their probability distribution over the range of results).

The other items should themselves be important enough, so it would fit well with a university cursus, so that it can be "automated" for a lot of things. The way of asking for predictions should be made so as to maximize bad predictions: for example the students are asked to give estimations in front of their peers (if that's shown to get them to overestimate), but afterwards not reminded of the prediction they gave nor of whether it came true (so that they don't deliberately try to make it come true).

It could also be extended to other events like "when I'll turn in my thesis" or even "whether I'll be single in a year" or "how much I'll weight in six months".

The more subjects they have to estimate on, the better. At the end, measure the Bayes-score.

This could be combined to some more "dramatic" and explicit rationality tests (see the other comments) to constitute the scoring method of a university rationality course. The explicit rationality tests would also help take a bit of attention away from the day-to-day probability estimates on exams and stuff, to diminish the "only rational when deliberately thinking about it" phenomenon.

Oh, also - ask the students for an estimate before the exam and after the exam (but before they have a chance of talking to someone else). Maybe even a week before and a week after too.

There is a recent trend of 'serious games' which use video games to teach and train people in various capacities, including military, health care, management, as well as the traditional schooling. I see no reason why this couldn't be applied to rationality training.

I always liked adventure style games as a kid, such as King's Quest or Myst, and wondered why they aren't around any more. They seemed to be testing rationality in that you would need to guide the character through many interconnected puzzles while figuring out the model of the world and how best to achieve the goals of the protagonist. It seems like the perfect video game genre for both developing and testing rationality skills.

Specifically, I've thought of a microcosm of the real world, taking place in a different setting yet similar enough to our real world that there would be analogues to religion, science, politics, etc. As you progress through the game, say from child to adult, you learn about the world and see how different beliefs and strategies effect the game. Players would encounter similar challenges to the real world but be disconnected enough not to put up a defense mechanism, yet involved enough to care about the outcome. Add MMO et al features to taste.

I'm not sure if this has already been said, but does the "biases" literature not already contain a lot of perfectly good (although probably overly game-able) rationality tests? Just pick an experiment at random from Tversky and Kahneman and see how well the people in the school do.

Of course, there is a problem of people learning how to do some of these tests, but I'm pretty sure there are some that could be reworked so that they're pretty damned hard to pass even if you're well-acquainted with the literature. I'm thinking particularly those where half of the subjects are asked a different question to the other half, and the results compared - e.g., tests for the Lake Wobegon effect, for Social Attribution Bias, etc.

(haven't looked through comments, so this may have been suggested many times over)

In a college-level rationality course, it would be most appropriate for a portion of the grade to be determined by an artificial economy. That is, set up a currency and a (relatively even) starting distribution, add (probabilistic) opportunities for investment (perhaps linked to other important parts of the course) and, most importantly, make defection possible, anonymous and easy. Make it, as much as possible, like a vast array of one-shot (or known number of iterations) Prisoner's Dilemmas.

Then allow students to organize into institutions with rules. Well-taught rationalists should be able to construct a very strong economy along these lines; poorly-taught ones will be only rational enough not to cooperate out of an irrational sense of honor. A student's final grade on that component will be the logarithm of their final wealth, curved as little as possible.

It would take a well-designed setup, of course, to ensure that we're truly measuring rationality and not (say) merely group cameraderie; but I think it could be worked out in a satisfactory way.

The main upshot of this as regards rationality verification: if two different rationality curricula run the same economy setup, a consistently better growth rate of one class economy is evidence of the second kind that more complete rationality is being taught. The students have a much bigger incentive towards their own grade than towards the reputation of the class, so it should be a pretty decent test.

I'm tempted to say "have them play poker", except it uses lots of domain-specific knowledge as well as general rationality. Perhaps if you could generate random games from a large enough space that people don't build up game-specific skills, and the games just end up testing general rationality? While poker-like games don't test all aspects of rationality, there are some things like "ability to keep making good decisions when frustrated / bored / angry" that these games test very well.

I think people would develop skill at the whole class of games...but at the same time, they would be improving their rationality.

Maybe there is a simple thing, which rational people can't do - always get wrong.

Some not very good examples could be:

Skipping with closed eyes.

Telling a lie to a stranger without it being discovered

Saying - "Ooops, I' m wrong," quickly enough

Going to church and sitting thru' a whole sermon without getting very very upset

Multi-tasking

Irony

Understanding metaphors metaphorically.......

Another key feature of [edit] group rationality is the ability to not be swayed by what the social group thinks.

There are simple experiments (though I cannot think of the relevant keywords) where a test subject is put in a room full of confederates, all of whom estimate one line segment to be longer than another when the two lines are in fact the same length.

EDIT: Conforming to the group opinion (on average) increases the probability that you are right, thus improving individual truth-tracking. But adding more conformers to the LW community just screws it over. In the limit of infinitely many perfect conformers, the community would display irreversible belief hysteresis; as soon as >50% of the group believed X, all the conformers would switch to believing X, and they would stay that way.

EDIT: The Asch experiments found that only 25% of people would consistently report the truth their own eyes were telling them. Thus most people don't make good group rationalists.

This comment thread makes clear the need to distinguish between creating good individual rationalists and good group rationalists. Optimizing for group rationality and individual rationality are different tasks.

A gold-plated ignorance prior will be awarded to the first commenter who finds the link for me...

Let's see...

• Prediction contests are an obvious one.
• Also, perhaps, having people compete at newly designed games, so that everyone has the same amount of time to learn the rules and how to win, given the rules.
• Perhaps we could design puzzles that intentionally have places where one would make a mistake, error, or wrong choice, and such errors are visible (to an observer who knows the puzzle) when made.

deleted

Give them a motivation that is higher than the drive to game the test. I'm an immortalist. I don't want to die. I could deceive myself and others in many ways about my skills, purposes, beliefs, but in the end I can't do that at the expense of my chances of not dying. Finding a similarly important purpose, something that might even be gamed, but for which gaming means you loose. Some real life test.

Maybe, measuring someone's capability to win. I have often wondered if being rational correlates with being succesful in society. I can't be sure, though it seems to be it should, if it doesn't then I suppose it either means there's a problem with a rationality that would leave you worse off, or more likely, that you aren't being rational enough, or do not have enough mental ressources to use that rationality to make a difference. Bounded rationality, always an issue.

Capability to win could be measured in many ways, economical success for instance, or any other existing societal position of power or prestige. Of course any single of those may be gamed, but it's ok to cheat, if cheating brings you closer to what you want, then it is rational to game. However, the goal that you hold, and for which you are vying, may not be very interesting. Empty fame, etc.

It would be best to have a personal goal set, and known, and measure how a person fares as to that goal; a goal difficult enough to require the proper use rationality to win in society, that would require to apply rationality to a very large and diverse bunch of situations, a goal that you'd want to preserve.

Can't help much to determine what that would be, I have my own thing to protect, as I said, not sure what it might be for other people. It doesn't work all the time either. Sometimes short time goals are vying for dominance over my actions, and I'll give in to them, even if it means getting farther from my own personal long term goal. That's a lack of willpower, not a lack of rationality at work there I think.

Send rationalists to do consulting work where real money is involved, for example techdirt:

http://www.techdirt.com/

The Techdirt group blog uses a proven economic framework to analyze and offer insight into news stories about changes in government policy, technology and legal issues that affect companies’ ability to innovate and grow.

Here you basically get paid for good insights. A "team" of rationalists could be sent in to dominate this particular arena, thereby validating the technique. Basically any online arena where real money can be made is fair game. Trading in Second Life, for example.

A friend of mine, the most consistently rational person I know of, once told me that his major criteria for whether a piece of information is useful is if it can allow him to forget multiple other pieces of information, because they are now derivable from his corpus of information, given this new fact.

I have a vague feeling that there should be a useful test of rationality based on this. Some sort of information modeling test whereby one is given a complex set of interrelated but random data, and a randomly-generated data-expression language. Scoring is based on how close to optimal once gets on writing a generator for the given data in the given language.

Unfortunately, I think this is someone one could explicitly train for, and someone with knowledge of data compression theory would probably be at an advantage.

An interesting idea would be to feed people the scientific data that ancient or medieval scientists had and see whether they reproduced all the incorrect but (given the limited knowledge) plausible theories that were invented.

This would work especially well on the vast numbers of people in our society who don't know any science anyway.

In fact just finding some sufficiently obscure area of current science would suffice. There's so much of it... How much of contemporary paleontology or inorganic chemistry could I re-invent?

I once succeeded in deriving the solution to the cubic and quartic equations invented by Tartaglia in the 1500's. It took me all day and almost the whole night, plus some tidying up the next day. I think it made a good exercise in rationality, and was harder than I thought it would be.

Related to this, many elite universities face the same kind of problem that we have here: they have to put a lot of effort into sorting out the good wrote learners from the creative thinkers in their crop of prospective students. To do this they have various entrance exams, plus interviews.

One way to test the effectiveness of rationality arts would be to offer rationality training to 17/18 year olds who were soon to be interviewed by these top universities. If our group did significantly better than one would have predicted based on their grades, then we would have a quantifiable and objective signal of effectiveness.

Hmm. Some off the top of my head:

• Look for studies that have recognized a certain bias, then use that information to come up with reasoning problems where the participants have to reach the correct answer without falling prey to the biases. Somewhat vulnerable to people studying to beat the test, though can potentially be defeated by creatively combining several different biases and applying them into new situations. Downside: coming up with lots of different scenarios where one may fall victim to biases is a lot of work. Perhaps come up with suitable computer games where success depends on avoiding biased behavior, and the scenarios can be automatically generated?
• Calibration tests. These could be auto-generated, drawing on a far wider field of information than the current ones.
• As the above two, but subjects are forced to write down their reasoning. This may be more helpful in making them reflect more on their reasoning, than for actual verification - somebody's train of thought can be very hard to interpet, since they'll never write down everything that influenced their decision.

I'm not sure what effect you're     !
going for, but indenting by four    !
spaces allows you to do things like !
this.                               !

* Bullet lists look like this.

1. Ordered lists look like this.

Something the masters (and students) of each school can do to keep it real:

The Winning Tournament: Organise a yearly or so event. A group of clever, evil people select and creates a number of "games" or tests, if you'd rather. Wannabe masters of rationality can compete against each other for the title, pride and  glory. 

The type of games and tests should be kept varied. Some could be contests where participants randomly compete against each other, other might be battle royals where people can form alliance and all around try as hard as they can to win. Others are just games where there is a correct solution to be figured out from clues, the fastest you get there the more points you gain, but if you get to the wrong solution, well, that's really worse than having been slow.

Physical abilities and expertise in fields should be kept out of these games if possible, since they are powerful "noise" sources. 

The tests should be really hard, so a committee of clever, evil people who approach the task of creating the games with a certain degree of evil glee is recommended. They can get inspirations from real life problems that have caused disasters and routinely mess up experts are recommended. Games that are similar to real problems people can meet in real life are recommended (for example, having to mediate an agreement between other groups of participants, or just by hearing the description of a situation).

A good example was the Darwing Game here on Lesswrong, though it seriously advantaged programmers it had the effect to spark a lot of interesting plans.

For organising it... if you keep things "fun" enough it should be doable to find interested people. If you keep things "amazing" enough, rationality might even start to get a bit of "awesomeness" reputation from it. It seems possible to even organise it online. You could charge a small fee to participate it to cover expenses (something proportionated to the "fun" you'd get participating) and perhaps use a part of it as prize money for the winner.

A group of intelligent people acts as "graders" of the game, to create an individual score that hopefully could be more or less coherent between tournaments, so you'd know how much you've improved from the last one.

Clearly, this is just as good as the games and tests that are put into it. Crucial aspects would be how much does a game has a victory that correlates with rationality skill rather than just some more specific skills, how much does a game has a victory that correlates with being able to "win" in common real life problems. But I think the committee could have fun in creating said games, and I'd expect people skilled in thinking and rationality to be advantaged. Raw native intelligence would be a major source of noise, but that can't really be helped I think.   

Something you could do to test a hundred student

A number of "games" or "situations" where a mathematical solution is calculated is devised, and such games are handed in a yearly test people can do online. 

The betting game with the lights in "A Technical Explanation of a Technical Explanation" is a good example.  People are asked to bet each round, with evidence coming from a mathematical simulation of the game being provided bit by bit. More than a scoring function can be used for such betting games, so players also have to calculate what the best winning strategy is depending on the circumstance.

Deduction games can also be played, where you have to use the clues (again, we are supposing types of games where we can calculate the exact or most likely solution for the evidence provided)

For such games, elaborated rules can be devised, not necessarily going with simple random distributions. Perhaps some scoring system for correctly stating the rule could be used? Occam's Razor might apply to how likely you are to have a complex rule vs. a simple rule.

Other possible games can be devised in decision theory, with the task of maximising a given utility function, predicting the moves of other entities is incorporated as well. For extra fun, some of these games will be connected, if a certain number of people will be asked to choose the move of the entity A in a scenario, and a number of player will be asked to choose the move of the entity B in the same scenario, then the percentages of the players' decisions will be used to choose A and B moves, evaluating the final score each player got in that scenario depending on his individual decision. 

Arguments with fallacies, biases or other errors of cognition can be provided (not in every test there will be one, and participants won't know how many there are). Participants would choose the correct option from a long list of options, and will have to also mark the numerical number of the line where the fallacy appears to specify which is what, so the scoring system can be automatised.

We can also put in some hard logic exercises for good measure.

Participants receive a global score and more detailed ones, which are determined using time and precision.

Note: I think this kind of test measures more the "theoretical skill" of the participant, and not whether they can apply it in real life.

Something you could use as a test even if people have an incentive to game it

Class project: a group of rationalists choose an ongoing problem in a particular field. A group of experts in the field is selected to act as judges. 

A number of rationalists offers to collectively tackle the issue. When they sign up for the game they specify the number of hours they could put into this (a minimum number is required for participation). Applicants are required to specify why they think they could help solving that particular problem, those that aren't judged suited to the task might be refused by the judges.

Ways to communicate with each other are provided. The participants have to organise the work between themselves, and be able to propose a solution, analysis, or something that would provide progress in said field. Each participants or group of participants will be assigned a task based on the number of hours they offered to put in, if they can't complete their tasks this affect negatively their score (which also depends on the number of hours). Of course groups are expected to be able to take care of such problems by themselves.

When the final work is produced (which has to suit standard requirements for their fields) it is graded by the judges, to give a "quick score". 

More importantly, attempts are made to put the work under the scrutiny of the fields it belongs to (for example, by publishing it in a scientific journal). If this scrutiny goes well, it goes into the "real score". Progress made in solving said problem are monitored. If the work produced by the rationalist group ends up being right, it goes into the "real score". If the project leads to solving the problem, it super goes into the "real score" and badges or something are handed out to everyone who worked on it and received an adequate score.

Stupid idea: Have a handful of students from each school volunteer to be assigned extremely difficult, real-world tasks, such as "become an officer at Microsoft within the next five years". These people would be putting any other of their life plans on hold, so you'd need to incentivize them with some kind of reward and/or sense of honor/loyalty to their school.

I doubt a few minutes of pondering will provoke any significantly insightful thoughts, but on the off chance that they do here's what I've got:

A major pitfall of most tests is that they can end up examining a wide variety of confounding variables. For example if the test for rationality is based on a written prompt then it selects against those with dyslexia in spite of their rationality. If it's based on a spoken prompt then it selects for those with similar accents to the test-giver, or against those who had it read to them in a strange way. Ideally since the thing that we're selecting for is (I assume) practical reasoning skills, we would want the test to have some similarities to real life.

Thus the thought that comes to mind is an escape room which can be set up and run essentially-identically for each participant, whose puzzling elements require you to make Bayesian updates on multiple propositions that you were given an idea of the likelihood at the start. In order to avoid biasing the tests in favor of those with more general knowledge, the propositions would ideally be totally fictitious. It occurs to me that the elements of real-world pressure and communication would bias the test against those prone to anxiety, but given that that's a common problem when you're called on to apply your rationality skills in reality I think that may be an acceptable flaw, if no other options are obviously superior.

Two ideas I got after 5 minutes (by the clock :)) thinking.

If the tests are stressful and mentally (and possibly physically) exhausting, then even if it is still possible to prepare just for the test, it will not be as far from preparing for the "real thing". So, something like Initiation Ceremony could be done periodically and not just for initiation.

Give the students "stories" and see if they can make heads or tails of them. (How accurately can they guess the omitted details? Can they predict how it continues? Etc.) But, where can you get real stories? An authored story is very bounded in usefulness for this.
The idea: we have court cases. A lot of them, in all kind of domains, dating back to centuries. And they are very real, even if it's distorted (fake evidence, false testimony), it's done by someone for some concrete reason, which can be analyzed rationally. This might require learning some Law, but even without formal training many non domain-specific cases can be understood with moderate work. And Law is one of the oldest applications of human rationality.

Both of the ideas are mostly applicable to the second use-case: measuring a bunch of students in a school, but not good for comparing schools or designing a standardized "rationality test".

I should note that per the EY’s request I haven’t read the other comments before posting, so sorry if I duplicate anything.

The ability to make predictions in advance seems like one of the most important important, and assuming that you have enough time easiest to test measures of rationality. For the experimental and potentially the organizational level success on the prediction markets seems like an obvious choice, that also has the benefit of showing how good the person is at avoiding certain money related biases. There would of course need to be some controls in terms of equal access to capital and access to information, but I think we can work that out.
At the reputational level set up something like the David Brin prediction wiki for EY http://earthbydavidbrin.pbworks.com/w/page/15607657/Predictions though in this case we would be focusing on ones he explicitly makes rather than stuff culled from a work of fiction.

Maybe something that tests "certainty faking"? I really don't know how to construct it, per se, may use a FACS test to see how much a person is trying to convey that they're very certain of something when they aren't. That would just be conscious faking, of course; you'd still need something to assess when someone is expressing their feeling of certainty vs. the data. Maybe something like Texas Hold 'Em, except with bets being placed on how accurate the probabilities are (e.g. randomized variations of situations like the cancer scenario at EY's Bayes page).?

Sorry if I'm not articulating this well, hopefully it's good enough to live up to the stupid idea criteria, if not the good idea. Oh, and I didn't read any of the comments, so I don't know if this has been suggested.

I'm reminded of your own introduction to Bayes. Even a really good test won't do a darn bit of good if rationalists are vanishingly rare.

There are lots of proposals which basically say, let somebody predict the development of a situation they're previously unfamiliar with. But that'll probably be very heavily a test of IQ, and while rationality would certainly help your performance in such scenarios, it seems to me that IQ will regardless be a bigger factor. Same with using real-life performance as a factor.

I'm not opposed to using such scenarios, and I proposed something like that myself, but I do think that the scenarios have to be specifically designed so that they're likely to trigger known biases (even if in a subtle way). You can't just use totally random historical events or police cases.

I get the feeling that the real problem here is repeatability. It's one thing to design a test for rationality, it's another to design a test that could not be gamed once the particulars are known. Since it probably isn't possible to control the flow of information in that way, the next-best option might be to design a test so that the testing criteria would not be understood except by those who pass.

I'm thinking of a test I heard about years ago. The teacher passes out the test, stressing to the students to read the instructions before beginning. The instructions specify that the answer to every question is C. The actual questions on the test don't matter, of course, but it's a great test of reading comprehension and the ability to follow instructions. Plus, the test is completely repeatable. All of the test questions could leak out, and still only those who deserve to pass would do so. If you are willing to assume that people who pass would not be willing to cheat (unlikely in this test, possible in a rationality test), then you would have an ungameable test.

A rationality test in this model might be one where an impossible task is given, and the correct response would be to not play.

Like R.A.W. has said, "The more you see yourself acting like a cosmic smuck thus less of a cosmic smuck you will become". I think it is very important that the environment stresses awareness of moment to moment actions and thoughts. If not, I think decent application of the knowledge of rationality will be very hard indeed.

If this is an important aspect of your 'school', then I think it would be hard to game the system without actually learning what is supposed to be learned. This would especially be true when it is a part of the reputation heirarchy. Sure, some could mimic to gain status but others with actual awareness would see through them easily.

I seem to be years late to this party, but I've heard the LW culture isn't opposed to commenting on old posts. In the interest of "breadth" I'll answer anyway after at least five minutes of thought, without looking at the other answers first (though I've probably seen subsequent posts that have been influenced by this one by now).

So there are three categories of tests here. In order of strictness: those for masters, those for students, and those for employees?

There are many skills under the "rationality" umbrella. Enumerate them and test separately. Maybe there are some we don't know yet. How do we test for those? There's also a difference between epistemic and instrumental rationality. Epistemic seems easier to test and is probably required for instrumental. But instrumental is what we really want. Some of my test suggestions will only test a part of "rationality".

Schools and science have a lot of experience measuring things like this. Can we learn from them?

Every test I've come up with seems to be in one of two categories: toy problems, or real-life problems. The real-life problems are better for the masters, perhaps; and the toy problems for the students. The toy problems are less real, but more replicable. I thought we're supposed to hold off on proposing solutions to avoid attractors like categorization prematurely limiting our scope. But we've been asked to brainstorm. Can we break out of these categories?

Some Ideas:

• Give the students a sum to invest for a small business, and time limit, then see how much they make. Require strict record keeping to prevent cheating. Noisy.

• Give them a sum to invest in a prediction market, then see how much they make.

• Use more direct calibration tests. Make students give probabilities for things. See how often they're right.

• A student must catch specific examples of cognitive errors/fallacies in a video. (Arguably the important part is to catch one's own errors, and the ability to find others' errors doesn't prove that.)

• Make a student write an essay before the term. The instructor will find examples of cognitive errors in it, but keep it secret. Then after the term, the student must review his essay and find as many errors in his former thinking as possible. This will measure personal improvement, but might not help measure relative to peers, since they're all taking different "tests".

• SAT-style multiple-choice exam. This can test knowledge of the material, and synthesis too (or so the test writers claim) to a limited extent.

• Like the three integers test, the master can play the role of nature, while the students play the role of a scientist, trying to figure out a simple rule by "experiment". Grading can be on the number of questions asked, the time taken, the difficulty of the rule, or the number of these questions answered correctly. The instructor must be strictly forbidden from giving hints that could ruin the results. This is actually very similar to debugging software. Maybe this kind of test could be computerized, with "nature" as an opaque program and, students writing code that interacts with it as their "experiments". They then may have to write code to emulate the rule. If it passes the unit tests, a human instructor can confirm if it implements the same rule. This can also give students a feel for what it's like to do science correctly.

• Competition programming AI to win at game-theory-inspired challenges. See how they compare to well-known strategies. May be hard to keep challenges secret. Could the payoff grid be randomized?

• Life outcome survey over years. Are they "winning" more versus control group? May be hard to define. Slow. We should do this, but we shouldn't wait for it before developing the program.

• Masters can actually try to accomplish something. Maybe improve life outcomes in a third-world country or something. To be meaningful, it would have a control group, competitors, a time limit and a budget.

Generate a fantasy world with certain rules of magic. The goal is to figure out precisely what those rules are, all the while working towards some end goal. Perhaps this could be run by a handful game masters who know exactly what the rules are supposed to be, or magics are input into a computer program, so no one knows for sure. One would promise to keep the rules secret once figure out. This would encourage proper hypothesis testing and thoughtful use of evidence, especially if resources are limited. I suspect this wouldn't just be a one-off, but a repeatable exercise if one had multiple worlds or the ability to arbitrarily generate the system. Perhaps one could engage in duels using the uncovered magics, and would be able to encourage creativity by applying these in different ways. I'd imagine one could use systems much like those in role playing games, but but qualitatively based perhaps?

There was a card game based around hypotheses in a class I took once which I've improved upon somewhat here: https://lordregexrationalist.wordpress.com/2015/09/29/rationalist-belief-card-game/

If rhetoric is the dark arts, then rationalists need a defense against the dark arts.

I've always seem debates as a missed opportunity for rationality training/testing. Not for debaters, but for the audience.

When you have two people cleverly arguing for an answer, that is an opportunity for the audience to see if they can avoid being suckered in. To keep things interesting, you could randomize the debate so that one, bother, or neither debater is telling the truth. (Or course in the toughest debates, the debaters are both partially true and the audience needs to find out what is the real answer.) And if we want to keep the students from compartmentalizing what they have learned, we probably need to make the debates a mix of real world and abstract debates. We might also have easy, medium, and hard difficulty debates, but you don't tell the audience beforehand.

I think that this would be useful thing, because lots of place already have debate clubs and public debates. All we would need to have an audience game running in the background.

I think that the most helpful part of the lesson would come after the debate and after the audience has been scored on their confidence intervals. If we can get the debaters to explain the rhetorical tricks they used, so the audience can recognize them in the future and hopefully not fall for them a second time.

http://lesswrong.com/lw/3h/why_our_kind_cant_cooperate/

I can't help but think the focus on competition is a fairly bad idea. If a student can raise the entire group's score by 10%, that is far more commendable than raising their own individual score by 20%. We don't want high-scoring individuals, we want to win. That's something which is quite often done as part of a group, in the real world.

When I started thinking about this I realized that testing for rationality is pretty complicated! The hardest part about it is determining the 'most rational person' in a group. If the 'most rational person' is a member of the group being tested, how can the testers determine who they are if the testers are less rational than them? Does a tester's ability to recognize the best of the test group depend on whether the tester is biased, and how they are biased? And who would test the testers, then?

Regardless, here's an idea or two.

A Multilevel test: Biases may be fairly easy to test for, in general. The are relatively well defined. Someone who is known to be fairly unbiased in one respect or another could run tests for that bias.

An experimental test: Learning ability could be tested. A list of 100 skills, facts and methods could be presented to individuals in a test group. Some of the items on the list would be false, untenable or in some other way illogical. Members of the group would have to learn the useful ones, thus demonstrating their ability to overcome any biases they may have to learning this-or-that concept, trait, skill or whatever. They would also have to not learn the bogus ones, demonstrating their ability to recognize bad ideas, harmful methods and false facts, etc. Groups that did well in the experiment would could be held up as examples of how one ought to approach learning.

I just suggested a relevant rationality test here: http://www.overcomingbias.com/2009/03/how-spend-rationality-test.html

Experimental methods for measuring rationality can be converted into organizational tools through the measurement of biological traits that are minimally malleable. For instance, you could map genomic and brain structure information to experimental tests of particular biases or bias-promoting traits, and then use those biological markers as ungameable indicators. Unfortunately, while this could help organizations get more rational employees (possibly deriving economies of scale), it would be much less useful for measuring improvement.

deleted

Vladimir Gritsenko mentioned Rational Debating on an old post. It looks like it would be a useful addition to the list.

Make a very detailed audit of the habits, hobbies, books, music, shoes, watch, cell-phone etc. etc. etc. of the top/average/bottom contributors to LW. Are there correlations? Match to new candidates.

Here is a stupid one: Detective stories. Like Encyclopedia Brown, but subtler. And with false leads. I don't think normal mass-market detective stories would work, because they may try to deliberately choose an irrational answer to surprise you. But special ones written by rationalists for rationalists could be a fun distraction if nothing else.

Role play. Build a corpus of fictional scenarios too big to memorize and present a random subset in the test.

Also, standard tests on rationality lore and mathematics would work to a degree because they're correlated with actual rationality.

What we need is a rationality equivalent of a katana or a machine gun. One for each student, some basic training and even ninja masters go down pretty quickly (unless they really can dodge bullets). Occupatio "weapon of mass rationality".

I ended up going in a  completely different direction with this: I intend to test my OWN rationality, and I figure that if rationality is about WINNING, about being EFFECTIVE, then I ought to find direct measures of the things I want, and test myself in 6 months or so (timeframe dependent on the toughness/length of the task). This will, in other words, be a test of my ability to understand the territory insofar as that understanding makes me more effective at a given task.

The things in particular, a few subgoals of my personal life-optimization:

• artistic endeavors and life enjoyment: engaging in things like art or gamedev or other mediums while aiming for MAXIMUM FUN.
• being RESTED, having ENERGY to do things (I have a tendency to burn out due to overworking myself)
• studying AI alignment (and the general objective of "actually have a positive effect on whether we're all going to die or not")

the tests in question:

• for each artistic project (i.e. a drawing, a game, et cetera), I post it to a public forum and, with said post, I add a simple strawpoll asking "would you judge this work as 'complete'?" I would not be allowed to argue with the results, and I'd be judged by the number of projects I complete, with "complete" defined by the poll being greater than 80%. It's hard to directly measure fun, but "completing quality projects and showing them to others" seems like a good enough way to achieve it, for me in particular.
• I've already got a method for measuring this: each day, if I notice that I'm tired during my free time, I force myself to rest until I stop being tired, and I record the time in a spreadsheet. Then, I sum up all the rest time accumulated over a period of a month. The lower it is, the better my sleep patterns hypothetically are. I predict that, if I had myself do this for as long as a month, I'd avoid pushing myself to my productive limit - that would induce burnout, which would eventually FORCE me to rest during free time, and a lot.
• A good measure of "you are making quality posts" would be either something like summed upvotes, or maximum upvotes on a post in a given period, or maximum links from other posts made by other users. That last one seems difficult, but also points me in the right direction of "look into what posts got referenced the most, and try to make those kinds of insights."

I'm a little bit nervous about taking on the 1st or 3rd test - I'm not sure if I could pull them off - but I suppose that's the right feeling to have, if they're hard but accurately so. 

An idea that might be both unsustainable and potentially dangerous, but also potentially useful, is to have someone teach as a final test. Less an exam and more a project (with oversight?). Of course, these trainees could be authentic or disguised testers.

Problems with this idea (non-exhaustive): - Rationality doesn't necessarily make you good at teaching, - Teaching the basics badly are likely to have negative effects on the trainee, - This could potentially be gamed by reformulated regurgitation.

So... What behaves differently in the presence of Rationality. I like Brennan's idea of time pressure, though he himself demonstrates that you don't need to have finished training for it, and it doesn't really hit the mark.

Or: What requires Rationality? Given Hidden Knowledge (may only require facts that are known, but not to them), one could present new true facts that need to be distinguished from new well-crafted falsehoods (QM anyone?^^). This still only indicates, but it may be part of the process. If they game this by studying everything, and thinking for themselves, and coming to correct conclusions, I think that counts as passing the test. Maybe I am currently not creative enough though. This test could also be performed in isolation, and since time would probably be a relevant component, it would likely not require huge amounts of resources to provide this isolation. Repeat tests could incorporate this (or seemingly incorporate it) too.

If you wanted to invest more effort, you could also specifically not isolate them, but put them in a pressured situation (again, I am being influenced by memories of a certain ceremony. But it is simply really good.) This doesn't have to be societal pressure, but this kind at least makes rash decisions less likely to be costly.

I can't really formulate the idea concretely, but: A test inspired by some of ye olden psychology experiments might provide double yield by both testing the rationality of the person in question and also disabuse them of their trust. Though I can see a lot of ways this idea could go awry.

An issue that most if not all of my tests run into is that they limit what could be taught, since it is still part of the test. This is a problem that should be solved, not just because it irritates me, but because this also means that random chance could easier change the results.

This is, I think, because so far all tests check for the correct answer. This, in itself, may be the wrong approach. Since we try to test techniques which have an impact on the whole person, not "just" their problem solving. I would for example hope that a crisis situation would on average benefit from the people being trained in rationality, not just in regards to "the problem solving itself", but also the emotional response, the ability to see the larger picture, prioritization and initial reaction speed, and so on.

(Maybe having them devise a test is a good test...^^ Productive, too, on the whole.)

(I can think of at least one problem of yours that I still haven't solved, though I therefore can't say whether or not my not-solving-it is actually showing a lack of rationality[though it's likely], or rather depends on something else. Not sure if I should mention it, but since you (thankfully) protect the answer, I don't think that I need to. This, still, is asking for a correct answer though.)

That's all I can think of for now. Though I am not really satisfied... Do I need to be "at a higher level" to be able to evaluate this, since I don't fully grasp what it is that should be tested yet? Seems like either an option or a stop sign..

One large theme I've seen in biases is the tendency to affirm positions you already hold, by treating evidence and arguments with imbalance.

So my idea, is to purposefully select arguments from both sides of highly controversial issues such as gun control, abortion, or whatever is polarized at the time period. Then riddle the arguments with mistakes, and challenge the student to find errors in both sides of the issues.

Possibly having a bank of possible rational missteps that they must dole out to different arguments, or a free form analysis that has to be well justified and is subjectively judged by a group of rationalists.

Take any cognitive bias that is supported by previous experimental data. Replicate to confirm.

Subject students to various training regimens, with control group.

Test again for presence of cognitive bias, note any improvements.

Repeat, repeat again for other known cognitive biases.

Not perfect, but it should be enough to make some headway.

Also just subjecting a student to a battery of tests, ideally creative stuff potentially involving real life scenarios not just written tests, to look for all sorts of cognitive biases.

Should the student try to game the system by learning, well great!

How about a test that causes people to build and use mental models and formulas? People are asked to estimate primarily numeric facts based on other facts. In each question, give people a set of "measured facts"* and ask people to estimate more relevant facts/consequences via back-of-envelope calculations (or a computer program, for more precision). But unlike a normal math word problem, set up the test so that, say, 2/3 of the questions cannot be accurately estimated with only the information given. Among that 2/3, half can be accurately estimated by adding some common-sense info (e.g. that most people work about 40 hours a week, that life expectancy is about 80 years, that almost half of American voters vote Republican, etc.), and the other half require more esoteric information that test-takers will rarely have. For all the questions, test-takers need to build a simple mental model or formula that would allow them to do the calculation, state any information they need that is missing, and try briefly to look up the info online in order to compute a reasonable estimate. If they can't do this, they need to express the answer in terms of unknown variables and then guess what the values of the variables are. They must also state relevant assumptions.

This is a means both to improve rationality as well as test it.

Example question set:
Background: in some types of accidents at some types of nuclear plants, radioactive substances can be released into the atmosphere (radioactive substances emit ionizing radiation). It is medically plausible that there is no perfectly safe dose of ionizing radiation in human tissue, and that radiation damage to DNA is cumulative, because cells repair some DNA damage very slowly, or never, and this damage can lead to cancer years after radiation exposure. This is known as the linear no-threshold hypothesis: that the health risk is proportional to exposure and there is no safe dose. If residents are promptly evacuated during an accident, the primary risk to their health upon returning will be from long-term exposure to radioactive cesium, which mainly causes a type of cancer called non-CLL leukemia.**
• A metastudy reports that the excess relative risk (ERR) of non-CLL leukemia from 100 mGy of radiation is about 19% (this means that people get non-CLL leukemia 19% more often than normal).**
• The normal rate of leukemia in the U.S. is about 14 diagnoses per 100,000 people per year. About 1.5% of people are diagnosed with leukemia at some point in their lifetime
• The normal death rate of leukemia is about 6.4 per 100,000 people per year in the U.S.
• One third of leukemia cases are CLL leukemia cases.
• Another study estimates that in the U.S. there are about 16,000 excess deaths annually due to electricity generation emissions, which is a low rate compared to some developing countries. The researchers estimate that 91% of these deaths were the result of emissions from coal-fired power plants.
• There are 328 million people in the U.S. and 7.5 billion in the world
• About 65% of all electricity worldwide is produced by burning fossil fuels. About 10% of electricity is from nuclear plants and 38.3% is from coal.
• Assume two-thirds of cancer cases and deaths from a nuclear accident occur outside the city where the accident occurred***

Scenario: suppose that another nuclear accident were to happen, one somewhat more serious than Fukushima, inside a city of one million people, in a developed country. Suppose that all evacuated persons return to their homes after one month and, as a result, are exposed to 100 mGy of radiation on average, mostly from cesium. Assume that half of this radiation dose occurs in the first 10 years and that most of it has occurred within 40 years***.

Questions:
1. Estimate the chance that the radiation will cause non-CLL leukemia in a particular, random person in the city at some point in their lives.
2. Estimate the chance that the radiation will kill a particular, random person in the city after they move back.
3. Estimate the total number of non-CLL leukemia cases caused by the radiation (over 40+ years).
4. Estimate the total number of people that will die as a result of the radiation (over 40+ years).
5. Assume that all nuclear accidents worldwide, combined, cause this number of deaths once every 20 years (e.g. in a 20-year period there might be two accidents, each half as serious as this one). What is the expected number of deaths per year in a randomly selected city of about one million people?
6. Estimate the number of excess deaths caused by power generation in that same city (i) per year, and (ii) over a 40-year period, if all its electricity came from fossil fuels instead of the nuclear plant.
7. Brainstorm additional factors that might change your estimates above.
8. Brainstorm other considerations that would be relevant to evaluating safety of nuclear power compared to alternatives.

Example answers:
1. Assumptions: All people have lives of average length (80 years). Age distribution in the city is uniform from 0 to 80. Leukemia risk is elevated uniformly after exposure for the rest of the person's life. All developed countries have similar leukemia rates. Leukemia is diagnosed very soon after it develops. Leukemia risk does not vary by age (this is not true, but on the other hand, I question whether it was appropriate for the metastudy to use ERR instead of excess absolute risk (EAR)). Radiation exposure probably drops off mostly according to cesium's half-life, but to simplify the calculation, assume 50% of the 100 mGy dose is delivered linearly in the first 10 years and the other 50% linearly over the following 30 years.
• Normal non-CLL leukemia risk is 14*2/3 = 9.333 per 100,000 per year
• A random person has on average 40 years of life left (50% of an 80-year lifetime)
• Excess risk of non-CLL leukemia is 19%, so 9.333*0.19 = 1.773 per 100,000 once the full dose happens.
• But there's a long delay before reaching the full dose... integrating over my approximate exposure function, average excess incidence should average 1.773/2/2 per 100,000 in the first 10 years and 1.773*0.75 over the next 30. Neglecting young and old people to simplify the calculation, the lifetime risk is about 1.773*0.25*10 + 1.773*0.75*30 = 44.3 per 100,000 over 40 years, so the lifetime risk is about 0.0443%, or 1 in 2260.

Fun fact 1: Before writing this LessWrong post, I did a calculation like this to learn about the risks of radiation, because I couldn't find any research estimating what I wanted to know. Radiation risks seem to be among the world's best-kept secrets. I'd rather see a peer-reviewed paper answer "how likely is X amount of radiation to kill me" than rely on my "napkin" model, but I haven't found any such research.
Fun fact 2: the answer increases if your starting point is "1.5% of people are diagnosed with leukemia at some point in their lifetime" since "14 per 100,000 people per year" only adds up to 1.12% per 80-year lifetime. I don't know why these numbers don't match up.
Fun fact 3: I should really be using a simple (Monte Carlo) computer model for this with exponential decay of radiation exposure... no idea if it would raise or lower my estimate.

2. (Further) Assumptions: Non-CLL leukemia is the only cause of death driven by radiation. Years of life left after the first cell turns cancerous is negligible. Probably both assumptions are significantly wrong, but the first assumption underestimates deaths and the second overestimates them so it seems like a wash.
• 6.4/14 = 45.7% of cases are fatal so the risk is 0.0443%*0.457 = 0.0202% or 1 in 4939.

3. Assumption: cancer screenings do not increase as a result of the accident (I'm sure this is wrong). There will be about 0.000443*1,000,000 = 443 excess cases in the city and about 487*3 = 1329 excess cases total
4. There will be about 1329*6.4/14 = about 607 excess deaths total

5. There will be 607/20 = 30.3 deaths worldwide per year from all nuclear accidents. Given a world population of 7.5 billion, that's about 0.004 deaths in a city of one million. The risk increases somewhat in cities that contain their own nuclear plant, if the plant is one of the more hazardous (read: old) models.

6. In a random U.S. city, the expected deaths per million in the U.S. from fossil fuels is 16'000/328=48.8 per year. (i) Assuming air pollution's effects are mainly local and 100% of power generation comes from fossil fuels, the expectation for a U.S. city is 16'000/328/0.65 = 75 deaths per year due to fossil fuels. (ii) which is 3001 deaths over a 40-year period (4.5x higher than the nuclear meltdown scenario).

7.
• Increased screening due to concern about the risk will increase the rate of cancer diagnoses, but not rates of cancer, and cancer death rates may be reduced by early detection.
• Radiation could cause other types of cancer deaths (I heard, for example, that short-lived iodine isotopes can cause thyroid cancer, but that this can be mitigated with iodine pills).
• Etc.: I'm getting tired but you get the idea

8.
• Regulations passed after Post-Three-Mile-Island probably increase safety a lot in newer reactors (but make new plants cost-prohibitive to certify and build)
• Nuclear waste increases long-term risk (but less than most people think, I would add)
• It has been suggested that terrorists could steal nuclear fuel and build a bomb with it. (I don't know if this is remotely plausible, but I do know that reactor-grade uranium is not directly usable in a bomb.)

• Deaths during plant construction and related mining should be similar between nuclear and fossil fuel plants; solar plant construction seems like it should be safer than nuclear, oil/coal, and wind.

• Though deaths from fossil fuels are more numerous, each death is expected to be less bad because it should happen near the end of a person's life due to many years of lung damage, whereas in the nuclear case, some young people will be affected. It's strange to me that fossil fuel deaths are not measured as "years of life lost" instead.

* The "facts" can be real or based on back-of-envelope calculations, but the test-taker is to assume the information is factual. If it is not factual, and concerns the real world, it mustn't be excessively off-the-mark because humans can't simply erase misinformation from our minds so it's best not to intentionally mess with us.
** This is roughly correct AFAIK but I'm not an expert. Also, the metastudy strangely neglects to model time, e.g. it does not say that the risk is elevated for the rest of peoples lives, or that it is elevated for X years, or anything time-related like that. I don't see why risk would be elevated for life—if damage will cause a cell to turn cancerous, why would it wait 20 years to do so?—but conservatively this is my mental model anyway. I've seen a study that indicates 100 mGy is more than the average dose avoided by relocating residents of Fukushima; note also that mGy and mSv are the same SI units, so I don't understand the difference.
*** This datum is made-up as I haven't found information about it.

After going through this exercise I think the formulas need to be more explicit... really we should write a program for nontrivial models, e.g....

// TODO: turn into Monte Carlo simulation

let ExcessLifetimeChanceOfCancer = 0, BaseNonCLLLRisk = (14.0*2/3)/100'000, Dose = 0, InitialYearlyDose = ??

let CesiumHalfLifeYears = 30.17, YearlyDecayFactor = 0.5**(1/CesiumHalfLifeYears), ERRPermGy = 0.19/100

for year in 1..YearsOfLifeLeft {

. Dose += InitialYearlyDose

. InitialYearlyDose *= YearlyDecayFactor

. ExcessLifetimeChanceOfCancer += BaseNonCLLLRisk * ERRPermGy / 100

}

print(Dose) // TODO: pick initial dose so that total tends to be 100 or a bit less

print(ExcessLifetimeChanceOfCancer)

And also there would be need of numerous easier exercises than this one.

Erm... let me be Brennan and go with the "obvious". Find problems whose solutions are known in some field but not widely, provide the initial data and results of additional experiments on request (with "too expensive to perform" being a possible result). Then have two measures:

1)Someone who is _also not an expert_ checks solutions for, well, everything you discuss here. Biases, effort, mysterious answers - you name it. (For effort, you might need to register when every thought was written, not just what it was.)

2)An expert checks the dataset used - what of the really conducted experiments students failed to request and which of them were actually useful.

The 'test even if gamed' reminds me of a labyrinth. Suppose there are several ways of reaching the end, and the participants can't know which way they are set upon, because it is chosen randomly. They are asked questions from outside of their domain of knowledge (it would need a big database to pick from), constructed in such a way that it is impossible to pick the right answer without knowing about various cognitive biases (e.g., the conjunction fallacy etc.) The questions can be independently rated for apparent difficulty, and masters will be given the hardest ones. (I don't know what makes some questions seem simpler even if the answer is still wrong. I asked people to rate questions in my post 'Before the seed. I. Guesswork', and somehow people chose exactly one to be 'the easiest to formulate hypotheses about', but I don't know how they did it. Plus, few people answered at all.)

How about asking people:

i) What is rationality for you?

ii) How rational are you?

iii) How will you prove it?

The askee can then ask the asker: Do you agree?

And then we have a conversation. Both parties have to agree on the final score.

Basic true/false test; reverse stupidity is not intelligence but rationalists tend to have fewer false beliefs. Taking the test upon entering the school would prevent the school from teaching to the test and the test could be scored on multiple areas of which one is a cunningly disguised synonym for rationality and the others are red herrings so that irrationalists have no incentive to lie on the test.

It seems like rationality overlaps so many different fields that it does not seem very plausible to be able to test rationality specifically. Political and ethical debates though seem to contain a lot of elements dealing with rationality.

Although this post is old now, I'll still enter my ideas (good or bad) before reading the other comments...

1. Video games. Expertise in one video game is not good enough; ideally, speed rationality of 100 people could be tested on a new game none of them had seen before.

2. Along similar lines, ask the 100 people to cooperate in a large artificial project which requires that number of people, such as the manufacture of a complicated item invented for the day. It should be complex enough that cooperation is needed; IE, involve several complex skills such as a specific (ideally new, or new to the group) origami folding. Each individual is scored on how many of the item they have in their possession by the end of the day. It would be possible to work alone, but more efficient to work in groups and split profits. (Ideally the item is complex enough to encourage very large groups, tending towards everyone needing to work together.) This has the additional benefit of being a good learning experience.

Try to simulate the apparently supernatural/ create other hoaxes and see who can debunk them. There is enough domain specific knowledge that it wouldn’t work too well with individuals, particularly if they have a motivation to game the system. Still if a school doesn’t generally increase its students ability to deal with the apparently supernatural and false information it’s almost certainly bad sign.

Experimental and Organizational tests seem to be the most important test types here; if the students and methods are able to show they're capable, and are measurably better than the students of another craft, then their school is obviously doing something better than other schools anyway, no Reputational test needed. So I'll concentrate on those.

What do we need for an experimental test? We need a way of comparing the strengths of students and ideas, to see which are stronger. The problem here is that there's not really a standard unit of rationality. If you want to measure something's volume, you can put it in a water bath and measure how many mL it displaces. If you want to measure someone's rationality... you're a bit out of luck.

I'm not well versed enough yet in cognitive sciences to propose a unit of raw intelligence/rationality measurement, and a way of at least estimating it. Until such a metric is apparant, I think we can make do with comparative testing. Take two students and have them perform some test of rationality that returns less rational, more rational, or equally rational as a rough comparison of the two. Perform it on an entire school, and you can rank each student. Perform it between similarly ranked students in two schools, and you can determine which school is better. Roughly. A test like this could also potentially serve as an organizational test.

What tests would I propose as an experiment? How about something like having the students competitively build a weirdtopia? (http://lesswrong.com/lw/xm/building_weirdtopia/) You could have a panel of randomly selected scifi fans read one of the two weirdtopias (don't compare them side by side, we're trying to get their honest opinion about one of the stories, not their comparison of the two) and rate 1-10 how much they'd like to live in that weirdtopia. The student with a higher voted paper is more rational, and if the stories are about equally weighted, we have two roughly equal weirdtopias.

That... doesn't test every facet of rationality I know. However, using tests as a way of comparing two students is something that a lot more tests could be adapted to, without necessarily having to make a measurable yardstick of rationality. Just need to figure out which aspect of rationality you want to test, look at papers and stories that display this aspect, have the two students write a similar paper using their own skills, and compare the two.

I might use something similar to The Book from Neal Stephenson's Anathem, but less deliberately harmful and more confusingly-related-to-reality. Something where, in order to succeed, you must Change Your Mind, at least partially. If possible, include a real scenario where you must apply the knowledge in a charged context, where people are most prone to irrationality.

A debate-like environment seems like an obvious example for a martial arts-like competition.

Hmm... To me, a master of rationality might seem to be able to debate fairly well with other heads of powerful schools, such as philosophy and physics. I myself can pose some interesting questions to physics knowledgeable people, and refute offhand philosophical stupidity in a stride.

To test students for rationality I guess is easier to test for debiasing, by making classical bias experiments?

I need to mull this over with my fellow Bayesian conspirators.