Rationality Exercises Prize of September 2019 ($1,000)post by Ben Pace (Benito) · 2019-09-11T00:19:51.488Z · score: 85 (25 votes) · LW · GW · 18 comments
Why exercises? What does this look like? What am I looking for in particular? Examples of things I think could have exercises Curated Sequences Curated Posts None 19 comments
I'm giving out $1,000 of prize money for the best exercises submitted in the next two weeks on a topic of interest to the LW community. I'm planning to distribute $1,000, with $500 of that go to the first place.
To submit some exercises, leave a comment here linking to your exercise(s) by midnight at the end of Friday 20th September PDT (San Francisco time), and I'll announce the winners by the Friday two weeks later (give me the time to try a bunch out). You're welcome to post them as a LW post, on your shortform feed, or privately link them to me in a PM if you want, though I'll be publishing all the entries that win a prize.
I'd like to be to practice using ideas, and to know whether I actually understand them.
I want to concretely practice the art of rationality [LW · GW] (and other arts [LW · GW]), but I don't have many natural affordances to do that. If people added exercises to their posts, I think that I'd do them. I do sabbath-like recovery days, and I have a weekly session with Jacob Lagerros where we work on problems in the HPMOR-recommended book Thinking Physics, both of which I find exceedingly valuable. I'd love to spend more time playing with other ideas people put forward on LW.
I also think it's surprisingly common for me and a friend to achieve a double illusion of transparency [LW · GW] where we're both using a concept or phrase in conversation, but actually have a very different referent in mind. I think small tests and checks can zoom in surprisingly quickly on miscommunication.
So I'm running the prize to get some exercises, for me and for others on LW who want to try them out.
I could talk more about why exercises are valuable, but a lot of my thinking here is downstream of reading the book Thinking Physics, so I'd rather just let its author, Lewis Carroll Epstein, speak instead. (This is from the opening of the book, all formatting is original.)
The best way to use this book is NOT to simply read it or study it, but to read a question and STOP. Even close the book. Even put it away and THINK about the question. Only after you have formed a reasoned opinion should you read the solution. Why torture yourself thinking? Why jog? Why do push-ups?
If you are given a hammer with which to drive nails at the age of three you may think to yourself, "OK, nice." But if you are given a hard rock with which to drive nails at the age of three, and at the age of four you are given a hammer, you think to yourself, "What a marvellous invention!" You see, you can't really appreciate the solution until you first appreciate the problem.
What are the problem of physics? How to calculate things? Yes - but much more. The most important problem in physics is perception, how to conjure mental images, how to separate the non-essentials from the essentials and get to the hear of a problem, HOW TO ASK YOURSELF QUESTION. Very often these questions have little to do with calculations and have simple yes or no answers: Does a heavy object dropped at the same time and from the same height as a light object strike the earth first? Does the observed speed of a moving object depend on the observer's speed? Does a particle exist or not? Does a fringe pattern exist or not? These qualitative questions are the most vital questions in physics.
You must guard against letting the quantitative superstructure of physics obscure its qualitative foundation. It has been said by more than one wise old physicist that you really understand a problem when you can intuitively guess the answer before you do the calculation. How can you do that? By developing your physical intuition. How can you do THAT? The same way you develop your physical body - by exercising it.
Let this book, then, be your guide to mental pushups. Think carefully about the questions and their answers before you read the answers offered by the author. You will find many answers don't turn out as you first expect. Does this mean you have no sense for physics? Not at all. Most questions were deliberately chosen to illustrate those aspects of physics which seem contrary to casual surmise. Revising ideas, even in the privacy of your own mind, is not painless work. But in doing so you will revisit some of the problems that haunted the minds of Archimedes, Galileo, Newton, Maxwell, and Einstein. The physic you cover here in hours took them centuries to master. Your hours of thinking will be a rewarding experience. Enjoy!
What does this look like?
Here are great exercises that have been on LessWrong in the past.
- Scott Garrabrant and Sam Eisenstat's Fixed Points Exercises [LW · GW], which had dozens of commenters completing them and submitting their answers.
- Eliezer's Highly Advanced Epistemology 101 For Beginners [LW · GW] has many meditations - while not exactly exercises, they were key problems that commenters gave answers to and then got to see Eliezer's answers in the subsequent post. Eliezer also previously challenged readers to not solve Free Will, but to dissolve it [LW · GW]. It had several setup and follow-up posts that helped.
- Johns Wentworth posted exercises in chemistry and deck-building [LW · GW] to grapple with the concept of slackness.
- RobinZ made some exercises [LW · GW] to test the reader's understanding of Making Beliefs Pay Rent in Anticipated Experience.
- Alkjash set a final exam in his hammertime sequence [LW · GW] on rationality, inviting people to invent their own rationality technique, leading 7+ readers to write their own posts with their results.
- Eliezer created an exercise prize once before, $50 for any exercise that CFAR actually tested, and $500 for any suggestion that was turned into a CFAR class. They asked for exercises that taught people to Check Consequentialism [LW · GW], to Be Specific [LW · GW], and to Avoid Motivated Cognition [LW · GW]. Winners who got the full $550 were Palladias's Monday/Tuesday Game [LW · GW] and Stefie_K's Vague Consultant Game [LW · GW].
- CFAR has a rationality checklist on their website. It doesn't have correct answers, but it operationalises a lot of problems in a helpful way.
In my primer on Common Knowledge [LW · GW], I opened with three examples and asked what they had in common. Then, towards the end of the post, I explained my answer in detail. I could've trivially taken those examples out from the start, included all the theory, and then asked the reader to apply the theory to those three as exercises, before explaining my answers. There's a duality between examples and exercises, where they can often be turned into each other.
But this isn't the only or primary type of exercise, and you can see many other types of exercise in the previous section that don't fit this pattern.
What am I looking for in particular?
While I'm open to most possible subjects, let me add one operational constraint: it should be an exercise that more than 10% of LessWrong commenters can understand after reading up to one-to-three posts you've specified, or after having done your prior exercises. As a rule I'm generally not looking for a highly niche technical problems. (Though it's fine to assume people have read any curated LW sequence.)
I asked Oli for his thought on what makes a good exercise, and he said this:
I think a good target is university problem sets, in particular for technical degrees. I've found that almost all of my learning in university came from grappling with the problem sets, and think that I would want many more problem sets I can work through in my study of both rationality and AI Alignment. I also had non-technical classes with excellent essay prompts that didn't have as clear "correct" answers, but that nevertheless helped me deeply understand one topic or another. I think both technical problem sets and good essay prompts would be great submissions for this prize, though I'd encourage providing at least suggested solutions (probably best posted behind spoiler tags).
(What are spoiler tags? Hover over the text of this post to read the black box below.)
This is a spoiler tag! To add this to your post or comment, see the instructions in the FAQ that's accessible from the frontpage on the left-menu.
I'm interested in exercises that help teach any key idea that I can't already buy a great textbook for, although if your exercises are better than those in most textbooks, then I'm open to it too.
I think technical alignment exercises will be especially hard to do well, because many people don't understand much of the work being done in alignment, and the parts that are easy to make exercises for often aren't very valuable or central.
I think that it's often easier to build exercises for very explicit, legible concepts (e.g. things that look more like math), and while that's really valuable, I'm also really excited about exercises for other ideas too.
Examples of things I think could have exercises
Definitely exercises for any curated post [LW · GW] or curated sequence [LW · GW] on LessWrong. I've taken a look through our curated posts, here are a few I think could really benefit from great exercises (though the tractability varies a lot on these).
Here are examples from Curated Sequences.
- Rationality: AI to Zombies
- Applause Lights
- Oli suggests taking an article from the web and working through the applause lights, then inviting readers to do the same and showing them your graded solution.
- Fake Explanations [LW · GW]
- Making History Available [LW · GW]
- I can imagine here trying to write a journal entry / newspaper article from a year like 1750, then comparing it with something from the time, and finding out what actually was salient when living through that part of history (as opposed to what is salient to me about it now). But I bet someone else can think of a better idea.
- Universal Fire [LW · GW]
- I recently had a visceral experience when a piece of fiction I was reading casually mentioned that it had covered up alien activity on the sun by publishing some fake data in the physics community, and implied it was a thing they'd often done. It just took me out of the book, because that's a world-breaking thing. You can't just lie to physicists and expect society to have developed okay. They'll notice, or else progress will stop. I imagine I could write a bunch of short world-descriptions like this, and ask readers to say which ones violate stronger laws.
- The Bottom Line [LW · GW]
- Positive Bias: Look Into The Dark [LW · GW]
- Dark Side Epistemology [LW · GW]
- The Proper Use of Humility [LW · GW]
- Practice distinguishing between modesty and humility.
- Argument Screens Off Authority [LW · GW]
- Reversed Stupidity is Not Intelligence [LW · GW]
- Scott's Writing
- The Pyramid and the Garden [LW · GW]
- I feel like Scott uses this insight all the time, but I don't have it internalised. Oli suggests a simple version where you learn how easy it is to explain coincidences, by taking a random word from one column and a random word from another column then explain how they're deeply related. But I'd like to practice the opposite too, where you break down a claim by seeing how many degrees of freedom the claimant had when picking it.
- The Noncentral Fallacy [LW · GW]
- Guided by the Beauty of Our Weapons [LW · GW]
- Meditations on Moloch [LW · GW]
- Good luck.
- Proving Too Much [LW · GW]
- Schelling Fences on Slippery Slopes [LW · GW]
- The Blue-Minimizing Robot [LW · GW]
- Basics of Human Reinforcement
- Time and Effort Discounting
- Ego syntonic thoughts and values
- Other curated sequences
- Mistakes with Conservation of Expected Evidence [LW · GW]
- Oli and I came up with two ideas here:
- Make a bunch of arguments using conservation of expected evidence, some using false inferences, and get student to show which ones are true and which ones are false.
- More technical exercise: I give you basics of probability theory, and you prove conservation of expected evidence yourself. Also I have hints you can un-spoiler if you wish to.
- Why Subagents? [LW · GW]
- Coherent decisions imply consistent utilities [LW · GW]
- Asymmetric Justice [LW · GW]
- Thoughts on human models [LW · GW]
- Disentangling arguments for the importance of AI Safety [LW · GW]
- Less Competition, More Meritocracy [LW · GW]
- This piece covers a lot of ground without giving you space to think for yourself. A version with meditations in the middle, or asking you to make proofs about the things you just read, would be very valuable.
- Meditations on Momentum [LW · GW]
- Norms of Membership for Voluntary Groups [LW · GW]
- I think that it could be very interesting to apply these categories to real world situations. I think it could be very interesting to look at modern examples of new spaces (social media types) and look at them as interactions between these groups, and show where disagreements come from false assumptions about which group we're in.
- Embedded Agency [LW · GW]
- Coordination Problems in Evolution: Eigen's Paradox [LW · GW]
- The Rocket Alignment Problem [LW · GW]
- Unrolling social metacognition: Three levels of meta are not enough [LW · GW]
- Would love to practice reading lots of descriptions of social situations that invite the reader to unroll the social metacognition.
- Prediction Markets: When Do They Work? [LW · GW]
- Beyond Astronomical Waste [LW · GW]
- Meta-Honesty: Firming Up Honesty Around Its Edge-Cases [LW · GW]
- Decision theory and zero-sum game theory, NP and PSPACE [LW · GW]
- Tech economics pattern: "Commoditize Your Complement" [LW · GW]
- On exact mathematical formulae [LW · GW]
- Local Validity as a Key to Sanity and Civilization [LW · GW]
- A voting theory primer for rationalists [LW · GW]
- A Sketch of Good Communication [LW · GW]
- The Costly Coordination Mechanism of Common Knowledge [LW · GW]
- Argument, intuition, and recursion [LW · GW].
Robin Hanson has masses of brilliant ideas that I've not got the time to mine. Some that comes to mind are his more recent posts on automatic norms that I think could become some really great exercises.
Some of Nick Bostrom's ideas would be excellent too, like the unilateralist's curse, or the vulnerable world hypothesis, or the Hail Mary approach to the Value Specification Problem.
If you leave a public comment describing what sort of exercises you might want to try creating, I will try to reply with my best guess on whether it can be a good fit for this prize.
I'll repeat: To submit exercises, leave a comment here linking to your exercise(s) by midnight at the end of Friday 20th September PDT (San Francisco time), and I'll announce the winners by the Friday two weeks later (give me the time to try a bunch out). You're welcome to post them as a LW post, on your shortform feed, or privately link them to me in a PM if you want, though I'll be publishing all the entries that win a prize.
I look forward to trying out your exercises.
Comments sorted by top scores.