That You'd Tell All Your Friends

deleted

I am likely to buy the book for my friends in any case, but I did just think of something that would give me an extra incentive. I want my friends to understand consequentialism/utility.

Most people I know are very stuck in "morality means not doing anything bad" mode. Not only would they not push the fat person off the bridge in the trolley problem, but many (perhaps most) wouldn't so much as push the lever to switch tracks. A lot of them think the whole question is some sort of reductio ad absurdum of consequentialist philosophy ("you're a consequentialist? I thought that was discredited when people showed that philosophy could be used to justify murdering someone in that trolley problem"). I've had no success arguing against these people, and one of the things that impressed me most the first time I read Overcoming Bias was that it was a community of people who used consequentialist arguments without fumbling for excuses, who thought Torture Vs. Dust Specks was a real problem instead of just "It would be wrong to inflict either of those options on people, so it's not a moral issue" (which is what my old philosophy professor said when I mentioned it to her).

I don't think Overcoming Bias ever officially argued for consequentialism, but a book aimed at the general public might have to. Eliezer is one of the most persuasive writers I've ever read, and if I thought his book could convince my friends to think consequentially I would be recommending it even more than I would otherwise.

Uncertainty is in the mind, not in reality. I was going to say the Mind-Projection Fallacy, but might as well make it very specific and focused. If you can make everyone I know believe intuitively that embracing "I don't know" does not cause reality to crumble... that'd be great.

We can get to "I don't currently know" later. :)

Something to Protect seems like the best way to get people to care about Rationality. I'd definitely want them to read that.

Thou Art Godshatter: gives an intuitive grasp for why and how human morality is complex, but that not any complex thing will do.

Clearly the earlier material is more important than the later. Include stuff like "The Bottom Line," "Update Yourself Incrementally," "Think Like Reality," "Conservation of Expected Evidence," "Avoiding Your Belief's Real Weak Points," the Fake Utility Functions sequence, &c. Also consider including the material between "But There's Still a Chance, Right?" through "0 and 1 are Not Probabilities," and "Mind Projection Fallacy" through "If You Demand Magic, Magic Won't Help." Ooh, and reprint the "Twelve Virtues"!

Don't mention quantum mechanics or the Singularity. Don't mention morality except for something along the lines of "Feeling Rational."

Politics is the mind-killer.

• Firstly, the combined ideas of "something to protect" and "rational agents should win" and "joy in the merely real." The idea that you should want to be rational not because it's inherently virtuous or important, but because it will allow you to get what you ultimately want, whatever that might be. The person I want to give this book to, currently believes that rationality is defined by cold, bloodless disinterest in the world, and thus has no interest in it.

• Secondly, the combined ideas of "writing the bottom line first" and "guessing the teacher's password" and "your ability to be more confused by fiction". That you cannot first choose an opinion from the ether, and then figure out how to argue for it. In the effort to correlate your beliefs with reality, it is only your beliefs that you are capable of changing. The person I want to give this book to, currently believes that it is better to "win" a debate than to "lose", and thus defends his beliefs against all comers without first finding out where they are come from.

• Thirdly, the idea that nature is allowed to set impossibly high standards and flunk you even if you do everything humanly possible to meet them. The person I want to give this book to, currently thinks that "I'm doing the best I can" is a viable excuse for failure.

Of course there should be advice on how, specifically, to be more rational, and the many failure modes possible. These three ideas are primarily the motivational ones: that rationality is necessary to anything you want to accomplish, and yet so difficult that it will take you a lifetime of effort to maintain a fighting chance of doing so. They are the ideas that, if internalized, will make people really want to try harder.

Just a week or two ago I found out -- to my utter astonishment -- that at least two people whom I see regularly have religious doubts about evolution -- in one case even about the age of the earth. They played various familiar cards, e.g.

-"You have faith in science [i.e. just like I have faith in religion]"

-"Evolution may work as science, but I must suspend judgement on whether it's true"

-"I believe in evolution within species"

-"How do you know the radioactive decay rates have remained constant?"

-"Are you a person of faith? [Answer: No.] You see? That's why this conversation will go absolutely nowhere." (Btw, anyone else notice how the religious are always the first ones to declare this?)

I was too shocked to be able to respond effectively. Would that I had had The Book to hand them.

I would ideally like to see a book of concrete controversies and scenarios that illustrate the principles of rationality. I envision it almost like a set of exercises. As you've noted before, the knowledge that a problem has been solved before tends to kill motivation, so they would need to be framed in a way to elicit curiosity. Quantum mechanics and p-zombies worked well for me, but I was already a long time traditional rationalist. Unfortunately, math tends to be scary to my friends and concepts like p-zombies wouldn't be considered relevant to the real world. Because an intro book has a lot of ground to cover, this may not be entirely feasible, but readers should have something to try their hand at.

For material to build up to controversies, I'd start with an explanation of guessing the teacher's password. I bet the concept rings true with most people who have sat through American high schools. Science classes give lots of answers but few explanations. The distinction between real and fake explanations also allows you to hold out the promise of real explanations to build curiosity.

Next, standard examples like the application of Bayes to medical tests and the Wason selection task (on letters and numbers vs. ages and beverages) are good introductory problems. They quickly show the necessity of a strict approach to rationality.

Next, Bayesian scoring à la Techical Explanation. I think my friends would love this, if it were explained right. Once you have a score, the urge is to maximize it. This brings home the points that there is a definite way to judge beliefs, the importance of making a prediction before hand, and the balance that must be struck between vagueness and precision. Beliefs have to pay rent, otherwise they are messing up your score.

Once predictions have been introduced, the book has to cover "the map != the territory" and the Mind Projection fallacy. Jaynes on the physics of coin-flipping did this for me. More "real-world" examples like drug testing might be useful here.

That is a lot of ground to cover, but it feels like a solid core. After that, practice on real issues (suitably divorced from ingrained beliefs) is important, both to hone skills and to make the reader feel like these principles are useful and important.

Tsuyoku Naritai.

Detailed explanation of why thought has rules, including minds as engines.

I'd have to say "12 virtues of rationality", though I'm not sure it counts since I don't think it was posted on OB and I wouldn't actually buy a book regardless of what's in it.

Science fiction with mind-blowing philosophy and hard science, preferably without in-jokes.

I might buy a concrete and practical handbook and recommend it to my friends. For example, I did that with the anti-depression handbook "Feeling Good". However, they wouldn't read it.

Greg Egan, Charlie Stross, and Nick Bostrom's Dragonology were all effective doors to interesting discussions, discussions that indicated my friends "got it". Nick Bostrom's more academic-ish stuff just bounced.

Ockham's Razor: Keeping it Simple. Making it Real. Getting Things Done.

I'm not sure how useful this is, and I feel odd posting it this way (Intuitive Rationality?), but there is a 'feel' to when there is a fallacy camouflaged in the discussion. If the reader could learn to pay attention to it and dig for it when they feel it, I would consider that a worthwhile book.

I'm putting another vote for the physics guys. I understand that the QM sequence may be too tangential to be included in a book about rationality, but all of the posts on "philosophy of science" that made MWI obviously true seem worth including.

Since I'm a physics major and my friends are too, the whole "philosophy of science" bit is what I'd loan out the book for (The bit on baye's law occam's razor, belief in the implied invisible, "the map is not the territory", minds as cognitive engines and such)

I guess the "here's what we're trying to do" part seems more interesting than (and should be learned before) the "here's where human minds consistently don't do that" part.

I think the explanation of how and why quantum mechanics is perfectly understandable and explainable would be the thing most likely to make me give the book to someone else. In many ways, its also the most valuable thing that I've taken away from the Overcoming Bias site.

I started out in Physics, and while in College and University, I often encountered the "don't try to understand it -- just do the math" point-of-view and I never subscribed to it. I always felt that there should be a way to make it all make sense, but I was never capable of doing so myself.

The articles on QM from Overcoming Bias resolved many of my doubts about how things worked in this realm, while showing that everything could be described by a consistent model that actually made sense. Its hard to overstate how happy I was to see that.

What would the world look like if X is true? What would it look like if X is false? How does the world actually look?

To me, that's the heart of it, the thing even many scientists don't get on a gut level.

I still go back to saying the most important thing that you've ever written, for me personally, was CFAI section 2.2, where you argued against anthropomorphism in AI. That was the thing that cracked my mind loose and set it on its journey without feeling trapped by traditional views of rationality, intelligence, or wisdom. Not that this appeared on OB as far as I can remember, but I think it's important enough that it should appear somewhere in the book.

I would like you to persuade my friends to hold their beliefs with much less confidence than they currently have. Overcoming Bias had this effect on me.

The general idea of science as Bayesianism. It would be a great resource for the irrational and scientists alike.

High status makes it more difficult to understand what other people are saying.

A friendly, concise overview of some of the most important techniques for improving thought, such as identifying cached thoughts and tabooing your words, holding off on proposing solutions, recognizing fake justification, leaving a line of retreat, righting wrong questions, identifying true sources of disagreement, etc.

This presupposes some motivating discussion of cognitive biases and why it is so unnatural to think rationally.

How to Bind Yourself To Reality is the number one thing people should GET. But my guess is that this one might not be teachable.

Make them catch the bug. Leave the reader enthusiastic to learn more. That's the #1 thing that would make me want to buy it for others, being confident what they get from it won't be a flash in the pan.

If your book makes less money for you than it's bibliography makes for Amazon, I'd call it a success.

Inferential Differences, so the understand(part of) why they have problems understanding other stuff I try to explain.

Clarity and transparency. One should be able to open the book to a page, read an argument, and see that it is right.

(Obviously this trades off against other values -- and is in some measure a deception --, but it's the kind of thing that impresses my friends.)

One Argument against an Army, Politics is the Mind-killer and related posts. (Actually, I'm surprised those posts don't reference each other, in my mind they're pretty close.)

Politics is probably the domain where I see the most emotional thinking around me (probably because people have little incentive to be actually right, and strong social incentives to go align with their peers).

I often promote Intentional Insights by talking about updating beliefs. I highlight the value of overcoming the negative emotions associated with learning about making a mistake, and instead encourage associating positive emotions with acknowledging reality and wanting to know the truth. This lets people take the best thing that they can from the mistake and move forward to achieve more!

.

Practical stuff. Real world applications. Preferentially with a inferential distance small. And personally i like anecdotes a lot. How Someone with his rationality foo managed to solve his problems.

If I gave the book to a friend it would probably be for carefully-argued futurism content.

A chapter on Science vs Probability Theory. eg Science isn't strict enough.

The stuff on quantum mechanics. I don't know that I'm a representative sample, though, because I know physicists.

What rationality really is? A concrete definition, and not a "virtue which is nameless"..

I don't buy books, and I don't have friends.