New edition of "Rationality: From AI to Zombies"

post by Rob Bensinger (RobbBB) · 2018-12-15T21:33:56.713Z · score: 78 (29 votes) · LW · GW · 23 comments

MIRI is releasing a new edition of Rationality: From AI to Zombies, including the first set of R:AZ print books. As of this morning, print versions of Map and Territory (volume 1) and How to Actually Change Your Mind (volume 2) are now available on Amazon (1, 2), and we'll be rolling out the other four volumes of R:AZ over the coming months.

R:AZ is a book version of Eliezer Yudkowsky's original sequences, collecting a bit under half of his Overcoming Bias and LessWrong writing from November 2006 to September 2009. Map and Territory is the canonical place to start, but we've tried to make How to Actually Change Your Mind a good jumping-on point too, since some people might prefer to dive right into HACYM.

The price for the print books is $6.50 for Map and Territory, and $8 for How to Actually Change Your Mind. The new edition is also available electronically (in EPUB, MOBI, and PDF versions) on a pay-what-you-want basis: 1, 2. The HACYM ebook is currently available for preorders, and should be delivered in the next day.

The previous edition of R:AZ was a single sprawling 1800-page ebook. I announced at the time that we were also going to release a paper version divided into six more manageable chunks; but this ended up taking a lot longer than I expected, and involved more substantive revisions to the text.

Changes going into the new edition include:

Oliver and Ben also plan to post the digital versions of M&T and HACYM to R:AZ on LessWrong — initially as new posts, though the URLs and comment sections of new and old versions may be merged in the future if LW adds a feature for toggling between post revisions.


Comments sorted by top scores.

comment by Said Achmiz (SaidAchmiz) · 2018-12-19T22:51:44.837Z · score: 23 (6 votes) · LW · GW

(I imagine this comment properly belongs here on Less Wrong, when the new versions are posted here; perhaps the mods can move it, then. For now, posting it here because I can’t find a better place for it.)

From the introduction to Map and Territory, section “Noticing Bias”:

Imagine meeting someone for the first time, and knowing nothing about them except that they’re shy.

Question: Is it more likely that this person is a librarian, or a salesperson?

Most people answer “librarian.” Which is a mistake: shy salespeople are much more common than shy librarians, because salespeople in general are much more common than librarians—seventy-five times as common, in the United States.[1]

This is base rate neglect: grounding one’s judgments in how well sets of characteristics feel like they fit together, and neglecting how common each characteristic is in the population at large.[2]

Reading this, I immediately noticed that integrating base rates is not sufficient to make the “librarian” answer a mistake; and therefore we cannot conclude that the reason why people answer thus, is base rate neglect.

In fact, the text does not establish that answering “librarian” is wrong. Consider the claims:

  1. “salespeople in general are much more common than librarians—seventy-five times as common, in the United States”

  2. “shy salespeople are much more common than shy librarians”

The first claim is specific—we’re given a figure (“seventy-five times as common”)—and referenced. (The first footnote cites Weiten, Psychology: Themes and Variations, Briefer Version, Eighth Edition, 2010.) The second claim is neither quantified nor cited. Hmm.

But this is basic Bayes: in order to conclude #2 from #1, we also need another claim, call it #3: that observing that someone is shy does not provide strong enough evidence to overcome our prior probability distribution over a randomly selected person’s profession, and shift our posterior estimate such that “librarian” becomes more likely than “salesperson”. This claim can easily be false—namely, in the case that the proportion of librarians who are shy is at least seventy-five times greater than the proportion of salespeople who are shy.

If claim #3 is false, then answering “librarian” is correct! But also, even if claim #3 is true, then the error of someone answering “librarian” may be simply mis-estimating the relative rates of shyness among salespeople and librarians—which would not be an example of the base rate fallacy.

The text does not comment on this. Perhaps it is assumed that librarians aren’t shy at a rate 75 times greater than salespeople are. (But this is already a pedagogical flaw! And it ignores my second point in the paragraph above…) But is it true? From the provided information, we don’t know.

Note that my criticism of the text of M&T stands regardless of what the facts of the matter are—even if claim #3 is true, the text is, as I say above, quite flawed. Nevertheless, I was curious, and got my hands on a copy of Weiten 2010. The relevant bit is on page 270:

Steve is very shy and withdrawn, invariably helpful, but with little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure and a passion for detail. Do you think Steve is a salesperson or a librarian? (Adapted from Tversky & Kahneman, 1974, p. 1124)

Using the representativeness heuristic, participants tend to guess that Steve is a librarian because he resembles their prototype of a librarian (Tversky & Kahneman, 1982). In reality, this is not a very wise guess, because it ignores the base rates of librarians and salespeople in the population. Virtually everyone knows that salespeople outnumber librarians by a wide margin (roughly 75 to 1 in the United States). This fact makes it much more likely that Steve is in sales. But in estimating probabilities, people often ignore information on base rates.

(The book does not discuss this particular matter further, though it goes on to further discussion of base rate neglect.)

Note that the “75 to 1” figure, in particular, does not appear to be referenced; nor is there any discussion of the relative rates of shyness among salespeople and librarians. There also does not seem to be any discussion of the possibility, in general, that people are taking base rates into account but mis-estimating the strength of the evidence (though I read only that one section, and not the entire 750-page textbook, so perhaps I missed it).

The two in-text citations in the above quote refer to:

Tversky, A., & Kahneman, D. (1974). Judgments under uncertainty: Heuristics and biases. Science, 185, 1124–1131.

Tversky, A., & Kahneman, D. (1982). Judgment under uncertainty: Heuristics and biases. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases. New York: Cambridge University Press.

The first source (Tversky & Kahneman, 1974—a paper) contains no mention or discussion of the relative numbers of salespeople and librarians in the general population (and, in fact, describes an experiment where the base rates involved were not base rates in the general population, but rather base rates in an artificially constructed set, making the applicability questionable), nor of their relative rates of shyness.

In the second source (Tversky & Kahneman, 1982—a book), the relevant bit (on page 4) seems to be almost verbatim identical, with minor editing, to the first source. There is no additional information on any of the above topics.

So, tracing back the citations in Map and Territory, we seem to find no good basis at all for a strong conclusion that people who are asked the “salesperson vs. librarian” question ignore base rates; nor for rejecting the possibility that people merely mis-estimate relative rates of shyness among salespeople vs. librarians; nor any information on what those relative rates are; nor for the claim that “most” (or even “many”!) people answer in the alleged way; nor even for the claim that answering “librarian” is a mistake at all…!


I decided to check whether even the “75 to 1 salesperson:librarian ratio” claim is true (remember, said claim in Weiten 2010 is not sourced at all).

The U.S. Bureau of Labor Statistics seems to claim that there are approximately 138,200 librarians in the United States. The American Library Association gives the number as 166,164. (Differences in sources, definitions, and periods of measurement likely explain the discrepancy.)

Bureau of Labor Statistics figures show 14,522,580 as the number of employees in “Sales and Related Occupations”—though this includes many jobs, like “Cashier”, which do not seem to fit the intuitive definition of “salesperson” in the way relevant to the “shyness” question above.

Ignoring that quibble, and taking the average of the two figures above for number of librarians, yields a 96:1 salesperson:librarian ratio in the United States, which is less than an order of magnitude off from the 75:1 figure quoted in Weiten 2010—which difference may be caused by the passage of time. In any case, I think we can call this one factoid more or less confirmed.

comment by habryka (habryka4) · 2018-12-19T23:07:58.631Z · score: 4 (2 votes) · LW · GW

Yep, will move it over as soon as we have the new posts up.

comment by Said Achmiz (SaidAchmiz) · 2018-12-15T22:26:43.932Z · score: 12 (5 votes) · LW · GW

Is the new version being released under the same license terms (CC BY-NC-SA 3.0) as the previous version was?

comment by Rob Bensinger (RobbBB) · 2018-12-15T22:47:09.702Z · score: 13 (8 votes) · LW · GW

The new version is under CC BY-NC-SA 4.0.

comment by Viliam · 2018-12-15T23:36:43.338Z · score: 11 (8 votes) · LW · GW

This is impressive. I am not going to read the text again to see what has changed, so no opinion on that part, but visually, it finally looks like a book. (Not like a bunch of screenshots from a blog.) Especially how the hyperlinks were changed; that was quite painful previously.

A part of me hopes that after publishing the 6 parts of the original Sequences, the series will continue. There were other things written during those years, both by Eliezer and by others.

comment by NicholasKross · 2018-12-19T03:55:45.163Z · score: 8 (5 votes) · LW · GW

There is something weirdly powerful about these being in well-printed form, more approachable in form and content. This is something easier to recommend to friends without having to go "By the way this post references a thing that is no longer relevant / is poorly researched" like the Robber's Cave mentioned.

comment by Viliam · 2018-12-19T20:48:24.478Z · score: 5 (3 votes) · LW · GW

To put it bluntly, a science-related book with bad design just screams "crackpot".

comment by NicholasKross · 2018-12-21T04:31:08.242Z · score: 3 (2 votes) · LW · GW

Indeed. Reminds me of how science communication these days has kind of expanded, partly because the tools of (good) design are more accessible. A quick look at the (good) science channels on YouTube shows kinda the opposite. Of course, this can lead to other problems (if science + bad-design = crackpot, people can perceive BS + good-design as = true). Rhyme-as-reason effect, style-as-substance, etc.

comment by anna_macdonald · 2018-12-16T00:06:06.113Z · score: 9 (6 votes) · LW · GW

That sounds great.

Out of curiosity, does the glossary include terms that aren't particularly rationality-related, but which may not be familiar to less-scientifically-interested readers? (Examples: light cones, configuration space).

comment by Rob Bensinger (RobbBB) · 2018-12-16T00:14:45.275Z · score: 10 (7 votes) · LW · GW

Yep! It doesn't try to include literally every term or reference someone might want to google, but it includes terms like a priori, bit, deontology, directed acyclic graph, Everett branch, normative, and orthogonality, in addition to more rationality-specific terms. The kinds of terms we leave out are ones like "IRC" where some people might need to google the term, but it's not really important enough to warrant a glossary entry.

comment by Senarin · 2018-12-16T06:19:43.201Z · score: 5 (4 votes) · LW · GW

No hardcover option?

comment by Rob Bensinger (RobbBB) · 2018-12-16T07:22:43.274Z · score: 6 (4 votes) · LW · GW

Not at present. Some people requested that we release higher-quality versions, so that's been on our radar, and I'd be interested to hear what kinds of variants people would and wouldn't be interested in buying. (Full-color, leather-bound, hardcover, etc.)

comment by ioannes_shade · 2018-12-16T17:48:47.126Z · score: 9 (6 votes) · LW · GW


comment by habryka (habryka4) · 2018-12-16T18:16:22.198Z · score: 8 (5 votes) · LW · GW

Would definitely buy a Leatherbound version

comment by justinpombrio · 2018-12-16T17:00:17.227Z · score: 7 (5 votes) · LW · GW

One thing to keep in mind is that---whether or not it should---price suggests quality. The paperback books are cheap (are you selling them at-price?), which makes me think "mass production novel", rather than "deeply impactful nonfiction". It might be worth putting out an overpriced high-quality version for this reason alone.

And I would be very happy to buy a high-quality version of the books. I like hard covers. Leather-bound would be impressive.

comment by Rob Bensinger (RobbBB) · 2018-12-16T17:48:27.203Z · score: 6 (4 votes) · LW · GW

Yep, there are good reasons to go for a cheaper edition (e.g., people can buy dozens of copies to pass out without breaking bank) and also to go for a more expensive edition. It makes sense to have one version that's very optimized for affordability (the current version, which is good-quality but roughly at cost), and a separate version that's optimized for other criteria. My main uncertainty is about which features Less Wrong readers are likely to care the most about, and how much those features are worth to them.

comment by NicholasKross · 2018-12-19T03:58:50.216Z · score: 5 (3 votes) · LW · GW

Definitely think it's a good idea to go for the two optimized versions. Kinda like many (most?) of the classic novels from history: cheap mass edition + luxurious "pro" version. (Not that the content would actually differ between them, unless there's some good reason for that I'm not thinking of; "Hey, the leatherbound version has more original post text and tangents!").

comment by Senarin · 2018-12-17T05:44:43.324Z · score: 4 (3 votes) · LW · GW

Please have a non-leather hardcover option for us vegans. :)

comment by Rob Bensinger (RobbBB) · 2018-12-20T21:06:45.523Z · score: 3 (2 votes) · LW · GW

Yes :) I wasn't thinking real leather, though maybe synthetic leather also has signaling problems..!

comment by Eggrenade · 2018-12-19T21:34:39.882Z · score: 3 (2 votes) · LW · GW

I would love an elaborate collector's edition box set for my coffee table when all six books are revised. The version on my phone is the one I'm actually going to read, but this is the kind of thing I want friends to see in my apartment when they visit.

comment by Senarin · 2018-12-17T05:43:21.605Z · score: 0 (2 votes) · LW · GW

Please no real leather for us vegans. :)

comment by Said Achmiz (SaidAchmiz) · 2018-12-15T22:21:31.554Z · score: 5 (3 votes) · LW · GW

It seems that is currently down.


The new version of Map and Territory is also available electronically (in EPUB, MOBI, and PDF versions)

This site also seems to be down.

comment by Rob Bensinger (RobbBB) · 2018-12-15T22:40:07.161Z · score: 5 (3 votes) · LW · GW

Hmm, Chrome seems to have modified the URLs. Try and (no HTTPS) instead.

The link is currently a redirect to the MIRI R:AZ book page while we wait for TrikeApps to finish setting up the proper book page. I figured it was better to release now rather than wait for the finished website and the HACYM ebook, since the print edition will take a few days to deliver and some people will probably want to buy these as holiday presents.

(Amazon currently says it can deliver copies to me by Dec. 18-19 in California.)