Rationality: From AI to Zombies

The cover is incorrect :(

EDIT: If you do not understand this post, read essay 268 from the book!

Perhaps this is already discussed elsewhere and I'm failing at search. I'd be amazed if the below wasn't already pointed out.

On rereading this material it strikes me that this text is effectively inaccessible to large portions of the population. When I binged on these posts several years ago, I was just focused on the content for myself. This time, I had the thought to purchase for some others who would benefit from this material. I realized relatively quickly that the purchase of this book would likely fail to accomplish anything for these people, and may make a future attempt more difficult.

I think many of my specific concerns apply to a large percentage of the population.

• The preface and introductions appear aimed at return readers. The preface is largely a description of 'oops', which means little to a new reader and is likely to trigger a negative halo effect in people who don't yet know what that means. - "I don't know what he's talking about, and he seems to make lots of writing mistakes."
• There isn't a 'hook'. Talking about balls in urns in the intro seems too abstract for people. The rest of the sequences have more accessible examples, which most people would never reach.
• Much of the original rhetoric is still in place. Admittedly that's part of what I liked about the original posts, but I think it limits the audience. As a specific example, a family member is starting high school, likes science, and I think would benefit from this material. However her immediate family is very religious, to the point of 'disowning' a sister when they found out about an abortion ~25 years ago. The existing material uses religion as an example of 'this is bad' frequently enough that my family member would likely be physically isolated from the material and socially isolated from myself. 87% of America (86% global) have some level of belief in religion. The current examples are likely to trigger defensive mechanisms, before they're education about them. (Side-note: 'Waking Up: A Guide to Spirituality Without Religion – by Sam Harris' is a good book, but has this same exact issue.)
• Terminology is not sufficiently explained for people seeing this material with fresh eyes. As an example, ~15% of the way through 'New Improved Lottery' talks about probability distributions. There was no previous mention of this. Words with specific meanings, that are now often used, are unexplained. 'Quantitative' is used and means something to us, but not to most people. The Kindle provided dictionary and Wikipedia definitions are not very useful. This applies to the chapter titles as well, such as 'Bayesian Judo'.
• The level of hyperlinks, while useful for us, is not optimal for someone reading a subject for the first time. A new reader would have to switch topics in many cases to understand the reference.
• References to LessWrong and Overcoming Bias and only make sense to us.

Eliezer and Robb have done a lot to get the material into book state... but it's preaching to the choir.

Specifically what I think would make this more accessible:

• A more immediate hook along the lines of 'Practicing rationality will help you make more winning decisions and be less wrong.' (IE: keep reading because this=good and doable) Eliezer was prolific enough that I think good paragraphs likely already exist; but need connectors.
• Where negative examples are likely to dissuade large numbers of people, find better examples. In general avoid mentions of specific politics or religion in general. It's better to boil the frog.
• Move or remove all early references to Bayes. 'Beliefs that are rational are call Bayesian' means nothing to most people. Later references might as well be technobabble.
• Make sure other terminology is actually explained/understandable before it's used in the middle of an otherwise straightforward chapter. I'd try 1n & 2n-gramming the contents against Google Ngrams to identify terminology we need to make sure is actually explained/understood before casual use.
• Get this closer to a 7th grade reading level. This sets a low bar at potential readers who can understand 'blockbuster' books in English. (This might be accomplished purely with the terminology concern/change above)
• Change all hyperlinks to footnotes.
• Discuss LessWrong, Overcoming Bias, Eliezer, Hanson in the preface as 'these cool places/people where much of this comes from' but limit the references within the content.

Is there any ongoing attempt or desire to do a group edit of this into an 'Accessible Rationality'?

Just a reminder that mistakes/problems/errors can be sent to errata@intelligence.org and we'll try fix them!

Yay! Now I'm sending this to all of my friends!

One of the most common complaints about the old Sequences was that there was no canonical default order, especially for people who didn't want to read the entire blog archive chronologically.

I was tricked into doing this. Years ago someone posted an ebook claiming to be the Sequences, but was actually just every single Yudkowsky blog post from 2006 to 2010 -_-

It took until noticing that only Yudkowsky's side of the FOOM debate was in there that I realized what had happened

Good work guys!

This might be the excuse I need to finally go through the complete sequences as opposed to relying on cherry-picking posts whenever I encounter a reference I don't already know.

Excellent, thank you! Any update on when the real book will be available for purchase for those of us who don't do ebooks?

Can I know to who and where the money for the book goes?

I am impressed. The production quality on this is excellent, and the new introduction by Rob Bensinger is approachable for new readers. I will definitely be recommending this over the version on this site.

I paid $0 because I'd rather not pay transaction fees on a donation to charity. You can donate to MIRI directly here:

https://intelligence.org/donate/

And CFAR here:

http://rationality.org/donate/

The zip file has some extra Apple metadata files included. Nothing too revealing, just dropbox bits.

Congratulations, well done!

Side note: the "Glossary" link seems to be broken.

For reasons, I suggest that Bayesian Judo doesn't make EY look good to people who aren't already cheering for his team, and maybe it wasn't wise to include it.

More generally, the book feels a bit... neutered. Things like, for example, changing "if you go ahead and mess around with Wulky's teenage daughter" to "if you go ahead and insult Wulky". The first is concrete, evocative, and therefore strong, while the latter is fuzzy and weak. Though my impression may be skewed just because I remember the original examples so well.

I am thinking of recommending this to people, all of whom are unlikely to pay. Is having people acquire this for $0 who would otherwise not have read it beneficial or harmful to MIRI? (If the answer is "harmful because of paying for people to download it", I can email it to my friends with a payment link instead of directing them to your website.)

With SumatraPDF 3.0 on Windows 8.1 x64, the links in the PDF version do not show up. With Adobe Reader 11 on Windows 7 x86, they look fine. On the other hand, SumatraPDF can also handle the MOBI and EPUB versions.

I have been creating a tex version at: https://github.com/jrincayc/rationality-ai-zombies

Awesome! How large is it altogether (in words)?

Hi, and thanks for the awesome job! Will you keep a public record of changes you make to the book? I'm coordinating a translation effort, and that would be important to keep it in sync if you change the actual text, not just fix spelling and hyperlinking errors.

Edit: Our translation effort is for Portuguese only, and can be found at http://racionalidade.com.br/wiki .

I liked Robby's introduction to the book overall, but I find it somewhat ironic that right after the prologue where Eliezer mentions that one of his biggest mistakes in writing the Sequences was focusing on abstract philosophical problems that are removed from people's daily problems, the introduction begins with

Imagine reaching into an urn that contains seventy white balls and thirty red ones, and plucking out ten mystery balls.

The first (though not necessarily best) example of how to rewrite this in less abstract form that comes to mind would be something like "Imagine that you're standing by the entrance of a university whose students are seven tenths female and three tenths male, and observing ten students go in..."; with the biased example being "On the other hand, suppose that you happen to be standing by the entrance of the physics department, which is mostly male even though the university in general is mostly female."

Some unnecessary technical jargon that could have been gotten rid of also caught my eye in the first actual post: e.g. "Rational agents make decisions that maximize the probabilistic expectation of a coherent utility function" could have been rewritten to be more broadly understandable, e.g. "rational agents make decisions that are the most likely to produce the kinds of outcomes they'd like to see".

I could spend some time making notes of these kinds of things and offering suggested rewrites for making the printed book more broadly accessible - would MIRI be interested in that, or would they prefer to keep the content as is?

Don't have paypal or credit card or bitcoins or similar stuff, 0 price for now, I will look into donating from my Maestro debit card or maybe a direct transfer although international transfer rates may make that not worth the while. That and cash are the only methods I use - I rarely need anything I cannot buy with them. (I use gift cards purchased in shops for steam and google play.) I am thinking about purchasing some bitcoins for € for such donations purpose, if anyone can recommend a safe and debit card (or sofort.com) compatible service?

That's awesome!

Do people think there is value in making an audio book from this?

I was thinking it would be possible to do in a similar process to the HPMOR audiobook with people contributing different chapters. If there is interest in doing this and if it is permitted to be done then I will happily volunteer to coordinate the effort. If this idea does have support then given the discussion below about how the book could be improved, would it make more sense to postpone an audiobook to allow for sensible changes, or is that an unnecessary delay in search of unreachable perfection?

I just finished listening to the Audiobook version of Rationality: From AI to Zombie. Lots of thanks to Yudkowsky and everyone else that was involved in making this book and the audio book. I do not know who the reader of the audio book is, but thanks all the same.

I am writing this comment as my way of prizing this book. I will try to summarize what I have personalty learned from it, in the hope that someone who was involved, will read this post and fell some pride in having helped me in my self improvement. But I am also writing this comment because I just want to express my thoughts after finishing the book.

I have not have any major change of mind, but I have several minor ones, which might very well continue to grow.

Listening to Yudkowsky's words have made me more confident, because he is saying many things that I already intuitively knew, but I could not properly explain it my self, and could therefore not be sure I was right. I am still not 100% certain I am right, but I am more confident, and I believe that this is a good thing. Smart people should be confident. No, this is not hind site bias, because:

• I did not allays instantly agree, so I do know the difference.
• I been actively introspecting since I was 12, so I know most of my brains tricks.

I never set out to be a rationalist. I don't even remember having a pre-LessWrong concept for the word "rationalist". There where just, correct thinking and in-correct thinking, and obviously correct thinking is the way that systematically leads you to the truth, because how else would you measure correctness. Maybe this saved me from falling in to some of the rationalist tropes that Yudkovsky warns about. Or maybe I avoided them because I have read to little science fiction. Or maybe it was because I looked at these types of tropes and saw an author who kinged to the, obviously wrong, but warm and fussy, idea that every human has the same number of skill points.

I wonder who setts out to be rational, with out having something specific they need rationality for. Maybe the same kind of people that identifies as an atheist? I am an atheist, but I don't identify as such, because in my country, this is mostly a non issue.

I found LessWrong because my new boyfriend encouraged me to read here, and I actually got through the book, because I like audiobooks.

The pre-LessWrong me was a truth seeker, and as such, I though a lot about the way as applied to truth seeking. I had a crisis of faith, a several years a go, questioning the validity of science. But never really though about applying systematic reasoning to decision under uncertainty. When, in my past, I was confronted with a decision, which I did not know how to reason out, I used to deliberately hand over the decision to my feelings. Because, I reasoned, if I don't know what is right anyway, I might as well save me the fight of going against my impulses. I hope that I can use what I have learned here to do better.

An other thing I have realized is that I am such a pushover for perceived social norms. I have notice a significant mental shift in my brain, just from having some one in my ear, who casually mentions many words and cryonics, as if these where the most normal things in the world. Intellectually I was already convinced, I already knew the right answer before listening to the book, but I still needed the extra nagging, to get all of may brain on board with it. I think that this has been the single most important insight I got from the book.

One reason I have not tried to develop the art of rational decision making before, is that I knew that I was not strong enough to counter my emotional preferences. But I was wrong. I now have one, systematically applicable self hack, and probably there are more out there to find. I have hope to be able to take charge of my motivation, and I have reasons to fight for control.

Current me is an aspiring effective altruist. I do not strive to be a perfect altruist, because I do have some selfish preferences, that I do not expect to go away. But I am going to get my ass out of the comfortable bubble of I can''t do anything anyway, and do something. Though I have not decided yet if I am going to take the path of earn to give, or if I should get directly involved in some project my self. I am looking in to both ways.

Finlay, here is one my favorite quotes from the book:

I pause. “Well…” I say slowly. “Frankly, I’m not entirely sure myself where this ‘reality’ business comes from. I can’t create my own reality in the lab, so I must not understand it yet. But occasionally I believe strongly that something is going to happen, and then something else happens instead. I need a name for whatever-it-is that determines my experimental results, so I call it ‘reality’. This ‘reality’ is somehow separate from even my very best hypotheses. Even when I have a simple hypothesis, strongly supported by all the evidence I know, sometimes I’m still surprised. So I need different names for the thingies that determine my predictions and the thingy that determines my experimental results. I call the former thingies ‘belief’, and the latter thingy ‘reality’.”

Is a printed six-volume set still being worked on?

Does the book (especially the printed version) have training problems after sections? (I don't have it, sorry if the question is redundant).

Might be worth including the Amazon.co.uk and other store links.

A friend of mine is interested in reading this book, but would prefer a printed copy. Is there any chance that this book will be published any time soon?

Is there anything on procrastination? I'm tempted to buy this bookinstead cause the dude has an alright podcast too. I don't listen to it anymore cause it's boring and not consistently novel information but yeah.

When I feel like this I don't want to read chapters that are complex sounding like Rationality and Politics and Death Spirals that without having read the sequences, don't mean shit to me and could equally appear in some random Trotskyist propoganda from the weird organisation down the road.

When are these pop-rationality books gonna be replaced by a new generation of books on say Bonferri corrections for everyday life or a conceptual introduction to regression?

you're reaching neither the unitiated nor furthering the knowledge of the adepts. You're just preaching to the choir and making some coin from it! Defend your honour!

edit 1: fixed links

Sorry for my problem.I tried 15 times downloading,only once started and stopped at 1.5M/30.6M. Others can't even get to track. I wish to use another source ,or some kind friends could send the pdf to 513493106@qq.com?DEEPLY BOW for your help!