Help us Optimize the Contents of the Sequences eBook

post by lukeprog · 2013-09-19T04:31:20.391Z · score: 11 (12 votes) · LW · GW · Legacy · 73 comments

Contents

  Map and Territory
  Mysterious Answers to Mysterious Questions
  A Human's Guide to Words
  How to Actually Change Your Mind
    Politics is the Mind-Killer
    Death Spirals and the Cult Attractor
    Seeing with Fresh Eyes
    Noticing Confusion
    Against Rationalization
    Against Doublethink
    Overly Convenient Excuses
    Letting Go
  The Simple Math of Evolution
  Challenging the Difficult
  Yudkowsky's Coming of Age
  Reductionism
  Quantum Physics
  Metaethics
  Fun Theory
  The Craft and the Community
  Appendix
None
73 comments

MIRI's ongoing effort to publish the sequences as an eBook has given us the opportunity to update their contents and organization.

We're looking for suggested posts to reorder, add, or remove.

To help with this, here is a breakdown of the current planned contents of the eBook and any currently planned modifications. Following that is a list of the most popular links within the sequences to posts that are not included therein.

Now's a good time to suggested changes or improvements!

———

Map and Territory

Added …What's a Bias Again? because it's meant to immediately follow Why Truth, And….

Mysterious Answers to Mysterious Questions

No changes.

A Human's Guide to Words

No changes.

How to Actually Change Your Mind

Politics is the Mind-Killer

Removed The Robbers Cave Experiment because it already appears in Death Spirals and the Cult Attractor, and there in the original chronological order which flows better.

Death Spirals and the Cult Attractor

Removed The Litany Against Gurus because it already appears in Politics is the Mind-killer.

Seeing with Fresh Eyes

Removed Asch's Conformity Experiment and Lonely Dissent because they both appear at the end of Death Spirals. Removed The Genetic Fallacy because it's in the Metaethics sequence: that's where it falls chronologically and it fits better there with the surrounding posts.

Noticing Confusion

Removed this entire subsequence because it is entirely contained within Mysterious Answers to Mysterious Questions.

Against Rationalization

Added Pascal's Mugging (before Torture vs Dust Specks) because it explains the 3^^^3 notation. Added Torture vs Dust Specks before A Case Study of Motivated Continuation because A Case Study refers to it frequently.

Against Doublethink

No changes.

Overly Convenient Excuses

Removed How to Convince Me that 2+2=3 because it's already in Map & Territory.

Letting Go

No change.

The Simple Math of Evolution

Added Evolutionary Psychology because it fits nicely at the end and it's referred to by other posts many times.

Challenging the Difficult

No change.

Yudkowsky's Coming of Age

No change.

Reductionism

No change. (Includes the Zombies subsequence.)

Quantum Physics

No change. Doesn't include any "Preliminaries" posts, since they'd all be duplicates

Metaethics

No change.

Fun Theory

No change.

The Craft and the Community

No change.

Appendix

Includes:

———

Here are the most-frequently-referenced links within the sequences to posts outside of the sequences (with a count of three or more). This may help you notice posts that you think should be included in the sequences eBook.

Suggestions?

73 comments

Comments sorted by top scores.

comment by Bobertron · 2013-09-19T08:27:32.938Z · score: 21 (21 votes) · LW · GW

I tend to think of "Diseased thinking: dissolving questions about disease " as belonging into the "A Human's Guide to Words" sequence. As I remember, the example of "what is a desease" is much more relevant, motivating and enlightening than any example in Yudkowsky's sequence.

comment by David_Gerard · 2013-09-19T09:55:08.634Z · score: 14 (14 votes) · LW · GW

Will the QM sequence be checked over for mathematical errors? These have been used in practice to quickly dismiss the sequence.

comment by Luke_A_Somers · 2013-09-19T17:14:27.075Z · score: 6 (8 votes) · LW · GW

You can build arbitarily-phase-shifting optical components. There's no reason one couldn't make half-silvered mirrors with a coating that makes it act like Eliezer's... and any physicist ought to know this. Plus, the real issue is the total difference in phase across the two paths, and you can tweak that however you like by adjusting the path lengths.

SO, either fix it numerically or include a note to that effect, because there's no reason this needs to fall to a silly nitpick.

comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2013-09-19T21:03:34.063Z · score: 4 (4 votes) · LW · GW

To be clear, is the criticism just wrong, or should the sequence be adjusted? What exactly needs to be done to fix it numerically?

comment by Luke_A_Somers · 2013-09-20T03:21:50.352Z · score: 7 (7 votes) · LW · GW

Both. The sequence is technically off on that detail, but someone who knows their stuff should know better than to complain about it. The simplest fix is to just say that this is a special custom half-silvered mirror that has these phase shifts.

I was going to say that the cleanest fix would be to make the math right, but looking at the math, it seems that the real math is so much messier and harder to explain (half-silvered mirrors invert or not based on which side you half-reflected from!) that getting it right would muddy the waters far more than using your power of arbitrary setups to make an idealized apparatus.

comment by Paul Crowley (ciphergoth) · 2013-09-20T16:01:10.012Z · score: 3 (5 votes) · LW · GW

Motivated reasoning made its masterpiece there - the guy does know his stuff, but he basically found one tiny nit and went "THIS IS CLEARLY ALL BOGUS, AS AN EXPERT I IMPLORE YOU ALL TO IGNORE EVERY WORD". That was why I posted to Physics Exchange about it, to demonstrate to a friend that when he says "trust me, I'm an expert" you can't trust him.

comment by shminux · 2013-09-19T21:59:20.651Z · score: 4 (8 votes) · LW · GW

Not directly related, but just wanted to mention that unqualified statements like

"A photon heading toward A"—and it's out there in the territory.

are way too strong, and unnecessarily so.

A single photon is not a part of reality, it's just a simplified convenient model of electromagnetic emission and absorption, which happens to work well in certain circumstances. Consider the question "When was the photon emitted?". Unless you measure the recoil of the emitter (a classical measurement), you don't know. You can reconstruct this time if you use a sensitive enough photomultiplier in the detector, which is again a classical measurement.

A somewhat more detailed QED-based model describes spontaneous emission as unitary evolution of an excited atom into a superposition of the multitude of the states of the form (a recoiled ground state of atom * a corresponding excited state of the EM field). This is not a well-defined single photon, but rather a superposition of all possible photons emitted at all possible times in all possible directions. What makes it into a single photon is the post-selection process (selecting a single possible world out of infinitely many ones, in the MWI picture).

So "A photon heading toward A" is not the territory, it's a useful simplification of a more accurate model. Which only underscores the point that "distinct configurations are not distinct particles".

comment by Luke_A_Somers · 2013-09-20T03:22:01.294Z · score: 0 (0 votes) · LW · GW

Would you be satisfied if he pointed out that there's no way to achieve such a well-specified initial condition in real life?

comment by shminux · 2013-09-20T04:00:43.084Z · score: 1 (3 votes) · LW · GW

I would be happy if he did not claim that a specific model is the reality.

comment by Luke_A_Somers · 2013-09-20T12:53:47.613Z · score: 0 (2 votes) · LW · GW

That's a different point you raised elsewhere, yes. I meant on the point you raised above.

Would you be satisfied on this other front if he restricted himself to saying that whatever it ends up being, it's not going to be an objective collapse theory? A distressingly large number of people haven't gotten that memo.

comment by shminux · 2013-09-20T14:53:00.522Z · score: 1 (3 votes) · LW · GW

Oh, I don't disagree that claiming that some form of objective collapse is the "reality" is not very smart. While QM can be simulated this way, and usually is, there is no reason to expect that this ad hoc rule is as deep as it gets. Unless we are in a poorly written simulation.

comment by Strilanc · 2014-12-17T09:03:04.157Z · score: 1 (1 votes) · LW · GW

A concrete example of a paper using the add-i-to-reflected-part type of beam splitter is the "Quantum Cheshire Cats" paper:

A simple way to prepare such a state is to send a horizontally polarized photon towards a 50:50 beam splitter, as depicted in Fig. 1. The state after the beam splitter is |Psi>, with |L> now denoting the left arm and |R> the right arm; the reflected beam acquires a relative phase factor i.

The figure from the paper:

I also translated the optical system into a similar quantum logic circuit:

Note that I also included the left-path detector they talk about later in the paper, and some read-outs that show (among other things) that the conditional probability of the left-path detector having gone off, given that D1 went off, is indeed 100%. (The circuit editor I fiddle with is here.)

It's notable that my recreation uses gates with different global phase factors (the beam splitter is 1/2-i/2 and 1/2+i/2 instead of 1/sqrt(2) and i/sqrt(2)). It also ignores the mirrors that appear once on both paths. The effect is the same because global phase factors don't matter.

edit My ability to make sign errors upon sign errors is legendary and hopefully fixed.

comment by notsonewuser · 2013-09-19T15:16:49.733Z · score: 12 (12 votes) · LW · GW

I agree with moridinamael...nix "The Simple Truth". I know it's not really from the original sequences, but I think "The Useful Idea of Truth" should be used instead.

comment by Coscott · 2013-09-20T17:49:25.040Z · score: 3 (3 votes) · LW · GW

I think this adding "The Useful Idea of Truth" is the most important change. The first post is the most important, and even if you want to call it "Sequences 2006-2009," you should still open with a post designed not to be informative as much as to catch readers. I think the "The Useful Idea of Truth" is the current best option. What would be even better would be if Eliezer wrote a new intro.

comment by shminux · 2013-09-19T16:25:06.064Z · score: 0 (4 votes) · LW · GW

I liked The Simple Truth as a cute little story... I still have no idea what the intended message was supposed to be. It's not even clear if the guy who decided to experimentally and dramatically test his model of the world survives through quantum immortality.

comment by RomeoStevens · 2013-09-20T00:36:06.348Z · score: 3 (3 votes) · LW · GW

The intended message was that a lot of the dead ends in epistemology are pretty silly when considered explicitly.

comment by shminux · 2013-09-20T01:34:46.417Z · score: 0 (2 votes) · LW · GW

What was the dead end?

comment by RomeoStevens · 2013-09-20T05:59:55.634Z · score: 3 (3 votes) · LW · GW

every other sentence markos makes is a reference to some famous epistemological claim.

comment by Raiden · 2013-09-20T14:11:14.245Z · score: 1 (1 votes) · LW · GW

I really like the cute little story as you say, but agree that it isn't effective where it is. Maybe include it in the end as a sort of appendix?

comment by [deleted] · 2013-09-19T16:50:59.650Z · score: 6 (6 votes) · LW · GW

fake fake utility functions is an important subsequence, IMO, as is the one around thermodynamics and engines of cognition.

In fact I'd feel a bit uneasy cutting out any well referenced post. IMO the whole thing should be in chronological order, or an order strongly influenced by chronology, with just the worst stuff cut out.

comment by Raiden · 2013-09-20T14:12:58.919Z · score: 2 (2 votes) · LW · GW

I really would like a chronological order.

comment by Rob Bensinger (RobbBB) · 2013-09-20T08:09:12.722Z · score: 5 (5 votes) · LW · GW

I can think of two possible primary goals for this project:

  1. A prettier format for LWers to review the Sequences in, allowing easy full-text-search. A refresher and trophy.

  2. A tidier, better-organized, more approachable update of the Sequences to introduce entirely new people to the ideas therein.

If the latter, brevity is a virtue. Every 50 pages longer the eBook gets probably takes a significant chunk out of how many people read any of it at all. So there's a lot of reason to excise everything unnecessary to an Appendix. And to shunt the Appendix off into a separate eBook of its own. ("The Out-Of-Sequences.")

An alternative (or complementary) solution is to divide the Sequences into several separate volumes, each no more than 250 pages long. Then you can be given the option either of downloading a single eBook that collects these volumes, or of downloading the separate eBooks and choosing a favorite one to send to a friend. Makes for a drastically less intimidating Christmas present.

comment by lukeprog · 2013-09-20T19:16:31.021Z · score: 2 (2 votes) · LW · GW

We may later decide to release particular sequences as ebooks, especially the Quantum Physics sequence.

comment by alexvermeer · 2013-09-20T18:22:35.874Z · score: 1 (1 votes) · LW · GW

A prettier format for LWers to review the Sequences in, allowing easy full-text-search.

That's the one.

A tidier, better-organized, more approachable update of the Sequences to introduce entirely new people to the ideas therein.

This is outside the scope of this project.

comment by Ben Pace (Benito) · 2013-09-20T15:54:22.452Z · score: 0 (0 votes) · LW · GW

They'd be more easily bought as books then too.

comment by Eneasz · 2013-09-20T22:35:13.165Z · score: 4 (4 votes) · LW · GW

Expecting Short Inferential Distances may be a good idea, since it encapsulates the reason there are so many words/posts in the sequences.

I thought Religion's Claim to be Non-Disprovable is an amazing post, one of the most memorable, and it'd be a shame not to include it.

Likewise, The Hidden Complexity of Wishes was an incredibly intuitive way to explain that an AI doesn't have to hate you for it to destroy the world, it simply has to not have your values.

comment by beoShaffer · 2013-09-19T18:45:16.006Z · score: 3 (3 votes) · LW · GW

Here are the most-frequently-referenced links within the sequences to posts outside of the sequences (with a count of three or more). This may help you notice posts that you think should be included in the sequences eBook.

This seems to imply that the listed posts aren't part of the sequences, but several of the linked posts say they are part of a sequence, for example Decoherence is Falsifiable and Testable says it it part of the QM sequence.

comment by alexvermeer · 2013-09-20T18:24:58.618Z · score: 0 (0 votes) · LW · GW

Opps, you're right, there are a few that are already in the sequences that accidentally made it onto this list (I count three). All other links are to posts that are not in the sequences, but are linked to posts within the sequences. The list is auto-generated.

comment by somervta · 2013-09-19T22:09:55.336Z · score: 0 (0 votes) · LW · GW

I think it was just an auto-generated list.

comment by Furcas · 2013-09-19T16:04:12.837Z · score: 3 (3 votes) · LW · GW

No Highly Advanced Epistemology 101 for Beginners?

comment by kgalias · 2013-09-19T16:16:01.864Z · score: 5 (5 votes) · LW · GW

Luke has said it will be in a different ebook.

comment by shminux · 2013-09-19T15:23:30.585Z · score: 3 (21 votes) · LW · GW

I would rather the QM sequence was shortened to the low-controversy subset Eliezer described in An Intuitive Explanation of Quantum Mechanics and checked for technical accuracy. The pure MWI advocacy part belongs in some appendix, and the outright nonsense like "a Bayesian can become as smart as Einstein" should be chucked and never mentioned again.

comment by wedrifid · 2013-09-20T02:19:49.047Z · score: 3 (7 votes) · LW · GW

and the outright nonsense like "a Bayesian can become as smart as Einstein" should be chucked and never mentioned again.

Citation needed. I believe shminux made this up.

comment by TheOtherDave · 2013-09-20T02:45:00.384Z · score: 2 (2 votes) · LW · GW

He's not making it up out of whole cloth, though he's being significantly uncharitable.

More precisely, I think this is a reference to Einstein's Speed and related Sequence posts, where EY argues that Einstein's unusual success at understanding physical law was significantly due to updating on all available Bayesian evidence rather than just the subset of such evidence that non-Bayesian scientists use.

That said, it's of course a huge jump from "Einstein would not have been as successful had he not been a Bayesian" to "any Bayesian can be as successful as Einstein," and I don't recall EY (or anyone else) ever making the latter claim.

comment by wedrifid · 2013-09-20T02:55:58.781Z · score: 5 (7 votes) · LW · GW

More precisely, I think this is a reference to Einstein's Speed and related Sequence posts, where EY argues that Einstein's unusual success at understanding physical law was significantly due to updating on all available Bayesian evidence rather than just the subset of such evidence that non-Bayesian scientists use.

It seems to me that accusing someone of saying "outright nonsense" like this when they in fact did not say something like that and said only something vaguely related is an act that I would like to see discouraged when detected. Straw men are not welcome on lesswrong!

More precisely, the author 'made this up' because he believes it is acceptable to distort reality to that degree when arguing in this environment. Reception of the claim at the time I replied to it indicated that this belief is correct. I would prefer it if this were not so.

comment by TheOtherDave · 2013-09-20T13:13:05.236Z · score: 1 (1 votes) · LW · GW

I share your preference for discouraging straw men and uncharitable readings.

comment by novalis · 2013-09-20T17:54:04.592Z · score: 1 (3 votes) · LW · GW

+1

comment by Rob Bensinger (RobbBB) · 2013-09-19T21:28:11.356Z · score: 1 (1 votes) · LW · GW

I haven't found anything Eliezer's written about Einstein to not be useful. Could you explain why you don't like it (and/or specify what it is you dislike), or link me to an explanation?

If we want to shorten the QM stuff and explain MWI without belaboring its truth, I don't think it would be out of the question to commission a specialist like Amit Hagar or David Albert to write a short explanation of what the QM-interpretation fuss is all about, insert that right before the more important QM implications philosophy-of-science stuff (Think like reality, When science can't help, etc.), and then put Eliezer's technical explanations in an appendix. That would do a lot to mitigate the criticism of the Sequences for uncredentialed nonstandard physics espousal, and it would lose fewer readers whose math or physics backgrounds are weak.

comment by shminux · 2013-09-19T22:12:35.873Z · score: 4 (6 votes) · LW · GW

I haven't found anything Eliezer's written about Einstein to not be useful. Could you explain why you don't like it (and/or specify what it is you dislike), or link me to an explanation?

What has been proven wrong is the idea that explicit Bayesian thinking gives a physicist a significant rather than a marginal advantage. I don't know of any physicist who learned Bayesian thinking and suddenly became much more productive/successful/famous. You are likely to do better than without it, but you will never be as good as a noticeably smarter not-explicitly-Bayesian physicist, let alone Einstein.

Eliezer's waxing poetic about Barbour, who is a fringe scientist with intriguing ideas but without many notable achievements, is high on pathos, but not very convincing.

comment by Rob Bensinger (RobbBB) · 2013-09-23T10:03:15.109Z · score: 2 (2 votes) · LW · GW

Luke, Alex: After reading through most of Eliezer's 2006-2009 posts over the last 3 days, I suggest including in the eBook all the out-of-sequence articles you listed, except:

  • The Cartoon Guide to Löb's Theorem
  • Conjunction Fallacy (summarized adequately in lots of other articles, especially Burdensome details)
  • Cynical About Cynicism
  • Fake Fake Utility Functions
  • For The People Who Are Still Alive
  • Locate the hypothesis
  • The Martial Art of Rationality
  • The Modesty Argument
  • No One Knows What Science Doesn't Know
  • Optimization
  • Passing the Recursive Buck
  • Possibility and Could-ness
  • Priors as Mathematical Objects
  • Rational evidence
  • Rationality is Systematized Winning (don't need a whole article for this; to the extent it isn't just a recap of the Newcomb's Problem article, the point can be made in one sentence in the Introduction, or in a slightly edited What do I mean by rationality? and/or Incremental progress and the valley)
  • Surface Analogies and Deep Causes
  • The Two-Party Swindle (interesting but not essential, and copy-pastes content from other pages)
  • The Ultimate Source
  • The Unfinished Mystery of the Shangri-La Diet (this and Akrasia and Shangri-La are both too specific to be good core sequence candidates; their message is better covered in Beware of other-optimizing)
  • We Don't Really Want Your Participation
  • Why is the Future So Absurd? (use Stranger than history instead; showing is more powerful than telling)
  • You Only Live Twice (already in the FOOM eBook)
  • Zen and the Art of Rationality

From what I can tell, all the links to those pages within the Sequences are minor passing mentions, and their contents aren't particularly valuable for understanding the core content of the Sequences. Understanding Free Will, for instance, isn't needed for the rest of the Sequences, and EY actively discourages just copy-pasting his answer here.

I note that about 25 of the posts from the Quantum Mechanics / Timeless Physics sequence also aren't necessary for understanding the rest of the Sequences, differ in tone in interesting ways (they remind me of 2010-2013 Eliezer), and make the QM section look a heck of a lot more intimidating than it needs to. They also need new images to replace the quick pencil drawings, so I suggest leaving out the more technical stuff and then consolidating all the QM posts in a future eBook, once the images are polished up.

I can also suggest places to integrate the new additions that will make it possible to read the entire Sequences eBook from front to back without doing any hopping around. It doesn't take very much reshuffling to make the dependencies work sequentially in the Sequences.

comment by Paul Crowley (ciphergoth) · 2013-09-20T06:37:24.741Z · score: 2 (2 votes) · LW · GW

Do you have a list of blog posts that won't be in the book as a result?

On the one hand, I want to say "it's an ebook, stick them all in an appendix". On the other hand, the more slowly the progress bar at the bottom of the page moves the more readers are likely to give up. What can be left out should be.

comment by alexvermeer · 2013-09-20T18:29:06.598Z · score: 2 (2 votes) · LW · GW

Do you have a list of blog posts that won't be in the book as a result?

That's what the list at the bottom of the OP is for: posts that are currently not slotted to be in the book, but are linked to by sequence posts.

What can be left out should be.

I lean this way as well.

comment by Paul Crowley (ciphergoth) · 2013-09-20T22:05:46.085Z · score: 2 (2 votes) · LW · GW

D'oh! Thanks.

comment by protest_boy · 2013-11-16T01:44:43.650Z · score: 1 (1 votes) · LW · GW

Are there any updates on when this will be released?

comment by [deleted] · 2013-09-20T17:55:36.037Z · score: 1 (1 votes) · LW · GW

I have a comment that takes no issue with the central argument of the 'Mind Projection Fallacy' sequence post, it's just about an embellishment that is, I think, false and a little cringe-inducing:

From "Mind Projection Fallacy":

But the Mind Projection Fallacy generalizes as an error...to Kant's declaration that space by its very nature is flat, and Hume's definition of a priori ideas as those "discoverable by the mere operation of thought, without dependence on what is anywhere existent in the universe"...

This should be removed or altered. Kant's declaration that space is by its very nature flat is a declaration about the conditions of experience, not the world as such. He's explicit that space and its flatness are not features of the territory (so to speak), but necessary features of the map. He may be wrong about this, but it's not a case of the mind projection fallacy.

As to the Hume comment, this is a bit of a non-sequitor as phrased: a definition of a philosophical term can't be a case of the mind projection fallacy.

comment by Ben Pace (Benito) · 2013-09-20T06:16:01.352Z · score: 1 (1 votes) · LW · GW

I've been working on an introductory set of posts for a (religious) friend and Hpmor fan. I've been trying to find the posts that explain the various parts of the LW philosophy best, and I've come up with this.

I'd just like to comment a little on which posts I selected, and then recommend for the ebook.

I think zen-like little pieces are nice to start with, like the Twelve Virtues and The Simple Truth, but I think that the Simple Truth requires as either a pre-req or an immediate following by 'The Useful Idea of Truth', to make people see the meaning of the former, otherwise it can be lost entirely. I think f Useful was edited slightly it could fit into the ebook well enough on its own. Also the Map and Territory and Appreciating Cognitive Agorothms from HAE101FB I think are great to introduce core ideas.

I think the inferential distances should be in, so the reader realises things can take time to explain. Also, posts on Curiosity are very useful near the beginning, to give the reader a care for the truth after they later perhaps read something they disagree with.

I'd like to see all the ones that have been referenced over six times added, and also Conjunction Fallacy, Lawful Creativity and Illusion of Transparency.

Cheers. Hope there's an opportunity to get it as two or three physical volumes.

comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2013-09-19T21:02:15.680Z · score: 1 (1 votes) · LW · GW

I'd drop A Case Study of Motivated Continuation from Politics rather than trying to add two such unrelated prereqs.

comment by roystgnr · 2013-09-19T16:47:32.553Z · score: 1 (1 votes) · LW · GW

Has anyone considered A/B testing different versions of proposed changes or orderings?

comment by Paul Crowley (ciphergoth) · 2013-09-19T19:09:16.080Z · score: 1 (1 votes) · LW · GW

I don't think it's practical to A/B test a million word book.

comment by gwern · 2013-09-19T20:10:38.884Z · score: 2 (2 votes) · LW · GW

It's perfectly possible to A/B test it, it just depends on what you're varying and how it's being delivered. My website is well over a million words, and I A/B test changes to its appearance all the time.

comment by Douglas_Knight · 2013-09-20T21:05:34.840Z · score: 0 (0 votes) · LW · GW

You can A/B test whether doubling the length with appendices affects download completion rates.

comment by gwern · 2013-09-19T17:16:22.807Z · score: 1 (1 votes) · LW · GW

That's not necessarily a bad idea, but how do you A/B test different versions of a PDF? What is the response that is being measured?

comment by roystgnr · 2013-09-19T21:57:30.566Z · score: 0 (0 votes) · LW · GW

I'd test one chapter (or one sub-chapter) at a time. Reader-reported assessments of quality and ease of understanding would be the obvious thing to measure for any instructional book... but also, isn't CFAR trying to come up with various "rationality tests"? Accuracy of responses to test questions would be a great metric to try and optimize, and most instructional material benefits from having end-of-chapter questions anyway if only to help readers verify that they've gained actual understanding and not just an illusion thereof.

comment by LM7805 · 2013-09-19T17:31:44.281Z · score: 0 (0 votes) · LW · GW

You can embed arbitrary javascript in PDFs, so what about including "phone-home" text-boxes for marginalia in rather the same way that online editions of Real World Haskell and other programming books have comment threads for each paragraph? I'd think other metrics like time-on-page could be measured as well. This would need to be disclosed, of course, and not every reader can be expected to comment, but the "how" seems tractable at first impression.

I don't have any useful insights on what response to measure.

comment by gwern · 2013-09-19T18:03:49.591Z · score: 0 (0 votes) · LW · GW

You can embed arbitrary javascript in PDFs

This doesn't seem usefully true. Some googling for search queries like 'a/b testing PDFs' or 'PDF phone home' show no one discussing the topic of A/B testing different versions of PDFs, and Wikipedia indicates that only Adobe supports JS and even it produces a popup when you try to phonehome. So any A/B test is going to work on only a fraction of users (how many LWers still use Adobe Acrobat to read PDFs?), and it will alarm the ones it does work on ('is this PDF spyware‽').

comment by LM7805 · 2013-09-19T18:48:10.602Z · score: 0 (0 votes) · LW · GW

Foxit Reader supports javascript, and libpoppler (which powers evince and okular, among others) does as well.

Without something to measure, though, that's really just a technical curiosity.

comment by Lumifer · 2013-09-19T17:54:43.182Z · score: 0 (0 votes) · LW · GW

You can embed arbitrary javascript in PDFs, so what about including "phone-home" text-boxes for marginalia

I don't know how the security model works in various PDF readers, but wouldn't the javascript code be sandboxed, hopefully? Sane security practices shouldn't allow arbitrary code in PDFs to talk to random 'net addresses...

comment by LM7805 · 2013-09-19T18:15:56.056Z · score: 0 (0 votes) · LW · GW

If the PDF is signed by a certificate the user has manually installed, it can embed what Adobe calls "high privilege" javascript, which includes the ability to launch any URL. That's an extra step, which would discourage some users, but on the plus side it addresses the "who's given informed consent?" problem.

Momentarily donning a slightly darker hat: it is also possible for a PDF to launch an arbitrary executable (see pp. 30-34 of Julia Wolf's OMG WTF PDF, video). AIUI this requires no additional privileges.

comment by Lumifer · 2013-09-19T18:34:24.573Z · score: 0 (0 votes) · LW · GW

...a certificate the user has manually installed ...an extra step, which would discourage some users

My estimate for the value of that "some" is 95%+

Not to mention that most of the people who can be easily persuaded to manually install a cert on their PC probably already have a dozen toolbars in their browser... :-D

comment by Viliam_Bur · 2013-09-19T08:19:30.427Z · score: 1 (1 votes) · LW · GW

One metric that could be calculated automatically: how many articles in the book contain hyperlinks to the articles that appear later in the book, or don't appear in the book?

I guess we should try to make that number as small as possible, for convenient linear reading, but of course it has to be balanced against other concerns (such as keeping articles with similar topics together, so we can make chapters with a unified topic).

Another idea: identify the main topics of the book (they roughly correspond to the chapters you have now: "map and territory", "words", "politics", "reductionism", "quantum physics", "metaethics", "community", etc.), and mark each article with the corresponding tags.

The goal of this is to reduce the cognitive load while sorting the chapters. Instead of thinking "what was the Fake Morality article about, again?" (and trying to keep all those articles in your head), reduce all chapters to a short information: article name (or identifiers), article tags, names of referenced articles. And then only use this information to sort the articles. -- Perhaps by describing every articles in one line of a text file, easily moving them around in the text editor, and calculating the "non-backwards hyperlinks" metric automatically.

comment by Paul Crowley (ciphergoth) · 2013-09-19T11:42:16.157Z · score: 1 (1 votes) · LW · GW

I also wanted a list of forward references to assess this ordering against.

comment by alexvermeer · 2013-09-20T18:33:38.266Z · score: 0 (0 votes) · LW · GW

One metric that could be calculated automatically: how many articles in the book contain hyperlinks to the articles that appear later in the book,

I like this idea!

or don't appear in the book?

That's the list at the end of the OP.

comment by Paul Crowley (ciphergoth) · 2013-09-19T08:04:59.728Z · score: 1 (3 votes) · LW · GW

Should it open with "The Simple Truth", since it's a precursor to Map and Territory?

comment by wedrifid · 2013-09-20T03:13:52.871Z · score: 1 (1 votes) · LW · GW

Should it open with "The Simple Truth", since it's a precursor to Map and Territory?

It could be that The Simple Truth may work better after (some of the many) concepts it illustrates have already been conveyed. I like it because it demonstrates in a memorable way a sane response to various philosophical confusions. It may be less helpful for those who are less inclined to appreciate satirical allegory and who need to be persuaded by more sombre explanation. Comprehend then memorize is common advice to those using spaced repetition and probably generalises to other means of facilitating long term retention.

comment by Tenoke · 2013-09-19T11:14:59.462Z · score: 0 (0 votes) · LW · GW

Exactly what I was thinking! Also, I often send people 'The Simple Truth' when I start recommending them to read the LessWrong Sequences.

comment by moridinamael · 2013-09-19T12:34:43.522Z · score: 18 (20 votes) · LW · GW

I've mentioned this elsewhere, but, The Simple Truth is stylistically totally different from the rest of the Sequences, extremely long, and rather meandering. I think it's enjoyable if you've already bought in to the memeplex but utterly confusing if you haven't. Specifically, I tried to get my wife to read the Seauences and The Simple Truth was the article that she never got past. I know, n=1 and all, but I stand by my other points.

comment by satt · 2013-09-21T14:11:06.492Z · score: 3 (3 votes) · LW · GW

Plus, more generally, if I were putting the book together I'd want to shuttle the reader from page 1 to Mysterious Answers to Mysterious Questions as soon as I could, since MAMQ is where the Sequences really pulls up the landing gear and takes off. Putting "The Simple Truth" at the start puts another 7,000-odd words between the reader and that take-off. (And I agree that it's tortuous & opaque, which just makes it an even bigger roadblock. I'd hide it an appendix too.)

comment by Rob Bensinger (RobbBB) · 2013-09-21T19:55:21.398Z · score: 1 (1 votes) · LW · GW

I agree with this. I'd start with some bits and pieces of the Map/Territory sequence, followed by a crash course on assorted cognitive biases (because it won't make sense that Map/Territory talks at length about 'human irrationality' and 'bias' unless the reader is swiftly provided with examples), followed immediately by a (slightly modified) Mysterious Answers. Then How To Actually Change Your Mind.

comment by Viliam_Bur · 2013-09-19T20:09:11.442Z · score: 1 (1 votes) · LW · GW

It's a story you either love or hate. I happen to love it, but that's probably because (a) I already agree with the ideas, and (b) I have met people expressing the silly ideas in real life, so I know what it is about. -- In other words, it is a bit like an applause light for the believers. Not a good introduction.

I also happen to love the "twelve virtues"... so I am curious whether people have mixed reactions to that one, too. Unlike the simple truth, the twelve virtues is short. (More precisely, the individual twelve points are short; but the total length is still acceptable.)

comment by Rob Bensinger (RobbBB) · 2013-09-19T21:10:38.242Z · score: 0 (0 votes) · LW · GW

I neither love nor hate it. It's pretty good. The concept is great; the execution is a bit too injokey and unstructured. Perhaps the main problem is that you need to be pretty savvy and experienced regarding pop philosophy to know what the point of the allegory is (without expending a fair amount of effort), but the sorts of people most likely to make the crude errors The Simple Truth is correcting aren't likely to be particularly savvy.

I do love The Twelve Virtues, though I think they could easily be compressed to something cleaner and easier to remember (e.g., with less built-in redundancy). For instance, Relinquishment and Humility seem to just be special cases of Lightness and Evenness (with a dash of cog-sci Empiricism/Scholarship, mayhap).

If I were redoing the list, I'd go with Ten Virtues, something more like: Curiosity, Daring, Attentiveness, Lightness, Evenness, Simplicity, Precision, Rigor, Community, The Void.

('Argument' breaks up into Daring and Community, 'Empiricism' breaks up into Daring and Attentiveness and Simplicity and pretty much all of the other virtues, 'Scholarship' breaks up into Rigor and Community, etc. I'd still want virtues like Scholarship explicitly talked about, but as more complicated practices that come out of the primary-color virtues.)

comment by Tenoke · 2013-09-19T18:23:07.553Z · score: 1 (1 votes) · LW · GW

Yeah, I usually make it explicitly clear that the style of The Simple Truth is nothing like the style of the sequences. I also point out that some people are not fond of it when recommending it.

comment by Paul Crowley (ciphergoth) · 2013-09-19T06:29:07.255Z · score: 1 (1 votes) · LW · GW

The closing parenthesis is missing on the link to "Map and Territory (sequence)". If you compose in Markdown, you have to escape this with a backslash.

comment by fortyeridania · 2013-09-21T00:14:52.249Z · score: 0 (0 votes) · LW · GW

Like other commenters here, I'd like to see the Twelve Virtues make the cut.

I'm really excited about this book!