The Comprehension Curve

post by AllAmericanBreakfast · 2021-02-22T22:51:49.820Z · LW · GW · 9 comments

Contents

  Speed Reading
  The Comprehension Curve
  The Natural Pace Hypothesis And The Difficulty With Comprehension Tests
  From "Speed Reading" To "Eye Technique" and "Mental Technique"
  Where's The Comprehension Bottleneck?
None
8 comments

Tl;Dr: You might comprehend and learn more quickly by slowing down your reading, rather than by speeding it up. This post is mainly about laying down some terminology that I think is conceptually clarifying in thinking about learning, comprehension and reading speed.

Speed Reading

According to Google Trends, interest in speed reading has been on a pretty steady decline since 2004. It briefly became more popular in March 2014, perhaps due to an Atlantic article published that month, on the speed reading app Spritz.

Nevertheless, when a person starts thinking about "learning how to learn," speed reading is one of the natural things to investigate. Am I reading fast enough? What's the maximum possible words per minute at which I could still comprehend the material? Couldn't I compensate for low comprehension just by reading much, much faster? So they Google "speed reading," and resurrect the conversation.

The fundamental idea of speed reading is that by reducing wasted motion in the physical eye movements involved in reading, people can dramatically increase the number of words per minute (WPM) that they can read, without sacrificing comprehension. Speed reading advocates point to evidence that speed readers show no statistically significant difference from normal readers in performance on comprehension tests. For such research to be meaningful, it needs to be executed with care. This care is generally lacking, as I describe below.

Speed reading is most obviously helpful under a narrow set of conditions:

  1. The document you have to read is your only source of the information.
  2. The information it contains is easy for you to comprehend.
  3. You still need to read the document for some reason.

Real-world examples might include a lawyer reading boilerplate, a teacher reading the essays of young students, or a businessperson reading memos. Another might be a student who wants to honestly be able to say they read the Scarlet Letter, but who doesn't care about the book very much and plans to study for the test using CliffNotes.

One of the nice things about reading speed is that it's easy to quantify. You can paste text into an app like Spreeder and set the speed as fast as you'd like to go.

But if you primarily want to comprehend quickly, rather than merely read quickly, then it's not fundamentally your reading speed, but your comprehension rate, that you want to maximize.

The Comprehension Curve

Unfortunately, comprehension is hard to quantify. Despite this challenge, we can think about its relationship with reading speed conceptually in terms of the Comprehension Curve.

The Comprehension Curve is analogous to the Laffer Curve. Briefly, the concept of the Laffer curve is that if the government taxes at 0% or 100%, it will receive no revenue, either because it will take none of people's money, or because nobody will be willing to work. Thus, the ideal rate of taxation to maximize revenue is somewhere in between. If the current tax rate is higher than the ideal tax rate, the government could in principle cut taxes while increasing tax revenue.

The point of the Comprehension Curve is that each person, for a certain text at a certain time, has an ideal reading speed that maximizes their comprehension rate. If a person was reading faster than their ideal reading speed, they could comprehend more quickly by reading more slowly. Of course, it's hard to know whether an individual is reading faster, slower, or right at their ideal rate, just as it's hard to determine the ideal tax rate to maximize tax revenue. However, we can make some plausible assumptions, given below, to help us think about this problem in more depth.

The vertical axis on the Comprehension Curve needs some unpacking. "Comprehension rate" has four important pieces of nuance:

  1. It refers to comprehension, not learning in general. Comprehension is hard to define, but think of it as that first step in importing words from the page (or from an oral speech, a picture, graph, or sensation) into your mind. It's that form of learning that you can achieve at the same time as you read/listen/pay attention. By contrast, learning can also refer to long-term retention, ability to apply a concept to solve a problem, physical technique gained through practice, and real-world experience. Comprehension is just the first step in learning from a text, presentation, or demonstration, or other similar experience, where the material is transduced from the page into the mind.
  2. It refers to the rate of comprehension, not an amount. A student at the start of their study session has comprehended only a small amount, but may be comprehending at a fast rate.
  3. Comprehension rate is personal and contextual, not with reference to some human average or theoretical limit. Comprehension rate is specific to a particular text, reader, environment, level of background knowledge, and level of reading skill.

The Comprehension Curve lets us see that "speed reading" starts with the assumption that most people, most of the time, are on the left side of the Comprehension Curve. If true, then they could increase their rate of comprehension by reading faster.

But the personal and contextual nature of reading suggests that in many cases, this is not a plausible assumption. If a person hears that the average reader has a 250 WPM reading speed, and that skilled speed readers can go 1,000 WPM, then tries to paste a dense biology textbook into a speed-reading app, they might run into problems.

Compounding this practical problem, some speed reading advocates suggest to novices that they should continue practicing speed reading, despite their (initial) lack of comprehension. For example, an article on MindTools.com states:

"You will probably find that you retain very little information at first, but, as you train your brain and you become more comfortable with the technique, your comprehension should improve."

It's OK to accept a temporary setback in order to train in a more powerful method. For example, a pianist who learned poor technique as a child might temporarily play worse as they work to master a better technique. But this is only a useful tradeoff if the technique  is indeed more powerful once mastered.

If people who are trying to comprehend at a faster rate turn out to be in the middle or on the right side of the Comprehension Curve, then even mastery of speed reading will harm their comprehension. Even if they're on the left-hand side, if they overshoot their ideal reading speed, their comprehension rate will suffer.

Training people to accept and ignore a harm, under the assumption that it's a temporary harm, is a dangerous, if sometimes necessary, message.

The Natural Pace Hypothesis And The Difficulty With Comprehension Tests

Does it seem plausible that we could dramatically increase our reading speed from our natural pace and attain a higher comprehension rate? After all, don't people naturally find a pace that optimal for their own brain and needs? Let's call this the "natural pace hypothesis."

If we accept the natural pace hypothesis, then it's an unlikely hypothesis that altering a person's reading speed could improve their comprehension, or whatever other outcome they're optimizing for. We therefore need a lot of strong evidence to make us accept it.

Alternatively, it might be more plausible to assume that people habitually read at a fixed rate, regardless of the subject matter or their level of familiarity. Perhaps they always read at a pace of 250 words per minute, no matter whether they're reading a novel, an Ikea manual, or a math proof, out of sheer habit. If so, they might benefit by calibrating their reading speed to the specific text, but it's not clear that speeding up their reading is generally the right move on the Comprehension Curve.

A third possibility, though, is that people read at a pace that they learned as children. As experienced readers, they can dramatically increase their reading speed, yet remain generally stuck at that deeply ingrained, childhood pace. For that reason, we can assume that most people, most of the time, have an ideal reading speed that's faster than their habitual one. If so, that would make speed reading seem like a more plausibly useful general intervention.

One way that speed reading advocates claim to test the hypothesis that speed reading does not interfere with comprehension is by administering comprehension tests to speed readers and normal readers, and comparing the results. If the speed readers do at least as well as the normal readers, then speed reading hasn't harmed their comprehension. Even if speed reader comprehension is worse than that of normal readers, it may be made up for by the sheer amount they're able to take in.

Rayner, et al provide an excellent review both of the technique of speed reading and the flaws in many of these comprehension tests.

In addition to looking at real research, it's useful to think of what an ideal study, and interpretation of the evidence it provides, would look like.

"Speed reading" can be defined across three orders of magnitude. If the average pace is 250 WPM, is "speed reading" 500 WPM? 2,500 WPM? 25,000 WPM? When we have study results, we shouldn't extrapolate too much from them to other types of texts, learners, or reading speeds. For that reason, claims that "studies show speed reading [does/does not] work" should be treated with skepticism.

But more importantly, what do we make of it when a study shows that we find no significant difference between speed readers and normal readers, for a given technique, text, and reading speed?

We could interpret this in two ways:

  1. The studied approach to speed reading allows faster reading with no loss of comprehension. Speed reading works!
  2. The comprehension test was too hard, or it wasn't hard enough.

As an example of a too-easy test, imagine that the text is a children's book, and the questions are asking about the basics of character and plot. If this test is given to proficient adult readers, they might be able to pick everything up based on a quick scan.

An example of a too-hard test would be having a group of chemistry students read a passage on the derivation of the Michaelis-Menten equation. If the comprehension test questions ask them to extend and apply that equation to difficult word problems, it may be that both of them would have such a hard time that neither group was able to answer the questions correctly.

To do a good comprehension test, you'd need to find a good operationalization for "normal" and "speed" reading. Perhaps you could establish a "normal range" of reading speeds by asking a group of subjects to read the text once, at their own speed, and to then determine the interquartile range - the range between the top 75% and bottom 25% of reading speeds.

Then you'd establish a corpus of texts that all reliably produce the same "normal range," and test to see whether or not individual people tend to read all texts from this corpus at around the same speed.

Once you had a corpus of such texts, you might expose speed readers to them, presenting some texts at "normal range" speeds, and others at the speed reading rate you're testing. If the speed reading rate produced significantly different comprehension than the normal rate, this would be evidence of whether or not speed reading would be beneficial for most people, at that speed and for similar texts.

From "Speed Reading" To "Eye Technique" and "Mental Technique"

If you read the Rayner review, you'll notice that speed reading technique partially focuses on refining one's physical eye movements.

What if we referred to the collection of techniques meant to optimize these motions not as "speed reading" but as "eye technique?"

After all, if these techniques represent improved eye technique, this improvement need not be in terms of reading speed. It could be, for example, that a reader with excellent eye technique reads at exactly the same speed as the average population, yet has improved comprehension nonetheless. Perhaps their more carefully coordinated eye motions enhance the visual clarity of the text itself, or otherwise help the reader identify and focus on the most important words.

So it may be that the eye techniques that are normally referred to as "speed reading" are useful to learn, even if the "natural pace hypothesis" is correct and most people read at their ideal reading speed.

Another way of stating this is that advocates and detractors of "speed reading" tend to conflate two hypotheses:

  1. Practice with a particular set of eye techniques can be useful to readers.
  2. Rate of comprehension is bottlenecked by words read per minute.

Even if reading speed is not the bottleneck to faster comprehension rates for most people, better eye technique might still be helpful for other reasons.

Beyond "eye technique," Abram Demski points out in the comments that speed readers also focus on bypassing the inner monologue when reading. They encourage readers to avoiding hearing the words on the page in their head, and instead to practice translating words on the page directly into concepts in the mind. We can call this a form of "mental technique."

The theory behind this particular mental technique is that it avoids the delay of translating text into inner speech, and inner speech into concepts. This is plausible, but there could be other forms of mental technique, with other compelling justifications. No matter which mental technique is chosen, it need not be put to use to increase reading speed. It could probably also be used at normal reading speed in order perhaps to improve comprehension rate.

In the case of this "audio cortex bypass" mental technique, it could be that the bypass helps people engage other, more useful faculties of mind for supporting comprehension. Perhaps if people don't hear the word "normal distribution" when they read it in a statistics textbook, they'll instead visualize a normal distribution, which would give them genuinely useful information to work with that hearing the words simply does not.

However, it might also turn out that "audio cortex bypass" enhances reading speed, but harms comprehension, either through moving the reader too far to the right on the Comprehension Curve, or by interfering with a useful faculty of mind for comprehension. I have anecdotally found that engaging my audio cortex in a particular way dramatically enhances my comprehension, which I will describe in more detail in a future post.

This is mere anecdote. But the real problem here is that speed reading advocates assume that "audio cortex bypass" is the right way to go, and scientists dutifully test it. Neither of them seem to step back and reconsider how we might plausibly improve comprehension rates through improved reading techniques, be they eye techniques, mental techniques, or some other technique entirely.

Where's The Comprehension Bottleneck?

Thinking in terms of a "comprehension bottleneck" can point to what we really want to know. Is the average person's natural approach to reading for comprehension - their eye technique, speed, cognitive reflection - their ideal approach as well? In other words, can the rate of comprehension be improved through better technique? And if so, what is the limiting factor, or bottleneck? If we aren't achieving our personal maximum comprehension rate, what aspect of poor technique is holding us back?

A multi-step chemical reaction can only proceed at the rate of its slowest component reaction. It also has a theoretical maximum, determined by the rate of diffusion. Likewise, comprehension can only proceed at the rate of the slowest component technique: health-promotion and focusing technique, material-selection technique, environment-setup technique, eye technique, mental technique, attention-maintaining technique. It also has a theoretical maximum for an individual, determined by the physiology of their brain.

Again, what I refer to above as mental technique is the set of voluntary thought-behaviors that that are under conscious control: our ability to visualize and activate our sensory imagination, our inner monologue, our memory.

An individual interested in experimenting with finding and improving the technique that is their comprehension bottleneck might best approach this by having a rich set of techniques with which to experiment.

Some techniques are addressed by specialists, such as psychiatrists who can prescribe medication for ADHD. Other professional specialties that seem like they would be useful are deeply unconventional. How often does a professional researcher consult with a reading teacher to see if their technique can be improved? By contrast, even highly accomplished musicians continue to work with music teachers throughout their career.

What book or professional would a "professional reader" - a lawyer, journalist, scientist, anybody in a knowledge profession - turn to in order to assess and improve the broad range of techniques that factor into their comprehension bottleneck? The problem is fractured into bits, each dealt with either by a different professional or with no expert attention at all.

Finally, it seems valuable to contextualize comprehension as one possible learning bottleneck. A person might, for example, achieve a very fast rate of comprehension when they read a textbook, yet fail to remember almost any of it a year later. If their goal is long-term retention, then improving their rate of comprehension is not the bottleneck for their learning.

Yet there could be a relationship between these facets of learning. For example, a faster rate of comprehension would mean that a student could devote more time to reviewing previously-learned concepts, which might improve long-term retention. Likewise, better comprehension might improve performance on applied work, such as labs or problem sets, or improve motivation. Being able to think clearly about the difference and relationship between comprehension and learning, and to view learning as a decomposable set of processes, akin to chemical reactions, each with a technical limiting factor and theoretical maximum rate, seems like the correct approach.

In this state of insufficiency, the goal of this article is to bring some conceptual clarity to the subject of reading speed and comprehension. The most important concepts are the "comprehension rate," the Comprehension Curve, the comprehension and learning bottlenecks, and the revisionist concept of "eye technique" and "mental technique" as part of a set of linked reading comprehension techniques.

Thanks especially to Abram Demski for thoughtful comments.

9 comments

Comments sorted by top scores.

comment by abramdemski · 2021-02-23T03:23:07.992Z · LW(p) · GW(p)

One friend of mine decided to, as an experiment, read with a 1-minute interval timer (dings every minute), and force themselves to turn the page at every ding. Depending on the book, this is very fast -- quick googling suggests 500 words per minute. They didn't expect to keep reading at such a pace; it was an experiment. But after the experiment, they decided that they much preferred reading this way, and kept it up. The lowered comprehension was greatly compensated for by the ability to get content from many many books quickly.

A different friend of mine reads very slowly and thoroughly. This friend learned that many people have the experience of reading a paragraph and then realizing that they haven't understood it, and re-reading. They were quite surprised at this, and compared it to learning that lots of people just aren't conscious -- how can you read something without comprehending it?? How can you be in the process of reading a paragraph without comprehending it, without realizing that you're not comprehending it, for so long that you get to the end of a paragraph before you notice??

If the first friend wrote a book on reading, it might look similar to other books on speed reading (although I'd expect it to be many times better than the usual).

If the second friend wrote a book on reading, it would be a book on slow reading. It might involve stopping after every paragraph and seeing if you agree with the author's conclusions. It might involve looking up a lot of reference material in other books, before you've even gotten very far in the book you're trying to read. It might involve taking copious notes. By the time you finish a book, you might be able to re-write it yourself, but better.

I think I could very roughly put friends on a spectrum from speed reading to slow reading. The speed readers would generally have the characteristic of a broad knowledge base. They would bring in impressive amounts of knowledge to any conversation. However, if you want detail on subjects, you'll often stump them. Their knowledge is shallow. They'll often have to send you to read some paper or other to answer the questions they can't answer themselves. And the papers often won't back up their summaries because they read the paper only shallowly, and more or less trusted the abstract -- which, often, aren't an accurate summary of what's in the paper.

The slow readers would be the careful thinkers. They have a deep knowledge base, with fractal detail. If they read a paper, they can tell you what's wrong with the abstract, and what they think the real conclusions are. They won't settle for outside-view knowledge of anything; they require a gears-level understanding before they consider themselves to have learned something.

There's almost no ceiling to what you can understand from a given text. It's just a question of how far you care to go.

So, in addition to raw comprehension rate, there's also what kind of knowledge you want to foster. This probably varies from text to text. In some cases it's best to absorb a broad base of material rapidly. In other cases it's more useful to get a really detailed understanding, questioning all of the author's conclusions, working through everything yourself.

comment by Matt Goldenberg (mr-hire) · 2021-02-23T17:04:07.104Z · LW(p) · GW(p)

So, in addition to raw comprehension rate, there's also what kind of knowledge you want to foster. This probably varies from text to text. In some cases it's best to absorb a broad base of material rapidly. In other cases it's more useful to get a really detailed understanding, questioning all of the author's conclusions, working through everything yourself.

 

I tried to take a stab at when to do which model in this post. [LW · GW]

comment by abramdemski · 2021-02-23T02:51:26.403Z · LW(p) · GW(p)

I am a slow reader compared to most people I know, and have occasionally looked into speed reading to try to remedy this.

I was surprised about your description of speed reading as "eye technique", because although I've seen this sort of thing, I've also seen mental processing advice. If someone asked me to quickly summarize what I know about speed reading, I would have said "Speed reading guides usually say that you have to stop turning the words into imaginary auditory signals in a mental voice; this forces you to slow down, as you first convert the letters to mental sounds, and then convert the sounds to meaning. Instead, you should interpret the letters directly into words and meaning, without involving the auditory cortex."

(I've never succeeded in doing this, but it's led me to practice EG repeating nonsense words in my head while I read, to try and force myself not to use auditory processing.)

Anyway.

I think it's important to point out the difference between optimal comprehension per character vs optimal comprehension per second. A natural hypothesis to me (alternate to your natural pace hypothesis) is that people intuitively optimize comprehension per character, because it's more salient (they're not watching the clock as they read, after all).

On this hypothesis, most people will be a bit too low on the comprehension curve, because they aren't automatically optimizing for that.

Either way, I find it a priori very plausible that people aren't at max reading skill automatically, and can benefit from focused practice. Although I haven't noticed marked improvements, myself. I definitely endorse your re-framing in terms of optimizing comprehension rate rather than raw reading speed.

One of the most subjectively helpful frames (which I got from someone at a LessWrong meetup) was that speed reading is about optimizing information absorbtion. Notice when you already get what the author is trying to say. If you have a guess about what argument the next paragraph is going to make, spot-check your guess by reading a few phrases; if they're consistent with your expectations, you can probably skip ahead. Lots of people sort of "cargo cult" reading books, by reading the whole thing in order. It's often more efficient to skip around and find the information you actually want.

I think part of what's going on is that speed-reading advocates know this, and are really advocating techniques which encourage people to skip over things. For example, you mention:

  • Regressions: how often the eye travels to an earlier point in the text.

Avoiding regressions means you avoid re-reading stuff you just passed over. I re-read all the time, when I didn't really get what was said. Someone who advocates avoiding regressions is implicitly telling me: "Oh, you know what they said. You're probably re-reading just to check. Even if you misunderstood something, you'll probably be able to figure out what's really going on based on the next paragraph."

Which is not necessarily true. Or, even if true, is not necessarily optimal information absorbtion (because the risk of misunderstanding is pretty high with this sort of skimming).

In other words, I reject the idea that "eye technique" is really what it purports to be -- better general eye technique for reading. I think it's actually implicit advice to skim more and fill in details by inference, cashed out as details about eye movements which sorta force you to do this.

comment by AllAmericanBreakfast · 2021-02-23T05:04:27.211Z · LW(p) · GW(p)

I take it as a big compliment that you wrote such a long and thoughtful reply to my post! Thank you!

The distinction you draw between broad vs. deep readers is the reason I didn't operationalize comprehension. "Comprehension" is just the type of understanding you want to extract from the text on your particular read-through, defined as you please. Maybe let's think of them in terms of abstract units called Comprehendalons, analogous to a Utilon.

You could define a Deep-Comprehendalon as knowing "wrong with the abstract, and what they think the real conclusions are," in which case a very slow reading speed is ideal. An ideal reading speed for Deep-Comprehendalons might be 10 wpm, or even slower, and it might take you several hours to acquire just one.

A Shallow-Comprehendalon might be picking up an single atomic fact. An ideal reading speed for Shallow-Comprehendalons might be 500 wpm, or even faster, and you might be able to pick up a huge number of them in a short period of time.

One thing I infer from this framework is that Shallow-Comprehendalons don't add up to Deep-Comprehendalons. They are not fungible, not the same type of good. Optimizing for one may mean sacrificing the other.

However, that seems debatable. Even if it's true, the Comprehension Curve would still hold. You'd just have a different ideal reading speed and maximum comprehension rate for each type.

Interestingly, for me, fully engaging my auditory cortex has, I believe, really helped me to move closer to my maximum comprehension rate. I'll describe this in a future post. One of my motivations for writing this is that I think that speed reading advocates are doing something fundamentally good -- experimenting -- but doing it in a screwy way, where they invent a whole theory of why their method is the best, without exploring contrasting hypotheses. And then when sober scientists get to studying it, they approach the field by testing the claims of speed readers, rather than by reflecting a priori on what approach to learning ought to enhance comprehension. The latter is what I've tried to advance toward here.

I appreciate you bringing up the point that much of speed reading advice revolves not around eye technique, but around mental technique - the bypassing of the auditory cortex. That's true, and I just entirely left that out for no good reason. I'm going to edit the OP to include a reference to it, along with a credit to you for reminding me of it.

One of my future posts will discuss what I've noticed in regards to skimming. I'll sum it up for you, as practice and since you brought it up.

Let's consider the following sentence from a biochemistry textbook:

"Oxidative reactions of the citric acid cycle generate reduced electron carriers that are then reoxidized to drive the synthesis of ATP."

For someone like myself who's familiar with biochemistry, many of the individual words and phrases refer to concepts that I already understand well. But there are particular keywords within the sentence that "focalize" a new concept, build out of the others. Let me break down my experience reading it:

  1. "Oxidative reactions" - the word oxidative is key, and "contains" the concept of a reaction within it.
  2. "of the citric acid cycle" - citric is key, and automatically refers to "citric acid cycle"
  3. "generate reduced electron carriers" - reduced is key, and "generate" is implied by the connection between "citric [acid cycle]" and "reduced"
  4. "that are then reoxidized" - reoxidized is key
  5. "to drive the synthesis of ATP." - ATP is key, because I already know that the end result is to synthesize, rather than consume ATP.

So as I read this sentence, the words "oxidative," "citric," "reduced," and "ATP" get lodged in my working memory, repeated in my auditory cortex in a sort of earworm-like jingle. While they repeat, my eyes scan the rest of the words to observe how they link up. So I read with a two-layered awareness: the working memory jingle-words that isolate and relate the key concepts, and the non-auditory connecting words that give them meaning and relate them together. This is just the approach to reading that I'm playing with right now, and I make no claim that it's useful or ideal for myself or anybody else.

But it's an interesting riff on skimming and speed reading. Instead of divorcing myself from my auditory cortex, I use it for keywords only, while relying on non-auditory reading for connecting words, which I can largely skim through.

The only way to develop ideas like this is to experiment openly, with a goal not of reading quickly, but of being experimental with your approach and trying to intuitively feel your way toward a method that is satisfying and feels like you've comprehended the material well. I find that this approach makes it far easier to pay nuanced attention to the material, read for long stretches of time without fatigue, and relate concepts.

comment by abramdemski · 2021-02-23T17:20:51.025Z · LW(p) · GW(p)

Imagine for the sake of argument that we're happy to just absorb information, with no topic prioritization; we just want our map to get closer to the territory.

There's still the question of exactly what our loss function is: how much do we like being a specific distance from the truth? Our loss function could be more like Bayes loss, which punishes overconfidence highly (assigning probability close to zero for an event which actually is true gets us an arbitrarily bad score); or, it could be more like Brier score (which caps the amount you can lose for any particular wrong idea).

I think deep readers are more like Bayes-score maximizers: skimming the abstract and getting wrong information feels like a big risk. No quantity of improved beliefs can necessarily compensate for one mistake, because mistakes can be arbitrarily bad. A bayes-score maximizer skimming something is conscious of exactly how many grains of salt go with each thing learned, because getting that wrong can be very costly -- so they would feel a need to remember "I was speed-reading this, so I should doubt all my conclusions about it a little bit more".

Broad readers are more like Brier-score maximizers: skimming the abstract and getting a wrong conclusion is only a bounded risk, so it's easily balanced by the benefit of lots of knowledge. They don't feel an overwhelming need to count grains of salt, because 95% wrong is not that different from 100% wrong; so they happily accept a bunch of noisy information in, without worrying too much about careful tabulation of the noise.

I don't have too many intuitions about when Bayes score or Brier score will be closer to our true utility-of-knowledge functions. But I suspect deep reading is more useful for the kind of research where you're trying to generate really new things, like totally new hypotheses or new areas of mathematics. Whereas broad reading may be more useful for "applied" type research, where you're taking existing knowledge and using it in new areas.

comment by AllAmericanBreakfast · 2021-02-23T18:36:31.709Z · LW(p) · GW(p)

If we assume that the accuracy improvements to researching a given question are logarithmic, then it would make sense to read broadly on unimportant questions and read deeply on crucial questions.

Signaling also seems relevant here. It might be advantageous to be widely informed, or to be seen as the kind of person who only speaks on their domain of expertise.

There could also be times when you just need to be conversant in the subject enough to know who to delegate the deeper research to.

So in general, I would expect the value of broad vs. deep research to be highly contextual.

But I wonder if the same habits that may lead people to anchor on an inappropriate reading speed also lead them to anchor on a sometimes inappropriate reading depth. It's plausible to me that people who tend to read broadly by habit could reap significant gains by practicing deep reading on even an arbitrary subject, and vice versa.

It would be interesting if there was an equivalent to the DSM, but for reading habits. Could we imagine a test or a set of diagnostic criterion that could classify people both according to their level of reading proficiency, and also according to their habitual level of depth/breadth? So for example, a low-skill but deep reader might be a religious fundamentalist who has their text of choice practically memorized, yet who has very little familiarity with the nuances of interpretation. By contrast, we can imagine low-skill broad readers, who read all kinds of novels and newspapers but remember very little of it. And high-skill broad or deep readers, of course.

I think this is related to one of my perennial topics of interest, which is the path toward a specialization. A science student in undergrad or earlier reads broadly about science. At some point, if they continue on a scientific path, they eventually focus on a much narrower area, and their whole reading program focuses on acquiring knowledge that they perceive as directly useful to a specific research project.

As I've graduated into this phase, I've found that the deep, related, specialized, purposeful reading is vastly more satisfying than the broad, shallow, disconnected reading that came before. It makes me suspect that one reason people get turned off of science early is that they never get the experience of "cocooning" in a specialty in which all the articles you read are riffing off each other, interrelated, and building toward a goal. It's the closest thing that I've found to programming, which also entails building an interrelated construct to make predictions and do useful work.

I'm also interested in whether and how "broad reading" can be done with an equivalent sense of purpose. There's an article on Applied Divinity Studies, Beware the Casual Polymath, which I think can be characterized as a criticism of superficially high-skilled, but in fact low-skilled broad readers. It's pointing out that just because you're reading all kinds of Smart People Stuff doesn't mean that you're actually learning effectively.

I would imagine that a high-skilled broad reader would be somebody who has a role that involves lots of delegation and decision-making. The fictional example that comes to mind is a member of the Bartlett senior staff in the West Wing, who have to understand a huge number of issues of national significance, but only just enough to know who to delegate to or what positions are at least not-insane. For them, making reasonable, if not necessarily perfectly optimized, choices, but making a decision, is much more important than getting the exact right answer. So I would describe them as a depiction of a high-skilled, broad reader.

comment by gjm · 2021-02-23T00:17:08.137Z · LW(p) · GW(p)

When I was a child learning to read, most of the things I was reading were pretty easy to understand. I was clever and a good learner, and my natural way of reading is very quick, about 1000wpm for reasonably straightforward material. But I am no longer a child, and many of the things I want to read are not nearly so easy to understand (mostly because I want to read textbooks and technical papers and dense novels and so forth; probably also because at 50 I am less clever than I was at, say, 20). The techniques I acquired naturally when learning to read are a bad match for much of my actual reading, and if I want to understand things I need to go out of my way to slow things down.

I suspect the story in the paragraph above could be told with equal truth by many people around here.

As well as providing some confirmation for what AAB says about comprehension rate versus reading rate, I think this suggests that the idea of thinking of speed-reading techniques as "eye technique" that are likely to be helpful to everyone might be too optimistic. I would guess that the way I read quickly is similar to the way in which "speed readers" read quickly, mechanically at least, but unfortunately it gives me not only the ability to read quickly but also the habit of reading quickly. If I sit down to read something, the pattern in which my eyes naturally move is one that works well for stuff I can assimilate easily, and doesn't work so well for stuff I have to think harder about as I read. I suspect that in order for "eye technique" to be useful it has to become habitual, and I suspect that "eye technique" that's useful for reading quickly is systematically anti-helpful for reading slowly and deliberately. (But, for the avoidance of doubt, all of this is conjecture. I haven't studied "speed reading" techniques, I haven't compared them with how I read, I don't know whether if I went about it the right way I could make my fast-reading habits more optional, I don't know whether other people using similar techniques would form the same habits as mine, etc., etc., etc.)

comment by supposedlyfun · 2021-02-23T07:03:35.708Z · LW(p) · GW(p)

The part about habitual speed rings true to me. I am slowly working my way through the Bermúdez textbook on cognitive science and find that my brain "wants" to read at a customary pace, which is too fast for comprehension of a subject I'm relatively unfamiliar with (haven't touched a science textbook since high school).

Forcing myself to slow down gut-feels like going under the speed limit in the far-left lane on the interstate, like wasting time, even though I conscious-know it's approximately infinity times more important to understand the text than it is to be able to truthfully say that I did "reading" on all the words in it.