90% of anything should be bad (& the precision-recall tradeoff)

post by cartografie · 2022-09-08T01:20:56.170Z · LW · GW · 22 comments

Contents

  Introduction
  The precision-recall tradeoff
    Signal Detection Theory
    The Promising Threshold
    Optimising the Promising Threshold
  Applying Sturgeon's Edict
  Why do people publish too conservatively?
      FP is more salient than FN
      TP is more salient than FP
      Hindsight bias
      Poorly-calibrated heuristics
  Some practical recommendations
None
22 comments

Introduction

Sturgeon’s Law states that 90% of anything is crap.

So 90% of sci-fi books are crap,
    and 90% of medical research,
    and 90% of jazz solos,
    and 90% of LessWrong posts.

Etc, etc — 90% of anything is crap.

Now, people usually interpret Sturgeon’s Law pessimistically.

“What a shame!”  people think,  “If only all the sci-fi books were great!  And all the medical research, jazz solos, LessWrong posts, etc etc.”

I think these people are mistaken.  90% of anything should be crap.  In this article, I will explain why.

The precision-recall tradeoff

Imagine one day you read a brilliant poem, and you decide to read the poet’s complete works.  If you discover that every poem she ever wrote was brilliant, how might you feel about that?

Well, you might feel a little depressed.  If the poet — let’s call her Alice — only published brilliant poems, then there are probably dozens of brilliant poems that she left unpublished.  She could only have maintained a perfect track-record by being so conservative that a lot of brilliant work was lost forever.

We can make this slightly more formal using Signal Detection Theory.

Signal Detection Theory

When Alice publishes a good poem, that's a true-positive.
When Alice publishes a bad poem, that's a false-positive.
When Alice rejects a good poem, that's a false-negative.
When Alice rejects a bad poem, that's a true-negative.

We can define two measures of how wisely Alice publishes her work:

1. Precision = $\frac{TP}{TP+FP}$  is the proportion of her published poems which are good.
2. Recall = $\frac{TP}{TP+FN}$  is the proportion of her good poems which are published.

High precision means that most of her published poems are good, whether or not her unpublished poems are good as well.  High recall means that most of her good poems are published, whether or not her bad poems are published as well.

There is often a tradeoff between precision and recall.  Alice can only increase one at the cost of decreasing the other.  If Alice published more conservatively, then she’d achieve lower recall (boo! less good poems!), but she’d also achieve higher precision (yay! less bad poems!).  If Alice published more liberally, then she’d achieve lower precision (boo! more bad poems!), but she’d also achieve higher recall (yay! more good poems!).

In the next section, I will explain the origin of the tradeoff.

The Promising Threshold

Some of Alice’s poems are more promising than others, i.e. they have a higher likelihood of being good.  Suppose Alice writes down her poems $X_1,X_2,X_3\dots$ in descending promisingness, and let $p_n=P(X_n\in G)$ be the likelihood that the $n$th most promising poem is good.

Alice will publish any poem which has more than $\tau$ likelihood of being good, where $\tau \in [0,1]$ is her promising threshold.  If $\tau$ is small then Alice publishes liberally, and if $\tau$ is large then Alice publishes conservatively.

In the graph below, the green area shows her good poems and the red area shows her bad poems.  The dark area shows her published poems, and the light area shows her rejected poems.  The four areas represent $FP,TN,FN,TP$ in clockwise order.

By squinting at the graph, you can see that if Alice increases $\tau$ then there are fewer false-positives and more false-negatives, whereas if Alice decreases $\tau$ then there are fewer false-negatives and more false-positives.

Moreover, if Alice increases $\tau$ then precision increases and recall decreases, whereas if Alice decreases $\tau$ then precision decreases and recall increases. Note that in general, precision will be higher than $\tau$.

Optimising the Promising Threshold

The optimal value for $\tau$ is determined by the relative cost between false-negatives and false-positives.  If false-negatives are much costlier than false-positives, then recall should be higher than precision.  Otherwise, if false-positives are much costlier than false-negatives, precision should be higher than recall.

Consider the poem $X_n$ which has $p_n$ chance of being good and $(1-p_n)$ chance of being bad.  If Alice rejects the poem, then she can expect a cost of $p_n\times C_{FN}$, where $C_{FN}$ is the cost of each false-negative.  If Alice publishes the poem, then she can expect a cost of $(1-p)\times C_{FP}$, where $C_{FP}$ is the cost of each false-positive.  Alice should publish if and only if the expected cost of rejection is greater than the expected cost of publishing. This holds when $p_n\times C_{FN}\ge (1-p_n)\times C_{FP}$, or equivalently when $p_n\ge \frac{C_{FP}}{C_{FP}+C_{FN}}$.

So the optimal promising threshold ${\tau }^{\star }=\frac{C_{FP}}{C_{FP}+C_{FN}}$.

But what actually are the relative costs of a false-positive and a false-negative? What do we actually care about here?  Well, we mostly care about publishing good poems, regardless of whether a few bad poems sneak past.  Publishing a bad poem might be annoying, but it’s ultimately harmless.  In contrast, rejecting a great poem would be a tragedy.

${\tau }^{\star }$ is small, so (surprisingly) precision should be low.  For poetry, I claim precision should be about 10%, and likewise for sci-fi books, medical research, jazz solos, LessWrong posts, etc.

In other words, ninety percent of anything should be crap.

I call this “Sturgeon’s Edict”.  Whereas Sturgeon’s Law is descriptive, Sturgeon’s Edict is prescriptive.  If more than 10% of sci-fi books were good, Sturgeon's Edict would mandate that the industry published more liberally.

Applying Sturgeon's Edict

1. If you design nuclear reactors, then Sturgeon's Edict doesn't apply.  A false-positive is very costly, so you need a much higher precision than 10%.  You need to be precision-maxxing.
    
2. If you're a sci-fi author, or a jazz soloist, or LessWrong poster, or a nature photographer, or something like that, then Sturgeon's Edict definitely applies, because a false-negative is much costlier than a false-positive.  If your precision is above 10%, then you are probably rejecting good work.  You need to be recall-maxxing.
    
3. Suppose you're Franz Kafka on your deathbed.  You never managed to finish any of your full-length novels.  Should you instruct your friend to burn your surviving manuscripts?
   
   Well, against Kafka's explicit instructions, his manuscripts were actually published.  Most of his writings weren't very good, but some were brilliant, and low precision is exactly what Sturgeon's Edict recommends.  So Kafka shouldn't have asked his friend to burn his manuscripts — he should've been recall-maxxing, not precision-maxxing.
    
4. Suppose you’re Michaelangelo carving a marble sculpture for the Tomb of Pope Julius II.  You're halfway through carving when the pope changes his mind.  Should you finish the sculpture anyway?
   
   This is borderline case of Sturgeon's Edict, because it would take considerable effort to finish the sculpture, so the cost of a false-positive isn’t negligible.
    
5.  Sturgeon’s Edict doesn’t always recommend publishing more liberally.
   
   Consider the Library of Babel.  This is a collection of books containing every possible page of 3200 characters.  The library has almost 100% recall because every great idea can be found somewhere in the library.  However, the library has almost 0% precision, because almost every page in the library is meaningless gibberish.
   
   Because the library has 0% precision, it’s completely useless for finding great ideas.  So you can go too far in recall-maxxing!  A library would be functional if it followed Sturgeon’s Edict, and had a precision of 10% rather than 0%.

Why do people publish too conservatively?

Here are four plausible explanations of why people might publish too conservatively, in violation of Sturgeon's Edict.

FP is more salient than FN

Alice only ever sees the published poems of other poets, not their unpublished poems. So when someone publishes a bad poem, that’s a visible blunder, but when someone rejects a great poem, that’s an invisible tragedy. So false-positives are more salient to Alice than false-negatives. 

Because she focuses on salient costs, Alice publishes too conservatively.

TP is more salient than FP

Alice remembers the great poems that other poets published but forgets their bad poems, so true-positives are more salient to her than false-positives. Therefore she overestimates the precision of other poets.

Because she tries to copy that precision, she publishes too conservatively.

Hindsight bias

When Alice reads a great poem, she overestimates how sure the poet would've been that the poem was great. Likewise, when she reads a bad poem, she overestimates how sure the poet would've been that the poem was bad. This is called the Hindsight Bias [LW · GW].

So Alice assumes that, had the poet published more conservatively, only the bad poems would’ve been weeded out. A cautious poet still would’ve published the great poems, because they were predictably great.

Because she underestimates its harm, she publishes too conservatively.

Poorly-calibrated heuristics

In the ancestral environment, false-positives were costlier than they are today. It was more difficult to store information, communicate information, and search information — and therefore people had to be sure they were publishing something good.

Although this behaviour was well-calibrated to the ancestral environment, it's poorly-calibrated to the contemporary environment. Modern technology has lowered the cost of storing, communicating, and searching information by many orders of magnitude, so the cost of a false-positive is negligible. 

Because Alice's behaviour is poorly-calibrated, she publishes too conservatively.

Some practical recommendations

1. If the cost of a false-positive is negligible, then publish.
    
2. It will be easier to follow Sturgeon’s Edict if you always remember — perfectionism is an ego problem.
    
3. "But what if I haven’t yet overcome my ego problem? I don't want to publish bad work."
   
   I would recommend that you publish your less promising work anonymously. Or pseudonymously. This way your track-record doesn’t drop, but the work still gets published. You can always claim ownership later if the work is well-received.
    
4. A perfect track-record is a dangerous thing. Maybe you should publish something terrible so you overcome the mental hurdle of that first imperfection.
    
5. Go through your drafts. If you know you won't improve the draft, publish it now. If it's really worthless, then maybe publish the draft as a twitter thread.

22 comments

Comments sorted by top scores.