Has LessWrong been a good early alarm bell for the pandemic?

post by DirectedEvolution (AllAmericanBreakfast) · 2020-04-03T09:44:39.205Z · LW · GW · 20 comments

Contents

  If the sheer volume of conversation is our alarm bell, this site seems to have lagged behind the stock market by about a week.
None
20 comments

The S&P 500 began to crash on Feb. 20th.

On or prior to that date, there were 6 posts on LessWrong related to COVID-19 (thanks to habryka, who found 3 that I'd missed).

Jan 20 - Rationalist prepper thread [LW · GW]

Jan 26 - Concerning the Recent 2019-Novel Coronavirus Outbreak [EA · GW]

Feb 5 - Some quick notes on hand hygiene [LW · GW]

Feb 11 - Will nCoV survivors suffer lasting disability at a high rate? [LW · GW]

Feb 14 - Please Help Metaculus Forecast COVID-19 [LW · GW]

Feb 20 - Making sense of coronavirus stats [LW · GW]

As a bonus, Scott Alexander dropped a link in his Links 2/20 post saying "As of earlier this month, China’s coronavirus case numbers followed such a neat quadratic curve that they seem kind of like low-effort fakes. Not sure if this also applies to the current numbers."

Near-daily COVID posting began on Feb. 20th. After Feb. 27th, the number of LW COVID-19 posts has increased by about 4 posts per day to a total of nearly 200:

The New York Times has compiled a recent history of the pandemic.

Before Feb 20, what had happened?

By Feb. 23, the S&P 500 had started to descend, and Italy was beginning to lock its cities down. It wasn't until around Feb. 27th, after Trump requested a big bailout from Congress, Iran reported over 60 deaths, and cases had spread to Latin America, that LessWrong began to see accelerated COVID posting. The number spiked from around 1 post per day to the average of 4-5 posts per day we've seen since then.

If the sheer volume of conversation is our alarm bell, this site seems to have lagged behind the stock market by about a week.

20 comments

Comments sorted by top scores.

comment by habryka (habryka4) · 2020-04-03T17:39:13.017Z · LW(p) · GW(p)

You are missing a few posts that were posted earlier than February 20th. Here are some that I could find: 

February 11th by Jim: https://www.lesswrong.com/posts/h4GFHbhxE2pfiadhi/will-ncov-survivors-suffer-lasting-disability-at-a-high-rate [LW · GW]

February 5th by willbradshaw: https://www.lesswrong.com/posts/ztDYsD4v7AaAbWEDM/some-quick-notes-on-hand-hygiene [LW · GW]

January 29th on the EA Forum: https://forum.effectivealtruism.org/posts/g2F5BBfhTNESR5PJJ/concerning-the-recent-2019-novel-coronavirus-outbreak  [EA · GW]

I am pretty confident there were also some more comments that were earlier than Feb 20th, but I can't easily find them right now.

I also would be somewhat hesitant about saying that the markets crashed on February 20th. The market continued crashing for quite a while, and this is when Wei Dai wrote some comments about his investment strategy, [LW(p) · GW(p)] which, if you had followed it at that point would have still made you a good amount of money.

Replies from: matthew-barnett, Unnamed, Isnasene, AllAmericanBreakfast
comment by Matthew Barnett (matthew-barnett) · 2020-04-04T07:19:40.224Z · LW(p) · GW(p)
January 29th on the EA Forum

For vanity reasons, I'm going to point out that it was actually the 26th.

Replies from: elityre
comment by Eli Tyre (elityre) · 2021-01-02T04:44:15.156Z · LW(p) · GW(p)

And also correct credit allocation reasons!

comment by Unnamed · 2020-04-04T03:55:49.340Z · LW(p) · GW(p)

In the broader rationality/EA community there was also a Siderea post on Jan 30 and an 80K podcast on Feb 3 (along with a followup podcast on Feb 14).

These two, plus Matthew Barnett's late Jan EA Forum post (which you linked), are the three examples I recall which look most like early visible public alarms from the rationality/EA community.

Other writing was less visible (e.g., on Twitter, Facebook, or Metaculus), less alarm-like (discussions of some aspect of what was happening rather than a call to attention), or later (like the putanumonit Seeing the Smoke post on Feb 27).

comment by Isnasene · 2020-04-03T20:25:36.446Z · LW(p) · GW(p)

The question in this post is "was Less Wrong a good alarm bell" and in my opinion only one of those links constitute alarm bells -- the one on EAForums. Acknowledging/discussing the existence of the coronavirus is vastly different from acknowledging/discussing the risk of the coronavirus.

  • "Will ncov survivors suffer lasting disability at a high rate?" is a medical question that makes no implication about broader covid risk.
  • "Some quick notes on hand-hygene" does not mention the coronavirus in the main post (but to be fair does have a coronavirus tag). It does make an offhand reference implying the coronavirus could be a "maybe pandemic" but this isn't a concrete estimation of actual risk
  • "Concerning the recent 2019 novel coronavirus outbreak" is a fantastic post that makes concrete claims like it now seems reasonable to assign a non-negligible probability (>2%) to the proposition that the current outbreak will result in a global disaster (>50 million deaths resulting from the pathogen within 1 year). Per one of the comments, this was consistent with Metaculus.

Overall, I'd say that LessWrong was about on par with "having lunch conversations with Chinese-American coworkers" in terms of serving as an actual alarm bell. Moreover, in the case that we admit a weaker standard for what an alarm bell is, it's worth noting that we still don't really beat the stock market -- because it did respond to the coronavirus back in January. It just didn't respond strongly enough to convey an actual broad concrete risk.

I also would be somewhat hesitant about saying that the markets crashed on February 20th. The market continued crashing for quite a while, and this is when Wei Dai wrote some comments about his investment strategy, [LW(p) · GW(p)] which, if you had followed it at that point would have still made you a good amount of money.

As someone who pretty regularly follows Less Wrong, I missed Wei Dai's investment strategy which makes me lean in the direction that most casual readers wouldn't have benefitted from it. The linked comment itself also doesn't have very strong valence, stating " The upshot is that maybe it's not too late to short the markets." Low valence open-thread comments don't really sound like alarm bells to me. Wei Dai has also acknowledged [EA · GW]that this was a missed opportunity on EAforums.

Moreover, there was also an extremely short actionable window. On February 28th, the stock market saw a swift >5% bear market rally before the second leg of the crash which temporarily undid half the losses. Unless your confidence in "maybe its not too late to short the markets" was strong enough to weather through this, you would've probably lost money. This almost happened to me -- I sold the Thursday morning after Wei Dai's comment and bought back in Monday, netting a very meek ~3% gain.

Replies from: Vaniver
comment by Vaniver · 2020-04-03T22:35:05.145Z · LW(p) · GW(p)

"Will ncov survivors suffer lasting disability at a high rate?" is a medical question that makes no implication about broader covid risk.

This seems wrong to me, in part because the hypothesis that there could be widespread negative effects even for survivors was a compelling reason for 1) me to take it seriously (at the time, I estimated my disability risk was something like 5x the importance of my mortality risk) and 2) people to expect spread to be bad in a way that shows up in many indicators (like GDP).

Replies from: Isnasene
comment by Isnasene · 2020-04-04T00:07:03.983Z · LW(p) · GW(p)

Fair enough. When I was thinking about "broad covid risk", I was referring more to geographical breadth -- something more along the lines of "is this gonna be a big uncontained pandemic" than "is coronavirus a bad thing to get." I grant that the latter could have been a valid consideration (after all, it was with H1N1) and that claiming that it makes "no implication" about broader covid risk was a mis-statement on my part.

That being said, I wouldn't really consider it an alarm bell (and when I read it, it wasn't one for me). The top answer, Connor Flexman, states:

Tl;dr long-term fatigue and mortality from other pneumonias make this look very roughly 2x as bad to me as the mortality-alone estimates.
It’s less precise than looking at CoVs specifically, but we can look at long-term effects just from pneumonia.

For me personally:

  • A 2x increase in how bad Covid19 was in February was not cause for much alarm in general. I just wasn't that worried worried about a pandemic
  • The answer is based long-term effects of pneumonia, not covid itself (which isn't measurable). If I read something that said "hey you have a surprisingly high likelihood of getting pneumonia this year", I would be alarmed. This wasn't really that post
  • I was already kind of expecting that Covid could cause pneumonia based on typical coverage of the virus -- I wasn't surprised by the post in the way I'd expect to be if it was an alarm bell

I'll give the post some points for pointing out a useful, valuable and often-neglected consideration but I dunno. At that time I saw "you are in danger of getting coronavirus" posts as different from "coronavirus can cause bad things to happen" posts. And the former would've been alarm bells and the latter wouldn't've been.

comment by DirectedEvolution (AllAmericanBreakfast) · 2020-04-03T19:35:37.836Z · LW(p) · GW(p)

Thanks for finding these! Even with them included, though, there's an obvious sharp discontinuity in volume of posting starting on Feb. 27th. There was early information was available in this community. Some people were thinking hard about it. But I think based on this, the majority of us were still waiting for other people to tell us there was a catastrophe at hand. And we kept waiting and waiting until the stock market let us know.

Based on that, I'd say that we have not yet achieved any kind of substantial collective coordination or debiasing. By and large, we're still passively waiting for the consensus to come to us and shake us out of complacency. I think we should feel especially chagrined given the constant conversation around black swans, catastrophic risk (specifically around pandemics), and the high level of mathematical competence in this community.

Replies from: Vaniver, Raemon
comment by Vaniver · 2020-04-03T20:42:08.697Z · LW(p) · GW(p)

Even with them included, though, there's an obvious sharp discontinuity in volume of posting starting on Feb. 27th. 

Is this the right comparison? It seems to me that the interesting question is "what was the balance of information available on LW before the 20th?" and "how much information was there on LW before the 20th?", not "did the amount of discussion on LW increase over time?". In worlds where we had posted the perfect pandemic survival guide on Feb 11th, and then as more and more people realized the crisis was real and posted questions here, the posts graph would look a lot like the one you posted.

To the best of my knowledge, LW had very little minimization or pushback against preparation (and what pushback I recall was generally of precautions that probably were too extreme or could have been accomplished more cheaply).

Based on that, I'd say that we have not yet achieved any kind of substantial collective coordination or debiasing. By and large, we're still passively waiting for the consensus to come to us and shake us out of complacency.

I mean, this seems true in the sense that most online communities are 90% lurkers.

I think my main regret was something like "assuming more people were on top of this," explicitly or implicitly; I was spooked and preparing early enough that when I went on one of my last outings to get something with Duncan, our conversation spooked him and he started preparing, and then he was on top of it in time to help other people prepare. But you won't see any warnings from me on LessWrong or Facebook, because my reaction was closer to "ah now I have a bunch of chores to get ready myself, and I'm not making inferences on any private data, so others can come to the same conclusion if they want to" than "oh jeez I need to make sure everyone is aware of this / I need to publicly vouch for my inference to best shift the group epistemology."

But also most of the conversations that I was having about this were private, in one way or another; what should this group house do? What do I think of this draft doc my boyfriend's group house is working on? Some of this eventually made its way to LW, mostly through the actions of a few heroes, but also often after days of private discussion and fact-checking.

Replies from: AllAmericanBreakfast
comment by DirectedEvolution (AllAmericanBreakfast) · 2020-04-03T22:30:49.292Z · LW(p) · GW(p)

The only way it makes sense for the LW community to have been on top of this, yet for nCov posting on LW to explode only after Feb 27th, is if most of us were focused on practical prep for nCov and didn't have time to post about it extensively until that date.

It's hard for me to see a reason why we'd have such a sharp discontinuity in posting activity if that were the case.

It makes much more sense to say that we as a community just fundamentally didn't see what a huge deal this was going to be, and start getting obsessed by the implications, until that time.

I'm making two key claims about the state of this community regarding "effective rationality."

1) Those who did understand the seriousness of this epidemic early were too quiet online to act as leaders for the digital readership. Next time, they should be louder, post more, and drive the message home.

2) Most of the online community, including myself, is not yet able to put two and two together and depends on blaring social norms to decide what to care about. Yet we had all the information and skills we needed to come to the right conclusion earlier than the stock market. We need to ask how we could do better next time.

comment by Raemon · 2020-04-03T20:20:45.511Z · LW(p) · GW(p)

It seems like the relevant metric here is "when did people start preparing in some way?" not "when did posts increase." (This is a bit harder to check. I do naively expect that the turning point here would be Seeing the Smoke, which was indeed on February 28th which is inline with your analysis. I do think that this is still prior to most other people, or at least most Americans – not sure about other countries)

I do think a lot of the mechanisms by which rationalists I know did prepare was social – some rationalists were paying heavy attention to it, and they got their friends to be aware of it. This is less useful for people who aren't as socially entwined.

comment by df fd (df-fd) · 2020-04-05T05:48:45.803Z · LW(p) · GW(p)

as someone who is working in ED in Australia, I can say that Lesswrong has been consistently predict the behaviors of my hospital management by 2-3 days.

for example: I've read a lesswrong post arguing that covid-19 is serious and people should stock up while my manager was still telling us to just calm people down if asked. The stance was reversed 2-3 days later.

Slatestarcodex recommended the wide spread use of masks a whole week before my local health district recommened the same thing.

I am not certain that being slow is not part of the system design. but Lesswrong has allow me to predict future hospital policies quite consistently.

comment by Isnasene · 2020-04-03T21:17:52.239Z · LW(p) · GW(p)

While I agree with the specific claims this post is making (i.e. "Less Wrong provided information about coronavirus risk similar to or just-lagging the stock market"), I think it misses the thing that matters. We're a rationality forum, not a superintelligent stock-market-beating cohort[1]! Compared to the typical human's response to coronavirus, we've done pretty well at recognizing the dangers posed by the exponential spread of pandemics and acting accordingly. Compared to the very smart people who make money by predicting the economic effects of a virus, we've done expectedly mediocre -- after all none of us (including the stock market) really had any special information about the virus's trajectory.

Maybe it is disappointing if we lagged the stock market instead of being perfectly on pace with it but a week of lag is a pretty small amount of time in the grand scheme of things. And I'd expect different auditing methodologies/interpretations to have about that amount in variance. In any case, I don't really think that it's a big deal.

[1]That is, unless you count Bitcoin, which Eliezer Yudkowsky doesn't.

comment by Unnamed · 2020-04-03T22:34:03.635Z · LW(p) · GW(p)

I think this post is giving the stock market too much credit.

I'd date the start of the stock market fall as February 24 rather than February 20. The S&P close on Feb 20 & Feb 21 was roughly the same as it had been over the previous couple weeks, and higher than the close on Feb 7, 5, 4, or 3. The first notable dip happened on February 24th; that was the first day that set a low for the month of Feb 2020 (and Feb 25 was the first day that set a low for calendar year 2020).

Also, that was just the start of the crash. The stock market continued falling sharply and erratically for a couple more weeks, and didn't get within 10% of its current level until March 12th (2.5 weeks after it started its fall on Feb 24).

comment by ignoranceprior · 2020-04-05T08:29:14.844Z · LW(p) · GW(p)

The "Rationalist prepper thread" was actually posted on January 28, not January 20.

comment by ChristianKl · 2020-04-05T13:28:13.819Z · LW(p) · GW(p)

At my LessWrong meetup in Berlin on the 16/02 I used the example of Corona and preparing for the possibility of it being serious as example to explain what applied rationality is about.

On a local channel I did ask on 19/02 how to invest into the market to profit from it but I tried to be to smart about targeting instead of just buying shorts and later when I tried to buy short trival inconvenience of my bank saying that I have to call them for unlocking my account stopped me from investing.

comment by habryka (habryka4) · 2020-04-03T17:20:48.473Z · LW(p) · GW(p)

Fixed your embedded images for you.

Replies from: habryka4, AllAmericanBreakfast
comment by habryka (habryka4) · 2020-04-03T17:31:43.664Z · LW(p) · GW(p)

However, I think your image for the S&P 500 is the wrong image. Will replace it with the right one.

comment by DirectedEvolution (AllAmericanBreakfast) · 2020-04-03T19:22:36.073Z · LW(p) · GW(p)

Thanks very much! For future reference, is there something else I should be doing besides pasting a shareable link generated by Google docs into the image URL box?

Replies from: habryka4
comment by habryka (habryka4) · 2020-04-04T01:18:48.192Z · LW(p) · GW(p)

Sadly Google Docs really doesn't like being hosted on external websites, so I would try to upload them somewhere else (imgur works well for me).