What are some low-information priors that you find practically useful for thinking about the world? 2020-08-07T04:37:04.127Z · score: 19 (9 votes)


Comment by linch on Against Victimhood · 2020-09-23T06:13:26.454Z · score: 1 (1 votes) · LW · GW

Right now it's very hard to determine whether I agree or disagree with the article.

I think there are a lot of verbal claims here, and it feels almost entirely like a question of mood affiliation to determine how in-alignment I am with the central thesis/which direction of claims do I agree with/how much do I agree with them. 

Not telling you how to live your life, but I'd personally benefit from more numerical claims/quantified uncertainty.

Comment by linch on Why haven't we celebrated any major achievements lately? · 2020-09-13T19:06:11.808Z · score: 1 (1 votes) · LW · GW

Competitive programming, maybe? Though perhaps the skill ceiling is lower than in professional sports.

Comment by linch on Multitudinous outside views · 2020-08-19T19:37:31.616Z · score: 7 (5 votes) · LW · GW

Another in-the-field example of differing reference class intuitions here, on the Metaculus question:

Will Ghislaine Maxwell be alive on 1 January 2021?

The other commentator started with a prior of actuarial tables on death rates of 58 year old women in the USA, and argued that going from a base rate of 0.3% to 10% means a 33.3x increase in log-odds, which is implausibly high given the evidence entailed.

I thought actuarial tables were not a plausibly good base rate to start from, since most of the Ghislaine Maxwell-relevant bits are not from possibility of natural death.

Hopefully the discussion there is helpful for some lesswrong readers in understanding how different forecasters' intuitions clash "in practice."

Comment by linch on Multitudinous outside views · 2020-08-18T20:43:43.068Z · score: 5 (3 votes) · LW · GW

Some scattered thoughts:

1. I think it's very good to consider many different outside views for a problem. This is why I considered section 2.1 of Yudkowsky's Intelligence Explosion Microeconomics to be frustrating/a weak man, because I think it's plausibly much better to ensemble a bunch of weak outside views than to use a single brittle outside view.

"Beware the man of one reference class" as they say.

2. One interesting (obvious?) note on base rates that I haven't seen anybody else point out: across time, you can think of "base rate forecasting" as just taking the zeroth derivative (while linear regression is a first derivative, etc).


So which reference class is correct? In my (inside) view as a superforecaster, this is where we turn to a different superforecasting trick, about considering multiple models. As the saying goes, hedgehogs know one reference class, but foxes consult many hedgehogs.

I think while consulting many models is a good reminder, the hard part is choosing which model(s) to use in the end. I think your ensemble of models can often do much better than an unweighted average of all the models you've considered, since some models are a) much less applicable, b) much more brittle, c) much less intuitively plausible, or d) much too strongly correlated than other models you have.

As you've illustrated in some examples above, sometimes the final ensemble is composed of practically only one model!

4. I suspect starting with good meta-priors (in this case, good examples of reference classes to start investigating) is a substantial fraction of the battle. Often, you can have good priors even when things are very confusing.

5. One thing I'm interested in is how "complex" do you expect a reasonably good forecast to be. How many factors go into the final forecast, how complex the interactions between the parameters are, etc. I suspect final forecasts that are "good enough" are often shockingly simple, and the hard part of a forecast is building/extracting a "correct enough" simplified model of reality and getting a small amount of the appropriate data that you actually need.

Once an experienced analyst has the minimum information necessary to make an informed judgment, obtaining additional information generally does not improve the accuracy of his or her estimates. Additional information does, however, lead the analyst to become more confident in the judgment, to the point of overconfidence.

Experienced analysts have an imperfect understanding of what information they actually use in making judgments. They are unaware of the extent to which their judgments are determined by a few dominant factors, rather than by the systematic integration of all available information. Analysts actually use much less of the available information than they think they do.
There is strong experimental evidence, however, that such self-insight is usually faulty. The expert perceives his or her own judgmental process, including the number of different kinds of information taken into account, as being considerably more complex than is in fact the case. Experts overestimate the importance of factors that have only a minor impact on their judgment and underestimate the extent to which their decisions are based on a few major variables. In short, people's mental models are simpler than they think, and the analyst is typically unaware not only of which variables should have the greatest influence, but also which variables actually are having the greatest influence.

From Psychology of Intelligence Analysis, as summarized in the forecasting newsletter (emphasis mine).

If this theory is correct, or broadly correct, it'd point to human judgmental forecasting being dramatically different from dominant paradigms in statistical machine learning, where more data and greater parameters usually improve accuracy.

(I think there may be some interesting analogies with the lottery ticket hypothesis that I'd love to explore more at one point)

Comment by linch on Multitudinous outside views · 2020-08-18T20:20:15.972Z · score: 8 (3 votes) · LW · GW
And for COVID, I've written about my very early expectations - but maybe you think that a follow-up on why superforecasters mostly disagreed with my forecasts / I modeled things differently than them over the past 3-4 months would be interesting and useful

I'd be interested in this.

Comment by linch on Are we in an AI overhang? · 2020-08-08T09:49:17.114Z · score: 3 (3 votes) · LW · GW

Re hardware limit: flagging the implicit assumption here that network speeds are spotty/unreliable enough that you can't or are unwilling to safely do hybrid on-device/cloud processing for the important parts of self-driving cars.

(FWIW I think the assumption is probably correct).

Comment by linch on What are some low-information priors that you find practically useful for thinking about the world? · 2020-08-08T09:14:55.096Z · score: 2 (2 votes) · LW · GW

I think my base rate for basic comprehension failures is at a similar level!

Comment by linch on What are some low-information priors that you find practically useful for thinking about the world? · 2020-08-08T09:07:42.065Z · score: 1 (1 votes) · LW · GW

Wow thank you! I found this really helpful.

Comment by linch on Can you gain weirdness points? · 2020-07-31T09:00:21.310Z · score: 9 (4 votes) · LW · GW

Many of the thinker-heroes that we revere now, like Jeremy Bentham, Isaac Newton, Florence Nightingale, and Benjamin Franklin, among others, had ideas that were considered deeply weird within their time. Many of them were considered quite popular even within their own time.

Comment by linch on What a 20-year-lead in military tech might look like · 2020-07-30T06:55:34.678Z · score: 7 (4 votes) · LW · GW

Information request: How large are leads in military tech for wars historically? My naive impression was that there was a >20 year lead for, eg, US-Vietnam, USSR-Afghanistan, both the first and second Italy-Ethiopian wars, great emu war etc.

I'm also curious how much of a lead the US currently has over, e.g., other permanent members of the UN Security Council.

Comment by linch on A Personal (Interim) COVID-19 Postmortem · 2020-06-28T09:43:36.548Z · score: 7 (2 votes) · LW · GW

I agree with the following points:

  • That European countries very much appear to have this under control
  • That they did much better than the US and Latin America
  • Right-wing populist leaders did worse than I expected, in a non-coincidental way (Brazil's Bolsonaro is another example to add to the list).
  • "trying to control the narrative over dealing with problems is a particularly dangerous approach with infectious diseases" very strongly agreed. I'm a big fan of this write-up by NunoSempere, and this historian's touching reflection on the Spanish flu.

I think it's likely our disagreements are somewhat about framing than actual empirical differences. For example, "they seems poised to have gotten it under control before it ended up everywhere, though they didn't catch it enough to prevent spread at first, which would have been the goal" is a phrase I'd use to describe South Korea and Singapore, not Western Europe, where almost every locale had community transmission. I'd use "they caught it enough to prevent spread" to describe places like Mongolia with zero or close to zero community transmission, or contained community transmission to a single region.

I agree that Western European governments should get a lot of relative credit for managing to prevent more deaths, disability, and wanton economic destruction, despite being in an initially bad spot. But thousands of people nonetheless died, and those deaths appeared to be largely preventable (in a practical, humanly doable sense). So while I think we should also a) emphasize the relative successes (because in these dark times it's good to both hold on to hope and be grateful for what we have), and b) be unequivocally clear that the other Western governments mostly did better than the US, I do want to not lose sight of the target and also be clear that the relative failings of the US under Trump does not excuse the lesser failings of other institutions and governments.

Comment by linch on A Personal (Interim) COVID-19 Postmortem · 2020-06-26T22:01:44.733Z · score: 4 (3 votes) · LW · GW

I know the conversation these days is (rightly) about preventing presymptomatic transmission from the wearer, but I'm personally still at ~80% that masks probably protect the wearer at least a little, though agree that the effect may not be huge.

Comment by linch on A Personal (Interim) COVID-19 Postmortem · 2020-06-26T12:25:59.278Z · score: 15 (10 votes) · LW · GW
people find it far easier to forgive others for being wrong than being right

Harry Potter and the Half-Blood Prince

First of all, I really appreciate this postmortem. Admitting times when you were wrong couldn't have been an easy task, particularly if/when you staked a lot of your identity and reputation to being right. As EA and rationalist individuals and institutions become older and more professionalized, I'm guessing that institutional pressures will increasingly push us further and further away from wanting to admit mistakes; so I sincerely hope we get in the habit of publicly recognizing mistakes early on. (Unfinished list of my own mistakes, incidentally[1]). I hope to digest your post further and offer more insightful thoughts, but here are some initial thoughts:

Addendum on masks:

Another consideration about masks is that masks turn out in practice to be very reusable, a fact we (or at least I) should have investigated a lot more in early March.

On hospital-based transmission:

I don't know how much you believed in it, but as presented, this appears to be merely (ha!) a forecasting error rather than a strategic error. In the absence of a clear counterfactual, I don't think you were obviously wrong here, since it's quite plausible that if a lot of people like you ignored/downplayed the role of hospital-based transmission, it'd have gotten a lot worse.

On being a jerk re Jim and Elizabeth's post:

For what it's worth, I also (privately) asked them to take it down because I had similar considerations to you and thought the thing they wrote about masks was unilateralist-y and a bit of an infohazard. I think I was wrong there. But I think I mostly was object-level wrong about the relative tradeoffs and harms. To the extent I updated now, a) I updated object-level on how much I should cooperate or desire others to cooperate with specific institutions, and b) I updated broadly (but not completely) in general favor of openness and against censorship.

I continue to maintain that if I (and possibly you) had the same object-level beliefs as before, it was not incorrect to consider it an info-hazard (but not all object-level info-hazards are worth suppressing! Particularly if release promotes the relevant meta-level norms more than it harms), though of course not an existential one.

On superforecasting:

You said you think superforecasting is

materially worse than [you] hoped it would be at noticing rare events early.

I don't know how high your hopes were, but for what it's worth, I think this proves too much. I'm not sure about the exact aggregation algorithms that the Open Phil Good Judgement covid-19 project was running, but I feel like all I can realistically gather was that "of this specific set of part-time superforecasters that were on the Open Phil-funded project, more than 50% of them were way too optimistic."

While it's certainly some evidence against superforecasters being good at noticing rare events early, I don't think it's sufficient evidence against superforecasters being able to do this, and I definitely don't think this is a lot of evidence against superforecasting as a process.

As you weakly allude to, if you were on the project and paying attention more, you would probably have done better. Likewise, I know other superforecasters who I think were much more pessimistic than the GJ median. I suspect superforecasters who regularly read LessWrong and the EA Forum would have done better; and if I were to design a better system for superforecasting on rare events, I'd a) prime people to pay attention to a lot of rare events first, and b) have people train and score on log-loss or some other scoring system that's more punishing of overconfidence than Brier.

(All that said, I think Metaculus did okay but not great on covid-y questions relative to what someone with high hopes for prediction aggregation algorithms might reasonably expect).

On US Gov't Institutions:

I think there was a bunch of insights that your policy research experience has colored. For example, you mention how you trusted the FDA to have done a lot better under Scott Goettlieb. This might be obvious to you, but it's something I didn't even really think about until you highlighted this point. You also highlight a lot of useful specific uncertainties about whether the issue was political directors under Trump or nonpolitical directors of specific institutions. I think all of these things are very useful to know from the perspective of a policy researcher like yourself (and for students of US policy), since how to reform institutions is very decision-relevant to you and many other EAs.

That said, at a very coarse level, I think I'm a lot more cynical than you are implying with regard to how well US institutions would have handled this pre-Trump. It's possible we're not actually disagreeing, so I'm curious on your counterfactual probabilities on things being an order of magnitude better (<20,000 Americans dead of COVID-19 by now, say) in the following two worlds:

a) Clinton administration continuing all of Obama's policies?

b) Clinton administration continuing all of Obama's policies except for US CDC in China being equally understaffed as they are in our timeline.

My reasoning for why I'm generally pretty cynical (at least conditional upon this pandemic spreading at all, maybe a larger international presence could have helped contained it early) in those counterfactual worlds[2]:

1) There's sort of an existing counterfactual for preparedness of governments with a broadly American/Western culture but as competent at governance as a typical European country. It's called Europe. And I feel like every large geographically Western country was pretty bad at preparedness? People are praising Germany's response, but when it comes down to it, Germany has 9000+ confirmed covid-19 deaths in a population of 83 million, or >100 deaths/million, despite taking a large economic hit to suppress the pandemic. Japan had <1000 confirmed deaths in a population of 126.5 million. Now Japan was bad at testing, so maybe Japan actually had ~4000 deaths. But even at those numbers (~31 deaths/million), Japan still had <1/3 the number of deaths per capita as Germany. And object-level, Japan seemed to have screwed up a bunch of important things, so there's a simple transitivity argument where if a high-income country did worse than Japan, their policies/institutions couldn't have been that great.

Maybe I'm harping on this too much, but I really don't want us to succumb to the tyranny of low expectations here.

Now some culturally Western countries did fine (Australia, New Zealand). I'm not sure why they did well (maybe it's because they're islands, maybe seasonality is bigger than I think so Southern hemisphere had a huge initial advantage early on, maybe because they're around 10-15% East Asian so people had enough ties to China to be worrying earlier, maybe low population density, maybe their institutions are newer and better, maybe just luck), but regardless, I'd counterfactually bet on the response of Hillary's America looking more like a slightly less competent Europe or maybe Canada and less like Australia/NZ.

2) I didn't look at it that much, but at the high-level, the US response to 2009 H1N1 looked more competent, but ultimately the response didn't seem sufficient to have achieved containment if the mortality rates were as high as people thought it'd be? (Not sure of this, willing to be convinced otherwise on this one).

3) Some inside-view reasoning about specific actors.


Anyway, all these gripes aside, thank you again for your thoughtful (and well-written!) post. That couldn't have been easy to write, and I really appreciate it.

[1] Your post actually me to thinking about how I should be more honest/introspective about my strategic and not just predictive mistakes, so thanks for that! I plan to update the list soon with some strategic mistakes as well. For example, I considered myself to be on the "right" side of masks epistemically but not strategically.

[2] I'm maybe 35% on a) and 30% on b). A lot of the probability mass is considerations on there being enough chaos/sensitivity to initial conditions that this pandemic maybe wouldn't have happened at all, rather than Obama's or Hillary's response being an order of magnitude better conditional upon there being an epidemic.

Comment by linch on Simulacra Levels and their Interactions · 2020-06-26T09:08:42.627Z · score: 3 (2 votes) · LW · GW

First of all, I really appreciate this article! It helps me conceptualize cleanly some vocabulary that's flying around in the rationalist community that I previously didn't really understand.

To me, the most obvious missing archetype is The Truth-Giver or perhaps The Teacher.

The Teacher is concerned with only conveying truthful messages. She will usually tell the truth, but she may occasionally omit truthful things, or even (rarely) tell half-truths if she thinks it's easier to convey truthful messages via half-truth. Importantly, she's different from the Sage or the Pragmatist in that she's not concerned with other object-level consequences, only in conveying truthful messages.

Consider the claim:

There’s a pandemic headed our way from China

Suppose that The Teacher believes that the following is more correct:

There's a pandemic headed our way from Italy.

The Teacher will choose usually to clarify and say the full message, however if she only has one bit of response, she'll say "yes" to "Is there a pandemic headed our way from China?" Importantly (unlike the Pragmatist) she'll do this even if the perceived consequences are negative, as long as the subject gets more truthful information than they otherwise would have.

Against Level 1 and Level 2 players, The Teacher will never see a need to resort to Level 3. However, against a fully Level 3 player, she will (begrudgingly) issue utterances correctly conveying the ideological faction she's on, as that's the most relevant/only bit to transfer to fully Level 3 players.

I think Teacher roles are incredibly important in practical everyday communication, since all information is lossy, inferential gaps are common, attention and text is limited, etc. Indeed, I would go so far as argue that Teacher roles are often preferable to Oracle roles in murky situations if the purpose is to collectively seek truth.

Comment by linch on Thomas C. Schelling's "Strategy of Conflict" · 2014-01-31T19:23:31.925Z · score: 1 (1 votes) · LW · GW

Hi! First post here. You might be interested in knowing that not only is the broken radio example isomorphic to "Chicken," but there's a real-life solution to the Chicken game that is very close to "destroying your receiver." That is, you can set up a "committment" that you will, in fact, not swerve. Of course, standard game theory tells us that this is not a credible threat (since dying is bad). Thus, you must make your commitment binding, eg., by ripping out the steering wheel.