Risk and uncertainty: A false dichotomy?

post by MichaelA · 2020-01-18T03:09:18.947Z · LW · GW · 9 comments

Contents

  Overview
      But what do we really mean by “uncertainty”?
    Epistemic status
  What’s the distinction meant to mean?
  Does the distinction make sense?
    Certain knowledge of a probability?
    Zero knowledge of a probability?
  Decision-making given this distinction
    Principle of indifference
    Maximin
    Avoiding a lack of probabilities
    Do we have a choice?
  Unknown unknowns
  Closing remarks
None
9 comments

Overview

We’re often forced to make decisions under conditions of uncertainty. This may be empirical uncertainty (e.g., what is the likelihood that nuclear war would cause human extinction?), moral uncertainty (e.g., does the wellbeing of future generations matter morally?), or one of a number of other types [LW · GW] of uncertainty.

But what do we really mean by “uncertainty”?

According to [one] view, certainty has two opposites: risk and uncertainty. In the case of risk, we lack certainty but we have probabilities. In the case of uncertainty, we do not even have probabilities. (Dominic Roser [who argues against this view])

So what are we really talking about - risk, or (Knightian) uncertainty? What is such a distinction meant to mean? Does such a distinction make sense? What significance might this distinction have for how we resolve and make decisions given a lack of certainty? And what about unknown unknowns, black swans, and incomplete models?

These are the questions I discuss in this post, arriving at the following claims:

1. The risk-uncertainty distinction is usually not adequately specified; it lets a lot of the “work” be done by ambiguous phrases such as whether probabilities are “known” or “exact”.^[1]

2. Proponents of the risk-uncertainty distinction usually seems to discuss it as if it’s an absolute, binary distinction, or fundamental dichotomy; as if in some cases we really do “know” (or “really can estimate”, or whatever) the probabilities of interest, while in other cases we really can’t at all. To be clear, the alternative to this view is the idea that:

   • sometimes we have extremely little basis for estimating probabilities
   • other we times have extremely good basis for estimating probabilities
   • but these situations aren’t fundamentally different
   • there’s a continuum of possibilities in between those extremes
   • (I would obviously agree with the above four points, and thus with the idea of distinguishing between more “risk”-like situations and more “(Knightian) uncertainty”-like situations, along a continuum. This post [LW(p) · GW(p)] collects some terms and concepts relevant to this more continuous view.)

3. That’s a false dichotomy; no absolute, binary distinction can be made.

   • One quick quote to indicate that this view isn’t unique to me: “Bayesian approaches to probability treat it as a degree of belief and thus they do not draw a distinction between risk and a wider concept of uncertainty: they deny the existence of Knightian uncertainty” (Wikipedia).
   • (This post will essentially presume a Bayesian interpretation of probability, or possibly also argue for it, though I doubt that all the claims I make rely on the Bayesian interpretation.)

4. Thinking that there is such a binary distinction can lead to using strange and suboptimal decision-making procedures.

This post discusses each of these four claims in turn. I close by considering how unknown unknowns (or black swans, or incomplete models) fit into this picture.

This post doesn’t address the idea that, as a practical or heuristic matter, it might be useful to act as if there’s a risk-uncertainty distinction. My next post [EA · GW] will address that idea, and ultimately argue against it.

Epistemic status

The questions covered in this post all subject to substantial debate and have received some good treatments before. (I’d particularly recommend this short post by Ozzie Gooen, this paper by Dominic Roser [behind a paywall, unfortunately], and this [LW · GW] series [LW · GW] of [LW · GW] posts [LW · GW] by Nate Soares.) I’m also not an expert on these topics. Thus, this is basically meant as a collection and analysis of existing ideas, not as anything brand new. I’d appreciate feedback or comments in relation to any mistakes, unclear phrasings, etc. (and just in general!).

My three goals in writing this were to:

• Force me to clarify my own thinking and understandings.
• Provide an additional primer/resource on these topics, which will hopefully be fairly accessible, will take a different line of attack, and will collect in one place what seem to me to be the most important points. (This is particularly for the sake of those who don't already see the risk-uncertainty distinction as obviously flawed.)
• Set the scene for a later post, which will discuss essentially the exact same questions, but in the context of moral uncertainty in particular (which does not appear to have been adequately done before).

What’s the distinction meant to mean?

Wikipedia captures the everyday usage of the terms uncertainty and risk:

Uncertainty [is the ] lack of certainty, a state of limited knowledge where it is impossible to exactly describe the existing state, a future outcome, or more than one possible outcome.

[...] Risk [is a] state of uncertainty where some possible outcomes have an undesired effect or significant loss.

However, among some people and in some fields (particularly business and finance), it’s common to make a quite different risk-uncertainty distinction, like the one made in the quote at the beginning of this post. This different risk-uncertainty distinction (which is the one this post will focus on) is not about whether we’re talking about the possibility of something negative.

Instead, this distinction centres on something like whether we can “have”, “know”, “express”, “estimate”, or “quantify” the probabilities of interest (or perhaps, more specifically, “believable”, “justifiable”, or “precise” probabilities). If we can, we’re facing risk (even if the potential outcomes are all positive). If we can’t, we’re facing uncertainty (or Knightian uncertainty).^[2]

Here’s one way of explaining this risk-uncertainty distinction:

According to a seminal classification by Knight (1921), three types of uncertainty can be distinguished. In some situations, we can calculate the objective probabilities of events a priori, for instance, the chances of a (fair) die showing a six. In other cases, probabilities can be estimated statistically from data, for instance, the chances of being wrongly diagnosed [with] cancer when participating in regular screenings. Both cases are referred to as situations of risk.

Finally, there are situations with so many unique features that they can hardly be grouped with similar cases, such as the danger resulting from a new type of virus, or the consequences of military intervention in conflict areas. These represent cases of (Knightian) uncertainty where no data are available to estimate objective probabilities. While we may rely on our subjective estimates under such conditions, no objective basis exists by which to judge them (e.g., LeRoy & Singell, 1987). However, even when information is in principle attainable and chances could be estimated, an individual agent or agency may still lack the time or resources to do so—and thus face a situation of uncertainty. (bolding and line break added)

Here’s another similar explanation:

In the case of risk, the outcome is unknown, but the probability distribution governing that outcome is known. Uncertainty, on the other hand, is characterised by both an unknown outcome and an unknown probability distribution. For risk, these chances are taken to be objective, whereas for uncertainty, they are subjective.

Consider betting with a friend by rolling a die. If one rolls at least a four, one wins 30 Euros [...]. If one rolls lower, one loses. If the die is unbiased, one’s decision to accept the bet is taken with the knowledge that one has a 50 per cent chance of winning and losing. This situation is characterised by risk. However, if the die has an unknown bias, the situation is characterised by uncertainty.

The latter applies to all situations in which one knows that there is a chance of winning and losing but has no information on the exact distribution of these chances. (line break added)

And here’s another explanation from Roser:

In a locus classicus, Luce and Raiffa (1957, p. 13) distinguish risk and uncertainty according to whether probabilities are known or completely unknown or not even meaningful.

Does the distinction make sense?

What does it mean for a probability to be “known”? What does it mean for a probability to be “completely unknown”, or “not even meaningful”? Can we find a clear, sharp way of separating all probabilities into just the two categories of (a) those we can “know” and (b) those which must remain “completely unknown”?

As far as I can tell, the answer to that last question has to be “No.” This is essentially based on the following premises:

• P1: The answer being “Yes” would require it being the case that:

  • P1a) There are some probabilities that we can absolutely certainly know, and/or
  • P1b) There are some probabilities that we can have absolutely no knowledge about (and for which we can’t even use something like an uninformative prior).

• P2: P1a is false, because we can never validly be (or at least never should be) absolutely certain [LW · GW] of anything.

• P3: P1b is false, because we essentially always have at least some, incredibly flimsy basis for coming to a probability estimate for something, or, failing that, can use some type of uninformative prior. (I’m less confident in this premise than in the previous ones, and also less sure how to phrase what I mean.)

I can’t offer a proof of these premises (note: this doesn’t mean that a proof is impossible, just that I don’t know of one). Instead, what I’ll do below is try to illustrate why I believe those premises by:

• returning to examples that one of the authors quoted previously gave specifically for the purpose of highlighting the claimed distinction between risk and uncertainty
• arguing that these examples do not in fact show cases in which we can absolutely certainly know some probabilities and have absolutely no knowledge about others.

I claim that, in the absence of an alternative good example that does demonstrate P1a or P1b, this provides fairly strong evidence for P2 and P3, at least.

Certain knowledge of a probability?

In some situations, we can calculate the objective probabilities of events a priori, for instance, the chances of a (fair) die showing a six. In other cases, probabilities can be estimated statistically from data, for instance, the chances of being wrongly diagnosed [with] cancer when participating in regular screenings. Both cases are referred to as situations of risk. (source)

But how do we actually know that that’s a fair die? What if someone swapped it out at the last minute? And, even if it is what we’d typically call a “fair die”, how do we know that that means the odds it’ll show a six are ⅙? What if it’s become slightly eroded - entirely accidentally - such that it has a slightly higher chance of showing a six than showing another number? What if the person throwing the die knows how to throw it such as to increase or decrease the odds it’ll land on six?

Or as Yudkowsky’s flair for the dramatic puts it [LW · GW]:

The die might land on its edge; or get struck by a meteor; or the Dark Lords of the Matrix might reach in and write “37” on one side.

None of this stops me from happily believing that the odds are ⅙ that a die I have “very strong reason to believe” is “fair” will show a six. Nor will it stop me making bets on that basis. But it seems to me to highlight that there’s at least some doubt about what probability I should assign, and thus that this example doesn’t demonstrate P1a (i.e., doesn’t demonstrate that there are any probabilities I should be absolutely certain about).

Similar arguments could be run against the idea that the chances of being wrongly diagnosed with cancer can be absolutely, certainly known. (On top of the obvious possibilities like methodological errors in relevant studies, it’s also possible that the person’s very concept of cancer itself doesn’t line up well with reality or with the concept used by the relevant studies.)

Zero knowledge of a probability?

But might P1b be true - might there be probabilities we can have absolutely no knowledge about (and for which we can’t even use something like an uninformative prior)? I don’t believe I’ve ever encountered an example of such a probability. To return to the example given earlier:

there are situations with so many unique features that they can hardly be grouped with similar cases, such as the danger resulting from a new type of virus, or the consequences of military intervention in conflict areas. These represent cases of (Knightian) uncertainty where no data are available to estimate objective probabilities. While we may rely on our subjective estimates under such conditions, no objective basis exists by which to judge them (e.g., LeRoy & Singell, 1987). (source)

But are those situations really absolutely unique? Do we really have no (relevant) data available? What about previous viruses? What about previous military interventions? Of course situations may be very, very different in the case at hand, and that data may be of very little relevance, barely narrow things down at all, and still leave your guesses as very likely to be quite inaccurate. But isn’t it something?

To see that some data we have is at least slightly relevant, consider your reaction if I told you that the number of deaths from this new virus or new military intervention (which will occur sometime in the next 20 years) would be somewhere between 0 and 1 million, rather than somewhere between 1 trillion and 1 trillion & 1 million (i.e., more than the entire population of Earth). You’d be confident saying which of those is more likely, wouldn’t you?

And I think we’d both agree that that’s not just overconfidence - you do have legitimate reasons for your judgements there.

In fact, Tetlock’s work has empirically shown that it is possible to reliably do better than chance (and better than just “between 0 and 1 million”) in predicting events very much like those, at least over spans of a few years.^[3]

As with my rejection of P1a, what I’ve said is merely suggestive of P1b’s falsity - it’s still possible that there are some probabilities we can have absolutely no knowledge of. But I’ve seen various claimed examples of situations of (Knightian) uncertainty, and none have seemed to be ones in which we can have absolutely no knowledge of the probability. (Further discussion here [LW · GW].) And I suspect, though here I’m quite unsure, that if there was a good example, it’d still be possible to use something like an uninformative prior to deal with it, in the absence of anything better.

Altogether, I find myself fairly confident that Premises 1-3, or statements sufficiently like them, are true, and thus that it does not make sense to speak of an absolute, binary distinction between “risk” and “uncertainty”.

But why does that matter anyway? The next section discusses some decision-making procedures that people have proposed based on the idea that there is a risk-uncertainty distinction, and why these procedures seem strange and suboptimal.

Decision-making given this distinction

Many people argue that that maximising expected utility, or even just using expected value reasoning or explicit probabilities (EP) at all, is impossible or inappropriate when facing (Knightian) uncertainty rather than risk.^[4][5] As Roser writes: “According to a popular view, then, how we ought to make policy decisions depends crucially on whether we have probabilities.” Roser goes on to discuss three existing proposals for alternatives to standard usage of EPs when in situations of “uncertainty”, and why he rejects them; this is what I turn to now.

Principle of indifference

Roser:

The Principle of Indifference The first principle (also called the Principle of Insufficient Reason) suggests that if we have no probabilities, we ought to assign equal probabilities to all possible outcomes. We can then decide on the basis of these fabricated probabilities. Many of us tacitly and informally use this principle in everyday life. But it is also explicitly relied upon in the scientific literature. For example, in an article published in Science, Wigley and Raper (2001, p. 452) assume various emission scenarios to be equally likely on account of the fact that the IPCC’s third assessment report refused to make probability statements in this respect.

However, there is a widely acknowledged problem with the Principle of Indifference: there is often no natural way to individuate the consequences which one should treat as equally probable. [See Roser's paper for details.]

Maximin

Roser:

Maximin While the first strategy for decision-making without probabilities suggests making up probabilities, the second and most prominent strategy goes a different way. The Maximin Principle only characterizes consequences of policies as possible or impossible rather than as more or less probable. It proposes to rank policies by their worst possible consequence and to choose the policy with the least bad worst consequence.

The problem with Maximin lies in determining the worst possible consequence. There are various notions of possibility such as epistemic, logical, physical, or metaphysical possibility. According to any of these, the worst possible consequence of any climate policy currently on the table would seem to include the extinction of humanity or even worse catastrophes. This is hardly helpful for choosing among them.

The literature therefore suggests that we should not rely on a literal understanding of ‘‘possible’’ when espousing Maximin [cf. for example Gardiner (2006, p. 51) or Resnik (2003, pp. 336–337)]. Rather, if Maximin is to convince at all, the worst possible consequence must be understood as, say, the worst ‘‘realistic’’ consequence or the worst ‘‘plausible’’ consequence [which in turn could for example translate into us understanding the mechanism by which it could occur (cf. Shue 2010, p. 148)]. However, this brings up a problem. What does it mean for a consequence to be realistic? A consequence must count as realistic if the available evidence gives it at least some minimal support. Judging a consequence to have a certain minimal level of evidential support, however, precisely means judging it to have a certain minimal probability. Thus, realism judgements are nothing but probability judgements. ‘‘Realistic’’ is just another word for ‘‘not highly improbable’’.

This undermines the plausibility of Maximin as a decision-principle in the absence of probabilities, as it implies an availability of probabilities. If our evidence is such as to allow for a judgement about the realistic range of consequences, this same evidence surely allows for at least some probability judgements within and beyond that range. Maximin, however, suggests to arbitrarily limit our attention to the probability judgements needed for delineating the realistic range and not to make use of any probabilities within or beyond that range.

See Roser’s paper for what I see as a convincing example of the sort of problems maximin can lead to. Further discussion of why maximin, or something quite similar, seems a flawed approach can be found in this [LW · GW] series [LW · GW] of [LW · GW] posts [LW · GW] by Soares.^[6]

Avoiding a lack of probabilities

Roser:

Avoiding a Lack of Probabilities A third option for choosing policies in the absence of probabilities consists in limiting the menu to those options which allow for probability assignments. [...] This line of thinking is familiar from Ellsberg (1961). Consider the following scenario:

Scenario A: You must choose between drawing a lottery ticket (i) from an urn with 50% winning tickets or (ii) from an urn with an unknown proportion of winning and losing tickets.

In this scenario many would choose from the first urn. This is hardly irrational. However, assume a modified scenario:

Scenario B: You must choose between drawing a lottery ticket (i) from an urn with 1% winning tickets or (ii) from an urn with an unknown proportion of winning and losing tickets.

In that case, it is less clear what choice would be rational. The Principle of Avoiding a Lack of Probabilities would still recommend drawing a ticket from the first urn. But we might instinctively resist this (probably on the basis of an unjustified application of the Principle of Indifference). The problem is that it seems hopeless to justify our intuitive choices in either scenario A or B. Even though we are drawn to decide in line with the Principle of Avoiding a Lack of Probabilities in some cases (such as scenario A), a reason to support this intuition is not forthcoming.

Further discussion of situations like Scenarios A and B, and of the ambiguity aversion involved, can be found in the abovementioned series of posts from Soares.

A quote from Bostrom and Ćirković may further highlight the lack of rationale for, and potential scale of harms from, following the principle of Avoiding a Lack of Probabilities:

Although more rigorous methods are to be preferred whenever they are available and applicable, it would be misplaced scientism to confine attention to those risks that are amenable to hard approaches. Such a strategy would lead to many risks being ignored, including many of the largest risks confronting humanity. It would also create a false dichotomy between two types of risks – the ‘scientific’ ones and the ‘speculative’ ones – where, in reality, there is a continuum of analytic tractability.

Do we have a choice?

As Roser notes, despite all these issues, if there was an absolute, binary risk-uncertainty distinction, it might be preferable - or necessary - to use one of these three principles rather than a standard usage of EPs. However, as discussed above, it seems that there isn’t such a distinction. Thus, we do have a choice in the matter.

So it seems that, at least in the case of an ideal agent, it would be best to use whatever probabilities we do have, even if they’re incredibly poorly grounded, as they’d still be better than nothing. Roser writes:

It is difficult to see how foregoing whatever epistemic access to the features of the options at stake we have could lead to better policy decisions than making use of it. This would be comparable to a visually impaired pilot closing her eyes on account of the impairment rather than using whatever impressions she can glean from her limited eyesight. Discarding low-credentials-probabilities amounts to wasting information. [See here [LW(p) · GW(p)] for what Roser means by “low-credentials-probabilities”.]

My next post [EA · GW] will discuss the more complicated matter of to what extent that argument applies in practice, for actual humans, given issues like the time costs involved in using EPs and tendencies towards overconfidence and anchoring.

Unknown unknowns

Here’s one last statement of the risk-uncertainty distinction (this one from Holden Karnofsky):

"Risk" refers to the probabilities seen in well-understood and robust models, [...] such that one can be confident in what would happen over a large number of trials but not over a small number. "Uncertainty" refers to fuzzy, poorly modeled lack of understanding (or to "missing pieces of one's model")[.]

This quote makes more explicit the idea that (Knightian) uncertainty could be understood as including unknown unknowns (“missing pieces of one’s model”), rather than just “unknown” probabilities for the things one is modelling (which can be considered “known unknowns”). It could also be understood as including the related idea of “black swans”.

These concepts are sometimes ignored, sometimes implicit, and sometimes explicit in discussions of a risk-uncertainty distinction. Here I’ll explain how I think they fit into the picture. This is the section of this post where I’m least confident about both how true my beliefs are and how clearly I explain those beliefs.

I think unknown unknowns that influence the likelihood of whatever proposition we’re trying to work out the probability of are a key and common reason why the probability estimate we arrive at may be quite poorly grounded, flimsy, untrustworthy, etc. I think they’re also a key and common reason why the situation may look to some like a situation of Knightian uncertainty. But this doesn’t require fundamentally different approaches to creating or interpreting of one’s probability estimates. Soares [LW · GW] gives a useful example:

Consider the event "there is a cure for Alzheimer's disease 70 years from now".

As an aspiring Bayesian, I can assign a credence to this event. But as a denizen in a world of chaos, I can also expect black swan events. Dealing with the black swans doesn't require any new type of probability, though: I can account for it within the Bayesian framework.

Soares notes that black swans could include possibilities like “Within 70 years, human civilization will have collapsed”, but that obviously that particular example is no longer a black swan for us, as we’re now considering it. The black swans are those events we haven’t even thought of. He goes on:

I don't know where to expect black swans, but I can ask questions like "how will the average black swan affect Alzheimer's cures?". If I expect that most black swans will make Alzheimer's cures easier to achieve, then I adjust my credence upwards. If I expect the opposite, then I adjust my credence downwards.

And if I expect that I have absolutely no idea what the black swans will look like but also have no reason to believe black swans will make this event any more or less likely, then even though I won't adjust my credence further, I can still increase the variance of my distribution over my future credence for this event.

In other words, even if my current credence is 50% I can still expect that in 35 years (after encountering a black swan or two) my credence will be very different. This has the effect of making me act uncertain about my current credence, allowing me to say "my credence for this is 50%" without much confidence. So long as I can't predict the direction of the update, this is consistent Bayesian reasoning.

As a bounded Bayesian, I have all the behaviors recommended by those advocating Knightian uncertainty. I put high value on increasing my hypothesis space, and I often expect that a hypothesis will come out of left field and throw off my predictions. I'm happy to increase my error bars, and I often expect my credences to vary wildly over time. But I do all of this within a Bayesian framework, with no need for exotic "immeasurable" uncertainty.

But what if the unknown unknown doesn’t just influence what we’re trying to get a probability estimate for, but instead it is what we’re trying to get a probability estimate for? Soares discusses this too:

we are bounded reasoners, and we usually can't consider all available hypotheses. We can't expect to generate even the top ten most likely hypotheses, no matter how long we have to brainstorm. [...]

This phenomena is worrisome when attempting to predict the future, but we don't need a new kind of uncertainty to deal with the failure mode. In fact, this failure mode is nothing but a description of one of the limitations of a bounded Bayesian reasoner.

Bounded Bayesian reasoners should expect that they don't have access to the full hypothesis space. Bounded Bayesian reasoners can expect that their first-order predictions are incorrect due to a want of the right hypothesis, and thus place high credence on "something I haven't thought of", and place high value on new information or other actions that expand their hypothesis space.

That argument seems fairly sound to me. But parts of MIRI’s more recent writings on embedded agents [? · GW] being smaller than their world models have seemed to me to indicate that Soares' suggestions may be insufficient. But even if that’s the case, I think that's a separate problem, rather than something that reveals that there's a binary risk-uncertainty distinction. Essentially, I think it'd only show that one can't have a probability estimate for something one hasn't thought of, not that there are some propositions or evidence bases that, by their nature, fundamentally allow no probability estimates (even when one is looking at the proposition and trying to come up with an estimate).

(But again, I’m less sure of both my thinking and explanation on this, and think it’s somewhat tangential to the risk-uncertainty distinction, so I’ll leave that there.)

Closing remarks

In this post, I’ve:

• outlined what the risk-uncertainty distinction is meant to mean
• argued that no absolute, binary risk-uncertainty distinction exists
• argued that imagining that there is such a distinction can lead to strange and suboptimal decision-making procedures

But, practically speaking, for humans, could there be benefits to acting as if one believes there is such a distinction? I’ll cover this question in my next post [EA · GW] (ultimately arguing that the answer is probably “No”).

---

1. Additionally, it’s sometimes unclear whether proponents of the distinction are merely arguing (a) that people perceive such a distinction, so it’s useful to think about and research it in order to understand how people are likely to think and behave, or are actually arguing (b) that people should perceive such a distinction, or that such a distinction “really exists”, “out there in the world”. It seems to me that (a) is pretty likely to be true, but wouldn’t have major consequences for how we rationally should make decisions when not certain. Thus, in this post I focus exclusively on (b). ↩︎

2. It seems unfortunate that this distinction uses familiar words in a way that’s very different from their familiar usage. As Gooen writes, after introducing these less typical meanings of risk and uncertainty:

   Disagreeing with these definitions are common dictionaries and large parts of science and mathematics. In the Merriam-Webster dictionary, every definition of ‘risk’ is explicitly about possible negative events, not about general things with probability distributions. (https://www.merriam-webster.com/dictionary/risk)

   There is even a science explicitly called “uncertainty quantification”, but none explicitly called “risk quantification”.

   This is obviously something of a mess. Some business people get confused with mathematical quantifications of uncertainty, but other people would be confused by quantifications of socially positive “risks”.

   Roser further emphasises similar points:

   Not only in the past, but also today there is no universally accepted distinction between risk and uncertainty based on whether we have probabilities or not. This holds, of course, for everyday discourse which does not adhere to the technical distinction. But even in scientific and philosophical contexts, only some authors subscribe to the convention to use the terms risk and uncertainty in this specific way. For example, while the Intergovernmental Panel on Climate Change (IPCC) (2014, p. 155) draws a distinction between risk and uncertainty, it is completely different from the distinction at stake here. For instance, the IPCC takes probability density functions to be a common tool for characterizing uncertainty. One of the most famous microeconomics textbooks discusses decisions on the basis of probabilities under the heading of ‘‘Choice under Uncertainty’’ (Mas-Colell et al. 1995, ch. 6). More generally, there has been a long stream of economists who have dismissed the distinction and labelled it ‘‘sterile’’ or not ‘‘valid’’ (Hirshleifer and Riley 1992, p. 10; Friedman 1976, p. 282).

   ↩︎

3. Tetlock: “The best forecasters [...] we find are able to make between 10 and 15 distinguished… between 10 and 15 degrees of uncertainty for the types of questions that IARPA is asking about in these tournaments like whether Brexit is going to occur or if Greece is going to leave the eurozone or what Russia is going to do in the Crimea, those sorts of things. Now, that’s really interesting because a lot of people when they look at those questions say, “Well you can’t make probability judgements at all about that sort of thing because they’re unique.”

   And I think that’s probably one of the most interesting results of the work over the last 10 years. I mean, you take that objection, which you hear repeatedly from extremely smart people that these events are unique and you can’t put probabilities on them, you take that objection and you say, “Okay, let’s take all the events that the smart people say are unique and let’s put them in a set and let’s call that set allegedly unique events. Now let’s see if people can make forecasts within that set of allegedly unique events and if they can, if they can make meaningful probability judgments of these allegedly unique events, maybe the allegedly unique events aren’t so unique after all, maybe there is some recurrence component.” And that is indeed the finding that when you take the set of allegedly unique events, hundreds of allegedly unique events, you find that the best forecasters make pretty well calibrated forecasts fairly reliably over time and don’t regress too much toward the mean.” ↩︎

4. Note that one doesn’t necessarily have to accept maximising expected utility if one rejects the binary risk-uncertainty distinction. (E.g., one might choose a modification of maximisation of expected utility to avoid the issue of Pascal’s mugging.) Roser writes:

   while I believe that we always have probabilities, this paper refrains from taking a stance on how we ought to decide on the basis of these probabilities. The question whether we have probabilities is completely separate from the question how we ought to make use of them. Here, I only ask the former question. The two issues are often not kept separate: the camp that is in favour of relying on probabilities is often associated with processing them in line with expected utility theory. I myself am in favour of relying on probabilities but I reject expected utility theory (and related stances such as cost-benefit analysis), at least if it comes as a formal way of spelling out a maximizing consequentialist moral stance which does not properly incorporate rights.

   By the same token, the question of whether there’s a binary risk-uncertainty distinction is relevant even if one rejects maximisation or expected utility, as Roser also notes: “A lack of probabilities is not only a challenge for expected utility theory—which is known for its need for probabilities—but for all practical reasoning.” ↩︎

5. For example, Andreas Mogensen writes:

   In section 2.2, I reject what I take to be a naïve but tempting response [to the problem of cluelessness]. On this view, we needn’t worry about our ignorance of the indirect effects of our actions, because ignorance of this kind is compatible with expected utility maximization. This underestimates the depth of the problem. Using the jargon proposed by Knight (1921), I suggest that we are dealing not with risk, but uncertainty: ignorance of a kind so deep that our evidence does not warrant assigning precise probabilities to all relevant contingencies. Under Knightian uncertainty, the doxastic attitudes of an agent who proportions her beliefs to the evidence cannot be represented by a unique probability function relative to which an expected utility for each act may be defined.

   Indeed, it seems to me that the whole idea of cluelessness is premised on the idea of an absolute, binary risk-uncertainty distinction, or on something that’s very much like that distinction and that suffers from the same problems as that distinction does. I therefore hope to later write a post attempting to “dissolve [LW · GW]” the problem of cluelessness, using arguments quite similar to the above. For now, part of my thoughts on the matter can be found in two comments here [EA(p) · GW(p)], or extrapolated from what I say in this post. ↩︎

6. In Soares’ posts, he refers to what he’s critiquing as “maximiz[ing] minimum expected utility given [one’s] Knightian uncertainty”. I believe that this is either the same as or very similar to maximin, but it’s possible I’m wrong about that. ↩︎

9 comments

Comments sorted by top scores.