What’s this probability you’re reporting?

post by Eric Chen (Equilibrate), Sami Petersen (sami-petersen) · 2023-04-14T15:07:42.844Z · LW · GW · 9 comments

Contents

    Deterministic
    Stochastic
    Ensemble
  Takeaways
None
9 comments

It’s unclear what people mean when saying they’re reporting a probability according to their inside view model(s). We’ll look through what this could mean and why most interpretations are problematic. Note that we’re not making claims about which communication norms are socially conducive to nice dialogue. We’re hoping to clarify some object-level claims about what kinds of probability assignments make sense, conceptually. These things might overlap.

Consider the following hypothetical exchange:

Person 1: “I assign 90% probability to X”

Person 2: “That’s such a confident view considering you might be wrong”

Person 1: “I’m reporting my inside view credence according to my model(s)”

This response looks coherent at first glance. But it’s unclear what Person 1 is actually saying. Here are three kinds of model(s) they could be referring to:

1. Deterministic: There is a model that describes the relevant parts of the world, some deterministic laws of motion, and therefore a description of how it will evolve through time. There are no probabilities involved.
2. Stochastic: There is a model that describes the relevant parts of the world, and the evolution of the model is stochastic. Model-based probabilities here correspond to precise statements about random variables within the model. 
3. Ensemble: There is a set of models, deterministic or stochastic, that describe the evolution of the world. You have some way of aggregating over these. E.g., if nine models say “X happens” and one says “X doesn’t happen”, you might assign $P\left(X\right)=90\%$ if you have a uniform prior over the ten models.

There are troubles with all of these.

Deterministic

Models are often deterministic. When an engineer says a bridge is “unlikely” to collapse, it’s not necessarily because their model outputs probabilities; it could simply be because they aren’t confident that the model fully captures everything relevant. A deterministic model will not have any probabilities associated with it. So if someone is using such a model to assign a credence to some proposition, the source of uncertainty has to come from outside the model.

Stochastic

In a stochastic model, model-based probabilities correspond to very precise statements about the random variables in the model and their distributions. These random variables tend to correspond to genuine indeterminism in the system or at least the best we can formalise at some level of description for highly complex or chaotic systems. Examples include exogenous shocks in DSGE macroeconomic models or the evolution of entropy in statistical mechanics. The choice of adding a stochastic component to a model is very particular; its justification is usually based on features of the system, not simply one’s uncertainty—that’s what overall credences are for. This is a subtle point in the philosophy of science, but in slogan form: stochastic components in models represent emergent indeterministic macrodynamics rather than credences.^[1]

Ensemble

You could claim that “model-based probabilities” are weighted averages of the outputs you get from several models. But this comes scarily close to your all things considered view. If not, then which models are you averaging over? What procedure are you using to choose them? How is it justified? If you chose the ten models you find most applicable, why is ten the magic number? Simply stating a probability that’s based on several models, but that decidedly isn’t your overall confidence, is quite uninformative. Most of the information plausibly comes from the choice of how many models you are including and how you weigh them. And if this is not based on your overall view, what is it? Stochastic models and Bayesian agents have clearly-defined probabilities but ensembles don’t. It’s unclear what people are using to distinguish an ensemble’s average from their all things considered view. If you’re doing this rather peculiar thing when reporting probabilities, it would be useful to know the procedure used.

---

The fundamental issue is this. Credences are unambiguous. For a Bayesian, the statement “I assign 90% probability to X” is perfectly well-defined. And we have a well-established epistemic infrastructure for handling credences. We know what to do with credences in models of decision-making (including betting),^[2] opinion aggregation,^[3] information transmission,^[4] peer disagreement,^[5] and more.^[6] We have powerful theorems showing how to use credences in such ways. In contrast, “model-based” or “inside view” probabilities do not clearly correspond to well-defined objects, neither in the abstract nor in people’s heads. (Deterministic? Stochastic? Which models? Which aggregation procedure?) As a result, there does not exist a corresponding infrastructure for handling the various objects that they could refer to.

As an aside, we believe the various possible disambiguations of model-based probabilities can in fact correspond to useful objects for decision-making. But to justify their usefulness, we need to depart pretty drastically from Bayesian orthodoxy and examine which decision-making heuristics are rational for bounded agents. These can be perfectly reasonable choices but require justification or at least some clarification about which are being used.^[7]

Takeaways

1. Credences are well-defined, have well-studied properties, and fit into a wider epistemic and decision-making infrastructure. Things like “the probability on my main models” or “the probabilities generated by my initial inside view impression” don't have these properties. They are ambiguous and may just lack a referent.
2. If your probability statements refer to something other than a credence, it is worthwhile to clarify precisely what this is to avoid ambiguity or incoherence.
3. If you are reporting numbers based on a particular heuristic decision-making approach, this is more messy, so extra care should be made to make this clear. Because this leaves the well-trodden path of Bayesian epistemology, you should have some reason for thinking these objects are useful. There is an emerging literature on this and we'd be quite excited to see more engagement with it.^[8]

1. ^{^}

   See Wallace (2012) Chapter 4.1 for a particularly lucid explanation.

2. ^{^}

   E.g., expected utility theory (Bradley 2017).

3. ^{^}

   E.g., Dietrich (2010).

4. ^{^}

   E.g, cheap talk (Crawford and Sobel 1982) and Bayesian persuasion (Kamenica and Gentzkow 2011).

5. ^{^}

   E.g., Aumann’s (1976) agreement theorem.

6. ^{^}

   Stochastic models are in a similar position in that we know how to handle them in principle. But we doubt that something like “my model has a mean-zero, variance ${\sigma }^2$ exogenous shock component” is what people mean by “model-based” or “inside view” probabilities.

7. ^{^}

   Very briefly, in the literature on decision-making under deep uncertainty (DMDU), the use of a small collection of models (roughly interpretations 1 and 2 above) corresponds to what is called scenario-based decision-making. And the use of a large ensemble of models (roughly interpretation 3) corresponds to a popular method developed by the RAND Corporation termed robust decision-making. See Thorstad (2022) for some reasons for thinking these are good heuristics for decision-making under severe or Knightian uncertainty. But the key part for now is that this is a very immature field compared to Bayesian epistemology, and so thinking in these terms should be done as clearly as possible.

8. ^{^}

   E.g., Roussos et al (2022) and Thorstad (2022).

9 comments

Comments sorted by top scores.