The Case for Frequentism: Why Bayesian Probability is Fundamentally Unsound and What Science Does Instead

post by lsusr · 2022-04-03T20:52:50.683Z · LW · GW · 29 comments

Contents

    Land War in Eurasia
    Bayes' Theorem
    Bayesian Chains
      Corollary: Irresolvable Prior Disagreement
    Stack Traces
    The Belief-Value Uncertainty Principle
    Summary
  Frequentism
  Reductionism
None
29 comments

Land War in Eurasia

Bayes theorem is a tool for updating probabilities according to evidence.

Consider a real-world example of something important I was recently wrong about. On February 23, I found out Russia had invaded Ukraine [LW · GW]. When disruptive events happen, you must respond immediately. I quickly read what the experts were saying on Foreign Affairs and combined it with my existing knowledge of history, technology and geopolitics. I wrote "There is going to be a war. Ukraine is going to lose. The question is how much, how quickly and on what terms."

Then I remembered that war is unpredictable [LW · GW] and war experts are often wrong. I added a qualifiers before publishing. "There is probably going to be a war. Ukraine is probably going to lose. The question is how much, how quickly and on what terms."

A week later, reports of under-equipped soldiers began trickling in and I discovered that Russia's economy was much smaller than I had assumed. I updated my analysis [LW · GW]. As the war calcified, I finally had time to research current weapons technology [LW · GW] and build my own model of the war from a tactics-level foundation.

Bayes' Theorem

Over the course of the war, several observations changed my probabilities of an outcome.

1. Experts believe Russia will win.
2. Evidence arrived that Russian troops might be under-equipped.
3. I discovered the Russian economy is the size of Florida's.
4. Yep. It's not just propaganda. Russian troops really are under-equipped. Video evidence confirms it.
5. Russia's assault has slowed to a crawl.

Each time new evidence came in, my analysis of the situation shifted away from a Russia blitzkreig-style fait accopli.

Bayesian probability treats probability as a subjective state that is updated in response to evidence.

• Let $A$ denote the a Russian fait accompli.
• Let $B$ denote the belief of experts that Russia will perform a fait accompli.

For example, my initial prior probability $P\left(A\right)$ of "Russia will perform a fait accompli" started out at 0.5^[1].

$$P\left(A\right)=0.5$$

When I discovered that experts believed Russia would win, I updated my estimate. I estimated a probability of 0.9 that experts would believe in a Russian victory (conditional upon a Russian victory) $P\left(B_1|A\right)$. I estimated the probability of experts to believe in a Russian victory $P\left(B_1\right)$ to be 0.5.

$$\begin{array}{ccc}P\left(B_1|A\right)& =& 0.9\\ P\left(B_1\right)& =& 0.5\end{array}$$

After observing experts believed in a Russian victory, I wanted to find out the probability $P\left(A|B_1\right)$ of a Russian victory conditional on experts believing in a Russian victory. Bayes' Theorem states that the probability $P\left(A|B_1\right)$ of $A$ given $B_1$ equals the prior probability $P\left(A\right)$ of $A$ multiplied by an update factor $\frac{P\left(B_1|A\right)}{P\left(B_1\right)}$.

$$\begin{array}{ccc}P\left(A|B_1\right)& =& P\left(A\right)\frac{P\left(B_1|A\right)}{P\left(B_1\right)}\\ & =& 0.5\frac{0.9}{0.5}\\ & =& 0.9\end{array}$$

Thus, I arrived at a probability 0.9 "Ukraine is probably going to lose. The question is how much, how quickly and on what terms."

Bayesian Chains

There are unconfirmed hints suggest that Russian troops might be under-equipped. Evidence $B_2$ is has arrived.

• My prior probability "Russia will perform a fait accompli conditional on the observed fact that experts have predicted a fait accompli" $P\left(A|B_1\right)$ is 0.9.
• My probability $P\left(B_2\right)$ of $B_2$ conditional on $P\left(B_2|B_1\right)$ is 0.2.
• We can use the Bayes equation to calculate new probability of Russian victory $P(A|B_1,B_2)$ conditional on $P\left(A|B_1\right)$.

$$\begin{array}{ccc}P(A|B_2,B_1)& =& P\left(A|B_1\right)\frac{P(B_2|A,B_1)}{P\left(B_2|B_1\right)}\\ & =& P\left(A\right)\frac{P\left(B_1|A\right)}{P\left(B_1\right)}\frac{P(B_2|A,B_1)}{P\left(B_2|B_1\right)}\end{array}$$

The precise numbers aren't important. What matters is how Bayesian probabilities are chained together. Each bit of evidence $B_i$ multiplies our prior probability by an update factor $U\left(B_i\right)=\frac{P\left(B_i|\text{priors}\right)}{P\left(B_i\right)}$ to get a new probability. Suppose we have a finite set of observations $\{B_1,B_2,\dots ,B_n\}$. We can chain them together.

$$P(A|B_1,B_2,\dots ,B_n)=P\left(A\right)\prod _{i=1}^{n}U\left(B_i\right)$$

But now we have a problem. For every $ϵ,{\prod }_{i=1}^{n}U\left(B_i\right),P(A|B_1,B_2,\dots ,B_n)>0$, there exists $P\left(A\right)>0$ such that $P(A|B_1,B_2,\dots ,B_n)<ϵ$. In other words, for any finite observable evidence, there exists a sufficiently strong prior such that we can ignore the evidence.

Sufficiently strong priors are unfalsifiable. All knowledge that isn't garbage is based on the concept of falsifiability. Reality is that which is falsifiable. If a statement isn't falsifiable then that statement isn't about reality.

Bayesian probability ultimately rests on an unfalsifiable conviction.

Corollary: Irresolvable Prior Disagreement

If two people disagree about $P\left(A\right)$ but agree about all $U\left(B_i\right)$ then it will be impossible for them to come into agreement because no $U\left(B_i\right)$ is ever allowed to equal zero. Finite evidence cannot resolve a priori disagreement.

Stack Traces

The most important question in rationalism is "Why do you believe what you believe?" Suppose you asked a Bayesian "Why do you believe $A|B_1$ with confidence $P\left(A|B_1\right)$? there are two ways a Bayesian can respond.

1. The Bayesian can claim $P\left(A|B_1\right)$ is a prior. That's not empiricism. That's just blind faith.
2. The Bayesian can use Bayes Theorem: I believe $A|B_1$ with confidence $P\left(A|B_1\right)$ because $P\left(A|B_1\right)=P\left(A\right)\frac{P\left(B_1|A\right)}{P\left(B_1\right)}$.

Option (1) is a claim of blind faith. Option (2) is three claims of blind faith. If you ask a Bayesian "Why do you believe $B_1$ with confidence $P\left(B_1\right)$?" the answer is either blind faith "it's a prior" or will be "Because I believe $B_{11},B_{12},B_{13}$." If you ask "Why do you believe $B_{11},B_{12},B_{13}$?" then the logical chain either never terminates or it terminates in blind faith.

That's not even the worst problem with Bayesian philosophy.

The Belief-Value Uncertainty Principle

Suppose you asked me "What is the sum of $2+2$?" Usually, I would answer "$4$" but if I were in the Ministry of Love's Room 101 and they were threatening me with torture then I would answer "$5$".

A person's beliefs are not observable. Only a person's behavior is observable. Bayesians believe that people believe things and then optimize the world according to a value function. But there is no way to separate the two because:

• For every observed behavior and every possible belief there exists a value function which could produce the behavior.
• For every observed behavior and every possible value function there exists a belief which could produce the behavior.

A person's beliefs and behavior are intimately related the way a quantum wave packet's position is related to its momentum. You cannot observe both of them independently and at the same time.

Summary

Bayesian Probability has the following problems.

• The answer to "Why do you believe $x$?" is always reducible to priors, which are non-falsifiable. Evidence has no effect on priors.
• Rational agents with wildly differing priors are (usually) unable to come into even approximate agreement when provided with scarce evidence.
• Rational agents who disagree about unconditional priors $P\left(A\right)$ but who agree about evidence likelihood $P\left(B_i\right)$ and conditional priors $P\left(A|B_i\right)$ should be able to come into agreement. Instead, Bayesians who disagree about unconditional priors $P\left(A\right)$ while agreeing about evidence likelihood $P\left(B_i\right)$ and conditional priors $P\left(A|B_i\right)$ are provably unable to ever reach exact agreement if they use Bayes' Theorem. This is the opposite of how empiricism should work.
• Identifying someone else's beliefs requires you to separate a person's value function from their beliefs, which is impossible.

Frequentism

Frequentism defines probability as the limit of an event's relative frequency across a large number of independent trials. Frequentist probability is objective because it defined in terms of falsifiable real-world observations. An objective definition of probability can be used to resolve disagreements between scientists. A subjective definition cannot. That's why Frequentist probability is used as the foundation of science and Bayesian probability isn't.

The purpose of Frequentist probability is to bound uncertainties. Suppose you observe 100 white swans and 0 black swans. The probability you will observe a black swan by repeating the same experiment is probably not orders of magnitude higher than $\frac{1}{100}$. Suppose the probability of observing a black swan is actually $\frac{1}{10}$. The probability of observing zero black swans in a sample of 100 swans is ${0.9}^{100}\approx 3\times {10}^{-5}$. That's tiny.

If you can run a large number of independent trials then it is easy to show with extremely high confidence that the frequency of an event is below some $ϵ>0$. Our Frequentist confidence in theories like electromagnetism [LW · GW] has so many nines 0.999999… that it rounds to 1.

There are three dangers to Frequentist probability:

• Sample reporting bias. Sample reporting bias is prevented via basic scientific hygiene such as pre-registering experiments.
• Long tails. Frequentist probability will get you killed if you apply it naïvly to a long-tailed anti-inductive system like financial markets. That's because Frequentist probability optimizes median (and other percentile) outcomes whereas long-tailed systems punish you according to average outcomes. Bhuntut has a cool argument [LW · GW] about how the log utility of average outcomes [LW · GW] is a consequence of optimizing for median (and other percentile) outcomes.
• Small data. [LW · GW] Frequentism defines probability in terms of a large number of independent trials. How do you construct probabilities when you don't have "a large number of independent trials"?

Reductionism

Imagine you want to build a space elevator. Space elevators are expensive. You cannot build a million different versions and observe which ones work. You must get it right the first time. How can you do this when Frequentist probability requires "a large number of independent trials" to be meaningful at all?

You silo the problem into modular components.

Material reduction is the art of breaking a physical system into small subsystems with bounded interdependence. A space elevator depends strongly on the tensile strength of its core material. Laboratory material science experiments are cheap compared to building space elevators. You can build small bits of carbon nanotube table on Earth, test its tensile strength and expose it to space-like radiation. Do this over and over again until you're confident in the integrity of your materials.

Break your problem into small components. Break those components into smaller components. Break those components into even smaller components. Do this until you get to the level of atoms and photons. (We know how to go even smaller than that but applying quantum field theory to an engineering challenge usually causes more problems then it solves.) If you're confident about how a material behaves then you can be confident how an object build out of that material behaves.

Material reduction allows you to leverage your high confidence about the behavior of small things (which you have lots of data about) to confidently predict the behavior of big things (which you have little data about).

Most importantly, it's falsifiable. If I believe the result of a small-scale experiment is $x$ and you think it's $\neg x$ then we can quickly be brought into agreement by just running the experiment.

---

1. The probabilities used in this article here are retconned for illustrative purposes only. I did not use precise numeric probabilities in my analysis at the time. ↩︎

29 comments

Comments sorted by top scores.