What's so special about likelihoods?

post by mfatt · 2024-08-10T01:07:58.592Z · LW · GW · 0 comments

Contents

No comments

This is a coin.

It might be biased.

This is Bayes’ theorem.

$P\left(A|B\right)=\frac{P\left(B|A\right)\times P\left(A\right)}{P\left(B\right)}$

Bayes’ theorem tells us how we ought to update our beliefs given evidence.

It involves the following components:

• $P\left(A|B\right)$, called the posterior; it is the probability of A given B. In the case of the coin this is the probability that the coin is biased given the result of an experiment (i.e., a sequence of flips).
• $P\left(B|A\right)$, called the likelihood, is the probability of B given A. For our coin example this would be the probability of some particular ratio of heads to tails, given that the coin is biased.
• $P\left(A\right)$ is called the prior. This is the probability that the coin is biased, before we consider any evidence.
• $P\left(B\right)$ is the marginal. This is probability of us getting some sequence of head and tails, before considering any evidence.

The overall shape of the theorem is this:

Posterior $\propto$ likelihood $\times$ prior. 

---

If you were explain this to a high-school student, they might ask this naïve question:

Why should we bother to go through the process of calculating the likelihood and prior at all? Why can’t we just try and directly calculate the posterior? We have a formula for $P\left(A|B\right)$, namely $\frac{P(A\cap B)}{P\left(B\right)}$.

Maybe you'll say "That formula is fine but not useful in real life. It's usually tractable to go via conditional updates rather than the high school definition."

But if conditionals are easy to get, why not just go directly to the posterior? What's even the difference between A and B? Aren't they just symbols? We could easily rearrange the theorem to calculate $P\left(B|A\right)$ as a function of $P\left(A|B\right)$.

What is it that makes using strings of coin flips to calculate biases more natural or scientific?

Perhaps it is ease. If it is the case that for some reason calculating $P\left(B|A\right)$ is easier, what makes it easier?

Perhaps it is usefulness. If likelihoods are what's worth publishing, not posteriors, why are they worthier?

How do you spot a likelihood in the wild?

0 comments

Comments sorted by top scores.