About probabilities

Ah yes this was confusing to me for a while too, glad to be able to help someone else out with it!

The key thing to realise for me, is that the probability of 21 heads in a row changes as you toss each of those 21 coins.

The sequence of 21 heads in a row does indeed have much less than 0.5 chance, to be precise ${0.5}^{21}$,   which is 0.000000476837158.  But it only has such a tiny probability before any of those 21 coins have been tossed. However as soon as the first coin is tossed, the probability of those 21 coins all being heads changes. If first coin is tails, the probability of all 21 coins being heads goes down to 0, if first coin is heads the probability of all 21 coins being heads goes up to ${0.5}^{20}$. Say you by unlikely luck keep tossing heads. Then with each additional heads in a row you toss, the probability of all 21 coins being heads goes steadily up and up, til by the time you've tossed 20 heads in a row, the probability of all 21 being heads is now.... 0.5, i.e. the same as a the probability of a single coin toss being heads! And our apparent contradition is gone :)

The more 'mathematical' way to express this would be: The unconditional probability of tossing 21 heads in a row is ${0.5}^{21}$, i.e. 0.000000476837158 but the probability of tossing 21 heads in a row conditional on having already tossed 20 heads in a row is $0.5$.

$P\left(\text{21 heads}\right)={0.5}^{21}=0.000000476837158$

$P\left(\text{21 heads}|\text{20 heads}\right)=0.5$

Let me know if any of that is still confusing.

 

A sequence of 100 heads is only half as likely as a sequence of 99 heads. Which is why the probability of the 100th coinflip being head is exactly one half.

One way to think of this: Uncertainty (at least on this level) is in the observer, not the coin. It comes up heads or comes up tails, with 100% chance of the thing that actually happens.

Before the flip, you assign 50% to each outcome, but that’s your uncertainty, not the coin’s, and the result may as well be secretly predetermined by the universe. After you’ve seen 20 heads, that part is now probability 100% (it’s knowledge, not uncertainty, on your part), and the next flip is still 50/50 (to your knowledge, presuming you have reason to trust the coin and not update toward an unfair flipper).

Are there 2 (or more) types of probabilities and I am just mixing them up

Yes, there are conditional probabilities and unconditional probabilities.

The unconditional probability of 21 heads in a row is 0.5^21^[1].
The conditional probability of 21 heads in a row given that the first 20 were all heads is 0.5.

Conditional probability is just a division: the conditional probability of some event A given that B happened is just the unconditional probability of both A and B divided by the probability of B. In symbols: P(A | B) = P(A & B) / P(B).

Bayes' Law comes from simple algebra on this.

1. ^{^}

   As is common, this assumes that the coin flips are independent of one another. An alternative might be that the coin was flipped "lazily" such that it more often shows the same face as the previous flip, but over the long run still flips 50% heads. A "properly" flipped coin should not depend upon the results of any or all previous flips.