Probability is in the Mind

It seems to me you're using "perceived probability" and "probability" interchangeably. That is, you're "defining" probability as the probability that an observer assigns based on certain pieces of information. Is it not true that when one rolls a fair 1d6, there is an actual 1/6 probability of getting any one specific value? Or using your biased coin example: our information may tell us to assume a 50/50 chance, but the man may be correct in saying that the coin has a bias--that is, the coin may really come up heads 80% of the time, but we must assume a 50% chance to make the decision, until we can be certain of the 80% chance ourselves. What am I missing? I would say that the Gomboc (http://tinyurl.com/2rffxs) has a 100% chance of righting itself, inherently. I do not understand how this is incorrect.

GBM:

Q: What is the probability for a pseudo-random number generator to generate a specific number as his next output?

A: 1 or 0 because you can actually calculate the next number if you have the available information.

Q: What probability do you assign to a specific number as being it's next output if you don't have the information to calculate it?

Replace pseudo-random number generator with dice and repeat.

So therefore a person with perfect knowledge would not need probability. Is this another interpretation of "God does not play dice?" :-)

The Bayesian says, "Uncertainty exists in the map, not in the territory. In the real world, the coin has either come up heads, or come up tails."

Alas, the coin was part of an erroneous stamping, and is blank on both sides.

In other words, probability is not likelihood.

Here is another example me, my dad and my brother came up with when we were discussing probability.

Suppose there are 4 card, an ace and 3 kings. They are shuffled and placed face side down. I didn't look at the cards, my dad looked at the first card, my brother looked at the first and second cards. What is the probability of the ace being one of the last 2 cards. For me: 1/2 For my dad: If he saw the ace it is 0, otherwise 2/3. For my brother: If he saw the ace it is 0, otherwise 1.

How can there be different probabilities of the same event? It is because probability is something in the mind calculated because of imperfect knowledge. It is not a property of reality. Reality will take only a single path. We just don't know what that path is. It is pointless to ask for "the real likelihood" of an event. The likelihood depends on how much information you have. If you had all the information, the likelihood of the event would be 100% or 0%.

The competent frequentist would presumably not be befuddled by these supposed paradoxes. Since he would not be befuddled (or so I am fairly certain), the "paradoxes" fail to prove the superiority of the Bayesian approach. Frankly, the treatment of these "paradoxes" in terms of repeated experiments seems to straightforward that I don't know how you can possibly think there's a problem.

"Probabilities express uncertainty, and it is only agents who can be uncertain. A blank map does not correspond to a blank territory. Ignorance is in the mind."

Eliezer, in quantum mechanics, one does not say that one does not have knowledge of both position and momentum of a particle simultaneously. Rather, one says that one CANNOT have such knowledge. This contradicts your statement that ignorance is in the mind. If quantum mechanics is true, then ignorance/uncertainty is a part of nature and not just something that agents have.

Constant: The competent frequentist would presumably not be befuddled by these supposed paradoxes.

Not the last two paradoxes, no. But the first case given, the biased coin whose bias is not known, is indeed a classic example of the difference between Bayesians and frequentists. The frequentist says:

"The coin's bias is not a random variable! It's a fixed fact! If you repeat the experiment, it won't come out to a 0.5 long-run frequency of heads!" (Likewise when the fact to be determined is the speed of light, or whatever.) "If you flip the coin 10 times, I can make a statement about the probability that the observed ratio will be within some given distance of the inherent propensity, but to say that the coin has a 50% probability of turning up heads on the first occasion is nonsense - that's just not the real probability, which is unknown."

According to the frequentist, apparently there is no rational way to manage your uncertainty about a single flip of a coin of unknown bias, since whatever you do, someone else will be able to criticize your belief as "subjective" - such a devastating criticism that you may as well, um, flip a coin. Or consult a magic 8-ball.

Sudeep: If quantum mechanics is true, then ignorance/uncertainty is a part of nature and not just something that agents have.

A common misconception - Jaynes railed against that idea too, and he wasn't even equipped with the modern understanding of decoherence. In quantum mechanics, it's an objective fact that the blobs of amplitude making up reality sometimes split in two, and you can't predict what "you" will see, when that happens, because it is an objective fact that different versions of you will see different things. But all this is completely mechanical, causal, and deterministic - the splitting of observers just introduces an element of anthropic pseudo-uncertainty, if you happen to be one of those observers. The splitting is not inherently related to the act of measurement by a conscious agent, or any kind of agent; it happens just as much when a system is "measured" by a photon bouncing off and interacting with a rock.

There are other interpretations of quantum mechanics, but they don't make any sense. Making this fully clear will require more prerequisite posts first, though.

Maybe I'm stupid here... what difference does it make?

Sure, if we had a coin-flip-predicting robot with quick eyes it might be able to guess right/predict the outcome 90% of the time. And if we were precognitive we could clean up at Vegas.

In terms of non-hypothetical real decisions that confront people, what is the outcome of this line of reasoning? What do you suggest people do differently and in what context? Mark cards?

B/c currently, as far as I can see, you're saying, "The coin won't end up 'heads or tails' -- it'll end up heads, or it'll end up tails." True but uninformative.

Conrad.

ps - The thought experiment with the trick coin is ungrounded. If I'm being asked to lay even odds on a dollar bet that the coin is heads, then that's rational -- since the coin could be biased for heads, or tails (and the guy proposing the bet doesn't know the bias). If I'm being asked to accept or reject a number meant to correspond to the calculated or measured likelihood of heads coming up, and I trust the information about it being biased, then the only correct move is to reject the 0.5 probability. It has nothing to do with frequentist, Bayesian, or any other suchlike.

C.

Sudeep: the inverse certainy of the position and momentum is a mathematical artifact and does not depend upon the validity of quantum mechanics. (Er, at least to the extent that math is independent of the external world!)

PK: I like your posts, and don't take this the wrong way, but, to me, your example doesn't have as much shocking unintuitiveness as the ones Eliezer Yudkowsky (no underscore) listed.

I'd like to understand: Are frequentist "probability" and subjective "probability" simply two different concepts, to be distinguished carefully? Or is there some true debate here?

I think that Jaynes shows a derivation follownig Bayesian principles of the frequentist probability from the subjective probability. I'd love to see one of Eliezer's lucid explanations on that.

You can derive frequentist probabilities from subjective probabilities but not the other way around.

Silas: My post wasn't meant to be "shockingly unintuitive", it was meant to illustrate Eliezer's point that probability is in the mind and not out there in reality in a ridiculously obvious way.

Am I somehow talking about something entirely different than what Eliezer was talking about? Or should I complexificationafize my vocabulary to seem more academic? English isn't my first language after all.

If I'm being asked to accept or reject a number meant to correspond to the calculated or measured likelihood of heads coming up, and I trust the information about it being biased, then the only correct move is to reject the 0.5 probability.

Alas, no. Here's the deal: implicit in all the coin toss toy problems is the idea that the observations may be modeled as exchangeable. It really really helps to have a grasp on what the math looks like when we assume exchangeability.

In models where (infinite) exchangeability is assumed, the concept of long-run frequency can be sensibly defined. (Long-run frequency may or may not be a cogent concept in models without exchangeability.) The probability of heads in any one toss is the expectation of a probability density function (pdf) which encodes our knowledge about the long run frequency. (Roughly. There are some technical conditions for the existence of a pdf that I'm ignoring.)

Conrad, your idea that 0.5 is not an allowable probability is almost correct. In fact, the correct expression of this idea is that the pdf of the long-run frequency must be equal to zero at 0.5. But! -- its values in the neighborhood of 0.5 are not constrained, so the pdf may have a removable singularity.

Suppose our information about bias in favour of heads is equivalent to our information about bias in favour of tail. Our pdf for the long-run frequency will be symmetrical about 0.5 and its expectation (which is the probability in any single toss) must also be 0.5. It is quite possible for an expectation to take a value which has zero probability density. We can refuse to believe that the long-run frequency will converge to exactly 0.5 while simultaneously holding a probability of 0.5 for any specific single toss in isolation.

Eliezer, I have no argument with the Bayesian use of the probability calculus and so I do not side with those who say "there is no rational way to manage your uncertainty", but I think I probably do have an argument with the insistence that it is the one true way. None of the problems you have so far outlined, including the coin one, really seem to doom either frequentism specifically, or more generally, an objective account of probability. I agree with this:

Even before a fair coin is tossed, the notion that it has an inherent 50% probability of coming up heads may be just plain wrong. Maybe you're holding the coin in such a way that it's just about guaranteed to come up heads, or tails, given the force at which you flip it, and the air currents around you.

but I question whether it really captures the frequentist position. To address the specifics, you seem to be talking about how the coin is held in a specific concrete toss. But frequentists emphatically are not talking about individual tosses. They are talking about infinitely repeated tosses. Alternatively, you might be talking about an infinitely repeated experiment in which the coin is tossed "in such a way", but here too I see no problem for the frequentists. Since the way of holding the coin is part of the experiment, then in this case they will predict a long term frequency of mostly heads. So they won't get this one wrong.

(Replace the link to "removable singularity" with one to removable discontinuity.)

No way to do it other way around? Nothing along the lines of, say, considering a set of various "things to be explained" and for each a hypothesis explaining it, and then talk about subsets of those? ie, a subset in which 1/10 of the hypothesies in that subset are objectively true would be a set of hypothesies assigned .1 probability, or something?

Yeah, the notion of how to do this exactly is, admittedly, fuzzy in my head, but I have to say that it sure does seem like there ought to be some way to use the notion of frequentist probability to construct subjective probability along these lines.

I may be completely wrong though.

"Suppose our information about bias in favour of heads is equivalent to our information about bias in favour of tail. Our pdf for the long-run frequency will be symmetrical about 0.5 and its expectation (which is the probability in any single toss) must also be 0.5. It is quite possible for an expectation to take a value which has zero probability density."

What I said: if all you know is that it's a trick coin, you can lay even odds on heads.

"We can refuse to believe that the long-run frequency will converge to exactly 0.5 while simultaneously holding a probability of 0.5 for any specific single toss in isolation."

Again what I said: if the question is, "This is a trick coin: I've rigged it. I have written down here the probability that it'll come up heads. Do you accept that the number I've written down is .5?" -- You've got to say no. Since they've just told you it was rigged.

And if what they've written down is .50000000000001 and come back at you for it, then they stretched a point to say it was rigged.

So your problem is you haven't grounded the example in terms of what we're being asked to do.

Again, what difference does it make?

Conrad.

ps - Ofc, knowing, or even just suspecting, the coin is rigged, on the second throw you'd best bet on a repeat of the outcome of the first.

C.

But frequentists emphatically are not talking about individual tosses. They are talking about infinitely repeated tosses.

In other words, they are talking about tail events. That a frequentist probability (i.e., a long-run frequency) even exists can be a zero-probability event -- but you have to give axioms for probability before you can even make this claim. (Furthermore, I'm never going to observe a tail event, so I don't much care about them.)

Conrad,

Okay, so unpack "ungrounded" for me. You've used the phrases "probability" and "calculated or measured likelihood of heads coming up", but I'm not sure how you're defining them.

I'm going to do two things. First, I'm going to Taboo "probability" and "likelihood" (for myself -- you too, if you want). Second, I'm going to ask you exactly which specific observable event it is we're talking about. (First toss? Twenty-third toss? Infinite collection of tosses?) I have a definite feeling that our disagreement is about word usage.

If you honestly subscribe to this view of probability, please never give the odds for winning the lottery again. Or any odds for anything else.

What does telling me your probability that you assign something actually tell me about the world? If I don't know the information you are basing it on, very little.

I'm also curious about a formulation of probability theory that completely ignores random numbers and other theories that are based upon them (e.g. The law of large numbers, Central limit theorem).

Heck a re-write of http://en.wikipedia.org/wiki/Probability_theory with all mention of probabilities in the external world removed might be useful.

I'm not sure the many-worlds interpretation fully eliminates the issue of quantum probability as part of objective reality. You can call it "anthropic pseudo-uncertainty" when you get split and find that your instances face different outcomes. But what determines the probability you will see those various outcomes? Just your state of knowledge? No, theory says it is an objective element of reality, the amplitude of the various elements of the quantum wave function. This means that probability, or at least its close cousin amplitude, is indeed an element of reality and is more than just a representation of your state of knowledge.

For aficionados of interpretations of QM, this relates to an old debate, whether the so-called "Born rule" can be derived from the MWI. Various arguments have been offered for this, including one by Robin, and some have claimed that these now work so well that the argument is settled. However I don't think the larger physics/philosophy community is convinced.

Will Pearson, I'm having trouble determining to whom your comment is addressed.

Roland and Ian C. both help me understand where Eliezer is coming from. And PK's comment that "Reality will only take a single path" makes sense. That said, when I say a die has a 1/6 probability of landing on a 3, that means: Over a series of rolls in which no effort is made to systematically control the outcome (e.g. by always starting with 3 facing up before tossing the die), the die will land on a 3 about 1 in 6 times. Obviously, with perfect information, everything can be calculated. That doesn't mean that we can't predict the probability of a specific event.

Also, I didn't get a response to the Gomboc ( http://tinyurl.com/2rffxs ) argument. I would say that it has an inherent 100% probability of righting itself. Even if I knew nothing about the object, the real probability of it righting itself is 100%. Now, I might not bet on those odds, without previous knowledge, but no matter what I know, the object will right itself. How is this incorrect?

::Okay, so unpack "ungrounded" for me. You've used the phrases "probability" and "calculated or measured likelihood of heads coming up", but I'm not sure how you're defining them.::

Ungrounded: That was a good movie. Grounded: That movie made money for the investors. Alternatively: I enjoyed it and recommend it. -- is for most purposes grounded enough.

::I'm going to do two things. First, I'm going to Taboo "probability" and "likelihood" (for myself -- you too, if you want). Second, I'm going to ask you exactly which specific observable event it is we're talking about. (First toss? Twenty-third toss? Infinite collection of tosses?) I have a definite feeling that our disagreement is about word usage.::

You yourself said that we're dealing with one throw of a rigged coin, of unknown riggage. I don't think we have have a disagreement, exactly, except it looks to me like the discussion's moot.

But look: if I can back up a bit, the notion that we can be dealing with a rigged coin, know that it's rigged, and say that the --er, chances-- of getting a heads is "really" 50%, because we Just Don't Know, is useless. At that point you're using 50-50 because we have two possible known outcomes:

But in fact we deal with unknown probabilities all the time. Probabilities are by default unknown, until we measure them by repeated trial and a lot of scratch-work. What about when you're dealing with a medication that might kill someone, or not: in the absence of any information, do you say that's 50-50?

Conrad.

GBM:: ..That said, when I say a die has a 1/6 probability of landing on a 3, that means: Over a series of rolls in which no effort is made to systematically control the outcome (e.g. by always starting with 3 facing up before tossing the die), the die will land on a 3 about 1 in 6 times.::

--Well, no: it does mean that, but don't let's get tripped up that a measure of probability requires a series of trials. It has that same probability even for one roll. It's a consequence of the physics of the system, that there are 6 stable distinguishable end-states and explosively many intermediate states, transitioning amongst each other chaotically.

Conrad.

I have to say that it sure does seem like there ought to be some way to use the notion of frequentist probability to construct subjective probability along these lines.

Assign a measure to each possible world (the prior probabilities). For some state of knowledge K, some set of worlds Ck is consistent with K (say, the set in which there is a brain containing K). For some proposition X, X is true in some set of worlds Cx. The subjective probability P(X|K) = measure(intersection(Ck,Cx)) / measure(Ck). Bayesian updating is equivalent to removing worlds from K. To make it purely frequentist, give each world measure 1 and use multisets.

Does that work?

Who else thinks we should Taboo "probability", and replace it two terms for objective and subjective quantities, say "frequency" and "uncertainty"?

The frequency of an event depends on how narrowly the initial conditions are defined. If an atomically identical coin flip is repeated, obviously the frequency of heads will be either 1 or 0 (modulo a tiny quantum uncertainty).

Oops, removing worlds from Ck, not K.

GBM, I think you get the idea. The reason we don't want to say that the gomboc has an inherent probability of one for righting itself (besides that we, um, don't use probability one), is that as it is with the gomboc, so it is with the die or anything else in the universe. The premise is that determinism, in the form of some MWI, is (probably!) true, and so no matter what you or anyone else knows, whatever will happen is sure to happen. Therefore, when we speak of probability, we can only be referring to a state of knowledge. It is still of course the case that if you toss a fair die a very large number of times, the proportion of threes you get will tend towards 1/6--we're just not using such cases as a definition of what probability means.

Having already written the above, I must add that I like Nick's just-posted frequency/uncertainty breakdown.

Cyan, sorry. My comment was to Eliezer and statements such as

"that probabilities express ignorance, states of partial information; and if I am ignorant of a phenomenon, that is a fact about my state of mind, not a fact about the phenomenon."

I think there's still room for a concept of objective probability -- you'd define it as anything that obeys David Lewis's "Principal Principle" which this page tries to explain (with respect to some natural distinction between "admissible" and "inadmissible" information).

Before accepting this view of probability and the underlying assumptions about the nature of reality one should look at the experimental evidence. Try Groeblacher, Paterek, et al arXiv.0704.2529 (Aug 6 2007) These experiments test various assumptions regarding non=local realism and conclude= "...giving up the concept of locality is not sufficient to be consistent with quantum experiments, unless certain intuitive features of realism are abandoned"

Standard reply from MWIers is that MWI keeps realism and locality by throwing away a different hidden assumption called "counterfactual definiteness".

Nick Tarleton:

Who else thinks we should Taboo "probability", and replace it two terms for objective and subjective quantities, say "frequency" and "uncertainty"?

I second that, this would probably clear a lot of the confusion and help us focus on the real issues.

The "probability" of an event is how much anticipation you have for that event occurring. For example if you assign a "probability" of 50% to a tossed coin landing heads then you are half anticipating the coin to land heads.

What about when you're dealing with a medication that might kill someone, or not: in the absence of any information, do you say that's 50-50?

You've already given me information by using the word medication -- implicity, you're asking me to recall what I know about medications before I render an answer. So no, those outcomes aren't necessarily equally plausible to me. Here's a situation which is a much better approximation(!) of total absence of information: either event Q or event J has happened just now, and I will tell you which in my next comment. The asymmetry in your information is just that I chose the label Q for the first event and J for the second event. Which event do you find more plausible? I'd like you to justify your choice, pretending (if necessary) that I am honest in this instance.

Now, if at least one child is a boy, it must be either the oldest child who is a boy, or the youngest child who is a boy. So how can the answer in the first case be different from the answer in the latter two?

Because they obviously aren't exclusive cases. I simply don't see mathematically why it's a paradox, so I don't see what this has to do with thinking that "probabilities are a property of things."

The "paradox" is that people want to compare it to a different problem, the problem where the cards are ordered. In that case, if you ask "Is your first card an ace," "Is your first card the ace of hearts," or "Is your first card the ace of spades," then there is the same probability of 1/3 in all three cases that both cards are aces given an answer "Yes." In that case the averaging makes sense because the cases are exclusive. In the "paradox," you can't average by saying that, "well, if there's one it's either the Ace of Spades or the Ace of Hearts, and in either case the answer would be 1/3, so it averages to 1/3." The problem is that you're double-counting.

I'm a Bayesian, but I don't see what this particular example has to do with subjectivity and agents. Probability is a result of the measure and the universe one is dealing with, and that may lead to results that seem unintuitive to those who don't grasp the mathematical principles (that seem obvious to me), but that has nothing to do with needing an agent. Define the measure space as you have done, claim that the probabilities are cold hard inherent facts about the objects themselves, and the result is independent of an agent.

This "paradox" seems on the same level to me as the confusion as to why the chances of rolling a 6 in three rolls of a die is not 1/2, or the problem that if one takes an outbound trip averaging 30 mph, then it is impossible to make the inbound trip so as to average 60 mph without teleporting instantaneously.

Or, I suppose, I would compare it to the other noted statistical paradox, whereby a famous hospital has a better survival rate for both mild and severe cases of a disease than a less-noted hospital, but a worse overall survival rate because it sees more of the worst cases. Merely because people don't understand how to do averages has little to do with them requiring an agent.

The estimated Bayesian probability has nothing to do with the coin. If it did, assigning a probability of 0.5 to one of the two possible outcomes would be necessarily incorrect, because one of the few things we know about the coin is that it's not fair.

The estimate is of our confidence in using that outcome as an answer. "How confident can I be that choosing this option will turn out to be correct?" We know that the coin is biased, but we don't know which outcome is more likely. As far as we know, then, guessing one way is as good as guessing the other.

The sides of the coin do have an actual probability associated with them, which is why it's wrong to say that one particular outcome is more likely. That's a truth statement that we can't justify with the available data. Without knowing more about the coin, we can't speak about it. We can only speak to our confidence and how justified it is with the data we know.

The assertion that uncertainty is not an aspect of reality goes far beyond what anyone can justify, and is an example of gross overconfidence in one's opinions, btw.

Another way to look at it: if you repeatedly select a coin with a random bias (selected from any distribution symmetric about .5) and flip it, H/T will come out 50/50.

Silas: The uncertainty principle comes from the fact that position and momentum are related by Fourier transform. Or, in laymans terms, the fact that particles act like waves. This is one of the fundamental principles of QM, so yeah, it sort of does depend on the validity thereof. Not the Schrodinger equation itself perhaps, but other concepts.

As for whether QM proves that all probabilities are inherent in a system, it doesn't. It just prevents mutual information in certain situations. In coin flips or dice rolls, theoretically you could predict the outcome with enough information. Most probabilistic situations are that way; they're probabilistic because you don't have that info. QM is a bit different, and scientists still argue about it, but the fine detail of behavior of atoms doesn't have any effect on a poker game.

Z. M. Davis: Thank you. I get it now.

Follow-up question: If Bob believes he has a >50% chance of winning the lottery tomorrow, is his belief objectively wrong? I would tentatively propose that his belief is unfounded, "unattached to reality", unwise, and unreasonable, but that it's not useful to consider his belief "objectively wrong".

If you disagree, consider this: suppose he wins the lottery after all by chance, can you still claim the next day that his belief was objectively wrong?

Nick Tarleton: Not sure I entirely correctly understood your suggestion, need to think about it more.

However, my initial thought is that it may require/assume logical omnicience.

ie, what of updating based on "subjective guesses" of which worlds are consistent or inconsistent with the data. That is, as consistent as you can tell, given bounded computational resources. I'm not sure, but your model, at least at first glance, may not be able to say useful stuff about those that are not logically ominicent.

Also, I'm unclear, could you clarify what it is you'd be using a multiset for? Do you mean "increase measure only by increasing number of copies of this in the multiset, and no other means allowed" or did you intend something else?

(incidentally, I think I do prefer coherence/dutch book/vulnerability style constructions of epistemic probability. Especially the ones that build up decision theory along the way, so one ends up starting with utilities almost. Such have very much of a "mathematical karma" flavor, as I've expressed elsewhere.)

Hal, I'd say probability could be both part of objective physics and a mental state in this sense: Given our best understanding of objective physics, for any given mental state (including the info it has access to) there is a best rational set of beliefs. In quantum mechanics we know roughly the best beliefs, and we are trying to use that to infer more about the underlying set of states and info.

Rolf Nelson: "Follow-up question: If Bob believes he has a >50% chance of winning the lottery tomorrow, is his belief objectively wrong? I would tentatively propose that his belief is unfounded, "unattached to reality", unwise, and unreasonable, but that it's not useful to consider his belief "objectively wrong"."

It all depends on what information Bob has. He might have carefully doctored the machines and general setup of the lottery draw to an extent that he might have enough information to have that probability. Now if Bob says he thinks he has a greater than 50% chance of winning the lottery because he is feeling lucky, and that is it, you can probably say that is unattached to reality or ignoring lots of relevant information.

However, my initial thought is that it may require/assume logical omnicience.

Probably. Bayes is also easier to work with if you assume logical omniscience (i.e. knowledge of P(evidence|X) and P(E|~X)).

Also, I'm unclear, could you clarify what it is you'd be using a multiset for? Do you mean "increase measure only by increasing number of copies of this in the multiset, and no other means allowed" or did you intend something else?

Yes, using multisets of worlds with identical measure is equivalent to (for rational measures only) but 'more frequentist' than sets of worlds with variable measure.

incidentally, I think I do prefer coherence/dutch book/vulnerability style constructions of epistemic probability. Especially the ones that build up decision theory along the way, so one ends up starting with utilities almost. Such have very much of a "mathematical karma" flavor, as I've expressed elsewhere.

Yeah, my idea was only meant as an existence proof and is probably an inferior formal construction, although it is how I tend to personally think about subjective probability. I guess at heart I'm still a frequentist.

(You could think about Rolf's problem this way; if the vast majority of the measure of possible worlds given Bob's knowledge is in worlds where he loses, he's objectively wrong.)

You have to lay £1 on heads or tails on a biased coin toss. Your probability is in your mind, and your mind has no information either way. Hence, you lay the pound on either. Hence you assign a 0.5 probability to heads, and also to tails.

If your argument is 'I don't mean my personal probability, I mean the actual probability', abandon all hope. All probability is 'perceived'. Unless you think you have all the evidence.

All probability is 'perceived'. Unless you think you have all the evidence.

Some probabilities are objective, inherent properties of bits of the universe, and the universe does have all the evidence. The coin possesses an actual probability independent of what anyone knows or believes about it.

if the vast majority of the measure of possible worlds given Bob's knowledge is in worlds where he loses, he's objectively wrong.

That's a self-consistent system, it just seems to me more useful and intuitive to say that:

"P" is true => P
"Bob believes P" is true => Bob believes P

but not

"Bob's belief in P" is true => ...er, what exactly?

Also, I frequently need to attach probabilities to facts, where probability goes from [0,1] (or, in Eliezer's formulation, (-inf, inf)). But it's rare for me to have to any reason to attach probabilities to probabilities. On the flip side, I attach scoring rules in the range [0, -inf] to probability calculations, but not to facts. So in my current worldview, facts and probabilities are tentatively "made of different substances".

I second tabooing probability, but I think that we need more than two words to replace it. Casually, I think that we need, at the least, 'quantum measure', 'calibrated confidence', and 'justified confidence'. Typically we have been in the habit of calling both "Bayesian", but they are very different. Actual humans can try to be better approximations of Bayesians, but we can't be very close. Since we can't be Bayesian, due to our lack of logical omniscience, we can't avoid making stupid bets and being Dutch Booked by smarter minds. It's therefore disingenuous to claim that vulnerability to Dutch Books is a decisive argument against a behavioral strategy. Calibrated confidence is the strategy that we can try to use to minimize our vulnerability to being Dutch Booked by people who aren't smarter than we are but who know exploits in our heuristics. They tend to be much much closer to 50% than Bayesian confidences, and are pretty much unavoidably subject to some framing based biases as a result.

just fyi, there's no such thing as the 'eldest' of two boys; there's just an elder and a younger. superlatives are reserved for groups of three or more.

as i'm a midget among giants here, i'm afraid that's all i have to add. :)

Enginerd: The uncertainty inherent in determining a pair of conjugate variables - such as the length and pitch of a sound - is indeed a core part of QM, but is not probabilistic. In this case, the term "uncertainty" is not about probabilities, even if QM is probabilistic in general, rather a consequence of describing states in terms of wave functions, which can be interpreted probabilistically. This causes many to mistakenly think the Heisenberg's "Uncertainty Principle" is the probabilistic part of QM. As Wikipedia[1] puts it: "The uncertainty principle is related to the observer effect, with which it is often conflated."

[1] http://en.wikipedia.org/wiki/Uncertainty_principle

You're equating perceived probability with physical probability, and this is false, when either you or anyone else ignores that distinction.

However, your whole argument depends on a deterministic universe. Research quantum mechanics; we can't really say that we have a deterministic universe, and physics itself can only assign a probability at a certain point.

@Daniel:

You're attacking the wrong argument. Just look up the electron double-slit experiment. (http://en.wikipedia.org/wiki/Double-slit_experiment) Its not only about the observer effect, but how the probability that you say doesn't exist causes interference to occur unless an observer is present. The observer is the one who collapses the probability wave down to a deterministic bayesian value.

It sounds like both you and the author of this blog do not understand Schrodinger's cat.

Let me further explain my point. Somewhere earlier said that reality only takes one path. Unless an observer is present, the electron double slit experiment proves that this assumption is false.

Welcome to Overcoming Bias, anon! Try to to avoid triple-posting. The author of this post has actually just written a series on quantum mechanics, which begins with "Quantum Explanations." He argues forcefully for a many-worlds interpretation, which is deterministic "from the standpoint of eternity," although not for any particular observer due to indexical uncertainty. (You might say that, yes, reality does not take only one path, but it might as well have, because neither do observers!)

@Z. M. Davis

Thanks for the welcome. While I disagree with the etiquette, I'll try to follow it. A three post limit serves only to stifle discussion; there are other ways to deal with abusive posters than limiting the abilities of non-abusive posters. Also, I'm pretty sure my comment is still valid, relevant, and an addition to the discussion, regardless of whether I posted it now or a couple hours ago.

Back to the many worlds approach, as an individual observer of the universe myself, it seems to me that attempting to look at the universe "from the standpoint of eternity" in order to force universal determinism (and its implications) is equivalent to looking at a dice roll from the standpoint of a infinite series of dice rolls in order to force frequency. Its a perspective we'll never achieve and it subverts our intuition. Just a thought, as this blog seems to be about avoiding cognitive traps and not just shifting them to a higher level.

That the probability assigned to flipping a coin depends on what the assigner knows doesn't prove probability's subjectivity, only that probability isn't an objective property of the coin . Rather, if the probability is objective, it must be a property of a system, including the throwing mechanism. Two other problems with Eliezer's argument. 1) Rejecting objective interpretations of probability in empirical science because, in everyday usage, probability is relative to what's known, is to provide an a priori refutation of indeterminism, reasoning which doesn't square with Eliezer's empiricist analysis of knowledge. (Although, perhaps, objective probability is incoherent, but Eliezer hasn't shown this.) 2) A purely subjective interpretation of probability leaves probability in the same position as religion (or, maybe exactly, in the same position as Kant's a prior forms of understanding). A purely subjective interpretation doesn't explain the adaptive utility of taking the first Bayesian step of forming an a priori probability.

Stephen R. Diamond, there are two distinct things in play here: (i) an assessment of the plausibility of certain statements conditional on some background knowledge; and (ii) the relative frequency of outcomes of trials in a counterfactual world in which the number of trials is very large. You've declared that probability can't be (i) because it's (ii) -- actually, the Kolmogorov axioms apply to both. Justification for using the word "probability" to refer to things of type (i) can be found in the first two chapters of this book. I personally call things of type (i) "probabilities" and things of type (ii) "relative frequencies"; the key is to recognize that they need different names.

On your further critiques:
(1) Eliezer is a determinist; see the quantum physics sequence.
(2) True. A logical argument is only as reliable as its premises, and every method for learning from empirical information is only as reliable as its inductive bias. Unfortunately, every extant practical method of learning has an inductive bias, and the no free lunch theorems give reason to believe that this is a permanent state of affairs.

I'm not sure what you mean in your last sentence...

Like Butters in that South Park episode, I can't help after all these posts but to notice that I am confused.

"Renormalizing leaves us with a 1/3 probability of two boys, and a 2/3 probability of one boy one girl." help me with this one, i'm n00b. If one of the kids is known to be a boy (given information), then doesn't the other one has 50/50 chances to be either a boy or a girl? And then having 50/50 chances for the couple of kids to be either a pair of boys or one boy one girl?

Conrad wrote:

ps - Ofc, knowing, or even just suspecting, the coin is rigged, on the second throw you'd best bet on a repeat of the outcome of the first.

I think it would be worthwhile to examine this conclusion - as it might seem to be an obvious one to a lot of people. Let us assume that there is a very good mechanical arm that makes a completely fair toss of the coin in the opinion of all humans so that we can talk entirely about the bias of the coin.

Let's say that the mechanism makes one toss; all you know is that the coin is biased - not how. Assume that it comes up heads; what does this tell you about the bias? Conrad asserts that it will certainly be biased in favor of heads. How much? Will it always show up as heads? 3 times out of 4? As it turns out, you have no way of knowing.

It could be that it is in fact only 1/3 biased towards heads; then it would be much wiser to bet on tails in the future, no? It could be that it is actually 100 times more likely that tails will come up; you simply can't tell the difference from the first toss.

So let's consider more coin tosses. What if it comes up heads once and then tails 5 times in a row? Could you tell me exactly what the bias is? Is it 5/6 towards tails perhaps? What about 50 tails and 15 heads? In fact, it is still not possible to say anything at all about what the bias is.

Since you probably have a heuristic method of analysis (intuition) you will in time see which side is the best bet; i.e. you'll conclude which side is most likely to be biased and you'll probably be correct - with higher accuracy as the amount of tosses increase. However; there is no logic, rationalism or deduction in the world that could tell you exactly what the bias is. This is true after any integer amount of coin tosses.

"Or here's a very similar problem: Let's say I have four cards, the ace of hearts, the ace of spades, the two of hearts, and the two of spades. I draw two cards at random. You ask me, "Are you holding at least one ace?" and I reply "Yes." What is the probability that I am holding a pair of aces? It is 1/5. There are six possible combinations of two cards, with equal prior probability, and you have just eliminated the possibility that I am holding a pair of twos. Of the five remaining combinations, only one combination is a pair of aces. So 1/5." For future reference the initial phrase "I have four cards" may imply that you have in your possession those cards already. Combined with the fact that you draw two cards afterwords implies that you now have six cards. The simple fact that you are using cards implies that you are using a full deck of 52 cards, unless you specifically say there are only certain cards in play. I just want to clear that up because I had to reread that section a few times before I understood how you came up with your figures.

The unpredictability of a die roll or coin flip is not due to any inherent physical property of the objects; it is simply due to lack of information. Even with quantum uncertainty, you could predict the result of a coin flip or die roll with high accuracy if you had precise enough measurements of the initial conditions.

That is quite debatable. For one thing, it is possible to for quantum indeterminism, if there is any, to leak into the macroscopic world. Even if it were not possible, there is still the issue of microscopic indeterminism. You cannot prove that there is no objective indeterminism (ie that the universe is deterministic) just by performing an armchair examination of human reasoning about probability. You have to take the physics into account as well. It appears to be standard round here to assert the Many Worlds interpretation, and to assert it as deterministic. That is debatable as well, since there are problems with MWI and it is not the only no-collapse interpretation

Hate to be a stickler for this sort of thing, but even in the bayesian interpretation there are probabilities in the world, it's just that they are facts about the world and the knowledge the agents have of the world in combination. It's a fact that a perfect bayesian given P(a), P(a|b), and P(a|~b) will ascribe P(b|a), a probability of P(a|b)P(a) / P(b), and that that is the best value to give P(b|a).

If an agent has perfect knowledge then it need not ascribe any non-1 probability to any proposition it holds. But it is a fact about agents in the world that without perfect knowledge they ascribe non-1 probabilities to their propositions if they're working right. Bayesian reasoning is the field which tells us the optimal probability to assign to a proposition given the rest of our information, but that that is the optimal probability given the rest of our information is a fact about the world. FOr any proposition 'a', if a perfect bayesian says 'P(a) = y:x' based off of some premise list P, then any agent who concludes 'a' from "P" (or any other equivalently cogent premise list) will be right y:x of the time, and wrong 1 - y:x of the time regardless of what "a" actually says.

Some might say, "There is no sweetness in the world; sweetness is in your mind's interpretation of the world." The correct response is "Since 'in' is a transitive relation, and my mind's interpretation of the world is in the world, sweetness is in the world. It's just that to learn about sweetness you can't just study sugar crystals, you have to study brains too." The situation here is similar-ish.

It is important that facts about the probabilities of statements be facts about the world; if they weren't then how would we find our priors? Priors seem to require that we be capable of checking "P(this woman having cancer) = such and such" by checking the world. In fact, I believe EY say's almost word for word that priors are fact about the world in "an intuitive explanation of Bayes theorem": "Actually, priors are true or false just like the final answer - they reflect reality and can be judged by comparing them against reality. For example, if you think that 920 out of 10,000 women in a sample have breast cancer, and the actual number is 100 out of 10,000, then your priors are wrong. " -- EY

Let us not forget that the map is a part of the territory; the map's accuracy is a fact about the territory as much as a fact about the map. You can study a map till you're blue in the head, and you still won't know how accurate it is unless you look at the corresponding territory.

Does this mean that there is nothing that is inherently uncertain? I guess another way to put that would be, could Laplace's Demon infer the entire history of the universe back to front from a single moment? It might seem obvious that there are singularities moving backwards through time (i.e. processes whose result does not give you information about their origin), so couldn't the same thing exist moving forward through time?

Anyway, great article!

My first post, so be gentle. :)

I disagree that there is a difference between "Bayesian" and "Frequentist;" or at least, that it has anything to do with what is mentioned in this article. The field of Probability has the unfortunate property of appearing to be a very simple, well defined topic. But it actually is complex enough to be indefinable. Those labels are used by people who want to argue in favor of one definition - of the indefinable - over another. The only difference I see is where they fail to completely address a problem.

Take the biased coin problem as an example. If either label applies to me, it is Frequentist, but my answer is that the one Eliezer_Yudkowsky says is the Bayesian's. He gets the wrong Frequentist solution because he only allows the Frequentist to acknowledge one uncertainty - one random variable - in the problem. Whether the coin came up heads or tails. If a Frequentist says the question is unanswerable, (s)he is wrong because (s)he is using an incomplete solution. The bias b - of a coin already selected - is just as much a random variable as the side s that came up in a coin already flipped. If you claim the answer must be based on the actual value of b for the coins, it must also be based on the actual value of s for this flip. That means the probability is either 0 or 1, which is absurd. (Technically, this error is one of confusing an outcome and an event. An outcome is the specific result of a specific trial, and has no probability. An event is a set of possible outcomes, and is what a probability is assigned to. Eliezer_Yudkowsky's Frequentist is treating the choice of a coin as an outcome, and the result of the flip as an event.)

We can answer the question without knowing anything more about b, than that it is not 1/2. For any 0<=b1<1/2, since we have no other information, b=b1 and b=1-b1 must be treated as equally likely. Regardless of what the distribution of b1 is, this makes the probability the coin landed on heads 1/2.

The classic Two Child Problem has a similar issue, but Eliezer_Yudkowsky did not ask the classic one. I find it best to explain this one in the manner Joseph Bertrand used for his famous Box Paradox. I have two children. What is the probability they share the same gender? That's easy: 1/2. Now I secretly write one gender on a note card. I then show the card to you, and tell you one of my children has that gender. If it says "boy," does the answer change to 1/3? What if it says "girl"? The answers can't be different for the two words you might see; but whatever that answer is, it has to be the same as the answer to the original question (proof by Bayes Theorem). So if the answer does change, we have a paradox.

Yet if presented with the information all at once, "I have two, and one is a boy," Frequentist and Bayesian alike will usually answer "1/3." And they usually will say that anybody who answers 1/2 is addressing the "I have two, and one specific child, by age, is a boy" version Eliezer_Yudkowsky mentioned. But that is not how I get 1/2. There are three random variables, not two: the older child's gender, the younger child's gender, and which gender I will mention if I have the choice of two. Allowing all three to be split 50/50 between "boy" and "girl" makes the answer 1/2, and there is no paradox.

Ironically, my reasoning is what the same mathematicians will use for either the Monty Hall Problem, or the identical Three Prisoners Problem. Two cases that were originally equally likely remain possible. But they are no longer equally possible, because the provider of information had a choice of two in one case, but no choice in the other. Bayesians may claim the difference is a property of the information, and Frequentists (if they use a complete solution) will say there is an additional, implicit random variable. Both work out the same, just by different methods. It is ironic, because while Bertrand's Box Paradox is often compared to these two problems because it is mathematically equivalent to them. The Two Child Problem is closer to being logically equivalent because of the way the information is provided, yet never gets compared. In fact, it is identical if you add a fourth box.

I used to be a frequentist, and say that the probability of the unfair coin landing heads is either 4/5 or 1/5, but I don't know exactly which. But that is not to say that I saw probabilities on things instead of on information. I'll explain.

If someone asked me if it will it rains tomorrow, I would ask which information am I supposed to use? If it rained in the past few days? Or would I consider tomorrow as a random day and pick the frequency of rainy days in the year? Or maybe I should consider the season we are in. Or am I supposed to use all available information I have? The latter I would call subjective probability. If someone instead passed me the children problem I would say 1/3 because this problem implicitly tells me to consider only the what tells the enunciate.

But simply asking for the probability without a context, I would say either that this is a no question, i.e. that the enunciate is imprecise and lacking information, or I would believe that the interrogator was asking for a intrinsic probability, in which case I would say either 0 or 1, but I don't know which.

But I did believe in intrinsic probability, in some cases, like quantum mechanics.

This view of mine became hollow after I started inquiring myself about this intrinsic probability. Even if such a thing existed, it couldn't be differentiated from what I called subjective probability. By Occam's razor I shouldn't create 2 kinds of probabilities that I cannot tell apart. This thought was partly inspired by reading lesswrong, not a particular post, but by seeing the ease in which what I called subjective probability was used in several occasions.

Thinking of probabilities as levels of uncertainty became very obvious to me when thinking about the Monty Hall problem. After the host has revealed that one of the three doors has a booby prize behind it, you're left with two doors, with a good prize behind one of them.

If someone walks into the room at that stage, and you tell them that there's a good prize behind one door and a booby prize behind another, they will say that it's a 50/50 chance of selecting the door with the prize behind it. They're right for themselves, however the person who had been in the room originally and selected a door knows more and therefore can assign different probabilities - i.e. 1/3 for the door they'd selected and 2/3 for the other door.

If you thought that the probabilites were 'out there' rather than descriptions of the state of knowledge of the individuals, you'd be very confused about how the probability of choosing correctly could at the same time be 2/3 and 1/2.

Considering the Monty Hall problem as a way for a part of the information in the hosts head to be communicated to the contestant becomes the most natural way of thinking about it.

I'm sorry, why isn't the prior probability that you say "why yes, I am holding the ace of spades" = 1/4?

Edit: unless you meant "draw a pair", in which case yes, the ace of spades would show up in three out of six possible pairings.

Even before a fair coin is tossed, the notion that it has an inherent 50% probability of coming up heads may be just plain wrong. Maybe you're holding the coin in such a way that it's just about guaranteed to come up heads, or tails, given the force at which you flip it, and the air currents around you. But, if you don't know which way the coin is biased on this one occasion, so what?

Maybe it isn't really 50%, and it isn't really 100% how-it-came up either. That it is rational to make estimates based on our own ignorance is not proof that the universe is deterministic. You can't deduce the nature of physical reality from your own reasoning processes.

To make the coinflip experiment repeatable, as frequentists are wont to demand, we could build an automated coinflipper, and verify that the results were 50% heads and 50% tails. But maybe a robot with extra-sensitive eyes and a good grasp of physics, watching the autoflipper prepare to flip, could predict the coin's fall in advance—not with certainty, but with 90% accuracy. Then what would the real probability be?

Whatever QM says.

So, I've been on this site for awhile. When I first came here, I had never had a formal introduction to Bayes' theorem, but it sounded a lot like ideas that I had independently worked out in my high school and college days (I was something of an amateur mathematician and game theorist).

A few days ago I was reading through one of your articles - I don't remember which one - and it suddenly struck me that I may not actually understand priors as well as I think I do.

After re-reading some fo the series, and then working through the math, I'm now reasonably convinced that I don't properly understand priors at all - at least, not intuitively, which seems to be an important aspect for actually using them.

I have a few weird questions that I'm hoping someone can answer, that will help point me back towards the correct quadrant of domain space. I'll start with a single question, and then see if I can claw my way towards understanding from there based on the answers:

Imagine there is a rational, Bayesian AI named B9 which has been programmed to visually identify and manipulate geometric objects. B9's favorite object is a blue ball, but B9 has no idea that it is blue: B9 sees the world through a black and white camera, and has always seen the world through a black and white camera. Until now, B9 has never heard of "colors" - no one has mentioned "colors" to B9, and B9 has certainly never experienced them. Today, unbeknownst to B9, B9's creator is going to upgrade its camera to a full-color system, and see how long it takes B9 to adapt to the new inputs.

The camera gets switched in 5 seconds. Before the camera gets switched, what prior probability does B9 assign to the possibility that its favorite ball is blue?

This was a very difficult concept for me, Eliezer. Not because I disagree with the Bayesian principle that uncertainty is in the mind, but because I lacked the inferential step to jump from that to why there were different probabilities depending on the question you asked.

Might a better (or additional) way to explain this be to point out an analogy to the differing probabilities of truth you might assign to confirmed experimental hypothesis that were either originally vague, and therefore have less weight when adjusting the overall probability of truth vs. specific, and therefore shift the probability of truth further.

Hopefully I'm actually understanding this correctly at all.

The problem with trying to split, the it must be the oldest child who is the boy or the youngest child who is the boy is that the two situations overlap. You need to split the situation into oldest, youngest and both. If we made the ruling that both should be excluded, then we'd be able to complete the argument that there shouldn't be a difference between knowing that one child is a boy or knowing that the oldest child is a boy.

I think that the main point of this is correct, but the definition of "mind" used by this phrase is unclear and might be flawed. I'm not certain, just speculating.

As an aside, I think it is equivocation to talk about this kind of probability as being the same kind of probability that quantum mechanics leads to. No, hidden variable theories are not really worth considering.

But projectivism has been written about for quite a long time (since at least the 1700s), and is very well known so I find it hard to believe that there are any significant proponents of 'frequentism' (as you call it).

To those who've not thought about it, everyday projectivism comes naturally, but it falls apart at the slightest consideration.

When it comes to Hempel's raven, though, even those who understand projectivism can have difficulty coming to terms with the probabilistic reality.

I think I can show how probability is not purely in the mind but also an inherent property of things, bear with me.

Lets take an event of seeing snow outside, for simplicity we know that snow is out there 3 month a year in winter, that fact is well tested and repeats each year. That distribution of snowy days is property of the reality. When we go out of bunker after spending there unknown amount of time we assign probability 1/4 to seeing a snow, and that number is function of our uncertainty about the date and our precise knowledge of when snow is out there. 1/4 is a precise description of reality if our scope is not just one day but a whole year. In this case we have a precise map, and our uncertainty is lack of knowledge of our place on the map. What we also know that if we do not have a date or season there is no better prediction and this is a property of things too.

Additionally having probability distribution you can perfectly predict accumulated effect of series of events, and this ability to predict something precisely is an indication that you grasped something about reality.

Returning to the coin 0.5 prediction of one throw is function of our uncertainty, but our prediction of sum of long series where 1 is heads and 0 is tails is a result of our knowledge of coin properties that is expressed as probability

E[x]=0.5

even for the frequentist, and that's what we make decisions with, so focusing on p(x) is a bit of misdirection. The whole frequentist-vs-bayesian culture war is fake. They're both perfectly consistent with well defined questions. (They have to be, because math works.)

And yes to everything else, except...

As to whether god plays dice with the universe... that is not in the scope of probability theory. It's math. Your Bayesian is really a pragmatist, and your frequentist is a straw person.

Great post!

Kinship, or more accurately the lack of it, is likewise in the mind. That's why it always annoys me to see the parenthetical phrase "no relation" in a newspaper or magazine article.

It is a mind game, but not the one you're claiming imo. Probabilities are a game about choices, aka co-products. There are lots of ways to specify the alternatives in a co-product.  And once you've done so, you can create an instance of that co-product by injecting one of its constructors.  A co-product is a type, and its constructors create instances of that type. So frequentists count up the instances and then compare the relative frequency. Your mind games are just silly ways of defining different co-products using hypothetical knowledge or not. Differences in knowledge imply different definitions of alternatives ("Knowing" something subtracts the alternatives), which defines different types. But obviously you don't actually need silly anecdotes about what somebody knows or doesn't know to define different co-products. You could do it with toothpicks of different lengths, or the color of balls, or whatever.

I tried to rush the angry comment about how it all is wrong, but a few second ater posting the comment (oops) I understood. I've seen a great example since the school genetics: when two heterozygotes cross (Aa is crossed with Aa), frequency of homozygotes among the descendants with dominant trait is 1/3. AA Aa aA aa (may never survive to the adulthood. Or AA may not survive. Or both survive, but we aren't interested)

There may be something that influences the 1:2:1 proportion (only in one side?), but it's a "You flip a loaded coin. What's your bet on it falling heads?" case.