Bead Jar Guesses

"We'll bypass the novice mistake of calling it .5, of course; just because the options are binary (red or non-red) doesn't make them equally likely. It's not like you have any information."

Well, if you truly had no information, 0.5 would be the correct (entropy maximizing given constraints) bet. If you have no information you can call it "A or !A" or "!B or B" and it sounds the same- you can't say one is more likely.

By assigning a different probability, you're saying that you have information that makes the word "red" means something to you, and that it's less likely than half (say because there are 11 other "colors").

Likewise, if I say how likely is A and how likely is !A? You have to say 0.5. If A turns out to be "I'm gonna win the lottery tomorrow" then you can update and P goes to near zero. You didn't screw up though, since It could have just as easily been "I won't win the lottery tomorrow". If you don't think that it's just as likely, then that is information.

When you hear people saying "winning the lottery is 50/50 because either you win or you don't", their error isn't that they "naively" predict 0.5 in total absence of information. Their problem is that they don't update on the information that they do have.

I think more of our beliefs are bead jar guesses than we realize, but because of assorted insidious psychological tendencies, we don't recognize that and we hold onto them tighter than baseless suppositions deserve. [said Alicorn in the original]

If someone wants to do the work of linking the fairly abstract discussion here to how we think about making decisions in the real world, I think we would all benefit greatly.

We're to assume here that Omega, being a strange sort of deity, has selected the question "Is the bead red?" via some process that has no expected correlation whatsoever to the actual color. Correct?

I've just started reading Jaynes on prior formation, and I'd love to see more posts here on the topic. Maybe I'll write one if I ever have the chance to get some reading done.

As far as this problem goes, I agree we have some information about other colors. I want to know what Omega counts as "red" though, because that will go a long way in determining what sort of prior we'd assign.

Based on my limited understanding of physics, if we assume the bead only reflects a single wavelength, then it would be red if the wavelength were between 620 and 750nm. The visible spectrum goes from 380 to 750nm, so a uniform distribution over wavelength gives a probability of 0.34.

Most objects reflect multiple wavelengths though. In this case, the color of an object could be characterized as a distribution over the visible spectrum of the reflected light. To count as red, the distribution would need a mean between 620 and 750nm. We might need an additional constraint on the shape of the distribution so it isn't too spread out. I don't know how you'd begin calculating the measure of distributions that meet these constraints though.

I think drawing a red bead will marginally increase our probability of red.

In the end, I think our estimate is much more likely to depend on our expectation of Omega's motives than knowledge about colors, though.

It's not like you have any information. Assuming you don't think Omega is out to deliberately screw with you

These are contradictory assumptions. I have no information. I have no idea what Omega is out to do. The whole point of invoking Omega is to obliterate meaningful priors. I'm with byrnema here, probabilities are a tool for making maximally effective use of information. Without any such information, the only correct answer to "what is your probability" is "I don't have one".

Interestingly, figuring out the answers to questions of this kind, basically about prior, we are dealing with issues similar to those in elicitation of human values. In both cases, the answer is hidden in our minds, never in explicit and consistent form, with no hope of constructing a precise model that will give the answer. The only way to approximate the solution is to consider arguments for and against, consider relared situations, think, and listen to your inner voice, to intuitive response that says that it's proper to save a child, and you agree, that it's proper to assign a probability of no less that .03 to seeing red, and you agree, and that can lie to you, beware.

P.S. Keywords for Google scholar: 'utility elicitation', 'prior elicitation', 'proper scoring rules'.

Alicorn, I think it'd be appropriate to add the following link at the beginning of the article:

Related to: Priors as Mathematical Objects.

It also kinda answers your questions.

Even if Omega had asked about the bead being lilac, and you'd dutifully given a tiny probability, it would not have surprised you to see a lilac bead emerge from the jar.

I see this conclusion as a mistake: being surprised is a way of translating between intuition and explicit probability estimates. If you are not surprised, you should assign high enough probability, and otherwise if you assign tiny probability, you should be surprised (modulo known mistakes in either representation).

Predicting the second bead given the color of the first one can also be expressed as probability estimates for joint observations, made before you observe the color of the first bead. What is the probability that you'll see two reds? That you'll see a red followed by a non-red? Non-red following by a red? Two non-reds? Then crunch the numbers through the definition of conditional probability/Bayes' theorem.

But because you start with no information, it's very hard to gather more. Suppose Omega reaches into the jar and pulls out a red bead. Does your probability that the second bead will be red go up... down... [or] stay the same...?

My intuition here is to start with an uninformative prior over possible bead-generating mechanisms. (You still have the problem of how to divide up the state space, but that's nothing new.) If a red bead comes out first, I update the probabilities that I assign to each mechanism and proceed from there.

Where exactly that leads seems likely to depend on a bunch of assumptions that I'm too lazy to think through right now (e.g. whether the urn has a finite number of beads in it) but it seems a reasonable way of proceeding in principle.

A human example of this situation.

For me, the Omega problem described in the post presents the following conundrum: what is a probability in the limit of no information?

Suppose we employ a pragmatic perspective: the "probability of an event", as a mathematical object, is a tool that is used to summarize information about the past and/or future occurrence of that event. In the limit of no information, using the pragmatic view, there is no justification for assigning a probability not because we don't know what it is, but because it has no use in summarizing information.

If you don't care about being pragmatic, if you want to define a probability because everything must have a probability between 0 and 1, then I don't see how you would be justified in eliminating information (constraining the space of possible probabilities) or making up information (arbitrarily picking a probability) by specifying a value for the probability.

So finally, if Omega asked me what is the probability of him choosing a red ball, I would be forced to say something like, "To the extent that I can guess what you might be meaning by probability -- respectfully, Omega, it's not entirely well-defined -- the probability is p where 0<p<1."

How could I truthfully say more?

I find this problem interesting because it's a gray area in the correct approach to working with definitions. One of the perspectives I've encountered on Less Wrong is that in "real world" situations you need to let definitions be defined as you go. Finding the solution to a problem involves finding the right/good/proper definition, so that the definition you end up with tells you which world you're in. See Disputing Definitions).

This problem represents a context where you cannot find a good workable definition for probability (other than as the original abstract mathematical object), because you're not allowed to constrain which world you're in, because that would require making up information.

This post confused me enormously. I thought I must be missing something, but reading over the comments, this seems to be true for virtually all readers.

What exactly do you mean by "bead jar guess"? "Surprise"? "Actual probability"? Are you making a new point or explaining something existing? Are you purposely being obscure "to make us think"?

I propose replacing this entire post with the following text:

Hey everybody! Read E.T. Jaynes's Probability Theory: The Logic Of Science!

Thinking about how an Occamian learner like AIXI would approach the problem, it would probably start from the simplest domain theory "beads have a color, red is a color I've heard mentioned, therefore all beads are red", p=1. If the first bead was grey, it would switch to "all beads are grey", p=0. The second bead is red, "half and half", p = 0.5, and so on, ratcheting up theories from the simplest first.

I think this post could have been more formally worded. It draws a distinction between two types of probability assignment, but the only practical difference given is that you'd be surprised if you're wrong in one case but not the other. My initial thought was just that surprise is an irrational thing that should be disregarded ― there's no term for "how surprised I was" in Bayes' Theorem.

But let's rephrase the problem a bit. You've made your probability assignments based on Omega's question: say 1/12 for each color. Now consider another situation where you'd give an identical probability assignment. Say I'm going to roll a demonstrated-fair twelve-sided die, and ask you the probability that it lands on one. Again, you assign 1/12 probability to each possibility.

(Actually, these assignments are spectacularly wrong, since they give a zero probability to all other colors/numbers. Nothing deserves a zero probability. But let's assume you gave a negligible but nonzero probability to everything else, and 1/12 is just shorthand for "slightly less than 1/12, but not enough to bother specifying".)

So as far as everything goes, your probability assignments for the two cases look identical up to this point. Now let's say I offer you a bet: we'll go through both events (drawing a bead and putting it back, or rolling the die) a million times. If your estimate of the probability of red/one was within 1% of correct in that sample, I give you $1000. Otherwise, you give me $1000.

In the case of the die, we would all take the bet in a heartbeat. We're very sure that our figures are correct, since the die is demonstrated to be fair, and 1% is a lot of wiggle room for the law of large numbers. But you'd have to be crazy to take the same bet on the jar, despite having assigned a precisely identical chance of winning.

So what's the difference? Isn't all the information you care about supposed to be encapsulated in your probability distribution? What is the mathematical distinction between these two cases that causes such a clear difference in whether a given bet is rational? Are we supposed to not only assign probabilities to which events will occur, but also to our probabilities themselves, ad infinitum?

I'd say like the second example, most of our beliefs are bead jar guesses informed by untrustworthy informants. Namely our parents and other adults around when we were young.

Do you see this as being sort of like Jimmy's metauncertainty?

Also, if Omega pulled out a bead and then asked you about the next one, the Rule of Succession would be a good place to start making guesses.

Omega asking you that question, "What's the probability that the bead will be red" is itself information about the beads - Omega is more likely to ask that question in cases where the color is relevant.

To do things properly, you could suppose that Omega is inspiring himself from existing bead-color-guessing logic puzzles (there are plenty of examples of those in human history), and you could assign probabilities to each type of guessing games, noting how many types of beads there are etc. You can also collect statistics about which combinations of colors are often used for puzzles (red-and-blue and black-and-white seem common), and from all that, have a probability to assign to each color coming out, knowing that "will the bead be red" is a reasonable question.

That kind of thinking seems like closer to what you can come up with with limited computational resources, and it might give you a probability of 0.5 for red.

Apparently, the term you are searching for is "Second Order Probability".

See here for a paper: www.dodccrp.org/events/2000_CCRTS/html/pdf_papers/Track_4/124.pdf

I've just started reading Jaynes on prior formation, and I'd love to see more posts here on the topic. Maybe I'll write one if I ever have the chance to get some reading done.

As far as this problem goes, I agree we have some information about other colors. I want to know what Omega counts as "red" though, because that will go a long way in determining what sort of prior we'd assign.

Based on my limited understanding of physics, if we assume the bead only reflects a single wavelength, then it would be red if the wavelength were between 620 and 750nm. The visible spectrum goes from 380 to 750nm, so a uniform distribution over wavelength gives a probability of 0.34.

Most objects reflect multiple wavelengths though. In this case, the color of an object could be characterized as a distribution over the visible spectrum of the reflected light. To count as red, the distribution would need a mean between 620 and 750nm. We might need an additional constraint on the shape of the distribution so it isn't too spread out. I don't know how you'd begin calculating the measure of distributions that meet these constraints though.

I think drawing a red bead will marginally increase our probability of red.

In the end, I think our estimate is much more likely to depend on our expectation of Omega's motives than knowledge about colors, though.

I'm sure not sure I understand the point of this post. Are you saying that guesses without any information are inherently unfounded?

How would guessing 50% on the first pull be any worse since, by definition of the problem, you have no information? As soon as you have seen one bead, however, you have perfect historical information which is better than none.

Assuming that Omega is picking at random, it makes sense to me to simply pick a random percentage on the first pull and then swing to 0% or 100% once you see a bead. Update again on the second bead to 0%, 50%, or 100%. Continuing the process should eventually converge on the correct answer.

(Note) If "0" and "100" make you uncomfortable, switch to the appropriate very close value of your choice.

In my opinion, a more interesting question is what game Omega can devise in which revealing your probability estimate is part of the winning strategy. If he asks you to name even odds and bet with him, for example, you could name ridiculous odds if you wanted more money. The only thing I can think of is for Omega to pour out part of the jar and reward you depending on your deviation from the correct percentage.