Updating, part 1: When can you change your mind? The binary model

What matters isn't so much finding the right answer, as having the right approach.

At least as far as I'm concerned, that's the main reason to spend much time here. I don't care whether the answer to Sleeping Beauty is 1/2 or 1/3, that's a mere curio.

I care about the general process whereby you can take a vague verbal description like that, map it into a formal expression that preserves the properties that matter, and use that form to check my intuitions. That's of rather more value, since I might learn how my intuitions could mislead me in situations where that matters.

Is there any real-group analog to the answer to problem t becoming mutual knowledge to the entire group? I can't think of a single disagreement here EVER to which the answer has been revealed. Further, I don't expect much revelation until Omega actually shows up.

The Sleeping Beauty problem and the other "paradoxes" of probability are problems that have been selected (in the evolutionary sense) because they contain psychological features that cause people's reasoning to go wrong. People come up with puzzles and problems all the time, but the ones that gain prominence and endure are the ones that are discussed over and over again without resolution: Sleeping Beauty, Newcomb's Box, the two-envelope problem.

So I think there's something valuable to be learned from the fact that these problems are hard. Here are my own guesses about what makes the Sleeping Beauty problem so hard.

First, there's ambiguity in the problem statement. It usually asks about your "credence". What's that? Well, if you're a Bayesian reasoner, then "credence" probably means something like "subjective probability (of a hypothesis H given data D), defined by p(H|D) = p(D|H) p(H) / p(D)". But some other reasoners take "credence" to mean something like "expected proportion of observations consistent with data D in which the hypothesis H was confirmed".

In most problems these definitions give the same answer, so there's normally no need to worry about the exact definition. But the Sleeping Beauty problem pushes a wedge between them: the Bayesians should answer ½ and the others ⅓. This can lead to endless argument between the factions if the underlying difference in definitions goes unnoticed.

Second, there's a psychological feature that makes some Bayesian reasoners doubt their own calculation. (You can try saying "shut up and calculate" to these baffled reasoners but while that might get them the right answer, it won't help them resolve their bafflement.) The problem somehow persuades some people to imagine themselves as an instance of Sleeping Beauty selected uniformly from the three instances {(heads,Monday), (tails,Monday), (tails,Tuesday)}. This appears to be a natural assumption that some reasoners are prepared to make, even though there's no justification for it in the problem description.

Maybe it's the principle of indifference gone wrong: the three instances are indistinguishable (to you) but that doesn't mean the one you are experiencing was drawn from a uniform distribution.

I would like you to publish any results you may generate with your script, and promise to upvote them even if the results do not prove anything, as long as they are presented roughly as clearly as this post is.

So... why does this post have such a low rating? Comments? I find it bewildering. If you're interested in LessWrong, you should be interested in finding out under what conditions people become less wrong.

Re: "I had expected that people would read posts and comments by other people, and take special note of comments by people who had a prior history of being right, and thereby improve their own accuracy."

FWIW, I think that was how I originally approached the problem. Rather than trying to solve it directly, I first looked at your response, and Robin Hanson's response. After a little reflection, I concluded that you agreed with each other - and that you had both focused on the key issues - and got the answer right.

At that time, most of the rest of the thread was people saying the problem was ambiguous and needed a bet to clear it up - and a fair bit of confusion - with very little defense of the standard answer.

This is a really interesting topic, there are heaps of things I want to say about it. I was initially waiting to see what your results were first, to avoid spoilers with my guesses, but that's no way to have a conversation.

First - I think there's an error in the program: When you compute p[i][j] you take a sum then divide by N, but it looks like you should divide by the number of guesses you are adding, which can be more than N since it includes multiple rounds of guesses.

My (inconsistent) thoughts about how the model would behave:

• They'd quickly learn the ratio of correct initial guesses everyone had, and make near-perfect use of that information. But they don't distinguish between the initial guesses and later updates, so that's not right.

• Even the bad guessers will get most of their updated estimates right by the end, so their opinions will be assumed to correlate with the truth. If you then went back and posed everyone a new question, all the bad guessers could significantly mislead everyone. That's not the procedure in your code, but you could try it.

• At the start of the simulation, all the guessers are simply seeing who else agrees with them. The good guessers might be converging to a correct consensus, while the bad guessers could converge to the opposite. But as the simulation progressed and the answers were revealed, the bad guessers would lose confidence in their whole subgroup, including themselves, and follow the good guessing group.

Ideas for variants:

• Make the initial guess accuracy depend on both guesser accuracy and problem difficultly/deceptiveness. I proposed a formula for this in my previous comment. In this case, the best way to update from the initial guesses would seem to be to follow the average opinion of a few of the best guessers and maybe the reverse of the worst few guessers, but I'm not sure how it would play out in the simulation where you don't know who they are, and you have to update on each other's updated guesses.

• Make the initial guess accuracy depend on both the skill of the guesser and the difficulty of the question, but vary what weight is given to skill - some questions can be just as hard for skilled guessers as everyone else. In this case, a way to update from an initial guess would be to look at enough of the best guessers that you're confident which way they guess on average (you'd need to sample more if they are near 50%)

• Repeat the exercise - after the first set of N answers are revealed, continue with N more questions. This time the guessers start with data about each other's accuracy. Then after they are done, N more, etc.

• Instead of everyone getting the same number of updates, let some update more often.

• Instead of updating everyone and revealing one answer each round, randomly pick between updating a random person and randomly revealing a correct answer just to one person, which they will be certain of for the rest of the game. You could give different people different chances of updating from group opinions, and of getting the correct answer revealed. Since people don't know who's had what answers revealed they don't stop counting them when evaluating each other's accuracy.

The Sleeping Beauty Challenge

Maybe I'm naive, but I actually think that we can come close to consensus on the solution to this problem. This is a community of high IQ, aspiring rationalists.

I think it would be a good exercise to use what we know about rationality, evidence, biases, etc. and work this out.

I propose the following:

1. I will write up my best arguments in favor of the 1/2 solution. I'll keep it shorter than my original post.

2. Someone representing the thirders will write up their best arguments in favor of the 1/3 solution

3. Before reading the others' arguments, we will assume that they are right, and that reading it will only confirm our beliefs (this is hard to do, but I find that this approach can be helpful)

4. We cannot respond for at least 24 hours. (this will give us time to digest the arguments, without just reacting immediately)

5. We will then check to see if there is agreement

6. If we still disagree, we can have some discussion (say, via email) to see if progress can be made

7. We will post our original two arguments and conclusion here (maybe in a new post)?

What do you think?

I tried to set this up in such a way to reduce some of the known biases that prevent agreement. Am I missing something?

Possible pitfall: if we come to an agreement, people who disagree with our conclusion might say it's because one of us was a poor representative of their viewpoint. However, I think we'd still move a step towards consensus.

What say you?

from random import shuffle, randint

flips=1000

HEADS=0
TAILS=1

# individual cells
HMW = HTW = HMD = HTD = 0.0
TMW = TTW = TMD = TTD = 0.0

def run_experiment():
    global HMW, HTW, HMD, HTD, TMW, TTW, TMD, TTD

    coin = randint(HEADS,TAILS)

    if (coin == HEADS):
      # wake Beauty on monday
      HMW+=1
      # drug Beauty on Tuesday
      HTD+=1

    if (coin == TAILS):
      # wake Beauty on monday
      TMW+=1
      # wake Beauty on Tuesday too
      TTW+=1

for i in range(flips):
  run_experiment()

print "Total samples where heads divided by total samples ~P(H):",(HMW+HTW+HMD+HTD)/(HMW+HTW+HMD+HTD+TMW+TTW+TMD+TTD)
print "Total samples where woken F(W):",HMW+HTW+TMW+TTW
print "Total samples where woken and heads F(W&H):", HMW+HTW
print "P(W&H)=P(W)P(H|W), so P(H|W)=lim F(W&H)/F(W)"
print "Total samples where woken and heads divided by sample where woken F(H|W):", (HMW+HTW)/(HMW+HTW+TMW+TTW)

> beauty <- function(n) {
+ 
+     # Number of times the coin comes up tails
+     ntails <- 0
+ 
+     # Number of times SB wakes up
+     wakings <- 0
+ 
+     for (i in 1:n) {
+ 
+         # It's Sunday, flip the coin, 0 is heads, 1 is tails
+         coin <- sample(c(0, 1), 1)
+         ntails <- ntails + coin
+ 
+         if (coin == 0) {
+             # Beauty wakes up once, Monday
+             wakings <- wakings + 1
+         } else {
+             # Beauty wakes up Monday, then Tuesday
+             wakings <- wakings + 2
+         }
+     }
+ 
+     return(c(ntails / wakings, ntails / n))
+ 
+ }
> beauty(5)
[1] 0.1666667 0.2000000
> beauty(50)
[1] 0.375 0.600
> beauty(500)
[1] 0.3036212 0.4360000
> beauty(5000)
[1] 0.3314614 0.4958000
> beauty(50000)
[1] 0.3336354 0.5006800

I'd like a variant where there is both person accuracy p[i] and problem easiness E[j], and the odds of person i getting the correct answer initially on problem j are p[i] E[j] : (1-p[i])(1-E[j])

Ideally the updating procedure for this variant wouldn't treat everyone's opinions as independent, but it would also be interesting to see what happens when it mistakenly does treat them as independent.

The poll on the subject:

http://lesswrong.com/lw/28u/conditioning_on_observers/1ztb

...currently has 75% saying 1/3 and 25% saying 1/2. (12:4)

Collective intelligence in action?

In modeling Bayesians (not described here), I have the problem that saying "I assign this problem probabilty .5 of being true" really means "I have no information about this problem."

My original model treated that p=.5 as an estimate, so that a bunch of Bayesians who all assign p=.5 to a problem end up respecting each other more, instead of ignoring their own opinions due to having no information about it themselves.

I'm reformulating it to weigh opinions according to the amount of information they claim to have. But what's the right way to do that?

This suggestion contains a classic bootstrapping problem. If only I knew I was good at statistics, then I'd be confident of analysing this problem which tells me whether or not I'm good at statistics. But since I'm not sure, I'm not sure whether this will give me the right answer.

I think I'll stick to counting.

Comment moved.

I'm interested in hearing others responses to these questions:

What do you think will happen when I run the program, or its variants? What other variants would you like to see tested?

As for this one:

*Is there a fundamental problem with the model?

As you know, that depends on what we want to use the model for. It ignores all sorts of structure in the real world, but that could end up being a feature rather than a bug.