Navigating disagreement: How to keep your eye on the evidence

Anyone who hasn't already, check out Anna's OB post, Share likelihood ratios, not posterior beliefs.

(Anna: you write lots of great stuff; link it up!).

It was written about a year ago, but it's actually a good follow up to this post. The point is that, ideally, people would share raw observations. But sometimes that's too slow, so instead we should share a form of summarized evidence. Sharing opinions is a noisy way to do that, because other peoples' prior beliefs get needlessly mixed in with the observations, and then with your opinion, just like the sort of agreement ritual Anna describes here.

It's much better if rational people share Bayes factors from independent tests. That is, you ask your friends "By what multiplicative factor did your priors increase"? Anna gives a rough example of two rational friends with cynical and optimistic priors, who know a third party John in different contexts (i.e. substantially independent tests). If the optimist says "John is a terrible person", the cynic, knowing the optimist's priors, can tell there must have been a significant update, i.e. a large Bayes factor, hence significant evidence, that John is really a terrible person. But if the cynic said that, the optimist wouldn't learn much.

This doesn't work as easily if you share common observations with your friends (rendering your tests non-independent) and consider your character judgement updates relatively unsusceptible to computational errors. In that case, you have to abstract what parts of your current opinion comes from the unshared observations, estimate the Bayes factor (significance of evidence) for those observations only, and share those factors instead. Or just resort to describing the unshared observations explicitly.

Let me give an argument in favor of #4, doing what the others do, in the thermometer problem. Now we seem to have them behaving badly. I think in practice many people would in fact look at other thermometers too in making their guesses. So why aren't they doing it? Two possibilities: they're stupid; or they have a good reason to do it. An example good reason: some thermometers don't read properly from a side angle, so although you think you can see and read all of them, you might be wrong. (This could be solved by #3, writing down the average of the cards, but that doesn't work if everyone tries it since everyone is waiting for everyone else to go first.)

Only if we add a stipulation to this problem, that you are usually right when everyone else is wrong, would it be a good idea to buck the crowd. And even then there is the danger that the others may have some private information that supports their seemingly illogical actions.

“is an accurate belief” is a property of the belief only

Technically, it's a property of the (belief, what-the-belief-is-about) pair. Beliefs can't be accurate by themselves; there has to be an external referent to compare them with. (Only-)self-referencing beliefs degrade straighforwardly to tautologies or contradictions.

The thermometer answer is wrong, you're ignoring that you're on a game show On a game show the producers try to organize things such that few people (or only one person) wins a challenge. As such I would expect all but one thermometer to be in error. Furthermore by watching old episodes of the show I could tell if only one thermometer will be right or if several contestants also succeed at each challenge and therefore either pick the small clump or the lone outlier.

You’re asked to estimate the number of jelly-beans in a jar. You have a group of friends with you. Each friend privately writes down her estimate, then all of the estimates are revealed, and then each person has the option of changing her estimate.

How should you weigh: (a) your own initial, solitary estimate; (b) the initial estimates of each of your friends; (c) the estimates your friends write down on paper, after hearing some of the others’ answers?

I start by asking them how they made their initial estimates, and how they used others'.

This might seem ridiculously obvious, or outside the intent of the thought experiment, and maybe it is and this comment is needless noise. But, I'm sort of worried that the focus on individual estimation and guessing about others' belief-formation processes that I see in most of the discussion here, might lead us to want to overuse our cool individual estimation techniques when asking people about their opinions is easy and a better choice. I'm not sure how well-founded that worry is, but in any case, I'd like to see more discussion on LW of group truth-seeking; and if nothing else, it seems like a good idea to explicitly prime the thought "ask people what they think!".

You should consider all the information you have, from all the (randomly allocated, and so informationally symmetric) thermometers. It doesn’t matter who was handed which thermometer. Forming accurate beliefs is normally about this simple. If you want the most accurate beliefs you can get, you’ll need to pay attention to the evidence. All of the evidence. Evenly.

This gives the impression that you think that normally one can just collect all the relevant evidence, after which you don't need to consider anyone else's opinion. I suppose it depends on what sort of world you live in, but that seems far from the normal situation to me.

Let’s say I want to know whether it’s safe for my friend to bike to work. My own memories are truth indicative, but so are my friends’ and neighbors [and online surveys]... The trouble is my own memories arrive in my head with extreme salience, and move my automatic anticipations a lot; while my friend’s have less automatic impact, and those of the surveyed neighbors still less...our automatic cognition tends not to weigh the evidence evenly at all. <

I sometimes wonder, though, if giving one's own experiences greater weight in situations like these (though not in the thermometer situation) is rational:

• People lie (especially in online surveys); first hand evidence should be more valuable than evidence whose validity is in question
• There are a large number of unknown and unanalyzed factors, some of which may vary with the individual: (I'm less/more coordinated and accident-prone, I am on better/worse terms with the rough crowd in the neighborhood, etc). This information may not be obvious enough to consciously consider.

If I have a sneezing fit every single time I encounter a bullfrog, and no one's ever heard of a bullfrog allergy, and medical science doesn't support the notion, it's still going to be difficult (and I think possibly irrational) to arrive to the pond without a kleenex. It seems to me that in gray-area situations with strong personal evidence, admitting you don't know why you don't know why is at least as rational as concluding you're wrong.

My take on how to get the best estimates, in separate comments for tidier discussion threads:

Should you discount multiple opinions which are based on the same information source?

Therein lies a tail... Even in the first, thermometer, problem, there is still a question about whether to average or to take the median. Roughly speaking, if one expects some form of independent random additive noise in each thermometer, the choice of what to do with outliers depends on what one's prior for the expected noise distribution looks like. If one expects a gaussian, the variance of the distribution is finite, and one does better by averaging the readings. If one expects a distribution with long tails, with an unbounded variance, then one wants to pick the estimate more nearly from the median. Intermediate choices include throwing out some outliers and averaging the remaining samples or ranking the samples, then doing a weighted average of the samples based on how far from the median rank they are. A nice example for the Cauchy distribution is at http://kmh-lanl.hansonhub.com/publications/maxent93.pdf

Problem 1: Jelly-beans

I entered such a contest as a child. I mentally divided the jar into its different conic sections, took a string and measured each section in units of jelly beans, then computed its volume in jelly beans.

I came in second. There's always someone whose wild guess is better than your best calculations.

Problem 1 is basically a noisy thermometers problem, except that the noise is gaussian in the estimate of the length/density along one dimension, not the number given. So I would take the cube root of each answer (including my own), then average them, then cube the result to make my estimate. If I thought one person was a particularly good or bad estimator, I would apply that as a weighting in the middle step.

For problem 2, the answer is that you should be able to test whether you are upset directly, using introspection (perhaps combined with a mental trick or two), and if you do it right, the result of this test should be much better evidence of your mental state than your roommate's observation would be. However, performing this test is a skill and the problem description doesn't mention having it. So if you've explicitly practiced inspecting your mental state, then you should mostly ignore your roommate, but if you haven't then you should listen to him.

(But do note that the question of what your mental state is, is entirely separate from the question of whether you've been doing your fair share of the chores. That question can only be answered by actually enumerating a sample of the chores, partitioning it into the set you did and the set he did, and comparing the set sizes.)

(Written before and edited where marked after reading the comments.)

1. I look at the jar and estimate how many jellybabies wide and tall is the filled volume, do some mental arithmetic, and reduce the answer according to an estimate of the packing fraction, which I would expect to be the greatest source of error.

Anyone else can do the same thing, if they're smart enough to not just pull a figure out of the air. If I think the other contestants are unlikely to be, I ignore them. It's like the thermometer example, except that I have reason to think my thermometer is better than most other people's.

If I think that everyone else is smart enough to make an estimate along those lines, then it is more like the original thermometer problem. But I'm not going to just take an average. If my estimate is 1000, and someone else's is 300, that's too big a discrepancy to explain by minor variations. It casts doubt on the assumption of identical thermometers. Assuming that I only have the other people's estimates, and there's no opportunity for discussion, I'll search for reasons why we might have come up with completely different answers, but if I find no error in my own, I'll discard all such outliers.

If I work in the confectionery trade and know the packing fraction for jellybabies, that elevates my confidence in my own estimate and again I ignore the others...unless this competition is being held at a confectionery trade show.

In general, averaging the estimates is only valid if the estimates are believed to be of similar worth. If you know that the estimates are all unbiased but with differing variances, then you can work out some optimally weighted average that puts more weight on the more accurate estimates but does not discard the less accurate ones. However, if estimates are wildly different, the assumption may be a bad one.

BTW, a real-world example is the assessment of conference papers by the programme committee. Each paper will have been refereed by, say, four members. The typical procedure is that if they all say it's excellent, it's accepted without discussion. Likewise, reject it if they all say it's rubbish. For uniformly middling assessments, the question is where to set the bar for acceptance. The only papers where a real discussion is required are the ones where the referees disagree. The disagreements are resolved by sharing evidence, not by an Aumann-like compromise based on sharing posteriors.

2. Recognise that whoever is at rational fault, agreement is not possible in the current state of things. Start recording who does what housework when, then return to the matter after some suitable time, with evidence to decide the issue.

3. Case 2 was set up to be symmetrical. Case 3 is different: rationality is right and religion is wrong. I continue in that belief. How I conduct my discussions thereafter with my religious friend is a separate matter.

4. I'm not sure how much rational perfection and common knowledge to assume of Alfred and Betty in this problem, but even if I assume that they are perfect reasoners with common priors, then I can't see my way to proving anything about the ordering of their second estimates. (Added after reading the comments: Alfred or Betty's estimate of the probable ordering of their second estimates is a different matter.) I suppose that some version of Aumann's theorem says that on iterating the process they must eventually converge.

I've been thinking about something like this.

If there's two people arguing, there's a 50% chance a given one is right. There is generally no reason to believe the the correct one happens to be you.

That only really adds a bit of evidence against you, which doesn't seem like much. That said, if the other person realizes this, and doesn't change their mind, their evidence was one bit stronger than you previously thought.

Furthermore, if they realize that you know that and haven't changed your mind, and they still don't change their mind, that adds another bit.

Etc.

If both people understand this, they will either immediately figure that the one who's more sure is probably right, or just not be sure.

Therein lies a tail... Even in the first, thermometer, problem, there is still a question about whether to average or to take the median. Roughly speaking, if one expects some form of independent random additive noise in each thermometer, the choice of what to do with outliers depends on what one's prior for the expected noise distribution looks like. If one expects a gaussian, the variance of the distribution is finite, and one does better by averaging the readings. If one expects a distribution with long tails, with an unbounded variance, then one wants to pick the estimate more nearly from the median. Intermediate choices include throwing out some outliers and averaging the remaining samples or ranking the samples, then doing a weighted average of the samples based on how far from the median rank they are. A nice example for the Cauchy distribution is at http://kmh-lanl.hansonhub.com/publications/maxent93.pdf

What's stopping you from saying "Hey, why don't we average our thermometers?" You should at least see if they're updating on your suggestion for proper procedure before you update on their default procedure.

No real beef with the main issue, but as for the Extra Credit Problems:

1. I imagine I would weigh it depending on the group of friends I had with me, and previous experience with each of them in the field of guessing, geometry, even basic arithmetic. After I considered all that, I would then adjust my answer depending on each one's credibility.

2. Think back on what I've done compared to what he's done. His emotional concern is actually irrelevant, because its entirely possible he has gotten upset for reasons such as the dishes are supposed to be done on a rotation, but he uses more dishes when he knows its your turn, while yours stay consistent. However, the reverse is possible. Depending on the situation, a renegotiation of housework may be necessary, so I would suggest an accurate tracking of use to work ratios.

3. According to the question, there is no right answer, as you're both arguing opinions, and like the haircut, they cannot be resolved. However, this question is heavily biased. Are there no such things as rational Christians? Carl Sagan and Stephen Hawking have both made convincing arguments stating that a rational person should have difficulty declaring atheism (For Carl Sagan's especially well thought out and rational perspective, read "The Demon Haunted World"). He is wrong, but it is entirely possible you are biased, from the Dawkin's School of Atheist Fundamentalism. However, as you're the rational one, the burden of proof is on you, not him. We should always ask for rationality from our discussion partners, but it is irrational to demand a higher standard of them than of ourselves. If we are unwilling to consider that we may be wrong, how can we expect them to reciprocate?

It's a complicated idea, because his methodology is so fundamentally unsound. However, as an Atheist as opposed to an agnostic, you have declared there is absolutely no higher being. If you are every bit as unwilling to examine your own beliefs as he is, you are no less fundamentalist, and you are therefore both biased. A good skeptic should follow both the scientific and Socratic method's, being equally capable of proving his own hypothesis (in this case, that God does not exist), and of understanding his opponents argument as if it were his own. This is a case of universal absolutes, as opposed to the previous problem, which was about temporal difficulties. A non-fundamentalist atheist, I.E., a rational Atheist, must be willing to truly understand the other side's viewpoint, because otherwise they are as equally guilty as their religious fundamentalist counterpart. Or worse, as a religious person who has studied science and can legitimately argue against your claims is actually being more rational than you, who has chosen to do no research, and to consider nothing.

Sorry that was so long, but it was requested we search the questions themselves for bias. Is there actually any reason there can not be a Christian and an Atheist with an Equal level of Rationality? If so, why was this question worded towards a solely Christian extreme, portraying the Atheist as rational while the Xtian is not? Once again, for emphasis, from the way this question is worded, neither is rational, and both are being depressingly biased.

4. It is more likely Alfred's will be smaller, although here is a chance Betty may increase her estimate as well. They should both eventually stabilize, however, for no other reason than because they both know that Betty is more knowledgeable, so if she sticks to an answer, it has a better chance of being correct.