What Evidence Filtered Evidence?

We really want to know: what are the typical filters applied in particular areas of life, and thus what evidence does testimony there give us? Doctors, lawyers, parents, lovers, teachers and so on - what filters do they collectively produce on the evidence they get?

We had a related discussion my blog a little while ago - your expert input would be most welcome.

Each statement that he makes is valid evidence - how could you not update your probabilities? ... But then the clever arguer can make you believe anything he chooses, if there is a sufficient variety of signs to selectively report. That doesn't sound right.

What's being overlooked is that your priors before hearing the clever arguer are not the same as your priors if there were no clever arguer.

Consider the case if the clever arguer presents his case and it is obviously inadequate. Perhaps he refers to none of the usual signs of containing a diamond and the signs he does present seem unusual and inconclusive. (Assume all the usual idealizations, ie no question that he knows the facts and presents them in the best light, his motives are known and absolute, he's not attempting reverse psychology, etc) Wouldn't it seem to you that here is evidence that box B does not contain the diamond as he says? But if no clever arguer were involved, it would be a 50/50 chance.

So the prior that you're updating for each point the clever arguer makes starts out low. It crosses 0.5 at the point where his argument is about as strong as you would expect given a 50/50 chance of A or B.

What lowers it when CA begins speaking? You are predictively compensating for the biased updating you expect to do when you hear a biased but correct argument. (Idealizations are assumed here too. If we let CA begin speaking and then immediately stop him, this shouldn't persuade anybody that the diamond is in box A on the grounds that they're left with the low prior they start with.)

The answer is less clear when CA is not assumed to be clever. When he presents a feeble argument, is it because he can have no good argument, or because he couldn't find it? Ref "What evidence bad arguments".

So the prior that you're updating for each point the clever arguer makes starts out low. It crosses 0.5 at the point where his argument is about as strong as you would expect given a 50/50 chance of A or B.

I don't believe this is exactly correct. After all, when you're just about to start listening to the clever arguer, do you really believe that box B is almost certain not to contain the diamond? Why would you listen to him, then? Rather, when you start out, you have a spectrum of expectations for how long the clever arguer might go on - to the extent you believe box A contains the diamond, you expect box B not to have many positive portents, so you expect the clever arguer to shut up soon; to the extent you believe box B contains the diamond, you expect him to go on for a while.

The key event is when the clever arguer stops talking; until then you have a probability distribution over how long he might go on.

The quantity that slowly goes from 0.1 to 0.9 is the estimate you would have if the clever arguer suddenly stopped talking at that moment; it is not your actual probability that box B contains the diamond.

Your actual probability starts out at 0.5, rises steadily as the clever arguer talks (starting with his very first point, because that excludes the possibility he has 0 points), and then suddenly drops precipitously as soon as he says "Therefore..." (because that excludes the possibility he has more points).

I mostly concur, but I think you can (and commonly do) get some "negative" information before he stops. If CA comes out with a succession of bad arguments, then even before you know "these are all he has" you know "these are the ones he has chosen to present first".

I know that you know this, because you made a very similar point recently about creationists.

(Of course someone might choose to present their worst arguments first and delay the decent ones until much later. But people usually don't, which suffices.)

g, agreed.

I don't believe this is exactly correct. After all, when you're just about to start listening to the clever arguer, do you really believe that box B is almost certain not to contain the diamond?

Where do you get that A is "almost certain" from? I just said the prior probability of B was "low". I don't think that's a reasonable restatement of what I said.

Your actual probability starts out at 0.5, rises steadily as the clever arguer talks (starting with his very first point, because that excludes the possibility he has 0 points), and then suddenly drops precipitously as soon as he says "Therefore..." (because that excludes the possibility he has more points).

It doesn't seem to me that excluding the possibility that he has more points should have that effect.

Consider the case where CA is artificially restricted to raising a given number of points. By common sense, for a generous allotment this is nearly equivalent to the original situation, yet you never learn anything new about how many points he has remaining.

You can argue that CA might still stop early when his argument is feeble, and thus you learn something. However, since you've stipulated that every point raises your probability estimate, he won't stop early. To make an argument without that assumption, we can ask about a situation where he is required to raise exactly N points and assume he can easily raise "filler" points.

ISTM at every juncture in the unrestricted and the generously restricted arguments, your probability estimate should be nearly the same, excepting only that you need compensate slightly less in the restricted case.

Now, there is a certain sense of two ways of saying the same thing, raising the probability per point (presumably cogent) but lowering it as a whole in compensation.

But once you begin hearing CA's argument, you know tautologically that you are hearing his argument, barring unusual circumstances that might still cause it not to be fully presented. I see no reason to delay accounting that information.

Tom, if CA's allotment of points is generous enough that the limit makes little difference then it's no longer true that "you never learn anything new about how many points he has remaining" because he'll still stop if he runs out.

If he knows that he's addressing Eliezer and that Eliezer will lower his probability estimate when CA stops, then indeed he'll carry on until reaching the limit (if he can), but in that case what happens is that as he approaches the limit without having made any really strong arguments Eliezer will reason "if the diamond really were in box B then he'd probably be doing better than this" and lower his probability.

Suppose you meet CA, and he says "I think you should think the diamond is in box B, and here's why", and at that instant he's struck by lightning and dies. Ignoring for the sake of argument any belief you might have that liars are more likely to be smitten by the gods, it seems to me that your estimate of the probability that the diamond is in box B should be almost exactly 1/2. (Very slightly higher, perhaps, because you've ruled out the case where there's no evidence for that at all and CA is at least minimally honest.)

Therefore, your suggestion that you lower your probability estimate as soon as you know CA is going to argue his case must be wrong.

What actually happens is: after he's presented evidence A1, A2, ..., Ak, you know not only that A1, ..., Ak are true but also that those are the bits of evidence CA chose to present. And you have some idea of what he'd choose to present if the actually available evidence were of any given strength. If A1, ..., Ak are exactly as good as you'd expect given CA's prowess and perfectly balanced evidence for the diamond's location, then your probability estimate should remain at 1/2. If they're better, it should go up; if they're worse, it should go down.

Note that if you expect a profusion of evidence on each side regardless, k will have to be quite large before good evidence A1 ... Ak increases your estimate much. If that's the case, and if the evidence really does strongly favour box B, then a really clever CA will try to find a way to aggregate the evidence rather than presenting it piecemeal; so in such situations the presentation of piecemeal evidence is itself evidence against CA's claim.

G, you're raising points that I already answered.

Only in the sense that you've said things that contradict one another. You said that knowing that you're listening to CA modifies your prior estimate of P(his preferred conclusion) from the outset, and then you said that actually if you stop him speaking immediately then your prior shouldn't be modified. These can't both be right.

I don't see any way to make the "modified prior" approach work that doesn't amount to doing the same calculations you'd do with the "modified estimation of evidence provided by each point made" approach and then hacking the results back into your prior to get the right answer, and I don't see any reason for preferring the latter.

Of course, as a practical matter, and given the limitations of our reasoning abilities, prior-tweaking may be a useful heuristic even though it sometimes misleads. But, er, "useful heuristics that sometimes mislead" is a pretty good characterization of what's typically just called "bias" around here :-).

"Has it ceased to be true that, in such-and-such a proportion of Everett branches or Tegmark duplicates in which box B has a blue stamp, box B contains a diamond?"

I am baffled as to why a person who calls himself a Bayesian continually resorts to such frequentist thinking. And in this case, it's spectacularly inappropriate frequentist reasoning. Unless someone used a Geiger counter to decide whether or not to put a diamond in the box, quantum-level uncertainty is utterly irrelevant to this problem.

In a way, no one can really trust the theory of natural selection until after they have listened to creationists for five minutes; and then they know it's solid.

You tread on dangerous ground here. Shouldn't the detail & scope of its predictions (the rent) be the criterion by which we evaluate any theory? Though creationists' poor arguments may be suggestive of the indefensibility of their position, this alone does not prove them wrong, and certainly does not confirm evolution.

On a game show, you are given the choice of three doors leading to three rooms. You know that in one room is $100,000, and the other two are empty. The host asks you to pick a door, and you pick door #1. Then the host opens door #2, revealing an empty room. Do you want to switch to door #3, or stick with door #1?

The answer depends on the host's algorithm. If the host always opens a door and always picks a door leading to an empty room, then you should switch to door #3. If the host always opens door #2 regardless of what is behind it, #1 and #3 both have 50% probabilities of containing the money. If the host only opens a door, at all, if you initially pick the door with the money, then you should definitely stick with #1.

Which means that (when Monty's algorithm isn't given or when there's uncertainty about how accurate the problem statement is) people who don't switch are making a very defensible choice by the laws of decision theory. For what plausible reason would he (open a door and) offer to let you switch unless he stood to gain if you did? (Answer: To trick you on the meta level, of course.)

I'm not sure if I can parse the title correctly:

What Evidence Filtered Evidence?

I understand it to mean: "What evidence are you using to filter the evidence". Is that correct?

plus the fact of the host choosing to open door #2.

As long as I don't know his motives (or on what level the host is playing, to put it in HPMOR terms) I can't infer anything from what the host does. He might have opened door 2 because the money isn't behind door 1 and I get another chance. Or because it is behind 1 and he wants me to switch so the company can keep the money. Knowing I should integrate his motives into the equation doesn't mean I can.

Or am I missing something essential here?

I'm trying to incorporate this with conservation of expected evidence: http://lesswrong.com/lw/ii/conservation_of_expected_evidence/

For example: "On average, you must expect to be exactly as confident as when you started out. Equivalently, the mere expectation of encountering evidence—before you've actually seen it—should not shift your prior beliefs." -Eliezer_Yudkowsky AND "Your actual probability starts out at 0.5, rises steadily as the clever arguer talks (starting with his very first point, because that excludes the possibility he has 0 points)" -Eliezer_Yudkowsky

Appear to be contradictions, given that each point= a piece of evidence (shininess of box, presence of blue stamp, etc).

The cherry picking problem appears to be similar to the witch trial problem. In the latter any piece of evidence is interpreted to support the conclusion, while in the former evidence is only presented if it supports the conclusion.

You can't expect your probabilities to on average be increased before seeing/hearing the evidence.

I think only if you do have a large background of knowledge, with a high probability that you are already aware of any given piece of evidence. But if you hear a repeat evidence, it simply shouldn't alter your probabilities, rather than lower it. I'm having a hard time coming up with a way to properly balance the equation.

The only thing I can think of is if you count the entire argument as one piece of evidence, and use a strategy like suggested by g for updating your priors based on the entire sum?

But you don't necessarily listen to the entire argument. Knowing about hypothetical cut off points below which they wont spend the time to present and explain evidence means with enough info you could still construct probabilities. If time is limited, can you update with each single piece of evidence based on strength relative to expected?

What if you are unfamiliar with the properties of boxes and how they are related to likelihood of the presence of a diamond? Any guesstimates seem like they'd be well my abilities at least.

Unless I already know a lot, I have a hard time justifying updating my priors at all based on CA's arguments. If I do know a lot, I still can't think of a way to justifiably not expect the probability to increase, which is a problem. Help, ideas?

PS. Thankfully not everyone is a clever arguer. Ideally, scientists/teachers teaching you about evolution (for example) will not be selective in giving evidence. The evidence will simply be lopsided because of nature being lopsided in how it produces evidence (entangled with truth). I don't think one has to actually listen to a creationist, assuming it is known that the scientists/source material the teacher is drawing from are using good practice.

Also, this is my first post here, so if I am ignorant please let me know and direct me to how I can improve!

What my question was about was filtered evidence.
Filtered evidence is a special case, in which that entanglement between the source and matter of the belief = 0.
Using Eliezer_Yudkowsky's example from "the bottom line":
http://lesswrong.com/lw/js/the_bottom_line/
The only entanglement between the claim of which sort of evidence I will be presented with, "box B contains the diamond!", is with whether the owner of box A or box B bid higher (owner of box B, apparently).

Not with the actual content of those boxes (unless there is some relation between willingness to bid and things actually entangled with presence/absence of a diamond).


Therefore, being told of this will not alter my belief about whether or not box B is the one that contains the diamond.
This is the lead-in to the questions posed in my previous/first post.

Knowing that the source is filtered, every single piece of evidence he gives you will support the conclusion the diamond is in box B. Yet you simply cannot expect every single piece of evidence to on average increase your belief that it is in box B.
While the arguments are actually entangled, their selective presentation means P(E)=1 and P(~E)=0.
You can't balance the equations other than by not modifying your beliefs at all with each piece of evidence.
Probability of the diamond being in box B = probability of the diamond being in box B given that you are shown evidence, meaning the evidence doesn't matter.
The direction of the evidence that passes through the filter (the clever arguer) is entangled with which box-owner bid more money. Not with which box actually contains the diamond.
Thus, it sounds like you simply should not modify your beliefs when faced with a clever arguer who filters evidence. No entanglement between evidence's direction and the truth/conservation of expected evidence.

My problem: not taking into account evidence entangled with reality doesn't sit well with me. It sounds as though it should ultimately be taken into account, but I can't immediately think of an effective process by which to do it.

This post and The Bottom Line previously, are extremely useful for cases where someone is trying to convince you of something you know very little about. Advertising seems like the most common example, although media coverage on obscure topics may fall into this category if you don't bother to look at the other side of the issue. This only applies in instances where two elements are present: (1) the media is motivated to bias their account (perhaps because people prefer news sources that confirm what they already believe, or because people prefer sensationalized stories over watching grass grow) AND (2) the subject must be obscure enough that you only encounter information about it from a single source, or from additional sources with identical biases.

If either of these elements is missing, however, you can't discard all that evidence as part of a selection bias. It may still be subject to some media bias, but you'd have to supply a counterargument to justifiably claim media bias, rather than being able to hypothesize that there exists a large body of data opposing the arguments viewpoint.

Another way of reading this might be to always check at least 2 sources with substantially different viewpoints if you want to be reasonably sure of a fact. Anything those sources have in common might also bias your information, although it may just add random noise in some cases, rather than true selection bias.

No, you don't need update you assumption. If clever arguer choose to argue about what box is contained a diamond - and not bet his own money on that..... It is sure sign that he have absolutely no idea about this, so all his speeches also just can't contain a usefull information, only total bullshit. It is like updating your beliefs about future fliping a coin. Coin just don't contain information about future- therefore useless for predicting. Also with clever arguer.

I try put it in other words. Arguer is clever. He doesn't sure what box is containing a diamond- i.e. he believe in 50/50. Else- he just bouth box, that he think contain diamond. He has a more information about box, then you. So, how you can think that you have more certain data, that one box contain a diamond -than arguer, if you have less information than he?

Also, I wonder - if somebody hired two clever arguers, one of them will persuaded one person, that diamond in the left box, and the other will argue to second person that diamond in the right box. And clever arguers is so good, that they victims almost sure in that... Isn't it almost as creating new diamond out of air ?