Posts
Comments
If you are not going to do an actual data analysis, then I don't think there is much point of thinking about Bayes' rule. You could just reason as follows: "here are my prior beliefs. ooh, here is some new information. i will now adjust my believes, by trying to weigh the old and new data based on how reliable and generalizable i think the information is." If you want to call epistemology that involves attaching probabilities to beliefs, and updating those probabilities when new information is available, 'bayesian' that's fine. But, unless you have actual data, you are just subjectively weighing evidence as best you can (and not really using Bayes' rule).
The thing that can be a irritating is when people then act as if that kind of reasoning is what bayesian statisticians do, and not what frequentist statisticians do. In reality, both types of statisticians use Bayes' rule when it's appropriate. I don't think you will find any statisticians who do not consider themselves 'bayesian' who disagree with the law of total probability.
If you are actually going to analyze data and use bayesian methods, you would end up with a posterior distribution (not simply a single probability). If you simply report the probability of a belief (and not the entire posterior distribution), you're not really doing conventional bayesian analysis. So, in general, I find the conventional Less Wrong use of 'bayesian' a little odd.
I feel like this creates more misconceptions than it clears up. It's very dismissive of something that is really in the early phases of being studied.
The primary effect that reading this had on me was the change in state from [owning a cloak hadn't occurred to me] to [owning a cloak sounds awesome; i am unhappy that i hadn't thought of it on my own]
I agree. Good point.
The answer to the question "what proportion of phenotypic variability is due to genetic variability?" always has the same answer: "it depends!" What population of environments are you doing this calculation over? A trait can go from close to 0% heritable to close to 100% heritable, depending on the range of environments in the sample. That's a definition problem. Further, what should we count as 'genetic'? Gene expression can depend on the environment of the parents, for example (DNA methylation, etc). That's an environmental inheritance. I just think there is an old way of talking about these things that needs to go away in light of current knowledge.
I agree with you that we still can learn a lot from these studies.
Adoption studies are biased toward the null of no parenting effect, because adoptive parents aren't randomly selected from the population of potential parents (they often are screened to be similar to biological parents).
Twin studies I think are particularly flawed when it comes to estimating heritability (a term that has an incoherent definition). Twins have a shared pre-natal environment. In some cases, they even share a placenta.
Plus, the whole gene vs. environment discussion is obsolete, in light of the findings of the past decade. Everything is gene-environment interaction.
wait, this isn't well done satire?
I don't think the questions even make much sense. We don't live in the world that we once thought we did, where genotype to phenotype results from DNA->RNA->protein model. The real action is in the switches, which are affected by the environment (and so on).
I'm not opposed to ever using terms like "realist." I'm opposed to it as it was used in the main post, where people who agree my views are realists, and people who do not are denialists.
It implies that people who reject their claims are not being real. I want to be a realist, but I certainly have seen no evidence that any particular race is more likely to commit unscrupulous acts if you control for environment (if that was even possible). It's a propaganda term, like '[my cause] realist.'
Downvoted for use of the term 'race realism' (that's verbal bullying).
Because if TDT endorsed the action, then other people would be able to deduce that TDT endorsed the action, and that (whether or not it had happened in any particular case) their lives would be in danger in any hospital run by a timeless decision theorist, and then we'd be in much the same boat. Therefore TDT calculates that the correct thing for TDT to output in order to maximize utility is "Don't kill the traveler," and thus the doctor doesn't kill the traveler.
TDT could deduce that people would deduce that TDT would not endorse the action, and therefore TDT is safe to endorse the action. It seems like the gist of this is: (a) i've decided that killing the traveler is wrong (based on something other than TDT) and (b) TDT should do the right thing.
I upvoted and like this post. Some of it just strikes me as magical
Genes just aren't as much of the story as we thought they were. Whether or not a gene increases fitness might depend on whether it is methylated or not, for example. Until recently, we didn't realize that there could be transgenerational transmittance of DNA methylation patterns due to environmental factors.
And as it turns out, all these predictions are correct.
I think your conclusion is largely correct, but I see a lot of overconfidence here, particularly in the evolutionary psych section. The selish gene theory was a good one, but wrong (see epigenetics).
"The Bridge". There was one person who survived and said he changed his mind once he was airborne. My recollection of the movie is that most of the people who jumped had been wanting to die for most of their lives. Even their family members seemed at peace with it for that reason.
The first one is flawed, IMO, but not for the reason you gave (and I wouldn't call it a 'trick'). The study design is flawed. They should not ask everyone "which is more probable?" People might just assume that the first choice, "Linda is a bank teller" really means "Linda is a bank teller and not active in the feminist movement" (otherwise the second answer would be a subset of the first, which would be highly unusual for a multiple choice survey).
The Soviet Union study has a better design, where people are randomized and only see one option and are asked how probable it is.
You have to realize that a great number of things are discussed in these proceedings that the mind just can't deal with, people are simply too tired and distracted, and by way of compensation they resort to superstition.
-- Kafka, The Trial
Justice is an artefact of custom. Where customs are unsettled its dictates soon become dated. Ideas of justice are as timeless as fashions in hats.
-John Gray, Straw Dogs
Who has not experienced the chilling memory of the better things? How it creeps over the spirit of one's current dreams! Like the specter at the banquet it stands, its substanceless eyes viewing with a sad philosophy the make-shift feast.
-Theodore Dreiser, The Titan
If you look at Table 2 in the paper, it shows doses of each vitamin for every study that is considered low risk for bias. I count 9 studies that have vitamin A <10,000 IU and vitamin E <300 IU, which is what PhilGoetz said are good dosage levels.
The point estimates from those 9 studies (see figure 2) are: 2.88, 0.18, 3.3, 2.11, 1.05, 1.02, 0.78, 0.87, 1.99. (1 favors control)
Based on this quick look at the studies, I don't see any reason to believe that a "hockey stick" model will show a benefit of supplements at lower dose levels.
"And I don't expect I will ever have to do that."
You do not sound 100% certain.
It would be nice if the top scoring all-time posts really reflected their impact. Right now there is some bias towards newer posts. Plus, Eliezer's sequences appeared at OB first, which greatly reduced LW upvotes.
Possible solution: every time a post is linked to from a new post, it gets an automatic upvote (perhaps we don't count it if linked to by same author). I don't know if it's technically feasible
I'd be glad to discuss it.
That would be great. I'd love to see the results.
In the first example, you couldn't play unless you had at least 100M dollars of assets. Why would someone with that much money risk 100M to win a measly 100K, when the expected payoff is so bad?
In cases where a scientist is using a software package that they are uncomfortable with, I think output basically serves as the only error checking. First, they copy some sample code and try to adapt it to their data (while not really understanding what the program does). Then, they run the software. If the results are about what they expected, they think "well, we most have done it right." If the results are different than they expected, they might try a few more times and eventually get someone involved who knows what they are doing.
Good find. Thanks.
Error finding: I strongly suspect that people are better at finding errors if they know there is an error.
For example, suppose we did an experiment where we randomized computer programmers into two groups. Both groups are given computer code and asked to try and find a mistake. The first group is told that there is definitely one coding error. The second group is told that there might be an error, but there also might not be one. My guess is that, even if you give both groups the same amount of time to look, group 1 would have a higher error identification success rate.
Does anyone here know of a reference to a study that has looked at that issue? Is there a name for it?
Thanks
Yes, that's a good point. Tthat would be considered using a data augmentation prior (Sander Greenland has advocated such an approach).
only if you keep specifying hyper-priors, which there is no reason to do
In the second example the person was speaking informally, but there is nothing wrong with specifying a probability distribution for an unknown parameter (and that parameter could be a probability for heads)
Hm, good point. Since the usual thing is .5, the claim should be the alternative. I was thinking in terms of trying to reject their claim (which it wouldn't take much data to do), but I do think my setup was non-standard. I'll fix it later today
Very good examples of perceptions driving self-selection.
It might be useful to discuss direct and indirect effects.
Suppose we want to compare fatality rates if everyone drove a Volvo versus if no one did. If the fatality rate was lower in the former scenario than in the latter, that would indicate that Volvo's (causally) decrease fatality rates.
It's possible that it is entirely through an indirect effect. For example, the decrease in the fatality rate might entirely be due to behavior changes (maybe when you get in a Volvo you think 'safety' and drive slower). On the DAG, we would have an arrow from volvo to behavior to fatality, and no arrow from volvo to fatality.
A total causal effect is much easier to estimate. We would need to assume ignorability (conditional independence of assignment given covariates). And even though safer drivers might tend to self-select into the Volvo group, it's never uniform. Safe drivers who select other vehicles would be given a lot of weight in the analysis. We would just have to have good, detailed data on predictors of driver safety.
Estimating direct and indirect effects is much harder. Typically it requires assuming ignorability of the intervention and the mediator(s). It also typically involves indexing counterfactuals with non-manipulable variables.
as an aside: a machine learning graduate student worked with me last year, and in most simulated data settings that we explored, logistic regression outperformed SVM
In my opinion, the post doesn't warrant -90 karma points. That's pretty harsh. I think you have plenty to contribute to this site -- I hope the negative karma doesn't discourage you from participating, but rather, encourages you to refine your arguments (perhaps get feedback in the open thread first?)
How about spreading rationality?
This site, I suspect, mostly attracts high IQ analytical types who would have significantly higher levels of rationality than most people, even if they had never stumbled upon LessWrong.
It would be great if the community could come up with a plan (and implement it) to reach a wider audience. When I've sent LW/OB links to people who don't seem to think much about these topics, they often react with one of several criticisms: the post was too hard to read (written at too high of a level); the author was too arrogant (which I think women particularly dislike); or the topic was too obscure.
Some have tried to reach a wider audience. Richard Dawkins seems to want to spread the good word. Yet, I think sometimes he's too condescending. Bill Maher took on religion in his movie Religulous, but again, I think he turned a lot of people off with his approach.
A lot has been written here about why people think what they think and what prevents people from changing their minds. Why not use that knowledge to come up with a plan to reach a wider audience. I think the marginal payoff could be large.
But: "You can be a virtue ethicist whose virtue is to do the consequentialist thing to do"
Please elaborate.
Perhaps a better title would be "Bayes' Theorem Illustrated (My Ways)"
In the first example you use shapes with colors of various sizes to illustrate the ideas visually. In the second example, you using plain rectangles of approximately the same size. If I was a visual learner, I don't know if your post would help me much.
I think you're on the right track in example one. You might want to use shapes that are easier to estimate the relative areas. It's hard to tell if one triangle is twice as big as another (as measured by area), but it's easier to do with rectangles of the same height (where you just vary the width). More importantly, I think it would help to show math with shapes. For example, I would suggest that figure 18 has P(door 2)= the orange triangle in figure 17 divided by the orange triangle plus the blue triangle from figure 17 (but where you show the division by shapes). When I teach, I sometimes do this with Venn diagrams (show division of chunks of circles and rectangles to illustrate conditional probability).
It seems to me that the standard solutions don't account for the fact that there are a non-trivial number of families who are more likely to have a 3rd child, if the first two children are of the same sex. Some people have a sex-dependent stopping rule.
P(first two children different sexes | you have exactly two children) > P(first two children different sexes | you have more than two children)
The other issue with this kind of problem is the ambiguity. What was the disclosure algorithm? How did you decide which child to give me information about? Without that knowledge, we are left to speculate.
We should blame and stigmatize people for conditions where blame and stigma are the most useful methods for curing or preventing the condition, and we should allow patients to seek treatment whenever it is available and effective.
I think you said it better earlier when you talked about whether the reduction in incidence outweighs the pain caused by the tactic. For some conditions, if it wasn't for the stigma there would be little-to-nothing unpleasant about it (and we wouldn't need to talk about reducing incidence).
I agree with your general principle, but think it's unlikely that blame and stigma are ever the most useful methods. We should be careful to avoid the false dichotomy between the "stop eating like a pig" tactic and fat acceptance.
Sandy's husband is an asshole, who probably defends his asshole behavior by rationalizing that he's trying to help her. He's not really trying to help her (or if he is, he knows little about psychology (or women)).
Blame and judgment are such strong signaling devices that I think people rarely use it for the benefit of the one being judged. If it happens to be the best tactic for dealing with the problem, well, that would be a quite a coincidence.
--
I liked your post a lot, in case that wasn't clear. I think you are focusing on the right kinds of questions.
Sorry I was slow to respond .. busy with other things
My answers:
Q1: I agree with you: 1/3, 1/3, 2/3
Q2. ISB is similar to SSB as follows: fair coin; woken up twice if tails, once if heads; epistemic state reset each day
Q3. ISB is different from SSB as follows: more than one coin toss; same number of interviews regardless of result of coin toss
Q4. It makes a big difference. She has different information to condition on. On a given coin flip, the probability of heads is 1/2. But, if it is tails we skip a day before flipping again. Once she has been woken up a large number of times, Beauty can easily calculate how likely it is that heads was the most recent result of a coin flip. In SSB, she cannot use the same reasoning. In SSB, Tuesday&heads doesn't exist, for example.
Consider 3 variations of SSB:
Same as SSB except If heads, she is interviewed on Monday, and then the coin is turned over to tails and she is interviewed on Tuesday. There is amnesia and all of that. So, it's either the sequence (heads on Monday, tails on Tuesday) or (tails on Monday, tails on Tuesday). Each sequence has a 50% probability, and she should think of the days within a sequence as being equally likely. She's asked about the current state of the coin. She should answer P(H)=1/4.
Same as SSB except If heads, she is interviewed on Monday, and then the coin is flipped again and she is interviewed on Tuesday. There is amnesia and all of that. So, it's either the sequence (heads on Monday, tails on Tuesday), (heads on Monday, heads on Tuesday) or (tails on Monday, tails on Tuesday). The first 2 sequences have a 25% chance each and the last one has a 50% chance. When asked about the current state of the coin, she should say P(H)=3/8
The 1/2 solution to SSB results from similar reasoning. 50% chance for the sequence (Monday and heads). 50% chance for the sequence (Monday and tails, Tuesday and tails). P(H)=1/2
If you apply this kind of reasoning to ISB, where we are thinking of randomly selected day after a lot of time has passed, you'll get P(H)=1/3.
I'm struggling to see how ISB isn't different from SSB in meaningful ways.
My NT 'data' are from conversations I've had over the years with people who I have noticed are particularly good socially. But of course, there is plenty of between person variability even within NT and AS groups.
The thing that I have been most surprised by is how much NTs like symbols and gestures.
Here are some examples:
Suppose you think your significant other should have a cake on his/her birthday. You are not good at baking. Aspie logic: "It's better to buy a cake from a bakery than to make it myself, since the better the cake tastes the happier they'll be." Of course, the correct answer is that the effort you put into it is what matters (to an NT).
Suppose you are walking through a doorway and you are aware that there is someone about 20 feet behind you. Aspie logic: "If I hold the door for them they will feel obligated to speed up a little, so that I'm not waiting too long. That will just inconvenience them. Plus, it's not hard to open a door. Thus, it's better for them if I let the door close." To the NT, you are just inconsiderate.
Suppose you are sending out invitations to a graduation party. You know that one of your close friends is going to be out of town that weekend. Aspie logic: "There is no reason to send them an invitation, since I already know they can't go. In fact, sending them an invitation might make them feel bad." If your friend is an NT, it's the wrong answer. They want to know they are wanted. Plus, it's always possible their travel plans will get canceled.
In each of these 3 examples the person with AS is actually being considerate, but would not appear that way to an NT.
Yes, I've read that paper, and disagree with much of it. Perhaps I'll take the time to explain my reasoning sometime soon
Anthropic reasoning is what leads people to believe in miracles. Rare events have a high probability of occurring if the number of observations is large enough. But whoever that rare event happens to will feel like it couldn't have just happened by chance, because the odds of it happening to them was so large.
If you wait until the event occurs, and then start treating it as a random event from a single trial, forming your hypothesis after seeing the data, you'll make inferential errors.
Imagine that there are balls in an urn, labeled with numbers 1, 2,...,n. Suppose we don't know n. A ball is selected. We look at it. We see that it's number x.
non-anthropic reasoning: all numbers between 1 and n were equally likely. I was guaranteed to observe some number, and the probability that it was close to n was the same as the probability that it was far from n. So all I know is that n is greater than or equal to x.
anthropic reasoning: A number as small as x is much less likely if n is large. Therefore, hypotheses with n close to x are more likely than hypotheses where n is much larger than x.
This is interesting. We shouldn't get a discontinuous jump.
Consider 2 related situations:
if Heads she is woken up on Monday, and the experiment ends on Tuesday. If tails, she is woken up on Monday and Tuesday, and the experiment ends on Wed. In this case, there is no 'not awake' option.
If heads she is woken up on Monday and Tuesday. On Monday she is asked her credence for heads. On Tuesday she is told "it's Tuesday and heads" (but she is not asked about her credence; that is, she is not interviewed). If tails, it's the usual woken up both days and asked about her credence. The experiment ends on Wed.
In both of these scenarios, 50% of coin flips will end up heads. In both cases, if she's interviewed she knows it's either Monday&heads, Monday&tails or Tuesday&tails. She has no way of telling these three options apart, due to the amnesia.
I don't think we should be getting different answers in these 2 situations. Yet, I think if we use your probability distributions we do.
I think there are two basic problems. One is that Monday&tails is really not different from Tuesday&tails. They are the same variable. It's the same experience. If she could time travel and repeat the monday waking it would feel the same to her as the Tuesday waking. The other issue is that, even though in my scenario 2 above, when she is woken but before she knows if she will be interviewed, it would look like there is a 25% chance it's heads&Monday and a 25% it's heads&Tuesday. And that's probably a reasonable way to look at it. But, that doesn't imply that, once she finds out it's an interview day, that the probability of heads&Monday shifts to 1/3. That's because on 50% of coin flips she will experience heads&Monday. That's what makes this different than a usual joint probability table representing independent events.
At this point, it is just assertion that it's not a probability. I have reasons for believing it's not one, at least, not the probability that people think it is. I've explained some of that reasoning.
I think it's reasonable to look at a large sample ratio of counts (or ratio of expected counts). The best way to do that, in my opinion, is with independent replications of awakenings (that reflect all possibilities at an awakening). I probably haven't worded this well, but consider the following two approaches. For simplicity, let's say we wanted to do this (I'm being vague here) 1000 times.
Replicate the entire experiment 1000 times. That is, there will be 1000 independent tosses of the coin. This will lead between 1000 and 2000 awakenings, with expected value of 1500 awakenings. But... whatever the total number of awakenings are, they are not independent. For example, one the first awakening it could be either heads or tails. On the second awakening, it only could be heads if it was heads on the first awakening. So, Beauty's options on awakening #2 are (possibly) different than her options on awakening #1. We do not have 2 replicates of the same situation. This approach will give you the correct ratio of counts in the long run (for example, we do expect the # of heads & Monday to equal the # of tails and Monday and the # of tails and Tuesday).
Replicate her awakening-state 1000 times. Because her epistemic state is always the same on an awakening, from her perspective, it could be Monday or Tuesday, it could be heads or tails. She knows that it was a fair coin. She knows that if she's awake it's definitely Monday if heads, and could be either Monday or Tuesday if tails. She knows that 50% of coin tosses would end up heads, so we assign 0.5 to Monday&heads. She knows that 50% of coin tosses would end up tails, so we assign 0.5 to tails, which implies 0.25 to tails&Monday and 0.25 to tails&Tuesday. If we generate observations from this 1000 times, we'll get 1000 awakenings. We'll end up with heads 50% of the time.
The distinction between 1 and 2 is that, in 2, we are trying to repeatedly sample from the joint probability distributions that she should have on an awakening. In 1, we are replicating the entire experiment, with the double counting on tails.
In 1, people are using these ratios of expected counts to get the 1/3 answer. 1/3 is the correct answer to the question about the long-run frequencies of awakenings preceded by heads to awakenings preceded by tails. But I do not think it is the answer to the question about her credence of heads on an awakening.
In 2, the joint probabilities are determined ahead of time based on what we know about the experiment.
Let n2 and n3 are counts, in repeated trials, of tails&Monday and tails&Tuesday, respectively. You will of course see that n2=n3. They are the same random variable. tails&Monday and tails&Tuesday are the same. It's like what Jack said about types and tokens. It's like Vladimir_Nesov said:
Two subsequent states of a given dynamical system make for poor distinct elements of a sample space: when we've observed that the first moment of a given dynamical trajectory is not the second, what are we going to do when we encounter the second one? It's already ruled "impossible"! Thus, Monday and Tuesday under the same circumstances shouldn't be modeled as two different elements of a sample space.
You said:
I don't mean to claim that as soon as Beauty awakes, new evidence comes to light that she can add to her store of bits in additive fashion, and thereby update her credence from 1/2 to 1/3 along the way. If this is the only kind of evidence that your theory of Bayesian updating will acknowledge, then it is too restrictive.
I don't think it matters if she has the knowledge before the experiment or not. What matters is if she has new information about the likelihood of heads to update on. If she did, we would expect her accuracy to improve. So, for example, if she starts out believing that heads has probability 1/2, but learns something about the coin toss, her probability might go up a little if heads and down a little if tails. Suppose, for example, she is informed of a variable X. If P(heads|X)=P(tails|X), then why is she updating at all? Meaning, why is P(heads)=/=P(heads|X)? This would be unusual. It seems to me that the only reason she changes is because she knows she'd be essentially 'betting' twice of tails, but that really is distinct from credence for tails.
The probability represents how she should see things when she wakes up.
She knows she's awake. She knows heads had probability 0.5. She knows that, if it landed heads, it's Monday with probability 1. She knows that, if it landed tails, it's either Monday or Tuesday. Since there is no way for her to distinguish between the two, she views them as equally likely. Thus, if tails, it's Monday with probability 0.5 and Tuesday with probability 0.5.