Regression To The Mean [Draft][Request for Feedback]

I would suggest leading with a scenario in which this bias is relevant.

I/e, "Bob hates his boss; Bob's boss is always criticizing employees, and never says anything positive about their performance. Bob's boss believes the evidence supports this policy, because every time bad performance has been criticized, performance has improved - and every time praise has been offered for good performance, performance has shown no improvement, or sometimes even gotten worse."

(I recommend you write something better than this, though. This particular example reads like something horrible out of an HR handbook written by an eldritch god who trying to pass as human. Extra eldritch god points for a continuing narrative in which Bob's boss manages to do everything precisely wrong.)

This is an excellent topic, and this is a good start, but I think this post could use some work. In fact, if you wanted focused help with this, I would be interested in writing this with you. You can email me at jsalvatier@gmail.com or talk to me on skype (jsalvatier).

The way you've described this is a little vague, I think, and also makes the concept seem more complex than it is. I also think it would be useful to explore ways in which regression to the mean can lead you astray.

The core of regression to the mean is not that complicated: take a set of variables for which it's semi realistic to say they're independently and identically distributed (that is they have the same distribution and each one is a separate realization from that distribution). For example, the number of pushups you can do each day this week, or the number of minutes you're early/late for school each day. If you notice that a value is unusually high compared to ones that came before it, then later values are likely to be lower than it. If one value is higher than the mean of the distribution, then ,on average, later values will be below it because the mean of these variables is the mean of the distribution.

Here's an illustration which I think might help: Lets say you're a math teacher and you have the students take two tests during the year. After a while, you notice that students who did really well on the first test tend to do worse than before on the next test. It may be tempting to conclude that this is because students are afraid of success, or worried about doing too much better than their peers or because students get lazy when they're successful. However, there's a simpler explanation, tests are noisy measurements, so students who did very well on the first test are probably good students but also got a bit lucky. If they had been unlucky, they might have still been quite good, but not at the top. Since on the next test each person is just as likely to be lucky as unlucky (the mean of luck is 0), they'll probably do worse the next time around.

You also mention stock returns. I think it would be good to explain why "go invest all your money in the mutual fund that did the best last year" isn't a very effective strategy.

Good explanation of the phenomenon. Some thoughts:

The first two quotes and your explanations for them are good. But in this part:

"These statements seem contradictory, yet both describe real effects. The apparent contradiction is caused by a phenomenon known as "regression to the mean,""

I would add a sentence stating that positive reinforcement is not the actual cause of the regression. This will end any confusion. Additionally, the example with the IQ test needs a bit more clarification. While the average human IQ is around 100, the kid doesn't necessarily regress to the same average. Most humans have their own "average" that they regress or improve to after periods of extreme positive or negative performance. So the kid might not regress at all if his natural ability is indeed that high.

I don't see the relationship between positive reinforcement and regression to the mean. As discussed in this post, positive reinforcement increases the frequency of a particular behavior, not the quality of the behavior. By contrast, regression to the mean tells us facts about the quality of future behavior, not the quantity.

Concretely, I reward my son for voiding in the potty to increase the frequency that he voids in the potty. (Sorry, potty training is on the mind). But I'm not sure what a quality measure would look like in this circumstance. By contrast, we can imagine a soccer goalkeeper who allows no goals for an entire season. Regression to the mean strongly suggests he won't achieve that feat next season. But what behavior is becoming more or less frequent?

I'm not saying that there's no insight in the convergence of the two concepts. But the current draft suggests a conflict that is not clearly demonstrated.

great post.

I think the measure of performance is something we can do almost to any varible factor, but been a average are not expectable after outlier performances. In the private sector people are recompensate to be exceptionale, even a tiny better then they peers. If people know that they are not so good in things that they think are so good, updates of you own performance would less frequent.

Added your blog to the Blogs by LWers list.

whenever there is a large random component in performance.

I think that is a good way of explaining regression to the mean, but it's possible that "large" may be misleading. Any random component of performance will produce to regression to mean, although the amount depends on the amount of randomness (or really, unmeasured-ness). I'm not sure what to do about this. Perhaps address it at the end. Or just mislead.

People make the errors you describe, so we need some way of talking about them, but here is someone who doesn't like the way people use the phrase "regression to the mean."

"In general, unusually bad performance improves after punishment, but good performance tends not to improve and sometimes even gets worse after praise is administered."

I interpreted this to mean that punishment and praise results in that general effect on the performance. If you mean that the immediate next action is better than the current one, then that's really misleadin.