Four Short Stories on Error Checking

post by brilee · 2012-06-25T04:58:47.681Z · LW · GW · Legacy · 8 comments

 

(Cross-posted from my blag)

1. My microwave clock has been broken since a power outage. It now reads something like 7 hours behind the real time. I've been too lazy to fix it, but the bright side is that it's 7 hours behind, and not 15 minutes behind. If it were 15 minutes behind, then who knows - I might mistake it to be the correct time, and end up fifteen minutes late to an appointment.

 

2. During the early years of computing, scientists and engineers were responsible for running numerical simulations of various sorts. To do this, they needed randomized starting conditions. Von Neumann favored the "middle square" method. While this method was not a very good pseudo-random number generator, its speed made up for its shortcomings in those early days. Additionally, a useful property of the middle-square method was that when it fell into a short cycle, it was immediately obvious. While other methods may have had undetectable cycles of intermediate length, the middle-square method would invariably output legitimate pseudorandom numbers for some time, then fall into a cycle of length 1, 2 or 4. (1)

 

3. One day, I was playing with some traffic models. My goal was to be able to correctly model the behavior of a line of cars as they accelerated from a standstill when the light turned from red to green. (2) I had collected some actual data at an intersection, and was planning to test my model against the data, as well as to fit some parameters. I ran a short program to fit these parameters and plot the actual times vs. my program's predicted times. To my astonishment, I had almost a perfect fit! Upon deeper inspection, it turned out that my program had merely gotten the times right by coincidence - while the cars behaved nicely long enough to get to the intersection, afterwards, their velocities oscillated with exponentially growing amplitude. I would have missed this if I had not insisted on checking the raw numbers from the simulation. I recoded my simulation and got a worse fit, but at least it didn't blow up as it had before.

 

(This last story is fiction. Or so, I hope)

4. The US News and World Report was doing its annual ranking of universities. They had recently changed the weightings on some of the subscores. Upon running their algorithm, an unexpected candidate rose to the top - Caltech! (Zing.) They concluded that there must have been a mistake with their algorithm, readjusted their weightings, and reran their algorithm. This time, Harvard rose to the top, as it should have. Happy with their results, US News and World Report published their university rankings and raked in a lot of dough.

 

Notes:

(1) Test the middle-square story. Is it actually true that all cycles are short? My code, if you want to play with it.

(2) The time for the nth car to reach the intersection is about 2*n seconds

 

8 comments

Comments sorted by top scores.

comment by Antisuji · 2012-06-25T16:11:33.129Z · LW(p) · GW(p)

The moral of the story: if something's going to fail, it's best if it does it in an obvious way. Actually, I'm not sure exactly what you're getting at with stories (3) and (4), but I'm probably just failing to connect the dots. Help?

Replies from: CronoDAS, brilee
comment by CronoDAS · 2012-06-25T18:42:42.654Z · LW(p) · GW(p)

#3 is an example of something that appears to be working but isn't.
#4 is an example of something that appears to be failing but isn't.

comment by brilee · 2012-06-25T16:24:03.808Z · LW(p) · GW(p)

You're very right about the moral of the story. I had intended it as a an allegory to publication bias in science, but I don't think I wrote the stories quite right.

comment by [deleted] · 2012-06-25T08:59:58.338Z · LW(p) · GW(p)

Added your blog to the Blogs by LWers list.

comment by mwengler · 2012-06-25T21:43:34.534Z · LW(p) · GW(p)

(Cross-posted from my blag)

Could you provide a link to your blag?

Replies from: brilee
comment by brilee · 2012-06-25T23:05:09.906Z · LW(p) · GW(p)

moderndescartes.com/blog

comment by Alicorn · 2012-06-25T07:28:15.108Z · LW(p) · GW(p)

Fixed your middle square link.