The Parable of the Boy Who Cried 5% Chance of Wolf

Why isn't the moral of the story "If you think statistically, take into account that most other people don't and optimize accordingly?"  

This echos an excellent post by Dan Luu that touches on problems you face when you build larger, less legible systems that force you to deal with normalization of deviance: https://danluu.com/wat/

The action items he recommends are:

Pay attention to weak signals

Resist the urge to be unreasonably optimistic

Teach employees how to conduct emotionally uncomfortable conversations

System operators need to feel safe in speaking up

Realize that oversight and monitoring are never-ending

Most of these go against what is considered normal or comfortable though:

1. It's difficult to pay attention to weak signals when people build awful attention traps (eg. tiktok, youtube, etc.)
2. People are commonly [overconfident](https://www.lesswrong.com/posts/ybYBCK9D7MZCcdArB/how-to-measure-anything#Step_2__Determine_what_you_know). 
3. Uncomfortable conversations are uncomfortable. In certain cultures, it's better to straight out lie rather than deliver bad news.
4. Few organizations have set up channels for upwards communication where front line employees can flag problems. It's better to not rock the boat.
5. Constant oversight and monitoring are mentally and physically draining.  These are also the easiest activities to cut from a budget because they're not visible until an accident happens.

What the boy should have done is establish an organization (Wolf-Spotters) whose responsibility is monitoring for signs of wolves. This organization could be staffed by professional or volunteer observers. They need to do periodic trainings and live-action drills (perhaps using a wolf suit). To the fund this, the boy should have first created a PR campaign to make people aware of the cost of unspotted wolves (death), then use that to get support of some local politicians.

(It's basically a fire department). 

If the boy was open to using the dark arts, he could have executed a false flag wolf attack. That would incentivize local politicians to support his cause.

Awesome line of thinking, and absolutely believable that most people don't think numerically very well.

However, it's also believable that there's an equilibrium effect here.  BECAUSE this is so unusual, and there's no particular reason to think this boy is better at estimating the risk of wolves than the median resident, it's easy to just ignore them.  This is especially true when it's a hassle to check for wolves and they've never needed to before.

If the boy were socially smart, he'd have separated the warning into two parts, each with justificatin: "There's a 5% chance a wolf is here" and "5% is risky enough that you should do something about it".  And here's how to check my reasoning.  

Since villagers aren't homogeneous, the boy just needs to convince a few people with status on the right dimensions, who need to convince those with status generally, who will convince everyone else.  That's pretty tenuous, so a lot of villages get eaten.  Humans are, indeed, dumb.

One quibble, there was a little bait and switch from someone with a well-calibrated model whose calibration just hasn't been well-evidenced, to...

You'll hear people saying that X will definitely fuck everything up very soon.
And it doesn't.
And when the catastrophe doesn't happen, don't over-update.
Don't say, "They cried wolf before and nothing happened, thus they are no longer credible."

These people ARE no longer credible as they are not estimating 5% chances but 95% chances, and the lack of an event, rather than being consistent with their model, is inconsistent with their model.

Your point is still well-taken, and I think the switch is a natural reflex given the infrequency of pundits attempting to make well-calibrated or even probabilistic judgments. For example, it has been noticeable to me to see Jamie Dimon publicly assigning probabilities to differing recession/not-recession severity bins rather than just sticking to the usual binary statements often seen in that space.

Hypothesis: This may be not just people being ignorant about probability. This may be people being unconsciously good at game theory. If the boy could be motivated with cross-entropy-loss worth of cookies, it would be another thing. If he can simply get the microphone and yell a number that would be enough for inciting the intended action but low enough to have practically no accountability - it may make more sense for society to actively make the "crying wolf" more binary. It doesn't penalise predication error, but yelling stuff into the microphone.

This may be called something like the "low-probability forecaster reputation problem".

People in the village were not necessarily wrong ex-ante to dismiss the warnings. They of course should not update much about forecasting abilities on not seeing a wolf after low-probability warnings. But they cannot know the accuracy of the forecasts. And if someone is an expert in certain low-probability events and nothing else, you cannot judge ability a lot based on the few forecasts. You need to examine the models used for forecasting, but what tells you that is a good use of your time? There are crackpots everywhere.

Therefore, people try to get bets or forecasts for things that are implied by the lp forecaster's model but happen more often and earlier. If your model implies big grey wolves for tonight with 15% chance, should we already see dead sheep by noon? If your model implies war between states x and y next year, what should we see next month that would not be expected by people with different models?

If you really specialize in forecasting low-probability events only and cannot show any evidence to show your model is right, then that is a tragedy. But it is understandable that people do not update based on your forecast.

The wolf-skeptics who stayed in bed felt smug.

If you perceive life as a zero-sum game, this is the correct move. If there is no wolf, the wolf-skeptics get higher status than their neighbors. If there is a wolf... everyone is eaten equally, no penalty for being a wolf-skeptic.

So what is the point where its correct to bin the sensor as alarmist?

Pondering this I lean that the main takeaway is that if you "cry" anything then no amount of what you cry can make it effective for low-probablity events. Crying "there is a 0.001% chance of wolf" is in practise going to be nearly equivalent to communicating about 20% chance of wolf. But if you say "there is a 0.001% chance of wolf" maybe that works for 5% and if you whisper you can get under 1%.

The second is that people can disagree what prepardness thresholds are proper. Say that the village runs out 10 times in responce to 5% reports and then can only harvest half the crop for being exhausted from running all the time killing 20% people to starvation. As a wolf-expert I am not likely to be the most well versed on food shortage events. So when I make judgement calls on where it is better to be safe than sorry I am likely to be atleast somewhat ignorant. So even an honest private trade-off might not be be socially optimal. The problem disappears if the village thinks the fortifications for nothing are worth the effort.

Small probabilities are hard to calculate accurately. How did the boy know that it was a 5% chance and not 0.001% chance? 

Maybe the real moral is that you shouldn't set up a village in an area where it seems even remotely plausible that there are wolves so enormous that they can eat everyone in a single night.

Or at least if you do, to set up defenses that are less flimsy than "have a boy run in yelling".