aarongertler's Shortform

I've been enjoying the Sold a Story podcast, which explains how many schools stopped teaching kids to read over the last few decades, replacing phonics with an unscientific theory that taught kids to pretend to read (cargo cult vibes). It features a lot of teachers and education scholars who come face-to-face with evidence that they've been failing kids, and respond in many different ways — from pro-phonics advocacy and outright apology to complete refusal to engage. I especially liked one teacher musing on how disconcerting it was to realize her colleagues were "refuse to engage" types.

The relatable topic and straightforward reporting make the podcast very accessible. It's a good way to share a story with people outside the LessWrong bubble that may get them angry in a way that supports rationalist virtues.

Epistemic status: Neither unique nor surprising, but something I felt like idly cataloguing.

An interesting example of statistical illiteracy in the field: This complaint thread about the shuffling algorithm on Magic: the Gathering Arena, a digital version of the card game. Thousands of unique players seem to be represented here.

MTG players who want to win games have a strong incentive to understand basic statistics. Players like Frank Karsten have been working for years to explain the math behind good deckbuilding. And yet, the "rigged shuffler" is a persistent belief even among reasonably engaged players; I've seen quite a few people try to promote it on my stream, which is not at all aimed at beginners.

(The shuffler is, of course, appropriately random, save for some "hand smoothing" in best-of-one matches to increase the chance of a "normal" draw.)

A few quotes from the thread:

How is that no matter how many people are playing the game, or how strong your deck is, or how great your skill level, I bet your winning percentage is 30% or less. This defies the laws of probability.

(No one ever seems to think the shuffler is rigged in their favor.)

As I mentioned in a prior post you never see these problems when they broadcast a live tournament.

(People who play in live tournaments are much better at deckbuilding, leading to fewer bad draws. Still, one recent major tournament was infamously decided by a player's atrocious draw in the last game of the finals.)

In the real world, land draw will not happens as frequent as every turns for 3 times or more. Or less than 2 to 3 turns, not drawing a land

(Many people have only played MTG as a paper game when they come to Arena. In paper, it's very common for people to "cheat" when shuffling by sorting their initial deck in a particular way, even with innocuous intent. When people are exposed to true randomness, they often can't tolerate it.)

Other common conspiracy theories about Arena:

• "Rigged matchmaking" (the idea that the developers somehow know which decks will be good against your deck, and ensure that you are matched up against it; again, I never see this theory in reverse)
• "Poker hands" (the idea that people get multiple copies of a card more often than would be expected)
• "50% bias" (the idea that the game arranges good/bad draws to keep players at a 50% win rate; admirably, these players recognize that they do draw well sometimes, but they don't understand what it means to be in the middle of a binomial distribution)