Comment by Mike G on Evaluating expertise: a clear box model · 2020-10-17T20:51:44.739Z · LW · GW

Good blog.

How many fields offer "short feedback loops over a long time period that are accurate"?

In my field, K-12 teacher coaching, there is rarely data on teacher performance that isn't "noisy or irrelevant or statistically biased." Gates Foundation spent $100 million to try to figure out this problem, but it proved thorny, both the stats and the politics.

Even in a data-loving field, like golf, with many competing experts (instructors) for hire, it's nearly impossible to know which ones actually generate the largest gains in their students. To use Waveman's point, it's hard to assess their relative skill at "fixing broken skills" because the data isn't available...even to the instructors.

What are the fields that best lend themselves to this sort of calculator?