Comment by Brian Bien (brian-bien) on Goodhart Taxonomy · 2023-01-21T13:45:54.518Z · LW · GW

> The fact that you choose an algorithm does not effect its performance, and you don't have to worry about Causal Goodhart. 

But now, I think you have to worry about a "Regressional Goodhart" 

Maybe this would be pedantic to point out, but your choice of the best-performing model on test data is likely to have done that well by chance, as the number of models evaluated increases (hence validation and test splits).