Contra papers claiming superhuman AI forecasting 2024-09-12T18:10:50.582Z

Comment by Peter Mühlbacher (peter-muehlbacher) on Contra papers claiming superhuman AI forecasting · 2024-09-16T09:01:28.603Z · LW · GW

I agree it's a high bar, but note that

1. a few particularly enthusiastic (&smart) humans still perform at roughly this level (depending on how you measure performance), so you wouldn't want it to be much lower, and
2. we only acknowledged that this is a fairly reasonable definition of superhuman performance—it's authors in these papers who claimed that their models were (roughly) on par with, or better than the crowd forecast.

We made the deliberate choice of not getting too much into the details of what constitutes human-level/superhuman forecasting ability. We have a lot of opinions on this as well, but it is a topic for another post in order not to derail the discussion on what we think matters most here.