Roadmap for a collaborative prototype of an Open Agency Architecture 2023-05-10T17:41:20.261Z

Comment by Deger Turan (deger-turan-1) on AI forecasting bots incoming · 2024-09-10T18:32:31.150Z · LW · GW

On behalf of the Metaculus team here: If @Dan H (or anyone) wants to actually easily, rigorously demonstrate a single-prompt forecasting bot that can outcompete Metaculus forecasters, I invite them to do so. We’ll even cover your API credits. Just enter your bot in the benchmarking series. We’ve completely solved the data leakage issue—which does appear to be a fatal flaw in Dan’s approach. I myself am skeptical that a bot using a single prompt can really compete given everyone's attempts so far, but I would love to be proven wrong!