Metaculus Introduces New Forecast Scores, New Leaderboard & Medals 2023-11-20T20:33:57.119Z
Metaculus Introduces AI-Powered Community Insights to Reveal Factors Driving User Forecasts 2023-11-10T17:57:11.837Z
Metaculus's New Sidebar Helps You Find Forecasts Faster 2023-11-08T20:56:12.392Z
Metaculus Launches Conditional Cup to Explore Linked Forecasts 2023-10-18T20:41:41.191Z
Metaculus Announces Forecasting Tournament to Evaluate Focused Research Organizations, in Partnership With the Federation of American Scientists 2023-10-03T16:44:17.620Z
Metaculus Launches 2023/2024 FluSight Challenge Supporting CDC, $5K in Prizes 2023-09-27T21:35:15.616Z
Gulf Breeze, Florida, USA – ACX Meetups Everywhere Fall 2023 2023-08-25T23:40:21.287Z
Metaculus Event: Forecast Friday, April 28th at 12pm ET — Speed Forecasting Session!  2023-04-27T19:50:44.343Z
[Event] Join Metaculus Tomorrow, March 31st, for Forecast Friday! 2023-03-30T20:58:57.315Z
[Event] Join Metaculus for Forecast Friday on March 24th! 2023-03-17T22:47:02.077Z
ChristianWilliams's Shortform 2023-03-01T02:43:58.128Z
Metaculus Introduces New 'Conditional Pair' Forecast Questions for Making Conditional Predictions 2023-02-20T13:36:19.649Z
Metaculus Year in Review: 2022 2023-01-06T01:23:07.050Z
Metaculus Announces The Million Predictions Hackathon 2022-11-10T20:00:00.392Z
Metaculus is building a team dedicated to AI forecasting 2022-10-18T16:08:11.039Z
Metaculus Launches the 'Forecasting Our World In Data' Project to Probe the Long-Term Future 2022-10-14T17:00:31.311Z
Metaculus is seeking experienced leaders, researchers & operators for high-impact roles 2022-07-10T14:27:38.197Z
Metaculus is seeking Analytical Storytellers to write essays fortified with testable predictions 2021-10-06T04:44:01.092Z
[Event] A Metaculus Open Panel Discussion: How Forecasts Inform COVID-19 Policy 2021-10-02T00:37:49.421Z


Comment by ChristianWilliams on Metaculus Introduces New Forecast Scores, New Leaderboard & Medals · 2023-11-24T20:20:25.452Z · LW · GW

Hi @Odd anon, thanks for the feedback and questions. 

1. To your point about copying the Community Prediction: It's true that if you copy the CP at all times you would indeed receive a high Baseline Accuracy score. The CP is generally a great forecast! Now, CP hidden periods do mitigate this issue somewhat. We are monitoring user behavior on this front, and will address it if it becomes an issue. We do have some ideas in our scoring trade-offs doc for further ways to address CP copying, e.g.: 

We could have a leaderboard that only considers the last prediction made before the hidden period ends to calculate Peer scores. This largely achieves the goal above: it rewards judgement, it does not require updates or tracking the news constantly. It does not reward finding stale questions.

Have a look here, and let us know what you think! (We also have some ideas we're tinkering with that are not listed in that doc, like accuracy metrics that don't include forecasts that are on the CP or +/- some delta.)

2. On indicating confidence:  You'll see in the tradeoffs doc that we're also considering the idea of letting users exclude a particular forecast from their peer score (Idea # 3), which could somewhat address this. (Interestingly, indicating confidence was attempted at Good Judgment Project, but ultimately didn't work and was abandoned.) 

We're continuing to develop ideas on the above, and we'd definitely welcome further feedback!

Comment by ChristianWilliams on Some Thoughts on Conditional Forecasts – Lessons from the 2020 Election · 2023-06-06T14:32:43.601Z · LW · GW

Interesting read, thanks for writing it up. FYI the link "The report on the 2022 results is now available" leads to a private Google Drive file. 

Comment by ChristianWilliams on [Event] Join Metaculus Tomorrow, March 31st, for Forecast Friday! · 2023-04-11T02:14:04.415Z · LW · GW

Fair point. Noted for future event posts. 

Comment by ChristianWilliams on ChristianWilliams's Shortform · 2023-03-01T02:44:04.700Z · LW · GW

Metaculus is conducting its first user survey in nearly three years. If you have read analyses, consumed forecasts, or made predictions on Metaculus, we want to hear from you! Your feedback helps us better meet the needs of the forecasting community and is incredibly important to us. 

Take the short survey here — we truly appreciate it! (We'll be sure to share what we learn.) 

Comment by ChristianWilliams on Progress links and tweets, 2022-11-01 · 2022-11-02T00:40:25.887Z · LW · GW

Prediction market Metaculus launches a “Forecasting Our World In Data” tournament (via @metaculus)

Hi, this link should be instead, it's missing the www