post by [deleted] · · ? · GW · 0 comments

This is a link post for


Comments sorted by top scores.

comment by ChristianKl · 2021-03-06T15:42:55.658Z · LW(p) · GW(p)

Since accurate calculation of bias requires large amounts of data, the group will officially begin its activities after we recruit a sufficient number of participants.

Have you thought about collaborating with existing platforms such as Metaculus?

comment by Oliterio · 2021-03-04T21:53:07.581Z · LW(p) · GW(p)

Great concept, B and C vectors makes a lot of sense as well. But how can we trust this AI and tests to be unbiased?

Replies from: Leonid
comment by Leonid · 2021-03-05T03:38:23.327Z · LW(p) · GW(p)

Quis custodiet ipsos custodes?

You are asking a very good question. Any person or group that is put in charge of the data might be tempted to modify the results to suit their agenda. In principle, this problem may be mitigated if there are several competing forecasting platforms. Scoring their accuracy on the same set of questions is likely to expose any systematic bias (just like it works with individual forecasters).