Building an Epistemic Status Tracker

post by rcu · 2022-06-22T18:57:34.198Z · LW · GW · 8 comments

Contents

8 comments

Based on my experiences in the rationalist community as well as the work of Tetlock and others in superforecasting, I was inspired to build something I've been wanting for a long time: an epistemic status tracker.

In its simplest form, it's a tool for making predictions, logging Bayesian updates, and analyzing your forecasting abilities. 

While Metaculus and other prediction markets are great for world events, making predictions about more obscure events is also key to developing intuition—I've resorted to logging these in my journal (in a format similar to Scott Alexander's), but this was always an imperfect solution.

Below are some sketches for the most basic version of the mobile app:

Of course, this is a personal, not-for-profit project that I'll be open-sourcing.

I've thought about including social features (e.g., friends can take the over or under on your predictions) and analytics (particularly the classic calibration chart).

Most of all, I'm hoping to hear feedback from other community members: what would you want to see included?

8 comments

Comments sorted by top scores.

comment by averykhoo · 2022-07-08T06:46:48.245Z · LW(p) · GW(p)

Personally I'd suggest adding a way to import/export to either json or csv (e.g. to enable power users to do bulk edits or stats/charts in excel). Tags (or categories) might be useful once the number of predictions is above 50 or so. Will it be available on Android?

comment by Raemon · 2022-06-22T19:41:40.990Z · LW(p) · GW(p)

How does it compare to PredictionBook?

Replies from: rcu
comment by rcu · 2022-06-23T01:04:23.659Z · LW(p) · GW(p)

Quite similar!

Having a prediction tracker on mobile is a big deal for me; predictions come up at unexpected times and sometimes it's a hassle to log in to a web app. Just a personal preference.

comment by RHollerith (rhollerith_dot_com) · 2022-06-22T19:11:52.089Z · LW(p) · GW(p)

How do you imagine you will enter text into the app? Using on on-screen keyboard?

Replies from: rcu
comment by rcu · 2022-06-23T01:05:12.752Z · LW(p) · GW(p)

Yes, exactly. Wasn't shown on the wireframes but each field would cause a keyboard (QWERTY, numerical, dates, etc.) to pop up, depending on context.

Replies from: rhollerith_dot_com
comment by RHollerith (rhollerith_dot_com) · 2022-06-23T20:20:04.311Z · LW(p) · GW(p)

Interesting. I am also curious what size of screen do you imagine (plan on) this keyboard being on? One of smartphone size?

I am surprised at how much text you are willing to enter on one's on-screen keyboard if indeed you are planning to use this app on a smartphone. (I don't own a smartphone, but am curious about them.)

comment by Vlad Sitalo (harcisis) · 2024-09-30T06:13:03.406Z · LW(p) · GW(p)

Did you ever end up building this, and if not what is the current best tool for this you're aware of?

Predictionbook seems to be dead. 
Foretold.io which I used before seems to be broken too (can't log-in into my account anymore)

Replies from: harcisis
comment by Vlad Sitalo (harcisis) · 2024-09-30T06:16:58.533Z · LW(p) · GW(p)

Found https://fatebook.io/ that seems alive, trying it out now