↑ comment by emr ·
2014-12-15T04:25:35.055Z · LW(p) · GW(p)
I recently wrote a small (python, command-line) program for prediction tracking. Above all, I wanted something that allowed very quick entry, that stored everything in a single plain-text human-readable file on my own computer, and that I could easily customize.
As far as features go, I also wanted a tagging system and to be able to do more sophisticated analysis. For example, I wanted to classify predictions as e.g."work" or "politics" when looking at accuracy, or to see how my calibration changed over different times or with how far in advance I had made a prediction.
Since the predictions that I care most about aren't very interesting to other people, I don't miss the social aspect. Nor do I want to obscure a prediction that contains personal details or store it separately.
If you wanted to predict that it will rain today, with 50% confidence, and tag the prediction with the "weather" and "external" tags, and be reminded to judge the prediction 2 days from now, you'd type this on the command line:
predict 'rain today' 50 -t weather external -d 2
The created entry in the log will look something like this:
"prediction": "rain today",
Running "predict --due" will either ask you about any predictions that are ready to be judged and update the corresponding log entry, or just change "state" to "due" so you can ctrl-f and edit the log directly. (Since the log is just text, you can manually edit any entry, like changing a prediction from "open" to "true" or "false").
Running "predict --stats" will dump statistics.
I'm very happy with this work flow, and I'm hoping to clean up the code and share it when I get a moment. It's all of ~250 lines, but I couldn't find what I wanted out there already.