Introducing bayescalc.io

Relevant xkcd

Thanks for the feedback, I'm really happy to hear that you already have uses for it!

You're right about needing examples; I'm thinking I'll add a tutorial that walks someone completely unfamiliar with Bayes' theorem through what it means and how it works, with lots of examples. That will take a while to design and write though.

I'm curious to know if other people felt the same way "How to use" part. I'm reluctant to make it more attention grabbing, because I want it to feel unobtrusive. My current thinking is that the main interface will catch the user's attention first, and if that's not clear they'll look at the wall of text to the right.

Instead of a wizard, I was thinking of adding a feature that explains what a specific component means when the user is hovering over it. Does that seem like it would address the issue adequately? I don't like wizards because I feel like they get in the way, but maybe that's an unusual preference.

Fixed now (but may require a cache refresh)!

Added! I hope you like the design :)

Thanks, I'm very glad you find it intuitive!

Only allowing the last piece of evidence to be deleted was a deliberate decision. The problem is that deleting evidence from the middle changes the meaning of all the likelihood values (the sliders) for all of the evidence below it, and which therefore may change in value. If I allowed it to be deleted anyway, it would make it very easy to mistakenly use the now incorrect values (and give the impression that that was fine). I know this makes it more annoying and inconvenient, but it's because the math itself is annoying and inconvenient!

The meaning of the e.g. Hypothesis B slider for Evidence #3 is "In what percentage of worlds where Hypothesis B is true would I see Evidence #3?" (hopefully this was clear, just reiterating to make sure we're on the same page). This is called the likelihood of Evidence #3 given Hypothesis B. When answering this, we don't use the fact that we've seen this piece of evidence (in this case that politicians are taking this seriously), which is always just going to be true for actual evidence. Hopefully that makes sense?

As for choosing this number, or the prior values, it's in general a difficult problem that has been debated a lot. My recommendation is that you make up numbers that feel right (or at least are not obviously wrong), and then play around with the sliders a bit to see how much the exact value effects things. The intended use of the tool is not to make you commit to numbers, but to help you develop intuition on how much to update your beliefs given the evidence, as well as to help you figure out what numbers correspond to your intuitive feelings.

If you're serious about choosing the right number, then here is what it takes to figure it out: Each hypothesis represents a model of how some part of the world works. To properly get a number out of it, you need to develop the model in technical detail, to the point where you can represent it with an equation or a computer program. Then, you need to set the evidence above the one you're computing the likelihood for to true in your model. You then need to compute what percentage of the time this evidence turns out to be true in the model. A nice general way to do this is to run the model a whole bunch of times, and see how often it happens (and if reality has been kind enough to instantiate your model enough times, then you might be able to use this to get a "base rate"). Or if your model is relatively simple, you might be able to use math to compute the exact value. This is typically a lot of work, and doesn't actually help train your intuition about the intuitive mental models you actually use on a day-to-day basis much. But going through this process is helpful for understanding what the numbers you make up are trying to be. I hope this is helpful and not just more confusing.

Thanks!

That was a deliberate decision designed to emphasize the core features of the app, but enough people have pointed this out now that I'm considering changing it.

More than two hypotheses are already supported, you just need to close the "How to use" box.

Thanks for the suggestions!

As ProgramCrafter mentioned, more (up to five) hypotheses are already supported. It's limited to 5 because finding good colors is hard, and 5 seemed like enough - but if you find yourself needing more I'd be interested to know.

The sliders already snap to tenth values (but you can enter more precise values in the textbox), and I think snapping to integers would sacrifice too much precision. It's plausible that fifths could be better though, I'll have to test that. I do want to introduce a way to allow for more precise control while dragging the sliders, which might address this concern to some extent by making it easy to stop at an integer value exactly if desired. But I haven't thought of a good interface for doing that yet.

That sounds cool, but I'm not sure how to make a good interface for that that wouldn't look too cluttered. I'm also worried people would misuse it for convenience. But I'll keep thinking about it!

Tooltips to explain things would be cool and I have a similar thing planned already.

That's a good idea, thanks!

Thank you! I'm glad you like those features, and I'm also glad to hear that the way the percent button feature worked was clear to you.

Regarding the possible improvements:

1. That's not a bug, it's just a limitation of the choice to show only one digit after the decimal. The number of decibels in case 2 for each evidence is 0.96910013..., whereas in case 1 it's exactly 10.

2. That's a deliberate nudge to suggest that the new hypothesis and decibel features are more advanced and not part of the essential core of the app.

3. That's a good idea, I'll probably do that at some point.

4. That's also a good idea but seems fairly complicated to implement, so it will have to wait until I've finished planned improvements with a higher expected ROI.

5. That's deliberate, because deleting evidence changes the meaning of the likelihoods for all subsequent evidence. Thus, having to delete all the evidence following the evidence you want to delete is a more honest way to convey what needs to be done, and prevents the user from shooting themselves in the foot by assuming that the subsequent likelihoods are independent. I'll explain this in the more fleshed out version of the help panel I have planned.

Hmm, you could use the slider to set the prior P for hypothesis A and it will set the prior for hypothesis B to 1 - P; does that not work for you for some reason?

The problem with having that behavior when you type in the number is that I want people to be able to enter the priors as odds, so I don't want to presume that the other numbers will change to allow for that.

Are you using a spaced repetition system like Anki? I find it to be great for learning theorems and formulas, you should try using that if you aren't already. It's literally like a memory hack, you can just take any information and embed it into your memory (assuming you find the time to go through your due cards every day)