New "Best" comment sorting system

Short version of how this is different, for those too lazy to click on the link: if you sort by "top", comments get sorted in a simple "the ones with the highest score go on top" order. This has the problem that it favors comments that were posted early on, since they're the ones that people see first and they've had a lot of time to gather upvotes. A good comment that's posted late might get stuck near the bottom because few people ever scroll all the way down to upvote it.

"Best" uses some statistical magic to fix that:

If everyone got a chance to see a comment and vote on it, it would get some proportion of upvotes to downvotes. This algorithm treats the vote count as a statistical sampling of a hypothetical full vote by everyone, much as in an opinion poll. It uses this to calculate the 95% confidence score for the comment. That is, it gives the comment a provisional ranking that it is 95% sure it will get to. The more votes, the closer the 95% confidence score gets to the actual score.

If a comment has one upvote and zero downvotes, it has a 100% upvote rate, but since there's not very much data, the system will keep it near the bottom. But if it has 10 upvotes and only 1 downvote, the system might have enough confidence to place it above something with 40 upvotes and 20 downvotes -- figuring that by the time it's also gotten 40 upvotes, it's almost certain it will have fewer than 20 downvotes. And the best part is that if it's wrong (which it is 5% of the time), it will quickly get more data, since the comment with less data is near the top -- and when it gets that data, it will quickly correct the comment's position. The bottom line is that this system means good comments will jump quickly to the top and stay there, and bad comments will hover near the bottom. (Picky readers might observe that some comments probably get a higher rate of votes, up or down, than others, which this system doesn't explicitly model. However, any bias which that introduces is tiny in comparison to the time bias which the system removes, and comments which get fewer overall votes will stay a bit lower anyway due to lower confidence.)

Not sure I fully understood that either. But they say it works well, so I guess I'll trust them!

How do the "Best", "Popular", and "Top" algorithms work?

Yay! Thanks Matt and the tricycle team (and anyone else) for continuing to improve LW.

Thanks Matt!

Thank you for taking the time to implement this, I've set it as my "sort by" criteria.

For me, this is not working for some of the posts,

e.g. http://lesswrong.com/lw/kd/pascals_mugging_tiny_probabilities_of_vast/?sort=top

I found a Reddit thread explaining the different comment sorting systems. Does LW use the same algorithms for each method?

http://www.reddit.com/r/TheoryOfReddit/comments/1y8rst/what_is_the_best_way_to_sort_top_best_new/

Missing from their list though are "popular" and "leading" (and "old", but that's pretty self-explanatory). I'm guessing "popular" is the same thing as "hot", judging based on what appears in my address bar when I sort that way. "Leading" is listed as "interestingness" in the address bar, which leads me to think it adds weight to comments that inspire a lot of discussion. My observations suggest that it also factors in votes though. Could someone please clarify further on what these algorithms do?

I've noticed that the "Best" sorting sometimes puts strongly downvoted comments (score -5 or less) above comments with scores closer to zero. Is this intentional or a bug?

What is the difference between Best, Popular, and Top?