Voting Weight Discussion

I have noticed that starting posts at the poster's karma creates a double counting: I think "Raymond posted this I should read it" and also "This has 3 points I should read it" but it has 3 points because Raymond wrote it. I then notice that it causes me to want to know how many points Raymond gives his initial posts so I can mentally subtract those 3 points and see if anyone else actually endorses this comment, so now I feel low-level pressured to memorize people's karma scores. Which seems bad.

A question I find myself asking is "do people write posts they themselves would not wish to upvote?" I think the answer is yes, e.g. Raymond's "good point, I will fix that" which is a good comment for him to write but which I do not think he would choose to upvote.

I do think that the sorting effect of attaching the person's karma to the comment is actively good and would like to keep it. That could be kept distinct from the displayed score, same way it is kept distinct from the person's karma score.

I also realize that I have a lot of strong preferences in such things, so we might want to consider (non-urgently) putting effort into giving people customization options for display of scores and sorting of comments.

I think the relative weighting is good and if anything is too weak, and I would consider taking out the initial point, which would mean that you need 5 Karma to vote at all. This seems fine since 5 Karma is one good comment, and the system can put your vote in retroactively. If anything it sends the right message that we want to know your opinion but do not yet put any weight on it! During the beta this might have odd effects since it does not appear that Karma has transferred from 1.0 (e.g. my posts on my first day here are starting at 1) so for now we should give everyone at least 1 point.

Inflation worries me a lot, though. As Raymond notes, when I see a 7 I instinctively think of that as 7 people voting up (or 7 more up than down). My brain knows what to think about that. If 20 is the new 7, then every time I see a number I'm doing math to translate back to the scale of people. This is together with the issue that comments start at 1+ rather than 0. The starting at 0 and going up slowly (since downvoting and stinginess) helped with the atmosphere and culture that LW1.0 projected. Each point mattered and getting them was hard and meaningful. It sent the message that one should be thoughtful (perhaps too thoughtful, but I am not worried about that error in 2.0 at least for now).

I find smaller numbers that map onto basic instincts, where I know what 1 means, much better and enabling of discrete, clean thinking than larger numbers. I despise the 'Over 9000' thing, and really appreciate Dragon Quest's starting you out dealing 4 damage and maxing out at 99 versus Final Fantasy starting you out dealing 121 and maxing out at 9999. Too much precision! Too many significant figures! I get that a Reddit post having 2730 upvotes actually got upvoted that many times but what does that even mean beyond going viral? How do I think about that?

Thus I think we should consider normalizing the scale, at least such that when you see a 10, on average it means 10 people voted up. If we get more people maybe even normalize such that the scale maxes out at 1 rather than starting at 1. We might even think about displaying on a log scale (although we should not do the actual long-term calculation that way, that would be quite bad).

Reactions--Vote with Emojis or Tags

Karma currently conflates multiple possible reations into a single datum. An upvote currently could mean "me too" or "I updated" or "I agree" or "others should read this", or "LOL", etc. A downvote could mean "fallacy" or "poor quality" or "disagree", etc. It's hard for posters and readers to discern intent.

Facebook and Github (and probably others) now have a small number of Emoji reactions, instead of just +/- or "Like". This is a more fine-grained feedback mechanism than a simple vote, but still easy (and familiar) enough for everyone to understand and use.

These give us additional low-effort feedback mechanisms that normally wouldn't be worth a written reply. Facebook's selection isn't appropriate for Less Wrong, but we could choose some better ones. They also need not be icons. Words or short phrases (i.e. "tags") will do. We'd probably want to pull memes about good discourse and their failure modes from the Sequences, e.g. #applause-light or #updated, etc., as well as all the well-known logical fallacies and biases. (These could also be links to the wiki for the uninitiated.)

For maximum flexibility, we could also give users the option to reply with a custom tag. Popular options could be added to the default suggestions, either automatically, or curated by the Sunshine Regiment. My concern with this approach is that custom tags may not have clear meanings, but a curated set could be linked to agreed-upon definitions in the wiki.

Tags also give us a good mechanism for straw polls. I've seen karma used this way on the old Less wrong, with a yes/no question asking for karma votes, (and a reply by the same author to counter balance it with opposite votes). The new weighted karma kind of destroys this feature. But with tags, we could vote with #yes #no or even #A #B #C #D, etc. for multiple-choice questions.

What happens if a user with 15 Karma upvotes something, gets to 30 karma, and then removes the upvote? Does it remove the original two karma, or the new three? Or are increases in karma weight retroactive, so your two vote becomes worth three when you reach 25 karma?

Nitpicking a misplaced parenthesis: I think the equation is probably

floor(log_5(karma+1))+1

rather than

floor(log_5(karma)+1)+1

New posts also start with their user having upvoted them once, so if you have 25 karma, you'll start with 3 points.

I have observed that this upvote does not effect the posters karma, this is a good design, but not clear from the text of this post.

Okay, the thoughts that originally prompted me to write this post:

1) Point Inflation - it currently feels weird/wrong to me that I'm getting more points than usual. And then, when I realize that 17 points actually just means maybe 5 people upvoted my post, that feels a little sad. However, this seems like a temporary problem, and over time I'd get used to it.

The new paradigm seems like "3-weight votes" is basically what you'll get as soon as you get the hang of things.

2) Not knowing how many people actually upvoted. I'm not sure how much I'd want this if I wasn't used to having it, but it seems like something that might be useful to actually be able to check.

3) 'Having' to upvote/downvote more than feels fair. I've upvoted a post and felt a little weird because it was a *slightly* good post that I wanted to incentivize it, but giving it 3 karma felt overkill for what I wanted to give.

Alternately: if something is hovering at 2 karma, and I think it was a little bit wrong or poorly-done in some way, I sometimes want to downvote it but then feel bad about kicking it into the negatives, which results in me not downvoting it at all.

4) The existing of voting weight isn't very intuitive - it feels at first like the site is just buggy, and this makes me want it to be more explicitly spelled out.

I have some thoughts on solutions, which I'll make separate comments so they're easier to upvote/downvote independently.

LW has a lot of old content, with old votes on it. This change will introduce a big disparity between the power of votes in the (pre-LW2) past and votes in the (post-LW2) future. I don't know that this is a problem, but it's a bit weird. (Consequence 1: when interpreting the score on a post or comment, you'll need to think about whether it's mostly been voted on recently or in the past. Consequence 2: when something has been voted on both recently and in the past, past voters' votes will have less influence on its score. Consequence 3: there will be a temptation once everything is integrated to go back to old material and retract-and-redo one's votes; people who choose to do this will in effect have more voting power -- as regards that old material -- than people who don't, and this may have quite a substantial effect on the karma scores of the people whose posts and comments those votes are made on.)

Bayesian Karma

Attention is a limited resource. I don't have the time or interest to read every comment on LessWrong. So what is karma even for? I use the karma score for one simple yes-no question: "Is reading this worth my time?".

Is displaying the number of upvotes minus downvotes really the best way to answer the question? Karma should not be about mere popularity, or path dependence based on whoever voted first. The weighting system is an improvement, but I think we can do better.

Display the estimated probability (as a percentage) that the post is worth my time.

Users will start with a reasonable personal karma score (say less that 50%), while established users with high karma on the old system will start with somewhat higher personal scores. Then the initial (prior) score of a post is the score of its author.

Users can then upvote or downvote posts, and this will be taken as Bayesian evidence about the quality of that post, shifting its score from its prior in the direction of the vote.

The probability that any given vote is accurate will be basted on the voters' karma percentage. Those with high personal karma will be assumed to have better judgement (because they write high karma posts), so their vote will have more weight.

And finally, the user's personal karma percentage will be adjusted from their prior by using the karma percentage of their posts as Bayesian evidence. This means that the personal karma percentage of a user is the estimated probability that the user's next post is worth reading.

A not-so-obvious consequence of weighted voting (in the form we currently have, as opposed to Rob's suggestion which I think I like a lot): if you know or suspect that something has been voted on only once since it was posted (or: since you last looked) then you can tell what voting weight the person who voted on it had, and hence get some information about who it was.

I've been thinking a lot about karma, as any numerical incentive system risks people going down incentive gradients, checking numbers and otherwise aiming for the number rather than the thing we want to target. This is a Hard Problem, since we 100% need karma for LW to work at all. The first level solution (in addition to weighting, which likely will help) is for everyone to be thinking hard about this and vote accordingly rather than voting locally and instinctively, but that feels insufficient. I'm planting this flag to encourage others to think more about karma problems outside of the weighting and inflation questions, which I'll address in a distinct comment while noting as an example that the Karma system encourages splitting comments in this way (which in this case I think is good).

On StackOverflow there's a minimum of 15 reputation to be able to cast upvotes. I think it's better when newly registered accounts don't have the ability to vote.

What happens to users with negative karma? (I have noticed one such user commenting on my posts.)

My karma from the old LessWrong did not port over. Is this normal?

Per-Paragraph Voting

I sometimes find myself agreeing (or disagreeing) with only part of a long comment.

Current options include replying with a quotation of the part in question, or voting on the whole thing based on the part. Sometimes this isn't worth the effort or seems unfair and I wish I could just vote on that one part.

Conciseness in comments is a virtue given our limited free time and attention, but I don't want to turn Less Wrong into Twitter. Due to inferential gaps, some concepts really do take long comments to get across, but aren't worth a top-level post. Our karma system should accommodate that.

I don't think it should be so fine-grained that we could insert a vote between every character. That's noise. It could really make a popular post hard to read. On the fine scale, per-sentence, or perhaps even per-punctuation mark is more reasonable.

But I suspect that the appropriate granularity is per paragraph. There's already a visual gap there, so you can insert karma votes (or tags) without hurting readability.

I'm not sure how to handle the case of edits. Minor typo fixes aren't a problem, but changing the number of paragraphs, or moving sentences around could be a problem. But there's nothing stopping us now from completely rewriting a comment to say something completely different. This is bad form, of course. And it might be bad form to change that in a deceptive way once it's voted on, but honest mistakes are possible.

I think edited posts (even comments) should have a publicly-visible history, like a wiki.

Is there a plan to import our old karma? I see I currently have none when I definitely have more than none on old LW.