Controversy - Healthy or Harmful?

Nah, I was talking more about the recent rash of posts on demographics and related topics in Stranger than History, the reactions to Eliezer's mass social engineering speculation in the April Fools' post, and a few other similar events. Though now that you mention it, people do seem to have grown a bit more trigger-happy with downvotes in recent months, block and otherwise.

If everyone had identical criteria for voting, we would see all postings having either large positive karma, karma near zero, or large negative karma. The more alike people are in their judgements, the less information the total score provides.

If you only can give 1 plus vote, 1 negative vote, or no vote at all, that seems to follow. If you rather could give, say 1-5 positive or negative Karma, we would see a greater variety of scores.

Also, note that many posts and especially comments have very few votes. This means that the votes actually cast will often not be typical of the whole population of possible voters in a system where people's votes vary considerably. In a system where people's votes are more alike, this obviously happens less frequently.

Yes! One click! A more complicated system would not be too complicated to use, but too complicated to be worth using. On Ebay, I'm happy to give feedback as positive/neutral/negative plus a few words of boilerplate, but I never use their 5-star scales for quality of packaging, promptness of delivery, etc. How do I rate a cardboard box out of 5?

I agree that one shouldn't have to rate, e.g. comments on say five different criteria. The system could be be somewhat more complex to comprehend, but you're right that it shouldn't be significantly more complex to use.

I think one obvious improvement is, though, to separate the posts into different categories which are to be assessed on different criteria. You could have one "objective information/literature review" section, one "opinion piece/discussion" section, one "meetup" section, and possibly a few more. In each section, you'd be rated on different criteria. That way, original pieces wouldn't be downvoted because they're not literature reviews, which seems to be Gunnar's (justifiable) complaint.

This system would be superior to the present, and no more complicated. I think further improvements are also possible, but those should be separately discussed.

People apply wildly different criteria.

This is a feature.

It is a property. It means some aggregation. But that is inevitable given a single bit.

In short, I think the karma system is excellent and sets a high bar for being improved on.

It is excellent compared to no rating or only single-direction voting. But is quite inferior to e.g. the slashdot system. Even a single-click system that provides different buttons for different types of posts would be better.

Thanks, that's very interesting. I was especially interested in this:

We can gauge each Superuser’s voting accuracy based on their performance on honeypots (proposed updates with known answers which are deliberately inserted into the updates queue). Measuring performance and using these probabilities correctly is the key to how we assign points to a Superuser’s vote.

So they measure voting accuracy based on some questions on which they know the true answer.

There is a difference between their votes and the kind of votes cast here, though; namely that on Less Wrong there is not in a strict sense a "true answer" to how good a post or comment is. So that tactics cannot be used.

On questions on which there is a true answer it is easier to track people's reliability and provide them with incentives to answer reliably. On questions which are more an issue of preference ("e.g. how good is this post?") that is harder.

Since you mention Slashdot, here's a little side effect of one of their moderation systems. At one point, they decided that "funny" shouldn't give posters karma. However, given the per-post karma cap of 5, this can prevent karma-giving moderation while encouraging karma-deleting moderation by people who think the comment overrated, potentially costing the poster tons of karma. As such, moderators unwilling to penalize posters for making jokes largely abandoned the "funny" tag in favor of alternatives.

I suspect that if an agree/disagree moderation option were added, it would likely suffer from a similar problem. Eg if we treated that tag reasonably and used it to try to separate karma gains/losses from personal agreement/disagreement, people would be tempted to rate a post they especially like as disagree/love/awe.

A more interesting idea, I think, would be to run correlations between your votes and various other bits, such as keywords, author, and other voters, to increase the visibility of posts you like and decrease the visibility of posts you don't like. This would encourage honest and frequent voting, and diversity. Conversely, it would cause people to overestimate the community's agreement with them (more than they would by default).

Interesting, and an interesting Slashdot link. I especially like the idea of "moderating the moderators". You do need to check whether people vote seriously in some way, it seems to me.

The only problem I see is Richard's concern below that multi-criterial systems, where you actually vote on all criteria, may turn out to be too cumbersome to use.