A few remarks about mass-downvoting

post by gjm · 2014-02-13T17:06:43.216Z · LW · GW · Legacy · 89 comments

To whoever has for the last several days been downvoting ~10 of my old comments per day:

It is possible that your intention is to discourage me from commenting on Less Wrong.

The actual effect is the reverse. My comments still end up positive on average, and I am therefore motivated to post more of them in order to compensate for the steady karma drain you are causing.

If you are mass-downvoting other people, the effect on some of them is probably the same.

To the LW admins, if any are reading:

Look, can we really not do anything about this behaviour? It's childish and stupid, and it makes the karma system less useful (e.g., for comment-sorting), and it gives bad actors a disproportionate influence on Less Wrong. It seems like there are lots of obvious things that would go some way towards helping, many of which have been discussed in past threads about this.

Failing that, can we at least agree that it's bad behaviour and that it would be good in principle to stop it or make it more visible and/or inconvenient?

Failing that, can we at least have an official statement from an LW administrator that mass-downvoting is not considered an undesirable behaviour here? I really hope this isn't the opinion of the LW admins, but as the topic has been discussed from time to time with never any admin response I've been thinking it increasingly likely that it is. If so, let's at least be honest about it.

To anyone else reading this:

If you should happen to notice that a sizeable fraction of my comments are at -1, this is probably why. (Though of course I may just have posted a bunch of silly things. I expect it happens from time to time.)

My apologies for cluttering up Discussion with this. (But not very many apologies; this sort of mass-downvoting seems to me to be one of the more toxic phenomena on Less Wrong, and I retain some small hope that eventually something may be done about it.)

89 comments

Comments sorted by top scores.

comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) · 2014-02-14T22:05:39.703Z · LW(p) · GW(p)

I've asked someone trusted to try to write a program to detect mass-downvoting and even check particular individuals, but we haven't been able to find anything! It's possible that the database export we're getting from the server admins is incomplete? I don't know.

Replies from: MugaSofer, gjm, Vaniver
comment by MugaSofer · 2014-07-10T20:18:20.440Z · LW(p) · GW(p)

Huh. Now that someone has been caught very much doing this, did you find out why you couldn't detect it before?

comment by gjm · 2014-02-14T22:33:54.607Z · LW(p) · GW(p)

Many thanks for looking into this!

What have you/they been trying to do? Unsupervised detection of mass-downvoting, or exploration of particular specific cases alleged to have occurred? If the latter, do the records not show the downvotes at all, or do they show downvotes from many different individuals in each case?

(Does the database contain information about when any given downvote happened? I guess probably not, which makes diagnosis more difficult.)

For example: I've been having ~5-10 recent and older comments downvoted per day, I think all during UK night-time or early morning (i.e., roughly 4pm to 2am Pacific), most days (I think all) for about the last 4 or 5. Approximately all of the recent-ish comments I've checked appear to have been downvoted exactly once. (A few very recent ones haven't been. A couple have been downvoted more than once; I guess that they were genuinely disliked on their (de)merits.)

If someone has the time to look, it would be interesting to know: Do the records show that those comments have been downvoted, and by whom? One downvoter, or a few, or many? Any signs of sockpuppetry, if many?

One possibility (though an unappealing one, not a very likely one a priori, and one that I feel a bit paranoid even mentioning) is a sort of downvoting ring of people willing to cooperate on downvoting at a rate just slow enough to avoid suspicion. That would be bad. And also sad.

[EDITED to add: I'd offer to help with the investigating, but as an interested party I probably shouldn't.]

[EDITED again to add: at 2014-02-15 08:41 GMT, it looks as if I haven't had a pile of downvotes in the last 8-10 hours. Maybe whoever it is has got bored, or maybe they've noticed evidence of someone looking. Or, of course, maybe I'm making the whole thing up, but anyone who finds that likely and cares to check can look at my comment history and see the evidence.]

Replies from: falenas108
comment by falenas108 · 2014-02-15T15:15:55.815Z · LW(p) · GW(p)

This is exactly the pattern for my downvoting too.

Replies from: Jiro
comment by Jiro · 2014-03-19T15:06:01.730Z · LW(p) · GW(p)

My last 100 comments contain exactly one comment that is not downvoted. I counted falenas108's and he has at least 50 comments in a row without one comment that is not downvoted. Starting with his first downvoted comment before the date of the above post, gwm has a string of 38 comments that are all downvoted, yet of his past 5 most recent comments (after he says the downvoter gave up) there are no downvotes.

It's obvious that we're all the victim of mass downvoting, and whatever Eliezer did didn't work. The system has to at least keep track of who downvoted which post, and it shouldn't be too hard for anyone with database access to get a count.

I suggest a simple change: for any logged-in user's own comments, display the name(s) of the people who downvoted him. I suspect that would fix the problem.

Replies from: Nornagest, CCC
comment by Nornagest · 2014-04-09T21:34:25.881Z · LW(p) · GW(p)

I suggest a simple change: for any logged-in user's own comments, display the name(s) of the people who downvoted him. I suspect that would fix the problem.

This would prevent mass downvoting, but at the cost of making votes socially significant. That's a bigger deal than it sounds like; it means that you can schmooze people by downvoting their enemies' comments (or by upvoting their own, if you extend it to that), and that you can incur the wrath of influential users if you dare to downvote theirs.

If you think there's too much politics in voting patterns here already, I guarantee this will make it ten times worse. About the best thing that could happen is people converging on a Facebook-style upvote-only pattern (maybe with exceptions for obvious trolls), and I still view that as distinctly inferior to Reddit's format for the purpose of promoting quality discussion.

Replies from: Jiro
comment by Jiro · 2014-04-10T05:17:35.799Z · LW(p) · GW(p)

How about the alternative of showing the name if greater than X percent of the user's last Y comments have been voted down by the same person? If you downvote 98 of a user's last 100 comments and they're not a blatant troll, you probably deserve to earn their wrath, influential or not.

If you don't want to do even that, then how about this instead: for each comment that was modded down, have a button to click for "moderation statistics". If you click on the button it will say something like "This comment was modded down by 2 users. User 1 has modded down 4 of your past 100 posts. User 2 has modded down 99 of your past 100 posts". Suspicious numbers like 99 out of 100 can be grounds for contacting an admin. There's nothing like having an actual human being doing actual adminning. This solution would also prevent someone from gaming the system by noticing that a name is displayed at 95% so he only mods down 94% of someone's posts.

Replies from: Nornagest
comment by Nornagest · 2014-04-10T05:20:39.370Z · LW(p) · GW(p)

At that point you might as well just write moderation tools for it, without requiring the user input step. Which wouldn't be a bad idea in theory, but it runs into the usual LW bottleneck of development time.

Replies from: Jiro
comment by Jiro · 2014-04-10T05:30:00.154Z · LW(p) · GW(p)

All that's actually needed in my case is an active admin that I can tell "this is an extremely suspicious pattern; please check it out". Having a button to display moderation statistics is just a way to make it harder for the admin to rationalize away not doing any adminning (or looking at it from the other side, for the user to be able to prove to the admin that the problem is worth taking the time to look into).

comment by CCC · 2014-03-24T11:57:58.743Z · LW(p) · GW(p)

I suggest a simple change: for any logged-in user's own comments, display the name(s) of the people who downvoted him. I suspect that would fix the problem.

How?

Let us say you suddenly discover that a user called (say) EvilDownvoter had been downvoting all your posts. How exactly does that help stop him?

Replies from: Jiro
comment by Jiro · 2014-04-06T08:47:31.121Z · LW(p) · GW(p)
  1. If they're also posting comments, revealing what they are doing would discredit them as a legitimate commentator, especially if history shows that they have an argument with me that they are trying to settle by forcing me off the site.
  2. If they're not posting comments, that means they have a single purpose account, which is an obvious troll.
  3. It would be possible to complain about them to an admin by name rather than complaining based on a statistical analysis of one's posts. It would be much harder for an admin to justify inaction, and much more likely for him to lose status given inaction, than if no name could be provided.
  4. Availability bias and related biases would make it easier to gain sympathy from others if the situation is easier to understand (no need to complain about Poisson distributions) and more specific (has a name attached).
comment by Vaniver · 2014-02-14T22:27:41.819Z · LW(p) · GW(p)

Can you give us some details on how the votes are stored in the server? This may be difficult/impossible to do in an offline fashion if the right sort of data isn't available.

Replies from: KnaveOfAllTrades, Risto_Saarelma
comment by KnaveOfAllTrades · 2014-02-15T20:14:36.422Z · LW(p) · GW(p)

I suspect that as with site modifications, those of us suggesting ways to find downvote stalkers would do best to figure out how LW works and do as much of the work as possible ourselves. So in this case, that'd probably mean downloading LW source code, figuring out the database structure, thinking of approaches to finding downvote stalkers, formalising them as database queries, then trying to get someone with database access to security check then run those queries. I suspect this because from what I gather Eliezer and those with database access (e.g. presumably Trike) tend to be busy enough or doing important enough other things that they are not willing to or it is not worth their time to do all this themselves, so we should do as much of it as possible to make things quicker for them.

Small amount of money to mouth: I did read through some of the webpages surrounding LW's source code, downloaded it, and spent a little time trying to figure out how the site and database work. But by the time I got to the point of looking at the code, I had little enough temporary motivation left and the relation of the scripts to each other and the difficulty of figuring out where to start was enough that I didn't get very far before I burned out for that night and haven't looked again since. :z

A guide to (learning) LW's code and database (even if just a few paragraphs along the lines of 'Start by looking at the main article display script, then move on to...' or commenting the scripts or something) might be higher leverage at this point with respect to improving the site than submitting small code improvements, since it might encourage several others to submit improvements. On the other hand, part of me suspects that the set of people held back just by that might actually be quite small (polarisation of would-be contributors into hardcore and indifferent with few in the middle--'if they were going to do it, they would have done it by now').

Given the distribution of coding ability here, it certainly seems ridiculous how slow stuff like this gets done, and I think it's due to trivial inconveniences, ugh fields, etc., of which figuring out the site and how to submit code etc. is possibly a large part.

Since Eliezer's response, I have slightly decreased my distribution over the level of downvote stalking, but there is still way too much evidence for me to honestly believe that there aren't any downvote stalkers; it would take at least an explanation of exactly what had been tried and possibly significant knowledge of database structure to convince me it's not happening at this point. So at present I defy the data.

comment by Risto_Saarelma · 2014-02-15T12:29:36.514Z · LW(p) · GW(p)

The server needs to explicitly remember every vote from every users for the interface where anybody can change or retract any of their past votes to be possible.

Replies from: Vaniver
comment by Vaniver · 2014-02-15T17:04:02.199Z · LW(p) · GW(p)

Right- but if it doesn't have a timestamp, then it's difficult to determine whether or not one user downvoted another user many times in a few minutes, which is a more reliable sign of the karmassassination problem than just how many times one user has downvoted another user.

You could go off comment timestamp- "has user X downvoted a contiguous block of comments from user Y, or are there holes (i.e. comments user X did not downvote)?"- but that's less useful, and more likely to catch the false positives of norm-enforcing users downvoting a repeated norm-breaker.

Replies from: Jiro
comment by Jiro · 2014-03-17T07:58:18.633Z · LW(p) · GW(p)

The past 80+ comments from me have all had at least one downvote. There is no reasonable way to interpret this other than as having a stalker.

And the solution to how not to catch false positives is to use some common sense. You're never going to have an automated algorithm that can detect every instance of abuse, but even an instance that is not detectable by automatic means can be detectable if someone with sufficient database access takes a look when it is pointed out to them.

Replies from: Vaniver, ChrisHallquist
comment by Vaniver · 2014-03-17T21:06:39.423Z · LW(p) · GW(p)

There is no reasonable way to interpret this other than as having a stalker.

Suppose we find the list of users who downvoted your recent comments, and there are fifteen users on that list, each of whom is an active poster in their own right. What conclusion would you draw from that?

(It may be that, when we actually find that list, there is one account, or a handful of mostly inactive accounts, that represent almost all of the downvotes, in which case 'stalker' is a reasonable conclusion. But it's not the only way the data could turn out.)

And the solution to how not to catch false positives is to use some common sense.

Common sense is costly. The point to doing this algorithmically is that you get a query result that says "these are the twenty cases that might be karmassassination" instead of "these are the twenty thousand cases that might be karmassassination" or "these are the zero cases that might be karmassassination."

It's also not particularly wise to run this check just on people who complain- part of the point of this is to prevent karmassassins from driving users away, which hasn't happened to the people who stuck around to complain (somewhat)- and at least a few users have a habit of downvoting any comments complaining about karma loss because they don't like comments that complain about karma loss, and so they'll be extra likely to show up on that list.

Replies from: Jiro
comment by Jiro · 2014-03-17T21:41:59.158Z · LW(p) · GW(p)

Suppose we find the list of users who downvoted your recent comments, and there are fifteen users on that list, each of whom is an active poster in their own right. What conclusion would you draw from that?

I'd conclude that this is an extremely weird statistical anomaly which is not one user moderating down comments, but looks almost exactly like it is. One user doing a lot of downmods has to apply the downmods to separate comments, so his downmods are spread out. 15 users producing the same total number of downmods independently of each other would produce something a lot closer to a Poisson distribution with an expected value of 1, and there should be a number of comments that have zero downmods just by chance.

comment by ChrisHallquist · 2014-03-17T20:51:55.312Z · LW(p) · GW(p)

And the solution to how not to catch false positives is to use some common sense. You're never going to have an aytomated algorithm that can detect every instance of abuse, but even an instance that is not detectable by automatic means can be detectable if someone with sufficient database access takes a look when it is pointed out to them.

Right on. The solution to karma abuse isn't some sophisticated algorithm. It's extremely simple database queries, in plain english along the lines of "return list of downvotes by user A, and who was downvoted," "return downvotes on posts/comments by user B, and who cast the vote," and "return lists of downvotes by user A on user B."

Replies from: Vaniver
comment by Vaniver · 2014-03-17T21:16:39.412Z · LW(p) · GW(p)

It's extremely simple database queries, in plain english along the lines of "return list of downvotes by user A, and who was downvoted," "return downvotes on posts/comments by user B, and who cast the vote," and "return lists of downvotes by user A on user B."

And then what will you do with that data? If you find that GrumpyCat666 cast most of the downvotes, does that mean that GrumpyCat666 is a karmassassin, or that GrumpyCat666 is one of the gardeners?

(I can't find the link now, but early on there was a coded rule to prevent everyone from downvoting more than their total karma. This prevented a user whose name I don't recall, who had downvoted about some massive fraction of all the comments the site had received, from downvoting any more comments, but this was seen as not helpful for the site, since that person was making the junk less visible.)

comment by Emile · 2014-02-13T17:38:30.035Z · LW(p) · GW(p)

Time for a poll!

Systematic downvoting of users (that aren't spambots or obvious trolls) is wrong. [pollid:600]

Systematic downvoting of users (that aren't spambots or obvious trolls) is against community norms (i.e. people should already know it's wrong). [pollid:601]

It would be nice if admins had a way of automatically detecting such behavior (e.g. running an SQL query to pick up patterns of mass-downvote) [pollid:602]

By the way, dear LessWrong reader, have you ever been a victim of mass downvotes? [pollid:603]

And have you engaged in it? [pollid:604]

(not that I think such polls should have power of law, it's just nice to have an idea of the opinion of the community)

Replies from: NancyLebovitz, FeepingCreature, RobbBB, solipsist, solipsist, eggman, ntroPi
comment by NancyLebovitz · 2014-02-13T21:31:36.025Z · LW(p) · GW(p)

I've done a minor version of mass downvoting-- I normally let mildly annoying comments go, but I'll downvote them if they're from one particular poster.

comment by FeepingCreature · 2014-02-14T02:00:42.657Z · LW(p) · GW(p)

I think if I'd notice that I always tend to downvote some person's comments when I see them, I might end up looking at their history for comments I'm likely to also disfavor. But I wouldn't downvote their comments just because they're the one who made them; I'd judge each comment individually. I don't know if this is against community norms.

comment by Rob Bensinger (RobbBB) · 2014-02-17T02:23:45.894Z · LW(p) · GW(p)

Two features of this poll make me doubt its usefulness:

  1. You don't define 'systematic'. I've gone through user histories and looked at lots of their old comments, doling out lots of positive or negative karma in a short period of time based on the quality of the old comments. But I have no idea whether that makes me a 'systematic downvoter' or 'mass downvoter' in the sense people here are worried about. Your last question is particularly ambiguous, since it's not clear whether 'e.g. looking if a user has a history of other bad comments' is meant to be an example of 'systematic' or of 'not systematic' down-voting.

  2. You don't allow people to express agnostic or highly uncertain views about anything. You allow 'tends toward X' and 'don't care', but not 'I don't know'. I think I gave overly extreme answers because I do care about this issue and about whether it's harming the community, but I don't have a clear view yet of whether it's net harmful.

comment by solipsist · 2014-02-18T02:25:01.375Z · LW(p) · GW(p)

Have to revise my poll answer. 92% of my comments from the past month were downvoted today.

comment by solipsist · 2014-02-17T22:10:29.287Z · LW(p) · GW(p)

I reported "never been mass downvoted", but in the past few hours 92% of my comments since January 29th were downvoted :-/.

comment by eggman · 2014-02-13T23:57:56.139Z · LW(p) · GW(p)

I wasn't logged in when I initially filled out this survey, and the cookies in my browser might have been acting funny for whatever reason. So, I may have mistakenly filled out this form twice. If I did without anonymity, ignore my duplicate result. If it was anonymous, and it appears a user did enter twice, and the results are the same, then ignore one of them, as it could easily be from me.

Replies from: army1987
comment by A1987dM (army1987) · 2014-02-14T08:57:58.729Z · LW(p) · GW(p)

So, non-logged in users are allowed to vote to polls? (Does anyone know if that's deliberate?)

Replies from: Vladimir_Nesov
comment by Vladimir_Nesov · 2014-02-14T17:38:14.495Z · LW(p) · GW(p)

Does anyone know if that's deliberate?

It's not true, at least normally. If you are logged out and press "Vote", you are asked to log in.

comment by ntroPi · 2014-02-14T16:38:01.128Z · LW(p) · GW(p)

I disagree about having this problem solved by moderators. Changing the karma system would be preferable i.e. by removing the downvotes or having downvotes only affect the individual post but not on the total karma of the user.

comment by IlyaShpitser · 2014-02-13T18:36:23.797Z · LW(p) · GW(p)

I have two questions:

(a) Has there been a common feature of victims of mass downvoting (perhaps political views?), and

(b) Why do people care about karma so much? I don't think there is a lot of signal there.

Replies from: gjm, Lumifer
comment by gjm · 2014-02-13T18:49:24.780Z · LW(p) · GW(p)

(a) There is some evidence that engaging in discussions on gender and taking a view nearer the "feminist" than the "reactionary" end of the spectrum sometimes results in getting hit by mass-downvoting. I'm not sure whether other (leftish?) political opinions provoke it.

(b) I don't know how much is "so much", and hence in particular whether it's true that people care about karma "so much". But the karma system is part of how LW is built; it's meant to be somewhat motivating for writers and somewhat informative for readers. Indiscriminate downvoting (or for that matter indiscriminate upvoting, though oddly one rarely hears of people complaining that this has happened to them) of a particular user's comments is liable to distort both.

Replies from: buybuydandavis, IlyaShpitser
comment by buybuydandavis · 2014-02-13T20:59:57.016Z · LW(p) · GW(p)

It's more informative for writers than readers, IMO.

Some of my most upvoted posts were ones where I was venting and expected to be dinged on. I often get downvoted for tone, so when a rant gets a large and consistently positive vote, I find that an interesting tell. I don't think I've hit on an interpretation yet, but those are my posts that I should be interpreting. I notice that I am confused, etc.

comment by IlyaShpitser · 2014-02-13T19:11:51.569Z · LW(p) · GW(p)

I don't know how much is "so much"

Enough to write a top level post about it.

Replies from: gjm
comment by gjm · 2014-02-13T20:41:38.016Z · LW(p) · GW(p)

I care about mass-downvoting because I think it is damaging to the LW community. If that counts as "caring about karma so much" then I suppose that indeed I do, but I don't really see why it should be surprising.

comment by Lumifer · 2014-02-14T17:34:15.878Z · LW(p) · GW(p)

Why do people care about karma so much?

Because people are not rational, and, in particular do not like negative feedback.

Replies from: Error
comment by Error · 2014-02-14T18:49:00.908Z · LW(p) · GW(p)

In this community it might be more accurate to say that people do not like useless negative feedback. I'm perfectly happy to get useful negative feedback, and I think most who participate here would agree.

[ETA: A heavily downvoted post is useful information; it says people don't like what you posted, and this is something you can analyze, ignore, or change appropriately. Mass-downvoting is discouraging without providing anything useful; it says someone doesn't like what you posted because you're you, and that's kind of hard to change.

My personal opinion is that mass-downvoting is basically trolling and that complaining about it is counterproductive for the same reason that responding to trolls is counterproductive.]

comment by Douglas_Knight · 2014-02-13T18:52:58.607Z · LW(p) · GW(p)

Check out this comment by Eliezer, within the last month.

I have always been skeptical of the value of posts like this. I have since become annoyed at the quantity of them. The fact that Eliezer responded in a particular place that was not of this form is evidence that they are not a good way of attracting his attention. Maybe you should contact him directly? (though I'd wait a full month from the above comment) If you want a statement from moderators, maybe you should contact them directly?

Replies from: Tenoke
comment by Tenoke · 2014-02-13T19:17:28.691Z · LW(p) · GW(p)

Maybe you should contact him directly?

In my experience contacting him directly has been useless and he claims that there is no way whatsoever to check a suspect's voting history.

Replies from: Douglas_Knight
comment by Douglas_Knight · 2014-02-13T20:27:03.655Z · LW(p) · GW(p)

His assertion that there is no way to check seems to me a better outcome than these posts shouting into the wind that don't get any response.
But now that he made the particular comment, one could contact him to ask for an update, rather than a whole request.

Replies from: ChrisHallquist, Tenoke
comment by ChrisHallquist · 2014-03-03T07:25:03.540Z · LW(p) · GW(p)

His assertion that there is no way to check seems to me a better outcome than these posts shouting into the wind that don't get any response.

Did he assert that, exactly? The comment you linked to sounds more like "it's difficult to check." Even that puzzles me, though. Is there a good reason for the powers that be at LessWrong not to have easy access to their own database?

Replies from: Douglas_Knight
comment by Douglas_Knight · 2014-03-03T16:28:03.420Z · LW(p) · GW(p)

My two paragraphs refer to two different things Eliezer said. The contrast is indicated by the word "but." Tenoke says that he asserted that, exactly. I assume Eliezer was just lying to get him to shut up.

There are a lot of reasons not to look directly at the database. But once the person wrote an acceptable query years ago, yes, it should be easy to just try it again.

Replies from: ChrisHallquist
comment by ChrisHallquist · 2014-03-03T16:55:01.220Z · LW(p) · GW(p)

Oh, I see now. But why would Eliezer do that? Makes me worry this is being handled less well than Eliezer's public statements indicate.

comment by Tenoke · 2014-02-13T20:31:26.289Z · LW(p) · GW(p)

His assertion that there is no way to check seems to me a better outcome than these posts shouting into the wind that don't get any response.

Sure, expect that I didn't exactly believe him.. even more so in the light of the comment you highlighted..

comment by ricketybridge · 2014-02-13T18:53:03.491Z · LW(p) · GW(p)

I agree. Getting downvoted feels bad man, no matter the reason.

Replies from: None
comment by [deleted] · 2014-02-14T03:58:35.177Z · LW(p) · GW(p)

This along with the semi-regular accounts of downvote abuse makes me question what advantages a +/- system has over a strictly + system. The ego threat of being downvoted seems more like a contribution deterrent than a learning signal. Is there anyone who could explain to me why the current system is better?

Replies from: gjm, ricketybridge
comment by gjm · 2014-02-14T09:14:41.204Z · LW(p) · GW(p)

Downvotes (plus a mechanism for making heavily-downvoted things less prominent, which LW has) provide protection against spammers and trolls.

Downvotes provide a non-cluttering way of indicating when a particular kind of comment (e.g., low-value template-y attempts at humour, which if not discouraged are apt to start taking over everything) are not appreciated by the community.

Downvotes provide a signal to people who simply aren't able or willing to make a positive contribution that they might do better to go elsewhere. Perhaps this last one is some of the motivation behind some or all mass-downvoting, but what evidence I have suggests that it's often done to people whose contribution is obviously positive to anyone who isn't so politicized that they see the mere presence of someone who thinks differently from them as a threat.

Replies from: ntroPi
comment by ntroPi · 2014-02-15T17:26:30.404Z · LW(p) · GW(p)

A less extreme modification of the karma system would be to keep the downvotes but change how karma is calculated for the users.

Karma could be defined as the sum of all votes of posts with positive total score. An alternative change would be to count only the upvotes and ignore downvotes completely for the karma calculation.

In both cases the general correlation between users that post great content and high karma would stay intact but mass downvoting would no longer feel as threatening. All the signaling benefits you mentioned would still work in this modified system.

Do you think these are acceptable changes to the karma system?

Replies from: gjm
comment by gjm · 2014-02-15T20:18:30.840Z · LW(p) · GW(p)

Doesn't seem crazy. I'd have to give it more thought before deciding whether it's likely to be an improvement. (Not that it particularly matters whether I think it's likely to be an improvement!)

But mass-downvoting would still be an abuse of the system and make karma less informative. Better to make it go away, if possible, either by preventing it or by disincentivizing it.

[EDITED to add: I mean "and make the scores of posts and comments less informative".]

Replies from: ntroPi
comment by ntroPi · 2014-02-15T21:38:08.589Z · LW(p) · GW(p)

I don't think prevention is very likely as EYs comment suggests that moderator intervention will be very hard or even impossible, so disincentivizing is probably the way. I hope my suggestions would remove a motivation for mass downvoting by making it impossible to attack someones karma.

comment by ricketybridge · 2014-02-14T05:30:15.662Z · LW(p) · GW(p)

Completely agreed. That's why some subs only do +, no -. I cannot defend the current system. ;-)

comment by hyporational · 2014-02-13T20:48:15.233Z · LW(p) · GW(p)

Engaging any controversial topic might earn you a fan it seems. You contributed to White Lies. There was a fair bit of disagreement in that thread and it wasn't pretty. Prismattic and Wedrifid also remarked in the previous open thread after participation in White Lies that they were mass downvoted. I got unusually many downvotes in the thread too, which I don't find surprising, but haven't experienced systematic downvoting.

Previous top level post

comment by falenas108 · 2014-02-13T19:29:03.365Z · LW(p) · GW(p)

Yeah, I've been getting this too (see here for when I posted in the open thread about it).

So far, I've lost about 300 karma.

comment by [deleted] · 2014-02-14T15:22:10.082Z · LW(p) · GW(p)

How easy would it be to have downvotes be on some kind of timer, where you could only downvote once every N minutes? (A time is arbitrary and flexible based on experimentation)

This seems as if it would prevent someone from trivially systematically going through and downvoting every post by a poster, but still allows for someone to read something and downvote it on a general basis.

Replies from: Viliam_Bur, Nornagest, gjm
comment by Viliam_Bur · 2014-02-15T10:04:32.544Z · LW(p) · GW(p)

If someone's goal is to mass-downvote all comments made by someone else, this wouldn't stop them at all. Just keep the page open in another tab, and every N minutes downvote the next comment.

For people who read quickly and vote in real time, it would be inconvenient. (For example, Stack Exchange has a similar thing: I can read a short comment in 1 second, but I am allowed to vote only once per 5 seconds. So I read 20 comments in a row and then close the page without voting, because I don't see a point in waiting.)

Replies from: CCC
comment by CCC · 2014-03-24T12:05:31.680Z · LW(p) · GW(p)

Given that the aim is to prevent mass downvoting of a single person, it could perhaps be more usefully implemented as a timer that prevents one person from downvoting the same user more than X times every N minutes (for some low X, perhaps only one or two).

This would have less effect on the casual browser (since, normally, you'd be reading comments from several people) while still making it harder for the mass downvoter.

There may need to be some changes to the database to implement this, of course.

Replies from: Viliam_Bur
comment by Viliam_Bur · 2014-03-24T14:22:44.541Z · LW(p) · GW(p)

A person can make five stupid comments in the same thread, in which case they would deserve five consecutive downvotes. (Making ten or more stupid comments in the same thread seems unlikely. Unless it's a troll, in which case their karma will become negative either way.)

By the way, I don't remember seeing five stupid comments by the same person recently, but it seems to me that a year or two ago it was relatively common, e.g. in political threads, where people kept replying to each other and refused to give up.

Replies from: CCC
comment by CCC · 2014-03-24T14:39:03.601Z · LW(p) · GW(p)

X=5 N=120, perhaps?

That allows up to five stupid comments from the same poster to be downvoted in rapid succession, but makes it a lot harder for the mass downvoter.

comment by Nornagest · 2014-02-14T18:50:52.424Z · LW(p) · GW(p)

I don't know anything about LW's internals, but in most sane architectures you'd want to have a last-downvote timestamp linked to each user's account, and to compare the timing of incoming downvotes against it. If the difference is too small, you reject the input and display a message similar to the one you get when you're about to post in a heavily downvoted thread. That message has a few issues -- it doesn't come up if the vote total upthread dropped below the threshold between when the page was loaded and when the post was submitted -- but since we only care about session-local behavior here, that shouldn't be a problem. Synchronization between the session and server time might be harder but that depends on details I don't know.

I can see a behavioral pitfall, though. I typically decide on votes after reading through a screen's worth or so of a thread, and this sometimes implies two or more downvotes (for example in the cases of short low-quality posts or discussions inappropriate for the site). I'd find it gratingly aversive to be forced to wait N seconds or minutes between posts in these cases; I don't know if this happens often enough to outweigh the mass-downvoting problem, though.

comment by gjm · 2014-02-14T19:31:20.785Z · LW(p) · GW(p)

I agree with Nornagest that this would be really annoying for many non-abusive posters.

I'm also not sure how effective it would be against abusive mass down-voters. Whoever's currently stalking me is systematically downvoting a bunch of my old comments every day. I assume s/he is just gradually going through LW's list of my comments. It doesn't seem like it would be difficult to do it gradually throughout the day, or even to write a computer program to do it automatically.

comment by buybuydandavis · 2014-02-13T20:49:30.266Z · LW(p) · GW(p)

What does "make my votes public" do?

I've never noticed any indication of anyone's identity associated with a vote. Does everyone have it set to private?

Replies from: Douglas_Knight
comment by Douglas_Knight · 2014-02-13T21:04:32.538Z · LW(p) · GW(p)

"Make my votes public" only applies to votes on top level posts, not votes on comments. Mr Hen turned it on, so his about page has, in addition to the normal "Comments" and "Submitted" tabs, tabs called "Liked" and "Disliked" (also, "Hidden," whatever that means*; some people have a "Drafts" section, but it's usually empty). Liked and Disliked show posts that he voted on. The point is for people who share his taste to get recommendations. It is not convenient for posters to find out how he voted on their posts, presumably by design.

* I'd expect Hidden to be related to the preferences choices "don't show me posts after I've [dis]liked them," but it doesn't seem to match in his case.

comment by trist · 2014-03-01T09:07:37.105Z · LW(p) · GW(p)

New datapoint on mass downvoting:

Sometime between this comment and my last comment, approximately all of my comments were downvoted exactly once. Seems kinda strange.

(I don't have anything that I want to post to Main prepared anyway, so karma's kinda a moot point, but I hoped this could be helpful if anyone ever does look into it, and times are included in downvote logs.)

Edit: Hehe! And within ten minutes, this one joined the rest of them...

comment by Strilanc · 2014-02-13T19:32:34.498Z · LW(p) · GW(p)

Relevant post from the stack overflow blog: Vote Fraud and You (not so much for the post itself as the comments and links).

Given that the karma system has value, does not function well when people mass-downvote, and that people are actually mass-downvoting, it's important to deal with the problem.

comment by Brillyant · 2014-02-13T21:47:27.665Z · LW(p) · GW(p)

It is interesting to me that the karma system continues to be a discussion on LW.

Is there a correct answer to what an optimized karma system entails? How optimized is the current system? Do people really care that much?

At this point, a high comment score signals to me (A) good, useful comment and (b) someone who is super invloved in LW and LW-approved stuff like CFAR or MIRI or whatever. I'd say A is the dominant signal, but B is getting stronger all the time.

As far as mass-downvoting, I think it is problem for some users and I believe there has got to be some way to mitigate its effects. I sense a apathy from whoever makes the decisions to do anything about mass-downvoting, or the karma system writ large. Why? Who knows. I'm apathetic about their apathy.

Replies from: Pfft, gjm
comment by Pfft · 2014-02-13T22:41:06.487Z · LW(p) · GW(p)

I don't feel this is very much an issue with the karma system, just a problem with "mean people suck". No matter what technical system is in place, there will probably be some ways for people to be meanies---sending lots of nasty personal messages from behind a TOR proxy, for instance. The fact that in this case they decided to (ab)use the voting system is incidental, and a solution need not involve making any dramatic changes to how the voting mechanisms work.

comment by gjm · 2014-02-13T21:54:57.736Z · LW(p) · GW(p)

I'd say A is the dominant signal, but B is getting stronger all the time.

You reckon the scores on individual comments are substantially a matter of the commenter's involvement in LW, CFAR, MIRI, etc.? That seems very surprising to me, and I can't say I've noticed it. If so, it's the same sort of problem as mass-downvoting: it would mean that comment scores are assessments of people rather than comments, and on the basis of something other than the quality of what they write. That would be very bad.

I'm apathetic about their apathy.

OK. I'm left wondering why you bothered to write that paragraph, though. Perhaps I'm apathetic about your apathy about those people's apathy.

Replies from: Brillyant
comment by Brillyant · 2014-02-13T22:25:16.443Z · LW(p) · GW(p)

You reckon the scores on individual comments are substantially a matter of the commenter's involvement in LW, CFAR, MIRI, etc.?

Yes, I reckon it. Though I suppose it would depend on your exact definition of 'substantial'. I think stuff EY says gets double digit votes with only limited exceptions, and I don't think it would matter much what he said as long as it was spelled right and generally on-topic. I think EY says a lot of interesting shit in highly interesting ways, but many of the vote counts for his comments are based of the fact he is a celebrity around here. Similar stuff is at play for others, I'd hypothesize. Nothing too surprising. I'm sure I'd tend to unconsciously favor the online comments of those I'd met in the flesh more too—that is, if I'd met anyone from LW in the flesh. I suppose the same could be said of people I'd interacted with for many years online.

OK. I'm left wondering why you bothered to write that paragraph, though. Perhaps I'm apathetic about your apathy about those people's apathy.

So, you are metapathetic. Now we're talking. :)

I commented cuz' it's sincerely interesting to me. I think the karma system is okay. And it is better than anything I could have devised. But I don't think it is anywhere near optimal. And on a website where people are fervant about optimizing stuff using rationality, it tickles my irony bone to see them use a non-optimal system to discuss how to optimize the universe.

Replies from: Nornagest
comment by Nornagest · 2014-02-13T22:48:58.536Z · LW(p) · GW(p)

I think stuff EY says gets double digit votes with only limited exceptions, and I don't think it would matter much what he said as long as it was spelled right and generally on-topic. I think EY says a lot of interesting shit in highly interesting ways, but many of the vote counts for his comments are based of the fact he is a celebrity around here.

I think this is... well, half right. I've seen plenty of Eliezer comments with zero, near-zero, or negative karma; his less contentful comments tend to stay around zero where they should be, and the negative ones tend to be a lot more negative than I'd expect them to be from their content. On the other hand, when he posts something insightful, he also tends to get voted up more than I'd expect from the content. Same goes for Lukeprog, Yvain, and the other high-karma users (though I can't recall ever seeing a negative-karma Yvain post, now that I think about it).

In other words, I suspect that community status serves as an amplifier rather than a scalar bonus. Since almost everyone has a positive karma balance, though, this hashes out to a positive effect on average.

(Incidentally, I have met a lot of the high-karma users in the flesh, but I don't think it affects my votes much on average. I think that in most cases I treat their meat presence and their online persona as quasi-separate entities in my mind.)

Replies from: ESRogs
comment by ESRogs · 2014-02-14T00:09:52.321Z · LW(p) · GW(p)

Part of the explanation for this (though probably not all of it), is people like me who go looking for recent comments from people we've identified as often saying insightful things. I don't indiscriminately upvote all such comments, but more eyeballs means more upvotes.

comment by JQuinton · 2014-02-14T18:29:24.968Z · LW(p) · GW(p)

I recently wanted to comment on a post that had a lot of downvotes. I think I was prompted to sacrifice some karma in order to post in it. Maybe something similar could be implemented for mass downvotes? Say if you downvote someone more than 5 times (or some threshold where it's less likely that you're downvoting the same person by chance) you have to sacrifice some karma to do so.

Replies from: gjm
comment by gjm · 2014-02-14T19:56:08.698Z · LW(p) · GW(p)

More than five times ever? That's going to be a problem for long-term participants.

More than five times in some fixed period? That would make more sense, though it might need extra information recorded in the database. The right way might be something like this. For each pair of users, store an "amount of recent downvoting" that decays exponentially over time and gets incremented with each downvote; then apply a karma toll that increases with the value of this. It would also be necessary to store the last time the downvote density was updated, so that the exponential decay can be applied properly.

comment by Stefan_Schubert · 2014-02-14T10:36:43.529Z · LW(p) · GW(p)

The Karma system has its advantages but the present version strikes me as a bit too crude. For one thing, a down-vote from someone who down-votes everything is not as good evidence that the post is actually bad than a down-vote from someone who barely ever down-votes anything. Also, a down-vote from a poster with lots of Karma is better evidence that the post is bad than a down-vote from a poster with lots of Karma.

My suggestion is therefore that for any given level of Karma that you have, there should be definite limit to the number of up votes and down votes you can make. If you went over that limit, your votes would get diluted, so that, e.g. one up-vote from you only would give the poster .8 Karma.

If such a system would be introduced, it would also be natural to let people give less-than-full votes (say .5 up-vote, .3 down-vote, etc).

The drawback of such a system is of course that it is more complicated, but I think it would give a better picture of what posts are good and what aren't.

Replies from: gjm
comment by gjm · 2014-02-14T10:49:41.251Z · LW(p) · GW(p)

There are online fora where downvoting costs you some of your own karma. That might be an effective compromise.

I like in principle the idea of weighting more informative votes more. I suppose what you'd want is that a vote counts for +/- log p where p is the fraction of the voter's votes that are the same as it. (I'm not sure whether you'd want to count "neither up nor down" as a vote for this purpose. Let's say we don't.) So if someone downvotes everything, her downvotes would be worth nothing, but then her first upvote would be worth log N where N is her total number of votes. If we take the logs to base 2, then a vote either way from someone who upvotes and downvotes equally would be worth exactly 1 unit.

That would be kinda fun. I think it wouldn't actually be that hard to implement (provided we didn't want changes in vote frequency to be retroactive -- which I think is correct) but it would complicate things and it might get gamed in ways more harmful than what we have now (e.g., casting a lot of meaningless votes on random comments to increase the power of one's votes in the other direction).

Replies from: Stefan_Schubert
comment by Stefan_Schubert · 2014-02-14T11:13:38.834Z · LW(p) · GW(p)

My suggestion is rather like this. A new poster would get the ability to make some upvotes and some negative votes for free (say 20; perhaps you'd get more upvotes than downvotes). I think that's necessary to encourage newcomers; otherwise they'd feel locked out.

As your karma increases, the number of positive and negative votes you can make increases. However, if you only ever give one down-vote, it's not like that's going to give the person receiving it minus 100 karma. There's still an upper limit on -1. Otherwise, single downvotes could have too large an impact.

My suggestion is thus a) that the votes of people who vote a lot (either up or down) would get diluted and b) that the votes of people with little Karma would get diluted.

One possible drawback is that there might be people who read a lot, are good at distinguishing good posts from bad, but which don't post and hence have little Karma. Such people's votes would get heavily diluted in this system. But I suppose that's a minor thing and all systems have their drawbacks after all.

Replies from: gjm
comment by gjm · 2014-02-17T10:38:37.277Z · LW(p) · GW(p)

I'm not sure that last one is really a drawback. It amounts to encouraging more active participation from people with good judgement. That would be a good thing.

I think I was unclear in my earlier comment: when I wrote "I suppose what you'd want is ..." I didn't actually mean I thought that was what you were proposing. I was ruminating on what happens if one takes seriously the idea of estimating how much evidence a given vote from someone with a given historical voting pattern provides.

I agree that just restricting the amount of voting each user can do is a simpler way to achieve something along similar lines.

comment by Sabiola (bbleeker) · 2014-02-14T11:09:32.242Z · LW(p) · GW(p)

Maybe it would help to do what they do on Reddit: disable voting after a certain amount of time.

Replies from: John_Maxwell_IV, gjm
comment by John_Maxwell (John_Maxwell_IV) · 2014-02-15T02:57:40.725Z · LW(p) · GW(p)

Hm, I always figured that was to decrease the load on their servers by making pages more cacheable.

Replies from: bbleeker
comment by Sabiola (bbleeker) · 2014-02-15T14:55:31.790Z · LW(p) · GW(p)

Two birds, one stone.

comment by gjm · 2014-02-14T13:15:40.207Z · LW(p) · GW(p)

That would certainly prevent the situation where A downvotes 10 old comments from B every day (which is what's been happening to me lately). It wouldn't stop A downvoting every new comment from B as it arrives, though, and that's almost as bad.

comment by Stefan_Schubert · 2014-02-15T08:52:11.886Z · LW(p) · GW(p)

As I understand the Karma system, you're supposed to upvote intelligent, methodical, readable and original articles, since the point of the Karma system is to provide evidence to other readers that the post (and poster) in question is interesting. That is, you're not supposed to upvote based on whether you agree with the conclusions. The same goes for downvotes, obviously: you shouldn't downvote articles or posts simply because you don't like the conclusions.

Still, I think that happens quite a lot. I got four downvotes within a short time-span a few hours ago on this post and it's comments (people who agreed with me were also were downvoted): http://lesswrong.com/r/discussion/lw/jo8/private_currency_to_generate_funds_for_effective/ Now I don't claim that my post and comments are master-pieces by any means, but I don't really see that they're so sub-par from the first point of view as to deserve a stream of down-votes. My suggestion is that some people don't like the conclusions and therefore downvote it.

I think people have a problem distinguishing these two things (in fact, I think it's a universal human trait and that very few people are able to fight their bias against posts containing conclusions they don't like successfully). If so, it might be a good idea to give two sorts of Karma: "conclusions-based Karma" and "argument-based Karma". Personally, I think it is the argument-based downvotes that count and that I wouldn't look so much at others' conclusion-based Karma, but it might still be useful to have it so as to make it salient that the quality of an argument is distinct from whether you like the conclusion or not.

Another idea is to force people to give arguments for their votes. Of course you couldn't do that with every vote but if someone has a very unusual voting pattern, I think it might be a good idea that that person would have to explain his or her votes.

My tentative opinion is nevertheless that the Karma system does more good than harm, but that it should be improved.

Replies from: satt
comment by satt · 2014-02-26T03:56:47.560Z · LW(p) · GW(p)

That is, you're not supposed to upvote based on whether you agree with the conclusions. The same goes for downvotes, obviously: you shouldn't downvote articles or posts simply because you don't like the conclusions.

I'd guess you differ from a lot of LW here. I definitely disagree with what I've quoted...although I won't downvote you for that!

It's good to downvote comments that are flagrantly wrong or misleading, and to upvote cogent comments, and sometimes that's going to amount to downvoting because I disagree, or upvoting because I agree. Sometimes the most efficient way to handle a crappy comment is to hit it with a downvote and move on, rather than getting bogged down in an argument.

comment by khafra · 2014-02-14T13:41:52.886Z · LW(p) · GW(p)

Yeah; maybe if one user downvotes comments by another user at more than 1 standard deviations above the rate at which that user collects downvotes from all other sources, we could just silently ignore voting from user #1 (and/or flag #1's account for voting pattern review, by a human)?

comment by EGarrett · 2014-02-14T05:55:16.103Z · LW(p) · GW(p)

Ironically, I found it annoying when Youtube appeared to disable downvotes, but that seems to be a handy solution to the issue. On the other hand, before that they simply didn't show the number of down or upvotes something had until it received a significant amount which helped also.

comment by moridinamael · 2014-02-14T03:06:41.515Z · LW(p) · GW(p)

The mere existence of the "downvote" mechanism is social and emotional poison.

It's like you're sitting around a table chatting with friends, and everyone has their hands under the table, with an annoying buzzer in one hand and a friendly chime in the other hand. And everyone is chatting politely, and then people realize that they can give "anonymous feedback" by pressing the annoying buzzer when they don't like what someone just said, and the chime when they approve of what someone just said.

But as soon as people start to do this, it is clear that, whatever the intention, the only possible actual outcome is noise. You say something and receive two chimes and a buzzer. What does that mean? What do you do with that? Alice responds, and then Bob responds to her, except he precedes his response with a buzzer, so he might as well have preceded the whole remark with "hey, asshole!" even though his comment would have been perfectly constructive without the buzzer.

Basically my point is the buzzer ruins everything. If you went to a party and people were using these buzzers you would leave.

Replies from: ThisSpaceAvailable, Barry_Cotter
comment by ThisSpaceAvailable · 2014-02-16T22:28:28.731Z · LW(p) · GW(p)

The downvote option does make it easier to be negative, but it also gives people an option in between ignoring trolls (and risking people thinking that other people don't have a problem with them) and engaging them (and rewarding them). It also lets people express disapproval of a post without wasting another post saying nothing but "I disagree". Of course, people still do downvote and then also post a comment that has no semantic content other than "I don't like you".

At a party, there are all sorts of feedback: smiles, laughs, nods, frowns, awkward silences, glares, etc. The upvote/downvote is a rough analog of that.

comment by Barry_Cotter · 2014-02-14T05:49:24.181Z · LW(p) · GW(p)

My first thought on reading this was of one person driven away from LW, Peter D. Jones by extensive downvoting. Reading his posts this was obviously a good thing. Enabling downvoting has driven away at least one person I really wish was still an active contributor, (wedrifid)[http://lesswrong.com/lw/jm7/open_thread_for_february_3_10/aj15/]. I think given that Hacker News doesn't have downvotes for stories but does for comments and employs hellbanning extensively I support the current system. Do you have any examples of good redditalikes without downvoting?

The buzzer would be annoying. Downvotes are just a signal people don't like what you wrote. Not equivalent.

Replies from: NancyLebovitz
comment by NancyLebovitz · 2014-02-14T17:26:34.290Z · LW(p) · GW(p)

I think I'd heard of hellbanning before, but not really registered it.

It's tempting, but in a way unfair because the hellbanned person is still posting, and not realizing they're being blanked out for everyone else until (perhaps) they leave because of the lack of response.

I don't know whether it would be better or worse to let hellbanned posters see and respond to each other's posts and comments. It's still sneaky. It would use up some system resources, but probably not very much, and it would be a public service to keep them occupied with each other.

comment by Dva · 2014-02-17T00:07:12.943Z · LW(p) · GW(p)

dude cool response