Automatic Rate Limiting on LessWrong

post by Raemon · 2023-06-23T20:19:41.049Z · LW · GW · 34 comments

Contents

  Current AutoRateLimits (as of June 22)
    Default Rate Limits
    Negative Karma Rate Limits
    Recent Karma Rate Limits
      Post Rate Limits
      Comment Rate Limits
  Q&A
    I got downvoted. How do I write stuff that won't be downvoted?
    I think I'm just getting downvoted because I disagree with LessWrong consensus. How is that fair? Don't you want to avoid becoming closed-minded?
None
36 comments

The LessWrong team recently began rolling out automatic rate limits. The general idea is if a user gets heavily downvoted, the site automatically restricts their posting and commenting privileges. (See Well-Kept Gardens Die By Pacifism [LW · GW] for some background on LessWrong's overall moderation philosophy)

After thinking awhile about various side-effects (see previous discussion [LW · GW]), I designed some rate limits based on the following user metadata:

• Total karma. i.e. all net upvotes/downvotes the user has ever received
• Recent karma: The net karma the user received from their last 20 comments and/or posts.
• Last month's karma: The karma the user received for their last 20 comments/posts within the last month (i.e. a subset of the "recent karma." determined by fetching all votes on the last 20 comments/posts, and then filtering for votes on posts/comments that were published in the last 30 days)
• Unique downvoter count: The number of individuals who've downvoted the user's recent content. This ensures that a single angry individual or small clique can't wreck your day.

The exact karma thresholds and rate-limit-strengths chosen are based on the LW team's experience with daily moderation maintenance. Every day, moderators look over ~20 posts and comments from new-ish users. We've built up some intuitions for what user-karma and comment/post-karma tends to correspond with stuff we want to see more of on LessWrong. And for the past month or so I've been checking how these four metrics apply to the current distribution of LW users.

The karma system isn't perfect. Sometimes you'll get randomly downvoted by someone for idiosyncratic or petty reasons. Sometimes it's hard for newer user's content to get noticed and upvoted. But people tend to be pretty hesitant to downvote stuff  – most comments just don't get voted on. Downvoted content from new users tends to be some combination of poorly argued, difficult to read, rude-without-actually-saying-much, or rehashing topics that have been discussed extensively on LessWrong without introducing new considerations. 

I suspect we'll iterate and fine-tune these rules over time. 

Later in this post I have advice on writing posts/comments that tend to be well received on LessWrong. If you've been downvoted and/or rate-limited, don't take it too hard. LessWrong has fairly particular standards. My recommendation is to read some of the advice at the end here and try again. 

Current AutoRateLimits (as of June 22)

We'll probably experiment with these a bit.  The exact implementation here involves a lot of different numbers, but here's a rough overview of the current rate limit philosophy:

• Default rate limits for users with less than 5 karma (aimed at new users).
• Moderately strong (but non-escalating) rate limits for users with negative karma.
• Escalating rate limits for users with negative karma and "negative recent karma".

Stronger rate limits require more unique downvoters, so that a single angry voter or small clique can't have too large an impact.

The hope is for users to mostly show up, get recognized if they write good content, and then get full posting permissions. If a user is getting net-downvoted, then when they go to post they'd they get a message looking something like:

Default Rate Limits

Users whose total karma is < 5 are limited to 3 comments a day, and 2 posts per week. 

The hope here is that these numbers are: a) high enough that most users can get started commenting/posting without having to deal with a rate limit, and b) low enough that a user writing low-quality content can't go on too big a commenting spree, before having to slow down a bit and learn some site norms. 

The default commenting rate limits apply to writing comments on your own posts. 

Negative Karma Rate Limits

If a user has -1 or less total karma, they can only write one comment per day and one post per two-weeks. 

This doesn't require multiple downvoters, since new users with negative karma are more likely to be spam, or fairly confused about LessWrong. I do think this will occasionally be unfair to new users who were posting in good faith (and I am sorry about that!). 

My intention here is to treat getting a negative-karma rate limit to be like getting caught in a spam filter – it happens sometimes unfortunately, but the consequences aren't too bad. You'll be able to try again the next day.

Negative karma rate limits also apply to writing comments on your own posts.

Recent Karma Rate Limits

We have an escalating series of rate limits based on recent karma. (Here "recent karma" means "karma from your last 20 comments/and/or/posts", and "last month karma" is the subset of recent karma on comments/posts from the past month)

[Edit: we've updated the escalating rate limits to only apply to users with negative karma. We basically only want them to apply in times when a moderator would have ordinarily wanted to give the user a temporary ban, and this seems important to avoid false positives about. We don't currently have a good enough downvote metric to avoid false positives. So instead, we're now using the escalating rate limits basically as a way to phase out users who the site is clearly "collectively judging" as net negative, while still giving them the ability to try again in a month.

This explanation kinda sucks, I will hopefully work on a clearer explanation]

Post Rate Limits

Note: these only apply to publishing posts. You can still create and work on drafts.

• -15 recent karma (4+ downvoters) 
  • 1 post per week
• -30 recent karma (5+ downvoters, -1 total karma)
  • 1 post per two weeks
• -45 recent karma (5+ downvoters, -1 total karma)
  • 1 post per three weeks
• -60 recent karma (5+ downvoters, -1 total karma)
  • 1 post per four weeks

Comment Rate Limits

Commenting rate limits felt a bit trickier than posts. 

People typically don't publish posts very often, and the posts either get upvoted or downvoted and people move on with their day. Comments are part of a conversation. Sometimes, it's really important for people to be able to write a ton of comments back-and-forth without anyone bothering to upvote each other. Sometimes, posting a ton of comments is annoying and spammy. 

These rate limits attempt to navigate that balance.

Most of these do not affect commenting on your own posts.

• 1 or less recent karma (no minimum downvoters)
  • 3 comments per day
  • i.e. if your past 20 comments aren't at least slightly net-upvoted, you go back to the default rate limit for new users. 
  • This particular rate-limit only applies to users with less than 2000+ karma. (When established users get into this situation, I think it's more likely that they're having a nerdsnipy back-and-forth about an obscure topic, whereas for newer users it's more likely that they're commenting a lot and being slightly annoying, but not quite annoying enough to get downvoted)
• -1 recent karma (3+ downvoters)
  • 1 comment / hour
  • i.e slow down a little, you may be getting into a heated exchange.
  • this stacks with the "3 comments per day" rate limit for users with 1-or-less recent karma. Unlike the previous rate limit, it applies to all users, even if they have 2000+ karma.
  • Applies to commenting on your own posts. (The main use case of this one is slowing and taking a breather when you're getting into a heated argument, and this seems potentially important even if that argument is on your own post) 
• -5 recent karma (4+ downvoters)
  • 1 comment per day
• -15 recent karma (4+ downvoters, 500 total karma or less)
  • 1 comment per 3 days
• -30 last month karma (5+ downvoters, -1 total karma or less)
  • 1 comment per week
  • Unlike the other comment rate limits, this one only uses karma from the last month. This is because if you got into a deep negative karma-hole, it'd become pretty punishingly unfair to dig yourself out (since you'd only get one comment per week). Instead, you most likely end up heavily rate limited for a month, and if you don't get additional downvotes during that month you can try again with a lighter rate limit.

You can read the specific implementation of the rate limits in this file.

Q&A

I got downvoted. How do I write stuff that won't be downvoted?

My recommendations here are pretty similar to Ruby's recommendations in the New User's Guide to LessWrong [LW · GW]. 

What gets upvoted and downvoted depends on a lot of factors. A post/comment might be really strong in some areas and weak in other areas. There are maybe two threads of advice I'd want to give:

• How to write content that actively provides value.
• How to avoid falling into particular pitfalls that people are especially likely to downvote.

The first one is in some sense simple: write content that is novel, correct and useful. There are a lot of paths towards this. It's a pretty complex problem though, beyond the scope of this post. Here's are two essays by Paul Graham, here's Sarah Constantin on writing Fact Posts [LW · GW], and Nonfiction Writing Advice by Scott Alexander. 

The rest of this will focus on "avoiding pitfalls". 

Focus on exploring "what predictions are you making and why?", as opposed to trying to win a fight. If you're coming from the Rest Of The Internet, you may be surprised by how far LessWrong takes this. There's a genre of comments that seem to be out to make someone look dumb or ridiculous. I recommend a focus on criticizing ideas, on the object-level, rather than trying to raise or lower someone's status.

I recommend reading the LessWrong Political Prerequisites [? · GW] sequence to get more context.

Write a clear introduction and make it sure it's nicely formatted (with appropriate paragraph breaks). If you're writing a post or pretty long comment, make sure it's easy to read and understand the point you're making. Err on the side of writing something shorter rather than longer if you can.

Be familiar with rationality basics like "beliefs are probabilistic [LW · GW] and should control anticipations [LW · GW]." Get curious about where you might be wrong [? · GW], avoid arguing over definitions [? · GW], etc. Read through the Sequences Highlights [? · GW]. 

Be easy to engage/argue with. If you disagree with a post author, try to state your reasoning and what would change your mind. Make concrete predictions. 

Aim for a high standard if you're contributing on the topic AI. As AI becomes higher and higher profile in the world, many more people are flowing to LessWrong because we have discussion of it. In order to not lose what makes our site uniquely capable of making good intellectual progress, we have particularly high standards for new users showing up to talk about AI. If you find yourself getting downvoted, I recommend shifting your stance towards "ask questions in the latest AI All Questions Open Thread [? · GW]".

I think I'm just getting downvoted because I disagree with LessWrong consensus. How is that fair? Don't you want to avoid becoming closed-minded?

If this was happening a lot, I'd be quite worried (not just about autoRateLimits but about LessWrong as a whole). 

I do think this happens nonzero. But I don't think it's usually quite what's going on.

The LessWrong community does have some biases. But the way I think those biases tend to play out depends on some specifics. I think comments tend to fall into this grid:

Low-quality consensus posts/comments (usually somewhat upvoted, or  heavily upvoted when they're funny  or particularly emotionally resonant)	High-quality consensus posts/comments (Usually pretty upvoted)
Low-quality contrarian posts/comments (usually somewhat downvoted, or  heavily downvoted if they're rude)	High-quality contrarian posts/comments (Usually heavily upvoted)

(There are, of course, some medium-quality contrarian posts that tend to get somewhat upvoted.) 

I think a lot of people on LessWrong are actively into well-written critiques of "the LessWrong consensus." (For a long time Holden's Thoughts on the Singularity Institute (SI) was the most upvoted post, and Paul's Where I agree and disagree with Eliezer is sort of a spiritual successor).

My actual sense, from looking at upvote/downvote patterns, is that people complaining about getting downvoted are in fact tending to write posts that are some combination of not well-argued, or retreading ground that's been discussed a bunch on LessWrong without adding new arguments (often failing to engage with the standard rebuttals and positions). 

I think these are basically correct things to get downvoted and deprioritized.

The question of "is it fair/reasonable for low-quality consensus comments to get upvoted?" is a fair one. I basically trust the LessWrong karma system to distinguish "stuff there should be less of" and "stuff that's at least plausibly good". I think reasonable people can disagree on whether LessWrong has good taste in what gets significantly upvoted. 

---

Hope that helps give some context on rate limits and how they fit into LessWrong. As noted, we'll likely iterate on this over time. 

I think it's also worth reading the New User's Guide to LessWrong [LW · GW], and the Sequences Highlights to get a good sense of what LessWrong is about, and what background knowledge it's built on.

34 comments

Comments sorted by top scores.