Duncan Sabien on Moderating LessWrong

Meta-meta point: the fact that you used your own post as an example, rather than a post where you weren't involved, made a very large difference in how I engaged with this Moderating LessWrong article. It prevented me from finishing it. Specifically, when I got to the point where you quoted Benquo, I felt honor-bound to stop reading and go check the source material, which turned out to be voluminous and ambiguous enough that I ran out of time before I came back. This is because, long ago, I made it a bright-line rule to never read someone quoting someone they disagree with in a potentially-hostile manner without checking the original context. That wouldn'tve felt necessary if the examples were drawn from discussions where you weren't involved.

(Meta, in case it's relevant to anyone: it's felt to me like LW has been deluged in the last few weeks by people saying things that seem very clearly wrong to me (this is not at all specific to this post / discussion), but in such a way that it would take a lot of effort on my part to explain clearly what seems wrong to me in most of the cases. I'm not willing to spend the time necessary to address every such thing or even most of them, but I also don't have a clear sense of how to prioritize, so I approximately haven't been commenting at all as a result because it just feels exhausting. I'm making an exception for this post more or less on a whim.)

There's a lot I like about this post, and I agree that a lot of benquo's comments were problematic. That said, when I put on my circling hat, it seems pretty clear to me that benquo was speaking from a place of being triggered, and I would have confidently predicted that the conversation would not have improved unless this was addressed in some way. I have some sense of how I would address this in person but less of a sense of how to reasonably address it on LW.

There's something in Duncan's proposed norms in the direction of "be responsible for your own triggered-ness." And there's something I like about that in principle, but also I think practice almost nobody on LW can do this reliably, including Duncan, and I would want norms that fail more gracefully in the presence of multiple people being triggered and not handling it ideally. At the very least, I want this concept of being triggered to be in common knowledge (I think it's not even in mutual knowledge at the moment) so we can talk about it when it's relevant, and ideally I'd want norms that make it okay to say things like "hey, based on X, Y, and Z I suspect you're currently a little triggered, do you want to slow down this conversation in A, B, or C way?" without this being taken as, like, a horrendous overreaching accusation.

There's a set of moderation-challenges that the post doesn't delve into, which are the ones I struggle most with – I don't have a clear model of what it'd mean to solve these, whereas the challenges pointed to in the OP seem comprehensible, just hard. I'm interested in thoughts on this.

1. Difficulty with moderating just-over-the-line comments, or non-legibly-over-the-line comments

The most common pattern I run into, where I'm not sure what to do, is patterns of comments from a given user that are either just barely over the line, or where each given comment is under the line, but so close to a line that repetition of it adds up to serious damage – making LW either not fun, or not safe feeling.

The two underlying generators I'm pointing at here seem to be:

• Not actually approaching a discussion collaboratively. Sometimes this is legit bad faith, and sometimes it's just... not enough willingness to do interpretive labor, or a vague underlying sense that you're not trying to be on the same team.
• Not being up-to-speed enough to contribute to a discussion. This feels harshest to take action on, even when I'm putting on my maximally friendly hat. Sometimes someone shows up and clearly hasn't read the sequences. Sometimes they have read all the relevant background info but just don't seem to be getting what the conversation is about, or are derailing it into a more basic version of itself (when it was aiming to be a higher level conversation)

These involve the most misunderstandings.

2. Negative Space

Sometimes, it's not what a commenter has said – it's what they haven't said. They're making some points that seem reasonable, but they're ignoring other points.

Sometimes this is because they're actually ignoring points that are uncomfortable, or they don't have access to a mental-skill that allows them to notice or process something.

Other times, they respond to a bunch of comments with another full blown essay that addresses everything at once (but, take weeks/months/years to get around to it).

This article gave me a bunch of food for thought. I don't think it addresses my main cruxes re: previous disagreements I've had with Duncan, but it definitely gave me some new ideas and new vantage points to view old ones.

(Note 1: I won't be commenting on Duncan's comments on Benquo's comments because I'm still in the process with chatting with Benquo about it. I have a number of relevant disagreements with both Ben and Duncan, hope to resolve those disagreements at some point, but meanwhile don't have the bandwidth I'd require to engage with both of them at once)

Some thoughts so far:

1. Hierarchy of Goals

The hierarchy of "purposes of LessWrong" that Duncan describes is roughly the same one I'd describe. A concern or difference in framing I have here is that several the of the stages reinforce each other in a cyclical fashion.

I'm not quite sure you can cleanly prioritize truth over truthseeking culture.

If our culture isn't outputting useful accumulation of knowledge, then it's failing at our core mission. Definitely. But in the situations where truthseeking-culture vs truth seem to be in conflict, I think it's often because there's a hard problem that defies easy answers. (see "Decoupled vs Contextual [LW · GW]" and "Tensions in Truthseeking [LW · GW]" for rough examples).

2. "Every Comment Gets Read, and acted on in some fashion"

I agree that this is an important goal worth striving for. I don't think it's achievable for the immediate future due to resource constraints, but useful to have in mind as the bar to measure against.

I think we've actually recently made a lot of progress on 80/20ing this – the new moderator sidebar forces someone to engage with all new posts, and all "at-risk" comments – those created by new users, or which have been downvoted or reported.

I liked Duncan's proposed UI of "all un-read-by-moderators comments appear highlighted to mods". I think it might be worth implementing something similar so that we gain a visceral sense of how close or far we currently are to "100% comment-check-coverage."

3. Limited Resources, tool building

We do have pretty limited resources, and a fairly high bar for who we trust as a moderator (esp. at the level that Duncan is describing here).This is exacerbated because half the mods also being developers, so there is a direct tradeoff between building tools, and engaging in high bandwidth communication (as well as putting organizational capacity into finding more moderators we trust, vs, say, building out open source documentation).

So, for the immediate future I do lean towards solutions like "build tools that help make moderation as easy as possible", rather than "make sure to fully engage with every comment that doesn't uphold our standards." (for example, right now there isn't actually a tool that with one click, locks an entire subthread – I can block replies to an individual comment, I can lock all comments on a post, but I can't lock all replies to all children of a comment, and this crimps the ability to put-things-in-stasis until a time when I have enough energy to moderate thoroughly.

3. Collegial Culture

I do think all things being equal, high-bandwidth communication is ideal. I like the general thrust of the approach described in the initial example:

Notice the details in the example above—they’re not random; most of them were put there deliberately and are doing important work. The phrase “appears to me to be” serves to highlight the critic’s awareness of uncertainty, that they may have misinterpreted things or missed detail. The framing of “thing we don’t do around here” is boundary-enforcing but not morally charged—it’s not that the behavior is fundamentally bad or wrong, just that it’s not part of our specific subcultural palette.

The phrase “if I’m understanding you correctly” foreshadows a crux [LW · GW]—if I’m understanding you, then I believe X, but if my understanding changes, I might not believe X anymore. The invocation of a “standard rationality move” (such as applying reductionism, checking the inverse of the hypothesis, or setting a five-minute timer) reinforces the shared culture of the site and models better behavior for newcomers. And the “thoughts?” at the end—especially if it occurs within a context where such invitations are demonstrably genuine, and not just lip service—actively draws the other person back into the conversation. The sum total of all of these little touches turns what might otherwise be the beginning of a fight into a cooperative, collaborative dynamic.

This basically is how I'd prefer myself to engage with most comments that don't live up to a standard, and insofar as I don't communicate that way it's (usually) because I'm either under time-pressure/stress/triggered, or perhaps more generally I haven't processed the overall strategy into a S1-response that flows smoothly in high stakes situations.

I think Vaniver does this sort of thing better than me. It was helpful to me to have some of the details of this spelled out so that I could pay more attention to them.

The main potential disagreement I have here is the scale of intervention within a single conversation that Duncan is suggesting. I can definitely imagine this turning out to be important, but I can also easily imagine it turning out to derailing most conversations into meta commentary on themselves.

4. Operationalization...

The part where I periodically disagree with Duncan is when it comes to the nuts and bolts of what sort of culture we're actually trying to build. I have a lot to say on this, but at least some of it is stuff that I'd like to chat with Duncan about in a separate format from this, because I think the internet is uniquely bad for hashing out the details.

To me it seems there is a certain tension between In Defense of Punch Bug and this post.

As I understand it, while "In defense of Punch Bang" in some parts argues people should not spend huge amount of attention on basically random noise, this calls for very high attention to detail on the part of moderators, like

"be attentive enough to be the one to catch the slipped-in hidden assumption in the middle of the giant paragraph, and to point out the uncharitable summary even when it’s carefully and politely phrased, and to follow the subthread all the way down to its twentieth reply".

This sounds surprisingly similar as a call why people should be diligently watching for microaggressions: so I would just point to the In Defense of Punch Bang for a reasonable counterargument.

Hm. I am in favor of high standards of discourse but I am remarkably resistant to Duncan imposing his high standards of discourse because he has remarkably different social/discourse intuitions from me.

I think it's helpful to have some sort of system to make sure that every comment gets read, but I think the ownership checkbox is potentially a bad way to do it. I'm mostly thinking of the incentives for moderators, here; it seems highly plausible that someone comes across a comment that feels off but they don't really know how to handle it; this means they don't want to say "yep, this one's mine" (because they don't want to handle it) but also not checking it is wrong somehow.

One of the things that I had considered when proposing the Sunshine Regiment was a 'report' button on every comment, available to all users, which was basically a "something about this should be handled by someone with more time and energy"--not necessarily "this post should get removed" but "oh man, I really want to see how Vaniver would respond to this comment," or something.

I also suspect there's something like Stack Exchange's edit queue that could be good, where several of the important pieces are 1) multiple eyes on any particular thing and 2) tracking how much people have eyes on things and 3) tracking when people's judgments disagree.

I think one of the main things I updated on is that a model I had of user discussions with users (that it's important to hug the most important part of the query, i.e. chasing and focusing on cruxes) is not applicable to mod actions (where it's important to catch each of the norm violations, instead of simply the most serious violation).

There are a handful of reasons for this, the most serious of which is the "that which is not punished is allowed"--Raemon made a comment on the Benquo's top comment that pointed out what Raemon saw as a defect in Benquo's presentation, and separated that from [LW(p) · GW(p)] a criticism of Benquo's argument. But that meant that later, when I criticized the argument, Benquo responded with [LW(p) · GW(p)] 'but Raemon wasn't criticizing my argument.' If I only point out what seems to me like the most serious error in a post, then there's a (reasonable!) argument that all the things that went unsaid weren't at the threshold of correction, and if I only point out the error that's easier to respond to, the same sort of thing goes through.

I really like when people put effort into providing alternatives-to/critiques-of how I work, so thanks for this.

It doesn’t mean I can always reply promptly, alas. Right now I’m in the process of taking all my finals (so I can get a visa and move to the Bay), and this will continue for a few weeks :(. Oli’s just coming back from the same, while Ray’s holding the fort.

It’s also regularly the case that doing detailed public write-ups of considerations around a decision/approach isn’t the right use of effort relative to just *building the product*, and that applies to things like detailed comments on this too. So far with LW I and Oli have spent much less time writing about where we’re going than just going there, which I think has worked well. (Ray has actually done a lot of writing of his thinking, somehow. Well done Ray.)

Then again, write-ups are a big part of building moderation, so I’ll see in a few weeks when I come to this.

Moved to meta.

This seems like it's pointing at a good thing. As a data point, the proposed responses to my comments would all have seemed friendly and helpful to me, and I'd have had an easy time engaging with the criticism.

His draft response to my VW comment would probably have motivated me to add a note to my initial comment deprecating that reference, or edit it out entirely (with a note clarifying that an edit had been made).

The asymmetry comment he pointed to as helpful (thanks, Vaniver!) actually would have been helpful, if SilentCal hadn't already taken the initiative (thanks, SilentCal!) and clarified what he took the term to mean.

I'm concerned that the described examples of holding individual comments to high epistemic standards don't seem to necessarily apply to top-level posts, or linked content- one reason I think this is bad is that it is hard to precisely critique something which is not in itself precise, or which contains metaphor, or which contains example-but-actually-pointing-at-a-class writing where the class can be construed in various different ways.

Critique of fuzzy intuitions and impressions and feelings often involves fuzzy intuitions and impressions and feelings, I think- and if this stuff is restricted in critique but not in top level content it makes top level content involving fuzzy intuitions and impressions and feelings hard to critique, despite I think being exactly the content which needs critiquing the most.

Strong comment standards seem like they would be good for a space (no strong opinion on whether LW should be that space), but it would probably want to also have high standards in top level posts, possibly review and feedback prior to publication, to keep them up to the same epistemic standards. Otherwise I think moderation argument over which interpretations of vague content were reasonable would dominate.

Additionally, strong disagree on "weaken the stigma around defensiveness" as an objective of moderation. One should post arguments because one believes they are valid, and clarify misunderstandings because they are wrong, not argue or post or moderate to try to save personal status. It may be desirable to post and act with the objective of making it easier to not be defensive, but we still want people in themselves to try to avoid taking it as a referendum on their person. In terms of fairness, I'm not sure how you'd judge it- it is valid for the part people have most concerns about to not be the part which is desired to be given the most attention, I think, in even formal peer review. It's also valid for most people to disagree with and have critiques of a piece of content. The top level post author (or the link post's author) doesn't have a right to "win"- it is permissible for the community to just not think a post's object level content is all that good. If there was to be a fairness standard that justified anything, it'd certainly want to be spelled out in more detail and checked by someone other than the person feeling they were treated unfairly.

There is a potential discussion to be had someday about whether upvotes and downvotes themselves can be considered to be "in error" given the specific milieu of LW, or whether upvotes and downvotes are sacred à la free speech. Looking at votes on this and other threads, I have frequently had the sense that people were Doing It (Objectively) Wrong, not fundamentally as in wrong-opinions, but culturally as in participation-in-this-culture-means-precommitting-to-supporting-X-and-othering-Y.

I'm aware that there is a strongly-felt libertarian argument against this (a sort of red-versus-white disagreement, in MTG terms) and it'd be interesting to see whether LW wants to have a standard in common knowledge around this question.

In short, what makes LessWrong different from everywhere else on the internet is that it’s a place where truth comes first,

Because all those science, logic, maths and philosophy forums are just full of lies.