Succinctly explain the main point.

When we're processing many new user-posts a day, we don't have much time to evaluate each post. 

So, one principle I think is fairly likely to become a "new user guideline" is "Make it pretty clear off the bat what the point of the post is." In ~3 sentences, try to make it clear who your target audience is, and what core point you're trying to communicate to them. If you're able to quickly gesture at the biggest-bit-of-evidence or argument that motivates your point, even better. (Though I understand sometimes this is hard).

This isn't necessarily how you have to write all the time on LessWrong! But your first post is something like an admissions-essay and should be optimized more for being legibly coherent and useful. (And honestly I think most LW posts should lean more in this direction)

In some sense this is similar to submitting something to a journal or magazine. Editors get tons of submissions. For your first couple posts, don't aim to write something that takes a lot of works for us to evaluate.

Corollary: Posts that are more likely to end up in the reject pile include...

• Fiction, especially if it looks like it's trying to make some kind of philosophical point, while being wrapped in a structure that makes that harder to evaluate. (I think good fiction plays a valuable role on LessWrong, I just don't recommend it until you've gotten more of a handle of the culture and background knowledge)
• Long manifestos. Like fiction, these are sometimes valuable. They can communicate something like an overarching way-of-seeing-the-world that is valuable in a different way from individual factual claims. But, a) I think it's a reasonable system to first make some more succinct posts, build up some credibility, and then ask the LessWrongOSphere to evaluate your lengthy treatise. b) honestly... your first manifesto just probably isn't very good. That's okay. No judgment. I've written manifestos that weren't very good and they were an important part of my learning process. Even my more recent manifestos tend to be less well received than my posts that argue a particular object-level claim.

Some users have asked: "Okay, but, when will I be allowed to post the long poetic prose that expresses the nuances of the idea I have in my heart?"

Often the answer is, well, when you get better at thinking and expressing yourself clearly enough that you've written a significantly different piece. 

Some top-level post topics that get much higher scrutiny:

1. Takes on AI

Simply because of the volume of it, the standards are higher. I recommend reading Scott Alexander's Superintelligence FAQ [LW · GW] as a good primer to make sure you understand the basics. Make sure you're familiar with the Orthogonality Thesis and Instrumental Convergence. I recommend both Eliezer's AGI Ruin: A List of Lethalities [LW · GW]  and Paul Christiano's response post [LW · GW] so you understand what sort of difficulties the field is actually facing.

I suggest going to the most recent AI Open Questions [? · GW] thread, or looking into the FAQ at https://ui.stampy.ai/ 

2. Quantum Suicide/Immortality, Roko's Basilisk and Acausal Extortion. 

In theory, these are topics that have room for novel questions and contributions. In practice, they seem to attract people who seem... looking for something to be anxious about? I don't have great advice for these people, but my impression is that they're almost always trapped in a loop where they're trying to think about it in enough detail that they don't have to be anxious anymore, but that doesn't work. They just keep finding new subthreads to be anxious about.

For Acausal Trade, I do think Critch's Acausal normalcy [LW · GW] might be a useful perspective that points your thoughts in a more useful direction. Alas, I don't have a great primer that succinctly explains why quantum immortality isn't a great frame, in a way that doesn't have a ton of philosophical dependencies. 

I mostly recommend... going outside, hanging out with friends and finding other more productive things to get intellectually engrossed in.

3. Needing help with depression, akrasia, or medical advice with confusing mystery illness.

This is pretty sad and I feel quite bad saying it – on one hand, I do think LessWrong has some useful stuff to offer here. But too much focus on this has previously warped the community in weird ways – people with all kinds of problems come trying to get help and we just don't have the resources to help all of them.

For your first post on LessWrong, think of it more like you're applying to a university. Yes, universities have mental health departments for students and faculty... but when we're evaluating "does it make sense to let this person into this university", the focus should be on "does this person have the ability to make useful intellectual contributions?" not "do they need help in a way we can help with?"

A key question when I look at a new user on LessWrong trying to help with AI is, well, are they actually likely to be able to contribute to the field of AI safety? 

If they are aiming to make direct novel intellectual contributions, this is in fact fairly hard. People have argued back and forth about how much raw IQ, conscientiousness or other signs of promise a person needs to have. There has been some posts arguing that people are overly pessimistic and gatekeeping-y about AI safety.

But, I think it's just pretty importantly true that it takes a fairly significant combination of intelligence and dedication to contribute. Not everyone is cut out for doing original research. Many people pre-emptively focus on community building and governance because that feels easier and more tractable to them than original research. But those areas still require you to have a pretty understanding of the field you're trying to govern or build a community for.

If someone writes a post on AI that seems like a bad take, which isn't really informed by the real challenges, should I be encouraging that person to make improvements and try again? Or just say "idk man, not everyone is cut out for this?"

Here's my current answer.

If you've written a take on AI that didn't seem to hit the LW team's quality bar, I would recommend some combination of:

• Read ~16 hours of background content, so you're not just completely missing the point. (I have some material in mind that I'll compile later, but for now highlight roughly the amount of effort involved)
• Set aside ~4 hours to think seriously about the topic.  Try to find one sub-question you don't know the answer to, and make progress answering that sub-question. 
• Write up your thoughts as a LW post.

(For each of these steps, organizing some friends to work together as a reading or thinking group can be helpful to make it more fun)

This doesn't guarantee that you'll be a good fit for AI safety work, but I think this is an amount of effort where it's possible for a LW mod to look at your work, and figure out if this is likely to be a good use of your time.

Some people may object "this is a lot of work." Yes, it is. If you're the right sort of person you may just find this work fun. But the bottom line is yes, this is work. You should not expect to contribute to the field without putting in serious work, and I'm basically happy to filter out of LessWrong people who a) seem to superficially have pretty confused takes, and b) are unwilling to put in 20 hours of research work.

Draft in progress. Common failures modes for AI posts that I want to reference later:

Trying to help with AI Alignment

"Let's make the AI not do anything." 

This is essentially a very expensive rock. Other people will be building AIs that do do stuff. How does your AI help the situation over not building anything at all?

"Let's make the AI do [some specific thing that seems maybe helpful when parsed as an english sentence], without actually describing how to make sure they do exactly or even approximately that english sentence"

The problem is a) we don't know how to point an AI at doing anything at all, and b) your simple english sentence includes a ton of hidden assumptions.

(Note: I think Mark Xu sort of disagreed with Oli on something related to this recently, so I don't know that I consider this class of solution is completely settled. I think Mark Xu thinks that we don't currently know how to get an AI to do moderately complicated actions with our current tech, but, our current paradigms for how to train AIs are likely to yield AIs that can do moderately complicated actions)

I think the typical new user who says things like this still isn't advancing the current paradigm though, nor saying anything useful that hasn't already been said.

Arguing Alignment is Doomed

[less well formulated]

Lately there's been a crop of posts arguing alignment is doomed. I... don't even strongly disagree with them, but they tend to be poorly argued and seem confused about what good problem solving looks like. 

Arguing AI Risk is a dumb concern that doesn't make sense

Lately (in particular since Eliezer's TIME article), we've had a bunch of people coming in to say we're a bunch of doomsday cultists and/or gish gallopers.

And, well, I think if you're just tuning into the TIME article, or you have only been paying bits of attention over the years, I think this is a kinda reasonable belief-state to have. From the outside when you hear an extreme-sounding claim, it's reasonable for alarm bells to go off and assume this is maybe crazy.

If you were the first person bringing up this concern, I'd be interested in your take. But, we've had a ton of these, so you're not saying anything new by bringing it up. 

You're welcome to post your take somewhere else, but if you want to participate on LessWrong, you need to engage with the object level arguments.

Here's a couple things I'll say about this:

• One particularly gishgallopy-feeling thing is that many arguments for AI catastrophe are disjunctive. So, yeah, there's not just one argument you can overturn and then we'll all be like "okay great, we can change our mind about this problem." BUT, it is the case that we are pretty curious about individual arguments getting overturned. If individual disjunctive arguments turned out to be flawed, that'd make the problem easier. So I'd be fairly excited about someone who digs into the details of various claims in AGI Ruin: A List of Lethalities [LW · GW] and either disproves them or finds a way around them.
• Another potentially gishgallopy-feeling thing is If you're discussing things on LessWrong, you'll be expected to have absorbed the concepts from the sequences (such as how to think about subjective probability, tribalism, etc), either by reading the sequences or lurking a lot. I acknowledge this is as pretty gishgallopy at first glance, if you came here to debate one particular thing. Alas, that's just how it is [see other FAQ question delving more into this]

Here's a quickly written draft for an FAQ we might send users whose content gets blocked from appearing on the site.

The “My post/comment was rejected” FAQ

Why was my submission rejected?

Common reasons that the LW Mod team will reject your post or comment:

• It fails to acknowledge or build upon standard responses and objections that are well-known on LessWrong. 
  • The LessWrong website is 14 years old and the community behind it older still. Our core readings [link] are over half a million words. So understandably, there’s a lot you might have missed!
  • Unfortunately, as the amount of interest in LessWrong grows, we can’t afford to let cutting-edge content get submerged under content from people who aren’t yet caught up to the rest of the site.   
• It is poorly reasoned. It contains some mix of bad arguments and obviously bad positions that it does not feel worth the LessWrong mod team or LessWrong community’s time or effort responding to.
• It is difficult to read. Not all posts and comments are equally well-written and make their points as clearly. While established users might get more charity, for new users, we require that we (and others) can easily follow what you’re saying well or enough to know whether or not you’re saying something interesting.
  • There are multiple ways to end up difficult to read:
    • Poorly crafted sentences and paragraphs
    • Long and rambly
    • Poorly structure without signposting
    • Poor formatting
• It’s a post and doesn’t say very much.
  • Each post takes up some space and requires effort to click on and read, and the content of the post needs to make that worthwhile. Your post might have had a reasonable thought, but it didn’t justify a top-level post.
    • If it’s a quick thought about AI, try an “AI No Dumb Questions Open Thread”
  • Sometimes this just means you need to put in more effort, though effort isn’t the core thing.
• You are rude or hostile. C’mon. 

Can I appeal or try again?

You are welcome to message us however note that due to volume, even though we read most messages, we will not necessarily respond, and if we do, we can’t engage in a lengthy back-and-forth.

In an ideal world, we’d have a lot more capacity to engage with each new contributor to discuss what was/wasn’t good about their content, unfortunately with new submissions increasing on short timescales, we can’t afford that and have to be pretty strict in order to ensure site quality stays high.

This is censorship, etc. 

LessWrong, while on the public internet, is not the general public. It was built to host a certain kind of discussion between certain kinds of users who’ve agreed to certain basics of discourse, who are on board with a shared philosophy, and can assume certain background knowledge.

Sometimes we say that LessWrong is a little bit like a “university”, and one way that it is true is that not anybody is entitled to walk in and demand that people host the conversation they like.

We hope to keep allowing for new accounts created and for new users to submit new content, but the only way we can do that is if we reject content and users who would degrade the site’s standards.

But diversity of opinion is important, echo chamber, etc.

That’s a good point and a risk that we face when moderating. The LessWrong moderation team tries hard to not reject things just because we disagree, and instead only do so if it feels like the content it failing on some other criteria.

Notably, one of the best ways to disagree (and that will likely get you upvotes) is to criticize not just a commonly-held position on LessWrong, but the reasons why it is held. If you show that you understand why people believe what they do, they’re much more likely to be interested in your criticisms. 

How is the mod team kept accountable?

If your post or comment was rejected from the main site, it will be viewable alongside some indication for why it was banned. [we haven't built this yet but plan to soon]

Anyone who wants to audit our decision-making and moderation policies can review blocked content there.

↑ comment by Ben Pace (Benito) · 2023-04-06T20:33:14.294Z · LW(p) · GW(p)

It fails to acknowledge or build upon standard responses and objections that are well-known on LessWrong. 

• The LessWrong website is 14 years old and the community behind it older still. Our core readings [link] are over half a million words. So understandably, there’s a lot you might have missed!
• Unfortunately, as the amount of interest in LessWrong grows, we can’t afford to let cutting-edge content get submerged under content from people who aren’t yet caught up to the rest of the site.

I do want to emphasize a subtle distinction between "you have brought up arguments that have already been brought up" and "you are challenging basic assumptions of the ideas here". I think challenging basic assumptions is good and well (like here [LW · GW]), while bringing up "but general intelligences can't exist because of no-free-lunch theorems" or "how could a computer ever do any harm, we can just unplug it" is quite understandably met with "we've spent 100s or 1000s of hours discussing and rebutting that specific argument, please go read about it <here> and come back when you're confident you're not making the same arguments as the last dozen new users".

I would like to make sure new users are specifically not given the impression that LW mods aren't open to basic assumptions being challenged, and I think it might be worth the space to specifically rule out that interpretation [LW · GW] somehow.

Replies from: Ruby

↑ comment by Ruby · 2023-04-06T23:26:03.335Z · LW(p) · GW(p)

Can be made more explicit, but this is exactly why the section opens with "acknowledge [existing stuff on topic]".

Replies from: Benito

↑ comment by Ben Pace (Benito) · 2023-04-06T23:28:25.221Z · LW(p) · GW(p)

Well, I don't think you have to acknowledge existing stuff on this topic if you have a new and good argument. 

Added: I think the phrasing I'd prefer is "You made an argument that has already been addressed extensively on LessWrong" rather than "You have talked about a topic without reading everything we've already written about on that topic".

Replies from: Ruby, Ruby

↑ comment by Ruby · 2023-04-07T00:06:43.933Z · LW(p) · GW(p)

I do think there is an interesting question of "how much should people have read?" which is actually hard to answer.

There are people who don't need to read as much in order to say sensible and valuable things, and some people that no amount of reading seems to save. 

The half a million words is the Sequences. I don't obviously want a rule that says you need to have read all of them in order to post/comment (nor do I think doing so is a guarantee), but also I do want to say that if you make mistakes the the Sequences would teach you not to make, that could be grounds for not having your content approved.

A lot of the AI newbie stuff I'm disinclined to approve is the kind that makes claims that are actually countered in the Sequences, e.g. orthogonality thesis, treating the AI too much like humans, various fallacies involving words.
 

↑ comment by Ruby · 2023-04-06T23:36:45.543Z · LW(p) · GW(p)

How do you know you have a new and good argument if you don't know the standards thing said on the topic

And relatedly, why should I or other readers on LW assume that you have a new and good argument without any indication that you know the arguments in general?

This is aimed at users making their very first post/comment. I think it is likely a good policy/heuristic for the mod team in judging your post that claims "AIs won't be dangerous because X", tells me early on that you're not wasting my time because you're already aware of all the standard arguments. 

In a world where everyday a few dozen people who started thinking about AI two weeks ago show up on LessWrong and want to give their "why not just X?", I think it's reasonable to say "we want you to give some indication that you're aware of the basic discussion this site generally assumes".

Replies from: Making_Philosophy_Better

↑ comment by Portia (Making_Philosophy_Better) · 2023-04-07T01:53:11.400Z · LW(p) · GW(p)

I find it hilarious that you can say this, while simultaneously, the vast majority of this community is deeply upset they are being ignored by academia and companies, because they often have no formal degrees or peer reviewed publication, or other evidence of having considered the relevant science. Less wrong fails the standards of these fields and areas. You routinely re-invent concepts that already exist. Or propose solutions that would be immediately rejected as infeasible if you tried to get them into a journal.

Explaining a concept your community takes for granted to outsiders can help you refresh it, understand it better, and spot potential problems. A lot of things taken for granted here are rejected by outsiders because they are not objectively plausible.

And a significant number of newcomers, while lacking LW canon, will have other relevant knowledge. If you make the bar too high, you deter them.

All this is particularly troubling because your canon is spread all over the place, extremely lengthy, and individually usually incomplete or outdated, and filled in implicitly from prior forum interactions. Academic knowledge is more accessible that way.

Replies from: Ruby, lc, lahwran

↑ comment by Ruby · 2023-04-08T00:25:20.932Z · LW(p) · GW(p)

Writing hastily in the interests of time, sorry if not maximally clear.

Explaining a concept your community takes for granted to outsiders can help you refresh it, understand it better, and spot potential problems.

 

It's very much a matter of how many newcomers there are relative to existing members. If the number of existing members is large compared to newcomers, it's not so bad to take the time to explain things.

If the number of newcomers threatens to overwhelm the existing community, it's just not practical to let everyone in. Among other factors, certain conversation is possible because you can assume that most people have certain background and even if they disagree, at least know the things you know.

The need for getting stricter is because of the current (and forecasted) increase in new user. This means we can't afford to become 50% posts that ignore everything our community has already figured out. 

LessWrong is an internet forum, but it's in the direction of a university/academic publication, and such publications only work because editors don't accept everything. 

↑ comment by lc · 2023-04-07T05:09:36.553Z · LW(p) · GW(p)

the vast majority of this community is deeply upset they are being ignored by academia and companies, because they often have no formal degrees or peer reviewed publication, or other evidence of having considered the relevant science.

Source?

Replies from: lahwran

↑ comment by the gears to ascension (lahwran) · 2023-04-07T05:16:52.852Z · LW(p) · GW(p)

my guess is that that claim is slightly exaggerated, but I expect sources exist for an only mildly weaker claim. I certainly have been specifically mocked for my username in places that watch this site, for example.

Replies from: lc

↑ comment by lc · 2023-04-07T05:19:35.332Z · LW(p) · GW(p)

I certainly have been specifically mocked for my username in places that watch this site, for example.

This is an example of people mocking LW for something. Portia is making a claim about LW users' internal emotional states; she is asserting that they care deeply about academic recognition and feel infuriated they're not getting it. Does this describe you or the rest of the website, in your experience?

Replies from: lahwran

↑ comment by the gears to ascension (lahwran) · 2023-04-07T05:29:50.063Z · LW(p) · GW(p)

lens portia's writing out of frustrated tone first and it makes more sense. they're saying that recognition is something folks care about (yeah, I think so) and aren't getting to an appropriate degree (also seems true). like I said in my other comment - tone makes it harder to extract intended meaning.

Replies from: lc

↑ comment by lc · 2023-04-07T05:43:39.062Z · LW(p) · GW(p)

Well, I disagree. I have literally zero interest in currying the favor of academics, and think Portia is projecting a respect and yearning for status within universities onto the rest of us that mostly doesn't exist. I would additionally prefer if this community were able to set standards for its members without having to worry about or debate whether or not asking people to read canon is a status grab.

Replies from: lahwran

↑ comment by the gears to ascension (lahwran) · 2023-04-07T06:02:50.242Z · LW(p) · GW(p)

sure. I do think it's helpful to be academically valid sometimes though. you don't need to care, but some do some of the time somewhat. maybe not as much as the literal wording used here. catch ya later, anyhow.

↑ comment by the gears to ascension (lahwran) · 2023-04-07T05:13:08.159Z · LW(p) · GW(p)

strong agree, single upvote: harsh tone, but reasonable message. I hope the tone doesn't lead this point to be ignored, as I do think it's important. but leading with mocking does seem like it's probably why others have downvoted. downvote need not indicate refusal to consider, merely negative feedback to tone, but I worry about that, given the agree votes are also in the negative.

Replies from: Making_Philosophy_Better

↑ comment by Portia (Making_Philosophy_Better) · 2023-04-07T12:37:51.492Z · LW(p) · GW(p)

Thank you. And I apologise for the tone. I think the back of my mind was haunted by Shoggoth with a Smiley face giving me advice for my weekend plans, and that emotional turmoil came out the wrong way.

I am in the strange position of being on this forum, and in academia, and seeing both sides engage in the same barrier keeping behaviour, and call it out as elitist and misguided in the other but a necessary way to ensure quality and affirm your superior identity in your own group is jarring. I've found valuable and admirable practices and insights in both, else I would not be there.

Replies from: thoth-hermes

↑ comment by Thoth Hermes (thoth-hermes) · 2023-04-07T15:20:26.030Z · LW(p) · GW(p)

Any group that bears a credential, or performs negative selection of some kind, will bear the traits you speak of. 'Tis the nature of most task-performing groups human society produces. Alas, one cannot escape it, even coming to a group that once claimed to eschew credentialism. Nonetheless, it is still worthwhile to engage with these groups intellectually.

shift the default assumptions of LessWrong to "users by default have a rate limit of 1-comment-per day"

Natural times I expect this to be frustrating are when someone's written a post, got 20 comments, and tries to reply to 5 of them, but is locked after the first one. 1 per day seems too strong there. I might say "unlimited daily comments on your own posts".

I also think I'd prefer a cut-off where after which you're trusted to comment freely. Reading the positive-selection post (which I agree with), I think some bars here could include having written a curated post or a post with 200+ karma or having 1000 karma on your account.

Here are some principles that are informing some of my thinking here, some pushing in different directions

• Karma isn't that great a metric – I think people often vote for dumb reasons, and they vote highest in drama-threads that don't actually reflect important new intellectual principles. I think there are maybe ways we can improve on the karma system, and I want to consider those soon. But I still think karma-as-is is at least a pretty decent proxy metric to keep the site running smoothly and scaling.
• Because karma is only a proxy metric, I'd still expect moderator judgment to play a significant role in making sure the system isn't going off the rails in the immediate future
• each comment comes with a bit of an attentional cost. If you make a hundred comments and get 10 karma (and no downvotes), I think you're most likely not a net-positive contributor. (i.e. each comment maybe costs 1/5th of a karma in attention or something like that)
• in addition, I think highly upvoted comments/posts tend to be dramatically more valuable than weakly upvoted comments/posts. (i.e. a 50 karma comment is more than 10 times as valuable as a 5 karma comment, most of the time [with an exception IMO for drama threads]

The current karma system kinda encourages people to write lots of comments that get slightly upvoted and gives them the impression of being an established regular. I think in most cases users with a total average karma of ~1-2 are typically commenting in ways that are persistently annoying in some way, in a way that'd be sort of fine with each individual comment but adds up to some kind of "death by a thousand cuts" thing that makes the site worse.

On the other hand, lots of people drawn to LessWrong have a lot of anxiety and scrupulosity issues and I generally don't want people overthinking this and spending a lot of time worrying about it.

My hope is to frame the thing more around positive rewards than punishments.

I suggest not counting people's comments on their own posts towards the rate limit or the “barely upvoted” count. This both seems philosophically correct, and avoids penalizing authors of medium-karma posts for replying to questions (which often don’t get much if any karma).

risks with negative selection

There should be fast tracks that present no practical limits to the new users. First few comments should be available immediately upon registration, possibly regenerating quickly. This should only degrade if there is downvoting or no upvoting, and the limits should go away completely according to an algorithm that passes backtesting on first comments made by users in good standing who started commenting within the last 3-4 years. That is, if hypothetically such a rate-limiting algorithm were to be applied 3 years ago to a user who started commenting then, who later became a clearly good contributor, the algorithm should succeed in (almost) never preventing that user from making any of the comments that were actually made, at the rate they were actually made.

If backtesting shows that this isn't feasible, implementing this feature is very bad. Crowdsource moderation instead, allow high-Karma users to rate-limit-vote on new users, but put rate-limit-level of new users to "almost unlimited" by default, until rate-limit-downvoted manually.

I'm less optimistic than Ray about rate limits, but still think they're worth exploring. I think getting the limits/rules correct will be tricky since I do care about the normal flow of good conversation not getting impeded. 

I think it's something we'll try soon, but not sure if it'll be the priority for this week.

"users by default have a rate limit of 1-comment-per day" and "1 post per week."

Imagine a system that lets a user write their comments or posts in advance, and then publishes comments according to these limits automatically. Then these limits wouldn't be enough. On the other hand, if you want to write a comment, you want to write it right away, instead of only starting to write it the next day because you are out of commenting chits. It's very annoying if the UI doesn't allow you to do that and instead you need to write it down in a file on your own device, make a reminder to go back to the site once the timeout is up, and post it at that time, all the while remaining within the bounds of the rules.

Also, being able to reply to responses to your comments is important, especially when the responses are requests for clarification, as long as that doesn't turn into an infinite discussion. So I think commenting chits should accumulate to a maximum of at least 3-4, even if it takes a week to get there, possibly even more if it's been a month. But maybe an even better option is for all but one of these to be "reply chits" that are weaker than full "comment chits" and only work for replies-to-replies to your own comments or posts. While the full "comment chits" allow commenting anywhere.

I don't see a way around the annoyance of feasibility of personally managed manual posting schedule workarounds other than implementing the queued-posting feature on LW, together with ability to manage the queue, arranging the order/schedule in which the pending comments will be posted. Which is pretty convoluted, putting this whole development in question.

I think I'd prefer setting the bar lower, and instead using downvotes as a filter for merely low-quality (rather than abysmal-quality) content. For instance, most posts on LW receive almost no comments, so I'd suspect that filtering for even higher quality would just dry up the discussion even more.

I did the have the idea of there being regions with varying standards and barriers, in particular places where new users cannot comment easily and place where they can, as an idea.

I do agree Duncan's post is pretty good, and while I don't think it's perfect I don't really have an alternative I think is better for new users getting a handle on the culture here.

Agreed w.r.t. "basic questions" we could ask new users. The subreddit /r/ControlProblem now makes people take a quiz before they can post, to filter for people who e.g. know and care about the orthogonality thesis. (The quiz is pretty easy to pass if you're familiar with the basic ideas of AI safety/alignment/DontKillEveryoneism.)

Agreed. Talking from the perspective of a very occasional author, suppose I post a LW essay and then link it in r/slatestarcodex. I want it to be as easy as possible for readers to comment on my essay wherever I link it. If it's impossible or even just inconvenient for them to simply comment on the essay, they might not do so.

In which case, why would the author post their stuff on LW, specifically?

I think avoiding the negative selection failure modes is an important point. I'm mulling over how to think about it.

Do you have a thing you're imagining with "positive selection?" that you expect to work?

I think this works at universities because teachers are paid to grade things (they wouldn't do it otherwise) and students get some legible-to-the-world certificate once they graduate. 

Like, we already have a wealth of curriculum / as much content for newbies as they can stand; the thing that's missing is the peer reading group and mentors. We could probably construct peer reading groups (my simple idea is that you basically put people into groups based on what <month> they join, varying the duration until you get groups of the right size, and then you have some private-to-that-group forum / comment system / whatever), but I don't think we have the supply of mentors. [This is a crux--if someone thinks they have supply here or funding for it, I want to hear about it.] 

you can get these principles in other ways

I got them via cultural immersion. I just lurked here for several months while my brain adapted to how the people here think. Lurk moar!

I too expect that testing whether someone can talk as if they've absorbed the sequences would measure password-guessing more accurately than comprehension.

The idea gets me wondering whether it's possible to design a game that's easy to learn and win using the skills taught by the sequences, but difficult or impossible without them. Since the sequences teach a skill, it seems like we should be able to procedurally generate novel challenges that the skill makes it easy to complete.

As someone who's gone through the sequences yet isn't sure whether they "really" "fully" understand them, I'd be interested in taking and retaking such a test from time to time (if it was accurate) to quantify any changes to my comprehension.

Re: #1, could you comment on what additional functionality on top of this button would help you?

css - the repetition is to elevate the css specificity, overriding even css styles from sites that use !important in their code; apply to all sites with the stylus extension in chrome:

a:not(:visited):not([href="#"])[href]:not(:visited)[href]:not(:visited)[href]:not(:visited)[href]:not(:visited)[href], a:not([href="#"]):not(:visited)[href]:not(:visited)[href]:not(:visited)[href]:not(:visited)[href]:not(:visited)[href] * {
   color: #32a1ce !important;
}
a:visited[href]:not([href="#"]):visited[href]:visited[href]:visited[href]:visited[href], a:visited[href]:not([href="#"]):visited[href]:visited[href]:visited[href]:visited[href] * {
   color: #939393 !important;
}

It's also quite plausible to me that carefully prompted language models, with a few dozen carefully explained examples and detailed instructions on the decision criteria, would do a good job at this specific moderation task. Less clear what the payoff period of such an investment would be so I'm not actually recommending it, but it's an option worth considering IMO.

I would be interested in helping out with a newbie comment queue to keep it moving quickly so that newbies can have a positive early experience on lesswrong, whereas I would not want to volunteer for the "real" mod team because I don't have the requisite time and skills for reliably showing up for the more nuanced aspects of the role.

Were such a proposal to be adopted, I would be likewise willing to participate.

I really appreciate the thought here (regardless of whether it works out) :) 

Scott Garrabrant once proposed being able to add abstracts to posts that would appear if you clicked on posts on the frontpage. Then you could read the summary, and only read the rest of the post if you disagreed with it.

I appreciate the consideration. I don’t know that I particularly have anything novel or interesting to say about the post in question; I think it mostly stands (or, rather, falls) on its own, and any response I could make would merely repeat things that I’ve said many times. I could say those things again, but what would be the point? Nobody will hear them who hasn’t already heard. (In any case, some decent responses have already been written by other commenters.)

There is one part (actually a quote from Vaniver) which I want to object to, specifically in the context of my work:

There’s a claim I saw and wished I had saved the citation of, where a university professor teaching an ethics class or w/e gets their students to design policies that achieve ends, and finds that the students (especially more ‘woke’ ones) have very sharp critical instincts, can see all of the ways in which policies are unfair or problematic or so on, and then are very reluctant to design policies themselves, and are missing the skills to do anything that they can’t poke holes in (or, indeed, missing the acceptance that sometimes tradeoffs require accepting that the plan will have problems). In creative fields, this is sometimes called the Taste Gap, where doing well is hard in part because you can recognize good work before you can do it, and so the experience of making art is the experience of repeatedly producing disappointing work.

In order to get the anagogic ascent, you need both the criticism of Socrates and the courage to keep on producing disappointing work (and thus a system that rewards those in balanced ways).

In my professional (design) experience, I have found the above to be completely untrue.

My work is by no means perfect now, nor was it perfect when I started; nor will I claim that I’ve learned nothing and have not improved. But it’s simply not the case that I started out “repeatedly producing disappointing work” and only then (and thereby) learned to make good work. On the contrary, I started out with a strong sense and a good understanding of what bad design was, and what made it bad; and then I just didn’t do those things. Instead of doing bad and wrong things, I did good and correct things. Knowing what is good and what is bad, and why, made that relatively straightforward.

(Is there a “rationality lesson” to be drawn from this? I don’t know; perhaps, perhaps not. But it stands as a non-metaphorical point, either way.)

You mean for the new comments by unapproved users? Those won't be visible at all.

I think more downvoting being the solution depends on the goals. If our goal is only to maintain the current quality, that seems like a solution. If the goal is to grow in users and quality, I think diverting people to a real-time discussion location like Discord could be more effective. 

Eg. a new user coming to this site might not have any idea a particular article exists that they should read before writing and posting their 3 page thesis on why AI will/wont be great, only to have their work downvoted (it is insulting and off-putting to be downvoted) and in the end we may miss out on persuading/gaining people. In a chat a quick back and forth could steer them in the right direction right off the bat.

If you apply some of these norms, then imo there are questionable implications, i.e. it seems weird to say that one should have read the sequences in order to post about mechanistic interpretability on the Alignment Forum.

The AI Alignment Forum was never intended as the central place for all AI Alignment discussion. It was founded at a time when basically everyone involved in AI Alignment had read the sequences, and the goal was to just have any public place for any alignment discussion. 

Now that the field is much bigger, I actually kind of wish there was another forum where AI Alignment people could go to, so we would have more freedom in shaping a culture and a set of background assumptions that allow people to make further strides and create a stronger environment of trust. 

I personally am much more interested in reading about mechanistic interpretability from people who have read the sequences. That one in-particular is actually one of the ones where a good understanding of probability theory, causality and philosophy of science seems particularly important (again, it's not that important that someone has acquired that understanding via the sequences instead of some other means, but it does actually really benefit from a bunch of skills that are not standard in the ML or general scientific community). 

I expect we will make some changes here in the coming months, maybe by renaming the forum or starting off a broader forum that can stand more on its own, or maybe just shutting down the AI Alignment Forum completely and letting other people fill that niche. 

similarly, I've been frustrated that medium quality posts on lesswrong about ai often get missed in the noise. I want alignmentforum longform scratchpad, not either lesswrong or alignmentforum. I'm not even allowed to post on alignmentforum!

some recent posts I've been frustrated to see get few votes and generally less discussion:

• https://www.lesswrong.com/posts/JqWQxTyWxig8Ltd2p/relative-abstracted-agency [LW · GW] - this one deserves at least 35 imo
• www.lesswrong.com/posts/fzGbKHbSytXH5SKTN/penalize-model-complexity-via-self-distillation [LW · GW]
• https://www.lesswrong.com/posts/bNpqBNvfgCWixB2MT/towards-empathy-in-rl-agents-and-beyond-insights-from-1 [LW · GW]
• https://www.lesswrong.com/posts/LsqvMKnFRBQh4L3Rs/steering-systems [LW · GW]
• ... many more open in tabs I'm unsure about.

There's maybe two tiers of new user:

• User that has never posted or commented before
• User that has posted or commented a couple times, but not been given a stamp-of-approval by moderators which means we don't have to pay special attention to them anymore.

Until a user has been approved, moderators at least glance at every comment and post they make.

Language models are potentially very capable of making the process of citing dupes much more efficient, and I'm going to talk to AI Objectives about this stuff [LW · GW] at some point in the next week and this is one of the technologies we're planning on discussing.

This is a cool suggested use of language models. I'll think about whether/how to implement it on LW.

A thing I'm finding difficult at the moment is that criticism just isn't really indexed in a consistent way, because, well, everyone has subtly different frames on what they're criticizing and why.

It's possible there should be, like, 3 major FAQs that just try to be a comprehensive index on frequent disagreements people have with LW consensus, and people who want to argue about it are directed there to leave comments, and maybe over time the FAQ becomes even more comprehensive. It's a lot of work, might be worth it anyway

(I'm imagining such an FAQ mostly linking to external posts rather than answering everything itself)

It's likely to apply starting roughly 4 months ago (i.e. when ChatGPT was released). But, this is just a guess and we may make changes to the policy.

It seems to me that building trust by somehow confirming that a person understands certain important background knowledge (some might call this knowledge a "religious story", those stories that inspire a certain social order wherever they're common knowledge), but I haven't ever seen a nice, efficient social process for confirming the presence of knowledge within a community. It always seems very ad hoc. The processes I've seen too demand very uncritical, entry-level understandings of the religious stories, or just randomly misfire sometimes, or are vulnerable to fakers who have no deep or integrated understanding of the stories, or sometimes there will be random holes in peoples' understandings of the stories that cause problems to occur even when everyone's being good faith. Maybe this stuff just inherently requires good old fashioned time and effort and I should stop looking for an easy way through.

Highly recommend this, wish the UI was more discoverable. For people who want this themselves:  you can change how posts are weighted for you by clicking "customize feed" to right of "front page". You'll be shown some default tags, and can also add more. If you hover over a tag you can set it to hidden, reduced (posts with tag treated as having 50% less karma), promoted (+25 karma), or a custom amount. 

It occurs to me I don't know how tags stack on a given post, maybe staff can clarify? 

I have it at -200.