The LessWrong 2022 Review: Review Phase

post by RobertM (T3t) · 2023-12-22T03:23:49.635Z · LW · GW · 7 comments

Contents

  Nomination Phase Results
  Inline Reacts!
  Prizes?  Prizes!
  Final Voting
None
7 comments

This year's LessWrong review [? · GW] nomination phase ended a few days ago, with 339 posts nominated.  For comparison, 291 posts were nominated in the 2021 review.

Nomination Phase Results

Here are the current top-20 posts by vote total:

  1. AGI Ruin: A List of Lethalities [LW · GW
  2. MIRI announces new "Death With Dignity" strategy [LW · GW
  3. Simulators [LW · GW
  4. Where I agree and disagree with Eliezer [LW · GW
  5. Reward is not the optimization target [LW · GW
  6. Six Dimensions of Operational Adequacy in AGI Projects [LW · GW
  7. You Are Not Measuring What You Think You Are Measuring [LW · GW
  8. Epistemic Legibility [LW · GW
  9. Let’s think about slowing down AI [LW · GW
  10. It Looks Like You're Trying To Take Over The World [LW · GW
  11. Staring into the abyss as a core life skill [LW · GW
  12. Counterarguments to the basic AI x-risk case [LW · GW
  13. Sazen [LW · GW
  14. Losing the root for the tree [LW · GW
  15. The shard theory of human values [LW · GW
  16. Limerence Messes Up Your Rationality Real Bad, Yo [LW · GW
  17. Models Don't "Get Reward" [LW · GW
  18. Toni Kurz and the Insanity of Climbing Mountains [LW · GW
  19. Butterfly Ideas [LW · GW
  20. On how various plans miss the hard bits of the alignment challenge [LW · GW

(I'm sensing a bit of a theme...)

More than 60 posts have already been reviewed, but that leaves quite a few posts that have yet to receive any reviews, including many of the most-upvoted ones.  If you want to see which posts are most under-reviewed, you can switch your sorting to Magic (Needs Review) [? · GW][1].  Maybe you have thoughts on Paul's thoughts [LW · GW] on Eliezer's thoughts?

Inline Reacts!

We've got these new nifty inline reacts which you can leave on posts (not just comments!); you may have noticed them.  I encourage you to make good use of these when reviewing posts.  (Typos should now be a lot less annoying to report, if you're inclined to do so.)

But maybe you have more sophisticated intentions.

Prizes?  Prizes!

Last year we awarded prizes [LW(p) · GW(p)] for good reviews.  This year we will also award prizes!  We're aiming for something similar to last year's, though we haven't yet worked out the details (size, scope, etc).

Briefly noting what you liked or didn't like about a post is good. But here are some specific things we're particularly excited for, from last year

New Information. My favorite reviews are ones that give new information to the reader. This could be a concrete fact, or a rebuttal to a key argument in a post, or a new way the post is relevant that you hadn't considered. It could be a new frame about how to even think about the post.

Concrete use cases. I think reviews that say "this concretely helped me, here's how" are also great, especially if they give specifics. I think it's both helpful and rewarding to individual authors to hear how their post was useful, so they can do more of that. (I think this is also helpful on a broader level, so that the collective LW userbase can see what kinds of effects are realistic)

i.e. rather than saying "this post has helped me a bunch over the years", say "here are two specific times that it helped me, and how."

Thoughts on how the post could be improved. Two years later, if a post still seems like good reference material, but is confusing or poorly argued, give advice on how to improve it. Discuss the use-case for the readership you have in mind.

Reflect on the Big Picture. How do various posts fit together into something greater than the sum of their parts? What major conversations have happened on LessWrong and what have you taken from them?

Brevity/clarity.  I'm not saying "optimize for shortness at the expense of saying anything substantive", just, note that all-else-being-equal, taking less space to convey the key information is helpful.

Final Voting

The review phase ends on January 14th, which is when final voting starts.

 

  1. ^

    Or click that link!

7 comments

Comments sorted by top scores.

comment by habryka (habryka4) · 2024-01-10T22:04:40.068Z · LW(p) · GW(p)

First round of review prizes for this year:

$200: 

$100: 

$50: 

comment by MondSemmel · 2023-12-22T09:34:58.740Z · LW(p) · GW(p)

Last year we awarded prizes for good reviews.  This year we will also award prizes!  We're aiming for something similar to last year's, though we haven't yet worked out the details (size, scope, etc).

What are you looking for in reviews? What makes for good ones?

Replies from: Raemon
comment by Raemon · 2023-12-22T09:41:34.297Z · LW(p) · GW(p)

I just edited the post to include a recap of some things that make a good review, 

comment by MondSemmel · 2023-12-22T09:32:09.810Z · LW(p) · GW(p)

For clarity, are the review & post results going to be separated into AI and non-AI stuff again, like they IIRC were in some previous year? I'd like to review some non-AI posts, but wouldn't bother doing so if all the top spots are going to be won by AI posts by default.

Replies from: Raemon
comment by Raemon · 2023-12-22T09:39:32.286Z · LW(p) · GW(p)

I'm not sure this question is that meaningful – we carve up the results in different ways for different purposes. In previous years we've put the top 50 things into books. We may not do exactly that again, but I think taking stock of the top 50 posts is still a worthwhile thing to do. Epistemic Legibility clocks in at #8 followed by non-AI posts. I think the non-AI content from the year was still doing something meaningful I want people to think about and process.

(Or, to put another way: if it turned out that not dividing up the categories was making people feel unmotivated to do non-AI posts, I'd probably go ahead and break them up into categories, because it does seem important to me)

Replies from: MondSemmel
comment by MondSemmel · 2023-12-22T10:01:33.885Z · LW(p) · GW(p)

we carve up the results in different ways for different purposes.

I meant for stuff like prizes etc.

Overall my attitude is like, if the top 7 posts are all AI posts, that's not because they're necessarily better than the best non-AI posts, but rather because AI has been The Topic since 2022, plus the readership has dramatically shifted towards AI content. At which point we might as well declare LW to be a full-time AI site and consider all the rest to be mere hobbyist content =(. Such a ranking outcome would disincentivize authors from writing about the latter. Better to split the ranking into two top-25s or something.

Also, if the rankings are not split up, then if one only visits LW for AI or non-AI content, that gives an annoying strategic incentive to review-downvote all the other content. That doesn't occur if the rankings are separate.

Replies from: Raemon
comment by Raemon · 2023-12-22T10:28:17.370Z · LW(p) · GW(p)

Ah, gotcha. Yeah this does update me that we should split out the prizes in some way along the lines you're suggesting here. (I'm not involved with final decisions on the Review this year since I'm focused on some other projects, but this does seem like an issue)