Review Voting Thread

post by Ben Pace (Benito) · 2020-12-30T03:23:06.075Z · LW · GW · 132 comments

We're half way through the second annual review, and 121 posts have been nominated

We've had more than double the number of individual nominations than last year, but on reviews we're still playing catchup. Last year we had 118 reviews, yet this year we've only had 51 so far.

When there's so many posts, it can be daunting to figure out which ones to review, so to help out, I'm making this thread. Every comment on this thread will be a post, and you should vote on which ones you would like to read a review of. 

A review is something ideally that puts it in context of a broader conversation, describes its key contributions, its strengths and flaws, and where more work can be done.

(Or something else. Many people who write self-reviews often give a different flavor of review. And I've read many great short reviews, e.g. Jameson Quinn and Zvi last year did a lot of short reviews that communicated their impression of the post quite clearly.)

So I'm going to leave 122 comments on this post. 121 comments will just be a post title, and the other one will be for thread meta. (Search "Meta Thread".) I will remove my own votes from them, so they all start at zero. 

Please vote on the comments to show how much you'd like to see reviews of different posts! Feel free to add a comment about what sort of review you'd like to see.

(Yes, I will probably get a lot of karma from this thread. Mwahaha you have fallen for my evil trap.)

(Also, my thanks to reviewers magfrump and Zvi with 5 each, johnswentworth with 6 reviews, and to fiddler with 10 (!), all thoughtful and valuable.)


Comments sorted by top scores.

comment by Ben Pace (Benito) · 2020-12-30T03:28:15.832Z · LW(p) · GW(p)

What failure looks like [LW · GW] by paulfchristiano.

Replies from: johnswentworth
comment by johnswentworth · 2020-12-30T14:41:47.263Z · LW(p) · GW(p)

I'm specifically interested in a review of this post by someone who found these scenarios novel.

comment by Ben Pace (Benito) · 2020-12-30T03:48:16.313Z · LW(p) · GW(p)

Risks from Learned Optimization: Introduction [LW · GW] by evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabrant.

comment by Ben Pace (Benito) · 2020-12-30T03:40:52.549Z · LW(p) · GW(p)

Book summary: Unlocking the Emotional Brain [LW · GW] by Kaj_Sotala.

Replies from: mr-hire
comment by Matt Goldenberg (mr-hire) · 2020-12-30T03:55:09.395Z · LW(p) · GW(p)

This is an excellent review. One thing I thought it could do better is in the vein of epistemic spot checks, pointing out places where the authors conjectures are far ahead of the science For instance, AFAICT, memory reconsolidation has only been proven for up to a few weeks in Mice models, but they talk about it being used to reconsolidate childhood memories in the book.

comment by Ben Pace (Benito) · 2020-12-30T03:58:49.047Z · LW(p) · GW(p)

No, it's not The Incentives—it's you [LW · GW] by Zack_M_Davis.

Replies from: mr-hire
comment by Matt Goldenberg (mr-hire) · 2020-12-30T04:15:14.459Z · LW(p) · GW(p)

I think there's a lot of value in this post articulating a certain frame. Don't know how this works since it's a link post, but would love to see a review that more explicitly pointed at the frame, how it's useful and not useful, and pointed out the ways the author could make the frame more explicit.

comment by Ben Pace (Benito) · 2020-12-30T03:36:38.568Z · LW(p) · GW(p)

Book Review: Secular Cycles [LW · GW] by Scott Alexander.

comment by Ben Pace (Benito) · 2020-12-30T03:26:59.252Z · LW(p) · GW(p)

Thoughts on Human Models [LW · GW] by Ramana Kumar, Scott Garrabrant.

comment by Ben Pace (Benito) · 2020-12-30T03:45:28.579Z · LW(p) · GW(p)

Paper-Reading for Gears [LW · GW] by johnswentworth.

comment by Ben Pace (Benito) · 2020-12-30T03:52:23.711Z · LW(p) · GW(p)

Unconscious Economics [LW · GW] by jacobjacob.

comment by Ben Pace (Benito) · 2020-12-30T03:57:57.061Z · LW(p) · GW(p)

Rule Thinkers In, Not Out [LW · GW] by Scott Alexander.

comment by Ben Pace (Benito) · 2020-12-30T03:44:42.838Z · LW(p) · GW(p)

Relevance Norms; Or, Gricean Implicature Queers the Decoupling/Contextualizing Binary [LW · GW] by Zack_M_Davis.

Replies from: mr-hire
comment by Matt Goldenberg (mr-hire) · 2020-12-30T03:51:01.897Z · LW(p) · GW(p)

I made a comment when that post first came out that I thought this was missing the mark on what contextualizing is actually trying to get at (in particular, focusing on language meanings rather than the consequences of language use).  I think this whole sequence by Zach is at it's best when it focuses on the upsides of decoupling, and at it's worst when it tries to explain contextualizing, and would like to see a review that covers both those strengths and weaknesses.

comment by Ben Pace (Benito) · 2020-12-30T03:32:41.015Z · LW(p) · GW(p)

The AI Timelines Scam [LW · GW] by jessicata.

comment by Ben Pace (Benito) · 2020-12-30T03:25:40.324Z · LW(p) · GW(p)

S-Curves for Trend Forecasting [LW · GW] by mr-hire.

Replies from: mr-hire
comment by Matt Goldenberg (mr-hire) · 2020-12-30T15:27:22.743Z · LW(p) · GW(p)

I would love to see someone review this post!  In particular, there was a critique about "falsifiability" - would love to hear the exact problems with what's unfalsifiable, and address them in a subsequent edit.  

comment by Ben Pace (Benito) · 2020-12-30T03:55:52.162Z · LW(p) · GW(p)

The Zettelkasten Method [LW · GW] by abramdemski.

comment by Ben Pace (Benito) · 2020-12-30T03:55:27.520Z · LW(p) · GW(p)

Meta Thread

Replies from: ryan_b
comment by ryan_b · 2020-12-31T16:36:37.562Z · LW(p) · GW(p)

I'm surprised by the presence of negative scores on some posts. Is this to be interpreted as "please do not review this post" or as "please get to the others first"?

All three seem to be good explanatory pieces in and of themselves; I wonder if there is a kind of performance penalty where if a post does not seem like it would benefit much from a review process, it gets pushed to the back of the line. This isn't bad really, in fact it seems fairly efficient, I just didn't expect it.

Replies from: Benito
comment by Ben Pace (Benito) · 2020-12-31T19:30:28.227Z · LW(p) · GW(p)

+1 was a bit surprised. Don't think it matters too much. Except mildly think it increases the chance those posts get reviewed.

comment by Ben Pace (Benito) · 2020-12-30T03:52:12.735Z · LW(p) · GW(p)

Blackmail [LW · GW] by Zvi.

comment by Ben Pace (Benito) · 2020-12-30T03:48:40.354Z · LW(p) · GW(p)

Understanding “Deep Double Descent” [LW · GW] by evhub.

Replies from: johnswentworth
comment by johnswentworth · 2020-12-30T14:50:53.694Z · LW(p) · GW(p)

Specifically interested in reviews covering related work, replication, follow-up, etc.

comment by Ben Pace (Benito) · 2020-12-30T03:41:42.286Z · LW(p) · GW(p)

Turning air into bread [LW · GW] by jasoncrawford.

Replies from: Raemon
comment by Raemon · 2020-12-30T05:26:19.478Z · LW(p) · GW(p)

I (weakly) think this one is probably more important than the progress studies post on concrete. 

comment by Ben Pace (Benito) · 2020-12-30T03:59:41.205Z · LW(p) · GW(p)

Make more land [LW · GW] by jefftk.

comment by Ben Pace (Benito) · 2020-12-30T03:56:02.641Z · LW(p) · GW(p)

Gears vs Behavior [LW · GW] by johnswentworth.

comment by Ben Pace (Benito) · 2020-12-30T03:54:56.450Z · LW(p) · GW(p)

Bioinfohazards [LW · GW] by Spiracular.

comment by Ben Pace (Benito) · 2020-12-30T03:54:26.853Z · LW(p) · GW(p)

Why Subagents? [LW · GW] by johnswentworth.

comment by Ben Pace (Benito) · 2020-12-30T03:54:03.324Z · LW(p) · GW(p)

Selection vs Control [LW · GW] by abramdemski.

comment by Ben Pace (Benito) · 2020-12-30T03:45:06.886Z · LW(p) · GW(p)

Mental Mountains [LW · GW] by Scott Alexander.

comment by Ben Pace (Benito) · 2020-12-30T03:38:52.432Z · LW(p) · GW(p)

The Power to Teach Concepts Better [LW · GW] by Liron.

Replies from: mr-hire
comment by Matt Goldenberg (mr-hire) · 2020-12-30T04:00:54.835Z · LW(p) · GW(p)

I made some critiques of this sequence when it first came out, related to the implicit framing that specificity is always good, and generalist is always sloppy ( this a frame, I don't think it's stated explicitly). Similar to my comment about Zach's sequence, I think this sequence is at its best when talking about the benefits of specificity, and at its worst when talking about the problems with non-specificity.

I'd like to see a review that highlights these merits while pointing out the missed perspectives. Bonus points if it relates to the some of the notions about withholding specificity discussed in alkjashs post on Babble, which was included in last year's review.

comment by Ben Pace (Benito) · 2020-12-30T03:33:39.603Z · LW(p) · GW(p)

The Costs of Reliability [LW · GW] by sarahconstantin.

comment by Ben Pace (Benito) · 2020-12-30T03:53:05.348Z · LW(p) · GW(p)

In My Culture [LW · GW] by Duncan_Sabien.

comment by Ben Pace (Benito) · 2020-12-30T03:42:22.721Z · LW(p) · GW(p)

Evolution of Modularity [LW · GW] by johnswentworth.

comment by Ben Pace (Benito) · 2020-12-30T03:38:13.868Z · LW(p) · GW(p)

What is operations? [LW · GW] by Swimmer963.

comment by Ben Pace (Benito) · 2020-12-30T03:31:57.347Z · LW(p) · GW(p)

Circle Games [LW · GW] by sarahconstantin.

comment by Ben Pace (Benito) · 2020-12-30T03:56:13.053Z · LW(p) · GW(p)

Total horse takeover [LW · GW] by KatjaGrace.

comment by Ben Pace (Benito) · 2020-12-30T03:30:45.317Z · LW(p) · GW(p)

Reframing Impact [LW · GW] by TurnTrout.