How Should We Critique Research? A Decision Perspective

post by gwern · 2019-07-14T22:51:59.285Z · score: 49 (12 votes) · LW · GW · 4 comments

This is a link post for https://www.gwern.net/Research-criticism

4 comments

Comments sorted by top scores.

comment by Raemon · 2019-07-15T20:42:02.487Z · score: 6 (3 votes) · LW · GW

I'm too busy to do so today, but I'd appreciate it if someone wrote up a comment that distilled this down into summary.

comment by gwern · 2019-07-15T22:12:19.701Z · score: 5 (3 votes) · LW · GW

Does the abstract not work for you?

comment by Raemon · 2019-07-15T22:24:45.468Z · score: 7 (4 votes) · LW · GW

Hmm. Maybe it does. I guess what I normal do when writing a distillation is check whether the abstract was sufficient.

(I have not yet read the post in full, but predicted that if I had, I'd want something that looked more like this [LW · GW] as a distillation)

I generally want two things out of a distillation:

  • An 80/20 of the post that does at least some work to clarify any assumptions the post is working in, and give some examples that of the higher level details of the post that get across some pieces of the post's generators, as well as it's content. (This is mostly for people who don't have time to read the whole post)
  • A skimmable document that I can use to refer back to the post in medium resolution, after I've actually read it. Where the point is to keep markers for each major concept the post introduces within a single visual field I can easily use to expand my working memory limitations.
comment by romeostevensit · 2019-07-15T09:06:20.485Z · score: 2 (1 votes) · LW · GW

I call it decision leverage, for handiness, and looking for it changes casual conversation as well as analysis.