Game that might improve research productivity

post by Jack R (Jack Ryan) · 2022-03-26T07:00:11.599Z · LW · GW · 3 comments

Contents

  Insight Hunt
None
3 comments

To figure out how to make progress on problems that require cognitive effort, it seems useful to think of times in the past when you’ve made lots of cognitive progress. For me, high school math camp was the most cognitively productive time.

It seems plausible that the main reason I was productive at math camp was because of the competitive atmosphere. In light of this, I want to experiment with systems that harness people’s competitiveness to drive learning and insight, since maybe this can increase productivity substantially. “Insight Hunt” is a way this could be done. 

I suspect the below idea is somewhat far from the ideal and that it could change a fair bit in response to user feedback. If anyone tries an Insight Hunt, please leave your feedback and results (including the name of the winner if you want increased motivation) in the comments! Also, feel free to suggest alternative games/systems which leverage competitiveness for cognitive productivity.

Lastly, I suspect that Insight Hunt works best when the stuff the players want to learn is more “shallow” i.e. knowledge/facts that require few inferential steps from what most people know to understand. This is because if one person wants to learn linear algebra (not shallow), and another person knows linear algebra, the game will devolve into a lecture (I suspect). Examples of shallow knowledge include alignment theory (given some pre-requisites, like basic ML; this is the primary use-case I'm excited about) and psych study results. Non-examples include theoretical physics and math.
 

Insight Hunt

3 comments

Comments sorted by top scores.

comment by TLW · 2022-03-28T06:52:03.402Z · LW(p) · GW(p)

Be careful that this doesn't devolve into... I'm not going to say clickbait, because it's a little longer timescale than that. Flashes in the pan?

Things that sound great for the first minute or two but fall apart on deeper inspection.

Replies from: Jack Ryan
comment by Jack R (Jack Ryan) · 2022-03-28T10:01:59.665Z · LW(p) · GW(p)

I'd be interested in you sharing any reasons why you think this might fall apart, e.g. any insights you've gained from deeper inspection.

Replies from: TLW
comment by TLW · 2022-03-29T02:10:16.122Z · LW(p) · GW(p)

I've made a career of being good at finding 'weird' failure modes[1]. I won't give further details about said career here.

=====

In general: look for the perverse incentives. Especially in competitive settings, or where money is on the line.

 In this case, the actual target is something along the lines of 'teach the judge the basics of alignment theory', and the metric is 'say things that the judge believes is good and interesting within the next minute'.

Well... what happens when the judge has a subtle[2] misconception? Seekers are incentivized to go along with it rather than fight it. This helps the metric but hurts the underlying target.

What happens when there's a single step in a knowledge chain that takes more than a minute to be interesting? Seekers are incentivized to ignore that knowledge chain. This helps the metric but hurts the underlying target.

Etc.

  1. ^

    Of course, I'm firmly on the side of having found >10 of the last 5 failures. Things wouldn't be great if everyone was me, but having one of me on a team can work reasonably well.

  2. ^

    Read: would take longer than a minute to get the judge to realize that it was a problem at all.