new study finds performance enhancing drugs for chess

post by morganism · 2017-01-27T00:04:39.039Z · LW · GW · Legacy · 14 comments

This is a link post for https://worldchess.com/2017/01/25/special-report-new-study-finds-performance-enhancing-drugs-for-chess/

Contents

14 comments

14 comments

Comments sorted by top scores.

comment by Douglas_Knight · 2017-01-27T15:58:49.982Z · LW(p) · GW(p)

The abstract is really unimpressive. The effect was not statistically significant. Was it even positive?* It is only by throwing out games lost on time that they find improved performance. Systematically throwing out games lost and finding a lot of games won is a standard error. Did it improve the moves they made, or did it cause them just to ruminate on lost games? They could have tested this, asking computers how good the moves were, but I don't think that they did. Even if the moves were better, was it because of the drugs, or because of the additional time spent, a decision that might have been possible without drugs? The claim that drugs made the players consume more time was interesting, though.

* Update: yes, the effect was positive before throwing out the games lost on time. The effect size was 7%, half of that reported in the abstract after throwing out losses on time. More concretely, the improvement was turning 2.5% of games from losses into wins and 2% of games from draws into wins.

Replies from: Douglas_Knight, WhySpace_duplicate0.9261692129075527
comment by Douglas_Knight · 2017-02-01T16:55:19.975Z · LW(p) · GW(p)

Addendum: Stimulants mess with your sense of time. Time management is an important part of playing chess, but surely players don't do it by looking at the clock, but by heuristics about how much to think, heuristics that could be messed up by their sense of time. So I think that the time management would be improved by practice with the drugs. Most chess players probably have practice at varying levels of caffeine, so the study is a more fair comparison of that drug than the others.

Added later: Indeed, caffeine lead to fewer losses on time than modafinil and methylphenidate, although more than placebo, even though raw win/loss/draw numbers were the same for the three drugs. (Oddly, although the score excluding games lost on time puts modafinil and methylphenidate together, a more general metric of performance controlling for time groups together methylphenidate with caffeine.)

comment by WhySpace_duplicate0.9261692129075527 · 2017-02-01T16:19:26.185Z · LW(p) · GW(p)

I'm worried that I found the study far more convincing than I should have. If I recall, it was something like "this would be awesome if it replicates. Regression toward the mean suggests the effect size will shrink, but still." This thought didn't stop me from still updating substantially, though.

I remember being vaguely annoyed at them just throwing out the timeout losses, but didn't discard the whole thing after reading that. Perhaps I should have.

I know about confirmation bias and p-hacking and half a dozen other such things, but none of that stopped me from overupdating on evidence I wanted to believe. So, thanks for your comment.

Replies from: Lumifer
comment by Lumifer · 2017-02-01T16:52:50.526Z · LW(p) · GW(p)

An interesting concept -- un-updating. Should happen when you updated on evidence that turned out to be wrong/mistaken, so you need to update back and I suspect that some biases will be involved here :-/

comment by morganism · 2017-01-27T00:06:40.767Z · LW(p) · GW(p)

Modafinil improved performances by 15 percent, methylphenidate by 13 percent, and caffeine by around 9 percent."

"Dr. Lieb said that the researchers went into the study with the expectation that the stimulants would not show much benefit. “We primarily thought that it is not possible to enhance high cognitive tasks and were astonished to find such results,” he said."

Replies from: madhatter
comment by madhatter · 2017-01-27T00:42:03.757Z · LW(p) · GW(p)

I'm not surprised. But I also don't see much utility from this study; most people already believe that coffee helps them focus.

Replies from: shev, Benquo
comment by shev · 2017-01-27T06:27:14.239Z · LW(p) · GW(p)

Don't you think there's some value of doing a more controlled study of it?

comment by Benquo · 2017-01-27T03:45:15.509Z · LW(p) · GW(p)

Something about "makes play better but slower" feels especially persuasive to me.

Replies from: madhatter
comment by madhatter · 2017-01-27T19:17:51.461Z · LW(p) · GW(p)

Actually, as a tournament player I feel I can help explain the slowness:

The article suggests that this isn't due to increased computational speed or focus, but I think that's wrong. Playing slowly doesn't imply thinking slowly. In a chess game, you have a certain amount of time overall, and often when the position is very complicated players will spend half an hour delving into variations and sub-variations. If it's hard to concentrate, they may just rely on low-calc alternatives, and play faster.

Replies from: Elo
comment by Elo · 2017-01-27T22:33:20.266Z · LW(p) · GW(p)

Agreed.

comment by James_Miller · 2017-01-27T15:58:26.788Z · LW(p) · GW(p)

There might be great value in setting up a way for an individual to evaluate his personal performance improvement from different drugs so you know how to optimize your performance on days when it really matters such as when you take an important test.

Replies from: Lumifer
comment by Lumifer · 2017-01-27T17:13:37.331Z · LW(p) · GW(p)

...paging gwern

:-)

comment by ChristianKl · 2017-01-28T09:02:05.746Z · LW(p) · GW(p)

What the hell is an improvement in Chess by 15%? How many Elo points are we talking about?

Replies from: Good_Burning_Plastic
comment by Good_Burning_Plastic · 2017-01-28T13:49:04.025Z · LW(p) · GW(p)

400*log10(1.15), I'd guess?