Low legibility of Cognitive Reflection Test dramatically improves performance?

Thanks! I've followed references and I think I have the original paper: http://pages.stern.nyu.edu/~aalter/intuitive.pdf

We recruited 40 Princeton University undergraduate volunteers at the student campus center to complete the three-item CRT (Frederick, 2005). Participants were seated either alone or in small groups, and the experimenter ensured that they completed the questionnaire individually. Those in the fluent condition completed a version of the CRT written in easy-to-read black Myriad Web 12-point font, whereas participants in the disfluent condition completed a version of the CRT printed in difficult-to-read 10% gray italicized Myriad Web 10-point font. Participants were randomly assigned to complete either the fluent or the disfluent version of the CRT (...) As predicted, participants answered more items on the CRT correctly in the disfluent font condition ... Whereas 90% of participants in the fluent condition answered at least one question incorrectly, only 35% did so in the disfluent condition.

Excellent! I'll do my exams upside down.

ETA: It was a joke - I don't really intend to do this in my future exams.

(crossposting this forum post from July, I think it has some bearing on this idea)

So I was thinking about scientists, and how they frequently write things that laypeople can't begin to parse. Do they do this to look smart? All scientists live in a crippling fear that what they are doing is not important or groundbreaking, so like a squid they generate a defensive cloud of technical terminology to obscure that fact. For a time I thought this was the whole of the explanation, when I was at my nadir of cynicism. Then I thought, well, maybe scientists just don't write very well. The things they write are unreadable not because of some plot to make them unreadable, but because it is the natural state of text to be unreadable, and there's no optimization process pulling scientific writing away from that. Then I grew kinder to scientists, and thought, maybe it's not that they can't produce readable text, but that they don't bother, because the point of scientific writing is to communicate facts to other scientists efficiently, and to hell with laypeople.

I have a new theory.

Human beings are not good at science. Their brains do not naturally work by looking at evidence and updating probabilities based on it, they pattern-match the things they see to the things they've seen before and treat them the same. The fundamental operation of the human brain is a cache lookup, and to the extent that humans have rationality it is a high-level construct built on top of it. Scientific language is an attempt to force a cache miss, a page fault, so that humans are forced to actually bring their rationality to bear instead of assuming that they already know the answers because particle physics is basically like billiards and I'm good at billiards. Take "climate change" (please). When the scientific community started talking about "climate change" instead of "global warming", people called it a cynical political move. Through this lens, though, it looks like a desperate attempt to regain scientific neutrality for a topic that has flooded everyone's cache due to widespread popular contention. Akin to the euphemism treadmill and the dysphemism treadmill, we have here the formalism treadmill.

"On the Pedagogical Motive for Esoteric Writing", Arthur Melzer 2007:

What evidence and what arguments can be produced in support of the controversial suggestion, first made by Leo Strauss now over 65 years ago, that most earlier philosophers wrote esoterically and, what is more, that they did so, not merely from fear of persecution, but with an eye to enhancing their pedagogical effectiveness? I argue here that the inherent paradoxes of philosophical education combined with the inherent shortcomings of writing led many earlier thinkers to see the pedagogical necessity of something like the “Socratic method.” And esoteric writing—a rhetoric of riddling concealment—is the closest literary approximation to the Socratic method.

My opinion of the general Straussian suggestion has been heightened by the recent claims of finding musical structures in Plato's dialogues, who had been one of the major proposed users of esotericism.

Author continues:

Cognitive strain, whatever its source, mobilizes System 2, which is more likely to reject the intuitive answer suggested by System 1.

System 1 is the impulsive, unconscious, eager but not very intelligent aspect of the mind. System 2 is slow, conscious and more thoughtful, but "lazy" and prone to accept suggestions from the System 1. Theory is that inducing cognitive strain diverts more mental resources to the System 2, which then tends to do a proper job at solving the test.

thanks, fixed

I don't think it's self-evident that effort put in recognizing letters should translate into significant improvement in problem solving. For example, it could be expected that this lower-level burden would drain cognitive "energy" from higher functions trying to solve the mathematical problem.