A Bat and Ball made me Sad

If you look in the appendix, you'll see we ran something similar to that, where we told them to "consider 10". It increased solution rates from 13% to 19%.

I partially agree. But I'll push back a little. The 23% wrong answers are not random key mashing. 80% of them type the number "10". Instead of copying the single digit that they are instructed to copy, they are looking further from the answer blank, finding two three-digit numbers, subtracting one from the other, and typing the two-digit difference. That is more work than the correct answer, not less.

But still, I do agree that if people had accuracy incentives, the correct answer rate would probably go up.... Then again, maybe, with monetary incentives, people would be even more likely to think the provided answer was a trap.  If I had to bet, I'd say that incentives would increase the 77% "5" rate. But I wouldn't be totally shocked if it went down.

This response is incorrect. Firstly, Google Consumer Surveys is very different from MTurk. MTurk users are paid to pay attention to the task for a given amount of time, and they are not 'paid ads'. 

"People are incentivized to complete these as soon as possible and there is little penalty for inaccuracy."

This is generally untrue for MTurk - when you run online surveys with MTurk:

1) You set exclusion criteria for people finishing too quickly
2) You set attention check questions that, when you fail, exclude a respondent
3) A respondent is also excluded for certain patterns of answers e.g. answering the same for all questions or sometimes for giving contradictory answers
4) The respondent is rated on their response quality, and they will lose reputation points (and potentially future employment opportunities) for giving an inaccurate response

You can apply these criteria more or less rigorously, but I'd assume that the study designers followed standard practice (see this doc), at least.

I'm not claiming that MTurk is a very good way of getting humans to respond attentively, of course. There are lots of studies looking at error rates: https://arxiv.org/ftp/arxiv/papers/2101/2101.04459.pdf and there are obvious issues with exclusively looking at a population of 'professional survey respondents' for any given question.

So I'm not exactly sure how this survey causes me to update. Perhaps especially because they're slightly rushing the answers, it's genuinely interesting that so many people choose to respond to the (perceived) moderate complexity task (which they assume is: "calculate 110 - 100"), rather than either the simple task: "write out the number 5". 

I guess it depends on what your priors already were but 23% is far higher than the usual 'lizardman', so one update might be to greatly expand how much error is associated with any survey. If the numbers are that high, it gets harder to understand many things (unless more rigorous survey methods are used etc)

See also drossbucket 2018 [? · GW].

This study was done via Google ads that block content, which means the people taking the study were thinking "How do I get this ad to go away so I can watch my video". 

The boldface was part of the problem presented to participants.    It is a famous question because so many people get it wrong. 

Humans are dumb, but not this dumb. This is about people not bothering to answer questions since they're rushing. See other comments.

I feel like handling human irrationality is a major motivation for agent foundations style research. What is the type of human values? How do we model human (ir)rationality to disentangle (unstable) goals from (imperfect) decison making procedures when figuring out what humans want. What's the thing which the human brain is an approximation of and how does it approximate that ideal? I know getting a model of human irrationality is important to Vanessa, and she thinks working on meta-cognition will make some progress on that. Scott Garrabrants work, of which geometric rationality is the latest example, makes some headway on these problems. I'd go even further and say that a lot of non-agent foundation agendas are motivated by this problem. For instance, Steve Byrnes agenda tackles the "modelling human rationality" part more directly than any other agenda I can think of. 

But that's the thing: there's been minor progress at best and researchers who work on these problems are pessimistic about further progress. I certainly don't know of anything that really looks like it would work. Maybe our perspective is wrong. I would guess that shard theorists believe shard-theory would agree with that claim, and would state that shard theory looks like a more promising route to answering these questions (p~0.4). 

So whilst the problem isn't explicitly written about that much, I think a lot of researchers would contest that the problem is being ignored.