Humans are very reliable agents

The best current image models have ~99.9% reliability on MNIST, an easy handwriting recognition task.

That's pretty much 100%. Have you looked at the hopelessly ambiguous examples https://towardsdatascience.com/going-beyond-99-mnist-handwritten-digits-recognition-cfff96337392 or the mislabeled ones https://arxiv.org/pdf/1912.05283.pdf#page=8 https://cleanlab.ai/blog/label-errors-image-datasets/ ? I'm not sure how many of the remaining errors are ones where a human would look at it and agree that it's obviously what the label is. (This is also a problem with ImageNet these days.)

And yes, a lot of this is going to depend on reference classes. You point to guns as incredibly safe, which they are... as long as we exclude gun suicides or homicides, of course. Are those instances of human unreliability? If we are concerned with bad outcomes, seems like they ought to count. If you grab a gun in a moment of disinhibition while drinking and commit suicide, which you wouldn't've if you hadn't had a gun, in the same way that people blocked from jumping off a bridge turn out to not simply jump somewhere else later on, you're as dead as if a fellow hunter shot you. If a pilot decides to commit suicide by flying into a mountain (which seems to have happened yet again in March, the China Eastern Airlines crash), you're just as dead as if the wings iced up and it crashed that way. etc

From Marvin Minsky,

We rarely recognize how wonderful it is that a person can traverse an entire lifetime without making a really serious mistake, like putting a fork in one's eye or using a window instead of a door.

The US has around one traffic fatality per 100 million miles driven; if a human driver makes 100 decisions per mile

A human driver does not make 100 "life or death decisions" per mile. They make many more decisions, most of which can easily be corrected, if wrong, by another decision.

The statistic is misleading though in that it includes people who text, drunk drivers, tired drivers. The performance of a well rested human driver that's paying attention to the road is much, much higher than that. And that's really the bar that matters for self driving car, you don't want a car that is doing better than the average driver who - hey you never know - could be a drunk.

The point about an AI being better than a human on average, but the worst case AI being much worse than a human seems like a critical insight! I haven't thought about it this way, but it matches what Eliezer keeps saying about AI dangers: AI is much more dangerous "out of distribution". A human can reasonably reliably figure out a special case where "222+222=555" even though 1+1=2, but an AI trained on a distribution will likely insist on a bad "out of distribution" action, and occasionally have a spectacular but preventable fatality when used as FSD autopilot, without a human justification of "The driver was drunk/distracted/enraged".

One nitpick on the estimates: a fatality may not be the best way to evaluate reliability: humans would probably have a lot more errors and near-misses than an AI ("oh sh*t, I drifted into a wrong lane!"), but fewer spectacular crashes (or brains sliced through, or civilians accidentally shot by fully autonomous drones).

Thought-provoking post, though as you hinted it's not fair to directly compare "classification accuracy" with "accuracy at avoiding catastrophe". Humans are probably less reliable than deep learning systems at this point in terms of their ability to classify images and understand scenes, at least given < 1 second of response time. Instead, human ability to avoid catastrophe is an ability to generate conservative action sequences in response to novel physical and social situations - e.g. if I'm driving and I see something I don't understand up ahead I'll slow down just in case.

I imagine if our goal was "never misclassify an MNIST digit" we could get to 6-7 nines of "worst-case accuracy" even out of existing neural nets, at the cost of saying "I don't know" for the confusing 0.2% of digits.

Disagree with the framing metaphor being one of binary decisions that lead to crashes when made wrong. We intentionally design our systems so that wide deviation along the relevant dimensions don't cause problems. For example, joysticks allow much more precise steering than steering wheels. But in testing this was found to be undesirable, you don't actually want precision at the expense of sensitivity. Joystick users were more precise but crashed more because it was easier to accidentally give an extreme input.

Humans are very reliable agents for tasks which humans are very reliable for.

For most of these examples (arguably all of them) if humans were not reliable at them then the tasks would not exist or would exist in a less stringent form.

I'm not sure the what the right tags for this are, curious about people's thoughts. "Reliability" seems like it should be a fairly recurring topic, and something about "evaluating benchmarks" also seems plausible, not sure if we already have tags that are close to those.

I see it as less of "Humans are more reliable than AI" but more of "Humans and AI do not tend to make the same kind of mistakes". And the everyday jobs we encounter have been designed/evolved around human mistake patterns so we are less likely to cause catastrophic failures. Keeping the job constant and replacing humans with AI would obviously lead to problems. 

For well-defined simple jobs like arithmetic, AI has a definite accuracy edge compared to human beings. Even for complex jobs, I am still unsure if human beings have the reliability edge. We had quite a few human errors leading to catastrophic failures in earlier stages of complex projects like space explorations, nuclear power plants, etc. But as time passes on we rectify the job so human errors are less likely to cause failure. 

Curated. This is sort of a simple and obvious point. I think I separately had a sense of how reliable humans are, and how reliable ML typically is these days. But I found it viscerally helpful to have driven home the point of how far we have to go.

Note: This post has been recorded as part of the LessWrong Curated Podcast, and an be listened to on Spotify, Apple Podcasts, and Libsyn.

Worth pointing out that although go anywhere/level 5 self-driving isn't a solved problem, computers are already more reliable than humans in the domain of highway driving.  

I do agree with the general point that AI fails (often in bizarre ways) at a rate far too high to be acceptable for most safety-critical tasks.

Somewhat related: Estimating the Brittleness of AI.

Some credit to the road and vehicle engineers should probably be given here.

There are design decisions that make it easier for humans to avoid crashes, and that reduce the damage of crashes when they do occur.

Not sure how many of those nines in the human reliability figure represent a 'unit of engineering effort' by highway/vehicle designers over the last hundred years, but it isn't zero.

I wouldn't call the low death rate from surgery humans being highly reliable. Surgery used to be much deadlier. Humans have spent many many years improving surgical methods (tools, procedures, training), including by using robotic assistance to replace human activity on subtasks where the robots do better. Surgery as practiced by current trained humans with their tools & methods is highly reliable, but this reliability isn't something inherent to the humans as agents.

Solid post, but I think in particular mentioning GPT distracts from the main point. GPT is a generative model with no reward function, meaning it has no goals that it's optimizing for. It's not engineered for reliability (or any other goal), so it's not meaningful to compare it's performance against humans in goal-oriented tasks.

Humans can also still work reliably with some of their brain cut out. Artificial neural networks seem more fragile.