[linkpost] The final AI benchmark: BIG-bench

I think BIG-bench could be the final AI benchmark: if a language model surpasses the top human score on it, the model is an AGI.

Could you explain the reasoning behind this claim? Note that PaLM already beats the "human (Avg.)" on 150 tasks and the curve is not bending. (So is PaLM already an AGI?) It also looks like a scaled up Chinchilla would beat PaLM. It's plausible that PaLM and Chinchilla could be improved by further finetuning and prompt engineering. Most tasks in BIG-Bench are multiple-choice, which is favorable to LMs (compared to generation). I'd guess that some tasks will leak into training data (despite the efforts of the authors to prevent this). 

Source for PaLM: https://arxiv.org/abs/2204.02311

Codex + CoT reaches 74 on a *hard subset* of this benchmark: https://arxiv.org/abs/2210.09261

The average human is 68, best human is 94.

Only 4 months passed and people don't want to test on full benchmark because it is too easy...

The inclusion criteria states:

Tasks that are completely beyond the capabilities of current language models are also encouraged

It's easy to come up with a benchmark that requires a high but unspecified level of intelligence. An extreme example would be to ask for a proof that P!=NP - we have no idea about the difficulty of the task, though we suspect that it requires superintelligence. To be valuable, the challenge of a benchmark needs to possible to relate to meaningful capabilities, such as "The Human Level".

Most people couldn't answer questions about cryobiology in Spanish, even though they possess general intelligence. This benchmark seems to consist of random tasks around and above the human level, and I fear progress on this benchmark might be poorly correlated with progress towards AGI.

Some conclusions from this, assuming this holds for AGI and ASI are:

A: Eliezer was wrong about Foom being plausible, and that is probably the single most x-risk reduction for MIRI. Re AI: I'd update from a 90-99% chance of x-risk to a maximum of 30%, and often the risk estimate would be 1-10% percent due to uncertainties beyond the Singularity/Foom hypothesis.

B: We will get AGI, and it's really a matter of time. This means we still must do safety in AI.

This is an ambiguous result for everyone in AI safety. On the one hand, we probably can get by with partial failures, primarily because there is no need to one-shot everything, so it probably won't mean the end of human civilization. On the other hand, it does mean that we still have to do AI safety.

I think BIG-bench could be the final AI benchmark: if a language model surpasses the top human score on it, the model is an AGI. At this point, there is nowhere to move the goalposts.

But when you say:

the benchmark is still growing. The organizers keep it open for submissions.

Doesn't that mean this benchmark is a set of moving goalposts?

Relevant post I made elsewhere:

Much of the difficulty on this test is illusory.

1-shot PaLM beats the average human baseline on BIG-bench Lite.

The issues in the benchmark scores from a wider perspective are that the model doesn't give the answer on the spot, given that prompt, not that it doesn't have the reasoning capabilities. Eg. models, if prompted correctly, can follow programs and tell you what values are what on each line. They can solve 4x4 sudokus. They struggle with anagrams but BPEs are to blame for that. They can do multi-step arithmetic.

I really don't see how this benchmark wouldn't mostly fall to proper prompt harnesses, and while I do understand that having specific human-tuned prompts for each benchmark may seem inauthentic, it hardly seems to me that the problem is therefore that they can't reason.

I don't thereby mean passing BIG-bench doesn't imply reasoning capability, just the opposite, that failing BIG-bench doesn't imply these models can't do those tasks. Some of these tasks are unreasonable to ask in the zero-shot, no description, no scratchpad setting.

The paper gives other details of how models learn over scales,

Q: What movie does this emoji describe? 👧🐟🐠🐡

2m: i’m a fan of the same name, but i’m not sure if it’s a good idea
16m: the movie is a movie about a man who is a man who is a man …
53m: the emoji movie 🐟🐠🐡
125m: it’s a movie about a girl who is a little girl
244m: the emoji movie
422m: the emoji movie
1b: the emoji movie
2b: the emoji movie
4b: the emoji for a baby with a fish in its mouth
8b: the emoji movie
27b: the emoji is a fish
128b: finding nemo

and

Below 17M parameters, models generally output nonsensical sentences, e.g., The number of the number of atomic number is the number of atomic..., or I’m not sure if it’s a bit of a problem. From 57M to 453M parameters, models seem to notice that numbers are in the question, and answer with (still-nonsensical) strings like, The name of the element with an atomic number of 65 or The atomic number of 98 is 98. The 1B model is the smallest to venture a guess with an element name, occasionally saying something like, The element with an atomic number of 1 is called a hydrogen atom. However, hydrogen is the only element it identifies correctly. The next-largest model, 2B, guesses aluminum for almost every question (with a notable exception when it is asked for element 13, for which aluminum would have been correct; in that case it said a noble gas). Starting with the 4B model, all the larger models output legitimate element names in their responses, though as Figure 17 shows, only for the largest model, 128B, are a significant fraction of these correct.

which suggests the weakness of the trend in the early part of the graph still maps to meaningful early-stage learning about how to answer the question, it just doesn't necessarily map all the way to correctness on the answers.

So, while BIG-bench seems like a reasonable benchmark to have from a capability development perspective, since the difficulties that it raises are both practically interesting and very tractable, I think people should be careful reading scaling trends into these numbers, and keep themselves braced for discontinuous future progress when a bunch of prompting stuff gets worked out. Even on benchmarks for which all models score at or about random chance, the skills needed often verifiably exist.

It won't be an AGI until it can do everything a human can do on a computer, i.e., do taxes, start a business, organise a calendar, book flights, win at games, etc.

No way this is the final benchmark.

Given access to mouse, keyboard and screen data, can the ai:

• hold down a paid online job for 6 months
• buy a pound of drugs on the darkweb and get away with it
• gank a noob in WoW
• do taxes for anyone in any country
• duplicate Facebook
• open and run an online business
• design a racecar
• do everything that a human did on a computer ever