Modern Transformers are AGI, and Human-Level

Well I’m one of the people who says that “AGI” is the scary thing that doesn’t exist yet (e.g. FAQ [LW · GW]  or “why I want to move the goalposts on ‘AGI’” [LW · GW]). I don’t think “AGI” is a perfect term for the scary thing that doesn’t exist yet, but my current take is that “AGI” is a less bad term compared to alternatives. (I was listing out some other options here [LW · GW].) In particular, I don’t think there’s any terminological option that is sufficiently widely-understood and unambiguous that I wouldn’t need to include a footnote or link explaining exactly what I mean. And if I’m going to do that anyway, doing that with “AGI” seems OK. But I’m open-minded to discussing other options if you (or anyone) have any.

Generative pre-training is AGI technology: it creates a model with mediocre competence at basically everything.

I disagree with that—as in “why I want to move the goalposts on ‘AGI’” [LW · GW], I think there’s an especially important category of capability that entails spending a whole lot of time working with a system / idea / domain, and getting to know it and understand it and manipulate it better and better over the course of time. Mathematicians do this with abstruse mathematical objects, but also trainee accountants do this with spreadsheets, and trainee car mechanics do this with car engines and pliers, and kids do this with toys, and gymnasts do this with their own bodies, etc. I propose that LLMs cannot do things in this category at human level, as of today—e.g. AutoGPT basically doesn’t work, last I heard. And this category of capability isn’t just a random cherrypicked task, but rather central to human capabilities, I claim. (See Section 3.1 here [LW · GW].)

Furthermore, when we measure that competence, it usually falls somewhere within the human range of performance. 

I think that for this to be meaningfully true, the LLM should be able to actually replace humans at a given task. There are some very specific domains in which this is doable (e.g. creative writing assistant), but it seems to me that they are still mostly too unreliable for this. 

I've worked with getting GPT-4 to act as a coach for business customers. This is one of the domains that it excels at - tasks can be done entirely inside a chat, the focus is on asking users questions and paraphrasing them so hallucinations are usually not a major issue. And yet it's stupid in some very frustrating ways that a human wouldn't be. 

For example, our users would talk with the bot at specific times, which they would schedule using a separate system. Sometimes they would ask the bot to change their scheduled time. The bot wasn't interfaced to the actual scheduling system, but it had been told to act like a helpful coach, so by default it would say something like "of course, I have moved your session time to X". This was bad, since the user would think the session had been moved, but it hadn't.

Well, easy to fix, right? Just add "if the user asks you to reschedule the session or do anything else that requires doing something outside the actual conversation, politely tell them that you are unable to do that" to the prompt. 

This did fix the problem... but it created a new one. Now the bot would start telling the user "oh and please remember that I cannot reschedule your session" as a random aside, when the user had never said anything about rescheduling the session. 

Okay, so what about adding something like "(but only tell this if the user brings it up, don't say it spontaneously)" to our prompt? That reduced the frequency of the spontaneous asides a little... but not enough to eliminate it. Eventually we just removed the whole thing from the prompt and decided that the occasional user getting a misleading response from the bot is better than it randomly bringing this up all the time.

Another basic instruction that you would think would be easy to follow would be "only ask one question at a time". We had a bit in a prompt that went like "Ask exactly one question. Do not ask more than question. Stop writing your answer once it contains a question mark." The end result? GPT-4 happily sending multi-question messages like "What is bothering you today? What kinds of feelings does that bring up?".

There are ways to fix these issues, like having another LLM instance check the first instance's messages and rewrite any that are bad. But at that point, it's back to fragile hand-engineering to get the kinds of results one wants, because the underlying thing is firmly below a human level of competence. I don't think LLMs are (or at least GPT-4 is not) yet at the kind of level of high reliability involved in human-level [LW · GW] performance.

I agree the term AGI is rough and might be more misleading than it's worth in some cases. But I do quite strongly disagree that current models are 'AGI' in the sense most people intend.

Examples of very important areas where 'average humans' plausibly do way better than current transformers:

• Most humans succeed in making money autonomously. Even if they might not come up with a great idea to quickly 10x $100 through entrepreneurship, they are able to find and execute jobs that people are willing to pay a lot of money for. And many of these jobs are digital and could in theory be done just as well by AIs. Certainly there is a ton of infrastructure built up around humans that help them accomplish this which doesn't really exist for AI systems yet, but if this situation was somehow equalized I would very strongly bet on the average human doing better than the average GPT-4-based agent. It seems clear to me that humans are just way more resourceful, agentic, able to learn and adapt etc. than current transformers are in key ways.
• Many humans currently do drastically better on the METR task suite (https://github.com/METR/public-tasks) than any AI agents, and I think this captures some important missing capabilities that I would expect an 'AGI' system to possess. This is complicated somewhat by the human subjects not being 'average' in many ways, e.g. we've mostly tried this with US tech professionals and the tasks include a lot of SWE, so most people would likely fail due to lack of coding experience.
• Take enough randomly sampled humans and set them up with the right incentives and they will form societies, invent incredibly technologies, build productive companies etc. whereas I don't think you'll get anything close to this with a bunch of GPT-4 copies at the moment

I think AGI for most people evokes something that would do as well as humans on real-world things like the above, not just something that does as well as humans on standardized tests.

↑ comment by abramdemski · 2024-03-26T22:00:40.220Z · LW(p) · GW(p)

With respect to METR, yeah, this feels like it falls under my argument against comparing performance against human experts when assessing whether AI is "human-level". This is not to deny the claim that these tasks may shine a light on fundamentally missing capabilities; as I said, I am not claiming that modern AI is within human range on all human capabilities, only enough that I think "human level" is a sensible label to apply.

However, the point about autonomously making money feels more hard-hitting, and has been repeated by a few other commenters. I can at least concede that this is a very sensible definition of AGI, which pretty clearly has not yet been satisfied. Possibly I should reconsider my position further.

The point about forming societies seems less clear. Productive labor in the current economy is in some ways much more complex and harder to navigate than it would be in a new society built from scratch. The Generative Agents paper gives some evidence in favor of LLM-base agents coordinating social events.

Replies from: michael-chen, None

↑ comment by mic (michael-chen) · 2024-03-29T02:17:19.908Z · LW(p) · GW(p)

I think humans doing METR's tasks are more like "expert-level" rather than average/"human-level". But current LLM agents are also far below human performance on tasks that don't require any special expertise.

From GAIA:

GAIA proposes real-world questions that require a set of fundamental abilities such as reasoning, multi-modality handling, web browsing, and generally tool-use proficiency. GAIA questions are conceptually simple for humans yet challenging for most advanced AIs: we show that human respondents obtain 92% vs. 15% for GPT-4 equipped with plugins. [Note: The latest highest AI agent score is now 39%.] This notable performance disparity contrasts with the recent trend of LLMs outperforming humans on tasks requiring professional skills in e.g. law or chemistry. GAIA's philosophy departs from the current trend in AI benchmarks suggesting to target tasks that are ever more difficult for humans. We posit that the advent of Artificial General Intelligence (AGI) hinges on a system's capability to exhibit similar robustness as the average human does on such questions.

And LLMs and VLLMs seriously underperform humans in VisualWebArena, which tests for simple web-browsing capabilities:

I don't know if being able to autonomously make money should be a necessary condition to qualify as AGI. But I would feel uncomfortable calling a system AGI if it can't match human performance at simple agent tasks.

Replies from: nathan-helm-burger

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-03-30T20:01:54.797Z · LW(p) · GW(p)

I think METR is aiming for expert level tasks, but I think their current task set is closer in difficulty to GAIA and VisualWebArena than what I would consider human expert level difficulty. It's tricky to decide though, since LLMs circa 2024 seem really good at some stuff that is quite hard to humans, and bad at a set of stuff easy to humans. If the stuff they are currently bad at gets brought up to human level, without a decrease in skill at the stuff LLMs are above-human at, the result would be a system well into the superhuman range. So where we draw the line for human level necessarily involves a tricky value-weighting problem of the various skills involved.

↑ comment by [deleted] · 2024-03-27T01:46:41.129Z · LW(p) · GW(p)

However, the point about autonomously making money feels more hard-hitting, and has been repeated by a few other commenters. I can at least concede that this is a very sensible definition of AGI, which pretty clearly has not yet been satisfied. Possibly I should reconsider my position further.

This is what jumped out at me when I read your post. Transformer LLM can be described as a "disabled human who is blind to motion and needs seconds to see a still image, paralyzed, costs expensive resources to live, cannot learn, and has no long term memory". Oh and they finished high school and some college across all majors.

"What job can they do and how much will you pay". "Can they support themselves financially?".

And you end up with "well for most of human history, a human with those disabilities would be a net drain on their tribe. Sometimes they were abandoned to die as a consequence. "

And it implies something like "can perform robot manipulation and wash dishes, or the "make a cup of coffee in a strangers house" test. And reliably enough to be paid minimum wage or at least some money under the table to do a task like this.

We really could be 3-5 years from that, if all you need for AGI is "video perception, online learning, long term memory, and 5-25th percentile human like robotics control". 3/4 elements exist in someone's lab right now, the robotics control maybe not.

This "economic viability test" has an interesting followup question. It's possible for a human to remain alive and living in a car or tent under a bridge for a few dollars an hour. This is the "minimum income to survive" for a human. But a robotic system may blow a $10,000 part every 1000 hours, or need $100 an hour of rented B200 compute to think with.

So the minimum hourly rate could be higher. I think maybe we should use the human dollar figures for this "can survive" level of AGI capabilities test, since robotic and compute costs are so easy and fast to optimize.

Summary :

AGI when the AI systems can do a variety of general tasks, completely, you would pay a human employee to do, even a low end one.

Transformative AGI (one of many thresholds) when the AI system can do a task and be paid more than the hourly cost of compute + robotic hourly costs.

Note "transformation" is reached when the lowest threshold is reached. Noticed that error all over, lots of people like Daniel and Richard have thresholds where AI will definitely be transformational, such as "can autonomously perform ai research" but don't seem to think "can wash dishes or sort garbage and produce more value than operating cost" is transformational.

Those events could be decades apart.

Replies from: abramdemski

↑ comment by abramdemski · 2024-03-28T17:56:36.349Z · LW(p) · GW(p)

And you end up with "well for most of human history, a human with those disabilities would be a net drain on their tribe. Sometimes they were abandoned to die as a consequence. "

And it implies something like "can perform robot manipulation and wash dishes, or the "make a cup of coffee in a strangers house" test. And reliably enough to be paid minimum wage or at least some money under the table to do a task like this.

The replace-human-labor test gets quite interesting and complex when we start to time-index it. Specifically, two time-indexes are needed: a 'baseline' time (when humans are doing all the relevant work) and a comparison time (where we check how much of the baseline economy has been automated).

Without looking anything up, I guess we could say that machines have already automated 90% of the economy, if we choose our baseline from somewhere before industrial farming equipment, and our comparison time somewhere after. But this is obviously not AGI.

A human who can do exactly what GPT4 can do is not economically viable in 2024, but might have been economically viable in 2020.

Replies from: None

↑ comment by [deleted] · 2024-03-28T21:06:24.445Z · LW(p) · GW(p)

Yes, I agree. Whenever I think of things like this I focus on how what matters in the sense of "when will agi be transformational" is the idea of criticality.

I have written on it earlier but the simple idea is that our human world changes rapidly when AI capabilities in some way lead to more AI capabilities at a fast rate.

Like this whole "is this AGI" thing is totally irrelevant, all that matters is criticality. You can imagine subhuman systems using AGI reaching criticality, and superhuman systems being needed. (Note ordinary humans do have criticality albeit with a doubling time of about 20 years)

There are many forms of criticality, and the first one unlocked that won't quench easily starts the singularity.

Examples:

Investment criticality: each AI demo leads to more investment than the total cost, including failures at other companies, to produce the demo. Quenches if investors run out of money or find a better investment sector.

Financial criticality: AI services delivered by AI bring in more than they cost in revenue, and each reinvestment effectively has a greater than 10 percent ROI. This quenches once further reinvestments in AI don't pay for themselves.

Partial self replication criticality. Robots can build most of the parts used in themselves, I use post 2020 automation. This quenches at the new equilibrium determined by the percent of automation.

Aka 90 percent automation makes each human worker left 10 times as productive so we quench at 10x number of robots possible if every worker on earth was building robots.

Full self replication criticality : this quenches when matter mineable in the solar system is all consumed and made into either more robots or waste piles.

AI research criticality: AI systems research and develop better AI systems. Quenches when you find the most powerful AI the underlying compute and data can support.

You may notice 2 are satisfied, one eoy 2022, one later 2023. So in that sense the Singularity began and will accelerate until it quenches, and it may very well quench on "all usable matter consumed".

Ironically this makes your central point correct. Llms are a revolution.

I believe that the important part of generality is the ability to handle new tasks. In particular, I disagree that transformers are actually as good at handling new tasks as humans are. My mental model is that modern transformers are not general tools, but rather an enormous Swiss army knife with billions of specific tools that compose together to only a limited extent. (I think human intelligence is also a Swiss army knife and not the One True Tool, but it has many fewer tools that are each more general and more compositional with the other tools.)

I think this is heavily confounded because the internet is so huge that it's actually quite hard to come up with things that are not already on the internet. Back when GPT-3 first came out, I used to believe that widening the distribution to cover every task ever was a legitimate way to solve the generality problem, but I no longer believe this. (I think in particular this would have overestimated the trajectory of AI in the past 4 years)

One way to see this is that the most interesting tasks are ones that nobody has ever done before. You can't just widen the distribution to include discovering the cure for cancer, or solving alignment. To do those things, you actually have to develop general cognitive tools that compose in interesting ways.

We spend a lot of time thinking about how human cognitive tools are flawed, which they certainly are compared to the true galaxy brain superintelligence. But while humans certainly don't generalize perfectly and there isn't a sharp line between "real reasoning" and "mere memorization", it's worth keeping in mind that we're literally pretrained on surviving in the wilderness and those cognitive tools can still adapt to pushing buttons on a keyboard to write code.

I think this effect is also visible on a day to day basis. When I learn something new - say, some unfamiliar new piece of math - I generally don't immediately fully internalize it. I can recall some words to describe it and maybe apply it in some very straightforward cases where it obviously pattern matches, but I don't really fully grok its implications and connections to other knowledge. Then, after simmering on it for a while, and using it to bump into reality a bunch, I slowly begin to actually fully internalize the core intuition, at which point I can start generating new connections and apply it in unusual ways.

(From the inside, the latter feels like fully understanding the concept. I think this is at least partly the underlying reason why lots of ML skeptics say that models "don't really understand" - the models do a lot of pattern matching things straightforwardly.)

To be clear, I agree with your argument that there is substantial overlap between the most understanding language models and the least understanding humans. But I think this is mostly not the question that matters for thinking about AI that can kill everyone (or prevent that).

I agree with virtually all of the high-level points in this post — the term "AGI" did not seem to usually initially refer to a system that was better than all human experts at absolutely everything, transformers are not a narrow technology, and current frontier models can meaningfully be called "AGI".

Indeed, my own attempt to define AGI a few years ago was initially criticized for being too strong, as I initially specified a difficult construction task, which was later weakened to being able to "satisfactorily assemble a (or the equivalent of a) circa-2021 Ferrari 312 T4 1:8 scale automobile model" in response to pushback. These days the opposite criticism is generally given: that my definition is too weak.

However, I do think there is a meaningful sense in which current frontier AIs are not "AGI" in a way that does not require goalpost shifting. Various economically-minded people have provided definitions for AGI that were essentially "can the system perform most human jobs?" And as far as I can tell, this definition has held up remarkably well.

For example, Tobias Baumann wrote in 2018,

A commonly used reference point is the attainment of “human-level” general intelligence (also called AGI, artificial general intelligence), which is defined as the ability to successfully perform any intellectual task that a human is capable of. The reference point for the end of the transition is the attainment of superintelligence – being vastly superior to humans at any intellectual task – and the “decisive strategic advantage” (DSA) that ensues.¹ The question, then, is how long it takes to get from human-level intelligence to superintelligence.

I find this definition problematic. The framing suggests that there will be a point in time when machine intelligence can meaningfully be called “human-level”. But I expect artificial intelligence to differ radically from human intelligence in many ways. In particular, the distribution of strengths and weaknesses over different domains or different types of reasoning is and will likely be different² – just as machines are currently superhuman at chess and Go, but tend to lack “common sense”. AI systems may also diverge from biological minds in terms of speed, communication bandwidth, reliability, the possibility to create arbitrary numbers of copies, and entanglement with existing systems.

Unless we have reason to expect a much higher degree of convergence between human and artificial intelligence in the future, this implies that at the point where AI systems are at least on par with humans at any intellectual task, they actually vastly surpass humans in most domains (and have just fixed their worst weakness). So, in this view, “human-level AI” marks the end of the transition to powerful AI rather than its beginning.

As an alternative, I suggest that we consider the fraction of global economic activity that can be attributed to (autonomous) AI systems.³ Now, we can use reference points of the form “AI systems contribute X% of the global economy”. (We could also look at the fraction of resources that’s controlled by AI, but I think this is sufficiently similar to collapse both into a single dimension. There’s always a tradeoff between precision and simplicity in how we think about AI scenarios.)

I'm saying "transformers" every time I am tempted to write "LLMs" because many modern LLMs also do image processing, so the term "LLM" is not quite right.

"Transformer"'s not quite right either because you can train a transformer on a narrow task. How about foundation model: "models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks".

I think this mostly just reveals that "AGI" and "human-level" are bad terms.

Under your proposed usage, modern transformers are (IMO) brutally non-central [LW · GW] with respect to the terms "AGI" and "human-level" from the perspective of most people.

Unfortunately, I don't think there is any defintion of "AGI" and "human-level" which:

• Corresponds to the words used.
• Also is central from the perspective of most people hearing the words

I prefer the term "transformative AI", ideally paired with a definition.

(E.g. in The case for ensuring that powerful AIs are controlled, we use the terms "transformatively useful AI" and "early tranformatively useful AI" [LW · GW] both of which we define. We were initially planning on some term like "human-level", but we ran into a bunch of issues with using this term due to wanting a more precise concept and thus instead used a concept like not-wildly-qualitatively-superhuman-in-dangerous-domains [LW · GW] or non-wildly-qualitatively-superhuman-in-general-relevant-capabilities.)

I should probably taboo human-level more than I currently do, this term is problematic.

I agree 100%. It would be interesting to explore how the term "AGI" has evolved, maybe starting with Goertzel and Pennachin 2007 who define it as:

a software program that can solve a variety of complex problems in a variety of different domains, and that controls itself autonomously, with its own thoughts, worries, feelings, strengths, weaknesses and predispositions

On the other hand, Stuart Russell testified that AGI means

machines that match or exceed human capabilities in every relevant dimension

so the experts seem to disagree. (On the other hand, Stuart & Russell's textbook cite Goertzel and Pennachin 2007 when mentioning AGI. Confusing.)

In any case, I think it's right to say that today's best language models are AGIs for any of these reasons:

• They're not narrow AIs.
• They satisfy the important parts of Goertzel and Pennachin's definition.
• The tasks they can perform are not limited to a "bounded" domain.

In fact, GPT-2 is an AGI.

Cf. DeepMind's "Levels of AGI" paper (https://arxiv.org/abs/2311.02462), calling modern transformers "emerging AGI" there, but also defining "expert", "virtuoso", and "superhuman" AGI.

I agree. GPT-4 is an AGI for the kinds of tasks I care about such as programming and writing. ChatGPT4 in its current form (with the ability to write and execute code) seems to be at the expert human level in many technical and quantitative subjects such as statistics and programming.

For example, last year I was amazed when I gave ChatGPT4 one of my statistics past exam papers and it got all the questions right except for one which involved interpreting an image of a linear regression graph. The questions typically involve understanding the question, thinking of an appropriate statistical method, and doing calculations to find the right answer. Here's an example question:

Times (in minutes) for a sample of 8 players are presented in Table 1 below. Using an appropriate test at the 5% significance level, investigate whether there is evidence of a decrease in the players’ mean 5k time after the six weeks of training. State clearly your assumptions and conclusions, and report a p-value for your test statistic.

The solution to this question is a paired sample t-test.

Sure, GPT-4 has probably seen similar questions before but so do students since they can practice past papers.

This year, one of my professors designed his optimization assignment to be ChatGPT-proof but I found that it could still solve five out of six questions successfully. The questions involved converting natural language descriptions of optimization problems into mathematical formulations and solving them with a program.

One of the few times I've seen GPT-4 genuinely struggle to do a task is when I asked it to solve a variant of the Zebra Puzzle which is a challenging logical reasoning puzzle that involves updating a table based on limited information and using logical reasoning and a process of elimination to find the correct answer.

Perhaps AGI but not human level. A system that cannot drive a car or cook a meal is not human level. I suppose it's conceivable that the purely cognitive functions are at human level, but considering the limited economic impact I seriously doubt it. 

Maybe a better question than "time to AGI" is time to mundanely transformative AGI. I think a lot of people have a model of the near future in which a lot of current knowledge work (and other work) is fully or almost-fully automated, but at least as of right this moment, that hasn't actually happened yet (despite all the hype).

For example, one of the things current A(G)Is are supposedly strongest at is writing code, but I would still rather hire a (good) junior software developer than rely on currently available AI products for just about any real programming task, and it's not a particularly close call. I do think there's a pretty high likelihood that this will change imminently as products like Devin improve and get more widely deployed, but it seems worth noting (and finding a term for) the fact that this kind of automation so far (mostly) hasn't actually happened yet, aside from certain customer support and copyediting jobs.

I think when someone asks "what is your time to AGI", they're usually asking about when you expect either (a) AI to radically transform the economy and potentially usher in a golden age of prosperity and post-scarcity or (b) the world to end.

And maybe I am misremembering history or confused about what you are referring to, but in my mind, the promise of the "AGI community" has always been (implicitly or explicitly) that if you call something "human-level AGI", it should be able to get you to (a), or at least have a bigger economic and societal impact than currently-deployed AI systems have actually had so far. (Rightly or wrongly, the ballooning stock prices of AI and semiconductor companies seem to be mostly an expectation of earnings and impact from in-development and future products, rather than expected future revenues from wider rollout of any existing products in their current form.)

I agree it is not sensible to make "AGI" a synonym for superintelligence (ASI) or the like. But your approach to compare it to human intelligence seems unprincipled as well.

In terms of architecture, there is likely no fundamental difference between humans and dogs. Humans are probably just a lot smarter than dogs, but not significantly more general. Similar to how a larger LLM is smarter than a smaller one, but not more general. If you doubt this, imagine we had a dog-level robotic AI. Plausibly, we soon thereafter would also have human-level AI by growing its brain / parameter count. For all we know, our brain architectures seem quite similar.

I would go so far as to argue that most animals are about equally general [LW · GW]. Humans are more intelligent than other animals, but intelligence and generality seem orthogonal. All animals can do both inference and training in real time. They can do predictive coding. Yann LeCun calls it the dark matter of intelligence. Animals have a real world domain. They fully implement the general AI task "robotics".

Raw transformers don't really achieve that. They don't implement predictive coding, and they don't work in real time. LLMs may be in some sense more (e.g. in language understanding) intelligent than animals, but that was already true, in some sense, for the even more narrow AI AlphaGo. AGI signifies a high degree of generality, not necessarily a particularly high degree of intelligence in a less general (more narrow) system.

Edit: One argument for why predictive coding is so significant is that it can straightforwardly scale to superintelligence. Modern LLMs get their ability mainly from trying to predict human written text, even when they additionally process other modalities. Insofar text is a human artifact, this imposes a capability ceiling. Predictive coding instead tries to predict future sensory experiences. Sensory experiences causally reflect base reality quite directly, unlike text produced by humans. An agent with superhuman predictive coding would be able to predict the future, including conditioned on possible actions, much better than humans.

I found this an interesting but complex read for me -- both the post and the comments. I found a number of what seemed good points to consider, but I seem to be coming away from the discussion thinking about the old parable of the blind men and the elephant.

I very much agree with this. You're not the only one! I've been thinking for a while that actually, AGI is here (by all previous definitions of AGI). 

Furthermore, I want to suggest that the people who are saying we don't yet have AGI will in fact never be satisfied by what an AI does. The reason is this: An AI will never ever act like a human. By the time its ability to do basic human things like speak and drive are up to human standards (already happened), its abilities in other areas, like playing computer games and calculating, will far exceed ours. Moreover, AIs don't have desires that are anything like ours. So there will come a time when AIs can do all the things people do - but half the internet will still be saying, "But it doesn't take children to the park, because it doesn't have emotional intelligence, therefore it's still not real AGI." That is, because AI is not like us, there will always be some human activities that AI does not do; and there will always be people who claim that this means AI cannot do those things; and they will therefore suggest that AGI has not been achieved.

The much more interesting position right now is to recognise, as the OP does, that AGI is already here; and that AIs are still not very much like us; and to wonder what that means. The next gen AIs will be obviously much cmarter than us, and yet they still won't make money in pizza shops, as one commenter above suggested. I'll go out on a limb here, and say, no AI will ever open a pizza shop. And in fact, that's a stupid expectation to have of these fabulous aliens. It's nothing more or less than saying, X doesn't do what I would do, therefore X is wrong/not intelligent. It's the most parochial approach to a new thing that you could possibly take.

A less parochial approach is to ask: so, these alien beings are now among us. Rather than keep compliaining that they don't do what I do, can we ask: what do they do?

As an example of where that takes us: They're intelligent but they don't have intentions. Does that tell us that intentions are really at the heart of human consciousness, not intelligence? AIs are intelligent but they don't feel pain. Are they morally salient? If not, does that imply that people are morally salient not because we're smart, but because we hurt? Etc.

I've gotten push-back from almost everyone I've spoken with about this

I had also expected this reaction, and I always thought I was the only one who thinks we have basically achieved AGI since ~GPT-3. But looking at the upvotes on this post I wonder if this is a much more common view. 

I agree that "general intelligence" is a concept that already applies to modern LLMs, which are often quite capable across different domains. I definitely agree that LLMs are, in certain areas, already capable of matching or outperforming a (non-expert) human.

There is some value in talking about just that alone, I think. There seems to be a bias in play - preventing many from recognizing AI as capable. A lot of people are all too eager to dismiss AI capabilities - whether out of some belief in human exceptionalism, some degree of insecurity, some manner of "uncanny valley" response, something like "it seems too sci-fi to be true", or something else entirely.

But I don't agree that the systems we have are "human level", and I'm against using "AGI", which implies human or superhuman level of intelligence, to refer to systems like GPT-4.

Those AIs are very capable. But there are a few glaring, massive deficiencies that prevent them from being broadly "human level". Off the top of my head, they are deficient in:

• Long term memory
• Learning capabilities
• Goal-oriented behavior

I like the term "subhuman AGI" for systems like GPT-4 though. It's a concise way of removing the implication of "human-level" from "AGI", and refocusing on the "general intelligence" part of the term.

Thank you for writing this. I have been making the same argument [LW(p) · GW(p)] for about two years now, but you have argued the case better here than I could have. As you note in your edit it is possible for goal posts to be purposefully moved, but this irks me for a number of reasons beyond mere obstinacy:

1. The transition from narrow AI to truly general AI is socially transformative, and we are living through that transition right now. We should be having a conversation about this, but are being hindered from doing so because the very concept of Artificial General Intelligence has been co-opted.

2. The confusion originates I think from the belief by many people in the pre-GPT era that achieving general intelligence is all that is required to kick off the singularity. GPT demonstrates quite clearly that this belief is false. This doesn’t mean the foomers/doomers are wrong to be worried about AI, but it is a glaring hole in the standard arguments for their position, and should be talked about more, but confusion over terminology is preventing that from happening.

3. Moving goalposts to define AGI as radically transformative and/or superhuman capabilities is begging the question. To say that we haven’t achieved AGI because modern AI hasn’t literally taken over the world and/or killed all humans is to assume that unaligned AI would necessarily lead to such outcomes. Pre-2017 AI x-risk people did routinely argue that even a middling-level artificial general intelligence would be able to enter a recursive self-improvement cycle and reach superhuman capabilities in short order. Although I have no insider info, I believe this line of thinking is what led to EY’s public meltdown a year or so ago. I disagree with him, but I respect that he took his line of thinking to its logical conclusion and accepted the consequences. Most of the rationalist community has not updated on the evidence of GPT being AGI as EY has, and I think this goalpost moving has a lot to do with that. Be intellectually honest!

The AI x-risk community claimed that the sky was falling, that the development of AGI would end the human race. Well, we’re now 2-7 years out from the birth of AGI (depending on which milestone you choose), and SkyNet scenarios seem no closer to fruition. If the x-risk community wants to be taken seriously, they need to confront this contradiction head-on and not just shift definitions to avoid hard questions.

Obvious bait is obvious bait, but here goes.

Transformers are not AGI because they will never be able to "figure something out" the way humans can.

If a human is given the rules for Sudoku, they first try filling in the square randomly.  After a while, they notice that certain things work and certain things don't work.  They begin to define heuristics for things that work (for example, if all but one number appears in the same row or column as a box, that number goes in the box).  Eventually they work out a complete algorithm for solving Sudoku.

A transformer will never do this (pretending Sudoku wasn't in its training data).  Because they are next-token predictors, they are fundamentally incapable of reasoning about things not in their training set.  They are incapable of "noticing when they made a mistake" and then backtracking they way a human would.

Now it's entirely possible that a very small wrapper around a Transformer could solve Sudoku.  You could have the transformer suggest moves and then add a reasoning/planning layer around it to handle the back-tracking.  This is effectively what Alpha-Geometry does.

But a Transformer BY ITSELF will never be AGI.

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year. Will this post make the top fifty?