Near term discussions need something smaller and more concrete than AGI

post by ryan_b · 2025-01-11T18:24:58.283Z · LW · GW · 0 comments

Contents

  Motivation
  What I Want
  Build the Tool
  Review
None
No comments

Motivation

I want a more concrete concept than AGI[1] to talk and write with. I want something more concrete because I am tired of the costs associated with how big, inferentially distant, and many-pathed the concept of AGI is, which makes conversation expensive. Accounting for the bigness, inferential distance, and many-pathed-ness is very important for dealing with AGI properly - I believe they are causal in the risks AGI poses - but they drive a very high communication burden. By this I mean things like distinguishing between different meanings, summarizing a bunch of previous discussions, etc. This burden does not seem to have reduced much if at all over time, and I don't expect it to in the near future.

I consider that while the burden does not reduce over time, it does become easier to carry, because I can retain quite a bit of history and context. Now, to my great satisfaction, AGI is a much broader conversation across the public at large. Here is where I feel the burden becomes telling: the public at large does not have that history and context. For context, the book Superintelligence was published in 2014. That means we have 10 years worth of history and context on top of that book.

The problem is worse when trying to talk about things in the present time: topics like AGI timelines, regular AI impact on employment, regular AI impact on the economy, etc. Currently people are discussing very short timelines, where by short I mean ranges from already here (in a weak form) to 3 years. This creates urgency; we do not have the time for many rounds of conversation while carrying the heavy burden talking in terms of AGI demands. It feels to me like if these timelines are wrong, the people who hold them won't be able to change their mind before feeling forced into making big decisions; and further if these timelines are close to correct, we as a group won't have time to make progress on immediate questions.

Independently, and especially in the near term case, I think there are important ways that speaking in terms of AGI directly is wrong: we are trying to connect things at or near the object level to something many levels of abstraction up, with no clear conceptual way to go up, nor to come back down. I also notice that near-term-impact kinds of questions are...not what AGI is about. The idea of AGI is about the idea of intelligence - the early goal was to find the most general rules governing intelligence in the universe and then the associated goal of alignment is about how to guide AGI under those rules. Conversations around practical consequences of AGI are dogged by the need to somehow relate the thing a person is talking about to intelligence. I therefore think that AGI is the wrong rhetorical tool in a lot of cases. I want a new one.

What I Want

So what do I want out of this new rhetorical tool? I want something that is fast (to say), cheap (to think about), and powerful enough. I want it to fit places with limited bandwidth: verbal conversations, the comment section of LessWrong, maybe even chat messages. I'd like to be able to do thought experiments with it, a la Maxwell's Demon. I also need to make sure it doesn't duplicate any of the complaints I have about reasoning using AGI.

First: I want the new tool to be a short inferential distance from talking about current or near future events, so we can use it to tackle questions about jobs/economy/short timelines/etc. I think it needs to be about what AI can actually do in the same way that AGI is about intelligence. This buys me what I think of as short lateral inference: jobs are about what people can do; the economy is a big pile of what everyone does; short timelines mean AIs will be able to do a lot soon; etc. I don't want any built-in steps to relate what I am talking about to intelligence (or any third concept) first.

Second: I want it to be fairly concrete, but I also want it to be at least close to the level of abstraction of the problems we are trying to talk about. This buys me what I think of as short vertical inference distance: I can go down a level of abstraction (one specific type of job, like software engineer) or up a level of abstraction (maybe a sector of the economy, like AI) and return without much additional work. To do this, I'll need to come up from underneath, which is to say start with a concrete real thing and abstract up from there. By contrast, I understand AGI as essentially being a top-down type of abstraction: it begins with the idea of a lawful universe (in the sense of physical laws); then drops down from the laws to the general rules governing intelligence; then comes down to AGI, which acts under those rules. This is why AGI is about intelligence; its conceptual function in discussion is to be a fulcrum between us and the rules governing intelligence. My ideal case starts from the object level, and then goes up just enough levels to be convenient for the stuff I am trying to talk about.

Third: I want it to be able to reconnect to the big-picture questions surrounding AGI. I think this cashes out as being able to say reasonable things about what intelligence means in the context of the tool. The standard for reasonable I am looking for is really just clear and directionally correct. This should let discussions throw out wrong things and establish the right direction for the conversation to go in without too much effort, and also without falling away into an independent bubble entirely severed from the source.

Build the Tool

The about-ness criteria is straightforward. There is a pre-existing idea of intelligence vs capabilities [LW · GW] in public discussions. I therefore come down firmly on the side of capabilities. In lieu of Artificial General Intelligence, I give you the General Capability Machine (GCM). Keeping to the idea of being about what AIs can actually do, the General Capability Machine is a generalization over existing AIs, so that we can say anything any existing AI can do, the GCM does also.

For the object level grounding, we can look at how we measure AI capabilities now, which is benchmarks. AI benchmarks are a set of concrete tasks, usually of the same kind. Therefore I choose the task as the object level thing to ground the General Capability Machine on. However, we clearly can't leave it there: operating solely on tasks is basically just tracking the excruciating details of every AI in the world, which is no more manageable than the AGI burden I complained about upfront.

That being said it also feels like going directly to capabilities is a uselessly large jump. In the AGI case, consider the ARC benchmark: it is a series of concrete tasks that are designed to be difficult for LLMs but easy for humans, and therefore doing well is an indicator of intelligence in the LLM. This doesn't do much for my rhetorical purposes; ARC makes a strong correlational argument and indicator of success, but doesn't feed back into a better way to talk about intelligence (directly, anyway). Therefore I need something a level above tasks, that summarizes the task information well enough to help talk about capabilities.

Returning to benchmarks: the idea is that if an AI does well on the tasks in the benchmark, then it will do well on all tasks of the same type. In the context of humans, we have a natural choice of abstraction[2] for this - skills. When a person can do well on a type of task, we call it a skill; so I'll do the same for when an AI does well on a type of task. Doing a sanity check: in neural nets this is like the problem of doing well on the training distribution, but failing to do well outside of it. In benchmarks, models will sometimes do well on the benchmark tasks, but then in production it is discovered they routinely fail at the same kind of task. We compare to a human who does well in the Intro to Algebra course, but then gets tripped up on later problems that do stuff like use a,b instead of x,y. We would say they did not learn the proper skill, only memorized the problems in the original course. This closely matches the other two relationships - using the term skill for the benchmarks case seems solid. But how do capabilities enter the picture?

Capabilities are the thing we want to match with human capabilities, or else against the world. At bottom, doing anything out in the world tends to need a sequence of tasks, usually of several different types. So any time I would ask something like "is AI capable of X," I want to:

If the General Capability Machine has that set of skills on the list, then it has the capability. I can say "AI is capable of X."

The final criteria is that I can connect this way of thinking back to Artificial General Intelligence. I bridge the gap here through the idea of learning. What modern AI does with learning cashes out as new skills in the list of the General Capability Machine. This makes talking about AGI straightforward at least some of the time. For example, the question “how fast is AGI approaching?” becomes “how fast is GCM adding skills?” From there I could get more specific about what AGI means in terms of GCM descriptions.

Review

Talking about near term things like short timelines and employment using the idea of AGI is probably doomed because it is too big to use correctly in conversation.

Now I have my General Capability Machine, which is grounded in tasks AIs can do. When AIs do well on tasks of the same type, I call that a skill. To describe the General Capability Machine I make a list of its skills.

To talk about capability I break it down into the tasks it requires, then list the skills the tasks imply. I compare the list of skills the capability demands to the General Capability Machine list, and say the machine has the capability if all those skills are present. Now when I want to think about near term things like timelines or the ability to do a job, I can think in more concrete terms of what AIs can and cannot do.

I feel like the General Capability Machine is the near term rhetorical complement of AGI. We'll see how it works in practice.

 

  1. ^

    Including similar framings like Artificial Super Intelligence (ASI), which distinguishes the generality of Artificial General Intelligence from the superhuman requirement, and Transformative AI (TAI) which focuses on the question of impact.

  2. ^

    I mean this in a way conceptually similar to Natural Abstractions:

    Our physical world abstracts well: for most systems, the information relevant “far away” from the system (in various senses) is much lower-dimensional than the system itself. These low-dimensional summaries are exactly the high-level abstract objects/concepts typically used by humans.

    These abstractions are “natural”: a wide variety of cognitive architectures will learn to use approximately the same high-level abstract objects/concepts to reason about the world.

    Except in my case we are using abstractions that apply to humans and also applying them to machines, so we can relate what humans can do to what machines can do.

0 comments

Comments sorted by top scores.