Deepmind's Gato: Generalist Agent

Given your Prediction 2, it seems like maybe we are on the same page? You seem to be saying that a 1000x Gato would be AGI-except-limited-by-scale-of-training, so if we could just train it for a sufficiently long scale that it could learn to do lots of AI R&D, then we'd get full AGI shortly thereafter, and if we could train it for a sufficiently long scale that it could learn to strategically accumulate power and steer the world away from human control and towards something else, then we'd (potentially) cross an AI-induced-point-of-no-return shortly thereafter. This is about what I think. (I also think that merely 10x or even 100x probably wouldn't be enough; 1000x is maybe my median.) What scale of training is sufficiently long? Well, that's a huge open question IMO. I think probably 1000x the scale of current-Gato would be enough, but I'm very unsure.

That's the one, thanks!

it did update me some that we really don't need much of any fundamental advancement to build very big agents that learn on multimodal data

I've been saying this for some time now. (I often feel like I'm more confident in the bio anchors approach to timelines than Open Phil is, because I'm more willing to say "yes we literally could scale up 2020 algorithms and get TAI, given some engineering effort and enough good data, without any fundamental advances".)

My explanation for the negative transfer in ALE is that ALE isn't sufficiently diverse / randomized; you can see this in CoinRun (see "the diversity hypothesis" in Understanding RL vision), where you only get interpretable vision features for aspects of the environment that were randomized. In contrast, image classifiers trained on real world images have interpretable vision features at all layers except perhaps some of the later ones, and often lead to positive transfer on new tasks.

A big part of my model predicting what kind of transfer does and doesn't work in deep learning is figuring out to what extent I expect there to be large entangled variation in the features of the training data. If this variation is present, then I expect the neural network is forced to learn the real actual feature, and there isn't some other simpler program that happens to get it correct in only the training situations. If you have the real actual feature, then you're going to transfer better.

You usually don't get sufficient diversity with programmatically generated data, but you do get it with real-world data, because reality is laced together very tightly [LW · GW]. So I often expect transfer to be a lot harder with programmatically generated data (unless the transfer is only to things that "could have been" programmatically generated, as was the case in e.g. XLand).

(I was initially going to say I believed this in 2019, but looking back at my notes from the time, I wrote very related stuff but didn't actually write this particular thing. I've definitely been saying it in public talks since about the beginning of 2022. So I probably started believing this some time in between then.)

I agree that will happen eventually, and the more nuanced version of my position is the one I outlined in my comment [LW(p) · GW(p)] on CAIS:

Now I would say that there is some level of data, model capacity, and compute at which an end-to-end / monolithic approach outperforms a structured approach on the training distribution (this is related to but not the same as the bitter lesson). However, at low levels of these three, the structured approach will typically perform better. The required levels at which the end-to-end approach works better depends on the particular task, and increases with task difficulty.

Since we expect all three of these factors to grow over time, I then expect that there will be an expanding Pareto frontier where at any given point the most complex tasks are performed by structured approaches, but as time progresses these are replaced by end-to-end / monolithic systems (but at the same time new, even more complex tasks are found, that can be done in a structured way).

I think when we are first in the situation where AI systems are sufficiently competent to wrest control away from humanity if they wanted to, we would plausibly have robots that take in audiovisual input and can flexibly perform tasks that a human says to them (think of e.g. a household robot butler). So in that sense I agree that eventually we'll have agents that link together language, vision, and robotics.

The thing I'm not that interested in (from a "how scared should we be" or "timelines" perspective) is when you take a bunch of different tasks, shove them into a single "generic agent", and the resulting agent is worse on most of the tasks and isn't correspondingly better at some new task that none of the previous systems could do.

So if for example you could draw an arrow on an image showing what you wanted a robot to do, and the robot then did that, that would be a novel capability that couldn't be done by previous specialized systems (probably), and I'd be interested in that. It doesn't look like this agent does that.

On surprises:

• I definitely agree that your timelines should take into account "maybe there will be a surprise".
• "There can be surprises" cuts both ways; you can also see e.g. a surprise slowdown of scaling results.
• I also didn't expect double descent and grokking but it's worth noting that afaict those have had ~zero effects on SOTA capabilities so far.
• Regardless, the original question was about this particular result; this particular result was not surprising (given my very brief skim).

On catastrophic forgetting:

I agree that catastrophic forgetting is becoming less of an issue at larger scale but I already believed and expected that; it seemed like something like that had to be true for all of the previous big neural net results (OpenAI Five, AlphaStar, language models, etc) to be working as well as they were.

Sorry, I think that particular sentence of mine was poorly written (and got appropriate pushback [LW(p) · GW(p)] at the time). I still endorse my followup comment [LW(p) · GW(p)], which includes this clarification:

The thing I'm not that interested in (from a "how scared should we be" or "timelines" perspective) is when you take a bunch of different tasks, shove them into a single "generic agent", and the resulting agent is worse on most of the tasks and isn't correspondingly better at some new task that none of the previous systems could do.

In particular, my impression with Gato is that it was not showing much synergy. I agree that synergy is possible and likely to increase with additional scale (and I'm pretty sure I would have said so at the time, especially since I cited a different example of positive transfer).

(Note I haven't read the mixed-modal scaling laws paper in detail so I may be missing an important point about it.)

I don't really know what you mean by a "general" agent. Here are some properties that I would guess it has (caveating again that I haven't read the paper in detail), which may or may not be related to what you mean by "generality":

1. Given an input, it can tell which task it is supposed to do, and then do the relevant tasks.
2. Some of the tasks do benefit from the training done on other tasks ("positive transfer"), presumably because some of the basic building blocks of the needed programs are the same ("look at the token that was one place prior" is probably helpful for many tasks).
3. It has some neurons that are used in multiple different tasks (presumably).
4. It cannot learn new tasks particularly quickly ("few-shot learning"), except inasmuch as that could already be done with language models.
5. It does not do any "learning with frozen weights" (i.e. the sort of thing where you prompt a language model to define a new word, and then it can use that word later on, without any gradient descent), except inasmuch as the specialized models would also do that learning.
6. It is about as well-modeled as an expected utility maximizer as the specialized models would be.