The surprising parameter efficiency of vision models

Ahh this is extremely simple and obvious to those with deep wisdom from computer graphics - the closer you get to low level physics cellular automata the more effective parameter sharing is. Conceptualize the space of all physics approximation functions organized by spatiotemporal scale. At the most detailed finest scale physics is uniform across space and time so a small amount of code/params describes all of spacetime. As you advance up the approximation abstraction tree to larger temporal spatial scales code complexity and specificity increases with language models sitting at the top.

At the lowest levels of vision all the features are spatially invariant and thus the natural connection matrix is highly compressible through simple weigh sharing, but diminishing exponentially as you advance up abstraction levels.

I suspect it is a combination of #3 and #5.

Regarding #5 first, I personally think that language models are being trained wrong. We'll get OoM improvements when we stop randomizing the examples we show to models during training, and instead provide examples in a structured curriculum.

This isn't a new thought, e.g. https://arxiv.org/abs/2101.10382

To be clear, I'm not saying that we must present easy examples first and then harder examples later. While that is what has been studied in the literature, I think we'd actually get better behavior by trying to order examples on a spectrum of "generalizes well" to "very specific, does not generalize" and then training in that order. Sometimes this might be equivalent to "easy examples first", but that isn't necessarily true.

I recognize that the definitions of "easy" and "generalizes" are nebulous, so I'm going to try and explain the reasoning that led me here.

Consider the architecture of transformers and feed-forward neural networks (specifically not recurrent neural networks). We're given some input, and we produce some output. In a model like GPT, we're auto-regressive, so as we produce our outputs, those outputs become part of the input during the next step. Each step is fundamentally a function $F\left(S_1\right)->S_2$.

Given some input, the total output can be thought as:

def reply_to(input):

    output = ""

    while True:

        token = predict_next(input + output)

        if token == STOP:

            break

        output += token

    return output

We'd like to know exactly what `predict_next` is doing, but unfortunately, the programmer who wrote it seems to have done their implementation entirely in matrix math and they didn't include any comments. In other words, it's deeply cursed and not terribly different from the output of Simulink's code generator.

def predict_next(input):

    # ... matrix math

    return output

Let's try to think about the capabilities and constraints on this function.

• There is no unbounded `loop` construct. The best we can do is approximate loops, e.g. by supporting an unrolled loop up to some bounded number of iterations. What determines the bounds? Probably the depth of the network?
• If the programmer were sufficiently deranged, they could implement `predict_next` in such a way that if they've hit the bottom of their unrolled loop, they could rely on the fact that `predict_next` will be called again, and continue their previous calculations during the next call. What would be the limitations on this? Probably the size of each hidden layer. If you wanted to figure out if this is happening, you'd want to look for prompts where the network can answer the prompt correctly if it is allowed to generate text before the answer (e.g. step-by-step explanations) but is unable to do so if asked to provide the answer without any associated explanations.
• How many total "instructions" can fit into this function? The size of the network seems like a decent guess. Unfortunately, the network conflates instructions and data, and the network must use all parameters available to it. This leads to trivial solutions where the network just over-fits to the data (analogous to baking in a lookup table on the stack). It's not unsurprising that throwing OoM more data at a fixed size NN results in better generalization. Once you're unable to cheat with over-fitting you must learn algorithms that work more efficiently.

The reason why I'm discussing the network in the language of instructions, stack space, and loops is because I disagree with a blanket statement like "scale is all you need". I think it's obvious that scaling the neural network is a patch on the first two constraints, and scaling the training data is a patch on the third constraint.

This is also why I think that point #3 is relevant. If GPT-3 does so well because it's using the sea of parameters for unrolled loops, then something like Stable Diffusion at 1/200th the size probably makes sense.

To tie this back to point #5:

• We start with a giant corpus of data. On the order of "all written content available in digital form". We might generate additional data in an automated fashion, or digitize books, or caption videos.
• We divide it into training data and test data.
• We train the network on random examples from the training data, and then verify on random examples from the test data. For simplicity, I'm glossing over various training techniques like masking data or connections between nodes.
• Then we fine-tune it, e.g with Q&A examples.
• And then generally we deploy it with some prompt engineering, e.g. prefixing queries with past transcript history, to fake a conversation.

At the end of this process, what do we have?

I want to emphasize that I do not think it is a "stochastic parrot". I think it is very obvious that the final system has internalized actual algorithms (or at least, pseudo-algorithms due to the limitation on loops) for various tasks, given the fact that the size of the data set is significantly larger than the size of the model. I think people who are surprised by the capabilities of these systems continue to assume it is "just" modeling likelihoods, when there was no actual requirement on that.

I also suspect we've wasted an enormous quantity of our parameters on embedding knowledge that does not directly contribute to system's capabilities.

My hypothesis for how to fix this is vaguely similar to the idea of "maximizing divergence" discussed here https://ljvmiranda921.github.io/notebook/2022/08/02/splits/.

I think we could train a LLM on a minimal corpus to "teach" a language^[1] and then place that LLM inside of a larger system that we train to minimize loss on examples teaching logic, mathematics, and other components of reasoning. That larger system would distinguish between the weights for the algorithms it learns and the weights representing embedded knowledge. It would also have the capability to loop during the generation of an output. For comparison, think of the experiments being done with hooking up GPT-4 to a vector database, but now do that inside of the architecture instead of as a hack on top of the text prompts.

I think an architecture that cleanly separates embedded knowledge ("facts", "beliefs", "shards", etc) from the algorithms ("capabilities", "zero-shot learning") is core to designing a neural network that remains interpretable and alignable at scale.

If you read the previous paragraphs and think, "that sounds familiar", it's probably because I'm describing how we teach humans: first language, then reasoning, then specialization. A curriculum. We need language first because we want to be able to show examples, explain, and correct mistakes. Especially since we can automate content generation with existing LLMs to create the training corpus in these steps. Then we want to teach reasoning, starting with the most general forms of reasoning, and working into the most specific. Finally, we grade the system (not train!) on a corpus of specific knowledge-based activities. Think of this step as describing the rules of a made-up game, providing the current game state, and then asking for the optimal move. Except that for games, for poems, for math, for wood working, for engineering, etc. The whole point of general intelligence is that you can reason from first principles, so that's what we need to be grading the network on: minimizing loss with respect to arbitrarily many knowledge-based tasks that must be solved using the facts provided only during the test itself.

1. ^{^}

   Is English the right language to teach? I think it would be funny if a constructed language actually found a use here.

A thing that stands out more starkly to me than the number of parameters is that GPT-3 is able to produce human-like text by passing a merely 12K-dimensional vector along its residual stream, through merely 96 successive steps of elementary transformations (even if each transformation is coded by lots of parameters). No features that require more contemplation than that can be computed during inference, deliberative thought that's not written out in tokens needs to fit in there. The whole context needs to be comprehended within that limit, from the first word to the last one.

Perhaps GPT-4's advantage is in having enough layers to also think a bit more about what's going on and carry out more complicated plans [LW · GW] without thinking out loud in tokens, capturing in the middle layers the features that human brain would need multiple recurrent passes to formulate. But the low dimension is still shocking, it wasn't obvious with older NLP work that this order of amount of data is enough to capture human-level cognition, since 1K-dimensional embeddings are also good for indexing/search and it wasn't clear how far up the nuance would need to go.

4.) Our assessment of LLM abilities is wrong and existing LLMs are just vastly superhuman and GPT-2 style models are actually at human parity. This seems strongly unlikely from actually interacting with these models, but on the other hand, even GPT-2 models possess a lot of arcane knowledge which is superhuman and it may be that the very powerful cognition of these small models is just smeared across such a wide range of weird internet data that it appears much weaker than us in any specific facet. Intuitively, this would be that a human and GPT-2 possess the same 'cognitive/linguistic power' but that since GPT-2's cognition is spread over a much wider data range than a human, it's 'linguistic power density' is lower and therefore appears much less intelligent in the much smaller human-relevant domain in which we test it. I am highly unclear whether these concepts are actually correct or a useful frame through which to view things.

I think LLMs are great and plausibly superhuman at language, it's just that we don't want them to do language, we want them to do useful real-world tasks, and hijacking a language model to do useful real-world tasks is hilariously inefficient.

If you consider pure language tasks like "Here's some information in format X, please reshuffle it to the equivalent in format Y", then GPT-4 seems vastly superhuman. (I'm somewhat abusing terms here since the "language" task of reshuffling information is somewhat different than the "language" task of autoregressively predicting information, but I think they are probably way more closely related tasks than if you want to apply it to something useful? Idk.) Can't remember anything about how good GPT-2 was at this, not sure I even bothered to try it.

Our brains were not trained for image generation (much). They were trained for converting 2d image into the understanding of the situation. Which AI still struggles with and needs the help of LLMs to have anywhere good results.

On the other hand, the tens of billions of tokens fed to LLMs is orders of magnitude beyond what humans could ever experience.

Nitpick, but one of the LLaMA versions was trained on 1.4T tokens and GPT-4 probably had an even larger training dataset.

"Trillions of tokens" feels more accurate for SOTA language models.

I also think (4) is a big piece of this. GPT-4 isn't that good at programming, but it's shockingly good for someone with no long-term memory doing everything in their head with limited or no ability to self-reflect.

Try transforming the language task and the image task into the same format and comparing the two. It's easy to rasterize images so that they become transformed into strings of colored dots. For language, replace each token with a colored dot and do so uniformly across all texts. You have now transformed each text into a string of colored dots.

Now take, say, the first billion dots from your (transformed) image collection and the first billion dots from your (transformed) text collection. Which string has the higher entropy?

Thoughts:

1. Strongly agree re the unfair comparison
2. Don't know enough neuroscience or machine learning to have an opinion here
3. Same as above
4. Seems unlikely. Capabilities emerging at particular levels of scale (this phenomenon seems present even in GPT-3.5 -> GPT-4) makes me doubt that GPT-2 has any sort of human level linguistics competence. That said, the task that GPT performs when modelling language (next token prediction), is not the same task that humans perform, and it seems likely that GPT-2 was already superhuman at that task? Alternatively, performance on downstream tasks is not an "ecological evaluation" of GPT.
5. LLaMA demonstrates that it is possible to tradeoff training data size against model size and attain performance gains even with Chinchilla over trained models; this also seems to reduce the inference compute cost

Let's contrast this to the brain. Previously

The hyperlink for "Previously" is broken.

I am not sure how did you come to the conclusion that current models are superhuman. I can visualize complex scenes in 3D for example. Especially under some drugs :)

And I don't even think I have an especially good imagination. 

In general, it is very hard to compare mental imagery with Stable Diffusion. For example, it it is hard to imagine something with many different details in different parts of the image but it is perhaps a matter of representation. An analogy could be that our perception is like a low-resolution display. I can easily zoom in on any area and see the details.

I wouldn't say that current models are superhuman. Although I wouldn't claim humans are better either, it is just very unobvious how to compare it properly and probably there are a lot of potential pitfalls.

So 1) has a large role here. 

In 2) CNNs are not a great example (as you mentioned yourself). Vision transformers demonstrate similar performance. It seems that inductive bias is relatively easy to learn for neural networks. I would guess it's similar for human brains too although I don't know much about neurobiology.

3) Doesn't seem like a good reason to me. There are modern GANs that demonstrate similar performance to diffusion models, also there are approaches which make diffusion work in a very small number of steps, even 1 step showed decent results IIRC. Also, even ImageGPT worked pretty well back in the day.

4) Similarly to the initial claim, I don't think much can be confidently said about LLM language abilities in comparison to humans. I do not know what exactly it means and how to measure it. We can do benchmarks, yes. Do they tell us anything deep? I don't think so. LLMs are very different kinds of intelligence, they can do many things humans can't and vice versa.

But at the same time, I wouldn't say that visual models strike me as much more capable given the same size/same amount of compute. They are quite stupid. They can't count. They can't do simple compositionally.

5) It is possible we will have much more efficient language models, but again, I don't think they are much more inefficient than visual models.

My two main reasons for the perceived efficiency difference:

1. It is super hard to compare with humans. We may do it completely wrong. I think we should aspire to avoid it unless absolutely necessary.
2. "Language ability" depends much more on understanding and having a complicated world model compared to "visual ability". We are not terribly disappointed when Stable Diffusion consistently draws three zombies when we ask for four and mostly forgive it for weird four-fingered hands sometimes growing from the wrong places. But when LLMs do similar nonsense, it is much more evident and hurts performance a lot (both on benchmarks and in the real world). LLMs can imitate style well, they have decent grammar. Larger ones GPT-4 can even count decently well and probably do some reasoning. So the hard part (at least for our current deep learning methods) is the world model. Pattern matching is easy and not really important in the grand scheme of things. But it still looks kinda impressive when visual models do it.