Paper: Transformers learn in-context by gradient descent

That definition of "optimizer" requires

some objective function that is explicitly represented within the system

but that is not the case here.

There is a fundamental difference between

1. Programs that implement the computation of taking the derivative.  ($f\to f^{\prime }$, or perhaps $f,x\to f^{\prime }\left(x\right)$.)
2. Programs that implement some particular function g, which happens to be the derivative of some other function.  ($x\to g\left(x\right)$, where it so happens that $g=F^{\prime }$ for some $F$.)

The transformers in this paper are programs of the 2nd type.  They don't contain any logic about taking the gradient of an arbitrary function, and one couldn't "retarget" them toward $L_1$ loss or some other function.

(One could probably construct similar layers that implement the gradient step for $L_1$, but they'd again be programs of the 2nd type, just with a different hardcoded $g$.)

Calling something like this an optimizer strikes me as vacuous: if you don't require the ability to adapt to a change of objective function, you can always take any program and say it's "optimizing" some function.  Just pick a function that's maximal when you do whatever it is that the program does.

It's not vacuous to say that the transformers in the paper "implement gradient descent," as long as one means they  "implement [gradient descent on $L_2$ loss]" rather than "implement [gradient descent] on [$L_2$ loss]."  They don't implement general gradient descent, but happen to coincide with the gradient step for $L_2$ loss.

If in-content learning in real transformers involves figuring out the objective function from the context, then this result cannot explain it.  If we assume some fixed objective function (perhaps LM loss itself?) and ask whether the model might be doing gradient steps on this function internally, then these results are relevant.

I am aware of the definition of a mesa-optimizer. I'm just saying that this definition doesn't catch much of the danger or exciting stuff, since it's e.g. not so important if the model contains something that optimizes over e.g. $L\left(x\right)=(x^2-2{)}^2$. The danger comes when the inner objective is more consequentialist than that.

I also think that it's pretty bad to claim that something is only an optimizer if it's a power-seeking consequentialist agent. For example, this would imply that the outer loop that produces neural network policies (by gradient descent on network parameters) is not an optimizer!

If the outer loop isn't connected to some mechanism that evaluates the consequences of different policies in the real world, then you are probably training something that mimics the prespecified training data rather than searching for novel powerful policies. This isn't useless - the explosive growth of statistical models for various purposes proves so much - but it is unlikely to be dangerous unless coupled with some other process that handles the optimization.

So I'm not saying these things aren't optimizers, but much of the worry about mesa-optimizers is about consequentialist optimizers, not generic optimizers. Most functions are perfectly safe to optimize.