Comment by Osher Lerner (osher-lerner) on [AN #140]: Theoretical models that predict scaling laws · 2024-09-30T22:00:34.266Z · LW · GW

Oh hi! I linked your video in another comment without noticing this one. Great visual explanation!

Comment by Osher Lerner (osher-lerner) on [AN #140]: Theoretical models that predict scaling laws · 2024-09-30T21:50:10.749Z · LW · GW

Yes, that's precisely what I'm claiming!

Sorry if that wasn't clear.  As for how to establish that, I proposed an intuitive justification:

There is no mechanism fitting the model to the linear approximation of the data around the training points.

And an outline for a proof:

Take two problems which have the same value at the training points but with wildly different linear terms around them. A model perfectly fit to the training points would not be able to distinguish the two.

Let's walk through an example

1. Consider trying to fit a simple function, $f\left(x\right)=0$ . Let's collect a training dataset

$$D_{\text{train}}=\left[\right(x=0,y=0),(x=1,y=0),(x=2,y=0),\dots ]$$

• You optimize a perfect model on $D_{\text{train}}$ (e.g. a neural net with mapping $NN\left(x\right)=0$)
• Now let's study the scaling of error as you move away from training points. In the example, we achieved ${\nabla }_{x}L\left(x_{\text{train}}\right)=0$, since coincidentally $\forall xf\left(x\right)=NN\left(x\right)$

2. Consider a second example. Let's fit $f\left(x\right)=\sin \left(x/\pi \right)$. Again, we collect training data

$$D_{\text{train}}=\left[\right(x=0,y=0),(x=1,y=0),(x=2,y=0),\dots ]$$

• You optimize a perfect model on $D_{\text{train}}$ (using the same optimization procedure, we get a neural net with mapping $NN\left(x\right)=0$)
• Now, we see ${\nabla }_{x}L\left(x_{\text{train}}\right)\ne 0$ (we predict a flat line at $y=0$, and $L$ measures error from a sinusoid). You can notice this visually or analytically:

$$\text{derivation:}{\nabla }_{x}L\left(x_{\text{train}}\right)={\nabla }_x{\parallel f\left(x\right)-NN\left(x\right)\parallel }_{x=x_{\text{train}}}={{\nabla }_x\parallel f\left(x\right)-0\parallel }_{x=x_{\text{train}}}={\nabla }_x\parallel \sin \left(x/\pi \right){\parallel }_{x\in Z}\ne 0$$

The model is trained on $x_{\text{train}},f\left(x_{\text{train}}\right)$, and is independent of ${\nabla }_{x}f\left(x_{\text{train}}\right)$.  That means even if by happy accident our optimization procedure achieves ${\nabla }_{x}L\left(x_{\text{train}}\right)=0$, we can prove that it is not generally true by considering an identical training dataset with a different underlying function (and knowing our optimization must result in the same model)

On rereading your original argument:

since the loss is minimized, the gradient is also zero.

I think this is referring to ${\nabla }_{\theta }L\left(x_{\text{train}}\right)=0$, which is certainly true for a perfectly optimized model (or even just settled gradient descent). Maybe that's where the miscommunication is stemming from, since "gradient of loss" is being overloaded from discussion of optimization ${\nabla }_{\theta }L$, and discussion of Taylor-expanding $L$ around $x_{\text{train}}$ (which uses ${\nabla }_{x}L$)

Comment by Osher Lerner (osher-lerner) on [AN #140]: Theoretical models that predict scaling laws · 2024-09-24T08:12:47.814Z · LW · GW

Since we have perfectly fit the training data, at the training data point, the loss is zero; and since the loss is minimized, the gradient is also zero.

The linear term in this is not actually 0. There is no mechanism fitting the model to the linear approximation of the data around the training points. The model is only fit to the (0th order) value at the training points. 

To correctly state the above sentence:

• "perfectly fit the training data"$⟺$"the loss is 0 (at training points)" $\Rightarrow$ "(training) loss is minimized" $\nRightarrow$ "gradient (of test loss around a training point) is 0"

To prove this point:

• Take two problems which have the same value at the training points but with wildly different linear terms around them. A model perfectly fit to the training points would not be able to distinguish the two.

To see this visually:

• Check out this plot, which quotes the above sentence from this post as a footnote, but brilliantly visually demonstrates the contradiction: see how error increases linearly around training points (it's using interpolation between points, but the same point holds for a constant piecewise function)