Inner Optimization Mechanisms in Neural Nets

post by ProgramCrafter (programcrafter) · 2024-05-12T17:52:52.803Z · LW · GW · 1 comments

Contents

  Actual subnetwork design for point #3
  Conclusion
None
1 comment

I believe that current architecture of neural networks supports mesa-optimization: generally speaking, searching across some vectors in order to select one of them, which will be most useful for producing an answer.

Three inner optimization ways are already possible, and most likely there will be new ones.

1. Optimization on scaffolding level.
   1. Iterative diffusion models (e.g. Stable Diffusion) are very close to this, modifying intermediate pseudo-image each pass.
   2. LLMs with scratchpad can write down some amount of text - including plans, for instance - then compare them to calculate output.
2. Custom "neuron" functions embedded into the neural network, which would perform search (for instance, generate arbitrary incoming data to subnetwork, infer output and select the best). To my knowledge, this way is not currently used in any public models (but name "Q*" is pretty suspicious, reminding of "A*" - path searching algorithm in graph).
3. Option selection based on nonlinear activation functions - mostly, ReLU.
   If subnetwork has inputs $A$ and $B$, it's pretty easy to output $max(A;B)$. Additional information can be selected either by squeezing it into input numbers, or by building a slightly bigger subnetwork.

Actual subnetwork design for point #3

Let's suppose the neural network consists of layers, as common now - composition of matrix multiplication and activation function - and activation function $ReLU\left(x\right)=max(0;x)$.

1. A, B - inputs ($A\ge 0,B\ge 0$);
2. $C=max(0;A\ast 1+B\ast -1)$, $D=max(0;A\ast 0+B\ast 1)$
3. $E=max(0;C\ast 1+D\ast 1)$

$A<B⟶C=0,D=B⟶E=B=max(A;B)$

$A\ge B⟶C=A-B,D=B⟶E=A=max(A;B)$

This construction can be extended to select maximum out of $2^{k/2}$ options in k layers; possibly, even $2^{k-1}$ options.

Conclusion

I believe that inner optimization might exist in current neural networks, and that it can be used as evidence to approximate what future AIs can do at what levels of capability.

1 comments

Comments sorted by top scores.