Comment by Good Man (good-man) on A Mechanistic Interpretability Analysis of Grokking · 2022-10-13T17:15:44.537Z · LW · GW

I found your idea fascinating.  You have a good company too. Percy Liang's group just published a paper along this line of thought and showed transformer's effectiveness in learning "ML trainers":

What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

https://arxiv.org/abs/2208.01066