rai's Shortform

Kernel of something that might inspire someone else who knows more than I.

Assuming weights that have “grokked” a task are more interpretable, is there use in modifying loss functions to increase grokking likelihood? Perhaps by making it path dependent on the updates of the weights themselves?