On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback 2024-11-07T15:39:06.854Z

Comment by Adhyyan Narang (adhyyan-narang) on Latent Variables and Model Mis-Specification · 2022-06-27T23:01:32.650Z · LW · GW

Hi Jacob, I really enjoyed this post thank you! 

 

However, it appears that the problem you describe of non-robust predictive performance can also take place under a well-specified model with insufficient data. For instance, my recent paper https://arxiv.org/abs/2109.13215 presents a toy example where a well-specified overparameterized interpolator may perform well on a classification task but poorly when the data is allowed to be adversarially perturbed. 

 

Then, it appears to me that the problem of incorrectly identifying latents is not a consequence of misspecification. But more a consequence of the limitation of the data. Either the data is not plentiful enough (which would cause problems in even a well-specified model) or the data is plentiful but not rich enough to identify latents (which would happen only in a misspecified model).