Comment by Chloe Li (chloe-li-1) on Linear encoding of character-level information in GPT-J token embeddings · 2023-12-12T00:27:16.791Z · LW · GW

We show that linear probes can retrieve character-level information from embeddings and we perform interventional experiments to show that this information is used by the model to carry out character-level tasks.

These two links need permission to be accessed.