align your latent spaces

post by bhauth · 2023-12-24T16:30:09.138Z · LW · GW · 8 comments

This is a link post for https://www.bhauth.com/blog/thinking/align%20latents.html

Contents

8 comments

I learned very early the difference between knowing the name of something and knowing something.

— Richard Feynman

Here is a simplified diagram of the mind of someone reading English text and writing English text in response:

Here is a simplified diagram of the mind of someone fluent in English who is beginning to learn French and studying using flashcards with word pairs:

Red arrows indicate the main path of information from flashcards. Maybe you can see the reason I don't consider flashcards an optimal approach to learning languages. That approach can load word associations into a vector lookup system, but what's needed is encoder/decoder training. For that to happen with flashcards, the best-case scenario is training across the entire chain of two encoders to increase similarity at the concept latent space, and flashcards are inefficient for that because they provide a small amount of data without context. Anyone who understands neural networks knows that large amounts of in-context real data is better than repeatedly using small numbers of small fragments.

So, instead of using flashcards, I think it's better to read pairs of complete paragraphs with the same meaning, without repetition. For example, reading a book together with its translation. The foreign text not being understood as it's read doesn't mean that reading it is ineffective as an approach.

One reason flashcards are overrated is that a lot of language tests are for "vocabulary" because association between word pairs is easy to test.

 

Align your spine.

— Malcolm in the Middle

Once someone has become fluent in a 2nd language, they typically gain the ability to write and understand sentences using words from both languages. That implies integration of the encoders and decoders, which presumably happens via distillation, as in this diagram:

Considering that this happens with more practice and is seen in more-fluent people, it's probably better overall.

This integration of multiple systems for different languages seems to me to be fundamentally the same process as integration of understanding of multiple scientific fields. The equivalent of reading a book together with its translation for that would be reading a book that considers questions and problems from the perspective of one field and then from the perspective of a 2nd field. However, that's less available than translated literature, because there are fewer multidisciplinary experts than bilingual people, and multidisciplinary expertise is harder to evaluate and thus harder to find. Also, people capable of doing that would probably instead write a single version that integrates their understanding of both fields. That would be fine for learning from but possibly undesirable for students trying to pass a class or people trying to fit into an established culture of a field.

8 comments

Comments sorted by top scores.