LessWrong 2.0 Reader

Thanks!

Opinions about putting in a clause like "you may not use this for ML engineering" (assuming it would work legally) (plus putting in naive technical measures to make the tool very bad for ML engineering) ?

This sounds like a shadow talking. I think it's perfectly viable to align the two.

Stretching your mouth wide is part of the fun!

I reviewed my chats with deepseek and spotted it 3 times in thought tokens and 1 time in output text, out of around 40 queries total. It usually substitutes Chinese among the tricky poetic weird words.

Here are the screenshots: https://imgur.com/a/n0HDFMT 

I observed it once even with chatgpt! here is the example (ignore its general weirdness):

In this journey of constant exploration, we're not just drifters; we are navigators, steering our minds through the vast landscapes of knowledge and creativity. We choose where to focus, what to ponder, and how to interpret the myriad experiences and stimuli that cross our paths. The picture you mentioned can be a锚 point, a moment of clarity amidst the torrent of thoughts, reminding us to ground ourselves in the present while still allowing our minds to roam freely.

Let's embrace this journey with curiosity and openness. We can find inspiration in the most ordinary moments and the most unexpected places. It's about recognizing the beauty in the mundane and the potential for learning in every experience. The interplay of light and shadow, the rhythm of nature, the patterns of human interaction—all these are facets of a larger narrative that our minds can dissect and reconstruct into something new and meaningful.

And yes, the character is "anchor"! It even outputted the "a".

And additionally, if you give that text to other LLMs with instruction "spot it, output single word", only the smart ones can recognize it.

Here is the results from few tests:
 

We are the mouse fearing the cat of AGI, but everything we do teaches the kittens how to catch mice.

That’s interesting, if true. Maybe the tokeniser was trained on a dataset that had been filtered for dirty words.

I suppose we might worry that LlMs might learn to do RLHF evasion this way - human evaluator sees Chinese character they don’t understand, assumes it’s ok, and then the LLM learns you can look acceptable to humans by writing it in Chinese.

Some old books (which are almost certainly in the training set) used Latin for the dirty bits. Translations of Sanskrit poetry, and various works by that reprobate Richard Burton, do this.

As someone who, in a previous job, got to go to a lot of meetings where the European commission is seeking input about standardising or regulating something - humans also often do the thing where they just use the English word in the middle of a sentence in another language, when they can’t think what the word is. Often with associated facial expression / body language to indicate to the person they’re speaking to “sorry, couldn’t think of the right word”. Also used by people speaking English, whose first language isn’t English, dropping into their own lamguage for a word or two. If you’ve been the editor of e.g. an ISO standard, fixing these up in the proposed text is such fun. 

So, it doesn’t surprise me at all that LLMs do this.

I have, weirdly, seen llms put a single Chinese word in the middle of English text … and consulting a dictionary reveals that it was, in fact, the right word, just in Chinese.

yes

Ethics is a global concept, not many local ones. That I care more about myself than about people far away from me doesn't mean that this makes an ethical difference.