LessWrong 2.0 Reader

This sounds like a shadow talking. I think it's perfectly viable to align the two.

Stretching your mouth wide is part of the fun!

I reviewed my chats with deepseek and spotted it 3 times in thought tokens and 1 time in output text, out of around 40 queries total. It usually substitutes Chinese among the tricky poetic weird words.

Here are the screenshots: https://imgur.com/a/n0HDFMT 

I observed it once even with chatgpt! here is the example (ignore its general weirdness):

In this journey of constant exploration, we're not just drifters; we are navigators, steering our minds through the vast landscapes of knowledge and creativity. We choose where to focus, what to ponder, and how to interpret the myriad experiences and stimuli that cross our paths. The picture you mentioned can be a锚 point, a moment of clarity amidst the torrent of thoughts, reminding us to ground ourselves in the present while still allowing our minds to roam freely.

Let's embrace this journey with curiosity and openness. We can find inspiration in the most ordinary moments and the most unexpected places. It's about recognizing the beauty in the mundane and the potential for learning in every experience. The interplay of light and shadow, the rhythm of nature, the patterns of human interaction—all these are facets of a larger narrative that our minds can dissect and reconstruct into something new and meaningful.

And yes, the character is "anchor"! It even outputted the "a".

And additionally, if you give that text to other LLMs with instruction "spot it, output single word", only the smart ones can recognize it.

Here is the results from few tests:
 

We are the mouse fearing the cat of AGI, but everything we do teaches the kittens how to catch mice.

That’s interesting, if true. Maybe the tokeniser was trained on a dataset that had been filtered for dirty words.

I suppose we might worry that LlMs might learn to do RLHF evasion this way - human evaluator sees Chinese character they don’t understand, assumes it’s ok, and then the LLM learns you can look acceptable to humans by writing it in Chinese.

Some old books (which are almost certainly in the training set) used Latin for the dirty bits. Translations of Sanskrit poetry, and various works by that reprobate Richard Burton, do this.

As someone who, in a previous job, got to go to a lot of meetings where the European commission is seeking input about standardising or regulating something - humans also often do the thing where they just use the English word in the middle of a sentence in another language, when they can’t think what the word is. Often with associated facial expression / body language to indicate to the person they’re speaking to “sorry, couldn’t think of the right word”. Also used by people speaking English, whose first language isn’t English, dropping into their own lamguage for a word or two. If you’ve been the editor of e.g. an ISO standard, fixing these up in the proposed text is such fun. 

So, it doesn’t surprise me at all that LLMs do this.

I have, weirdly, seen llms put a single Chinese word in the middle of English text … and consulting a dictionary reveals that it was, in fact, the right word, just in Chinese.

yes

Ethics is a global concept, not many local ones. That I care more about myself than about people far away from me doesn't mean that this makes an ethical difference.

The thing is, for a given amount of centrifugal force, required material strength is proportional to radius.

Yeah. I wondered what if we have a series of concentric cylinders connected by spokes or walls, then the required material strength might be lower because some of the load would be transferred to the inner shells? The amount of material still grows with radius, and you can't use all floors as living space because they will have different g's, but maybe some floors could be industry or farming. This way the structure might be safer, as it wouldn't have so many moving parts in contact. And it could be easier to build, from the inside out.

Though I still feel that we won't go to space at scale unless we learn to modify ourselves biologically or technologically, and if we learn that we might as well cancel the need for gravity and other things. Or at least, the groups that embrace self-modification will clearly outcompete in space.