LessWrong 2.0 Reader

That’s interesting, if true. Maybe the tokeniser was trained on a dataset that had been filtered for dirty words.

I suppose we might worry that LlMs might learn to do RLHF evasion this way - human evaluator sees Chinese character they don’t understand, assumes it’s ok, and then the LLM learns you can look acceptable to humans by writing it in Chinese.

Some old books (which are almost certainly in the training set) used Latin for the dirty bits. Translations of Sanskrit poetry, and various works by that reprobate Richard Burton, do this.

As someone who, in a previous job, got to go to a lot of meetings where the European commission is seeking input about standardising or regulating something - humans also often do the thing where they just use the English word in the middle of a sentence in another language, when they can’t think what the word is. Often with associated facial expression / body language to indicate to the person they’re speaking to “sorry, couldn’t think of the right word”. Also used by people speaking English, whose first language isn’t English, dropping into their own lamguage for a word or two. If you’ve been the editor of e.g. an ISO standard, fixing these up in the proposed text is such fun. 

So, it doesn’t surprise me at all that LLMs do this.

I have, weirdly, seen llms put a single Chinese word in the middle of English text … and consulting a dictionary reveals that it was, in fact, the right word, just in Chinese.

yes

Ethics is a global concept, not many local ones. That I care more about myself than about people far away from me doesn't mean that this makes an ethical difference.

The thing is, for a given amount of centrifugal force, required material strength is proportional to radius.

Yeah. I wondered what if we have a series of concentric cylinders connected by spokes or walls, then the required material strength might be lower because some of the load would be transferred to the inner shells? The amount of material still grows with radius, and you can't use all floors as living space because they will have different g's, but maybe some floors could be industry or farming. This way the structure might be safer, as it wouldn't have so many moving parts in contact. And it could be easier to build, from the inside out.

Though I still feel that we won't go to space at scale unless we learn to modify ourselves biologically or technologically, and if we learn that we might as well cancel the need for gravity and other things. Or at least, the groups that embrace self-modification will clearly outcompete in space.

If there were no downsides to resolving a persistent issue, then why has it lasted so long??

If I understand correctly, your claim is that when we see long-standing issues like depression, chronic neck pain, or patterns of emotional avoidance persisting for years, it's more likely than not to be some sort of adaptive coping strategy—essentially a way the mind or body protects itself from harm–otherwise the issue would have been resolved. 

Why do you think this is more likely than a mundane explanations such as "bad luck in the genetic lottery, no obvious levers to pull"?

Is there a reason why every LLM tokenizer I've seen excludes slurs? It seems like a cheap way to train for AI assistant behavior. 

Also notable that numbers are tokenized individually - I assume this greatly improves its performance in basic arithmetic tasks as compared to GPTs.

Re: moral patienthood, I understand the Sam Harris position (paraphrased by him here as "Morality and values depend on the existence of conscious minds—and specifically on the fact that such minds can experience various forms of well-being and suffering in this universe.") as saying that anything else that supposedly matters, only matters because conscious minds care about it. Like, a painting has no more intrinsic value in the universe than any other random arrangement of atoms like a rock; its value stems purely from conscious minds caring about it. Same with concepts like beauty and virtue and biodiversity and anything else that's not directly about conscious minds.

And re: caring more about one's close circle: well, everyone in your close circle has their own close circle they care about, and if you repeat that exercise often enough, the vast majority of people in the world are in someone's close circle.

I remember reading about SAD treatment by lumens in Inadequate Equilibria, though I did not finish the book.