LessWrong 2.0 Reader

Pasek was a trans woman, Christian mentions in another comment having been in contact with Pasek 8 or 9 years ago, so... could it be the pronouns Pasek actually used back then?

I agree that from a functional perspective, we can interact with an LLM in the same way as we would another human. At the same time I’m pretty sure we used to have good reasons for maintaining a conceptual distinction.

I think of this through the lens of Daniel Dennett's intentional stance; it's a frame that we can adopt without making any claims about the fundamental nature of the LLM, one which has both upsides and downsides. I do think it's important to be careful to stay aware that that's what we're doing in order to avoid sloppy thinking.

Nate Soares' related framing [LW · GW] as a 'behaviorist sense' is also useful to me: 

If an AI causes some particular outcome across a wide array of starting setups and despite a wide variety of obstacles, then I'll say it "wants" that outcome “in the behaviorist sense”.

I can care about a genetically enhanced genius labrat line for their own sake, and be willing to cooperate with them on building a mutually beneficial world, because I've generally been raised and grown to care about other beings, but if the genius labrats attempted to control and permanently enslave me, it would certainly alter that dynamic for me.

"Over-represented relative to general population" and "only a few" can both be simultaneously true.

Like, if in the general population on average 1 in 1000 is trans, and among the rationalists it is 1 in 100, and you have 1000 rationalists living in the Bay Area... that still means only 10 trans rationalists you can talk to if you live in the Bay Area, one of them is Ziz. (The numbers are made up, just as an example.)

therefore longer contexts can elicit much richer classes of behaviour.

Up to and including Turing-completeness ('Ask, and it shall be given')

I think there's starting to be evidence that models are capable of something like actual introspection, notably 'Tell me about yourself: LLMs are aware of their learned behaviors' and (to a more debatable extent) 'Looking Inward: Language Models Can Learn About Themselves by Introspection'. That doesn't necessarily mean that it's what's happening here, but I think it means we should at least consider it possible.

I'm aware of high rates among LWers but it's still far from what we see among Zizians that we hear a lot about.

Ziz herself apparently believed that trans women were inherently more capable of accepting her "truth" for g/acc-ish reasons

interesting

Trans people are over-represented in the rationalist community, relative to general population. It should be evident to anybody who hangs out with a lot of rationalists (at least in "big rationality hubs") or attends big rationality meetups. But I think I also saw some census data (either SSC/ACX or the LW census) confirming that.

wait - do you consider that an insult? i snuggled with the best of them

(not OP) high base rates of transgenderism in LW-rationalism, particularly the sections that would be the most sensible to tenets of Ziz's ideology (high interest in technical aspects of mathematical decision theory, animal rights, radical politics), while being on average more socially vulnerable, and Ziz herself apparently believed that trans women were inherently more capable of accepting her "truth" for more mystical g/acc-ish reasons (though I can't find first-hand confirmation rn)