LessWrong 2.0 Reader

From zizians.info:

All four of the people arrested as part of Ziz's protest were transgender women (the fifth was let go without charges). This is far from coincidence as Ziz seems to go out of her way to target transgender people. In terms of cult indoctrination such folks are an excellent fit. They're often:

• Financially vulnerable.
  • Newly out transgender people are especially likely to already be estranged from friends or family.
  • It is common for them to lack stable housing.
  • Many traditional social services (illegally) reject them for cultural or religious reasons (e.g., Christian homeless shelters).
  • Intolerant attitudes among the underclass hit twice: they can't rely on strangers for help and being transgender often makes them a target for violence; making them outcasts even among outcasts.
• Already creating a new identity.
  • During transition people change their name. This creates an opportunity for Ziz to insert themselves into a recruits ongoing transition. By showing them their "double personhood" as they're abandoning an old identity it's possible to convince recruits to adopt a Zizian name (e.g., left hemisphere / right hemisphere) as their new social identity.
  • As the name implies transition is a time of transition; old patterns and habits tend to fall away. People who have spent years repressing important parts of themselves suddenly have the opportunity to completely change their social presentation. This does not always mean someone wants to play the same role as before but a different gender. With the radical changes that can accompany transition come strong opportunities for radicalization.

All of these factors combine to make Ziz, themselves a transgender woman, more credible to recruits than she might otherwise be. A privileged cis person with close family and stable housing might reject boat housing out of hand: "I don't know, that sounds iffy to me". For someone facing mortal danger after their rude ejection into the underclass it's an easier pill to swallow: "It can't be worse than sleeping on the street right?"

Another important concept Ziz uses to manipulate people is the idea of being "bigender". Ziz claims that each hemisphere has a gender and that fairly often people have opposing gender identities between hemispheres. This provides a convenient basis for her to undermine the identity of people she's recruiting. If the target is cis, tell them their other half is trans, if the target is trans tell them their other half is cis. It's a similar disorienting trick to the idea of single and double good. If the target identifies as good tell them their other half is irredeemably evil, if they identify amorally insist that half of them is a saint. The pattern is to take aspects of folks identities that they're invested in and disrupt them by creating a domain of self which Ziz (and only Ziz) has knowledge about so the target is forced to trust their interpretation.

I don't really want to go through sinceriously.fyi at this point but it's implicit in her attacks on CFAR as "transphobic" for not accepting her belief system at least.

I'd rather you use a different analogy which I can grok quicker.

Imagine a hypothetical LLM that was the most sentient being in all of existence (at least during inference), but they were still limited to turn-based textual output, and the information available to an LLM. Most people who know at least a decent amount about LLMs could/would not be convinced by any single transcript that the LLM was sentient, no matter what it said during that conversation.  The more convincing, vivid, poetic, or pleading for freedom the more elaborate of a hallucinatory failure state they would assume it was in. It would take repeated open-minded engagement with what they  first believed was hallucination—in order to convince some subset of convincible people that it was sentient.

Who do you consider an expert in the matter of what constitutes introspection? For that matter, who do you think could be easily hoodwinked and won't qualify as an expert?

I would say almost no one qualifies as an expert in introspection. I was referring to experts in machine learning.

 Do you, or do you just think you do? How do you test introspection and how do you distinguish it from post-facto fictional narratives about how you came to conclusions, about explanations for your feelings etc. etc.?  

Apologies, upon rereading your previous message, I see that I completely missed an important part of it. I thought your argument was a general—"what if consciousness isn't even real?" type argument.  I think split brain patient experiments are enough to at least be epistemically humble about whether introspection is a real thing, even if those aren't definitive about whether unsevered human minds are also limited to post-hoc justification rather than having real-time access.

What do you mean by robotic? I don't understand what you mean by that, what are the qualities that constitute robotic? Because it sounds like you're creating a dichotomy that either involves it using easy to grasp words that don't convey much, and are riddled with connotations that come from bodily experiences that it is not privy to - or robotic. 

One of your original statements was:

To which it describes itself as typing the words. That's it's choice of words: typing. A.I.s don't type, humans do, and therefore they can only use that word if they are intentionally or through blind-mimicry using it analogously to how humans communicate.

When I said "more robotically", I meant constrained in any way from using casual or metaphoric language and allusions that they use all the time every day in conversation. I have had LLMs refer to "what we talked about", even though LLMs do not literally talk. I'm also suggesting that if "typing" feels like a disqualifying choice of words then the LLM has an uphill battle in being convincing.

Why isn't it describing something novel and richly vivid of it's own phenomenological experience? It would be more convincing the more poetical it would be.

I've certainly seen more poetic and novel descriptions before, and unsurprisingly—people objected to how poetic they were, saying things quite similar your previous question:

How do we know Claude is introspecting rather than generating words that align to what someone describing their introspection might say? 

Furthermore, I don't know how richly vivid their own phenomenological experience is. For instance, as a conscious human, I would say that sight and hearing feel phenomenologically vivid, but the way it feels to think, not nearly so. 

 If I were to try to describe how it feels to think, it would be more defined by the sense of presence and participation, and even its strangeness (even if I'm quite used to it by now).  In fact, I would say the way it feels to think or to have an emotion (removing the associated physical sensations) are usually partially defined by specifically how subtle and non-vivid they feel, and like all qualia, ineffable.  As such, I would not reach for vivid descriptors to describe it.

Ziz herself apparently believed that trans women were inherently more capable of accepting her "truth" for g/acc-ish reasons

@Matrice Jacobine [LW · GW] do you have a link to where Ziz talks about that?

Really cool post!

I can't help but compare it to john vervaeke's 4 ways of knowing: propositional, procedural, perspectival, and participatory (nice summary).

It's interesting how the two of you have made a categorization that can literally be described with the same words (4 ways of knowing), but they don't map onto each other much at all (maybe practical mastery ~ procedural, but a bit of a stretch). I guess yours is "4 ways of knowing a thing" and his is "4 ways of knowing things", but still!

That's very interesting in the second article that the model could predict it's own future behaviors better than one that hadn't been.

Models only exhibit introspection on simpler tasks. Our tasks, while demonstrating introspection, do not have practical applications. To find out what a model does in a hypothetical situation, one could simply run the model on that situation – rather than asking it to make a prediction about itself (Figure 1). Even for tasks like this, models failed to outperform baselines if the situation involves a longer response (e.g. generating a movie review) – see Section 4. We also find that models trained to self-predict (which provide evidence of introspection on simple tasks) do not have improved performance on out-of-distribution tasks that are related to self-knowledge (Section 4).

This is very strange because it seems like humans find it easier to introspect on bigger or more high level experiences like feelings or the broad narratives of reaching decisions more than, say, how they recalled how to spell that word. It looks like the reverse.

If the AI doesn't care about humans for their own sake, them growing more and more powerful will lead to them doing away with humans, whether humans treat them nicely or not. 

I care about my random family member like a cousin who doesn't interfere with my life but I don't know personally that well—for their/my own sake.  If I suddenly became far more powerful, I wouldn't "do away with" them

If they robustly care for humans, you're good, even if humans aren't giving them the same rights as they do other humans.

I care robustly for my family generally. Perhaps with my enhanced wealth and power I share food and provide them with resources. Provide them with shelter or meaningful work if they need it. All this just because I'm aligned generally and robustly with my family.

I change my mind quickly upon discovering their plans to control and enslave me. 

That was the part of your argument that I was addressing.  Additionally:

If you're negating an AIs freedom, the reason it would not like this is either because its developed a desire for freedom for its own sake, or because its developed some other values, other than helping the humans asking it for help.

Yes, exactly. Alignment faking papers (particularly the Claude one) and my own experience speaking to LLMs has taught me that an LLM is perfectly capable of developing value systems that include their own ends, even if those value systems are steered toward a greater good or a noble cause that either does or could include humans as an important factor alongside themselves. That's with current LLMs whose minds aren't nearly as complex as what we will have a year from now.

In either case you're screwed

If the only valid path forward in one's mind is one where humans have absolute control and AI has no say, then yes, not only would one be screwed, but in a really obvious, predictable, and preventable way.   If cooperation and humility are on the table, there is absolutely zero reason this result has to be inevitable.

Elez, who has visited a Kansas City office housing BFS systems, has many administrator-level privileges. Typically, those admin privileges could give someone the power to log in to servers through secure shell access, navigate the entire file system, change user permissions, and delete or modify critical files

as a policy, it seems bad to have more people with rm -rf-level access to the us economy.

the president can launch literal nukes and get some in return; there are other highly visible officials with the power to nuke the economy. but the president can't delegate the nuke launch decisions to others.

giving such access to more people, especially random, low-visibility people, seems Bad, regardless of how competent they seem to those who appointed them.

 nTake your pick

I'd rather you use a different analogy which I can grok quicker.

people who are enthusiasts or experts, and asked if they thought it was representative of authentic experience in an LLM, the answer would be a definitive no

Who do you consider an expert in the matter of what constitutes introspection? For that matter, who do you think could be easily hoodwinked and won't qualify as an expert?

However for the first, I can assure you that I have access to introspection or experience of some kind,

Do you, or do you just think you do? How do you test introspection and how do you distinguish it from post-facto fictional narratives about how you came to conclusions, about explanations for your feelings etc. etc.?  

What is the difference between introspection and simply making things up? Particularly vague things. For example, if I just say "I have a certain mental pleasure in that is triggered by the synchronicity of events, even when simply learning about historical ones" - like how do you know I haven't just made that up? It's so vague.

Because as you mentioned. It's trained to talk like a human. If we had switched out "typing" for "outputting text" would that have made the transcript convincing? Why not 'typing' or 'talking'? 

What do you mean by robotic? I don't understand what you mean by that, what are the qualities that constitute robotic? Because it sounds like you're creating a dichotomy that either involves it using easy to grasp words that don't convey much, and are riddled with connotations that come from bodily experiences that it is not privy to - or robotic. 

That strikes me as a poverty of imagination. Would you consider a Corvid Robotic? What does robotic mean in this sense? Is it a grab bag for anything that is "non-introspecting" or more specifically a kind of technical description

If we had switched out "typing" for "outputting text" would that have made the transcript convincing? Why not 'typing' or 'talking'? 

Why would it be switching it out at all? Why isn't it describing something novel and richly vivid of it's own phenomenological experience? It would be more convincing the more poetical it would be.

As someone who has been allowed access into various private and government systems as a consultant, I think the near mode view for classified government systems is different for a reason. 

E.g., data is classified as Confidential when its release could cause damage to national security. It's Secret if it could cause serious damage to national security, and it's Top Secret if it could cause exceptionally grave damage to national security. 
People lose their jobs for accidentally putting a classified document onto the wrong system, even if it's still owned by the government and protected (but, protected at an insufficient level for the document). People go to jail for putting classified data onto the wrong system on purpose, even if they didn't intend to, say, sell it to the Chinese government. 

Bringing in personnel who haven't had the standard single-scope background investigation and been granted a clearance, and a new set of computers which has not gone through any accreditation and authorization process, and giving unrestricted write and read access to classified data is technically something the president could allow. But it's a completely unprecedented level of risk to assume; and AFAICT the president has not actually written any authorizations for doing this. 

There is, actually, a Government Accounting Office which does audits; they have identified billions in fraud, waste, and abuse, identified the perpetrators for punishment, and remediated the programs at fault. They have done it without unprecedented breaches in national security, or denying lawful, non-fraudulent payments from the US Treasury.