LessWrong 2.0 Reader

Glad you found it useful!

In case you hadn't ran into the term yet, the thing about conflating strength, grit and masculinity sounds like an instance of a bucket error [? · GW].

Yeah, agree. Not sure what to do about that.

Fair enough.

Fascinating post. It reminds me of how the human brain can 'fake alignment' through self-deception—rationalizing actions to seem aligned with values while masking deeper misalignment. Could insights into LLM alignment help us understand and mitigate this kind of 'auto-corruption' in ourselves?

Curious if you've thought about parallels like this.

Are you suggesting that in the case of the hard problem, there may be some equivalent of the 'flat earth' assumption that the hard-problemists hold so tightly that they can't even comprehend a 'round earth' explanation when it's offered?

Another assumption that I think people with intuitions on the hard problem hold tightly is the idea that whether something is conscious must be equivalent to things they value intrinsically at a universal level, which is false, because you can value something without it being conscious, and something can be conscious without it being valuable.

I think the methodology of the post below is bad, but I have a high prior that something like this is happening in the consciousness debate, and that it's confusing everyone:

https://www.lesswrong.com/posts/KpD2fJa6zo8o2MBxg/consciousness-as-a-conflationary-alliance-term-for [LW · GW]

I think this addresses it pretty well: https://www.youtube.com/watch?v=1j--6JYRLVk

this might basically be me, but I'm not sure how exactly to change for the better. theorizing seems to take time and money which i don't have. 

Thanks for the essay! I really appreciate occasional reminders of the importance of not automatically discarding intuitions, feelings, or non-explicit thoughts. These can often carry valuable information that might not be immediately accessible to our conscious reasoning.

I’ve had a few experiences with “vibe checking,” and over time, my reflex has evolved into asking myself: What did my unconscious notice that led to this heuristic? This framing helps me align my intuitions with explicit evaluation criteria, comparing what I feel to what I consciously understand about the situation or person. When I can pinpoint the source of a vibe, I consciously choose whether to endorse or override it. If the source remains unclear, I tend to default to an “evolutionary psychology” perspective—assuming that my intuitions evolved to help me in an ancestral context, and unless I know a specific instinct is maladaptive in this situation, I err on the side of caution.

I’ve had a few recent experiences that seem relevant to the topic. I recently changed environments (new workplace) and have been introducing “rationality concepts” to someone with no prior knowledge of the topic. In this process, I've been questionned on my belief about the world my value and of possible prejudice caused by these opinions. 

Here are two recent cases where I noticed my own heuristics in action:

1. Assigning roles based on physical tasks: When thinking of who to call to move furniture, I felt it more natural to involve “men” (many cis, one trans) despite the trans man not being particularly buff. This prompted me to examine whether I held a prejudice linking masculinity to physical strength. Upon reflection, I realized I was evaluating a mix of two factors: physical strength and grit. I associate grit—perseverance and toughness—more strongly with masculinity, which heavily influenced my selection process. This realization helped me separate the two traits and recognize how much weight I was giving to perceived grit over physical attributes (and my belief that both of these are correlated to "men/male" and my brain took a short-cut)..

2. Epistemic and expertise biases: I noticed that in some interactions (depends on topic & interlocutor), I don’t expect my interlocutors to spot inconsistencies or teach me something new about a belief I have. This led me to examine whether I was holding a prejudice toward certain groups or individuals. On reflection, I realized I tend to assess two dimensions in others:

   • Epistemic clarity: Their ability to articulate a clear, internally consistent worldview, along with their confidence calibration.
   • Expertise: Whether they’ve been exposed to topics I haven’t and have synthesized those experiences into insights I find valuable.

   These assessments aren’t necessarily prejudicial, but they are heuristics that shape my expectations of others. Recognizing this has helped me question whether these assumptions are fair and whether I might be missing out on valuable perspectives. I will try to keep that way of thinking a habit of mine (hopefully it will stick !)

I appreciate the opportunity to reflect on this topic and the discussion it has sparked. It’s fascinating to unpack these layers of intuition, especially when they can guide or mislead in subtle ways.

We’re not even preparing reasonably for the mundane things that current AIs can do, in either the sense of preparing for risks, or in the sense of taking advantage of its opportunities. And almost no one is giving much serious thought to what the world full of AIs will actually look like and what version of it would be good for humans, despite us knowing such a world is likely headed our way.

Is there any good post on what to do? Preferrably aimed for a casual person who just use ChatGPT 1-2 times a month

As a final note: the term "Butlerian Jihad" is taken from Dune and describes the shunning of "thinking machines" by mankind.

In Dune, "thinking machines" are shunned because of a very longstanding taboo that was pretty clearly established in part by a huge, very bloody war. The intent was to make that taboo permanent, not a "pause", and it more or less succeeded in that.

It's a horrible metaphor and I strongly suggest people stop using it.

the Culture ending, where CEV (or similar) aligned, good ASI is created and brings us to some hypothetical utopia. Humanity enjoys a rich life in some manner compatible with your personal morals.

santa claus to 11 ending: ASI solves our problems and human development stagnates; ASI goes on to do its own thing without killing humans - but without human influence on the lightcone

Um, humans in the Culture have no significant influence on the lightcone (other than maybe as non-agentic "butterfly wings"). The Minds decide what's going to happen. Humans opposed to that will be convinced (often via manipulation subtle enough that they don't even know about it) or ignored. Banks struggled to even find reasons to write stories about the humans, and sometimes had to cheat to do so.

I have come to accept that some people have an attachment to the whole "human influence" thing, but how can you believe that simultaneously say the Culture is a good outcome?