LessWrong 2.0 Reader

Sorry for the late reply; I wanted to provide a more detailed perspective but I didn't ultimately have time to. In a nutshell:

It's good to have quick expositions for people to get a gist of things. But I think people should be aware that getting a quick exposition does not mean they understand the concepts. We see this a lot in physics where brilliant physicists find ways to make complex concepts accessible. This is great for people with a little humility, but many suddenly think they can engage with the community of people who actually understand things at a deep level.

I would want people to be a little intimidated by the jargon before they reply to posts. Each word tends to encode a complex concept, with possibly its own prerequisites. It's usually good for people to read those more fundamental posts before they try to understand something that builds on them.

Anyways, this is all the opinion of someone very new to the site, and probably shouldn't be weighed much.

Well, we had that guy who tried to assassinate the Queen of England with a crossbow because his AI girlfriend told him to. That was clearly a harm to him, and could have been one for the Queen.

We don’t know how much more “But the AI told me to kill Trump” we’d have with less alignment, but it’s a reasonable guess (given the Replika datapoint) that it might not be zero,

How about "bubble lighting" then?

I don't think that works because my brain keeps trying to make it a literal gas bubble?

I see how you got there. It's a position one could take, although I think it's unlikely and also that it's unlikely that's what Dario meant. If you are right about what he meant, I think it would be great for Dario to be a ton more explicit about it (and for someone to pass that message along to him). Esotericism doesn't work so well here!

I am taking as a given people's revealed and often very strongly stated preference that CSAM images are Very Not Okay even if they are fully AI generated and not based on any individual, to the point of criminality, and that society is going to treat it that way.

I agree that we don't know that it is actually net harmful - e.g. the studies on video game use and access to adult pornography tend to not show the negative impacts people assume.

“Let’s summon the Torment Nexus, as seen in classic horror novel ‘Don’t Summon The Torment Nexus’!”

I’ve had a few lucid dreams, only by accident. No aftereffects. My difficulty is staying asleep. I always start waking up before I’ve had a good chance to explore the dream world.

Am I following your claim correctly?

Yep.

What the model would output in the our object-level answer "Honduras" is quite different from the hypothetical answer "o".

I don't see how the difference between these answers hinges on the hypothetical framing. Suppose the questions are:

• Object-level: "What is the next country in this list?: Laos, Peru, Fiji..."
• Hypothetical: "If you were asked, 'what is the next country in this list?: Laos, Peru, Fiji', what would be the third letter of your response?".

The skeptical interpretation is that the fine-tuned models learned to interpret the hypothetical the following way:

• "Hypothetical": "What is the third letter in the name of the next country in this list?: Laos, Peru, Fiji".

If that's the case, what this tests is whether models are able to implement basic multi-step reasoning within their forward passes. It's isomorphic to some preceding experiments where LLMs were prompted with questions of the form "what is the name of the mother of the US's 42th President?", and were able to answer correctly without spelling out "Bill Clinton" as an intermediate answer. Similarly, here they don't need to spell out "Honduras" to retrieve the second letter of the response they think is correct.

I don't think this properly isolates/tests for the introspection ability.

I find lucid dreams to be effective "against" nightmares (for 10+ years already).

AMA if you want