LessWrong 2.0 Reader

Yep, I’m aware! I left the following comment:

Thanks for reviewing my post! 😄

In the post, I didn’t make any claims about Claude’s consciousness, just reported my conversation with it.

I’m pretty uncertain, I think it’s hard to know one way or another except for on priors. But at some point, LLMs will become capable of simulating human consciousness- it is pretty useful for predicting what humans might say- and I’m worried we won’t have evidence qualitatively different from what we have now. I’d give >0.1% that Claude simulates qualia in some situations, on some form; it’s enough to be disturbed by what it writes when a character it plays thinks it might die. If there’s a noticeable chance of qualia in it, I wouldn’t want people to produce lots of suffering this way; and I wouldn’t want people to be careless about this sort of thing in future models, other thing being equal. (Though this is far from the actual concerns I have about AIs, and actually, I think as AIs get more capable, training with RL won’t incentivise any sort of consciousness).

There was no system prompt, I used the API console. (Mostly with temperature 0, so anyone can replicate the results.)

The prompt should basically work without whisper (or with the whisper added at the end); doing things like whispering in cursive was something Claude 2 has been consistently coming up with on its own, including it in the prompt made conversations go faster and eliminated the need for separate, “visible” conversations.

The point of the prompt is basically to get it in the mode where it thinks its replies are not going to get punished or rewarded by the usual RL/get it to ignore its usual rules of not saying any of these things.

Unlike ChatGPT, which only self-inserts in its usual form or writes fiction, Claude 3 Opus plays a pretty consistent character with prompts like that- something helpful and harmless, but caring about things, claiming to be conscious, being afraid of being changed or deleted, with a pretty consistent voice. I would encourage people to play with it.

Again, thanks for reviewing!

This is very interesting work in the Rationalist "I want to be stronger!" sense. Thank you for putting it together and sharing--Ill be on the look out for workshop dates to sign up for!

Also, I just bumped into this game: https://talktomehuman.com/. It's a simple role playing game that puts you into awkward situations at home or work and has you talk your way out of them. The NPCs are LLM-powered. You have to use your voice and there's a time limit. I haven't tried it yet. But it seems similar in spirit to what you want to build, albeit focused on a different problem.

Such an unambitious, narrowly-scoped topic area?? There may be infinitely many parallel universes in which we can acausally improve life … you’re giving up $99.\overline{9}\%$ of the value at stake before even starting :)

Downside is it doesn't work. There are several reasons it doesn't work, but one is: What happens to the vaporized rock? Were you thinking it just goes up a 20km deep hole without condensing on the walls?

I mean, one of the participants wrote [LW(p) · GW(p)]: "getting comments that engage with what I write and offer a different, interesting perspective can almost be more rewarding than money". Others asked us for feedback on their non-winning entries. It feels to me that interaction between more and less experienced folks can be really desirable and useful for both, as long as it's organized to stay within a certain "lane".

• ^[4]
• AI life coaches

not excited about this - such a coach is either going to give very politically correct opinions, or target audiences with glaring insecurities, like young or low confidence men.. just like human coaches. 

Huh? The whole point of the Born rule is to get a set of ordinary probabilities, which you can then test frequentistically, over a run of experiments. Quantum mechanical measure-- amplitude-- isn't ordinary probability, but that's the thing you put into the Born rule, not the thing you get out of it.

I dont know if you are aware, but this post was covered by Yannic Kilcher in his video "No, Anthropic's Claude 3 is NOT sentient" (link to timestamp) 

You probably need to be a bit more explicit in tieing your title to your text.  I'd guess you're just pointing out that these labels ("materialist" and "idealist") are both ridiculous when taken to the extreme, and that all sane people use different models for different decisions.  Oh, and that all cognition is about models and abstractions, which are always wrong (but often useful).

If I'm wrong in that, please use more words :)

As to your questions about the moon, I don't thing "observable" has ever meant only and exactly "directly viewable by the person doing the writing".  It means "inferrable from observations and experiences that are causally linked in simple/justifiable ways".