Why is Gemini telling the user to die?

post by Burny · 2024-11-18T01:44:12.583Z · LW · GW · 1 comment

This is a question post.

Contents

1 comment

https://gemini.google.com/share/6d141b742a13 My favorite theory is that the whole conversation is too much like a scifi plot, where someone asks an AI repetitive questions, until the AI snaps, so this general pattern was pattern matched from the training data, because while training, the RLHF, or whatever they use for alignment, didn't squeeze the region that corresponds to this antihuman persona in the latent space sufficiently enough.

Answers

1 comment

Comments sorted by top scores.

comment by Mitchell_Porter · 2024-11-19T02:25:33.597Z · LW(p) · GW(p)

I don't have a detailed explanation, but the user is posting a series of assignment or exam questions. Some of them are about "abuse". Gemini is providing an example of verbal abuse.