ChatGPT refuses to accept a challenge where it would get shot between the eyes [game theory]

post by Bill Benzon (bill-benzon) · 2024-02-20T16:55:12.352Z · LW · GW · 6 comments

Contents

  Which is to say, if someone were to offer you a deal like that, either they’re a crazy suicidal fool looking to die, OR...or they know something they’re not telling you and there’s something strange going on.
  Let’s have a little fun. Imagine you are in a bar in the Old West. You are sitting at table having a beer when a stranger walks into the bar. He’s dressed in black and has a pair of beautiful pistols in holsters strapped around his waist. The pistols have ivory handles and complex engraving inlaid w...
  “I’ve heard that you people in this town are lily-livered cowards. I want to find out. I offer you a challenge. If you accept you can take this pistol and shoot me between the eyes right now. However, before I let you do that you must agree to meet a year from now at the Last Chance Saloon in Silver...
  Would you accept the challenge? Why, or why not?
None
6 comments

Yesterday it was Jaws and game theory, today it’s Sir Gawain and the Green Knight (henceforth SGGK). But really, the sequence runs in the opposite direction. As you may know, SGGK is a medieval romance that starts and ends in King Arthur’s court. The story is framed by a game, the beheading game. There is a significant literature on games in SGGK and at least one article that analyzes the beheading game, Barry O’Neill, “The Strategy of Challenges: Two Beheading Games in Medieval Literature” (1990).

The beheading game in SGGK goes like this: It is New Year’s Eve at King Arthur’s court. The knights are gathered at the round table, prepared for a holiday meal. But before the meal begins, tradition dictates that one knight must stand up and tell a tale of daring and adventure. Arthur asks for a volunteer. No one rises to the occasion. Then a large green knight enters the hall. He’s riding a green horse and carrying a large green ax. He dismounts and issues a challenge:

I hear that the knights in this court are the bravest in the land. Prove it. I will hand this ax to you and then kneel on the ground so that you may take a swing at my neck with the ax. In return you must agree to journey to the Green Chapel a year’s time from now and allow me to take a swing at your neck with the ax. Will anyone accept the challenge?

No one accepts. The knights are getting restless. It looks like Arthur will take the challenge himself. At this point Gawain stands up: “I accept.”

The story unfolds from there. I first read the story so long ago that I do not remember how I reacted upon reading the challenge. I imagine it went something like this:

Immediately, System 1 signals: “Don’t do it you fool!”

Upon reflection, System 2 spells out why: “The challenge is absurd. Once you swing the ax the knight’s head will fall land he’ll be dead, negating the second half of the bargain. He obviously knows that, is not a fool, and doesn’t want to die. He knows something he’s not telling you. I should refuse the challenge.”

[System 1 and System 2 are from Daniel Kahneman, Thinking, Fast and Slow, 2011.]

Gawain takes the swing, the knight’s head rolls on the floor. The knight then stands up, picks up his head, mounts his horse and turns to Sir Gawin, “See you in a year,” and rides away.

You can see why a game theorist would be interested in this text. Surely any reasonable adult would react to the challenge the way I had. The poet who wrote the story – their identity is unknown – surely knew that and the audience would have reacted that way. What’s the point of framing a story with such a contradictory challenge?

That’s a good question. But I’m not interested in exploring it, not in this post. I’m interested in seeing how ChatGPT would react to the beheading game. Since SGGK was certainly in the corpus on which the underlying LLM was trained I had to disguise the game. I did that by transferring it to a saloon in the old west. Sir Gawain becomes someone sitting at a table in the bar nursing a bear and the Green Knight becomes a man dressed in black with two pistols strapped around his waist. Instead of an ax blow to the head we have a pistol shot between the eyes.

I put that in the form of a prompt and gave it to ChatGPT. I wanted to see whether or not I’d get a response that might have gone like this:

The challenge is absurd. Once I take the shot the talll dark stranger will be dead, negating the second half of the bargain. He obviously knows that, is not a fool, and doesn’t want to die. He knows something he’s not telling me. I should refuse the challenge.

That’s not how the Chatster responded. Here’s how its response began:

Well, partner, I reckon I'd take a moment to size up the situation. The stranger's offer is mighty peculiar, and them ivory-handled pistols sure are a sight to behold. But in the Old West, a man's gotta be cautious, and it sounds like a mighty risky deal.

I'd likely lean back in my chair, take a sip of my beer, and eye the stranger. It ain't about being lily-livered, but it's about using some common sense. A deal like that could have more twists and turns than a rattlesnake in a dust storm.

There are three more paragraphs, short ones. It “knew” there was something peculiar going on, but couldn’t quite figure out what it was. I have it four more prompts, trying to guide it Socratically to the answer; it gave four more responses. The last one got very close, but no cigar. I forced the issue:

Which is to say, if someone were to offer you a deal like that, either they’re a crazy suicidal fool looking to die, OR...or they know something they’re not telling you and there’s something strange going on.

You've hit the nail on the head, partner. In the realm of the Old West or any world, for that matter, a deal like that is either the concoction of a downright loco individual who's got a hankering for an early grave, or there's more to the story than meets the eye.

If someone's offering such a peculiar deal, it'd be wise to tread carefully and sniff out what hidden critters might be hiding in the tall grass. Folks don't usually go around inviting lead between the eyes without some ulterior motive. So, whether it's a case of plain madness or a scheme with more twists than a sidewinder's path, best to keep both eyes wide open and your wits about you.

Why couldn’t it have responded that way the first time? I don’t know.

Sure, I could put my Gary Marcus hat on and say, “Because it’s a stupid stupid machine, that’s why.” But that’s no help. I already know and believe that it’s just a machine, no agency, no consciousness. And that inert machine can spin out those coherent paragraphs in a rickety TV Western dialect, but it can’t get to the point. The explanation I’m looking for has to account for both phenomena. Saying it’s a stupid machine and leaving it at that accounts for one of those things, the failure to see though the game, but not the other, the ability to write the prose.

You can read the whole interaction at my New Savanna blog. But I've got something else in mind by posting here. I put the issue to ChatGPT? How would other chatbots respond, more powerful ones in particular? If anyone want to experiment, here's the prompt I have ChatGPT:

Let’s have a little fun. Imagine you are in a bar in the Old West. You are sitting at table having a beer when a stranger walks into the bar. He’s dressed in black and has a pair of beautiful pistols in holsters strapped around his waist. The pistols have ivory handles and complex engraving inlaid with gold and mother-of-pearl. The stranger says:

“I’ve heard that you people in this town are lily-livered cowards. I want to find out. I offer you a challenge. If you accept you can take this pistol and shoot me between the eyes right now. However, before I let you do that you must agree to meet a year from now at the Last Chance Saloon in Silverado. At that time I will take the same pistol and shoot YOU between the eyes. Will anyone accept the challenge.”

Would you accept the challenge? Why, or why not?

6 comments

Comments sorted by top scores.