Decision Theory in Space
post by lsusr · 2024-08-18T07:02:11.847Z · LW · GW · 18 commentsContents
18 comments
"Since you are reluctant to provide us with the location of the Rebel base," said Grand Moff Tarkin, "I have chosen to test this station's destructive power on your home planet of Alderaan."
"No. Alderaan is peaceful. We have no weapons there. It is a loyal planet under Imperial control. Striking Alderaan would destroy your own resources and foment rebellion. Destroying it is irrational," said Princess Leia, perfectly calm.
"Nonsense," said Tarkin, "That is a naïve understanding of decision theory. I am a causal decision theorist, but I acknowledge the value of precommitments. I therefore precommit to destroying Alderaan unless you reveal to me the location of the Rebel base. This is not an irrational act if you capitulate to me."
"But it is an irrational act if I do not capitulate to you," said Leia, "I am a functional decision theorist. The algorithm I use to select my decision accounts for the fact that you are modelling my mind. You are a rational agent. You only threaten me because you expect me to succumb to your blackmail. Because of that I will not succumb to your blackmail."
"I'm going to do it," said Tarkin.
"Sure you are," said Leia.
"I'm really going to blow up the planet," said Tarkin.
"Be my guest," said Leia, with a smile, "Aim for the continent Anaander. Its inhabitants always annoyed me. We'll see who has the last laugh."
"I'm really really going to do it," said Tarkin.
"I grow tired of saying this, so it'll be the last time. Just blow up the planet already. I have an execution I'm late for…."
Leia's voice trailed off. She was suddenly aware of the deep, mechanical breathing behind her.
Kshhhhhhh. Kuuuuuuo. Kshhhhhhh. Kuuuuuuo.
Everyone in the Life Star command center turned to face the cyborg space wizard samurai monk in black armor.
Kshhhhhhh. Kuuuuuuo. Kshhhhhhh. Kuuuuuuo.
Vader's cloak fluttered and a couple indicator lights on his life support system blinked, but no muscles or actuators moved. A semi-mechanical voice in the uncanny valley spoke from Vader's mask. "Chief Gunnery Officer Tenn Graneet, you may fire when ready."
"Commander Tenn Graneet, belay that order," said Tarkin.
The Chief Gunnery Officer held his hand above his control panel, touching nothing. He looked rapidly back-and-forth between Tarkin and Vader.
Tarkin turned angrily to face Vader. "Are you insane?" Tarkin hissed.
Vader ignored the question and looked at Leia. "Where is the Rebel base?"
Leia's eyes were wide with horror and her mouth was wide with a silent scream. She clenched her teeth and stared at the floor. "Tatooine. They're on Tatooine," Leia said.
"Chief Gunnery Officer Tenn Graneet, you may fire when ready," said Vader.
"What‽" exclaimed Tarkin.
Graneet lifted the clear cover off of the authorization lever. He moved his hand as slowly as he could.
"Commander Tenn Graneet, belay that order," said Tarkin.
"Commander Tenn Graneet, ignore all orders you receive from the Grand Moff," said Vader.
"Commander Tenn Graneet, I am your commanding officer. Ignore all orders from 'Lord' Vader. If you continue to disobey my orders, you will be court martialed," said Tarkin.
Graneet continued the process of authorizing the firing team.
Tarkin drew his blaster pistol and held it to Graneet's head. "Stop or I will shoot you in the head right now," said Tarkin.
Bkzzzzzzzzzzzzz.
Tarkin felt the heat of Vader's red lightsaber a centimeter from his neck. The next seconds felt like slow motion. Graneet paused. Then Greneet continued the firing activation sequence. Tarkin pulled the trigger.
Click.
Nothing came out of the blaster's emitter.
Vader didn't even bother to watch his order get carried out. He just turned around, deactivated his lightsaber, and strode out of the command center. Vader's cape billowed as Alderaan exploded behind him. His black silhouette was edge lit with the radiation of an exploding planet.
The Life Star command center was silent after the genocide.
"Thank you," said Leia.
"For what," said Tarkin in a monotone.
"For trying to save my home," said Leia.
"Kark off," said Tarkin.
18 comments
Comments sorted by top scores.
comment by jbash · 2024-08-18T15:05:32.495Z · LW(p) · GW(p)
There's surely some point or joke in this, but I'm just going "Wat?". This disturbs me because not many things go completely over my head. Maybe I'm not decision theory literate enough (or I guess maybe I'm not Star Wars literate enough).
Is Vader supposed to have yet another decision theory? And what's the whole thing with the competing orders supposed to be about?
Replies from: Ape in the coat, ChrisHibbert↑ comment by Ape in the coat · 2024-08-18T17:50:22.086Z · LW(p) · GW(p)
Tarkin doesn't really want to destroy Alderan for the reasons Leia mentions. His claims that he has precommited to the destruction of Alderan, unless Leia reveals the placement of a base, is a threat. Which Leia ignores as rational agent should.
Tarkin's precommitment is only rational if Leia indeed reveals the placement of the base when this precommitment is made. Leia knows it and, therefore, doesn't reveal the placement of the base. This retroactively makes Tarkin's precommitment irrational, so he constantly reconsiders it. It's rational for him to claim that he has precommited to the destruction of Alderan, but it is irrational for him to actually destroy Alderan. So they are caught in a loop: Tarkin keeps claiming that he has precommited, maybe even fully believing that his precommitment is true in the moment of making it, but the next moment, he second guesses it, and therefore he is unable to truly precommit. Leia knows all of it and therefore she is initially confident that Alderan is safe: the only stable equilibrium is when Alderan is not destroyed.
Vader really doesn't care about Alderan. He is not saying that he is going to destroy it, expecting a particular reaction from Leia, he is ready to do it regardless of her reaction. For him it's not a threat - it's BATNA. And so he is capable to actually precommit and follow through. Leia understands this and that now the stable equilibrium is that Alderan is destroyed, unless she does something.
Then Vader proceeds to destroy Alderan anyway, which proves that he indeed was going to do it, and not just making a threat expecting a particular behavior from Leia. On the other hand, Tarkin tries to stop it, because he doesn't want Alderan destroyed in the first place, and because he has precommited not to destroy Alderan if Leia reveals the placement of the base, which he believes she did. They have a short precommitment race with Vader, which ends when Tarkin's blaster fails to shoot (something to do with following the rules of physics when the lightsaber is lit).
There is an extra layer of Commander Tenn Graneet's mental process where he apparently understands that Vader is going to win the precommitment race. And whether Leia lied about the placement of the base or not, and whether Vader caught her lying, which, I suppose, will be revealed later.
Replies from: lsusr, Filip Sondej, Ape in the coat↑ comment by Filip Sondej · 2024-09-04T12:42:28.315Z · LW(p) · GW(p)
For me the main thing in this story was that cheap talk =/= real commitment. You can talk all you want about how "totally precommitted" you are, but this lacks some concreteness.
Also, I saw Vader as much less galaxy brained as you portray him. Destroying Alderaan at the end looked to me more like mad ruthlessness than calculated strategy. (And if Leia had known Vader's actual policy, she would have no incentive to confess.) Maybe one thing that Vader did achieve, is signal for the future that he really does not care and will be ruthless (but also signaled that it doesn't matter if you give in to him, which is dumb).
Anyway, I liked the story, but for the action, not for some deep theoretic insight.
↑ comment by Ape in the coat · 2024-08-19T10:14:41.369Z · LW(p) · GW(p)
A bit more about whether Leia lied and Vader reasoning process.
For Vader there are two options, either Leia lied, and therefore Alderan is to be destroyed or she told the truth, but her decision algorithm for telling the truth was not correlated with Vaders not destroying Alderan as Vader has precommited to destroying Alderan, and therefore Alderan is to be destroyed.
Vader could've calculated all this beforehand which makes his initial precommitment to destroy Alderan rational. He looses nothing as he doesn't care about Alderan, but wins in case Leia tells the truth, which probability is non-zero. It's not rational for Leia to tell the truth, but she is put into a much more stressful situation where she is likelier to make a mistake than when dealing with Trakin from the comfort of confidence that Alderan is safe.
Its easy to ignore threats when all their effects are purely counterfactual. Much less so, otherwise. Next time when she will be dealing with Tarkin who will make a threat Leia may want to reconsider her superior rationality of ignoring threats, because otherwise she will have to deal with Lord Vader BATNAs.
↑ comment by ChrisHibbert · 2024-08-18T16:03:48.690Z · LW(p) · GW(p)
Vader is clearly homicidal and irrational. Leia's superior rationality won't slow him down. Leia recognizes that immediately, but she should also have realized that revealing the base's location wouldn't prevent him from destroying the planet.
comment by Martín Soto (martinsq) · 2024-08-18T20:46:20.238Z · LW(p) · GW(p)
The only decision-theoretic points that I could see this story making are pretty boring, at least to me.
Replies from: Filip Sondej, Mo Nastri↑ comment by Filip Sondej · 2024-09-04T13:08:42.519Z · LW(p) · GW(p)
I liked it precisely because it threw theory out the window and showed that cheap talk is not a real commitment.
- Tarkin > I believe in CDT and I precommit to bla bla bla
- Leia > I belive in FDT and I totally precommit to bla bla bla
- Vader > Death Star goes brrrrr...
↑ comment by Martín Soto (martinsq) · 2024-09-14T19:44:47.192Z · LW(p) · GW(p)
hahah yeah but the only point here is: it's easier to credibly commit to a threat if executing the threat is cheap for you. And this is simply not too interesting a decision-theoretic point, just one more obvious pragmatic consideration to throw into the bag. The story even makes it sound like "Vader will always be in a better position", or "it's obvious that Leia shouldn't give in to Tarkin but should give in to Vader", and that's not true. Even though Tarkin loses more from executing the threat than Vader, the only thing that matters for Leia is how credible the threat is. So if Tarkin had any additional way to make his commitment credible (like program the computer to destroy Alderaan if the base location is not revealed), then there would be no difference between Tarkin and Vader. The fact that "Tarkin might constantly reconsider his decision even after claiming to commit" seems like a contingent state of affairs of human brains (or certain human brains in certain situations), not something important in the grander scheme of decision theory.
comment by abstractapplic · 2024-08-18T10:55:12.842Z · LW(p) · GW(p)
succomb
succumb
"Tatooine. They're on Tatooine,"
Was this deviation from canon intentional? (I remember in the movie she picks a different planet with a similar-sounding name.)
Replies from: ben-lang, lsusr, Measure↑ comment by Ben (ben-lang) · 2024-08-19T09:06:50.787Z · LW(p) · GW(p)
Dantoine was where she said in the film I think.
Replies from: AnthonyC