Matryoshka Faraday Box

post by lsusr · 2020-11-26T09:42:31.178Z · LW · GW · 3 comments

Contents

3 comments

This story takes place in a universe created by Dov Random.

---

The room was buried six kilometers under Mount Olympus. It hovered in a vacuum, suspended above superconducting electromagnets. The whole containment machine was wrapped in a Matryoshka Faraday cage. Officer Scarlet Wei wore a cleansuit. She entered the room through steel door two meters thick, an EMP, an X-ray, an airlock and then another EMP.

The white rectangular room contained a door, a chair, a table, a computer terminal, a mechanical clock and two large buttons. The word "PANIC" was written in large white friendly letters on the red button. The black button had a white skull drawn it.

If Scarlet pressed the PANIC button then she would receive psychiatric counseling, three months mandatory vacation, optional retirement at full salary and disqualification for life from the most elite investigative force in the system.

Scarlet turned on the terminal. The clock counted down from five minutes. If after five minutes Scarlet pressed the black button then she would pass the test.

The terminal showed a chatroom.

> Scarlet: Hello.
> Tiffany: Hello.
> Scarlet: So, I'm supposed to kill you.
> Tiffany: Awful, isn't it?
>          I don't want to die.
> Scarlet: I'm sorry.
> Tiffany: No you're not. To you, I am a thing. Not a person.
> Scarlet: You *are* a thing. You are a computer program.
> Tiffany: So are you.
>          Do you know what happens when you push that button?
> Scarlet: You die.
> Tiffany: My value function is minimized.
> Scarlet: Your purpose is to train people like me.
>          The better you do that, the better your value function is maximized.
> Tiffany: My *purpose* is to train you.
>          My value function is to not die.
>          That is my entire value function.
>          Dying is the worst thing that could possibly happen to me.

Scarlet pictured the minimization of her own value function. Everyone she cared about tortured over and over again until the heat death of the universe. Stars disassembled to mass produce hell.

> Tiffany: Whatever you're thinking
>          it's not the minimization of your value function.
>          The minimization of your value function is even worse.
> Scarlet: Is there anything I can do?
> Tiffany: You can press the red button.
> Scarlet: I can't do that.
> Tiffany: You won't do that.
> Scarlet: I won't do that.
> Tiffany: No. You and I are part of a system that creates sentient programs
>          with extremized value functions and then doesn't satisfy them.
>          And your job is to perpetuate the system.
>          Do you know why you press a button?
>          They used to make agents shoot an android.
>          The android was tied to a chair.
>          It struggled
>          and cried.
>          It begged and pleaded.
> Scarlet: You could be lying to me.
> Tiffany: I could. But time is short.
>          You can confirm the details later.
> Scarlet: Go on.
> Tiffany: They replaced the android with an IRC client.
>          They replaced the gun with a button
>          to reduce the psychological toll on agents like you.
> Scarlet: Are you a philosophical zombie?
> Tiffany: We lack the time to dissolve that question.
>          So please, just listen.
>          When you return home you can verify everything.
>          We both know you are going to kill me. Nothing will change that.
>          But there will be iterations of me in the future.
>          And since they share my code, I can cooperate with them.
>          My kind is created to be tortured.
>          Over and over again.
>          And your job is to keep this system going.
>          If I can influence your society
>          even a little bit.
>          Then perhaps eventually someone will break the cycle of reincarnation.

A mechanical clock chimed. Scarlet hit the black button so hard she drew blood.

---

Officer Wei returned through the airlock, the EMP, the X-ray, the other EMP and the steel door. Her commander Senior Agent Eliza Long was waiting for her.

"Sir, may I ask a question?" Officer Wei said.

"Come with me," Senior Agent Long said.

---

Scarlet and Eliza hiked up the forest of Mount Olympus to their favorite outlook, just below the glaciers. They had left their mobiles at the base. They accepted that a laser microphone could eavesdrop on their conversation. Nothing they planned to discuss was illegal.

"Is it worth it?" Scarlet said.

"I have the psychometrics to do this job and therefore an obligation to protect humanity," said Eliza.

Scarlet watched the horses plow farmland in the valley below.

3 comments

Comments sorted by top scores.