Opt out from the Funni

post by Coafos (CoafOS) · 2022-09-24T22:07:38.360Z · LW · GW · 1 comments

Contents

  Abstract
  Intro
  On Petrov
  "idealistic agent"
  Summary
  Closing remarks
None
1 comment

Abstract

There are situations, where inaction would lead to an interesting situation, which could go badly if you make a wrong decision. AKA The Funni.

However, sometimes you can take a preemptive action which would lead to a boring situation. You should consider this option more times because it might be better. Hence this post.

Epistemic vibes: this post was made by the opt-out gang.
Disclaimer: I bought "no" on the manifold.

Intro

Imagine the following: a trolly is hurtling towards a fork. You can pull a lever to change paths. If you pull the lever, the trolley will change to an empty track.

But the default track contains the classical 1-5 trolley, where you have to make a high-stakes morally-grey decision in an adrenalin-fueled split-second if you don't pull the first lever.

You know it is morally good to pull the first lever. With cold blood, the choice is completely clear. But that would be booooring, so what if

On Petrov

When your screen shows that five missiles are approaching, and the system is waiting for your answer, it's already late.

And it's not late because you have to follow up on your C. U. [? · GW] T. [? · GW] E. decision theory to make deterrence work. It's not late because you can be a renegade and don't let the world burn.

It's late because there is an option to burn the world.

"idealistic agent"

So you're saying that it would not be a problem for you, because, as far as you know, you will make the right decisions at the right time. And I think it's possible you are right.

However, it won't be you who makes this decision. It will be future you. And future you is a complex agent, they can be hungry or sleepy. They'll have system 1, which can override the right decision. They'll have an inclination to do funni.

That's why you have to protect yourself against these. The you here is plural; it's a very universal human trait. Just remember that in John 8:7 Jesus said:

Let any one of you who's a rational agent be the first to throw a nuke at her.

Summary

What I said, that you should find an option to opt out of a future conflict, only works if you are not currently in that conflict. If the world is moving toward bad state, then you should actively act against it.

Like, if you're reading this, and there is a big red button to do bad, don't do it. But if this red button will only appear in a day or two, then you should probably find a way to disable it.

The good world should be relatively stable; if your proposed option would lead to a world where you have to make some pivotal actions, then you should suspect that it might not be a good world at all, and you have to make it safer.

Closing remarks

Life is a strange game. Sometimes, the only winning move is

1 comments

Comments sorted by top scores.