Don't die with dignity; instead play to your outs

Okay, but isn’t “playing to your outs” encouraging people to do crazy or violent things?

I think this is the largest downside to this framing (though I'd include simply "counter-productive" along with the others), and comes out of the MtG-vs-reality disparity: in MtG you know your outs; in reality, you do not. (I know you mentioned this, but it needs more shake-you-by-the-lapels-and-scream-it-in-your-face)

This framing needs to come with prominent warnings along the lines of:
Do NOT assume that what you think is an out is certainly an out.
Do NOT assume that the potential outs you're aware of are a significant proportion of all outs.

The issue with crazy/violent plans is not only that they're usually dumb/unlikely-to-work, but that even when they would be sensible given particular implicit assumptions, those assumptions may not hold.

People don't know what they don't know. Playing to the-outs-I-can-think-of will feel like "playing to my outs" - while I may be witlessly screwing up a load of outs I didn't notice.

I guess this is why Eliezer went with the dignity framing - it suggests a default of caution, and perhaps deliberation. Playing to your outs suggest throwing caution to the wind (this is often how it works when you know your outs).

For anyone adopting this framing, I suggest an exercise where you first imagine having someone repeatedly scream YOU DO NOT KNOW ALL YOUR OUTS!! YOU WILL NEVER KNOW ALL YOUR OUTS!! at you for long enough that you know it in your bones.

I agree with the first point of "dying with dignity" doesn't seem like it accomplishes the goal it was intending to accomplish. I'm not yet sold on "play to your outs" accomplishing the right thing (my cruxes have to do with how people end up interpreting / acting-on / being-motivated-by the phrase).

I interpreted the die-with-dignity post to be saying something like:

1. P(doom) is very high. (I'm personally less confident of this than Eliezer but I'd be willing to substitute "P(doom) is likely enough to be high that we should start thinking about how we'd engage psychologically with that)"
2. When faced with doom, people's epistemics tend to get slippery, avoiding confronting the doom and coming up with plans that feel vaguely good without engaging with the hard problem. (I agree with this)
3. One way their epistemics get slippery is that they condition on being in worlds where the plans-they-could-think-of-might work, and once you've done that a couple times you're just living in a fantasy world. (I'm slightly philosophically confused about this, but I recall Eliezer once saying 'you get to do this approximately once without it being a bad idea', which seems plausible to me)
4. If people do fully accept high likelihood of failure, they have a hard time staying motivated [LW · GW] (both individuals, and as a group). (I agree with this)
5. When facing doom, people also tend to look for extreme strategies that burn the coordination commons, while not even really solving the problem. (I'm a bit confused about this but it seems true-ish)
6. Thinking in terms of "raising log-odds of survival while continuing to tackle the hard parts of the problem" is a useful reframe for motivation. (I'm confused about this. I'm not sure how much the "log-odds" part is supposed to be helping. I think there was also supposed to be some flavor of where Eliezer thought people's strategic plan-making thoughts should be going, but I don't think I could summarize it succinctly in a way he endorses)
7. Calling #6 "Die with dignity" is a good slogan. (I disagree with this)

"Die with dignity" seems overdeterminedly like a bad slogan because of landfish's aforementioned google of "dying with dignity", which is about terminally people voluntarily killing themselves. Seems like totally the wrong vibe, and not worth trying to win a memetic war on. 

I think Eliezer meant something like "die with dignity the way a soldier would" (where you keep fighting to end). I think "die fighting with dignity" is at least a marginal improvement (and even fits within the 5 word limit for slogans!). But, still doesn't feel quite right.

But... "Play to your outs"... doesn't seem at first glance to me like it really solves the problems in the middle? (#2, #3, #4, #5?). I feel like it encourages #5 (and Jeff's vague Q&A on "don't do that" doesn't seem reassuring to me). It also seems to encourage #3 (and again the vague admonishment to "not do that" doesn't seem that reassuring to me.)

I don't think "play to your outs" scales. When I imagine 100 people trying to do that who aren't part of a single company with shared leadership, I imagine them doing a bunch of random stuff that is often at cross-purposes.

Right now I feel like "play to your outs" gives me the illusion of having a strategy-and-life-philosophy, but doesn't feel like it directs my attention in particularly useful ways.

I don't have a great alternate slogan at the time, but I'm not sure the thing exactly bottlenecked personally on a slogan exactly.

(I'm not strongly confident it's not a good phrase here – If it turned out everyone ended up naturally doing sensible things when following it as a strategy/life-philosophy, coolio I guess. I see some people have responded positively to the post so far. I'm worried those people are responding more to the fact that someone gave them something less uncomfortable/depressing than 'die with dignity' to orient around, rather than actually getting directed in a useful way)

I like this post. I think that if Eliezer had written "MIRI announces new 'Play To Your Outs' strategy", it wouldn't have helped with some specific goals he has:

• Get people to face up to grim reality so that they are less tempted to violate ethical injunctions.
• Get people to face up to grim reality so that they are less tempted to self-deceive or quietly condition on a bunch of optimistic assumptions.

But given that someone's already read the "Death With Dignity" thing and internalized it as a reason not to do that stuff, I do expect that a lot of people will do better work if they're thinking in terms of "play to your outs" on a daily basis. (Not everyone, but a lot of people.)

Like, for me personally, I think it was important that I go through something like a period of mourning and acceptance. (The big one I remember was in October 2017.) But I also don't think that a "mourning and acceptance" vibe is the right air to be breathing hour-to-hour as I get work tasks done. (Again, speaking for me personally.)

It seems to me that it would be better to view the question as "is this frame the best one for person X?" rather than "is this frame the best one?"

Though, I haven't fully read either of your posts, so excuse any mistakes/confusion.

I strongly prefer the "dying with dignity" mentality for 3 basic reasons:

• as other people have mentioned, "playing to your outs" is too easy to misinterpret as conditioning on comfortable improbabilities no matter how much you try to draw the distinctions
• relatedly, focusing on "playing to your outs" (especially if you do so for emotional reasons) may make it harder to stay grounded in accurate models of reality (that may mostly output "we will die soon")
• Operating under the mindset that death is likely when AGI is still some ways around the corner and easy to ignore seems like it ought to make it easier to stay emotionally resilient and ready to exploit miracle opportunities if/when AGI is looming and impossible to ignore

Of these, the 3rd feels the most important to me, partly because I've seen it discussed least. It seems like if Eliezer's basic model is right, a significant portion of the good outcomes require some kind of miracle occuring at crunch time, which will presumably be easier to obtain if key players are emotionally prepared and not suddenly freaking out for the first time (on an emotional/subconscious level). I know basically nothing about psychology, but isn't it a bad sign if you retreat to "oh death with dignity is unmotivating, let's just focus on our outs" when AGI is less salient?

Just a dumb question, why do you/Eliezer talk about the log-odds and not just probability? Is there any reason behind this word choice?

Besides this, I'm fully on board, I enjoy this framing much more than the "die with dignity"

I don't think it's quite the same thing, but another relevant framing is to increase your luck surface area.

Even given the clarifications in this post, "playing to your outs" seems like it comes with some subtle denial. My sense is that it's resisting a pending emotional update?

I'm curious if this resonates.

I do like the framing of "playing to your outs" for various reasons. I'm not sure how I feel about which framing is better, but here are two thoughts.

1. Eliezer wanted [LW · GW] to post this on April Fools Day. But if it's an April Fools Day post, "playing to your outs" isn't very April Fools-y, while "death with dignity" (sorta) is.
2. The Death With Dignity post feels to me like it spends a decent amount of time recommending against doing unethical stuff, ie. the talk about consequentialism and second-order effects. If that's something Eliezer wanted to emphasize, "dignity" does seem like a good word. In your Q3 you question this premise that we need to have this conversation about unethical stuff in the first place. I don't feel strongly here, but I lean away from this. I lean towards thinking that it's important to note.

A problem with this framing that's related to, but I think not the same as, the one pointed out by Joe_Collman:

"Play to your outs" suggests focusing on this one goal even if what you have to do to have a hope of meeting that goal have other very bad consequences. This makes sense in many games, where the worst win is better than the best not-win. It may not make sense in actual life.

If you reckon some drastic or expensive action makes a 2% improvement to our chances of not getting wiped out by a super-powerful unaligned AI, then that improvement might be worth a lot of pain. If you reckon the same action makes a 0.02% improvement to our chances, the tradeoff against other considerations may look quite different.

But if you frame things so that your only goal is "win if possible" at the game of not getting wiped out by a super-powerful unaligned AI, and if either of those is the best "out" available, then "play to your outs" says you should do it.

Joe points out that you might be wrong about it being the best "out". I am pointing out that if it is your best "out" then whether it's a good idea may depend on how good an "out" it is, and that the "play to your outs" framing discourages thinking about that and considering the possibility that it might not be good enough to be worth playing to.

[EDITED to add:] Q3 at the end of the post is relevant, but I don't think the answer given invalidates what I'm saying. It may very well be true that most crazy/violent plans are bad, but that's the wrong question; the right question is whether the highest-probability plans (in the scenario where all those probabilities are very low) are crazy/violent.

Playing to your outs seems potentially VNM-consistent in ways the log-odds approach doesn't https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy?commentId=cWaoFBM79atwnCadX [LW(p) · GW(p)]

"playing to your outs" might be more commonly known as running a Hail Mary.

I like this framing so, so much more. Thank you for putting some feelings I vaguely sensed, but didn't quite grasp yet, into concrete terms.

This framing feels like a much more motivating strategy, even though it's pretty much identical to what Eliezer is proposing.

The distinction between your post and Eliezer's is more or less that he doesn't trust anyone to identify or think sanely about [plans that they admit have negative expected value in terms of log odds but believe possess a compensatory advantage in probability of success conditional on some assumption].

Such plans are very likely to hurt the remaining opportunities in the worlds where the assumption doesn't hold, which makes it especially bad if different actors are committing to different plans. And he thinks that even if a plan's assumptions hold, the odds of its success are far lower than the planner envisioned.

Eliezer's preferred strategy at this point is to continue doing the kind of AI Safety work that doesn't blow up if assumptions aren't met, and if enough of that work is complete and there's an unexpected affordance for applying that kind of work to realistic AIs, then there's a theoretical possibility of capitalizing on it. (But, well, you see how pessimistic he's become if he thinks that's both the best shot we have and also probability ~0.)

And he wanted to put a roadblock in front of this specific well-intentioned framing, not least because it is way too easy for some readers to round into support for Leeroy Jenkins strategies.

I do not find this inspiring.

Likewise.

In the normal usage, dying with dignity is about accepting your death and not fighting the inevitable. I do not find this inspiring.

It also might read as being about stuff like cryopreservation though.

Some possible outs and opportunities: Some semi-strong AI is deployed in a dangerous and very damaging Headline way a few before AGI is developed, allowing for a tiny sliver of a chance to rein the research sector with whatever the best ideas are for doing so.

A new Luddite movement (it could be us?) slows down research for an extra 10 years through sheer shouting, agitation, and political dealing, allowing for possibilities and ideas that might be helpful.

Well-defined safety guidelines which are enforced on research and actually stops the creation of anything not provably safe (which we somehow figure out).

I think another framing is anthropic-principle optimization; aim for the best human experiences in the universes that humans are left in. This could be strict EA conditioned on the event that unfriendly AGI doesn't happen or perhaps something even weirder dependent on the anthropic principle. Regardless, dying only happens in some branches of the multiverse so those deaths can be dignified which will presumably increase the odds of non-dying also being dignified because the outcomes spring from the same goals and strategies.

Not sure if it counts as an "out" (given I think it's actually quite promising), but definitely something that should be tried before the end:

“To the extent we can identify the smartest people on the planet, we would be a really pathetic civilization were we not willing to offer them NBA-level salaries to work on alignment.” - Tomás B. [LW(p) · GW(p)]

Megastar salaries for AI alignment work [EA(p) · GW(p)] 

[Summary from the FTX Project Ideas competition]

Aligning future superhuman AI systems is arguably the most difficult problem currently facing humanity; and the most important. In order to solve it, we need all the help we can get from the very best and brightest. To the extent that we can identify the absolute most intelligent, most capable, and most qualified people on the planet – think Fields Medalists, Nobel Prize winners, foremost champions of intellectual competition, the most sought-after engineers – we aim to offer them salaries competitive with top sportspeople, actors and music artists to work on the problem. This is complementary to our AI alignment prizes, in that getting paid is not dependent on results. The pay is for devoting a significant amount of full time work (say a year), and maximum brainpower, to the problem; with the hope that highly promising directions in the pursuit of a full solution will be forthcoming. We will aim to provide access to top AI alignment researchers for guidance, affiliation with top-tier universities, and an exclusive retreat house and office for fellows of this program to use, if so desired.

[Yes, this is the "pay Terry Tao $10M" thing. FAQ in a GDoc here.]

If trying to invent ever more powerful AI is too dangerous, the focus should temporarily be shifted to the REAL goal. That would be to invent a solution to the problem of human death. More specifically a method to "back up" the mind contents of a human brain. Like, if you can back up your life's work onto a USB, why not your memories and personality, at least somewhat? Figuring out how to do such a thing may or may not require godlike superhuman AI.