Applied Rationality Practice

I'm going to suggest some initial steps from Hammond, Keeney, and Raiffa's book, Smart Choices. The first step is problem definition, which I think accords with the advice discussed on this website to hold off on proposing solutions until you've examined the problem thoroughly. Here are some of their ideas:

• Be creative about your problem definition; don't necessarily state it the first way it pops into your mind. Is the problem simply about mitigating existential risk? Or is it about giving your life purpose and meaning? Or is it that you want to prolong your own life as long as you can, and anything that poses a risk to the entire human race also poses a risk to you? Be honest here; don't give the socially acceptable answer if it's not the real answer.

• Ask what triggered the decision. Why are you even considering it? This can provide a link to the essential problem, but beware of letting it lock you into thinking about the problem only in the way it first occurred to you. It sounds like the college issue, and perhaps the whole "what am I going to do with my life" question, is what prompted the decision. So is the problem really "where do I take my life next?", or "where should I go to college?", or "what career should I follow", with the issue of existential risk being an important consideration?

• Question the constraints in your problem statement. Is going to college necessary for you? Can you find any other hidden constraints or assumptions in your problem statement?

• Identify the essential elements. What are possible threats to human existence? Or, if this is about "what should I do with my life", figure out what your core values are, what makes you feel fulfilled, what things have you done in the past that made you feel good about yourself, etc.

• Understand what other decisions hinge on this decision. You've already identified college major and choice of university as decisions that hinge on how you are going to contribute to mitigating existential risk.

• Establish a sufficient but workable scope for your problem definition. In particular, make it clear what is not part of this decision, so as not to muddy the waters.

• Gain fresh insights by asking others how they see the situation. You're already doing this.

After problem definition, the next step is to clarify your objectives. What do you really want? What do you hope for? In the terminology of decision analysis, this is the step where you try to understand the rough outlines of your utility function, as it relates to your decision problem. Without this understanding you can't effectively compare and evaluate the alternatives.

You may have multiple objectives, and there may be some conflict between them. Don't let that stop you from listing all of them; you'll deal with any necessary tradeoffs in a later step. Don't rush this step, and don't settle for the immediate, obvious answers; figuring out what you really want is harder than you might think.

Here are some of HKR's thoughts on tackling this step:

• Objectives help you determine what information to seek. You'll generally seek information for two reasons: (1) to suggest alternatives to consider, and (2) to help you evaluate those alternatives. Knowing your objectives tells you what information you need for purpose (2).

• Often decision makers take too narrow a focus. If your list of objectives is brief and cursory, you may find after making your decision that you left out important considerations. Don't concentrate solely on the tangible and quantitative (lifespan, income, etc.); make sure you consider the intangible and subjective also (Is this fun? Is this something I really care about?)

• Don't limit your objectives by the ease of measurement. Income may be easier to measure than happiness, but the latter is important nonetheless.

• If a prospective decision sits uncomfortably in your mind, you may have overlooked an important objective. For example, suppose that you were considering taking a particular job and that it looked great according to all of your criteria: good current income, prospects for advancement, opportunities to learn and expand your skills, etc. If you still feel uneasy about it, figure out why. Does it require you to live in a location that you find unappealing? Does something about the people there or the work environment bother you? The answers to such questions will point to possible additional objectives you should include.

AKR suggest the following five-step process for eliciting your objectives:

1. Write down all the concerns you hope to address through your decision. Put together a wish list. Consider a great, if infeasible, alternative; what's so great about it? Consider a terrible alternative; why is it so bad? Ask people who have faced similar situations what they considered when they made their decision -- not what they chose, nor the choice they recommend for you, but what criteria and issues they considered.

2. Convert your concerns into succinct objectives. A short phrase consisting of an verb and an object works well, e.g., "enjoy life", "save humanity", "live forever".

3. Separate ends from means to establish your fundamental objectives. You may have heard these discussed as instrumental values vs. terminal values; Eliezer goes into great detail on the subject here. For example, when it comes to personal objectives, money itself is usually not a fundamental objective; it is, instead, a means of obtaining other things that you really care about (e.g., food, shelter, amusement) or that themselves are means (e.g., status) to getting other things you really care about (e.g., sex). A good exercise is to take each objective and ask why you want it; take the answer and ask why you want that; and continue until you arrive at something that needs no further justification -- you value it for itself. (This is similar to The 5 Whys, but aimed at discovering root values rather than root causes.) Further notes: The means objectives can stimulate ideas for alternatives to consider, and help you understand the problem better. The fundamental objectives -- and only the fundamental objectives -- are used to evaluate and compare alternatives.

4. Clarify what you mean by each objective. Ask, "what do I really mean by this?" Latch onto any fuzziness or ambiguity in the objective and resolve it. Imagine that you're explaining your objectives to someone who's being a bit obtuse.

5. Test your objectives. Compare several alternatives; does the one that comes up best, according to your stated objectives, feel like it could be better than the others? A second test is to see if your objectives would suffice to explain a prospective decision to someone else. (Imagine this person to be entirely non-judgmental so that you aren't tempted to be less than honest about your true objectives.) If you feel that A is a better choice than B, but you can't adequately explain why in terms of your stated objectives, then you may be missing something.

At this point you have a lot of introspection to carry out. You may find it useful to think out loud (or, more accurately, in print) with us. You don't have to achieve perfection here, but I would urge you to carefully think through these first two steps before proceeding further. However, there may be some things that you just can't get right on the first pass, and will only realize are important after you have proceeded further through the decision process. That's okay; you can make this an iterative process, returning to earlier steps when necessary.

This sounds like a good idea. I'll start.

Okay, first off, what existential risks do you think are most likely? Nuclear war, natural pandemic, engineered pandemic, UFAI, nanotech, climate change, or something else?

Second question: what are your talents? Computer science, biology, chemistry, writing, politics/persuading people, or something else? Especially note talents/skills that could make you a lot of money.

Once you've answered one or both of those, I'll know where we can go from there.

Also, I feel like many of the posts tagged "decision" or "decision theory" are more about logical AIs in contrived situations than the type of decision making we're trying to do here. Take the prisoner's dilemma for example.There are more than 2 individual playing in real life There are no arbitrary rules in real life. When does game theory apply in my every day life (and in trying to make this decision?)?