Urges vs. Goals: The analogy to anticipation and belief

I have also found Eat That Frog to be an unusually good collection of the major productivity techniques. Incidentally, I also heard about the book from Patri via Divia.

For a shorter and more rationality-friendly version of the book, I summarized it here:

EDIT: http://becomingeden.com/summary-of-eat-that-frog/

this analogy is arguably obvious, but it's deep, useful, and easy to overlook in its implications - there seem to be two major kinds of wanting:

and

Obvious but it needs to be said: People are as bad at looking into hospitals for their own health as for the sake of their parents' health.

I found neither of these things the least bit obvious. I hadn't realized Implication 4 until I had been reading Less Wrong for many months and it was not obvious in retrospect. I hadn't even considered the distinction between urges and goals at all, though it did seem obvious in retrospect - only in retrospect.

I say this because I have had a ton of trouble grasping the concept that things that are obvious to me aren't necessarily obvious to other people.

(Though I don't want to make the same mistake and assume that other people also have this problem.)

I really have seen multiple people (some of whom I significantly cared about) malfunctioning as a result of misinterpreting this point. As a stand-alone system for pulling your actions, urges have all kinds of problems. Urges can pull you to stare at an attractive stranger, to walk to the fridge, and even to sprint hard for first base when playing baseball. But unless coupled with goals and far-mode reasoning, urges will not pull you to the component tasks required for any longer-term goods. When I get into my car I have a definite urge for it not to be broken. But absent planning, there would never be a moment when the activity I most desired was to take my car for an oil change. To find and keep a job (let alone a good job), live in a non-pigsty, or learn any skills that are not immediately rewarding, you will probably need goals. Even though human goals can easily turn into fashion statements and wishful thinking.

I sort of run this way. Contrary to the description, though, I sometimes do get urges to clean, do laundry, etc. This usually occurs when I happen to be annoyed by the feel of dirt on my bare feet, or find my clothes hamper full, or some other stimulus triggers the behavior. Incidentally, I also am the one in my family who takes the cars for oil changes.

On the other hand, I also have no job. I have a hard time acting on anything that I don't have an urge to do; fortunately or unfortunately, I also have parents to provide me with reasons to have urges to do things I wouldn't otherwise have an urge to do. (This might also be why I once said I didn't have an understanding of how people did things they didn't feel like doing, because the process I use to decide what activity to do at any given moment seems to consist of weighing my various urges in order to figure out what it is that I "feel like" doing, and then doing it, which is a process that almost entirely relies on emotional/unconscious processes rather than conscious verbal reasoning.)

I think I might be living by urges alone. Whenever I see something about "goals" or "self-discipline" or "self-improvement" I immediately shut down and get miserable. My brain says "I don't want to, dammit!" Of course, people tell me I am self-disciplined, but I see that as merely being practical; if it makes any sense, I'm willing to be practical but severely freaked out by aspirational or normative thinking.

Andrew Critch's "greedy algorithm": Whenever you catch yourself really wanting to do something you want to want, immediately reward yourself - by feeding yourself an M&M, or if that's too difficult, immediately pumping your fist and saying "Yes!"

I have been doing this deliberately for a few months because I was starting to get fed up with fighting my instincts every time I chose to program for an hour and wanted to spend that hour reading science fiction, so I actually started standing and exercising to watch anime or read books I did not really need to, while rewarding myself for getting something done that was relatively high up my priority list by sitting / eating food I liked.

It seems like no matter how I try to rebalance the incentives and consequences, negative stimuli always seem to have a much stronger effect on my behaviour however, so although I get more studying done now I think it comes more from the significant increases in my perceived free time than from reward mechanisms.

• Anticipations, what we actually expect to see happen;
• Professed beliefs, the set of things we tell ourselves we “believe”, based partly on deliberate/verbal thought.

This distinction helps explain how an atheistic rationalist can still get spooked in a haunted house;

I apologize if this seems nitpicky, but the implication seems to be that in Yvain's post he is merely "professing" to not believe in ghosts, but "anticipating" that they exist. I believe the actual point of the post was that Yvain both professes and anticipates the nonexistence of ghosts (hence his willingness to place a bet with the bookie as he flees the mansion), he simply hasn't internalized this anticipation/belief on a gut-level.

Porting this back to Urges vs. Goals, perhaps the analogy shouldn't be "Urges : Anticipations", but instead "Urges : Gut-level Internalizations".

Kaj forms the generalization "as soon as my brain adopted a cause, my subconscious reinterpreted it as the goal of giving the impression of doing prestigious work for the cause". I worry that our community has a tendency to explain as e.g. status signaling or "people really don't care about X", observations that can also be explained by less malice/selfishness and more "our brains have known malfunctions at linking goals to urges".

I actually agree with this, and have somewhat changed my mind about the explanation in my original post, but I just haven't had the time to write about it. Hopefully this week.

EDIT: Done.

Andrew Critch's "greedy algorithm": Whenever you catch yourself really wanting to do something you want to want, immediately reward yourself - by feeding yourself an M&M, or if that's too difficult, immediately pumping your fist and saying "Yes!"

I'm adopting this. Could someone point me to the source? I tried to google for Andrew Critch's "greedy algorithm" but haven't found anything except this LW post. Update: Sent a PM to Andrew, asked for more details.

Update 2:

I tried this for a while but alas, it didn't stick - I couldn't find a trigger for this habit. To use programming terms, my brain never fires an event "you're wanting to do something you want to want", so I don't have a place to put the rewarding "habit-code" into.

On the other hand, I do notice when I actually do what I want to want to do, so I started rewarding myself for this, both mentally and physically (e.g. by grabbing some nuts / dried berries from the bowl, taking a good break, bragging to coworkers etc.)

Great post! "Aspiring consequentialist" has a nice ring to it.

I just realized I also have "Send Carl & Anna their wedding gift" on my to-do list.

We know a thing or two about the neurobiology behind the divide between urges and goals. Those interested can read about it here.

More generally, for the basic decision-making tools we have a collection (automatic application, automatic correction, deliberative application, deliberative correction). For goals, that's (wanting, liking, approving, approving of approving); for beliefs, (anticipation, learning/surprise, professed belief, correspondence with referent (taskian truth)).

For example, correcting wrong belief in belief (professed belief) that doesn't reflect more accurate anticipation then corresponds to getting rid of fake professed utility functions that don't reflect the actual detailed values. Both are errors of mishandling the tools for deliberative reasoning (about beliefs and goals), and fixing both of these errors should in theory improve the quality of one's decisions (or of theorizing about decision-making).

Andrew Critch's "greedy algorithm": Whenever you catch yourself really wanting to do something you want to want, immediately reward yourself - by feeding yourself an M&M, or if that's too difficult, immediately pumping your fist and saying "Yes!"

Closely related to the parenting advice of "Catch them being good" - which works wonders on kids. I expect it will generalize well to adults.

A lot of our community technique goes into either (1) dealing with "beliefs" being an evolutionarily recent system, such that our "beliefs" often end up far screwier than our actual anticipations; or (2) trying to get our anticipations to align with more evidence-informed beliefs.

Wow. I hadn't heard this expressed quite like this before... We have one territory and two maps, and we can help get both maps in sync with reality by getting them both in sync with each other.

Is (2) related to taking ideas seriously?

To me there seems to be more going on here than in Belief in Belief where Yudkowsky encourages people to do (1) - to update vacuous or nonsensical professed beliefs. From Yudkowsky's article I didn't pick up that (2) needs to happen as well.

And much the same way that a lot of craziness stems, not so much from "having a wrong model of the world", as "not bothering to have a model of the world", a lot of personal effectiveness isn't so much about "having the right goals" as "bothering to have goals at all" - where unpacking this somewhat Vassarian statement would lead us to ideas like "bothering to have something that I check my actions' consequences against, never mind whether or not it's the right thing" or "bothering to have some communication-related urge that animates my writing when I write, instead of just sitting down to log a certain number of writing hours during which I feel rewarded from rearranging shiny words".

Simple, useful preaching.

Thank you for this post. The central insight that we should consider instrumental rationality by analogy to epistemic rationality is something that has never occured to me before. I wish I had thought of it.

Besides an aspiring rationalist, these days I call myself an "aspiring consequentialist".

I think I'll do that too.

I think "goals" are the wrong way to look at it.

Very few people have a complete, coherent system of terminal values. The few who do usually seem to suffer from their excessive rigidity. I can't commit to an exhaustive set of goals, all the way to the end of my life. I've had to discard and change my plans too many times. What looks like a great idea today may turn out to be fruitless on inspection.

Instead of goals I think about resources. I don't know specifically what I'm going to want to do, but whatever it is, money will be helpful. As will health. And human capital. And a track record of accomplishments.

When a goal isn't a resource, it usually turns out to be a fake goal. "I want to learn Latin." Well, if learning Latin isn't going to help me with anything, then guess what? I'll never get around to learning Latin. It's just an empty aspiration. Learning awk, however, isn't an empty aspiration -- I learned the hard way how badly I need it. Awk is a resource.

It's not a perfect distinction. Sometimes a resource looks like a mere aspiration because its usefulness is too far away in time. Asking "is this a resource?" will tend to bias you towards what you need most now. But it's not a bad algorithm. (I think of it as gradient ascent.) You get done what you need to do, and you worry a lot less about what you vaguely think you should be doing.

He personally had the experience of believing "If the last day where I remember having gone to bed was a Tuesday today shouldn't be Monday but Wednesday". Before the belief got challenged by hard reality I have never paid any conscious attention to the belief. Getting it challenged on the other hand produced one of the three stongest feelings of cognitive dissonce that I felt in my life.

We all have a bunch of beliefs which a very reasonable but for which they are edge cases where the beliefs don't hold.

I think the common term for those beliefs is "common sense".

Then there a other little area of believes that we commonly call "perception". If I see a red carpet I believe that the carpet is red. It doesn't necessarily have to be and there a psychological tricks that one can use to give people false perceptions.

I strongly endorse your second and fourth points; thanks for posting this. They're related to Yvain's post Would Your Real Preferences Please Stand Up?.

This should be on the front page.

When I was in San Francisco, I recall the phrase "goals not roles" popping up a lot.

I find that it's a fairly easy way to remember that it's even a question whether I'm trying to accomplish something, or just do some things that make it look like I'm trying to accomplish it.

Important and timely (the next Melbourne LW meetup will focus on setting good goals, an exercise which has always confounded me).

I find particularly interesting the "wedding gift todo" example, where imagined achievement of the goal stands-in for actually achieving the stated goal (giving a wedding gift). We want to have and act on "goals" rather than "urges". But setting goals is the kind of activity where "urges" can dominate. To me this looks like the analogue of belief-in-belief. We want our reasoning processes to be reflexively consistent, but in practice they often fail to work that way.

Edit: And when I go back and look at "Belief in Belief", that's where Eliezer outlines the "invisible dragon" example, so my main point is already implicit in this post!

Well it seems to me that a rational goal professed by a rationalist should correspond to a few anticipations (that goal is achievable, that achieving the goal will achieve some rational super goal, serve an urge, or otherwise be positive). Not an analogy but a straightforward correspondence.

Unless of course one adopts goals of the form - suppose I am the leader of the tribe and you are regular member and I tell you to defend this hill. Or vice versa. And we adopt defending of the hill as a goal without any knowledge as to why we are defending this hill and how is that linked to the urge to live or other urges. Such knowledge may be hard to convey and requirement of communication of such knowledge would easily be a handicap (especially if the language is not very well developed).

Likewise for the passed down generations goal-set of the tribe, which itself is an entity that can evolve.

I imagine that sort of thing happened a lot all the way to couple millions years back.

Detachment of goals allows us to have all sorts of screwy goals ranging from more noble varieties like pursuing higher education (which dramatically decreases the reproduction rate) to really bad ones (suicidal attacks). [Note that from evolutionary point of view those goals are all rather screwy in so much that they don't improve reproduction rate of any particular gene]

I imagine that if biological evolution would be allowed to continue for a million years or two we would have more intelligent human with precisely 1 strong urge - to reproduce - everything else being derived as goals rationally. But as of now we got the urge system of a monkey that won't be able to arrive at any action or decision starting from such high level goal as reproduction.

Visualizing that accomplishment, and its positive rewarding consequences, until you have an urge for it to happen

I so have to try this hack. No agency without urgency?

This fits in reasonably well with an anti-akrasia framework I've been thinking over: Rephrase goal X as "I honestly believe that I will achieve X", and then carry on thinking until you actually have a reasonably solid case for believing that. This particular trick translates to breaking down the statement into "I will force myself to develop an urge to do X. And once I have an urge to do X I will surely do X because I tend to follow my urges".

"Status" / "prestige" / "signaling" / "people don't really care about" is way overused to explain goal-urge delinkages that can be more simply explained by "humans are not agents".

I think that those are sort of describing the same thing but at different levels of abstraction. Thinking in terms of evolutionary adaptations, our professed goals and actual behavior may differ for status/signaling reasons. But thinking about cognition we just come to the conclusion that we're not very agenty and aren't so bothered about the evolutionary reason why.

I agree with your article. I think that this example doesn't quite illustrate it:

Before you install Linux, do you think "What's the positive consequence of installing Linux?" or does it just seem like the sort of thing a free-software-supporter would do?

The first few times I did this, it was the second motivation. After a while, it became the first, namely that I got a system I had better control over, incorporating high-quality software. However, the first motivation was very good for the second. Without (lots of) people doing the sort of thing a free-software supporter would do, most of the good software I use out of work would not be available to me.

Your sentence has a strong/weak dichotomy whereas I think either motivation is acceptable.

Urges vary in strength, but it isn't usual to speak of one goal being stronger than another—except in the sense that it's powered by more urges. But goals, too, would seem to vary in strength. A goal's strength would bear some relationship to the expected value of striving to attain it.

You overstate the disconnection between urges and goals because you don't consider the consequences of goals having intrinsic strength, apart from their extrinsic association with urges. A stronger goal exerts a stronger pull to recruit urges to its service. Unless we're neurotic, we don't typically ignore our strongest goals because of a dearth of supporting urges.

Thought provoking post.

I got a lot out of this post, and it's obviously very high quality, but I have one humble gripe.

"and why Eliezer had to actually crash a car before he viscerally understood what his physics books tried to tell him about stopping distance going up with the square of driving speed. (I helped Anna revise this - EY.)"

I feel as if the parenthetical statement at the end of the quoted text would be unnecessarily alienating to an outside reader. Maybe it's that it feels unprofessional (I'm not really sure), but it seems like the kind of thing that would seem weird to me if I were reading this after having it sent to me by a friend. And this is definitely the kind of post I would send to a friend.

In other news, I agree vehemently about your point in the first footnote. I'll definitely start using the "chess player" example when I'm explaining belief in belief to others.

My workplace seems, at times, to be well-designed to align my urges and goals for me.

(also: Congratulations, Anna and Carl on your wedding!)

I love this post's anticipations : professed beliefs :: urges : professed goals. Planning seems more necessary (although I guess it's actually rare) than talking about your beliefs (which is easy to do to excess).

All the other cases involve not asking it, or not asking hard enough.

"the cases" is unclear. I assume you mean the rest of the "ways to screw up in choosing goals" yet to be listed.

Ideals can be more ungoaly because they're sometimes about faraway things or less ancestral things - it's probably easier to improve your agency on less idealy goals that link more quickly to urges -

Redundant and opaque. "idealy" and "ungoaly" aren't words. The entire paragraph states a simple point (that I agree with): we shouldn't automatically give up on the goals where agency is hardest - it may be worth making progress toward those goals, even if it's expensive, or incomplete.

The Inuit may not have 47 words for snow

The Inuit does not have 47 words for snow! Please, don't propagate this falsehood, especially on a 'rationality' blog.

Edit: Sorry I read incorrectly. My apologies! It says 'may not'...