The Power of Agency

I radically distrust the message of this short piece. It's a positive affirmation for "rationalists" of the contemporary sort who want to use brain science to become super-achievers. The paragraph itemizing the powers of agency especially reads like wishful thinking: Just pay a little more attention to small matters like fixity of purpose and actually acting in your own interest, and you'll get to be famous, rich, and a historical figure! Sorry, that is entirely not ruthless enough. You also need to be willing to lie, cheat, steal, kill, use people, betray them. (Wishes can come true, but they usually exact a price. ) It also helps to be chronically unhappy, if it will serve to motivate your extreme and unrelenting efforts. And finally, most forms of achievement do require domain-specific expertise; you don't get to the top just by looking pretty and statusful.

The messy, inconsistent, and equivocating aspects of the mind can also be adaptive. They can save you from fanaticism, lack of perspective, and self-deception. How often do situations really permit a calculation of expected utility? All these rationalist techniques themselves are fuel for rationalization: I'm employing all the special heuristics and psychological tricks, so I must be doing the right thing. I've been so focused lately, my life breakthrough must be just around the corner.

It's funny that here, the use of reason has become synonymous with "winning" and the successful achievement of plans, when historically, the use of reason was thought to promote detachment from life and a moderation of emotional extremes, especially in the face of failure.

Even a little bit of agenty-ness will have some lasting historical impact. Think of Benjamin Franklin, Teddy Roosevelt, Bill Clinton, or Tim Ferris.

Can you go into more detail about how you believe these particular people behaved more agenty than normal?

In case anybody asks how I was able to research and write two posts from scratch today:

It's largely because I've had Ray Lynch's 'The Oh of Pleasure' on continuous repeat ever since 7am, without a break.

(I'm so not kidding. Ask Jasen Murray.)

real agent with the power to reliably do things it believed would fulfill its desires. It could change its diet, work out each morning, and maximize its health and physical attractiveness. It could learn body language and fashion and salesmanship and seduction and the laws of money and win in every sphere of life without constant defeat by human hangups. It could learn networking and influence and persuasion and have large-scale effects on societies, cultures, and nations.

A lot of body language, fashion, salesmanship, seduction, networking, influence, and persuasion are dependent entirely on heuristics and intuition.

In the real world those that have less access to these traits (being people of the autistic spectrum, for example) tend to have a much harder time learning how to accomplish any of the named tasks. They also, for most of those tasks, have a much harder time seeing why one would wish to accomplish those tasks.

Extrapolating to a being that has absolutely no such intuition or heuristics then one is left with the question of what it is that they wish to actually do? Perhaps some of the severely autistic really are like this and never learn language as it never occurs to them that language could be useful and so have no desire to learn language.

With no built in programing to determine what is to be desired and what is not to be desired and no built in programing as to how the world works or does not work then how is one to determine what should be desirable or how to accomplish what is desired? As far as I can determine an agent without human hardware or software may be left spending its time attempting to figure out how anything works and figuring out what, if anything, it wants to do.

It may not even attempt to figure out anything at all if Curiosity is not rational but a built in heuristic. Perhaps someone has managed to build a rational AI but has neglected to give it built in desires and/or built in Curiosity and it did nothing so was assumed to not have worked.

Isn't even the desire to survive a heuristic?

You do not have the robust character you think you have, but instead are blown about by the winds of circumstance.

Am I wrong in taking this to be a one-liner critique of all virtue ethical theories?

I've been thinking about this with regards to Less Wrong culture. I had pictured your "deliberative thinking" module as more of an "excuse generator" - the rest of your mind would make its decisions, and then the excuse generator comes up with an explanation for them.

The excuse generator is primarily social - it will build excuses which are appropriate to the culture it is in. So in a rationalist culture, it will come up with rationalizing excuses. It can be exposed to a lot of memes, parrot them back and reason using them without actually affecting your behavior in any way at all.

Just sometimes though, the excuse generator will fail and send a signal back to the rest of the mind that it really needs to change something, else it will face social consequences.

The thing is, I don't feel that this stuff is new. But try and point it out to anyone, and they will generate excuses as to why it doesn't matter, or why everyone lacks the power of agency except them, or that it's an interesting question they'll get around to looking at sometime.

So currently I'm a bit stuck.

I loved this, but I'm not here to contribute bland praise. I'm here to point out somebody who does, in fact, behave as an agent as defined by the italicized statement: "reliably do things it believed would fulfill its desires" that continues with "It could change its diet, work out each morning, and maximize its health and physical attractiveness. " I couldn't help but think of Scott H Young, a blogger I've been following for months. I really look up to that guy. He is effectively a paragon of the model that you can shape your life to live it as you like. (I'm sure he would never say that though.) He actually referenced a Less Wrong article recently, and it's not the first time he's done it, which significantly increased my opinion of him. His current "thing" is trying to master the equivalent of a rigorous CS curriculum (using MIT's requirements) in 12 months. Only those on the Less Wrong community stand a good chance of not thinking that's pretty audacious.

Coming back to this post, I feel like it's selling a dream that promises too much. I've come to think of such dreams as Marlboro Country ads. For every person who gets inspired to change, ten others will be slightly harmed because it's another standard they can't achieve, even if they buy what you're selling. Figuring out more realistic promises would do us all a lot of good.

Excellent clarion call to raise our expectation of what agency is and can do in our lives, as well as to have sensible expectations of our and others' humble default states. Well done.

One way of thinking about this:

There is behavior, which is anything an animal with a nervous system does with its voluntary musculature. Everything you do all day is behavior.

Then there are choices, which are behaviors you take because you think they will bring about an outcome you desire. (Forget about utility functions -- I'm not sure all human desires can be described by one twice-differentiable convex function. Just think about actions taken to fulfill desires or values.) Not all behaviors are choices. In fact it's easy to go through a day without making any choices at all. Mostly by following habits or instinctive reactions.

In classical economics, all behaviors are modeled as choices. That's not true of people in practice, but possibly some people choose a higher percentage of their behaviors than other people do. Maybe it's possible to train yourself to make more of your behaviors into choices. (In fact, just learning Econ 101 made me more inclined to consciously choose my behaviors.)

I want to upvote this again.

It might be simply structural that the LessWrong community tends to be about armchair philosophy, science, and math. If there are people who have read through Less Wrong, absorbed its worldview, and gone out to "just do something", then they probably aren't spending their time bragging about it here. If it looks like no one here is doing any useful work, that could really just be sampling bias.

Even still, I expect that most posters here are more interested to read, learn, and chat than to thoroughly change who they are and what they do. Reading, learning, and chatting is fun! Thorough self-modification is scary.

Thorough and rapid self-modification, on the basis of things you've read on a website rather than things you've seen tested and proven in combination, is downright dangerous. Try things, but try them gradually.

And now, refutation!

So what's the solution?

To, um, what, exactly? I think the question whose solution you're describing is "What ought one do?" Of these, you say:

It wont feel like the right thing to do; your moral intuitions (being designed to operate in a small community of hunter gatherers) are unlikely to suggest to you anything near the optimal task.

That depends largely on your moral intuitions. I honestly think of all humans as people. I am always taken aback a little when I see evidence that lots of other folks don't. You'd think I stop being surprised, but it often catches me when I'm not expecting it. I'd suggest that my intuitions about my morals when I'm planning things are actually pretty good.

That said, the salient intuitions in an emotionally-charged situation certainly are bad at planning and optimization. And so, if you imagine yourself executing your plan, I would honestly expect it to feel oddly amoral. It won't feel wrong, necessarily, but it might not feel relevant to morality at all.

It will be something you can start working on right now, immediately.

This is ... sort of true, depending on what you mean. You might need to learn more, to be able to form a more efficient or more coherent plan. You might need to sleep right now. But, yes, you can prepare to prepare to prepare to change the world right away.

It will disregard arbitrary self-limitations like abstaining from politics or keeping yourself aligned with a community of family and friends.

Staying aligned with a community of family and friends is not an arbitrary limitation. Humans are social beings. I myself am strongly introverted, but I also know that my overall mood is affected strongly by my emotional security in my social status. I can reflect on this fact, and I can mitigate its negative consequences, but it would be madness to just ignore it. In my case - and, I presume, in the case of anyone else who worries about being aligned with their family and friends - it's terrifying to imagine undermining many of those relationships.

You need people that you can trust for deep, personal conversations; and you need people who would support you if your life went suddenly wrong. You may not need these things as insurance, you may not need to use friends and family in this way, but you certainly need them for your own psychological well-being. Being depressed makes one significantly less effective at achieving one's goals, and we monkeys are depressed without close ties to other monkeys.

On the other hand, harmless-seeming deviations probably won't undermine those relationships; they're far less likely to ruin relationships than they seem. Rather, they make you a more interesting person to talk to. Still, it is a terrible idea to carelessly antagonize your closest people.

Speaking about it would undermine your reputation through signaling. A true rationalist has no need for humility, sentimental empathy, or the absurdity heuristic.

No! If we're defining a "true rationalist" as some mythical entity, then probably so. If we want to make "true rationalists" out of humans, no! If you completely disregard common social graces like the outward appearance of humility, you will have real trouble coordinating world-changing efforts. I you disregard empathy for, say, people you're talking to, you will seem rather more like a monster than a trustworthy leader. And if you ever think you're unaffected by the absurdity heuristic, you're almost certainly wrong.

People are not perfect agents, optimizing their goals. People are made out of meat. We can change what we do, reflect on what we think, and learn better how to use the brains we've got. But the vast majority of what goes on in your head is not, not, not under your control.

Which brings me to the really horrifying undercurrent of your post, which is why I stayed up an extra hour to write this comment. I mean, you can sit down and make plans for what you'll learn, what you'll do, and how you'll save billions of lives, and that's pretty awesome. I heartily approve! You can even figure out what you need to learn to decide the best courses of action, set plans to learn that, and get started immediately. Great!

But if you do all this without considering seemingly unimportant details, like having fun with friends and occasionally relaxing, then you will fail. Not only will you fail, but you will fail spectacularly. You will overstress yourself, burn out, and probably ruin your motivation to change the world. Don't go be a "rationalist" martyr, it won't work very well.

So, if you're going to decompartmentalize your global aspirations and your local life, then keep in mind that only you are likely to look out for your own well-being. That well-being has a strong effect on how effective you can be. So much so that attempting more than about 4 hours per day of real, closely-focused mental effort will probably give you not just diminishing returns, but worse efficiency per day. That said, almost nobody puts in 4 hours a day of intense focus.

So, yes, billions are miserable, people die needlessly, and the world is mad. I am still going out tomorrow night and playing board games with friends, and I do not feel guilty about this.

The real question is: how big of an impact can this stuff make, anyway? And how much are people able to actually implement it into their lives?

Are there any good sources of data on that? Beyond PUA, The Game, etc?

Think of Benjamin Franklin, Teddy Roosevelt, Bill Clinton, or Tim Ferris.

It's Tim Ferriss.

People don't change their sense of agency because they read a blog post.

"In alien hand syndrome, the afflicted individual’s limb will produce meaningful behaviors without the intention of the subject. The affected limb effectively demonstrates ‘a will of its own.’ The sense of agency does not emerge in conjunction with the overt appearance of the purposeful act even though the sense of ownership in relationship to the body part is maintained. This phenomenon corresponds with an impairment in the premotor mechanism manifested temporally by the appearance of the readiness potential (see section on the Neuroscience of Free Will above) recordable on the scalp several hundred milliseconds before the overt appearance of a spontaneous willed movement. Using functional magnetic resonance imaging with specialized multivariate analyses to study the temporal dimension in the activation of the cortical network associated with voluntary movement in human subjects, an anterior-to-posterior sequential activation process beginning in the supplementary motor area on the medial surface of the frontal lobe and progressing to the primary motor cortex and then to parietal cortex has been observed.[167] The sense of agency thus appears to normally emerge in conjunction with this orderly sequential network activation incorporating premotor association cortices together with primary motor cortex. In particular, the supplementary motor complex on the medial surface of the frontal lobe appears to activate prior to primary motor cortex presumably in associated with a preparatory pre-movement process. In a recent study using functional magnetic resonance imaging, alien movements were characterized by a relatively isolated activation of the primary motor cortex contralateral to the alien hand, while voluntary movements of the same body part included the concomitant activation of motor association cortex associated with the premotor process.[168] The clinical definition requires “feeling that one limb is foreign or has a will of its own, together with observable involuntary motor activity” (emphasis in original).[169] This syndrome is often a result of damage to the corpus callosum, either when it is severed to treat intractable epilepsy or due to a stroke. The standard neurological explanation is that the felt will reported by the speaking left hemisphere does not correspond with the actions performed by the non-speaking right hemisphere, thus suggesting that the two hemispheres may have independent senses of will.[170][171]

Similarly, one of the most important (“first rank”) diagnostic symptoms of schizophrenia is the delusion of being controlled by an external force.[172] People with schizophrenia will sometimes report that, although they are acting in the world, they did not initiate, or will, the particular actions they performed. This is sometimes likened to being a robot controlled by someone else. Although the neural mechanisms of schizophrenia are not yet clear, one influential hypothesis is that there is a breakdown in brain systems that compare motor commands with the feedback received from the body (known as proprioception), leading to attendant hallucinations and delusions of control.[173]