On Doing the Improbable

The times I was able to get people to do things that they felt were too unlikely to commit to were largely about lowering the emotional costs of failure. The context is a bit different, but it seems likely that some of the same factors apply.

Using “writing HPMoR” as an example, there’s more than one thing failure could be taken to mean. One is “I tested a high risk high reward idea, and it didn’t pan out. I learned something useful about what kinds of things I can’t do (right away, at least), and it still strikes me as having been worth attempting, given what I knew at the time. If I keep trying high risk high reward ideas one of them is likely to pay out, because the idea that I’m limited by what social expectations would see as “modest” isn’t even worth taking seriously”. A completely different thing it could mean is “I was arrogant to think I had a chance at this. I learned nothing on the object level because I already knew I couldn’t do it, but on the meta level I learned that I was wrong to set this aside and hope. In hindsight, it was a mistake that never was worth trying in the first place, and if I keep trying high risk high rewards things I’m just going to keep failing because social expectations of what I’m capable of are *right*”. The people with the latter anticipation are going to be less thrilled about flipping that coin with a 50% chance of success because the other 50% hurts a lot more.

The former mindset *sounds* a lot better, and people are going to want to say “yeah, that one sounds right! I believe *that* one!” even when their private thoughts tend towards the latter mindset. If you try to get someone in the latter category to act like they’re in the former category, you’re going to run into motivation problems. You’re going to hear “You’re right, and I want to… I just can’t find the motivation”.

In order to get people to shift from “failure means I should be less confident and try less” to “failure means this particular one didn’t pan out, and it’s still worth trying more”, you have to be able to engage with (and pass the ‘ideological turing test’ of) their impulses to take failure as indicative of a larger problem. There is definitely a skill to this, and it can be tough when you can plainly see that the right answer is to “just try it”. At the same time, it’s a skill that can be learned and it does work for opening things up for change.

I recently got a chance to interview a couple people about this who'd done product management or similar at bay area tech companies.

They agreed that you can't run projects there unless you project near-certainty the project will succeed. However, they had a trick that had failed to occur to me prior to them saying it, which is to find a mid-scale objective that is all of: a) quite likely to have at least a bit of use in its own right; b) almost certainly do-able; and c) a stepping-stone for getting closer to the (more worthwhile but higher-failure-odds) goal. And work towards your harder goal via stepping-stones such as these, when doing product management for groups.

I'll be trying this out, probably. It reminds me of building modular code instead of spaghetti code.

Less confidently, I'd like to lightly recommend reading this blog post about Christopher Alexander's notions of how to make buildings into places where humans like to be, and about analogous ideas for how to do good software design. I don't have concrete take-aways to point to from that one, so read it at your own risk or don't, but it seems to me there may be patterns like this to how to make goals and subgoals that groups can have morale/orientation while building.

Looking at my own experience, the thing that motivated me to do things likely to fail is the expectation of getting other benefits, even if it failed. One such thing is "experience", but it could also be "it'll be fun" or "attempting will give you status even if you fail" or any number of other things.

Or, if there is feedback after relatively little effort (you find out after the first few chapters if people like it).

There's just something about "work hard for an extended period of time with no feedback until you find out if you won, which is a binary event with low odds" that turns people off, I guess.

I'm a bit surprised that most of the previous discussion here was focused on the "okay, so how do you actually motivate people?" aspect of this post.

This post gave me a fairly strong "sit bolt upright in alarm" experience because of it's implications on epistemics, and I think those implications are sneaky and far reaching. I expect this phenomenon to influence people's ability to think and communicate, before you get to the point where you actually have a project that people are hitting the hard parts of. 

People form models of what sort of things they can achieve, and what sort of projects to start, and how likely their friends are to succeed at things, and (I expect) this backpropagates through their entire information system.

The problem may not just affect the people on a given project – it could affect the people nearby (and the people nearby them in turn), in terms of what sort of feedback you can easily give each other. 

I've struggled with deciding whether to give people feedback that I don't think their project is likely to work, when I nonetheless think the project is the right thing to be working on, either for EV reasons or longterm-growth-as-a-person reasons, i.e. their next project will benefit from the skills they're gaining here. I know there are some people who'd lean hard into the "information is a key bottleneck, definitely don't withhold info like that." And they may be right. But if so, that still leaves us with a deep, crucial question of how to integrate epistemics and actual success on hard, long problems that lets you actually win at the instrumental part.

(I don't think the "pay people a lot" is actually sufficient, although it obviously helps. I think lack-of-clear-path-to-victory can be demoralizing even if you have plenty of money – that seems to be where a lot of 20th century ennui comes from. Also, many of the key projects here are early-stage where it's just hard to convince people to give you enough money to escape scarcity mindset)

I notice I don't have a very explicit model of what's going on – this comment felt more like I was ranting than clearly explaining things, not sure how it comes across to others. 

I hope to flesh this out (and hopefully make some empirical predictions) during the review phase.

Hmm, I worry that motivation is only part of the picture. There's also idiosyncrasy between agents in terms of ability and acceptance of outcome.

By this I mean that they would not have stuck to writing the Sequences or HPMOR or working on AGI alignment past the first few months of real difficulty [LW · GW],

True enough, though you can replace "would not" with "will not" or "did not".

without assigning odds in the vicinity of 10x what I started out assigning that the project would work

This is not the counterfactual I'd assign the most weight to. "Without being Eliezer" is probably too specific, but "without having Eleizer's history of rewarded iconoclasm" wouldn't be a stretch. It's extremely likely that you _ARE_ orders of magnitude more likely to succeed at these endeavors than the majority of people who say they're interested in the topic.

I'm painfully familiar with the issue of lack of group participation, since I can't even get people to show up to a meetup.

Because of that, I've been doing research on identifying the factors contributing towards this issue and how to possibly mitigate them. I'm not sure if any of this will be new to you, but it might spark more discussion.
These are the first ideas that come to mind:

1. For people to be intrinsically motivated to do something, the process of working on it has to be fun or fulfilling.

2. Extrinsic motivation, as you say, requires either money or a reason to believe the effort will accomplish more than other uses of one's time would. If it's a long-term project, the problem of hyperbolic discounting may lead people to watch TV or [insert procrastination technique here] instead, even if they think the project is likely to succeed.

3. If people already have a habit of performing an activity, then it takes less effort for them to participate in similar activities and they demand less benefit from doing so. Identifying habits that are useful for the task you have in mind can be tricky if it's a complex issue, but successfully doing so can keep productivity consistent and reduce its mental cost.

4. Building a habit requires either intense and consistent motivation, or very small steps that build confidence. Again, though, identifying very small steps that still make for good productivity early on may be tricky.

5. If you have trouble getting people to start joining, it may be good to seek out early adopters to provide social proof. However, the social proof may only work for people who are familiar with those specific early adopters and who take cues from them. In that case, you may need to find some regular early adopters and then identify trendsetters in society (habitual early adopters from whom many people take their cues) you could get on board, after which their followers will consider participating. (Then the danger becomes making sure that the participants understand the project, but at least you have more people to choose from.)

6. It may help to remind people from time to time what they're working towards, even though everyone already knows. Being able to celebrate successes and take some time to review the vision can go quite a ways in relieving stress when people start to feel like their work isn't rewarding.

From item 1, if people think they can get a benefit from working on a project even if the project fails, they might be willing to participate. Socializing with project members and forming personal relationships with them may help in this respect, since they'll enjoy working with people. Alternatively, you could emphasize the skills they'll pick up along the way.

From item 4, I've been working on 'mini-habits" (a concept I got from Stephen Guise) to lower my own mental costs for doing things, and it seems to be working fairly well. Then the trick becomes getting enough buy-in (per item 5) so you can get other people started on those mini-habits.

There are probably some other factors I'm overlooking at the moment. Since I haven't been able to get results yet, I can't say for sure what will work, but I hope this provides a helpful starting point for framing the problem.

But… what’s wrong with this?

If:

(a) Someone is “not personally madly driven to accomplish the thing”, and

(b) They are not being paid money to do it, and

(c) They don’t think it’s pretty likely to succeed…

… then… why should they keep working on it?

This is a very general question, and I'm not sure there's good general answers that don't make use of the specific types involved, such as 'convincing people who like doing X to do X is easier'. (As opposed to 'hard workers work harder/longer, so get hard workers.')

In absence of an answer, I have speculation.

People don't want to do new things.

Getting started is the hardest part of doing anything.

The end of a project is the hardest part.

Maybe people value successes - not in the sense of risk aversion, but maybe they want to a certain number of expected successes? .7+.7=1.4, so 2 things with a 70% chance of success, gets you an expected success.

Maybe people want a certain fraction of the projects they do to succeed? This would makes things difficult.

Maybe it's the type of thing, or something about it (doesn't sound interesting, requires acquiring new skills, or takes too long to produce value*).

*As compared to this. and things like it.

What are you trying to get people to do?

This post has influenced my evaluations of what I am doing in practice by forcing me to consider lowering the bar for expected success for high return activities. Despite "knowing" about how to shut up and multiply, and needing to expect a high failure rate if taking reasonable levels of risk, I didn't consciously place enough weight on these. This helped move me more in that direction, which has led to both an increased number of failures to get what I hoped, and a number of mostly unexpected successes when applying for / requesting / attempting things.

It is worth noting that I still need to work on the reaction I have to failing at these low cost, high-risk activities. I sometimes have a significant emotional reaction to failing, which is especially problematic because the emotional reaction to failing at a long-shot can influence my mood for multiple days or weeks afterwards.

I love how cleanly this brings up its point and asks the question. My answer is essentially that you can do this if and only if you can create expectation of Successful Failure in some way. Thus, if failing person's real mission can be the friends they made along the way or skills they developed or lessons learned, or they still got a healthy paycheck, or the attempt brings them honor, or whatever, that's huge.

Writing a full response is on my list of things to eventually do, which is rare for posts that are over a year old.

Focusing on the chance of success may be missing other factors. You need to also consider their belief in the chance that the problem is real and their belief that someone else won't solve it before them.

Someone who has, say a 30% belief in their ability to contribute to solving a problem will have very different motivations if they're 90% vs 10% confidant that the problem is real (and important) and if they have a 90/10% belief that someone else is going to solve it first.

I doubt it's as simple as multiplying those estimates, but that's an ok place to start.

From my experience with working with groups I've found a few things to work, all of them revolve around incentivizing or promoting positive feedback loops. This includes: bribery/payment (while most people think of this as being with money, since I work with 10-17 year olds it's usually candy or pizza), lowering the bar so that small bits of progress are celebrated greatly, making the actual task less boring (turning on some music, working with friendly people, making it look pretty), humor, taking breaks to promote sanity, having a comfortable working environment etc.

Luke already wrote [LW · GW] that there are at least four factors that feed motivation, and the expectation of success is only one of them. No amount of expectancy can increment drive if other factors are lacking, and as Eliezer notice, it's not sane to expect only one factor to be 10x the others so that it alone powers the engine.

What Eliezer is asking is basicall if anyone has solved the basic coordination problem of mankind, and I think he knows very well that the answer to his question is no. Also, because we are operating in a relatively small mindspace (humans' system 1), the fact that no one solved that problem in hundreds of thousands of years of cooperation points strongly toward the fact that such a solution doesn't exist.

What really helps is mortality and our inbred need to leave a legacy. It is better to pick a project with low probability of success than none at all. That can help you stick with something you only estimate to have a low chance of success, at least long enough to have sunk costs kick in. Does for me anyway.

This mechanism may only work for one man projects, or work in tight knit groups like bands of musicians. Your contribution to a big project doesn't feel like a legacy to the same degree.

The core idea of this post has occurred to me more than once when considering plans. I'm still working on how to relate to plans with low chances of success. On the one hand low chances of success suggest a bad plan, and being willing to "do the improbable" feels like it's an excuse for having a bad plan. On the other, sometimes you really want to be doing low probability, high EV plans. I'm uncertain over whether LW counts as this. Sometimes I think we can definitely succeed at big stuff, sometimes it more seems like high-EV low probability. I'm not sure.

But all in all, the idea here seems important to think more about. I'd actually like to see more thought on this (and perhaps do my own stuff here).

Also want to note that Anna's post on "going at half-speed" [LW · GW] seems at least a bit related.

Maybe it would be more fruitful to find the people who do want to cooperate than it is to convert the people who don't?

You should distinguish between "accomplishing one big thing after lots of work" (like writing HPMoR) and "accomplishing one tenth of similar goals after lots of work" (like having some kind of applications approved). The first is hard and usually just not something people find they must do. I think that's part of the problem. If everyone else gets by without writing HPMoR, surely I can. The second is annoying, but you can have a drink and forget about needing to be motivated and just keep at it.

For me, it would be hard even if I believed it necessary etc., because I cannot work alone (I tried). What I do must matter to someone "now", and in a very specific way. Cheering me only made me feel uncomfortable, even if it was deserved. People cheer you on when they want something from you, not just when they are happy for your sake, so after you hear the 'good job' part you wait for the other shoe to drop. Someone finding small bugs in what I do helps much more.