The Sin of Underconfidence

I wonder if the decline of apprenticeships has made overconfidence and underconfidence more common and more severe.

I'm not a history expert, but it seems to me that a blacksmith's apprentice 700 years ago wouldn't have had to worry about over/underconfidence in his skill. (Gender-neutral pronouns intentionally not used here!) He would have known exactly how skilled he was by comparing himself to his master every day, and his master's skill would have been a known quantity, since his master had been accepted by a guild of mutually recognized masters.

Nowadays, because of several factors, calibrating your judgement of your skill seems to be a lot harder. Our education system is completely different, and regardless of whatever else it does, it doesn't seem to be very good at providing reliable feedback to its students, who properly understand the importance of the feedback and respond accordingly. Our blacksmith's apprentice (let's call him John) knows when he's screwed up - the sword or whatever that he's made breaks, or his master points out how it's flawed. And John knows why this is important - if he doesn't fix the problem, he's not going to be able to earn a living.

Whereas a modern schoolkid (let's call him Jaden) may be absolutely unprepared to deal with math, but he doesn't know exactly how many years he's behind (it's hard enough to get this information in aggregate, and it seems to be rarely provided to the students themselves on an individual basis - no one is told "you are 3 years behind where you ought to be"). And Jaden has absolutely no clue why that matters, since the link between math and his future employment isn't obvious to him, and no one's explaining it to him. (School isn't for learning; as Paul Graham has explained, "Officially the purpose of schools is to teach kids. In fact their primary purpose is to keep kids locked up in one place for a big chunk of the day so adults can get things done. And I have no problem with this: in a specialized industrial society, it would be a disaster to have kids running around loose.")

Another modern schoolkid (let's call her Jaina) may be really skilled at math, but testing won't indicate this strongly enough (it works both ways; tests saturate at the high end - especially if they're targeting a low level of achievement for the rest of the class - and "you are 3 years ahead of everyone else in this room" is not feedback that is commonly given). And there's a good chance it won't be obvious to her how important this is, and how important becoming even more skilled is. And if she ends up being underconfident in her ability, and the feedback loop ("I know how skilled I am, I know why becoming stronger is important, and I know what I need to do") isn't established, then instead of learning plasma physics and working on ITER or DEMO, she goes into marketing or something. Maybe doing worthy things, but not being as awesome as she could have been.

My point, after this wondering, is that I agree with this post, and want to elaborate: structuring what you do so that you test yourself in the process of doing it is a good way to establish a feedback loop that increases your skill and the accuracy of your confidence in it. I find nothing wrong with the debating example in this post, but I worry that it makes self-testing sound like something that you should go out and do, separate from your everyday work. (Part of this, I think, is due to Eliezer's very unusual occupation.) My usual self-testing example is something like "can I write this program correctly on the very first try?". That's a hard challenge, integrated into my everyday work. Successfully completing it, or coming close, has allowed me to build up my skill ("the compiler in my head") and avoid the danger of underconfidence.

it was said by them that Christopher Hitchens should have watched the theist's earlier debates and been prepared, so I decided not to do that

I urge you to prepare properly. Not only Hitchens but Richard Carrier and several other atheists have been humiliated in debate with him, by their own admission. Winning at all is challenge enough, and would be a great service to the world. Given how much of a blow you would find it to lose having fully prepared, I urge you to to reconsider whether you're self-handicapping.

Unfair debate proposal

You want a debate in which the tables are tilted against you? I see a way to do that which doesn't carry the risks of your current proposal.

A bunch of us get together on an IRC channel and agree to debate you. We thrash out our initial serve; we then spring the topic and our initial serve on you. You must counter immediately, with no time to prepare. We then go away and mull over your counter, and agree a response, which you must again immediately respond to.

We can give ourselves more speaking time than you in each exchange, too, if you want to tilt the tables further (I'm imagining the actual serves and responses being delivered as video).

I've found some of the characterizations of Craig's arguments and debate style baffling.

When he debates the existence of god, he always delivers the same five arguments (technically, it's four: his fifth claim is that god can be known directly, independently of any argument). He develops these arguments as carefully as time allows, and defends each of his premises. He uses the kalam cosmological argument, the fine tuning argument, the moral argument, and the argument from the resurrection of Jesus. This can hardly be characterized as dumping.

Also, his arguments are logically valid; you won't see any, 'brain teaser, therefore god!' moves from him. He's not only a 'theologian'; he's a trained philosopher (he actually has two earned PHDs, one in philosophy and one in theology).

Finally, Craig is at his best when it comes to his responses. He is extremely quick, and is very adept at both responding to criticisms of his arguments, and at taking his opponent's arguments apart.

Debating William Lane Craig on the topic of god's existence without preparation would be as ill advised as taking on a well trained UFC fighter in the octagon without preparation. To extend the analogy further, it would be like thinking it's a good idea because you've won a couple of street fights and want to test yourself.

It sounds as though you're viewing the debate as a chance to test your own abilities at improvisational performance. That's the wrong goal. Your goal should be to win.

"The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means. Whenever you parry, hit, spring, strike or touch the enemy’s cutting sword, you must cut the enemy in the same movement. It is essential to attain this. If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him. More than anything, you must be thinking of carrying your movement through to cutting him."

By increasing the challenge the way you suggest, you may very well be acting rationally toward the goal of testing yourself, but you're not doing all you can to cut the opponent. To rationally pursue winning the debate, there's no excuse for not doing your research.

In choosing not to try for that, you'll end up sending the message that rationalists don't play to win. You and I know this isn't quite accurate -- what you're doing is more like a rationalist choosing to lose a board game, because that served some other, real purpose of his -- but that is still how it will come across. Do you consider this to be acceptable?

Cultivate a habit of confronting challenges - not the ones that can kill you outright, perhaps, but perhaps ones that can potentially humiliate you.

You may be interested to learn that high-end mountaineers apply exactly the strategy you describe to challenges that might kill them outright. Mick Fowler even states it explicitly in his autobiography - "success every time implies that one's objectives are not challenging enough".

A large part of mountaineering appears to be about identifying the precise point where your situation will become unrecoverable, and then backing off just before you reach it. On the other hand, sometimes you just get unlucky.

A slogan I like is that failure is OK, so long as you don't stop trying to avoid it.

While reading this post, a connection with Beware of Other-Optimizing clicked in my mind. Different aspiring rationalists are (more) susceptible to different failure modes. From Eliezer's previous writings it had generally seemed like he was more worried about the problem of standards (for oneself) that are too low -- that is, not being afraid enough of failure -- than about the opposite error, standards that are too high. But I suspect that's largely specific to him; others may need to worry more about being too afraid of failure. Hence I'm happy to see this post.

And so I wrote at once to the Bloggingheads folks and asked if they could arrange a debate. This seemed like someone I wanted to test myself against. Also, it was said by them that Christopher Hitchens should have watched the theist's earlier debates and been prepared, so I decided not to do that, because I think I should be able to handle damn near anything on the fly, and I desire to learn whether this thought is correct; and I am willing to risk public humiliation to find out.

This really bothers me, because you weren't just risking your own public humiliation; you were risking our public humiliation. You were endangering an important cause for your personal benefit.

There are three great besetting sins of rationalists in particular, and the third of these is underconfidence.

Were we ever told the other two?

gjm asks wisely:

What would you think of a musician who decided to give a public performance without so much as looking at the piece she was going to play? Would you not be inclined to say: "It's all very well to test yourself, but please do it in private"?

The central thrust of Eliezer's post is a true and important elaboration of his concept of improper humility, but doesn't it overlook a clear and simple political reality? There are reputational effects to public failure. It seems clear that those reputational effects often outweigh whatever utility is gained from an empirical "test" of one's own abilities: this is why international relations theory isn't a rigorous empirical science. We live in an irrational kaleidescope of power, driven by instinct and emotion, ordered only fleetingly by rhetoric and guile. In this situation, we need to keep our cards close to our chest if we want to win.

Mulciber adds something along the same lines:

By increasing the challenge the way you suggest, you may very well be acting rationally toward the goal of testing yourself, but you're not doing all you can to cut the opponent. To rationally pursue winning the debate, there's no excuse for not doing your research.

And Eliezer does seem to approve of this mode of thinking in some cases:

Of course this is only a way to think when you really are confronting a challenge just to test yourself, and not because you have to win at any cost. In that case you make everything as easy for yourself as possible. To do otherwise would be spectacular overconfidence, even if you're playing tic-tac-toe against a three-year-old.

So, to sum up my concern, how is this principle of pragmatism reconciled to your choice not to prepare? Isn't it best to test yourself in the peace and safety of your dojo, or in circumstances where the stakes are not high, and use every means available to resist on the actual field of battle?

We have lots of experimental data showing overconfidence; what experimental data show a consistent underconfidence, in a way that a person could use that data to correct their error? This would be a lot more persuasive to me than the mere hypothetical possibility of underconfidence.

I skimmed several debates with WLC yesterday, referenced here. His arguments are largely based on one and the same scheme:

1. Everythng must have a cause
2. Here's a philosophical paradox for you, that can't be resolved within the world
3. Since despite the paradox, some fact still holds, it must be caused by God, from outside the world

(Or something like this, the step 3 is a bit more subtle than I made it out to be.) What's remarkable, even though he uses a nontrivial number of paradoxes for the step 2, almost all of them were explicitly explained in the material on Overcoming Bias. At least, I was never confused while listening to his arguments, whereas some of his opponents were, on some of the arguments. I don't see WLC as possessing magical oratorial skills, but he bends the facts on occasion, and is very careful in what he says. Also, his presentations are too debugged to be alive, so it looks unnatural.

The general meta-counterargument would be to break this scheme, as he could present some paradox (e.g. anthropics) without clear known resolution, and through it bend his line. I'm sure he knows lots of paradoxes, so there is a real danger of encountering an unknown one.

He knows Bayesian math. On one occasion, he basically replied to a statement that there is no evidence for God that it's only relevant if you expect more evidence for God if it exists, as opposed to if it doesn't, and if you expect no evidence in both cases, this fact can't be lowered a priori probability. This, of course, contradicts the rest of his arguments, but I guess he'll say that those arguments are some different kind of evidence.

There is a children's puzzle which consists of 15 numbered square blocks arranged in a frame large enough to hold 16, four by four, leaving one empty space. You can't take the blocks out of the frame. You can only slide a block into the empty space from an adjacent position. The puzzle is to bring the blocks into some particular arrangement.

The mathematics of which arrangements are accessible from which others is not important here. The key thing is that no matter how you move the blocks around, there is always an empty space. Wherever the space is, you can always move a block into it, but however fast you move the blocks, they never fill the frame.

I have not heard of this theologian before, but ciphergoth's description of him firing off piles of pet arguments faster than you can point to the holes does suggest the sliding block metaphor, but he's playing with a much larger set of blocks. At any rate, it is utterly unlike the sedate, civilised pursuit of truth observed on Bloggingheads. It is theatre addressed to the audience, not the antagonist. As far as he is concerned, you would just be one of his supporting cast.

If a debate is arranged, I second the advice to prepare as well as possible, attending not just to the specific arguments he uses and how others fared against them, but also the theatrics. It may help that Bloggingheads does not have a live audience. I am 90% sure that if he agrees to debate you, he will ask to do so in front of one.

This post reminds me of Aristotle's heuristics for approaching the mean when one tends towards the extremes:

"That moral virtue is a mean, then, and in what sense it is so, and that it is a mean between two vices, the one involving excess, the other deficiency, and that it is such because its character is to aim at what is intermediate in passions and in actions, has been sufficiently stated. Hence also it is no easy task to be good. For in everything it is no easy task to find the middle, e.g. to find the middle of a circle is not for every one but for him who knows; so, too, any one can get angry- that is easy- or give or spend money; but to do this to the right person, to the right extent, at the right time, with the right motive, and in the right way, that is not for every one, nor is it easy; wherefore goodness is both rare and laudable and noble.

Hence he who aims at the intermediate must first depart from what is the more contrary to it, as Calypso advises-

Hold the ship out beyond that surf and spray.

For of the extremes one is more erroneous, one less so; therefore, since to hit the mean is hard in the extreme, we must as a second best, as people say, take the least of the evils; and this will be done best in the way we describe. But we must consider the things towards which we ourselves also are easily carried away; for some of us tend to one thing, some to another; and this will be recognizable from the pleasure and the pain we feel. We must drag ourselves away to the contrary extreme; for we shall get into the intermediate state by drawing well away from error, as people do in straightening sticks that are bent." (NE, II.9)

Did you write a cost function down for the various debate outcomes? The skew will inform whether overconfidence or underconfidence should be weighted differently.

Eliezer, does your respect for Aumann's theorem incline you to reconsider, given how many commenters think you should thoroughly prepare for this debate?

What is the danger of overconfidence?

Passing up opportunities. Not doing thing you could have done, but didn't try (hard enough).

Did you mean "danger of underconfidence"?

the abuse of doubt

New link [LW · GW]

the improper use of humility

New link [LW · GW]

Can anyone give some examples of being underconfident, that happened as a result of overcorrecting for overconfidence?

Eliezer said:

So if you have learned a thousand ways that humans fall into error and read a hundred experimental results in which anonymous subjects are humiliated of their overconfidence - heck, even if you've just read a couple of dozen - and you don't know exactly how overconfident you are - then yes, you might genuinely be in danger of nudging yourself a step too far down.

I also observed this phenomenon of debiasing being over-emphasized in discussions of rationality, while heuristic is treated as a bad word. I tried to get at the problem of passing up opportunities you mention when I said in my post on heuristic: "It's a mistake in cartography to have areas of your map that are filled in wrong, but it's also a mistake to have areas on your map blank that you could have filled in, at least with something approximate".

I think we need more success stories of human heuristic. Currently, the glut of information on biases and faulty heuristic is making these more cognitively available, leading to underconfidence.

Of course, it's easier to measure the gravity of mistakes of overconfidence, because we know the bad outcome, and we can speculate that it would have been avoided without the overconfidence. Yet in the case of mistakes of underconfidence, we don't know what we are missing out on, what brilliant theories were prematurely discarded, and what groundbreaking inventions were never created, because the creators (or their colleagues, investors, advisors, professors, whoever) were underconfident.

Yet we can look at examples of great discoveries, ideas, solutions, practices, and what what our lives, or the world, would be like if they had got nipped in the bud. Furthermore, there may be cases where two people (say, scientists or entrepreneurs) were both acquainted with the same evidence or theory, yet only one was confident enough about it to capitalize on it.

Playing tic-tac-toe against a three-year-old for the fate of the world would actually be a really harrowing experience. The space of possible moves is small enough that he's reasonably likely to force a draw just by acting randomly.

Can someone explain why we can't name the theist in question, other than sheer silliness?

This post reminds me of the phrase "cognitive hyper-humility," used by Ben Kovitz's Sophistry Wiki:

Demand for justification before making a move. Of course, this is not always sophistry. In some special areas of life, such as courtroom trials, we demand that a "burden" of specific kinds of evidence be met as a precondition for taking some action. Sophistry tends to extend this need for justification far beyond the areas where it's feasible and useful. Skeptical sophistry tends to push a sort of cognitive hyper-humility, or freezing out of fear of ever being "wrong"--or even being right but not fully justified. If you were to reason as the skeptic suggests that you should reason, you'd never be able to do anything in real life, because you'd never have sufficiently articulated and proven a priori principles to get started, nor evidence to justify your actions according to those principles, nor time to think this stuff through to the demanded degree.

If there was a message I could send back to my younger self this would be it. Plus that if it's hard, don't try to make it easier, just keep in mind that it's important. (By younger self, I mean 7-34 years old.)

IHAPMOE, but the post seems to assume that a person's "rationality" is a float rather than a vector. If you're going to try to calibrate your "rationality", you'd better try to figure out what the different categories of rationality problems there are, and how well rationality on one category of problems correlates with rationality on other categories. Otherwise you'll end up doing something like having greater confidence in your ethical judgements because you do well at sudoku.

Does the fact that I find this guy's formulation of the cosmological argument somewhat persuasive mean that I can't hang out with the cool kids anymore? I'm not saying it is an airtight argument, just that it isn't obviously meaningless or ridiculous metaphysics.

Slightly off-topic:

I don't know if it would be possible to arrange either of them, but there are two debates I'd love to see Eliezer in:

A debate with Amanda Marcotte on evolutionary psychology

and

A debate with Alonzo Fyfe on meta-ethics.

And conversely, as Ari observes:

If you’ve never hit the ground while skydiving, you’re opening your parachute too early.

This seems like a reflection of a general problem people have, the problem of not getting things done - more specifically, the problem of not getting things done by convincing yourself not to do them.

It's so much easier to NOT do things than do them, so we're constantly on the lookout for ways not to do them. Of course, we feel bad if we simply don't do them, so we first have to come up with elaborate reasons why it's ok - "I'll have plenty of time to do it later", "There's too much uncertainty", "I already got alot of work done today", etc. The underconfidence you're describing seems like another attempt at this rather than a peculiar habit of rationalists.

I try to fight this, semi-successfully, by remembering that it's only the RESULT that matters. If I want something, it doesn't matter what clever words or arguments I make to myself about doing it; in the end I either get it or I don't. And there's certainly nothing rational about convincing yourself to not get something you want; rationalists WIN, after all.

Eliezer should write a self-help book! Blog posts like the above are very inspiring to this perennial intellectual slacker and general underachiever (meaning: me).

I certainly can relate to this part:

"It doesn't seem worthwhile any more, to go on trying to fix one thing when there are a dozen other things that will still be wrong...

There's not enough hope of triumph to inspire you to try hard..."

Last paragraph, open parentheses missing. (I'm on a typo roll it seems)

Overconfidence is usually costlier than underconfidence. The cost to become completely accurate is often greater than the benefit of being slightly-inaccurate-but-close-enough.

When these two principles are taken into account, underconfidence becomes an excellent strategy. It also leaves potential in reserve in case of emergencies. As being accurately-confident tends to let others know what you can do, it's often desirable to create a false appearance.

A typo in the article: "What is the danger of overconfidence?" -> "What is the danger of underconfidence?"

...the third of these is underconfidence. Michael Vassar regularly accuses me of this sin, which makes him unique among the entire population of the Earth.

Well, that sure is odd. Guess that's why Vassar was promoted. It makes sense now.

Anyway, EY's history doesn't seem to me marked by much underconfidence. For example, his name has recently been used in vain at this silly blog, where they're dredging up all sorts of amusing material that seems to support the opposite conclusion.

Since I know EY has guru status around here, please don't jump down my throat. For the record, I agree with everything he says. I must, for the force of his rationality encircles me and compels me.

Anyway, for those who don't want to follow the link, here's the best part -- a bit of pasted materials in a comment by someone named jimf:

When Ayn [Rand] announced proudly, as she often did, 'I can account for every emotion I have' -- she meant, astonishingly, that the total contents of her subconscious mind were instantly available to her conscious mind, that all of her emotions had resulted from deliberate acts of rational thought, and that she could name the thinking that had led her to each feeling. And she maintained that every human being is able, if he chooses to work at the job of identifying the source of his emotions, ultimately

to arrive at the same clarity and control.

Barbara Branden, The Passion of Ayn Rand pp. 193 - 195

From a transhumanist acquaintance I once corresponded with:

Jim, dammit, I really wish you'd start with the assumption that I have a superhuman self-awareness and understanding of ethics, because, dammit, I do.