The Wrongness Iceberg

post by alfredmacdonald · 2013-02-04T09:02:54.914Z · LW · GW · Legacy · 27 comments

As soon as I got out of college I got a job at a restaurant. At the time I had never had a job at a restaurant, but my mom had known the owners and I felt obligated to avoid performing badly. Yet inevitably I did perform badly, and how this performance was evaluated would greatly affect my way of perceiving my mistakes.

If you're entrenched in an organization, there's a good chance you have an idea of what it is you're supposed to do and what mistakes you will or will not be making. But suppose you're in a position like this one: by way of your ignorance you know you're going to make a lot of mistakes, and it's just a question of when and how much. Further, you know that if you make too many mistakes, you make people you care about look bad. And finally, there are a lot of unknown unknowns: you don't know what possible mistakes and acts of ignorance exist to begin with, so many mistakes you've made you will be blind to.

The proactive thing to do, naturally, is to try to minimize how many mistakes you make.

There are two key ways to gauge the depth of being told you have made a mistake. The first way is to take mistakes literally, as if no other mistake exists, and any other mistake would be pointed out to you. So if you correct this mistake, everything else should be fine. This is how you'd expect to take mistakes if you were, say, under the supervision of an editor.

But the second kind is where the title of this writeup comes in. Not everyone is literal, or critical enough to notice every mistake. Much of the time, you'll only receive news of a mistake if many other mistakes are already afoot, and this mistake just happens to stand out from the set of mistakes you've already made. And since you don't know what mistakes you could be making, you don't know if there are many more mistakes under your level of awareness that you could be correcting for, but aren't.

In short, you're tasked with avoiding a wrongness iceberg: a mistake indicative of a nautical mile of mistakes below the surface and your level of awareness.

This is a debilitating position to be in, because your mental map of your performance prior to discovering the iceberg needs to be completely rewritten; in addition to accounting for all of the new areas you need to work on, you will likely account for the embarrassment of realizing that you have opened up a new frontier of mistakes to reflect on from your period of unaware incompetence.

While I don't think it's impossible that people exist who have never been in a situation like this, I think anyone who dives into a new field or skill is familiar, at least, with this feeling of brief yet total incompetence. And if you're in a field with enough depth and subjective calls to allow for a wrongness iceberg scenario, there might not be much you can do to prevent it. The most you can do is provide adequate resistance for the inevitable.

That's why I've created this mental model to think about it constructively. In every situation where I've faced a wrongness iceberg, the anxiety has been catastrophic. If you can at least deal with it, you can realize why it is you're anxious and what's going on with your assessment of your own mistakes. From experience, knowing that I'm worried about making this kind of iceberg-revealing mistake is helpful for mitigating my stress. And if you can somehow preempt an iceberg, that's even better.

side note: I've extended this concept to other domains, and it works well. A "dishonesty iceberg" is when one person's lie reveals a nautical mile of lies below the surface, and an "attraction iceberg" is when one person's expression of attraction toward you are indicative of a much greater level of internal attraction.

27 comments

Comments sorted by top scores.

comment by buybuydandavis · 2013-02-04T10:32:29.458Z · LW(p) · GW(p)

I'd say you're doing this the wrong way. You're trying to do a mountain of inference on a mole hill of data. Take more data. This has been of my issues in life, and I've found life gets easier when I just ask. There is the concept of managing up - having to manage your manager, doing what you can to make sure your manager gets what they need from you, even if they don't know how to arrange that themselves. If they're incompetent, their incompetence is your problem.

First, you want to be adequately trained. Too many people focus on getting a list of what to do, instead of a list of the figures of merit. What are we trying to accomplish? What are our goals? What does a good job look like? What does a bad job look like? Stephen Covey distinguishes this as the difference between gopher delegation and stewardship delegation.

Second, instead of worrying about icebergs, ask for feedback, good and bad. People refrain from criticism often because they are uncomfortable giving it, and expect you to be uncomfortable getting it. They only bring it up when their annoyance with your failure overcomes their reticence about criticizing you. That's not going to be a fun talk. Preempt that, and make clear that you want to do a good job, and want feedback to improve. That way they can bring up issues before their annoyance has mounted. When they've done that once or twice and it's not a horrible experience, it gets easier for them to give you feedback in the future.

Replies from: Protagoras, ThrustVectoring
comment by Protagoras · 2013-02-04T16:21:34.957Z · LW(p) · GW(p)

I don't know if a restaurant is such a case (probably it isn't; it seems especially common when you work for managers who have important duties besides managing), but there are jobs where the most important thing you need to do to please the manager is make sure you don't do anything that needs the manager's attention, because they don't have time to deal with you. Asking the manager for advice in such a case is obviously something to be done rarely and carefully.

Replies from: aelephant
comment by aelephant · 2013-02-05T00:00:08.120Z · LW(p) · GW(p)

I think you could make the argument that a place where the manager doesn't have time to deal with the people they manage is a poorly managed place. Maybe they need to hire assistant managers or more employees to do the non-managerial work.

Replies from: prase
comment by prase · 2013-02-06T18:30:48.618Z · LW(p) · GW(p)

That may be true but the argument is of little use for the employee.

Replies from: aelephant
comment by aelephant · 2013-02-07T00:01:30.293Z · LW(p) · GW(p)

It might be useful for the employee in determining they no longer want to work for a poorly managed company.

Replies from: prase
comment by prase · 2013-02-07T23:44:37.879Z · LW(p) · GW(p)

If they don't want to. But as an employee, I care about things that influence me directly; if the company is poorly managed to some degree but offers good wages, I still want to work for them, at least until I find something better. Trying to judge the management quality doesn't seem to be a good employee strategy.

comment by ThrustVectoring · 2013-02-04T20:51:35.877Z · LW(p) · GW(p)

I've managed up with a bipolar pizza shop manager by getting myself a notebook and making explicit notes to reference later about how she wants certain cleaning tasks done. It really works.

Replies from: buybuydandavis
comment by buybuydandavis · 2013-02-05T00:16:09.972Z · LW(p) · GW(p)

I'm hopeless if I don't write things down.

Also, if you write a bunch of stuff down, give it back to your manager to verify. Lots of talking happens. Communication happens better through a document that both people can look at.

comment by fubarobfusco · 2013-02-04T16:15:49.769Z · LW(p) · GW(p)

Interesting model. Another one to be on the lookout for is the "socioeconomic iceberg" where a person's passing comment on some aspect of their life history reveals a large mass of socioeconomic difference from you.

comment by Qiaochu_Yuan · 2013-02-04T16:53:44.154Z · LW(p) · GW(p)

When I've been in situations like this in the past I've been way, way too hesitant to ask for help and advice. The proactive thing to do is to ask for help and advice (especially in situations you expect many people to have been in; this becomes less useful the more unique your situation is). This doesn't just mean asking your boss for help and advice as buybuydandavis suggests (although this is a good idea if your boss is willing to help and advise): it means asking

  • your friends who have had similar jobs
  • your mom if she has some insight into what the owners will and won't care about
  • the internet

It took me way too long to realize that the heuristic "consult domain experts if I am not a domain expert" is a great heuristic. One reason is probably that asking for help and advice felt like admitting incompetence, which was an uncomfortable feeling for me. I am in the process of jettisoning this emotional response from my brain and replacing it with a recontextualization of asking for help and advice as doing research, but by querying people's brains instead of Wikipedia.

Replies from: None
comment by [deleted] · 2013-02-04T17:07:59.194Z · LW(p) · GW(p)

One reason is probably that asking for help and advice felt like admitting incompetence, which was an uncomfortable feeling for me.

I have this problem. Any advice on how to ask for advice?

Replies from: Qiaochu_Yuan, Vaniver
comment by Qiaochu_Yuan · 2013-02-04T17:36:46.087Z · LW(p) · GW(p)

Several ideas and techniques from the CFAR workshop are relevant:

  • Value of information: how much better can I make decisions after getting this advice?
  • Cultivating curiosity: gee, I wonder what advice I'll get?
  • Overcoming aversions / comfort zone expansion: what motives do I have for avoiding asking for advice? Does thinking about asking for advice make me feel bad? Do I view myself as someone who doesn't ask for advice? What do I expect will happen if I ask for advice and, taking the outside view, how realistic are those expectations?
  • Evaluating advice: now that I've gotten some advice, how much should I trust it (taking the outside view again)? Is the sample I've chosen to ask for advice from biased? How independent is the advice that I get from multiple people? When I asked for advice, did I accidentally prime the person I was asking in a particular direction?

The workshop included a practical session on comfort zone expansion (we went to a mall and did mildly uncomfortable things involving strangers) which I found helpful, mostly because it helped me internalize social pressure in the direction of comfort zone expansion. At the workshop this was abbreviated as CoZE and there was a half-serious idea floating around of awarding "CoZE points" to people whenever they expanded their comfort zones, which is the main technique I used the last time I tried asking for advice. (The other technique I used was a social commitment mechanism, but explaining how that fit in would make the story too long.)

comment by Vaniver · 2013-02-04T19:01:53.454Z · LW(p) · GW(p)

Any advice on how to ask for advice?

Try explicitly admitting to them that you care more about getting the job done correctly than appearing to be competent while doing the job.

comment by Kawoomba · 2013-02-04T09:34:13.079Z · LW(p) · GW(p)
  • While the "see one, do one, teach one" paradigm of learning by doing is prone to such an error mode, personally I don't find it as much of a problem when taking a more "supervised learning" approach - inferring your function from labelled training data. If you just shadow someone doing your exact job for a few days, chances are you'll avoid most mistakes just by copying the observed behavior. For a restaurant job - or most others - that should cover a majority of situations.

  • What about the iceberg iceberg, when noticing your first iceberg you realize there was a metric ton of icebergs under the iceberg.

Replies from: BerryPick6
comment by BerryPick6 · 2013-02-04T09:59:11.433Z · LW(p) · GW(p)

What about the iceberg iceberg, when noticing your first iceberg you realize there was a metric ton of icebergs under the iceberg.

Or a recursive iceberg, where you realize there's a whole nautical mile worth of rabbit hole left to go down?

Replies from: alfredmacdonald
comment by alfredmacdonald · 2013-02-04T10:42:27.601Z · LW(p) · GW(p)

I find that similar to the concept of fractal wrongness. What distinguishes an iceberg from a fractal is that you're in situations where someone is resisting exposing the whole iceberg for one reason or another. In the dishonesty scenario, you realize one lie reveals many others but only because that person has left you a tidbit of information that cracks their facade and allows you to infer just how deeply they've lied to you -- or in the case of attraction, an event or action that only would occur if they had a much greater level of attraction existing below the surface.

Replies from: DaFranker
comment by DaFranker · 2013-02-04T17:02:20.094Z · LW(p) · GW(p)

or in the case of attraction, an event or action that only would occur if they had a much greater level of attraction existing below the surface.

This seems misleading, à la Sherlock Holmes' "Eliminating the impossible". A charitable reading would parse as:

"or in the case of attraction, an event or action where the most probable world (as calculated with Bayes) in which it happens also requires a much greater level of attraction existing below the surface."

Just wanted to make sure I'm not inventing new interpretations and that there's no hidden inferential distance.

comment by wwa · 2013-02-05T13:12:24.701Z · LW(p) · GW(p)

Isn't anxiety the primary problem? The obvious solution to make less mistakes is to gather data and figure stuff out, but you're not asking about that. You're not afraid of making mistakes. You're afraid of people (including yourself) discovering your alleged incompetence. The obvious solution to that is to fix the anxiety problem. Yes, it might be hard and/or require external help, but you said it yourself :

the anxiety has been catastrophic

Replies from: alfredmacdonald
comment by alfredmacdonald · 2013-02-08T05:41:19.286Z · LW(p) · GW(p)

I don't currently work at a restaurant, so at the moment I'm afraid of nothing.

But for the purposes of the example, it's not about discovering mistakes or incompetence -- it's about your level of incompetence being much greater than you previously estimated, for reasons you were unaware of prior to being exposed to those reasons.

comment by TinyTimZamboni · 2013-03-12T03:48:01.937Z · LW(p) · GW(p)

Thanks so much, this is exactly what I've been going though at my new job, but hadn't been able to put words to it.

Is it possible though that sometimes the iceberg doesn't exist at all? I got a job as dishwasher at a restaurant when in high school. I fumbled food onto the floor, burned my hands, couldn't understand the head chef and was always working an hour after everyone else left. Feeling the iceberg below the waterline, I worked harder, skipping meals and never taking a break.

I left for a better job, and found out that I had been the best dishwasher they'd ever had in the position.

comment by Ari_Rahikkala · 2013-02-05T10:21:28.214Z · LW(p) · GW(p)

I wonder if you could use defect seeding in service jobs. Your boss's boss makes a deal with you that you deliberately make certain mistakes (of a severity that your boss should notice but that would not directly bother customers), and they'll keep those mistakes from going on your record. Then you wait to see whether your boss mentions those mistakes in your performance review, to get a better picture of how good of a handle they've got on what you're doing.

comment by [deleted] · 2013-02-04T16:29:51.143Z · LW(p) · GW(p)

The proactive thing to do, naturally, is to try to minimize how many mistakes you make.

I think you came close to saying the proactive thing to do just after this. You talked about correcting mistakes. One step behind that is not repeating mistakes. One step behind that is identifying mistakes. Mistakes unseen will be repeated. Identified mistakes might not be repeated, even if they cannot (at the time) be corrected.

Your model builds up your confidence and skills so keep it up! Part of that process is calving off parts of the mistake iceberg, never to be repeated again.

comment by [deleted] · 2013-02-11T03:09:28.987Z · LW(p) · GW(p)

The proactive thing to do, naturally, is to try to minimize how many mistakes you make.

To me, there seems to be something kind of off about this sentence. Suppose I'm trying to get better at a game like Starcraft. Starcraft is sufficiently complicated that attaining a basic level of still (by which I mean "macromanagement": being able to ensure that all your resources are being used somehow, without worrying about using them well) takes hours and hours of practice. And during that practice, you will inevitably make mistakes; the only way to avoid making mistakes is by not practicing. Indeed, every mistake teaches you something, so I'm tempted to say that what you want to do is to maximize the number of mistakes you make.

In short, it seems to me that minimizing the number of mistakes you make doesn't serve the purpose of making you more skilled. So what purpose does it serve?

Replies from: alfredmacdonald
comment by alfredmacdonald · 2013-02-11T21:20:19.535Z · LW(p) · GW(p)

Sure, in the very short run (starting from absolutely no knowledge of the game) you'd have to make mistakes to learn anything at all. But the process of getting better is a gradual decrease of the frequency of those mistakes. You'd want to minimize your mistakes as much as possible as you got better, because the frequency of mistakes will be strongly correlated with how much you lose.

I think you're seeing "try to minimize how many mistakes you make" and reading that as "trying to make no mistakes." There are certainly mistakes you'll have to make to get better, but then there are superfluous mistakes that some people may make while others won't, or catastrophic mistakes that would make you look really bad which you'd definitely want to avoid. The depth of mistakes can go much deeper than the necessary mistakes you'd have to make to get better, in other words.

Replies from: None
comment by [deleted] · 2013-02-11T22:26:51.303Z · LW(p) · GW(p)

Sure, in the very short run (starting from absolutely no knowledge of the game) you'd have to make mistakes to learn anything at all. But the process of getting better is a gradual decrease of the frequency of those mistakes. You'd want to minimize your mistakes as much as possible as you got better, because the frequency of mistakes will be strongly correlated with how much you lose.

Mm, that all sounds like it's true if you only play games against the same skill level of opponent. If you increase the difficulty level at the same speed that you gain speed, then you won't start winning more games. I guess it's true that you'll stop making some mistakes, but in addition, some things that previously weren't mistakes will become mistakes.

In any case, I guess it's certainly true that there are things you can do that will both decrease the number of mistakes that you make and increase your rate of learning, such as paying more attention.

comment by DuncanS · 2013-02-05T22:50:03.255Z · LW(p) · GW(p)

Congratulations - this is what it's like to go from the lowest level of knowledge (Knows nothing and knows not that he knows nothing.) to the second lowest level. (Knows nothing, but at least knows that he knows nothing.)

The practical solution to this problem is that, in any decent organisation there are people much more competent than these two levels, and it's been obvious to them that you know nothing for much longer than it's been obvious to you. Their expectations will be set accordingly, and they will probably help you out - if you're willing to take some advice.

comment by Cthulhoo · 2013-02-04T10:42:48.401Z · LW(p) · GW(p)

While entering a new field, you should be aware that there's at least an iceberg of ignorance you have to deal with. As you correctly noted, it's hard to tell which kind of mistakes this ignorance will lead you to make, but your prior for making mistakes should in general be pretty high. What I'm trying to tell is that you should be able to preemptively deal with the discovery of the tip of the iceberg: you should expect it to pop up at some point. Other people should in general be aware of it too; if they don't, just point it out beforehand. Usually the "I don't know how to do it, but I am willing to learn" position is the best one, and it's normally socially acceptable (except if you have previously sold yourself as an expert on the subject). Even if someone forgot how it was to be a newbie, he should know deep down that the only way to get rid of the iceberg is to slowly liquefy it: start small, get rid of the first tip, and then proceed to get rid of whatever comes to the surface next.