lynettebye's Shortform

Note, I consider this post to be “Lynette speculates based on one possible model”, rather than “scientific evidence shows”, based on my default skepticism for psych research. 

A recent Astral Codex Ten post argued that advice is written by people who struggle because they put tons of time into understanding the issue. People who succeeded effortlessly don’t have explicit models of how they perform (section II). It’s not the first time I’ve seen this argument, e.g. this Putanumonit post arguing that explicit rules help poor performers, who then abandon the rules and just act intuitively once they become good. 

This reminded me of a body of psych research I half-remembered from college called Choking under Pressure. 

My memory was that if you think about what you’re doing too much after becoming good, then you do worse. The paper I remembered from college was from 1986, so I found “Choking interventions in sports: A systematic review” from 2017. 

It turns out that I was remembering the “self-focused” branch of choking research. 

"Self-focus approaches have largely been extended from Baumeister’s (1984) automatic execution hypothesis. Baumeister explains that choking occurs because, when anxiety increases, the athlete allocates conscious attention to movement execution. This conscious attention interferes with otherwise automatic nature of movement execution, which results in performance decrements."

(Slightly worrying. I have no particular reason to doubt this body of work, but Baumeister's "willpower as muscle" -- i.e. ego depletion -- work hasn't stood upwell.) 

Two studies found that distraction while training negatively impacted performance. I’m not sure if this this was supposed to acclimatize the participants to distractions while performing or reduce their self-focus while training. (I’m taking the paper’s word and not digging beyond the surface on the numbers.) Either way, I feel very little surprise that practicing while distracted was worse. Maybe we just need fatal-car-crash magnitude effects before we notice that focus is good?  

Which makes it all the more surprising that seven of eight studies found that athletes performed better under pressure if they simultaneously did a second task (such as counting backwards). (The eighth study found a null result.) According to the theory, the second task helped because it distracted from self-focus on the step-by-step execution. 

If this theory holds up, it seems to support paying deliberate attention to explicit rules while learning but *not* paying attention to those rules once you're able to use them intuitively (at least for motor tasks). In other words, almost exactly what Jacob argued in the Putanumonit article.

Conclusions 

I was intrigued by this argument because I’ve argued that building models is how one becomes an expert.[1] After considering it, I don’t actually think the posts above offer a counter argument to my claim. 

My guess is that experts do have models of skills they developed, even if they have fewer models (because they needed to explicitly learn fewer skills). The NDM method for extracting experts’ models implies that the experts have models that can be coaxed out. Holden’s Learning By Writing post feels like an explicit model. 

Another possibility is that experts forget the explicit models after switching to intuition. If they faced the challenges more than five or ten years ago, they may not remember the models that helped them then. Probably uncoincidentally, this aligns neatly with Cal Newport’s advice to seek advice from someone who recently went through the challenges you’re now facing because they will still remember relevant advice.

Additionally, the areas of expertise I care about aren’t like walking, where most people will effortlessly succeed. Expertise demands improving from where you started. Both posts and the choking under pressure literature agree that explicit models help you improve, at least for a while. 

“Find the best explicit models you can and practice until you don’t need them” seems like a reasonable takeaway. 

[1] Note, there’s an important distinction between building models of your field and building models of skills. It seems like the main argument mostly applies to models of skills. I doubt Scott would disagree that models of fields are valuable, given how much time he’s put into developing his model of psychopharmacology.

Occasionally, I get asked for feedback on someone’s resume. I’m not really a resume-editing coach, but I can ask them what they accomplished in that roll where they’re just listing their duties. Over time, I’ve found I’m completely replaceable with this rock. 

You did X for Y company? Great! Why is it impressive? Did it accomplish impressive outcomes? Was it at an impressive scale? Did it involve impressive other people or companies? 

You wrote an application using Z software? Great! Why is it impressive? Did the code speed up run time by an impressive amount? Did it save an impressive amount of money? Did it enable an impressive research finding? Does it display an impressive amount of technical expertise? 

You published a thing? Great! Why is it impressive? Was it published somewhere impressive? Was it cited an impressive number of times? Did impressive people say good things about it?

Fabricated goals

I’m really good at taking an abstract goal and breaking it down into concrete tasks. Most of the time, this is super useful.

But if I’m not sure what would accomplish the high level goal, sometimes the concrete goals turn out to be wrong. They don’t actually accomplish the high-level more vague goal. If I don’t notice that, I’ll at best be confused. At worse, I’ll accomplish the concrete goals, fail at the high-level goal, and then not notice that all my effort isn’t accomplishing the outcome I actually care about.

I’m calling the misguided concrete goals as “fabricated goals”.  Because I’m falsely believing this goal is an option to get me to my high-level goal. 

The alternative feels pretty bad though. If I can’t break the vague goal into concrete steps that I know how to do, I need to be constantly feeling my way through uncertainty. In that situation, sometimes it’s good to pick a small, time-bound concrete goal and do it to see if it helps. But I need to be constantly checking in on whether it’s actually helping.

I’ve been practicing a lot this year with improving feedback loops, and it’s come a long way. Sitting with uncertainty for days and checking in on whether I’m making progress on the scale of minutes, though, that’s hard. I’ve heard early stages of research being called “wandering in the desert” and this feels similar. 

It’s so much easier to substitute a fabricated goal – I know what I need to do, I can measure my progress. It’s harder to sit with the uncertainty and hopefully, slowly feel my way toward some insight.