Perfectionism as depth-first search

post by MSRayne · 2020-12-29T01:49:22.945Z · LW · GW · 3 comments

Perfectionism could be generally defined as an approach to solving problems and completing tasks fixated primarily on minimizing perceived imperfection.

Depth-first search, meanwhile, is an algorithm for traversing tree structures, which starts at the root, then goes down a single branch until it hits a leaf. Upon reaching a leaf, it backtracks to the last node whose descendants aren't fully searched, and goes down a new branch. This process repeats until the whole tree is explored.

Recently, I realized, in a flash of insight while struggling to stop tweaking tiny flaws long enough to complete a project, that there is a possibly fruitful analogy between these two things.

Every task can be thought of as a tree of sub-tasks. In fact, while writing this very essay, I broke the main idea down into a tree of questions and answers, to help me think about it (essentially an outline, but with each statement having only questions as sub-elements, and each question only having answer statements beneath it). Similarly, when writing a program, one usually splits it into subroutines, themselves split into sub-subroutines.

Perfectionists tend to focus more on sub-tasks lower on the tree. For instance, when writing, I tend to fixate on word choice and comparatively neglect overall structure, and some people trying to tidy up might heavily clean each object, one by one, and burn out before touching most of what's in the room, instead of the standard approach - getting the whole room mostly clean prior to obsessing over specifics. The reason an aversion to imperfection motivates such behavior might be that smaller flaws are easier to perceive, and easier to fix - thus drawing more attention to themselves.

This interestingly parallels the way that depth-first search preferentially moves down the tree while searching - where searching the tree in this analogy represents working on tasks or sub-tasks, with the current node being visited representing the sub-task currently being worked on. A perfectionist will often be tempted to drill right down to a single detail, get it "right", then go to a related detail, and so on, wandering somewhat aimlessly through the leaves of the tree while barely touching the branches.

The reason this matters is that in most task trees, most value comes from working on nodes closer to the root - as higher-level sub-tasks usually lay necessary foundations on which lower-level details can be added later, and it is often much better to get every sub-task somewhat done, than just one or a few sub-tasks completely done. (Though, as a counterexample, in a complex machine, even a single part working incorrectly may make the whole machine fail - a case where perfectionism might be warranted!)

Perfectionism, though, spends most of its time far from the root, down in the dirty details. This means that (except in cases like that complex machine) it is usually less efficient than a more "breadth-first" work style would be - some way of doing tasks that prioritizes large sub-tasks, gradually building in smaller details after starting from a vague outline - similar to breadth-first search, an alternative to depth-first search which explores each horizontal level of the tree in turn rather than running up and down branches - the Veðrfölnir to depth-first search's Ratatoskr. Examples of working in a breadth-first way are outlining a program with dummy functions as a sanity check before filling in exactly how they work, and writing a messy first draft to get an idea down before editing it later.

Note that this analogy may not hold in cases where perfectionism is due to inaccurate measurement of value, rather than focus on details. Someone unable to decide how to do a task might try many approaches (like a breadth-first search) without going deeply into any, if they feel that they cannot afford to waste any effort on the wrong approach (going deeply into the wrong branch). This tends to be my problem, but in small, constrained tasks like writing an essay, a breadth-first approach (careful outlining, and refusing to tweak word choice until the very end) seems to work.

What I'd like to do now is work on ways to "breadth-ify" situations in which I commonly get sucked into perfectionistic tweaking, by creating incentives towards laying foundations first, which I hope would counteract my natural tendency to nitpick. I have some ideas about how to do this, and I used one to organize my thoughts in writing this - a variant of outlining that I am experimenting with, involving writing the outline as alternating layers of questions and answers, and applying a numeric priority score to each element of the outline which is proportional both to subjective importance, and to level on the tree - but I will have to explore this more. If anyone has ideas about how to do this, or how to apply this idea to things other than writing, please let me know!


Comments sorted by top scores.

comment by seed · 2020-12-29T12:32:25.230Z · LW(p) · GW(p)

Great observation! I was struggling with the same issue when I moved from studying math to graduate ML research. Depth-first search is the right approach to reading a math textbook. Say, you started to learn homology theory and realized you don't know what's an Abelian group. You should stop and go read about Abelian groups, or you won't understand what comes next. 

However, the same approach was getting me in trouble when trying to understand state-of-the-art in voice processing. I would start reading an article in the morning, and by the evening I'd finish no articles and find myself stuck in the middle of a textbook reading about some 50 years outdated method which wasn't relevant to modern research. So, I worked out that the effective approach to this task is breadth-first search: read the article from beginning to end, write down all unknown terms, go on to look up the most important one.

I think another reason why people might default to depth-first search is that BFS requires to store a list of unexplored nodes in memory and people don't have a lot working memory. So a note-taking system like Zettelkasten really helps with applying this approach more broadly in one's life.

Replies from: MSRayne
comment by MSRayne · 2020-12-29T13:37:46.550Z · LW(p) · GW(p)

Depth-first search is the right approach to reading a math textbook.

That's a great point! I never explicitly thought of it like that but it's clearly true now that you mention it. And not just math - nearly any scientific writing has the same quality where lack of knowledge about one idea or principle ruins your ability to understand any of it - these are examples of those "complex machines" which break if any part doesn't work.

A friend of mine mentioned that reading Wikipedia tends to be like your second example - going depth-first (chasing links) instead of breadth-first when trying to learn about a topic. You end up with a hundred tabs and no smarter than you were before. That's another failure mode where a breadth-first method like your note taking system presents advantages.

comment by Timothy Johnson (timothy-johnson) · 2020-12-29T02:13:19.833Z · LW(p) · GW(p)

XKCD made a similar comparison: xkcd: DFS.