"Metastrategic Brainstorming", a core building-block skill

post by Raemon · 2024-06-11T04:27:52.488Z · LW · GW · 5 comments


        Exploring multiple plans before committing.
        Come up with multiple "types" of metastrategies.
  Example: Writing this Blogpost
  Suggested Exercises
    Appendix: Existing available puzzles/games I've used
      Thinking Physics

I want to develop rationality training, which is aimed at solving confusing problems. 

Two key problems with "confusing problems" are:

  1. You might feel so confused and overwhelmed that you bounce off completely.
  2. You might be confused about what counts as progress, or where the most progress is possible, and accidentally work on the wrong thing.

A skill that helps with both of these is "metastrategic brainstorming" – the art of generating lots of potential good approaches; then choosing approaches that are likely to help. (And then, reflecting on whether those strategies worked)

Different situations call for different sorts of strategies. If a problem is confusing, you probably don't have a simple playbook for dealing with it. Different people also benefit from different sorts of strategies. So, while I can tell you a list of potential mental tools, what I most want you to practice is the art of identifying what would help you, in particular, with the situation in particular in which you find yourself.

My triggers for switching to "metastrategic brainstorming mode" are:

...and, I'm trying to solve a problem I expect to take at least 30 minutes (i.e. enough time it's worth spending at least a few minutes meta-brainstorming)...

...then I switch into "metastrategic brainstorming mode", which entails:

  1. Open up a writing doc.
  2. Ask myself "what are my goals?". If there are multiple goals, write them both down.
  3. Set a 5-10 minute timer, spend it brainstorming "meta-level strategies." Don't try to solve the object level problem. Just focus on generating strategies that might help you solve the problem. 
  4. Look at my list of meta-strategies, and see if there's one that I feel at least reasonably optimistic about. 
  5. If so, try that meta-strategy.
  6. If not, brainstorm more. (But: note that "take a break", "nap", and "ask a friend for help" all totally count as valid meta-strategies to try. Taking a nap is often pretty important, actually!)
  7. When/if I eventually solve my problem, take note of what strategies and meta-strategies I ended up using. Ideally, write them down somewhere I'm likely to remember them again.

I want to repeat emphasize "setting a real timer, for at least 5 and maybe up to 10 minutes, where you only allow yourself to generate meta-level strategies." (I think this is particularly valuable when you're training the skill)

Exploring multiple plans before committing.

Partly, this is because it just takes a little while to shift out of "object level mode". But, more importantly: because your problem is confusing, your ways of thinking about it might be somewhat off track. And, even if you'd eventually solve your problem, you might be doing it using a way less efficient method. 

In particular, many problems benefit from going "breadth first", where instead of barreling down the first plan you came up with, you try ~3 plans a little bit and see if one of them turns out to be way better than your initial plan.

Come up with multiple "types" of metastrategies.

When you're doing the 5-10 minutes of brainstorming, I recommend exploring a variety of strategies. For example, there are conceptual strategies like "break the problem down into smaller pieces." There are physical/biological strategies like "take a walk, or get a drink of water". There are social strategies like "ask a friend for help." (sometimes this isn't appropriate if you're training, but is a fine strategy to use on real world tasks)

Example: Writing this Blogpost

Right now I'm writing a blogpost on Metastrategic brainstorming. I actually found myself a bit stuck (a few paragraphs ago, before I finished the previous section). This seemed like a good opportunity to just demonstrate the technique right now.

First, what are my goals? They're basically:

With that in mind, I set a 6 minute timer. Here's what I came up with:

  1. Just set a 30 minute timer and write without stopping even if the words I'm writing feel dumb. (Sometimes that just works)
  2. Ask myself "What feels hard about this?" and engage curiously with whatever comes up. 
  3. Find a friend I want to explain this to, talk to them, and see how I end up explaining it to them.
  4. Take a walk.
  5. Take seven deep breaths.
  6. (I notice now that I feel stuck on generating metastrategies and feel "surely the ones so far are good enough and I can stop." Maybe that's true. But one of the things I want to illustrate here is generalized 'get yourself unstuck and keep generating good ideas'. So, I ask: if it turned out it was really important to keep going, what ideas might I still be missing?)
  7. Ask myself "What's the best version of this blogpost?" If I didn't want to settle for merely "having written up a decent post on Metastrategic Brainstorming?", what would be missing?
  8. Reflect more on why I decided to write this post right now. (Am I even doing the right thing?)

Okay, timer just went off. 

Sidebar: A few notes on how I generated these ideas:

In this case, I had a fair amount of experience trying tools to get over writers block, and I was mostly jogging my memory with those tools.

On step 6, where I noticed I had run out of steam, I knew from experience that asking "okay, but what if it was really important to keep going, what would I miss?" and "what would make this like 10x more impactful?" had previously been fruitful to ask.

If you're just starting out on metastrategy brainstorming, you probably won't have as clear a sense of what's helpful. I developed this skill with the Babble Challenge [LW · GW] series. A key thing is to relax your standards when you get stuck (i.e. if it's been a few minutes and you haven't successfully generated anything). Writing down a few "bad ideas" can grease your mental gears and get you generating some good ideas again.

There's an art to finding the right level of "babble/prune" ratio. In this case I felt like I had traction on getting "actual good ideas", and I didn't bother writing down ideas that I knew weren't actually going to be that useful in this context

How do I feel about those ideas? 

When I reflect on my two goals (write quickly, and convey a fairly deep skill), the strategies that feel most salient to me are:

  1. Mostly, focus on setting a short timebox, and shipping the post in some form at the end of the timebox even if it's not perfect.
  2. Imagine what questions the guy at the previous workshop would have, that this post might be able to answer. (The guy isn't around right now, but, I think simulating him might be good enough)

Those are still slightly in tension with each other. Also, because I want to get this done in one sitting, I don't want to spend too much time metabrainstorming. But I notice I don't quite expect those two tools to work quickly enough. So I decide to give myself ~1 more minute for metastrategizing. 

And the thing that comes up in that minute is "first, write an outline of what must be in the post when I'm done."

That list is:

That all feels doable. Something I feel slightly dissatisfied with is "but how do you generate strategies tho?". There is something magical-feeling about the process. I think I will mostly shrug and hope that the suggested exercises get help develop the skill, even if it initially feels opaque.

Suggested Exercises

The way I suggest learning this skill is, simply, to try tackling some problems that feel genuinely hard, which you don't have a good playbook for solving. (You can often learn more from failure than from success, if you're able to eventually look up the solution and get an explanation of it)

Various kinds of puzzles and games can make for good exercise test-beds. You can dial up the difficulty by giving yourself the goal of being very confident in your answer, or by trying to beat a video game on the very first try (despite limited information). You can also dial up the difficulty by trying to solve it faster (although I recommend first aiming to solve it "at all").

I've spent the past year exploring different puzzles, games and exercises. Sometimes I've written up particular exercises that took advantage of a given puzzle or game's strength. Here, I present them for you to consider in their raw form. 

My general approach is:

The last part is the most important part. Your goal is not to beat a given puzzle. Your goal is to find generalizable problem solving tools, and to learn the taste of whether a given tool is appropriate for a situation.

In the real world, you'll face confusing problems that don't have a clear answer, where there is no one to tell you what strategy to use. I'm hoping, with this exercise, you learn the art of "teaching yourself to fish", rather than me teaching you how to fish.

Appendix: Existing available puzzles/games I've used

Thinking Physics

I first started working on this skill in the context of Thinking Physics [LW · GW], a "reverse physics textbook" where instead of reading up on physics principles and then testing your knowledge with exercises, you are given a series of questions, which you try to solve (maybe taking multiple hours or even days to solve, from first principles), and then when you turn the page you'll see an explanation of the underlying physics phenomena.


I've found various puzzle and strategy games good for this exercise. A good videogame here [LW · GW] is easy to jump into without much preamble, and takes 30 to 120 minutes.

Some examples include:

You can read other games people have suggested in this One-Shot Strategy Game [LW · GW] thread, although I'm not sure they're all appropriate.


Comments sorted by top scores.

comment by Matt Goldenberg (mr-hire) · 2024-06-11T12:36:26.432Z · LW(p) · GW(p)

One thing to note - Brainstorming itself is a meta-strategy that may or may not be the best approach at certain points in the problem, to generate meta-strategic approaches.

Brainstorming for me has a particular flavor - it's helpful when I have a lot of ideas but don't know where to start, or when it feels like my mind just needs the starter cord pulled a few times.

Other times, I get a lot more out of a taking a walk and let my mind wander around the problem, not specifically listing out lanes of attack, but sort of holding the intention that one may show up as I think in a free associative way and walk.

Other times it's helpful for me to have a conversation with a friend, especially one who I can see has the right mind-shape to frame this sort of problem.

Other times it's helpful to specifically look through the list of meta-strategies I have, wandering around my Roam and seeing how different mental models and frameworks can frame the problem.


I guess what I'm saying is, it's helpful to separate the move of "oh, it's time to figure out what meta-strategy I can use" from "oh, it's time to brainstorm"

Replies from: mr-hire, Raemon
comment by Matt Goldenberg (mr-hire) · 2024-06-11T13:05:04.286Z · LW(p) · GW(p)

I just realized that this then brings the problem of "oh, but what's the meta-meta strategy i use), but I think there's just an element of taste to this.

comment by Raemon · 2024-06-11T22:18:54.950Z · LW(p) · GW(p)

Oh yeah. I may try to update the post to caveat this.

Relatedly: some people give me feedback that the process (or, the way I explain/demonstrate it) feels like some particular flavor of rigid/orderly in a way that doesn't work for everyone.

(I think that some of the people-for-whom-it-doesn't-work would benefit gaining some skills that would allow it to work. If you can't write stuff up your thought-process is doc, you're missing out on a lot of working-memory-extension options that will be limiting for you. But, writing stuff up in a doc, or doing so in a "brainstormy" way, is not always the right move, regardless.)

comment by Lorxus · 2024-06-11T23:48:51.171Z · LW(p) · GW(p)

@WhatsTrueKittycat [LW · GW] (meta?-)cogtech worth looking at, for effectiveness, elegance, and sheer breadth of applicability.

comment by jefa · 2024-06-11T11:50:47.999Z · LW(p) · GW(p)

I enjoyed this post, thanks.  Although I have never explicitly used this technique, I think the Lotus Blossom method of idea generation and problem solving is probably the best visualization of how I intuitively approach complex problem solving.