"Metastrategic Brainstorming", a core building-block skill

post by Raemon · 2024-06-11T04:27:52.488Z · LW · GW · 5 comments

Contents

        Exploring multiple plans before committing.
        Come up with multiple "types" of metastrategies.
  Example: Writing this Blogpost
  Suggested Exercises
    Appendix: Existing available puzzles/games I've used
      Thinking Physics
      Games
None
5 comments

I want to develop rationality training, which is aimed at solving confusing problems. 

Two key problems with "confusing problems" are:

1. You might feel so confused and overwhelmed that you bounce off completely.
2. You might be confused about what counts as progress, or where the most progress is possible, and accidentally work on the wrong thing.

A skill that helps with both of these is "metastrategic brainstorming" – the art of generating lots of potential good approaches; then choosing approaches that are likely to help. (And then, reflecting on whether those strategies worked)

Different situations call for different sorts of strategies. If a problem is confusing, you probably don't have a simple playbook for dealing with it. Different people also benefit from different sorts of strategies. So, while I can tell you a list of potential mental tools, what I most want you to practice is the art of identifying what would help you, in particular, with the situation in particular in which you find yourself.

My triggers for switching to "metastrategic brainstorming mode" are:

• I've just sat down to work on a problem I already know is hard.
• I've starting to feel stuck, annoyed or frustrated.
• I notice that I settled into the very first plan that occurred to me, and I have a sneaking suspicion it's not the best plan.

...and, I'm trying to solve a problem I expect to take at least 30 minutes (i.e. enough time it's worth spending at least a few minutes meta-brainstorming)...

...then I switch into "metastrategic brainstorming mode", which entails:

1. Open up a writing doc.
2. Ask myself "what are my goals?". If there are multiple goals, write them both down.
3. Set a 5-10 minute timer, spend it brainstorming "meta-level strategies." Don't try to solve the object level problem. Just focus on generating strategies that might help you solve the problem. 
4. Look at my list of meta-strategies, and see if there's one that I feel at least reasonably optimistic about. 
5. If so, try that meta-strategy.
6. If not, brainstorm more. (But: note that "take a break", "nap", and "ask a friend for help" all totally count as valid meta-strategies to try. Taking a nap is often pretty important, actually!)
7. When/if I eventually solve my problem, take note of what strategies and meta-strategies I ended up using. Ideally, write them down somewhere I'm likely to remember them again.

I want to repeat emphasize "setting a real timer, for at least 5 and maybe up to 10 minutes, where you only allow yourself to generate meta-level strategies." (I think this is particularly valuable when you're training the skill)

Exploring multiple plans before committing.

Partly, this is because it just takes a little while to shift out of "object level mode". But, more importantly: because your problem is confusing, your ways of thinking about it might be somewhat off track. And, even if you'd eventually solve your problem, you might be doing it using a way less efficient method. 

In particular, many problems benefit from going "breadth first", where instead of barreling down the first plan you came up with, you try ~3 plans a little bit and see if one of them turns out to be way better than your initial plan.

Come up with multiple "types" of metastrategies.

When you're doing the 5-10 minutes of brainstorming, I recommend exploring a variety of strategies. For example, there are conceptual strategies like "break the problem down into smaller pieces." There are physical/biological strategies like "take a walk, or get a drink of water". There are social strategies like "ask a friend for help." (sometimes this isn't appropriate if you're training, but is a fine strategy to use on real world tasks)

Example: Writing this Blogpost

Right now I'm writing a blogpost on Metastrategic brainstorming. I actually found myself a bit stuck (a few paragraphs ago, before I finished the previous section). This seemed like a good opportunity to just demonstrate the technique right now.

First, what are my goals? They're basically:

• Get this post written quickly. This is actually a pre-requisite for another post (I'm kinda procrastinating on the other post by writing this one because I hoped I could just bang it out in a couple hours)
• Try to convey a fairly opaque skill, to people who don't intuitively get it. It turned out at my previous workshop [LW · GW] that one participant didn't really have the skill and I hadn't set aside time for teaching it.

With that in mind, I set a 6 minute timer. Here's what I came up with:

1. Just set a 30 minute timer and write without stopping even if the words I'm writing feel dumb. (Sometimes that just works)
2. Ask myself "What feels hard about this?" and engage curiously with whatever comes up. 
3. Find a friend I want to explain this to, talk to them, and see how I end up explaining it to them.
4. Take a walk.
5. Take seven deep breaths.
6. (I notice now that I feel stuck on generating metastrategies and feel "surely the ones so far are good enough and I can stop." Maybe that's true. But one of the things I want to illustrate here is generalized 'get yourself unstuck and keep generating good ideas'. So, I ask: if it turned out it was really important to keep going, what ideas might I still be missing?)
7. Ask myself "What's the best version of this blogpost?" If I didn't want to settle for merely "having written up a decent post on Metastrategic Brainstorming?", what would be missing?
8. Reflect more on why I decided to write this post right now. (Am I even doing the right thing?)

Okay, timer just went off. 

Sidebar: A few notes on how I generated these ideas: In this case, I had a fair amount of experience trying tools to get over writers block, and I was mostly jogging my memory with those tools. On step 6, where I noticed I had run out of steam, I knew from experience that asking "okay, but what if it was really important to keep going, what would I miss?" and "what would make this like 10x more impactful?" had previously been fruitful to ask. If you're just starting out on metastrategy brainstorming, you probably won't have as clear a sense of what's helpful. I developed this skill with the Babble Challenge [LW · GW] series. A key thing is to relax your standards when you get stuck (i.e. if it's been a few minutes and you haven't successfully generated anything). Writing down a few "bad ideas" can grease your mental gears and get you generating some good ideas again. There's an art to finding the right level of "babble/prune" ratio. In this case I felt like I had traction on getting "actual good ideas", and I didn't bother writing down ideas that I knew weren't actually going to be that useful in this context

How do I feel about those ideas? 

When I reflect on my two goals (write quickly, and convey a fairly deep skill), the strategies that feel most salient to me are:

1. Mostly, focus on setting a short timebox, and shipping the post in some form at the end of the timebox even if it's not perfect.
2. Imagine what questions the guy at the previous workshop would have, that this post might be able to answer. (The guy isn't around right now, but, I think simulating him might be good enough)

Those are still slightly in tension with each other. Also, because I want to get this done in one sitting, I don't want to spend too much time metabrainstorming. But I notice I don't quite expect those two tools to work quickly enough. So I decide to give myself ~1 more minute for metastrategizing. 

And the thing that comes up in that minute is "first, write an outline of what must be in the post when I'm done."

That list is:

• Introduce the idea of metastrategy.
• Write down the simplified algorithm that I generally run.
• Write out at least one worked example (that's what I'm doing right now)
• Give a list of suggested exercises for practicing the skill.

That all feels doable. Something I feel slightly dissatisfied with is "but how do you generate strategies tho?". There is something magical-feeling about the process. I think I will mostly shrug and hope that the suggested exercises get help develop the skill, even if it initially feels opaque.

Suggested Exercises

The way I suggest learning this skill is, simply, to try tackling some problems that feel genuinely hard, which you don't have a good playbook for solving. (You can often learn more from failure than from success, if you're able to eventually look up the solution and get an explanation of it)

Various kinds of puzzles and games can make for good exercise test-beds. You can dial up the difficulty by giving yourself the goal of being very confident in your answer, or by trying to beat a video game on the very first try (despite limited information). You can also dial up the difficulty by trying to solve it faster (although I recommend first aiming to solve it "at all").

I've spent the past year exploring different puzzles, games and exercises. Sometimes I've written up particular exercises that took advantage of a given puzzle or game's strength. Here, I present them for you to consider in their raw form. 

My general approach is:

• Try to solve the problem for ~10 minutes
• Do ~10 minutes of meta brainstorming
• Try to solve the problem again. (alternate between object-level and meta-brainstorming however feels appropriate)
• When you're done, reflect on what you learned – which metastrategies turned out to actually help? What other situations do you expect them to apply to?

The last part is the most important part. Your goal is not to beat a given puzzle. Your goal is to find generalizable problem solving tools, and to learn the taste of whether a given tool is appropriate for a situation.

In the real world, you'll face confusing problems that don't have a clear answer, where there is no one to tell you what strategy to use. I'm hoping, with this exercise, you learn the art of "teaching yourself to fish", rather than me teaching you how to fish.

---

Appendix: Existing available puzzles/games I've used

Thinking Physics

I first started working on this skill in the context of Thinking Physics [LW · GW], a "reverse physics textbook" where instead of reading up on physics principles and then testing your knowledge with exercises, you are given a series of questions, which you try to solve (maybe taking multiple hours or even days to solve, from first principles), and then when you turn the page you'll see an explanation of the underlying physics phenomena.

Games

I've found various puzzle and strategy games good for this exercise. A good videogame here [LW · GW] is easy to jump into without much preamble, and takes 30 to 120 minutes.

Some examples include:

• The puzzle game Baba Is You has been particularly fruitful for me. (I've developed multiple exercises based on it)
• Battle for Polytopia. (note: this will start you off in a tutorial. I recommend exiting to the main menu, and then starting a new game, on "Hard" difficulty with 2-3 opponents)
• Into the Breach. (Note: has a minute or two of introduction you need to click past before you get to the first level)
• Luck Be a Landlord.

You can read other games people have suggested in this One-Shot Strategy Game [LW · GW] thread, although I'm not sure they're all appropriate.

5 comments

Comments sorted by top scores.