The "Think It Faster" Exercise

post by Raemon · 2024-12-11T19:14:10.427Z · LW · GW · 0 comments

Contents

    Example: 10x UI designers
  THE EXERCISE
  Part I: Thinking it Faster
    Steps you actually took
    Magical superintelligence steps
    Iterate on those lists
      What skills, if you’d trained for 20 or 100 hours, would have helped you find the answer intuitively?
      What principles, if you internalized and they came easily to mind, would have allowed you to make some of those leaps ~instantly, or at least much faster?
      What jumps-between-steps feel magical or unrealistic, in “magical short list”? 
      For the “original steps you took”, what steps could you have skipped? What would have been necessary to skip them?
      Overall, what takeaways do you want to remember for later?
      What's the broadest generalize that feels reasonable to draw?
    Generalizing, and not Overgeneralizing
    Skills into Principles
  Part II: Thinking It Faster The First Time
    Generalizing from this exercise 
      First, consolidate your list of skills and principles
      List past situations you could have benefited from those skills or principles
      List future situations where you suspect might benefit from those skills or principles.
      In the next week, what’s 1-3 tasks you’re doing that might benefit from those skills or principles?
    Anticipating Future Life Lessons
      In the next couple days, what's something you're planning to do that you expect to take a long time?
      ...what's something you expect to solve via tinkering/iteration without much of a plan, that you expect to take awhile?
    Getting Detailed, and TAPS
      For one of the past moments, think in detail about how principles/skills would apply.
      Take whatever actions you can take right now.
  Part III: The Five Minute Version
None
No comments

Ultimately, I don’t want to solve complex problems via laborious, complex thinking, if we can help it. Ideally, I'd want to basically intuitively follow the right path to the answer quickly, with barely any effort at all.

For a few months I've been experimenting with the "How Could I have Thought That Thought Faster?" [LW · GW] concept, originally described in a twitter thread by Eliezer:

Sarah Constantin: I really liked this example of an introspective process, in this case about the "life problem" of scheduling dates and later canceling them: malcolmocean.com/2021/08/int…

Eliezer Yudkowsky: See, if I'd noticed myself doing anything remotely like that, I'd go back, figure out which steps of thought were actually performing intrinsically necessary cognitive work, and then retrain myself to perform only those steps over the course of 30 seconds.

SC: if you have done anything REMOTELY like training yourself to do it in 30 seconds, then you are radically smarter/more able/etc than me and all the other people who do slower introspective practices.

SC: I don't know whether to be impressed or to roll to disbelieve.

EY: I mean I suspect that this actually requires something like a fast perceptual view of minds as engines and thoughts as doing work and like actually draws on my mind design knowledge, but, even so, I ask: Do you constantly look back and ask "How could I have thought that faster?"

SC: No, I've never asked that.

EY: Okay, well, every time I'm surprised by reality I look back and think "What about my model and my way of thinking could I change that would have predicted that better, without predicting a bunch of other things worse?"

I've been working to operationalize this as an exercise you can train repeatedly, rather than hoping to remember to do when reality hands you a surprise. You can do the exercise after any difficult cognitive task (either a toy puzzle exercise, or a day-job project that took a long while). 

What would have been necessary for you to just look at the situation, and automagically find the right solution (without overfitting, and without generalizing in a way that would cause you to think unproductive thoughts in other sorts of situations)?

The goal of this exercise is to identify:

…that move you to correct solutions to problems as quickly as possible. 

This overall builds into two deep skills: Asking yourself retrospectively “how could I have Thought that Faster?” And then, prospectively, learning to ask “what am I about to predictably spend too long thinking about, and how could I Think it Faster, the First Time?”

I don’t know the upper limits of these skills, but I am currently finding it fruitful to adopt the mindset of “if it took longer than 15 minutes and an LLM query, you probably took too long.” Relentlessly ask yourself how hours or days could have turned into 15 minutes.

There are some classes of ideas I currently can effortless generate, which used to take me a long time. Some of those skills are pretty general – consciously steering towards "what evidence would disconfirm this idea?", or recursively asking "why is this problem impossible, and how can I deal with that?" and generating useful answers that in fact help Deal With That.

It's clearly possible to gain skill at effortlessly steering towards the right answer. The question is "how quickly can this skill pay off?" and "how general are the solutions?"

I currently believe this is pretty likely to be worth most people's time to invest in. The evidence isn't clear yet, but I feel optimistic. When I first got started with this I'd spend ~an hour thinking through how could have Thought It Faster, but I've gotten it into a 5-10 minute exercise I can do multiple times per week when my day job hands me something difficult.

Example: 10x UI designers

You’ve probably heard stories about “10x programmers”, who just intuitively steer towards good decisions and get things done dramatically faster than most developers. But I spend most of my time doing UI design, and have lately been thinking about "10x UI designers."

I remember, 17 years ago, at my first “real” job at a printing company. A client wanted us to design a brochure for them. I was “good at art”, but this was my first assignment professionally making art for someone else. I thought about what they needed, I labored carefully for 4 hours, generating ideas and fiddling around in my graphic software.

Eventually my senior partner came to look at it. He said “eh, this has a lot of problems. Here, you should do it this way.” I can’t remember if he told me what to do, or just went and did it himself. But, he bypassed all my tedious work by just intuitively knowing how to solve this-particular-class of problem already, and moving directly to the good ideas instead of working through bad ones.

Okay, so, skill is a thing. He had 10 years of experience, I didn’t. 

More recently, I was working for three days on the design for the new Glossary Editor on posts. I was meanderingly exploring a few options, including a separate “table-of-contents” section, and a “show the jargon terms and definitions in detail” section. It was very complex and took up a lot of space and was overwhelming to look at. 

My goal was to make it so authors could quickly skim the AI generated jargon, and make decisions about approving ones they liked, without much effort. 

There were tradeoffs between:

  1. Making it easy to skim
  2. Making it easy to actually make final decisions, which required knowing enough about what the AI generated definitions actually said.
  3. Fitting it into a small amount of space, so it didn’t disrupt the experience of people who didn’t care about the glossary at all.
  4. Making it simple and elegant to think about.
  5. Accurately conveying all the tools and affordances.

I labored for 3 days, shuffling around where-the-tradeoffs lived, incrementally reducing some of the issues. 

Then Oliver Habryka came into the room, took a look, and said “man, your information hierarchy here is all over the place.” Then he fiddled around for 20 minutes and found something dramatically better than what I had at the time. 

Since I had recently been asking myself “how could I have Thought That Faster?”, I took this as an opportunity to ask “what the hell just happened, and how could I have done it myself without Oliver’s help?”. 

Mulling it over, I observed that although part of the answer was “UI specific design taste honed by years of experience”, there were some specific questions he was asking that pointed his attention in much more productive directions (i.e. “how can we give this a clear, unified, simple information hierarchy?”)

The underlying, general skill seemed to be:

In each domain (UI, programming, x-risk), there are specific tactical tools for actually “dealing with all constraints.” In this case, domain-specific principles included: “many buttons weren’t necessary to show users initially, until they started interacting with it”, and “instead of having a table-of-contents and a detail-view, build a single view that's an 80/20 of both views at once.”

But it seems like there is a general skill of:

I've since applied the "Actually Deal With Constraints" principle to other "Things I coulda thought faster" (such as how my Feedbackloop-first Rationality agenda has evolved over the past year, and how I could have made the same progress in like a month), and found it pretty valuable.

THE EXERCISE

Okay. So, you've just done either a Toy Exercise puzzle, or you just noticed in your Real World that you either spent a long time figuring something out, or were surprised by something (which you didn't figure out at all)

I recommend Thinking Physics [LW · GW] and Baba is You [LW · GW] as sources of puzzles to start grinding on this. For your day job, I recommend learning to notice [LW · GW] when you have the sneaking suspicion something took longer than it needed to. (Once you've got a bit of practice, I recommend applying this even to places you don't have that sneaking suspicion, but it did practically take a long time)

(Baba is You is particular good because it tends to surprise you, and you can practice micro-versions of this exercise on individual surprises within a single Baba is You level, which helps train the general reflex of Notice Surprise -> Ask how you could have Thought It Faster in the moment)

At a high level, you're going to ask:

"How could you have Thought it Faster?"

"What did you learn, which’ll let you Think It Faster The First Time, later?

Rather than go through each step exhaustively, in sequence, I recommend cycling through them: jot down a few quick ideas for each prompt, circling back to the first one, with each pass giving you a sense of how all the pieces fit together.

Part I: Thinking it Faster

Steps you actually took

In chronological order (as best you remember) what happened?

  1.  
  2.  

Magical superintelligence steps

If you were a waaaay smarter version yourself, or if you imagine some other waaaay (unrealistically) smarter genius, what is the shortest number of steps you can possibly imagine this taking? 

(Right now, it's okay for this to feel like cheating)

  1.  
  2.  

Iterate on those lists

Identify steps in the first list you could straightforwardly remove, or simplify. And, identify steps to add to the second list until it no longer feels like unrealistic cheating. (i.e. if you're not overfitting. The plan doesn't imply you should spend tons of cognitive overhead all the time on minor, unimportant clues)

Try these prompts to help you:

What skills, if you’d trained for 20 or 100 hours, would have helped you find the answer intuitively?

 

What principles, if you internalized and they came easily to mind, would have allowed you to make some of those leaps ~instantly, or at least much faster?

 

What jumps-between-steps feel magical or unrealistic, in “magical short list”? 

 

For the “original steps you took”, what steps could you have skipped? What would have been necessary to skip them?

 

Overall, what takeaways do you want to remember for later?

 

What's the broadest generalize that feels reasonable to draw?

 

Generalizing, and not Overgeneralizing

So far, I have mostly seen people fail generalize enough, rather than too much. It's certainly possible to fail in both directions, but I maybe suggest erring on the side of overgeneralizing, and wait until it actually hurts you to dial it back.

In the UI example above, here are a few takeaways I could have had:

  1. "In UI design, make sure not to be overwhelming, make sure to have a clear information hierarchy, try removing or simplifying or combining bits until you have an elegant but comprehensive tool."
  2. "In UI design, try to identify all the constraints for the final successful design, and follow a plan that can solve all of them."
  3. "In other design contexts, such as ritual design [LW · GW] or event center construction [LW · GW], make sure not to be overwhelming, present relevant information clearly, simplify or combine bits until you have an elegant solution."
  4. "In any difficult problem, identify all the necessary constraints, and follow a plan that can solve all of them."

Here's a short handle for each of those:

#1, #3 and #4 are each actually pretty useful to think about. I'll be doing a lot of UI design. The specific UI tactics will come up again, and I don't wanna have to rederive them from first principles every time. 

Many design principles transfer between domains. I design lots of kinds of things. It's useful to remember the design-specific tactics whenever they come up.

But "identify and deal with all the constraints" is an incredibly general tool. I should apply that all over the place, whenever I'm dealing with something pretty hard that I expect to spend at least several hours on.

One example of overgeneralization would be, in situations where I basically already know how to solve the problem (i.e. a similar UI design problem), if I were to go Full Meta and try to original see on the constraints instead of just executing on the tactics I already know how to do. 

Skills into Principles

Many people do the first steps, and then are like "but, it would have been impossible to have done better." I think this is almost always false. But, I haven't found it that useful to argue with that directly, and instead focus on "what are the skills that you hypothetically could have already trained on, which would have helped?"

This then prompts the followup question "okay, are those skills going to come up a lot in your life?" If so, maybe focus on training those skills more deliberately.

But, training skills is pretty slow and expensive. Developing subtle taste and reflections takes time. A thing I've found helpful is to try to translate skills into "principles" – straightforward instructions you can remind yourself, that help steer your mind towards the right sorts of cognition.

In the UI case, the "skill" is in noticing that something felt overwhelming about the original design, and visualizing what it'd be like to encounter the UI for the first time. There's a bunch of taste and imagination skill involved. It's worth cultivating that skill, but the "what are the constraints?" and "how can I create a clear, simple information hierarchy?" are questions that help guide thinking more directly.

Part II: Thinking It Faster The First Time

That was the easy part. 

The hard part is noticing when you're about to think something Too Slowly, and... do something else instead.

I don't yet have that crisp an exercise for this. If you did the Think It Faster exercise for a toy puzzle, the lessons might not generalize to whatever you're likely to do next week. But after you've done it several times, you'll start to notice patterns. It's easiest to notice the patterns when you start applying the exercise to your day job, since you probably do similar things in your day job a lot.

(One example: I was working on an LLM-prompting scaffold. I assumed I would need elaborate setup of examples to initially prompt it with, and spend days working on it and iterating on it. Eventually... it turned out the simplest, dumbest prompt with only one example did better than my elaborate setup. The generalization: "just try doing the dumb simple thing first." The very next day, we were working on some other problem where we tried an elaborate complicated thing and then realized eventually the simple dumb thing would work, and I kicked myself for not having remembered the lesson I'd explicitly noted from the day before)

To train this quickly, it's important to find a way to apply generalizations towards something soon, so you get to reinforce it before it fades from top-of-mind.

Here's my current prompts for thinking about this:

Generalizing from this exercise 

First, consolidate your list of skills and principles

 

List past situations you could have benefited from those skills or principles

 

List future situations where you suspect might benefit from those skills or principles.

 

In the next week, what’s 1-3 tasks you’re doing that might benefit from those skills or principles?

 

Anticipating Future Life Lessons

The flipside of "how can this exercise generalize to real life?" is "what real life situations are likely to benefit from some kind of exercise?".

So an alternate set of prompts are:

In the next couple days, what's something you're planning to do that you expect to take a long time?

 

...what's something you're confused about, where you're not sure how to do it?

 

...what's something you expect to solve via tinkering/iteration without much of a plan, that you expect to take awhile?

 

These might be situations that don't naturally lend themselves to the most obvious life lessons from the exercise you just did. But, they might give you clues about additional life-lessons to be on the lookout for. Or, might give you clues about which sorts of toy exercises are useful to apply this practice to.

 

Getting Detailed, and TAPS

After you've soaked in some basic ideas for takeways, and some practical places to apply them, you want to get a lot more detailed. Form explicit intentions about when to remind yourself of some advice, and see if it's helpful.

For one of the past moments, think in detail about how principles/skills would apply.

(Imagine doing this whole doc again, for that past moment, and how you wish you’d thought-it-faster then. Don’t do the whole-ass version of the doc, just briefly think about the key moments)

 

 

For the future moments, how would the skills or principles apply? What would you hope you do, in the moment, to avoid taking longer or making mistakes? (When you imagine failing to remember in the moment, why was that? What steps could you take to avoid forgetting?)

 

 

Write down 3 tactical plans for remembering and applying lessons from this exercise during the next week. (They can be bad plans, and they can be short/rough. Ideally, they should include some actions you take right now, and some actions you’ll take later)

 

 

Pick any of the plans that seem worthwhile. Make an explicit prediction about whether it’ll work. (If it doesn’t feel that likely to work, ask “how can I improve this plan?” until you’d be surprised if it failed.)

 

 

Take whatever actions you can take right now.

 

 

Part III: The Five Minute Version

Doing all of this thoroughly takes a long time. I recommend doing it thoroughly the first couple times, to build a complete model of how everything fits together.

But, ultimately, to practice a skill, you need to get a lot of reps in. You can't get a lot of reps in if you have to dedicate an hour each time.

So, what's the five minute version of this? When you look at everything you just thought about, what were the single most important thoughts you had? What prompts would have helped direct you to those important thoughts?

I recommend thinking about this quickly rather than slowly/deliberately, to help practice the art of "just actually think the most important thoughts, don't overthink it", which is it's own skill.

The next time you naturally stumble into a situation you could have Thought Faster, apply the 5 minute version of this exercise.

You can probably find at least 1-3 moments per week that would benefit from Thinking It Faster. 

0 comments

Comments sorted by top scores.