The "Think It Faster" Exercise

post by Raemon · 2024-12-11T19:14:10.427Z · LW · GW · 31 comments

Contents

  Example: 10x UI designers
    Okay, but how do you "Think it Faster?"
  THE EXERCISE
  Part I: Thinking it Faster
    Steps you actually took
    Magical superintelligence steps
    Iterate on those lists
      What skills, if you’d trained for 20 or 100 hours, would have helped you find the answer intuitively?
      What principles, if you internalized and they came easily to mind, would have allowed you to make some of those leaps ~instantly, or at least much faster?
      What jumps-between-steps feel magical or unrealistic, in “magical short list”? 
      For the “original steps you took”, what steps could you have skipped? What would have been necessary to skip them?
      Overall, what takeaways do you want to remember for later?
      What's the broadest generalize that feels reasonable to draw?
    Generalizing, and not Overgeneralizing
    Skills into Principles
  Part II: Thinking It Faster The First Time
    Generalizing from this exercise 
      First, consolidate your list of skills and principles
      List past situations you could have benefited from those skills or principles
      List future situations where you suspect might benefit from those skills or principles.
      In the next week, what’s 1-3 tasks you’re doing that might benefit from those skills or principles?
    Anticipating Future Life Lessons
      In the next couple days, what's something you're planning to do that you expect to take a long time?
      ...what's something you expect to solve via tinkering/iteration without much of a plan, that you expect to take awhile?
    Getting Detailed, and TAPs
      For one of the past moments, think in detail about how principles/skills would apply.
      Take whatever actions you can take right now.
  Part III: The Five Minute Version
None
31 comments

Note: there is a shorter "Think it Faster" worksheet [LW · GW]. I'm curious which one people find easier as an initial read.

 

Ultimately, I don’t want to solve complex problems via laborious, complex thinking, if I can help it. Ideally, I'd want to basically intuitively follow the right path to the answer quickly, with barely any effort at all.

For a few months I've been experimenting with the "How Could I have Thought That Thought Faster?" [LW · GW] concept, originally described in a twitter thread by Eliezer:

Sarah Constantin: I really liked this example of an introspective process, in this case about the "life problem" of scheduling dates and later canceling them: malcolmocean.com/2021/08/int…

Eliezer Yudkowsky: See, if I'd noticed myself doing anything remotely like that, I'd go back, figure out which steps of thought were actually performing intrinsically necessary cognitive work, and then retrain myself to perform only those steps over the course of 30 seconds.

SC: if you have done anything REMOTELY like training yourself to do it in 30 seconds, then you are radically smarter/more able/etc than me and all the other people who do slower introspective practices.

SC: I don't know whether to be impressed or to roll to disbelieve.

EY: I mean I suspect that this actually requires something like a fast perceptual view of minds as engines and thoughts as doing work and like actually draws on my mind design knowledge, but, even so, I ask: Do you constantly look back and ask "How could I have thought that faster?"

SC: No, I've never asked that.

EY: Okay, well, every time I'm surprised by reality I look back and think "What about my model and my way of thinking could I change that would have predicted that better, without predicting a bunch of other things worse?"

I've been working to operationalize this as an exercise[1] you can train repeatedly, rather than hoping to remember to do when reality hands you a surprise. You can do the exercise after any difficult cognitive task (either a toy puzzle exercise, or a day-job project that took a long while). 

What would have been necessary for you to just look at the situation, and automagically find the right solution? (without overfitting, or generalizing in a way that would cause you to think unproductive thoughts in other sorts of situations?)

The goal of this exercise is to identify:

…that move you to correct solutions to problems as quickly as possible. 

This overall builds into two deep skills: 

Asking yourself retrospectively “How could I have Thought that Faster?” 

And then, prospectively, learning to ask “What am I about to predictably spend too long thinking about, and How Can I Think it Faster, the First Time?”

I don’t know the upper limits of these skills, but I am currently finding it fruitful to adopt the mindset of “if it took longer than 15 minutes and an LLM query, you probably took too long.” 

Relentlessly ask yourself how hours or days could have turned into 15 minutes. Sometimes 15 minutes was literally possible. Most of the time, I find the act of aiming for 15 minutes to be illuminating, and reveal at least some important wasted time or new principles.

Example: 10x UI designers

You’ve probably heard stories about “10x programmers”, who just intuitively steer towards good decisions and get things done dramatically faster than most developers. But I spend most of my time doing UI design, and have lately been thinking about "10x UI designers."

I remember, 17 years ago, at my first “real” job at a printing company. A client wanted us to design a brochure for them. I was “good at art”, but this was my first assignment professionally making art for someone else. I thought about what they needed, I labored carefully for 4 hours, generating ideas and fiddling around in my graphic software.

Eventually my senior partner came to look at it. He said “eh, this has a lot of problems. Here, you should do it this way.” I can’t remember if he told me what to do, or just went and did it himself. But, he bypassed all my tedious work by just intuitively knowing how to solve this-particular-class of problem already, and moving directly to the good ideas instead of working through bad ones.

Okay, so, skill is a thing. He had 10 years of experience, I didn’t. 

More recently, I was working for three days on the design for the new Glossary Editor on posts. I was meanderingly exploring a few options, including a separate “table-of-contents” section, and a “show the jargon terms and definitions in detail” section. It was very complex and took up a lot of space and was overwhelming to look at. 

My goal was to make it so authors could quickly skim the AI generated jargon, and make decisions about approving ones they liked, without much effort. 

There were tradeoffs between:

  1. Making it easy to skim
  2. Making it easy to actually make final decisions, which required knowing enough about what the AI generated definitions actually said.
  3. Fitting it into a small amount of space, so it didn’t disrupt the experience of people who didn’t care about the glossary at all.
  4. Making it simple and elegant to think about.
  5. Accurately conveying all the tools and affordances.

I labored for 3 days, shuffling around where-the-tradeoffs lived, incrementally reducing some of the issues. 

Then Oliver Habryka came into the room, took a look, and said “man, your information hierarchy here is all over the place.” Then he fiddled around for 20 minutes and found something dramatically better than what I had at the time. 

Since I had recently been asking myself “how could I have Thought That Faster?”, I took this as an opportunity to ask “what the hell just happened, and how could I have done it myself without Oliver’s help?”. 

Mulling it over, I observed that although part of the answer was “UI specific design taste honed by years of experience”, there were some specific questions he was asking that pointed his attention in much more productive directions (i.e. “how can we give this a clear, unified, simple information hierarchy?”)

The underlying, general skill seemed to be:

In each domain (UI, programming, x-risk), there are specific tactical tools for actually “dealing with all constraints.” In this case, domain-specific principles included: “many buttons weren’t necessary to show users initially, until they started interacting with it”, and “instead of having a table-of-contents and a detail-view, build a single view that's an 80/20 of both views at once.”

But it seems like there is a general skill of:

I've since applied the "Actually Deal With Constraints" principle to other "Things I coulda thought faster" (such as how my Feedbackloop-first Rationality agenda has evolved over the past year, and how I could have made the same progress in like a month), and found it pretty valuable.

Okay, but how do you "Think it Faster?"

To fit this back into the "Think it Faster" context, there are two next steps:

In this context, how could I have figured out this principle for myself, before waiting for Oliver to demonstrate it could be done much more quickly?

Here are several options:

But this is incomplete – I need to followup and ask "but why didn't I think any of those particular thoughts the first time? What was the shortest-possible nudge from the actual past, to an alternate past where I figured it out?"

I do think I should have tried any of the general-rationality-principles sooner (Think It Faster, Metastrategy, Third Options). It's worth diagnosing But those aren't the shortest path – those are explicitly a path with an extra step of "think meta thoughts until I find the right object thought." They also require asking not only "how could I have had that Meta Thought faster", but also "how could I have gotten from that meta-thought to the right answer, without having seen Oliver's solution?".

I think the shortest path was a combo of "try the simplest things first." 

I think the most achievable inflection points were the moments I felt "ugh, this is a little hard to parse", and noticing "a little hard to parse" might be a bigger deal than I thought it was.

Ideally, I would have thought "let's just start with the simplest thing first" at the very beginning. (I started with "the most obvious thing" which is somewhat different from the "simplest thing"). I've since got a lot of experience seeing "do the fucking simplest thing" come a lot in Think It Faster exercises, so now it's in the bundle of general practices I should do way more often but I still don't do enough.

THE EXERCISE

Okay. So, you've just done either a Toy Exercise puzzle, or you just noticed in your Real World that you either spent a long time figuring something out, or were surprised by something (which you didn't figure out at all)

I recommend Thinking Physics [LW · GW] and Baba is You [LW · GW] as sources of puzzles to start grinding on this. For your day job, I recommend learning to notice [LW · GW] when you have the sneaking suspicion something took longer than it needed to. (Once you've got a bit of practice, I recommend applying this even to places you don't have that sneaking suspicion, but it did practically take a long time)

(Baba is You is particular good because it tends to surprise you, and you can practice micro-versions of this exercise on individual surprises within a single Baba is You level, which helps train the general reflex of Notice Surprise -> Ask how you could have Thought It Faster in the moment)

At a high level, you're going to ask:

"How could you have Thought it Faster?"

"What did you learn, which’ll let you Think It Faster The First Time, later?

Rather than go through each step exhaustively, in sequence, I recommend cycling through them: jot down a few quick ideas for each prompt, circling back to the first one, with each pass giving you a sense of how all the pieces fit together.

Part I: Thinking it Faster

Steps you actually took

In chronological order (as best you remember) what happened?

  1.  
  2.  

Magical superintelligence steps

If you were a waaaay smarter version yourself, or if you imagine some other waaaay (unrealistically) smarter genius, what is the shortest number of steps you can possibly imagine this taking? 

(Right now, it's okay for this to feel like cheating)

  1.  
  2.  

Iterate on those lists

Identify steps in the first list you could straightforwardly remove, or simplify. And, identify steps to add to the second list until it no longer feels like unrealistic cheating. (i.e. if you're not overfitting. The plan doesn't imply you should spend tons of cognitive overhead all the time on minor, unimportant clues)

Try these prompts to help you:

What skills, if you’d trained for 20 or 100 hours, would have helped you find the answer intuitively?

 

What principles, if you internalized and they came easily to mind, would have allowed you to make some of those leaps ~instantly, or at least much faster?

 

What jumps-between-steps feel magical or unrealistic, in “magical short list”? 

 

For the “original steps you took”, what steps could you have skipped? What would have been necessary to skip them?

 

Overall, what takeaways do you want to remember for later?

 

What's the broadest generalize that feels reasonable to draw?

 

Generalizing, and not Overgeneralizing

So far, I have mostly seen people fail to generalize enough, rather than too much. It's certainly possible to fail in both directions, but I maybe suggest erring on the side of overgeneralizing, and wait until it actually hurts you to dial it back.

In the UI example above, here are a few takeaways I could have had:

  1. "In UI design, make sure not to be overwhelming, make sure to have a clear information hierarchy, try removing or simplifying or combining bits until you have an elegant but comprehensive tool."
  2. "In UI design, try to identify all the constraints for the final successful design, and follow a plan that can solve all of them."
  3. "In other design contexts, such as ritual design [LW · GW] or event center construction [LW · GW], make sure not to be overwhelming, present relevant information clearly, simplify or combine bits until you have an elegant solution."
  4. "In any difficult problem, identify all the necessary constraints, and follow a plan that can solve all of them."

Here's a short handle for each of those:

#1, #3 and #4 are each actually pretty useful to think about. I'll be doing a lot of UI design. The specific UI tactics will come up again, and I don't wanna have to rederive them from first principles every time. 

Many design principles transfer between domains. I design lots of kinds of things. It's useful to remember the design-specific tactics whenever they come up.

But "identify and deal with all the constraints" is an incredibly general tool. I should apply that all over the place, whenever I'm dealing with something pretty hard that I expect to spend at least several hours on.

One example of overgeneralization would be, in situations where I basically already know how to solve the problem (i.e. a similar UI design problem), if I were to go Full Meta and try to original see on the constraints instead of just executing on the tactics I already know how to do. 

Skills into Principles

Many people do the first steps, and then are like "but, it would have been impossible to have done better." I think this is almost always false. But, I haven't found it that useful to argue with that directly, and instead focus on "what are the skills that you hypothetically could have already trained on, which would have helped?"

This then prompts the followup question "okay, are those skills going to come up a lot in your life?" If so, maybe focus on training those skills more deliberately.

But, training skills is pretty slow and expensive. Developing subtle taste and reflections takes time. A thing I've found helpful is to try to translate skills into "principles" – straightforward instructions you can remind yourself, that help steer your mind towards the right sorts of cognition.

In the UI case, the "skill" is in noticing that something felt overwhelming about the original design, and visualizing what it'd be like to encounter the UI for the first time. There's a bunch of taste and imagination skill involved. It's worth cultivating that skill, but the "what are the constraints?" and "how can I create a clear, simple information hierarchy?" are questions that help guide thinking more directly.

Part II: Thinking It Faster The First Time

That was the easy part. 

The hard part is noticing when you're about to think something Too Slowly, and... do something else instead.

I don't yet have that crisp an exercise for this. If you did the Think It Faster exercise for a toy puzzle, the lessons might not generalize to whatever you're likely to do next week. But after you've done it several times, you'll start to notice patterns. It's easiest to notice the patterns when you start applying the exercise to your day job, since you probably do similar things in your day job a lot.

(One example: I was working on an LLM-prompting scaffold. I assumed I would need elaborate setup of examples to initially prompt it with, and spend days working on it and iterating on it. Eventually... it turned out the simplest, dumbest prompt with only one example did better than my elaborate setup. The generalization: "just try doing the dumb simple thing first." The very next day, we were working on some other problem where we tried an elaborate complicated thing and then realized eventually the simple dumb thing would work, and I kicked myself for not having remembered the lesson I'd explicitly noted from the day before)

To train this quickly, it's important to find a way to apply generalizations towards something soon, so you get to reinforce it before it fades from top-of-mind.

Here's my current prompts for thinking about this:

Generalizing from this exercise 

First, consolidate your list of skills and principles

 

List past situations you could have benefited from those skills or principles

 

List future situations where you suspect might benefit from those skills or principles.

 

In the next week, what’s 1-3 tasks you’re doing that might benefit from those skills or principles?

 

Anticipating Future Life Lessons

The flipside of "how can this exercise generalize to real life?" is "what real life situations are likely to benefit from some kind of exercise?".

So an alternate set of prompts are:

In the next couple days, what's something you're planning to do that you expect to take a long time?

 

...what's something you're confused about, where you're not sure how to do it?

 

...what's something you expect to solve via tinkering/iteration without much of a plan, that you expect to take awhile?

 

These might be situations that don't naturally lend themselves to the most obvious life lessons from the exercise you just did. But, they might give you clues about additional life-lessons to be on the lookout for. Or, might give you clues about which sorts of toy exercises are useful to apply this practice to.

 

Getting Detailed, and TAPs

After you've soaked in some basic ideas for takeways, and some practical places to apply them, you want to get a lot more detailed. Form explicit intentions about when to remind yourself of some advice, and see if it's helpful.

For one of the past moments, think in detail about how principles/skills would apply.

(Imagine doing this whole doc again, for that past moment, and how you wish you’d thought-it-faster then. Don’t do the whole-ass version of the doc, just briefly think about the key moments)

 

 

For the future moments, how would the skills or principles apply? What would you hope you do, in the moment, to avoid taking longer or making mistakes? (When you imagine failing to remember in the moment, why was that? What steps could you take to avoid forgetting?)

 

 

Write down 3 tactical plans for remembering and applying lessons from this exercise during the next week. (They can be bad plans, and they can be short/rough. Ideally, they should include some actions you take right now, and some actions you’ll take later)

 

 

Pick any of the plans that seem worthwhile. Make an explicit prediction about whether it’ll work. (If it doesn’t feel that likely to work, ask “how can I improve this plan?” until you’d be surprised if it failed.)

 

 

Take whatever actions you can take right now.

 

 

Part III: The Five Minute Version

Doing all of this thoroughly takes a long time. I recommend doing it thoroughly the first couple times, to build a complete model of how everything fits together.

But, ultimately, to practice a skill, you need to get a lot of reps in. You can't get a lot of reps in if you have to dedicate an hour each time.

So, what's the five minute version of this? When you look at everything you just thought about, what were the single most important thoughts you had? What prompts would have helped direct you to those important thoughts?

I recommend thinking about this quickly rather than slowly/deliberately, to help practice the art of "just actually think the most important thoughts, don't overthink it", which is it's own skill.

The next time you naturally stumble into a situation you could have Thought Faster, apply the 5 minute version of this exercise.

You can probably find at least 1-3 moments per week that would benefit from Thinking It Faster. 

  1. ^

    I don't know that Eliezer would endorse this particular exercise. I asked him once for more detail on how he applied the skill, and it seemed like he'd been doing it so long that most of it was a compressed atomic action he couldn't easily unpack.

  2. ^

    There are corresponding emotional skills of being dissatisfied without being too worked up about it, or manic. 

31 comments

Comments sorted by top scores.

comment by James Stephen Brown (james-brown) · 2024-12-12T17:29:28.284Z · LW(p) · GW(p)

Thanks for this, nice writing.

The idea of 'thinking it faster' is provocative, because it seems to be over-optimising for speed rather than other values, where as the way you're implementing it is by generating more meaningful or efficient decisions which are underpinned by a meta-analysis of your process—which is actually about increasing the quality of your decision-making.

I think it's worthwhile seeing where we're wasting time. But often I find wasted time isn't what you'd expect it to be. As someone who also works in the creative industry, criticism is a lot easier than creating something out of whole cloth. Your senior partner, doesn't just have more experience, but is also a fresh pair of eyes looking at the product you're creating from a macroscopic (user's) perspective—this is much easier when you're not mired in the minutiae. I get this feedback in my job (a documentary editor) not only from people more experienced than me, but also those less experienced.

There a two things I have learned from experience:

1. Blocking out a scene is useful, even though the scene will never be in that form—the boring form of the scene makes it easier to step back and see the more creative way to approach the scene. The time spent making the picture clearer isn't wasted.
2. When working alone, step away and view your work from a fresh perspective (in my case the audience, in yours the user) to be your own director / senior partner.

That being said, I think it's well worth meta-analysing your own process and that of your more experienced colleagues, another thing I've learned is...

3. When someone you trust gives you changes you don't agree with, try them, they probably have a clearer perspective than you do.

Anyway, thanks for the post, I'm planning to implement your advice in my own job, it sounds like a worthwhile process. I actually think this third thing is likely to be a key lesson learned from meta-analysis, to not be stubborn and to pivot to the better solution more freely, what I call "back it up and break it".

Replies from: Raemon, Raemon
↑ comment by Raemon · 2024-12-12T17:37:05.434Z · LW(p) · GW(p)

The idea of 'thinking it faster' is provocative, because it seems to be over-optimising for speed rather than other values, where as the way you're implementing it is by generating more meaningful or efficient decisions which are underpinned by a meta-analysis of your process—which is actually about increasing the quality of your decision-making.

I considered changing it to "Think it Sooner", which nudges you a bit away from "try to think frenetically fast" and towards "just learn to steer towards the most efficient parts of your thought process, avoid wasted motion, and use more effective metastrategies." "Think It Sooner" feels noticeably harder to say so I decided to stick with the original (although I streamlined the phrasing from "Think That Thought faster" a bit so it rolled off the tongue)

Replies from: james-brown
↑ comment by James Stephen Brown (james-brown) · 2024-12-12T18:39:45.540Z · LW(p) · GW(p)

I see, I think you're right not to change it—it's just provocative enough to be catchy.

↑ comment by Raemon · 2024-12-12T17:30:54.363Z · LW(p) · GW(p)

I actually think this third thing is likely to be a key lesson learned from meta-analysis, to not be stubborn and to pivot to the better solution more freely, what I call "back it up and break it".

I'm not sure I understood this point, could you say more?

Replies from: james-brown
↑ comment by James Stephen Brown (james-brown) · 2024-12-12T17:35:02.570Z · LW(p) · GW(p)

Wow, that was quick. I mean, rather than scaffolding work that seems unproductive but is actually necessary, most creative time (for me at least) is wasted in resisting change (my number 3 point was about trying changes even if you don't immediately agree with them).

Replies from: Raemon
↑ comment by Raemon · 2024-12-12T17:37:36.621Z · LW(p) · GW(p)

Ah gotcha. Yeah, this is why Deliberate Grieving [LW · GW] is a core rationalist skill.

Replies from: james-brown
↑ comment by James Stephen Brown (james-brown) · 2024-12-13T01:36:02.542Z · LW(p) · GW(p)

That (deliberate grieving) was also an interesting read, yes, exactly.

comment by R S (r-s-2) · 2025-02-09T19:06:19.414Z · LW(p) · GW(p)

I basically discovered the same form of thinking after I learned the concept of amplification and distillation 

The long-term results of this was severe OCD that took me 1.5-2 years to cut back

This is not to say that it's a bad idea, or a bad idea for everyone 

But it's a very, very bad idea for some people 

Basically I would spend so many hours every day just thinking, stuck in thought loops, stuck trying to gain some value out of generalizing 

Trying to compress mental models that are impossible to learn implicitly into simplified models/memonics and then learn how to use them reflectively 

Etc etc 

It's a fucking mess if you go too far 

And it's hard to know if you've gone too far 

At least it was for me, until I was slowly working through it through months of therapy 

And then realized oh this is really all because of that stupid fucking thought pattern thing I'm obsessed with

Especially the idea of trying to think it faster the FIRST time

That in particular can be paralyzing, this sense that you should be able to think it the right way the first time 

Very slippery slope

It's like mindfulness, generally good up to a point, but it's absolutely possible to go too far and have it cripple your life

Replies from: Raemon
↑ comment by Raemon · 2025-02-09T23:31:43.661Z · LW(p) · GW(p)

Mm, that does make sense, thanks for the warning.

Part of my overall flow is "during working hours (where I'm trying to apply this sort of thing), any 'meta' I do is something that needs to pay off within a week." (I let myself do meta in weekends/evenings with less restriction, coming more out of my "hobby/after-hours-self-improvement" budget).

"Have a handle on when you're going overboard and need to focus back on just getting momentum on object-level progress" makes sense as an important foundational skill here. 

On my day-to-day, I do the 5-10 minute version of this in the morning.

Do you feel right now like you found a happy balance, or are you kinda in "recovering alcoholic, not even one drink of think-it-faster-meta is safe?"

Replies from: r-s-2
↑ comment by R S (r-s-2) · 2025-02-11T05:01:29.040Z · LW(p) · GW(p)

Yeah I like that approach 

Part of it's probably that I work very long hours often 7 days a week (blah blah stash money before ASI kills my differentiator)

The biggest thing is not having a goal of being able to prove a solution to myself 

Like stop the search early basically

Don't need to formalize or document everything or remember every finding

I think most of the time the brain actually does a pretty good job of gradually solving problems over time without conscious thought 

We don't think of it as thought because it's not conscious or subtitled by our auditory processing system

But it still thought & learning

Relying on that implicit system instead often gets me better results than logically formalized versions

I guess now I'm kind of debating the core premise of the post -- that it's even possible or useful overall to discover or learn in that way, or that it's real learning

But it probably is helpful for some % of people who are less tunnely / tic-prone

Maybe I'm just jealous that it works for some people after devoting (and largely wasting) so much time on it myself

Not sure

Replies from: Raemon
↑ comment by Raemon · 2025-02-11T05:25:31.861Z · LW(p) · GW(p)

FYI I do currently think "learn when/how to use your subconcious to process things" is an important tool in the toolbox (I got advice about that from a mentor I went to talk to). Some of the classes of moves here are:

  • build up intuitions about when it is useful to background process things vs deliberate-process them
  • if your brain is sort of subconsciously wandering in a rut, use a small amount of agency to direct your thoughts in a new direction, but then let them wander once you get them rolling down the hill in that new direction
comment by Luke Cheng (luke-cheng) · 2025-02-10T18:30:46.920Z · LW(p) · GW(p)

Do you have any real world feedback for this?  I think the idea is provisionally great, but the article would be 10x better at conveying the loop of learning to others if you gave many concrete examples where this actually helped and was worth the effort. My guess is that it is really exponential over time. 

I have some examples in my feedback loop documents where I do do this and it has really helped. 

I also have 2 extra versions where I try 1) using system 1 thinking explicitly for system 2, 2) I skip forward in time and just think about the entire next conversation that would unfold after the one in my present head. 

Replies from: Raemon
↑ comment by Raemon · 2025-02-11T00:38:01.002Z · LW(p) · GW(p)

The feedbackloops in escalating "realness" here for me are:

  • Do I identify principles/skills/habits/etc that seem like they should successfully cut down on time spent on things I regularly do?
  • Do I successfully identify moments where it seems like I should "think something faster the first time?", by applying a technique?
  • Do I do that? Does it seem to save time?

("does it seem to save time?" isn't an ironclad feedbackloop obviously. But, I think it + common sense is at least pretty good)

I've been doing some-kind-of-variant on this since 2023 with the Thinking Physics exercise "reflection portion". Everything in Skills from a year of Purposeful Rationality Practice [LW · GW] I think at least somewhat counts as habits that I've gained that allow me to think either think-things-faster, or, think-things-at-all.

I workshopped and ad-hoc "review your thinking for 10 minutes" after various exercises into the ~hour-long exercise you see here, a few months ago. In that time, some new things I try at least sometimes

  • Look at my checklist for debugging code, and do the things on it. These so far include:
    • "actually adopt a stance of 'form hypotheses and try to disprove them'"
    • "patiently follow the code all the way up the stack" (instead of bouncing off after the second step)
    • "binary search for where the problem is by commenting out ~half the code in the relevant section."

      (these may seem obvious but I'm just not that strong a developer, and exercises like this are the main mechanism by which I've gotten better at basic debugging skills)
  • Try the simple dumb thing first. (I still fail to do this an embarrassing amount of time, but am working on it)
  • When I notice myself flailing around myopically,
    • a) these days, try getting a Thinking Assistant [LW · GW] for the day.
    • b) back in December, when I first was noticing I was struggling to focus, I decided to write the Thinking Assistants post and spin up a Thinking Assistant community. The general form of that is "consider spinning up whole-ass subcommunities to deal with problems." (I knew from previous experience that finding a single thinking assistant was a brittle solution)
  • Also when I'm myopically flailing, try forming a more complete model of my constraints (as described in this blogpost), and then solve for them.

The first three things feel like they're straightforwardly working, although it's hard to tell how much they actually speed me up. (Often the thing I would previously do when failing to debug code was "ask someone for help", so it's less that there's a speedup exactly and more that I interrupt my colleagues less)

The fourth one, I feel like I'm still workshopping into a form that reliably works for me, because "make a map of the constraints" is made of a lot of subskills, which vary depending on the situation. I anticipate it turning into something pretty important over the next year but it's too early to tell.

comment by Ruby · 2025-02-09T06:25:55.531Z · LW(p) · GW(p)

Curated. I was reluctant to curate this post because I found myself bouncing off it some due to length – I guess in pedagogy there's a tradeoff between explaining at length (and you lose people) and you convey enough info vs keeping it brief and people read it but they don't get enough. Based on private convo, Raemon thinks length is warranted.

I'm curating because I do think this kind of project is valuable. Everyday it feels easier to lose our minds entirely to AI, and I think it's important to remember we can think better or worse, and we should be trying to do the former. 

I have mixed feeling about Raemon's project overall. Parts of it feel good, something feels missing (I think I'm partial to John Wentworth's claim elsewhere that you need a bunch of technical study in the recipe), but I except the stuff Raemon is developing to be helpful to have engaged with for anyone who gets better at thinking.

Replies from: Domenic
↑ comment by Domenic · 2025-02-10T06:19:34.131Z · LW(p) · GW(p)

It's interesting to compare this to the other curated posts I got in my inbox over the last week, What is malevolence? [LW · GW] and How will we update about scheming [LW · GW]. Both of those (especially the former) I bounced off of due to length. But this one I stuck with for quite a while, before I started skimming in the worksheet section.

I think the instinct to apply a length filter before sending a post to many peoples' inboxes is a good one. I just wish it were more consistently applied :)

comment by Raemon · 2024-12-13T19:19:23.290Z · LW(p) · GW(p)

Someone pointed out that I didn't really explain how "identify all the constraints" fit into "think it faster". I added another piece to the Example section to tie it back together more.

comment by agazi (mikael-1) · 2025-02-15T03:42:47.812Z · LW(p) · GW(p)

Well written. I find a large amount of my thinking process involves long lookups to some part of my brain that I can't visualize very well . This could be due to my particular split of inner monologue and abstract thought, but I'm now finding it challenging to optimize the speed of the abstract thought.

This feels like LLM interpretability with and without COT.

comment by Alex Vermillion (tomcatfish) · 2025-02-13T19:18:24.943Z · LW(p) · GW(p)

I'm curious which one people find easier as an initial read.

I used the random.org list randomizer to shuffle my reading order. It said worksheet then post.

Replies from: tomcatfish
↑ comment by Alex Vermillion (tomcatfish) · 2025-02-13T19:36:51.856Z · LW(p) · GW(p)

I did that. I didn't want to fill in the worksheet, but I read through it. Now I'm reading through this post and noting anything that I understood differently when I read the worksheet or that is worth mentioning:

  • The post has links and a story at the start that could draw some people away, but I didn't have an issue (except for noticing that other people could, yuck)
  • The link might cause people to be drawn away. I almost added it to my tabs and then decided to ignore it
  • The story about Yudkowsky might cause people to get distracted.
  • There's a lot in this one that details how you got here, which isn't really about why someone should read on. The worksheet didn't noticeably have this, which meant I got into your ideas faster.
  • The example about UI I like. It is helpful to see an example early on instead of a framework that I have to imagine an example for.
  • It is quite detailed though.

Overall, I think both had their ups and downs. The 30-seconds-of-thought synthesis, I think, is to try to make a worksheet, slap a small example problem and solution above it, then work through the example below. Your other sections can fall yet lower. This would give you a strong "So What?" hook like your example gave, while getting readers into the meat of your technique quickly. Working the example below makes the technique less impersonal, and would make the total feeling of reading your post be

  • So what? Oh, that would be nice to be able to do
  • Hrm. I need a bit more explanation
  • Oh, I see! That's not so bad
  • [reading your further sections]
Replies from: Raemon
↑ comment by Raemon · 2025-02-13T20:59:43.727Z · LW(p) · GW(p)

Thanks! I'll keep this in mind both for potential rewrites here, and for future posts.

comment by Kaleb Knotts (kaleb-knotts) · 2025-02-11T01:31:16.250Z · LW(p) · GW(p)

My takeaway from this and from real life experience is that similar memories are usually what allow us to be able to complete tasks better. Schema theory can apply to this concept for people. I was specifically thinking about how I viewed video game genres. I was talking to someone younger than me and I had to explain beat em ups. I went through a list of different games until I found one that they knew and they responded with "OH, those types of games". 

My next takeaway from this, NLP tokenizers work this way already i believe. Perhaps theres a way to apply this concept to your way or as the term i liked down below, thinking it sooner. Good post, please keep it up. 

comment by Domenic · 2025-02-10T06:30:13.591Z · LW(p) · GW(p)

These posts always leave me feeling a little melancholy that my life doesn't seem to have that many challenges where thinking faster/better/harder/sooner would actually help.

Most of my waking hours are spent on my job, where cognitive performance is not at all the bottleneck. (I honestly believe that if you made me 1.5x "better at thinking", this would not give a consistent boost in output as valued by the business. I'm a software engineer.) I have some intellectual spare-time hobbies, but the most demanding of them is Japanese studying, which is more about volume, exposure, and spaced repetition than clever strategies. I am intrigued by making myself more productive in my programming side projects, but I think the biggest force multiplier for me there is learning how to leverage AI agents more effectively. (Besides the raw time savings, rapid iteration speed can also lessen the need for thinking of the right solution the first time around.)

I can easily see how this would be an important skill for someone doing novel academic-ish research, however. And I wish some of the examples were about that, instead of Thinking Physics and video games!

comment by João Ribeiro Medeiros (joao-ribeiro-medeiros) · 2025-02-10T03:22:30.985Z · LW(p) · GW(p)

I love this! Clarity (and clearance xD [latin pun here]) is all we need.

One addition which I think can be relevant to the discussion on optimization and meta-strategizing is also the acknowledgement of the simple fact that to meta-strategize you need to apply time to something which does not immediately returns backlog completion. 

So to be quicker in the future, you have to invest time which at first will appear to slow you down, that's something which needs to be underscored given the reality of day jobs in engineering and related areas. 

Recognizing the need for principles, is also conditioned on in part disregarding the immediacy of the ungeneralized day-to-day task. That's meta-management right there.

Replies from: Raemon
↑ comment by Raemon · 2025-02-10T03:26:40.891Z · LW(p) · GW(p)

I agree that it requires upfront investment, but, a few comments on this post are reminding me "oh right the default thing is that everyone falls into The Meta Trap", wherein people invest in meta things that end up not paying off.

My solution to this is to set standards for myself that involve keeping up a "ship quickly" momentum, and generally aim to spend ~10% of your time on meta.

comment by PoignardAzur · 2025-02-09T09:13:47.131Z · LW(p) · GW(p)

First, I could have just gone and talked to Oliver earlier.

I'd say you're underrating that option.

Part of it is domain-specific: a lot of developers start off with very little agency, and get "try to do the thing yourself before asking to the teacher how to do the thing" drilled into them; it's easy to overcorrect to "never ask for help". Learning to ask for help faster is a valuable senior developer skill.

On a more general level, "asking for help faster" is a disgustingly common answer to the question "how could I have found the solution sooner?". Life isn't an exam, you don't get points off for talking to people. (And using ChatGPT, StackOverflow, etc.)

I recommend Thinking Physics and Baba is You as sources of puzzles to start grinding on this

Mhh. Interesting. I haven't played Baba Is You in a while. It has enough puzzles that I think you could actually practice some skills with it.

I might try your routine, though I'm a bit skeptical of it.

comment by mattmacdermott · 2024-12-13T20:13:29.612Z · LW(p) · GW(p)

Is Eliezer claiming to (train himself to perform only those steps) over thirty seconds, or train himself to (perform only those steps over thirty seconds)?

Replies from: Raemon, PoignardAzur
↑ comment by Raemon · 2024-12-13T20:15:26.840Z · LW(p) · GW(p)

I don't actually know for sure. 

The thing I think he meant was "he trains (over a longish period of time, or at least more than 30 seconds) to perform only the essential steps (in 30 seconds)". That's at least what I'm aiming at. (I'm setting the less ambitious initial goal of "~15 minutes.")

This essay doesn't actually focus much on the followup "drill yourself until you can actually do the steps in [30 seconds / 15 minutes]" because it feels early stage enough that I'm not quite sure which things make most sense to drill. 

Although now that I draw my attention to that I think I should maybe be prioritizing the followup drilling harder. I'm trying to have more Purposeful Practice be part of my life but it's labor-intensive.

Replies from: mattmacdermott
↑ comment by mattmacdermott · 2024-12-13T20:26:57.023Z · LW(p) · GW(p)

This might be the source of Sarah Constantin’s incredulity?

Replies from: Raemon
↑ comment by Raemon · 2024-12-13T20:28:39.484Z · LW(p) · GW(p)

I dunno, @sarahconstantin [LW · GW] do you remember?

(I'm also curious what @Eliezer Yudkowsky [LW · GW] thinks of this post, for that matter, if he's up for it)

↑ comment by PoignardAzur · 2025-02-09T08:58:31.098Z · LW(p) · GW(p)

Yeah, it seems a little weird to me that the post includes Eliezer's claim uncritically. "I totally train myself to improve on the spot, all the time" seems like a bold claim for someone who's admitted to having unusually low willpower?

comment by moreright2 · 2024-12-12T06:29:32.906Z · LW(p) · GW(p)

Good post. I think you incorrectly linked your local version of Feedbackloop-first Rationality.