Exploring the Idea Space Efficiently

I am not familiar with the neuroscience of what's going on, but a plausible story is that the areas of the brain corresponding to the examples he's just seen are activated.

This made me cringe. "I don't know the relevant science, but instead of looking it up, let me speculate anyway." To be fair, a lot of posts on LW probably do this, but few state it so explicitly. I'm only singling you out because you were so upright about it, which isn't how it should work. So I'm not downvoting you because of that.

The advice given at the end of this post seemed plausible to me, but the initial speculation about how the brain works seemed speculative and unsupported, and didn't really add anything to the post itself. I think the post could be improved by deleting everything between the initial paragraph and the one that starts "The fundamental thing you should do..."

What I would recommend that you do before that is come up with a map of the idea-space, describing where the possible solutions might be found

There is a formal technique for doing this, developed by Edward DeBono, and called a "concept fan". I did a quick web search but didn't find anything that actually explained the technique - everything I found on the web has a very shallow description that mostly misses the point of the technique, i.e., its systematic nature.

In a concept fan, you separate ideas from intentions, so that you see an idea like "carbon credits" as an idea for the intention of "reduce carbon use", which in turn is intended to "have less carbon in the atmosphere. You can then intentionally ask, "what are other ways to reduce carbon use?" or, "what are other ways to end up with less carbon in the atmosphere?", and so on.

(The technique is described in more detail in DeBono's book, "Serious Creativity"... a book which has as its main premise that creativity is something that can and should be approached systematically. Even when randomness is used (e.g. random words as provocations), they're recommended for use in a systematic way, for the specific purpose of keeping brains from going in circles over already-known territory, rather than as a source of mysterious inspiration.)

I would add that we should remember that proposing "categories in which solutions might fall" might itself be subject to priming/anchoring within category-space. =)

So for example, we could consider systematized ways to map the idea-space - I imagine that certain categories could be easily missed, and/or are very likely to show up for specific classes of problems.

I nominate this skill as something which is both worth practicing at the Center for Modern Rationality and is specific enough to do so.

This is a decent recapitulation of what has been discussed here to no end, what is missing is how to recover from the situation. Once you realize that you have been primed and anchored, what do you do to recover? (I have some ideas, but will refrain from proposing them just yet.)

Interesting concept! Come to think of it, I've found myself doing something similar to this, for pretty much the same reasons you decribed: I'll think of a solution, and then say "okay, cool solution -- now drop it from your mind, think about the problem again, and then come up with a different solution." Repeat a few times, then judge the solutions you've come up with. Here's how it might work for Simon, looking for examples for his calculus textbook:

1. Simon thinks of engineering, then agriculture, then irrigation, then comes up with a cool example involving marginal increases in crop yield from money spent on irrigation. He makes a note of it, then carefully stops thinking about it.

2. Simon thinks of something not engineering. Whenever he starts thinking of engineering, he stops thinking of engineering and instead thinks of something else. Specifically, he thinks of biology, then evolution, then comes up with an example involving peppered moths and changing allele frequencies caused by changing probabilities of being eaten by a predator before reproducing, coming from changes in the soot concentration in England during the Industrial Revolution.

3. Simon mentally blacklists engineering and biology, and decides to derive some basic equations for the motion of an object under continuous acceleration, like a perfectly spherical cow shot out of a cannon. Everybody loves those examples, right? I know I did.

In other words, as an alternative to doing breadth-first or simple depth-first search, you can do depth-first search while marking recently-explored areas of the idea space as off-limits since you've thought about them recently. (This sounds similar to Tabu search, if you enjoy over-stretching metaphors as much as I do.)

The thing I like most about this method is that it's really easy to remember to do -- I just need a reflex that says "now clear your head and think of a different solution" to make my saving roll against tunnel vision. I'm not sure I would have the willpower to start listing possible areas in idea-space before exploring any of them, at least not on a regular basis.

(By the way, whatever method ends up working best, avoiding tunnel vision is a really important skill. Using the method I just described, I managed to save myself several days of work about a week ago. That's pretty high payoff for such a simple habit of thought!)

Excellent post!

For instance, before looking at a single example of calculus, Simon might have written down a list of idea areas to explore: "jobs, personal life, the natural world, engineering, other". He would then take the first broad category, "jobs" and expand it into a longer list, perhaps "agriculture, teaching, customer service, manufacturing, research, IT, other". With this longer list, he can then focus on each area in turn and either expand it further if it seems especially rife with examples, or come up with an example from the area directly.

I am going to start searching for any opportunity I can find to use this method, because I'm really curious to see how well it works.

Hmmm, interesting. It seems like the trick is to just take "Be Specific" and reverse it - go up the abstraction ladder until you reach a more general area, one which is small enough to easily jot down a lot of general categories. Then take each of those and be more specific.

Basically, you're stuck in one "branch" of the "abstract tree", and if you just climb back to the root of the tree, you can then explore other branches. (I'm a programmer, so my trees are naturally upside down. For everyone else, apologies for the mixed directional metaphor :))

This is an interesting suggestion on how to be more creative in approaching a problem.

My hunch is that you could do scholarship and learn a lot more about creativity, however. For instance, did you know that being in a room with the color green makes people generate more creative ideas? (Source: 59 Seconds.) That's just a tidbit; there may be a lot more out there that's already known!

Off Topic

 What is the next number in the sequence 14, 15, 16, 17, 21, 23, 30, 33?

damn that 30, i can't think of any valid hypothesis that includes that.

It looks like the only reason Simon might stand a chance of success is because he is in StorySpace; he can start off with a broad search base because his narrator is leading him in the right direction. If I did the same thing, my research on calculus would turn into a manual on guinea-pig nutritional tips or something. The probability of a breakthrough from that direction is extremely improbable, which is why one would benefit from just a little bit of priming.