Cultivating a state of mind where new ideas are born

post by Henrik Karlsson (henrik-karlsson) · 2023-07-27T09:16:42.566Z · LW · GW · 20 comments

This is a link post for https://www.henrikkarlsson.xyz/p/good-ideas

Contents

  Part 1: Alexander Grothendieck
  First contact with the creative state
  Questions over answers
  Working with others without losing yourself
  Part 2: Ingmar Bergman
  Going sub-Bergman
  More techniques 
None
20 comments

In the early 2010s, a popular idea was to provide coworking spaces and shared living to people who were building startups. That way the founders would have a thriving social scene of peers to percolate ideas with as they figured out how to build and scale a venture. This was attempted thousands of times by different startup incubators. There are no famous success stories.

In 2015, Sam Altman, who was at the time the president of Y Combinator, a startup accelerator that has helped scale startups collectively worth $600 billion, tweeted in reaction that “not [providing coworking spaces] is part of what makes YC work.” Later, in a 2019 interview with Tyler Cowen, Altman was asked to explain why.

SAM ALTMAN: Good ideas — actually, no, great ideas are fragile. Great ideas are easy to kill. An idea in its larval stage — all the best ideas when I first heard them sound bad. And all of us, myself included, are much more affected by what other people think of us and our ideas than we like to admit.

If you are just four people in your own door, and you have an idea that sounds bad but is great, you can keep that self-delusion going. If you’re in a coworking space, people laugh at you, and no one wants to be the kid picked last at recess. So you change your idea to something that sounds plausible but is never going to matter. It’s true that coworking spaces do kill off the very worst ideas, but a band-pass filter for startups is a terrible thing because they kill off the best ideas, too.

This is an insight that has been repeated by artists, too. Pablo Picasso: “Without great solitude, no serious work is possible.” James Baldwin: “Perhaps the primary distinction of the artist is that he must actively cultivate that state which most men, necessarily, must avoid: the state of being alone.” Bob Dylan: “To be creative you’ve got to be unsociable and tight-assed.”

When expressed in aphorisms like this, you almost get the impression that creativity simply requires that you sit down in a room of your own. In practice, however, what they are referring to as solitude is rather something like “a state of mind.” They are putting themselves in a state where the opinions of others do not bother them and where they reach a heightened sensitivity for the larval ideas and vague questions that arise within them. 

To get a more visceral and nuanced understanding of this state, I’ve been reading the working notes of several highly creative individuals. These notes, written not for publication but as an aid in the process of discovery, are, in a way, partial windows into minds who inhabit the solitary creative space which the quotes above point to. In particular, I’ve found the notes of the mathematician Alexander Grothendieck and the film director Ingmar Bergman revealing. They both kept detailed track of their thoughts as they attempted to reach out toward new ideas. Or rather, invited them in. In the notes, they also repeatedly turned their probing thoughts onto themselves, trying to uncover the process that brings the new into the world.

This essay is not a definite description of this creative state, which takes on many shapes; my aim is rather to give a portrait of a few approaches, to point out possibilities.

Part 1: Alexander Grothendieck

It is as if there existed, for what seems like millennia, tracing back to the very origins of mathematics and of other arts and sciences, a sort of “conspiracy of silence” surrounding [the] “unspeakable labors” which precede the birth of each new idea, both big and small[.]

— Alexander Grothendieck, Récoltes et Semailles

In June 1983, Alexander Grothendieck sits down to write the preface to a mathematical manuscript called Pursuing Stacks. He is concerned by what he sees as a tacit disdain for the more “feminine side” of mathematics (which is related to what I’m calling the solitary creative state) in favor of the “hammer and chisel” of the finished theorem. By elevating the finished theorems, he feels that mathematics has been flattened: people only learn how to do the mechanical work of hammering out proofs, they do not know how to enter the dreamlike states where truly original mathematics arises. To counteract this, Grothendieck in the 1980s has decided to write in a new way, detailing how the “work is carried day after day [. . .] including all the mistakes and mess-ups, the frequent look-backs as well as the sudden leaps forward”, as well as “the early steps [. . .] while still on the lookout for [. . .] initial ideas and intuitions—the latter of which often prove to be elusive and escaping the meshes of language.”

This was how he had written Pursuing Stacks, the manuscript at hand, and it was the method he meant to employ in the preface as well. Except here he would be probing not a theorem but his psychology and the very nature of the creative act. He would sit with his mind, observing it as he wrote, until he had been able to put in words what he meant to say. It took him 29 months.

When the preface, known as Récoltes et Semailles, was finished, in October 1986, it numbered, in some accounts, more than 2000 pages. It is in an unnerving piece of writing, seething with pain, curling with insanity at the edges—Grothendieck is convinced that the mathematical community is morally degraded and intent on burying his work, and aligns himself with a series of saints (and the mathematician Riemann) whom he calls les mutants. One of his colleagues, who received a copy over mail, noticed that Grothendieck had written with such force that the letters at times punched holes through the pages. Despite this unhinged quality, or rather because of it, Récoltes et Semailles is a profound portrait of the creative act and the conditions that enable our ability to reach out toward the unknown. (Extracts from it can be read in unauthorized English translations, here and here.) 

First contact with the creative state

An important part of the notes has Grothendieck meditating on how he first established contact with the cognitive space needed to do groundbreaking work. This happened in his late teens. It was, he writes, this profound contact with himself which he established between 17 and 20 that later set him apart—he was not as strong a mathematician as his peers when he came to Paris at 20, in 1947. That wasn’t the key to his ability to do great work.

I admired the facility with which [my fellow students] picked up, as if at play, new ideas, juggling them as if familiar with them from the cradle—while for myself I felt clumsy, even oafish, wandering painfully up an arduous track, like a dumb ox faced with an amorphous mountain of things that I had to learn (so I was assured), things I felt incapable of understanding[.]

Grothendieck was, to be clear, a strong mathematician compared to most anyone, but these peers were the most talented young mathematicians in France, and unlike Grothendieck, who had spent the war in an internment camp at Rieucros, near Mende, they had been placed in the best schools and tutored. They were talented and well-trained. But the point is: being exceptionally talented and trained was, in the long run, not enough to do groundbreaking work because they lacked the capacity to go beyond the context they had been raised in.

In fact, most of these comrades who I gauged to be more brilliant than I have gone on to become distinguished mathematicians. Still, from the perspective of 30 or 35 years, I can state that their imprint upon the mathematics of our time has not been very profound. They’ve all done things, often beautiful things, in a context that was already set out before them, which they had no inclination to disturb. Without being aware of it, they’ve remained prisoners of those invisible and despotic circles which delimit the universe of a certain milieu in a given era. To have broken these bounds they would have had to rediscover in themselves that capability which was their birth-right, as it was mine: the capacity to be alone.

The capacity to be alone. This was what Grothendieck had developed. In the camp during the war, a fellow prisoner named Maria had taught him that a circle can be defined as all points that are equally far from a point. This clear abstraction attracted him immensely. After the war, having only a limited understanding of high school mathematics, Grothendieck ended up at the University of Montpellier, which was not an important center for mathematics. The teachers disappointed him, as did the textbooks: they couldn’t even provide a decent definition of what they meant when they said length! Instead of attending lectures, he spent the years from 17 to 20 catching up on high school mathematics and working out proper definitions of concepts like arc length and volume. Had he been in a good mathematical institution, he would have known that the problems he was working on had already been solved 30 years earlier. Being isolated from mentors he instead painstakingly reinvent parts of what is known as measurement theory and the Lebesgue integral. 

A few years after I finally established contact with the world of mathematics at Paris, I learned, among other things, that the work I’d done in my little niche [. . . had] been long known to the whole world [. . .]. In the eyes of my mentors, to whom I’d described this work, and even showed them the manuscript, I’d simply “wasted my time”, merely doing over again something that was “already known”. But I don't recall feeling any sense of disappointment. [. . .]

The three years of solitary work at Montpellier had not been wasted in the least: that intellectual isolation was what had allowed him to access the cognitive space where new ideas arise. He had made himself at home there.

Without recognizing it, I’d thereby familiarized myself with the conditions of solitude that are essential for the profession of mathematician, something that no-one can teach you. [. . .]

To state it in slightly different terms: in those critical years I learned how to be alone.

[. . .] these three years of work in isolation, when I was thrown onto my own resources, following guidelines which I myself had spontaneously invented, instilled in me a strong degree of confidence, unassuming yet enduring, in my ability to do mathematics, which owes nothing to any consensus or to the fashions which pass as law....

This experience is common in the childhoods of people who go on to do great work, as I have written elsewhere. Nearly everyone who does great work has some episode of early solitary work. As the philosopher Bertrand Russell remarked, the development of gifted and creative individuals, such as Newton or Whitehead, seems to require a period in which there is little or no pressure for conformity, a time in which they can develop and pursue their interests no matter how unusual or bizarre. In so doing, there is often an element of reinventing the already known. Einstein reinvented parts of statistical physics. Pascal, self-teaching mathematics because his father did not approve, rederived several Euclidean proofs. There is also a lot of confusion and pursuit of dead ends. Newton looking for numerical patterns in the Bible, for instance. This might look wasteful if you think what they are doing is research. But it is not if you realize that they are building up their ability to perceive the evolution of their own thought, their capacity for attention.

Questions over answers

One thing that sets these intensely creative individuals apart, as far as I can tell, is that when sitting with their thoughts they are uncommonly willing to linger in confusion. To be curious about that which confuses. Not too rapidly seeking the safety of knowing or the safety of a legible question, but waiting for a more powerful and subtle question to arise from loose and open attention. This patience with confusion makes them good at surfacing new questions. It is this capacity to surface questions that set Grothendieck apart, more so than his capacity to answer them. When he writes that his peers were more brilliant than him, he is referring to their ability to answer questions. It was just that their questions were unoriginal. As Paul Graham observes:

People show much more originality in solving problems than in deciding which problems to solve. Even the smartest can be surprisingly conservative when deciding what to work on. People who’d never dream of being fashionable in any other way get sucked into working on fashionable problems. 

Grothendieck had a talent to notice (and admit!) that he was subtly bewildered and intrigued by things that for others seemed self-evident (what is length?) or already settled (the Lebesgue integral) or downright bizarre (as were many of his meditations on God and dreams). From this arose some truly astonishing questions, surfacing powerful ideas, such as topoi, schemas, and K-theory.

Working with others without losing yourself

So far, we’ve talked about solitary work. But that has its limitations. If you want to do great work you have to interface with others—learn what they have figured out, find collaborators who can extend your vision, and other support. The trick is doing this without losing yourself. What solitude gives you is an opportunity to study what personal curiosity feels like in its undiluted form, free from the interference of other considerations. Being familiar with the character of this feeling makes it easier to recognize if you are reacting to the potential in the work you are doing in a genuinely personal way, or if you are giving in to impulses that will raise your status in the group at the expense of the reach of your work.

After his three years of solitary work, Grothendieck did integrate into the world of mathematics. He learned the tools of the trade, he got up to date on the latest mathematical findings, he found mentors and collaborators—but he was doing that from within his framework. His peers, who had been raised within the system, had not developed this feel for themselves and so were more susceptible to the influence of others. Grothendieck knew what he found interesting and productively confusing because he had spent three years observing his thought and tracing where it wanted to go. He was not at the mercy of the social world he entered; rather, he “used” it to “further his aims.” (I put things in quotation marks here because what he’s doing isn’t exactly this deliberate.) He picked mentors that were aligned with his goals, and peers that unblock his particular genius. 

I do not remember a single occasion when I was treated with condescension by one of these men, nor an occasion when my thirst for knowledge, and later, anew, my joy of discovery, was rejected by complacency or by disdain. Had it not been so, I would not have “become a mathematician” as they say—I would have chosen another profession, where I could give my whole strength without having to face scorn. [My emphasis.]

He could interface with the mathematical community with integrity because he had a deep familiarity with his inner space. If he had not known the shape of his interests and aims, he would have been more vulnerable to the standards and norms of the community—at least he seems to think so.

Part 2: Ingmar Bergman

Yet. Even if you know what it feels like to be completely open to where your curiosity wants you to go, like Grothendieck, it is a fragile state. It often takes considerable work to keep the creative state from collapsing, especially as your work becomes successful and the social expectations mount. When I listen to interviews with creative people or read their workbooks, there are endless examples of them lamenting how hard it is. They keep coming up with techniques, rituals, and narratives to block off and protect the mental space they need.

This is evident in the workbooks that Ingmar Bergman kept from 1955 to 2001. Starting around the time he wrote The Seventh Seal, where a young Max von Sydow plays chess against Death, Bergman kept detailed notes of his thoughts, ending after he’d finished the script to his final film, Saraband. It is a very fluid and loose set of notes. There is no logic or structure. One second, Bergman will be writing about his frustrations with the work, and then without warning, the voice will subtly shift into something else—he’s drifting into a monologue. (Werner Herzog does the same in his diaries, making notes about his day and then abruptly veering off into narrative and feverish metaphors.) These fragments that unexpectedly ooze out of Bergman gradually coalesce into films.

Going sub-Bergman

Bergman’s notebooks are filled with admonitions he gives himself, for example here, on March 18, 1960: “(I will write as I feel and as my people want. Not what outer reality demands.)” Or here, on July 16, 1955: “I must not be intimidated. It’s better to do this than a lousy comedy. The money I give no fuck about.” Being highly impressionable and introverted, he is crafting a defiant personality in the notebooks, a protective gear that allows his larval ideas to live, even those who seem too banal (“a man learns that he is dying and discovers that life is beautiful,” which turns into Seventh Seal). 

Another introverted and impressionable writer is Karl Ove Knausgaard. In a perceptive essay about Bergman’s workbooks (an essay that is, I should point out, partly fabulated in a way that perhaps says more about how Knausgaard works than Bergman), Knausgaard makes a remark about the reminders Bergman writes himself (“I must not be intimidated” etc). These kinds of reminders are, Knausgaard claims, of little use because they “belong to thought and have no access to those cognitive spaces where the creative act takes place, but can only point to them.” To access these spaces, the thought “I will write as I feel and as my people want” is not enough. Rather, Knausgaard writes:

In order to create something, Bergman had to go sub-Bergman, to the place in the mind where no name exists, where nothing is as yet nailed down, where one thing can morph into another, where boundlessness prevails. The workbook is this place—in it, Bergman could put anything he wanted, the entries he made there could be completely inane, cringingly talentless, heartrendingly commonplace, intensely transgressive, jaw-droppingly dull, and this was in part their purpose: they had to be free of censorship, in particular self-censorship, which sought to lay down constraints on a process that needed to be wholly unconstrained.

There is a difference between knowing what you need to do (be independent and true to the potential in your ideas) and something else entirely to know how to embody that. Orienting in the right way to your thoughts is a skill. Like all skills, it takes practice. You also need to have a rich mental representation of how it is supposed to feel to embody the state so that you can orient toward that. This feeling is what you use to measure the relative success of whatever techniques you employ.

To slip more easily into the state, many develop strict habits around their work, rituals even. This is also what Bergman does.

The first few years, in the late 50s, the entries in his workbook are sparse. But as he pushes into the height of his creative career Bergman sets up a strict routine where he writes in the book for three hours every day, from 9 to 12 am, stopping mid-sentence at the strike of the clock. The book becomes the main technique he uses to induce the state where films and plays and books can be born. A non-judgemental zone. He writes that the workbook needs to be “so unpresumptuous and undemanding and is intended to sustain like the mellowest woman almost any number of my peculiarities.”

This is a fairly common practice, crafting a ritual where you sit down at the same time every day, in the same chair, writing in the same kind of notebook, creating a repetitiveness that borders on self-hypnosis. This is what Hemingway did, it is what Mario Vargas Llosa does. 

More techniques 

Here are some other techniques people use to access and maintain the zone:

The mental states where new ideas can be born are hard to open up. And they are continually collapsing. The things you have to do to keep them from caving in will make people frown upon you—your tendency for isolation, working deep in the night, breaking norms. The zone is a place at the margin of society. But strangely enough, this fragile margin is where the ideas that make our society possible come from. Almost everything that makes up our world first appeared in a solitary head—the innovations, the tools, the images, the stories, the prophecies, and religions—it did not come from the center, it came from those who ran from it.

20 comments

Comments sorted by top scores.

comment by MondSemmel · 2023-07-27T15:40:21.235Z · LW(p) · GW(p)

I haven't had time to read your post yet, but you might appreciate Elizabeth's post on Butterfly Ideas [LW · GW].

comment by Raemon · 2023-08-02T02:18:29.214Z · LW(p) · GW(p)

Curated. 

In some sense some of these ideas are "obvious" to me, but I nonetheless realized I hadn't really deeply integrated them into my models of rationalsphere progress. (In particular since a lot of my work is connected to "campus building", which is sort of pushing in the opposite direction of this post's model)

I feel a bit confused about to evaluate the validity of the model here (seems like the sort of thing where all the data about is intrinsically going to be pretty messy?), but appreciate it as a datapoint.

comment by Algon · 2023-07-27T13:11:25.316Z · LW(p) · GW(p)

This post feels like it is claiming that removing the need to conform to others' beliefs is important for creativity, and speculating how you might do so. Is that right? If it is, then I'm eager to hear more. (And yes, I understand the irony of saying that.) This is an important topic to me as I can feel my thoughts are bent out of shape by social pressures. 

Being able to sense these pressures is a good thing, in my view. I'm not sure why I can; perhaps my "social detection" circuitry is one of the few parts of my brain which aren't crippled? But the upshot is that I might learn how to notice when I can think freely, and then cultivate those mind-states. 

And though I didn't intend to use some of the techniques you've described for exactly this purpose,  I have used them. And variants, too. 

Knausgaard's technique: I've tried writing so fast I can't self censor, but only for a few paragraphs at most. At least, for writing down thoughts whilst focusing [? · GW]/tuning cognitive strategies [LW · GW]. Though in those cases, the censor isn't social. Instead, it is past trauma. When I'm babbling [? · GW]ideas and writing them as a series of bullet points, I might get to a page or so. On some occasions when I've had to write something before a deadline, I will explicitly use Knausgaard's technique to get past a block. But without focusing on what I don't like, it doesn't help. Really, all of the techniques I've tried in this space require Gendlin's focusing to fully work.

Introducing a long delay: This technique generalizes beyond averting social pressure, in my experience. If you're stuck on a problem, there's probably something wrong with your framing. Coming back to it after you've forgotten the framing lets you automatically see it with fresh eyes. In practice, this looks like storing a problem in an Anki deck and thinking about it once you see it again. It works suprisingly often. Which makes me wonder if there's some sort of optimal scheduling to revisit old problems. If I remember correctly, Anki reminds you of a card when you've begun to forget it, so maybe you just need to increase durations by a tad. 

And speaking of software solutions, one idea I had whilst reading your post was to automatically re-name all individuals in a text you're reading. This software would replace Alexander Grothendieck with ... Kübra Éliane throughout the essay. Likewise for the other names. The problem with this technique is if the individual's depicted in an awesome manner within the text, you may still associate their ideas with awe. 

A potential fix might be tuning your ontology to decouple ideas from the people who generated them. Unfortunately, BWT doesn't explain what that means. Here's someone else's description of the skill from a commen [LW · GW] on LW:

I didn't know them and can only speak to how I did the tuning ontology thing. For about 2 weeks, I noted any time I was chunking reasoning using concepts. Many of them familiar LW concepts, and lots of others from philosophy, econ, law, common sense sayings, and some of my own that I did or didn't have names for. This took a bit of practice but wasn't that hard to train a little 'noticer' for. After a while, the pace of new concepts being added to the list started to slow down a lot. This was when I had around 250 concepts. I then played around with the ontology of this list, chunking it different ways (temporal, provenance, natural seeming clusters of related concepts, domain of usefulness, etc.). After doing this for a bit it felt like I was able to get some compressions I didn't have before and overall my thinking felt cleaner than before. Separately, I also spent some time explicitly trying to compress concepts into as pithy as possible handles using visual metaphors and other creativity techniques to help. This also felt like it cleaned things up. Compression helps with memory because chunking is how we use working memory for anything more complicated than atomic bits of info. Augmenting memory also relied on tracking very closely whether or not a given representation (such as notes, drawing etc.) was actually making it easier to think or was just hitting some other easily goodharted metric, like making me feel more organized etc.

With regard to 'tracking reality with beliefs' the most important thing I ever noticed afaict is whether or not my beliefs 1. have fewer degrees of freedom than reality and thus have any explanatory power at all and avoid overfitting, 2. vary with reality in a way that is oriented towards causal models/intervention points that can easily be tested (vs abstraction towers).

comment by rime · 2023-08-03T22:33:25.418Z · LW(p) · GW(p)

There's a funny self-contradiction here.[1]

If you learn from this essay, you will then also see how silly it was that it had to be explained to you in this manner. The essay is littered with appeals to historical anecdotes, and invites you defer to the way they went about it because it's evident they had some success.

Bergman, Grothendieck, and Pascal all do this.

If the method itself doesn't make sense to you by the light of your own reasoning, it's not something you should be interested in taking seriously. And if the method makes sense to you on its own, you shouldn't care whether big people have or haven't tried it before.

But whatever words get you into the frame of attentively listening to your own mind, I'm glad those words exist, even if it's sad that trickery was the only way to get you there.

6.54.
   My propositions serve as elucidations in the following way: anyone who understands me eventually recognizes them as nonsensical, when he has used them—as steps—to climb beyond them. (He must, so to speak, throw away the ladder after he has climbed up it.)
  He must transcend these propositions, and then he will see the world aright.

  1. ^

    Or, more precisely, a Wittgensteinian ladder.

Replies from: rime
↑ comment by rime · 2024-07-06T18:50:04.971Z · LW(p) · GW(p)

I'm sad this comment was interpreted as "combative" (based on Elizabeth's reaction). It's probably a reasonable prediction/interpretation, but it's far from what I intended to communicate. I wanted my comment to be interpreted with some irony: it's sad that this post has to be written like this in order to get through to most readers, because most readers are not already at the point where they can benefit from its wisdom unless it's presented to them in this manner.

comment by MSRayne · 2023-07-29T02:52:18.259Z · LW(p) · GW(p)

Great, apparently I'm in just the right place... I'm always alone and have few friends who might influence me to give up my wacky ideas! Wonderful.....

crickets

comment by L Rudolf L (LRudL) · 2024-12-10T15:32:56.268Z · LW(p) · GW(p)

This post rings true to me because it points in the same direction as many other things I've read on how you cultivate ideas. I'd like more people to internalise this perspective, since I suspect that one of the bad trends in the developed world is that it keeps getting easier and easier to follow incentive gradients, get sucked into an existing memeplex that stops you from thinking your own thoughts, and minimise the risks you're exposed to. To fight back against this, ambitious people need to have in their heads some view of how uncomfortable chasing of vague ideas without immediate reward can be the best thing you can do, as a counter-narrative to the temptation of more legible opportunities.

In addition to Paul Graham's essay that this post quotes, some good companion pieces include Ruxandra Teslo on the scarcity and importance of intellectual courage (emphasising the courage requirement), this essay (emphasising motivation and persistence), and this essay from Dan Wang (emphasising the social pulls away from the more creative paths). 

comment by jmh · 2023-08-04T21:56:35.336Z · LW(p) · GW(p)

I did like this post and, as a rather uncreative and boring mind most of the time, I find one aspect a bit understated. This comment is largely aligned with the old adage about inventiveness being 1% inspiration and 99% perspiration.

I don't think it's the ideas that are removed but the willingness to put some work into exploring the idea when other's are perhaps not on board. I do think that fits well with the point about not bringing an idea out into the world right away. Suggesting to me that one does spend some time teasing out various aspect of the core idea before trying to get much feedback from others. But one could then share in a partial way -- for instance, when stuck on some specific aspect of the general working out of the idea -- in order to get over some block/hurdle without really getting into the is this good or bad. That avoids the external disincentive to pursue or even succeed.

I think it also points to who you will seek help/assistance from as your creative efforts proceed. Generally you're looking for support and help not merely critical feedback. In the group setting like the shared workspace bit you loose control of that I suspect. 

comment by Jacob Watts (Green_Swan) · 2023-08-08T23:01:21.637Z · LW(p) · GW(p)

Great post!

As much as a I like LessWrong for what it is, I think it's often guilty of a lot of the negative aspects of conformity and coworking that you point out here. Ie. killing good ideas in their cradle. Of course, there are trade-offs to this sort of thing and I certainly appreciate brass-tacks and hard-nosed reasoning sometimes. There is also a need for ingenuity, non-conformity, and genuine creativity (in all of its deeply anti-social glory).

Thank you for sharing this! It helped me feel LessWeird about the sorts of things I do in my own creative/explorative processes and it gave me some new techniques/mindset-things to try.

I suspect there is some kind of internal asymmetry between how we process praise and rejection, especially when it comes to vulnerable things like our identities and our ideas. Back when I used to watch more "content creators" I remember they would consistently gripe that they could read 100 positive comments and still feel most affected by the one or two negative ones.

Well, cheers to not letting our thinking be crushed by the status quo! Nor by critics, internal or otherwise!

comment by SilverFlame · 2023-08-04T16:26:55.939Z · LW(p) · GW(p)

Another idea if you want to push against the mental pressure that kills good ideas, from Paul Graham’s recent essay on how to do good work: “One way to do that is to ask what would be good ideas for someone else to explore. Then your subconscious won't shoot them down to protect you.” I don’t know of anyone using this technique, but it might work.

This angle of attack sounds worth investigating for myself, especially because it can circumvent censorship for other reasons, such as resource availability or personal interests. I've had ideas before that I immediately knew weren't something I'd be interested in pursuing myself, and it would be a waste to automatically throw them out without trying to think of someone more willing to take up the torch.

Replies from: SilverFlame
↑ comment by SilverFlame · 2023-12-08T23:06:43.776Z · LW(p) · GW(p)

Circling back a few months later, I have some observations from trying out this idea:

  • I found myself tossing ideas to friends and acquaintances more often, which tended to improve my relationships with them somewhat
  • I noticed that some of the ideas I was preparing to hand off to someone else had glimmers of concepts I could use for other things, which had obvious benefits
  • I didn't notice any impact to my normal ideation/processing bandwidth as a result of the change in operating method
  • Sometimes ideas I handed off to someone else would circle back later and benefit one of my own projects, although I suspect the success rates for such second-order results will vary wildly

Overall, it seems to have been worth trying, and I'll probably keep it going.

Replies from: henrik-karlsson
↑ comment by Henrik Karlsson (henrik-karlsson) · 2023-12-09T12:32:50.226Z · LW(p) · GW(p)

Thank you for this update!

comment by romeostevensit · 2023-07-31T18:23:21.544Z · LW(p) · GW(p)

See also John Cleese on creativity, subsequently turned into a short book when it went viral.

And The Net and the Butterfly by Olivia Fox Cabane

comment by nim · 2023-07-27T14:31:22.023Z · LW(p) · GW(p)

Might the pressure you're referring to be described as the value/pursuit of monotonic improvement?

Might the solutions you're suggesting be described as techniques for getting out of local maxima in a field?

The isolation that I hear you pointing at, I would describe as thinking two moves ahead in a game to seek optimal states, instead of immediately ruling out all possible sets of moves where the first makes things "worse" in some way.

comment by blake.crypto · 2024-04-08T03:32:06.966Z · LW(p) · GW(p)

Is there an article that will help me have LESS ideas?

comment by Review Bot · 2024-03-03T19:19:19.028Z · LW(p) · GW(p)

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

comment by corruptedCatapillar · 2023-08-17T19:40:25.437Z · LW(p) · GW(p)

I really liked this essay a lot, especially how it shows the importance and necessity of creativity in such a rigorous field like mathematics for being able to explore and potentially end up wrong. This sprung up many thoughts:

  • Advice I had heard from Jacob Lagerros, "If you're not embarrassed by what you ship, you ship too late."

    • Contra the first technique the author mentions, having the audacity to put out a bad product seems good for feeling the fear but continuing to work anyways, continuing to explore what's unknown to you.

  • The etymology of the words experiment, experience, and finally expert all are based on the root "-per", meaning "to try, to dare, to risk".

    • In "Lean Manufacturing" Eric says "I’ve come to believe that learning is the essential unit of progress for startups." This is in response to putting out features of a product, finding out they don't work but realizing what it is that they got wrong and being able to update accordingly and reorient in the direction that leads to a better product.

  • Tip: work so fast you don't have time to self-censor:

    • Writing and editing are two very different processes, and writing as if you're writing the final result will be will stunt you.
    • I often write without spellcheck on, otherwise it stops the flow.

  • Quote from another Paul Graham essay "How to Think for Yourself", "To be a successful scientist, for example, it's not enough just to be correct. Your ideas have to be both correct and novel."

    • In your post you mention "being exceptionally talented and trained was, in the long run, not enough to do groundbreaking work because they lacked the capacity to go beyond the context they had been raised in"
    • There's also reminiscing of "definition of sanity is doing the same thing over and over again, expecting different results"; if you don't do something different then you won't get different things.
    • The author mentions "But strangely enough, this fragile margin is where the ideas that make our society possible come from" and I think that doesn't get emphasized enough in most formal schooling I've been a part of; instead of asking what people can bring to the table it's mostly "only make time for what others have done".

  • Personal advice on doing projects: Instead of waiting for the perfect opportunity, find a good one, do good and optimize from there (if you're in the state of not being established and aren't sure what to do). (This advice may be wrong, not certain.)

  • When writing a more important email I can't draft the response in the email response, I have to write it out in my notes. I type out the shitty, low quality ideas and then am able to see most importantly 1. the full idea, and 2. exactly what it is that doesn't work. If it's not being observed though, I stay with writer's block.

  • A note on the EA / Rat culture that I've been a part of; I've seen people move into this community and if they say something that's possibly not true / even moderately disagreeable, people will quip back some sort of "That's not true:" and dominate the conversation for a while. The intention is to update, but I see people instead stop adding to the conversation out of fear of being wrong and chastised.

  • I like this twitter thread that aims in a similar direction of this post, and exploring one's own thoughts

  • Maybe a more palatable piece of advice comes from "Sailling True North" by James Stavridis mentions "Creativity and innovation can be paralyzed by fear of failure."

    • In this context he's speaking of war (where some of the advice in the post isn't applicable), but being able to not cave into pressure from outside forces but still recognize and allow those pieces of you that very well could be wrong to surface. This is quite delicate, but I'd argue is massively important.
comment by corruptedCatapillar · 2023-08-17T19:20:15.352Z · LW(p) · GW(p)

Paul Graham


Link for the curious http://www.paulgraham.com/greatwork.html

comment by Bastumannen (karl-roekaeus) · 2023-08-12T18:06:09.718Z · LW(p) · GW(p)

Thanks for this post, and in particular for including Grothendieck as one of the examples to illustrate your ideas. I have thought that most people outside of mathematics, and even many that are studying math, are not familiar with him. So I like how you assume that such a reader will accept this and just start to read, in the first section, about a guy that is unknown to said theoretical reader. I think that shows respect for your audience.

There is one thing not related to your points but more to the work of Grothendieck that I would like to mention; your statement "It is his capacity to surface questions that set Grothendieck apart, more so than his capacity to answer them." (which I now took out of context, but still): I don't agree completely with this. For example, to a person not familiar with his work it sounds like as if he basically only came up with new questions. But equally important is that he was also able envision and to implement a solution to many of them. His SGA-seminar that ran for 9 years at IHES laid the foundation for what became modern algebraic geometry, and being a new theory it required a huge amount of ground work carried out during this seminar, large parts of which was done by Grothendieck himself. He then published the more foundational parts of this research, including the theory of schemes, in EGA, which is written mainly by him, with assistants from Dieudonné. Without this to me absurd amount of work on Grothendiecks part there would likely be nothing. At the time there was a need to generalize the concept of a variety to something defined over first any field and then also over any (commutative) ring, in particular the integers. Grothendieck's generalization is what is called a scheme, and in modern terminology a variety is a special type of scheme. So this being in the air at the time, someone else would have come up with some version of this theory. But the version that Grothendieck envisioned was also the one that became standard, and I doubt that that would have happened if Grothendieck himself did not actually put in the crazy amount of work that it required to go from this beautiful idea to a (more or less) finished theory. 

Moreover, it is really fascinating now to be able to read an English translation of "Récoltes..." that you so kindly provided links to. (Back when I was into this subject I could not find any translation and so I had not read it before.) In there he talks about his motivation for developing algebraic geometry over the integers, namely the Weil conjectures (already proved for curves by André Weil himself); these conjectures relate geometric properties (topology) of the the complex points of a variety to arithmetic properties of the defining polynomial equations of the variety (the behavior of the integer solutions to these equations). A simpler to state example of another conjecture relating topology to arithmetic, proved later by Faltings using Grothendieck's framework, is that there can only be a finite number of rational points on an algebraic curve of genus > 1. Here the genus is an invariant of the curve that becomes visible when we view the curve over the complex numbers. So the simplest example is the projective line, which can be defined by the equation x^2 + y^2 = 1 together with a point at infinity; viewed over the complex numbers this looks like a sphere (another name for this object is the Riemann sphere); it has genus 0 since there are no "holes" in it. (I should say that it is a curve due to it's algebraic dimension being 1; and if you look at its points with real coordinates you will see a curve (a circle); now we look at the points where the coordinates are allowed to be complex numbers, and then it will look like a sphere, hence a surface, this can potentially be a little confusing.) Next, an elliptic curve looks like a torus when viewed over the complex numbers, and so has it has one hole, hence genus 1. And if we look at a given curve over the complex numbers and it turns out that it has even more holes, then its genus will be greater than 1. So the genus is a topological invariant of the curve that we can see if we view the curve over the complex numbers. Now, the equation above has infinitely many solutions that are rational numbers; the number of rational points on an elliptic curve can vary and there is a whole theory for describing their structure; and what Faltings proved is that if a curve has genus >1 it can only have a finite number of rational points. For example, the projective curve defined by x^4+y^4=1 has genus 3, so by Faltings theorem it can only have a finite number of rational points, or in other words, the equation can only have a finite number of solutions where both x and y are rational numbers. (But for this particular equation it was proved already by Fermat that it only have the trivial rational solutions, or stated differently, the equation x^4+y^4=z^4 has only the trivial integer solutions where at least one of the coordinates are 0. As I'm sure everybody knows, Fermat also claimed to have a proof that this was true for all exponents larger than 2, but that wasn't proved until the 1990:s by Andrew Wiles, again using the framework laid out by Grothendieck as a foundation.) In short: The Weil-conjectures indicated a link between the topology (form) of the points with complex coordinates on a variety, and the integer solutions of the equations that defines the variety (arithmetic). Grothendieck envisioned a method to make use of this link that would first require him to generalize the definition of a variety to something that could exist over the integers. That generalization is what is known as a scheme.

So before Grothendieck it was only possible to talk about varieties as something that is defined over an algebraically closed field like the complex numbers. That's why I said "arithmetic properties of the defining equations" earlier. But from developing the theory of schemes, which can be define even over the integers while at the same time has classical varieties as a special type of scheme, it becomes possible to replace "arithmetic properties of the defining equations" with "properties of the rational points on the variety", thereby opening up the field of Arithmetic geometry. 

So back to "Récoltes...", Grothendieck talks about how he investigated this new undiscovered land; in this land, the Weil-conjectures was the capital, always visible at the horizon. But Grothendieck was exploring the whole country, down to the smallest cottage out in some distant province. I have to interpret this as that he was proving theorem after theorem that was needed for the foundation of his new theory but not directly related to the Weil-conjectures. In the end, Grothendieck and collaborators was able to prove 3 of the 4 Weil-conjectures using this newly developed theory. (The first of them had also been proved earlier using other methods). Then in 1970 Grothendieck left his position at IHES, to my understanding because they received funds from the military. Some years later, his student Deligné managed to prove the last of the Weil-conjectures, using the methods of Grothendieck, but not in the way that Grothendieck had envisioned (he wanted to do it by proving the so called "Standard conjectures", from which he already had showed that the last part of the Weil-conjectures would follow). He also left research mathematics for many years, but came back briefly in the 1980's with "Esquisse d'un Programme", including his "Dessins d'enfants" that can be used to describe Riemann surfaces, more precisely exactly those Riemann surfaces that arises as the complex points on an algebraic curve defined over the algebraic numbers. (Above we had two examples of such curves, the projective line and the Fermat-curve of genus 3). So in this program he actually surfaced the questions, whereas in the development of algebraic geometry there was already a need for a new framework, although your point that he was able to surface the right questions are valid also there. Anyway, this last research program is active to this day, but Grothendieck left it after a couple of years, during which he worked on the program alone. So in this case he was no longer able to put in the extreme work required to go from a question and idea to finished results. And I think this was my point (although it took a little more words than expected to convey it); that what really made Grothendieck so outstanding was not only ability to surface questions and his seminal ideas and visions, but equally much his ability to implement those ideas. (And to do that a second time was too much to ask; I assume that you are already familiar with what happened to Grothendieck later in his life, but if someone happens to read this and are not, I encourage you to read more about his life; in a way this is as close to the stereotypical mad genius that you can get, and it feels like a sad story, although that is not for me to judge but more up to Grothendieck himself when he was still alive.) 

comment by Ben Doyle (ben-doyle) · 2023-08-03T22:22:57.280Z · LW(p) · GW(p)

I thoroughly enjoyed your article and believe you could delve deeper into various topics you touched on. You've raised some compelling ideas, such as achieving a flow state and the importance of solitude in certain stages of a process. However, you don't present these as rigid rules for success, leaving room for flexibility and exploration.

This reminds me of historical figures like Ramanujan. Despite lacking a formal institutional mathematics education, his determination and, possibly, his solitary work style enabled him to contribute significantly to mathematics by pushing boundaries and innovating.

Going to save this because I am sure to come back to it and read it a few more times, thank you!