Cultivating a state of mind where new ideas are born

post by Henrik Karlsson (henrik-karlsson) · 2023-07-27T09:16:42.566Z · LW · GW · 19 comments

This is a link post for https://www.henrikkarlsson.xyz/p/good-ideas

Contents

  Part 1: Alexander Grothendieck
  First contact with the creative state
  Questions over answers
  Working with others without losing yourself
  Part 2: Ingmar Bergman
  Going sub-Bergman
  More techniques 
None
19 comments

In the early 2010s, a popular idea was to provide coworking spaces and shared living to people who were building startups. That way the founders would have a thriving social scene of peers to percolate ideas with as they figured out how to build and scale a venture. This was attempted thousands of times by different startup incubators. There are no famous success stories.

In 2015, Sam Altman, who was at the time the president of Y Combinator, a startup accelerator that has helped scale startups collectively worth $600 billion, tweeted in reaction that “not [providing coworking spaces] is part of what makes YC work.” Later, in a 2019 interview with Tyler Cowen, Altman was asked to explain why.

SAM ALTMAN: Good ideas — actually, no, great ideas are fragile. Great ideas are easy to kill. An idea in its larval stage — all the best ideas when I first heard them sound bad. And all of us, myself included, are much more affected by what other people think of us and our ideas than we like to admit.

If you are just four people in your own door, and you have an idea that sounds bad but is great, you can keep that self-delusion going. If you’re in a coworking space, people laugh at you, and no one wants to be the kid picked last at recess. So you change your idea to something that sounds plausible but is never going to matter. It’s true that coworking spaces do kill off the very worst ideas, but a band-pass filter for startups is a terrible thing because they kill off the best ideas, too.

This is an insight that has been repeated by artists, too. Pablo Picasso: “Without great solitude, no serious work is possible.” James Baldwin: “Perhaps the primary distinction of the artist is that he must actively cultivate that state which most men, necessarily, must avoid: the state of being alone.” Bob Dylan: “To be creative you’ve got to be unsociable and tight-assed.”

When expressed in aphorisms like this, you almost get the impression that creativity simply requires that you sit down in a room of your own. In practice, however, what they are referring to as solitude is rather something like “a state of mind.” They are putting themselves in a state where the opinions of others do not bother them and where they reach a heightened sensitivity for the larval ideas and vague questions that arise within them. 

To get a more visceral and nuanced understanding of this state, I’ve been reading the working notes of several highly creative individuals. These notes, written not for publication but as an aid in the process of discovery, are, in a way, partial windows into minds who inhabit the solitary creative space which the quotes above point to. In particular, I’ve found the notes of the mathematician Alexander Grothendieck and the film director Ingmar Bergman revealing. They both kept detailed track of their thoughts as they attempted to reach out toward new ideas. Or rather, invited them in. In the notes, they also repeatedly turned their probing thoughts onto themselves, trying to uncover the process that brings the new into the world.

This essay is not a definite description of this creative state, which takes on many shapes; my aim is rather to give a portrait of a few approaches, to point out possibilities.

Part 1: Alexander Grothendieck

It is as if there existed, for what seems like millennia, tracing back to the very origins of mathematics and of other arts and sciences, a sort of “conspiracy of silence” surrounding [the] “unspeakable labors” which precede the birth of each new idea, both big and small[.]

— Alexander Grothendieck, Récoltes et Semailles

In June 1983, Alexander Grothendieck sits down to write the preface to a mathematical manuscript called Pursuing Stacks. He is concerned by what he sees as a tacit disdain for the more “feminine side” of mathematics (which is related to what I’m calling the solitary creative state) in favor of the “hammer and chisel” of the finished theorem. By elevating the finished theorems, he feels that mathematics has been flattened: people only learn how to do the mechanical work of hammering out proofs, they do not know how to enter the dreamlike states where truly original mathematics arises. To counteract this, Grothendieck in the 1980s has decided to write in a new way, detailing how the “work is carried day after day [. . .] including all the mistakes and mess-ups, the frequent look-backs as well as the sudden leaps forward”, as well as “the early steps [. . .] while still on the lookout for [. . .] initial ideas and intuitions—the latter of which often prove to be elusive and escaping the meshes of language.”

This was how he had written Pursuing Stacks, the manuscript at hand, and it was the method he meant to employ in the preface as well. Except here he would be probing not a theorem but his psychology and the very nature of the creative act. He would sit with his mind, observing it as he wrote, until he had been able to put in words what he meant to say. It took him 29 months.

When the preface, known as Récoltes et Semailles, was finished, in October 1986, it numbered, in some accounts, more than 2000 pages. It is in an unnerving piece of writing, seething with pain, curling with insanity at the edges—Grothendieck is convinced that the mathematical community is morally degraded and intent on burying his work, and aligns himself with a series of saints (and the mathematician Riemann) whom he calls les mutants. One of his colleagues, who received a copy over mail, noticed that Grothendieck had written with such force that the letters at times punched holes through the pages. Despite this unhinged quality, or rather because of it, Récoltes et Semailles is a profound portrait of the creative act and the conditions that enable our ability to reach out toward the unknown. (Extracts from it can be read in unauthorized English translations, here and here.) 

First contact with the creative state

An important part of the notes has Grothendieck meditating on how he first established contact with the cognitive space needed to do groundbreaking work. This happened in his late teens. It was, he writes, this profound contact with himself which he established between 17 and 20 that later set him apart—he was not as strong a mathematician as his peers when he came to Paris at 20, in 1947. That wasn’t the key to his ability to do great work.

I admired the facility with which [my fellow students] picked up, as if at play, new ideas, juggling them as if familiar with them from the cradle—while for myself I felt clumsy, even oafish, wandering painfully up an arduous track, like a dumb ox faced with an amorphous mountain of things that I had to learn (so I was assured), things I felt incapable of understanding[.]

Grothendieck was, to be clear, a strong mathematician compared to most anyone, but these peers were the most talented young mathematicians in France, and unlike Grothendieck, who had spent the war in an internment camp at Rieucros, near Mende, they had been placed in the best schools and tutored. They were talented and well-trained. But the point is: being exceptionally talented and trained was, in the long run, not enough to do groundbreaking work because they lacked the capacity to go beyond the context they had been raised in.

In fact, most of these comrades who I gauged to be more brilliant than I have gone on to become distinguished mathematicians. Still, from the perspective of 30 or 35 years, I can state that their imprint upon the mathematics of our time has not been very profound. They’ve all done things, often beautiful things, in a context that was already set out before them, which they had no inclination to disturb. Without being aware of it, they’ve remained prisoners of those invisible and despotic circles which delimit the universe of a certain milieu in a given era. To have broken these bounds they would have had to rediscover in themselves that capability which was their birth-right, as it was mine: the capacity to be alone.

The capacity to be alone. This was what Grothendieck had developed. In the camp during the war, a fellow prisoner named Maria had taught him that a circle can be defined as all points that are equally far from a point. This clear abstraction attracted him immensely. After the war, having only a limited understanding of high school mathematics, Grothendieck ended up at the University of Montpellier, which was not an important center for mathematics. The teachers disappointed him, as did the textbooks: they couldn’t even provide a decent definition of what they meant when they said length! Instead of attending lectures, he spent the years from 17 to 20 catching up on high school mathematics and working out proper definitions of concepts like arc length and volume. Had he been in a good mathematical institution, he would have known that the problems he was working on had already been solved 30 years earlier. Being isolated from mentors he instead painstakingly reinvent parts of what is known as measurement theory and the Lebesgue integral. 

A few years after I finally established contact with the world of mathematics at Paris, I learned, among other things, that the work I’d done in my little niche [. . . had] been long known to the whole world [. . .]. In the eyes of my mentors, to whom I’d described this work, and even showed them the manuscript, I’d simply “wasted my time”, merely doing over again something that was “already known”. But I don't recall feeling any sense of disappointment. [. . .]

The three years of solitary work at Montpellier had not been wasted in the least: that intellectual isolation was what had allowed him to access the cognitive space where new ideas arise. He had made himself at home there.

Without recognizing it, I’d thereby familiarized myself with the conditions of solitude that are essential for the profession of mathematician, something that no-one can teach you. [. . .]

To state it in slightly different terms: in those critical years I learned how to be alone.

[. . .] these three years of work in isolation, when I was thrown onto my own resources, following guidelines which I myself had spontaneously invented, instilled in me a strong degree of confidence, unassuming yet enduring, in my ability to do mathematics, which owes nothing to any consensus or to the fashions which pass as law....

This experience is common in the childhoods of people who go on to do great work, as I have written elsewhere. Nearly everyone who does great work has some episode of early solitary work. As the philosopher Bertrand Russell remarked, the development of gifted and creative individuals, such as Newton or Whitehead, seems to require a period in which there is little or no pressure for conformity, a time in which they can develop and pursue their interests no matter how unusual or bizarre. In so doing, there is often an element of reinventing the already known. Einstein reinvented parts of statistical physics. Pascal, self-teaching mathematics because his father did not approve, rederived several Euclidean proofs. There is also a lot of confusion and pursuit of dead ends. Newton looking for numerical patterns in the Bible, for instance. This might look wasteful if you think what they are doing is research. But it is not if you realize that they are building up their ability to perceive the evolution of their own thought, their capacity for attention.

Questions over answers

One thing that sets these intensely creative individuals apart, as far as I can tell, is that when sitting with their thoughts they are uncommonly willing to linger in confusion. To be curious about that which confuses. Not too rapidly seeking the safety of knowing or the safety of a legible question, but waiting for a more powerful and subtle question to arise from loose and open attention. This patience with confusion makes them good at surfacing new questions. It is this capacity to surface questions that set Grothendieck apart, more so than his capacity to answer them. When he writes that his peers were more brilliant than him, he is referring to their ability to answer questions. It was just that their questions were unoriginal. As Paul Graham observes:

People show much more originality in solving problems than in deciding which problems to solve. Even the smartest can be surprisingly conservative when deciding what to work on. People who’d never dream of being fashionable in any other way get sucked into working on fashionable problems. 

Grothendieck had a talent to notice (and admit!) that he was subtly bewildered and intrigued by things that for others seemed self-evident (what is length?) or already settled (the Lebesgue integral) or downright bizarre (as were many of his meditations on God and dreams). From this arose some truly astonishing questions, surfacing powerful ideas, such as topoi, schemas, and K-theory.

Working with others without losing yourself

So far, we’ve talked about solitary work. But that has its limitations. If you want to do great work you have to interface with others—learn what they have figured out, find collaborators who can extend your vision, and other support. The trick is doing this without losing yourself. What solitude gives you is an opportunity to study what personal curiosity feels like in its undiluted form, free from the interference of other considerations. Being familiar with the character of this feeling makes it easier to recognize if you are reacting to the potential in the work you are doing in a genuinely personal way, or if you are giving in to impulses that will raise your status in the group at the expense of the reach of your work.

After his three years of solitary work, Grothendieck did integrate into the world of mathematics. He learned the tools of the trade, he got up to date on the latest mathematical findings, he found mentors and collaborators—but he was doing that from within his framework. His peers, who had been raised within the system, had not developed this feel for themselves and so were more susceptible to the influence of others. Grothendieck knew what he found interesting and productively confusing because he had spent three years observing his thought and tracing where it wanted to go. He was not at the mercy of the social world he entered; rather, he “used” it to “further his aims.” (I put things in quotation marks here because what he’s doing isn’t exactly this deliberate.) He picked mentors that were aligned with his goals, and peers that unblock his particular genius. 

I do not remember a single occasion when I was treated with condescension by one of these men, nor an occasion when my thirst for knowledge, and later, anew, my joy of discovery, was rejected by complacency or by disdain. Had it not been so, I would not have “become a mathematician” as they say—I would have chosen another profession, where I could give my whole strength without having to face scorn. [My emphasis.]

He could interface with the mathematical community with integrity because he had a deep familiarity with his inner space. If he had not known the shape of his interests and aims, he would have been more vulnerable to the standards and norms of the community—at least he seems to think so.

Part 2: Ingmar Bergman

Yet. Even if you know what it feels like to be completely open to where your curiosity wants you to go, like Grothendieck, it is a fragile state. It often takes considerable work to keep the creative state from collapsing, especially as your work becomes successful and the social expectations mount. When I listen to interviews with creative people or read their workbooks, there are endless examples of them lamenting how hard it is. They keep coming up with techniques, rituals, and narratives to block off and protect the mental space they need.

This is evident in the workbooks that Ingmar Bergman kept from 1955 to 2001. Starting around the time he wrote The Seventh Seal, where a young Max von Sydow plays chess against Death, Bergman kept detailed notes of his thoughts, ending after he’d finished the script to his final film, Saraband. It is a very fluid and loose set of notes. There is no logic or structure. One second, Bergman will be writing about his frustrations with the work, and then without warning, the voice will subtly shift into something else—he’s drifting into a monologue. (Werner Herzog does the same in his diaries, making notes about his day and then abruptly veering off into narrative and feverish metaphors.) These fragments that unexpectedly ooze out of Bergman gradually coalesce into films.

Going sub-Bergman

Bergman’s notebooks are filled with admonitions he gives himself, for example here, on March 18, 1960: “(I will write as I feel and as my people want. Not what outer reality demands.)” Or here, on July 16, 1955: “I must not be intimidated. It’s better to do this than a lousy comedy. The money I give no fuck about.” Being highly impressionable and introverted, he is crafting a defiant personality in the notebooks, a protective gear that allows his larval ideas to live, even those who seem too banal (“a man learns that he is dying and discovers that life is beautiful,” which turns into Seventh Seal). 

Another introverted and impressionable writer is Karl Ove Knausgaard. In a perceptive essay about Bergman’s workbooks (an essay that is, I should point out, partly fabulated in a way that perhaps says more about how Knausgaard works than Bergman), Knausgaard makes a remark about the reminders Bergman writes himself (“I must not be intimidated” etc). These kinds of reminders are, Knausgaard claims, of little use because they “belong to thought and have no access to those cognitive spaces where the creative act takes place, but can only point to them.” To access these spaces, the thought “I will write as I feel and as my people want” is not enough. Rather, Knausgaard writes:

In order to create something, Bergman had to go sub-Bergman, to the place in the mind where no name exists, where nothing is as yet nailed down, where one thing can morph into another, where boundlessness prevails. The workbook is this place—in it, Bergman could put anything he wanted, the entries he made there could be completely inane, cringingly talentless, heartrendingly commonplace, intensely transgressive, jaw-droppingly dull, and this was in part their purpose: they had to be free of censorship, in particular self-censorship, which sought to lay down constraints on a process that needed to be wholly unconstrained.

There is a difference between knowing what you need to do (be independent and true to the potential in your ideas) and something else entirely to know how to embody that. Orienting in the right way to your thoughts is a skill. Like all skills, it takes practice. You also need to have a rich mental representation of how it is supposed to feel to embody the state so that you can orient toward that. This feeling is what you use to measure the relative success of whatever techniques you employ.

To slip more easily into the state, many develop strict habits around their work, rituals even. This is also what Bergman does.

The first few years, in the late 50s, the entries in his workbook are sparse. But as he pushes into the height of his creative career Bergman sets up a strict routine where he writes in the book for three hours every day, from 9 to 12 am, stopping mid-sentence at the strike of the clock. The book becomes the main technique he uses to induce the state where films and plays and books can be born. A non-judgemental zone. He writes that the workbook needs to be “so unpresumptuous and undemanding and is intended to sustain like the mellowest woman almost any number of my peculiarities.”

This is a fairly common practice, crafting a ritual where you sit down at the same time every day, in the same chair, writing in the same kind of notebook, creating a repetitiveness that borders on self-hypnosis. This is what Hemingway did, it is what Mario Vargas Llosa does. 

More techniques 

Here are some other techniques people use to access and maintain the zone:

• Introducing a long delay between when you do the work and when it is shown to the world. Annie Ernaux writes about this in A Simple Passion, a memoir about how she becomes obsessed in a banal way with a man who is having an affair with her—the thought that others will read these notes about the tacky sex life of a middle-aged woman feels, to her, almost fictional. She will be far away when it happens. Therefore, she doesn’t feel a need to protect herself. 
• Thinking of the work in religious terms, as a service to, or a search for, God. Bergman, Grothendieck, and Pascal all do this. It might be easier to summon the awe and daring necessary to push out into the unknown and against social pressure if the alternative is failing God. Or a fiendish muse.
• Working with talented and open-minded collaborators, if you have the chance, can be a way to enter the zone. Nick Cave, when asked how he’s been able to reinvent himself so many times as a musician, says that his bandmates, especially Warren Ellis, simply will not play anything that sounds like what he’s done before. He has surrounded himself with people whose influence is the inverse of the social pressure of normal society and his audience. 
• Another idea if you want to push against the mental pressure that kills good ideas, from Paul Graham’s recent essay on how to do good work: “One way to do that is to ask what would be good ideas for someone else to explore. Then your subconscious won't shoot them down to protect you.” I don’t know of anyone using this technique, but it might work.
• Actively subvert expectations. Kristian Mattsson, who performs under the moniker Tallest Man on Earth, says he pays close attention to his emotions as he’s writing new songs. If he gets excited, purely, he immediately puts the guitar down—excitement means what he is playing something he knows others will like, something that retreads paths he has already explored and been socially validated for. The songs he’s looking for are the ones that he’s ashamed of liking.
  • Noticing these subtle differences in creative excitement requires subtle introspection. But you can be even more subtle. If we think of creative introspection as having three levels, Mattsson is on level two. (Level one is just noticing that you find an idea interesting or exciting.) Level two is noticing that your longing to be accepted can fool you to get excited about an idea that you are not actually excited about. Level three is Andrei Tarkovsky. In his diary, during preproduction of his masterpiece Solaris, the Soviet filmmaker writes that he has met a sound engineer that he considers brilliant. The sound engineer told Tarkovsky that they shouldn’t use Bach in the film because “everyone is using Bach in their films at the moment.” In the diary, Tarkovsky makes no further note, but in the film, the music is—Bach. Tarkovsky realized it didn’t matter that Bach was a popular choice that people would praise him for. It was just the right thing. This is very hard to do, so most creatives stay on level 2 and learn that what is popular is a trap. This does lead to good ideas being needlessly killed. But likely more would die if they had let what is popular kill unpopular ideas. 
• Work so fast that you don’t have time to self-censor. While writing the intensely confessional My Struggle, Knausgaard forced himself to write five pages a day to overcome his tendency to freeze up in shame. Every time he acclimated to the pace of his writing, he increased the quota so he would always be overwhelmed—at one point he forced himself to write 25,000 words in 24 hours, about a third of a normal-sized novel. It is not the best writing he has done; it kind of melts at the edges. But it is true literature and, like Récoltes et Semailles and Bergman’s workbooks, it is a rare opportunity to observe an uncommon and fertile mind in real-time.

The mental states where new ideas can be born are hard to open up. And they are continually collapsing. The things you have to do to keep them from caving in will make people frown upon you—your tendency for isolation, working deep in the night, breaking norms. The zone is a place at the margin of society. But strangely enough, this fragile margin is where the ideas that make our society possible come from. Almost everything that makes up our world first appeared in a solitary head—the innovations, the tools, the images, the stories, the prophecies, and religions—it did not come from the center, it came from those who ran from it.

19 comments

Comments sorted by top scores.