Investigating Fabrication

post by LoganStrohl (BrienneYudkowsky) · 2023-05-18T17:46:52.783Z · LW · GW · 14 comments

Contents

  1. Motivation
  2. Approach
  3. Lab Work, Part 1
    Finally, there is a feeling like a dungeon wall falling away to reveal a sunny meadow, which, if I watch closely enough, has some kind of stuttering dream-like quality to it. It’s as though I had to fall asleep, or trip through a magical door, or slip somehow, before I could experience it. But the dreaminess is quiet, and it’s largely drowned out by relief (if only briefly).
  4. Lab Work, Part 2
    slide
  5. Field Work, Part 1
    painting—or, perhaps i should say "fabrication", because this i think is the precise cognitive activity at the heart of "fabricated options"—has a characteristic *lightness*. there is no inertia. there is no resistance, no need for exertion or endurance. there is only dream-like ease. it's like gymnastics in super low gravity. it's like Minecraft in creative mode. it's like casting "accio solution" in the Harry Potter universe and watching things fly across the room right into your hands.
  6. Field Work, Part 2
  7. No Conclusion; Current Direction
    My main conclusion so far from my dalliance with fabricated options is that fabrication is a response to feeling trapped. That’s where it comes from, and what it’s made of. It’s a strategy for getting out of a too-tight possibility space, and its downfall is the way it moves only in imagination without staying in touch with the real world.
  Appendix
None
14 comments

Note 1: This was originally published in 2021. I'm now re-publishing it in 2023 as the penultimate post in The Nuts and Bolts Of Naturalism [? · GW], as a demonstration of the method. I intend to publish a collection of much-more-complete demonstrations in the future, and I plan for them to explicitly discuss each step as I've described it in the sequence. For now, though, I hope that this partial demo will give you a taste of one way the overall process can go.

(For a similarly partial demo by someone other than me, check out "Maps of Maps, and Empty Expectations [LW · GW]" by Nora Ammann.)

Note 2: This essay was written in response to “Lies, Damn Lies, and Fabricated Options [LW · GW]”. I expect it’ll be pretty confusing to read this one without having read that one first. However, I’m not the boss of you, and it isn’t obvious to me anyway which reading order is best overall. If prolonged confusion makes you grumpy, I recommend reading Duncan’s essay first. But if you try it backwards, I’d love to hear how that goes.


1. Motivation

When I read Duncan's essay "Lies, Damn Lies, and Fabricated Options [LW · GW]," it seemed to me to point toward something that's worth knowing for real, rather than just worth knowing about.

I think it does a good job of helping the reader build a new concept that could conceivably be taken as a hypothesis in important situations. The hypothesis is, “Perhaps this thing that seems to me like an option isn’t really a possibility, and it only seems like one because I’m in the grips of a map/territory conflation.” (Or, at least, “Perhaps the person I’m arguing with thinks their made-up option is real because [etc.].”)

But I wasn’t satisfied with my new concept “fabricated options”. There were some super important questions that I’d need to answer before I could put it to use.

For instance, 

So I set out to investigate option fabrication (the act), and what my brain is up to in the moments just before a fabricated option appears. I hoped that if I could learn where fabricated options come from, and what they are made of, I’d be a long way toward answering many of the practical questions on my list.


2. Approach

I did naturalism [? · GW] to this. (One of the reasons this essay felt worth writing and sharing is to give people more chances to see what naturalism looks like in practice.)

I assumed that there's sometimes A Thing Going On in my brain that in some ways resembles what Duncan called "fabricated options", and that my default understanding of the Thing is worse than I'd prefer. I set out to observe the Thing myself, directly and in real time, and to gradually build my own understanding of whatever I found. I tried to observe the Thing both in the "field" (as it occurs naturally in the course of daily life) and in the "lab" (as it occurs when I deliberately create controllable conditions that might give rise to it).

This was not a full naturalist study (which usually takes at least a month), but was instead a "let's see what comes of a week or two" dalliance. Still, it turned out to follow the arc of a mostly complete study: Sensitization, zooming in, zooming out, and experimentation. [ETA: Though I used different terms here, these correspond to Locating Fulcrum Experiences [? · GW], Getting Your Eyes On [? · GW], Collection [? · GW], and Experimentation [? · GW].]


3. Lab Work, Part 1

In accordance with the “try it immediately” heuristic, I started out with a relatively lab-like exercise right after reading the essay. I asked myself, "What is it like to fabricate options?" and in a search for relevant data, I began to scan my recent memories for "options" I might have "fabricated".

Multiple topics came to mind right away. One of them was “top surgery”, which I was scheduled to have in just a few days.

A thought occurred to me; it was one I've had before (it felt familiar as it arose), though it wasn't one I’d examined closely, or tried to put into words. I tried this time, though, with extra attention:

"Here's what I will do. I will have top surgery, but I’ll skip all those awful parts with pain and anesthesia drugs and so forth. I’ll just push a button, and my boobs will be gone. Simple.”

Although this was pretty clearly an instance of a “fabricated option”, I was at this point less interested in the thought itself, and more interested in what led me to turn my attention toward "top surgery" in the first place. What caused me to (correctly) expect to find The Thing there? I wanted to know in what directions I should open my introspective sense if I want to pick up more information about fabricated options. What properties does “top surgery” have that mark it as the natural habitat of the “fabricated options” creature I would like to study?

The “boob elimination button” thought was one I seemed to recreate frequently as I considered “top surgery”, so I had several opportunities to observe the conditions that gave rise to it. Here’s what always preceded it:

When I thought about top surgery, I felt good things, and I also felt bad things. As I imagined having a masculine chest and never having breasts again, I felt good. As I imagined going through surgery and recovering from it, I felt bad. And when my attention fell on the fact that I had scheduled top surgery, that it was in my future by default, I began to do and feel something I call “trying to squirm out from under the problem”.

“Trying to squirm out from under the problem” felt like this: 

When I thought about top surgery, it was like finding that something hard and heavy was on top of me. I did not like being under the hard heavy thing, and I didn’t know how to lift it. I was getting squished. It was uncomfortable under there, so I squirmed: I pushed up on this part of it with this hand, and that part of it with that foot, and I wriggled and writhed and tried to work myself out sideways. 

That’s what it felt like, in my body and my emotions.

The thoughts that happened while I was squirming went like this: 

Top surgery is scheduled? Ugh, I don’t like it. 

Maybe I could cancel top surgery? No, that doesn’t work, because then I’d keep having boobs. 

How about I go through with top surgery then? No, that’s no good either, because then I’d have to experience surgery and recovery. 

Well, let’s see… Maybe… Maybe I could cancel top surgery? 

*sigh* Still no, for the same reasons as last time.

I watched this pattern with multiple other topics, too, and it went almost exactly the same way with each of them, always with the same feelings of being trapped, squirming, and repeatedly hitting walls. The general form:

Well, [problem]. I don't like it.

I guess I could [option]? No, because [cost].

How about [second option]? No not that either.

Well maybe [the first thing again]? Still no.

I go through this cycle a few times, and as I do, it’s as though I’m banging my head, elbows, and knees against the walls of a much-too-small space. Inevitably, though, I at some point have a thought that goes,

What if [good parts of first option, and good parts of second option], but not [bad parts of either option]?

In the moments leading up to this new kind of thought, I usually feel frustration, restlessness, and desperation. There’s a bright flash of a moment that I want to label, “FINE!”, as shouted by someone stamping their foot or slamming a door. Then there is a “grasping” and a “stitching”, as I weave together “just the good things” and “without the bad ones”. 

Finally, there is a feeling like a dungeon wall falling away to reveal a sunny meadow, which, if I watch closely enough, has some kind of stuttering dream-like quality to it. It’s as though I had to fall asleep, or trip through a magical door, or slip somehow, before I could experience it. But the dreaminess is quiet, and it’s largely drowned out by relief (if only briefly).

Another thought I found by searching for a prolonged squirming feeling was, “I know! What I will do is, I will experience winter, with snow and Christmas and all of that, and instead of getting depressed like usual, this time I just won't get depressed!” 


4. Lab Work, Part 2

The other thing I did right after reading Duncan’s essay was order a gyroscope. Three days later, I played with it.

The main question I was investigating was, "What exactly is it like to imagine a gyroscope that's not weird?" 

This turned out to be a bit of a wrong question. Not wrong in the sense that I should have focused my attention on a different question instead; but wrong in the sense that I was employing some nonsensical concepts while asking it.

I tried very carefully to "imagine a gyroscope that's not weird". And what I found is that I'm not capable of imagining a gyroscope. Or an apple, or anything else. Not in the way I naively tend to think.

What I can do is conjure a blurry round-ish image and label it "spinning disk". I can conjure a crisp, bright tactile sensation of a sharp pressure on my finger and call it "where the gyroscope balances". I can conjure the sensations I'd expect to feel while tilting my wrist as my hand grasps a gyroscope, and I can imagine the pressure changing just as it would if I were holding a ball, or a phone, or a non-spinning gyroscope.

And if I'm not very slow and careful, it turns out, I can conclude from this imagining that some gyroscopes behave like solid objects even while their disks are spinning. Because hey, I've got a gyroscope in my imagination right now, and it's not being weird at all!

Only, I don’t have a gyroscope in my imagination.

It turns out that my brain is confused about imagination. 

It conjures visual images and textures and stuff, things that resemble "a gyroscope" or "an apple" or "the way my mother drives a car". It conjures them in order to manipulate them, to see what would happen under various circumstances. “What if I threw an apple at the wall?” I imagine doing so, and the thing I'm imagining goes thud-smoosh as it bounces off the other thing I’m imagining. This is most of how I predict stuff.

The problem is, my brain sort of thinks that when it manipulates a shiny red fruit-smelling object in imagination, what it's manipulating is an apple.

What it's really manipulating is part of itself, its own expectations and associations and so forth. When I use my own expectations to answer a question like, "What if I threw an apple at the wall?", of course the answer I get from imagination is whatever I already expected.

If my expectations are wrong, how will my imagination discover my error? It might let me discover my error by showing me that I have two conflicting expectations, which could give me space to ask whether one of them is wronger than the other (or something). Which is why it’s often worthwhile to deliberately investigate my expectations. This is (part of) what's going on when people change their mind when asked to bet, or when they do CFAR-style Murphyjitsu.

But imagination won't let me check a single expectation by itself, because it's taking that expectation as input. It's expectation working with expectation. My imagination is going to play along with itself. Even if I expect to notice some kind of contradiction, and cause myself to imagine that I've noticed the expectation being broken in some way, that brokenness isn't necessarily going to correspond to anything in the territory. I can tell my brain to see how a fantasy is a fantasy, and it's going to succeed, but that success will be another fantasy.

Reality, by contrast, resists individual mistaken expectations, and it does so lawfully.

Noticing the total lack of any feeling of resistance is what turned out to be so important about this session with the gyroscope. Whatever thing I was doing as I attempted to “imagine a gyroscope, but not weird” was full of ease, at least after I skipped over the question of what exactly the thing in my imagination was made of. The experience was full of effortlessness. It was full of “of course” and “simple” and “just”.

And the particular flavor of ease I found here was similar to the “open meadow” from before, the way I could slide into planning for “just the good things, without the bad”. It’s like shifting a car into neutral. There’s a tiny skip, and then all the resistance is gone.


5. Field Work, Part 1

By "field work," what I mean is that rather than engineering experiences specifically for the purpose of trying to investigate The Thing ("fabricated options," or whatever it is), I instead went about my day hoping to run into The Thing in a more natural, unplanned way. I hoped to encounter the phenomenon in its natural habitat, so to speak. 

Based on my early lab work, I made some guesses about what I might see or feel if my brain was just about to start doing The Thing, or had just done The Thing, or if I might be near The Thing in some other way. Whenever I ran into something that made me think I might benefit from paying extra attention, I snapped my fingers and took a moment to check what was going on in my thoughts and in my environment. Sometimes, I made a note in my phone to record my observations for later review.

I snapped my fingers many times. Each time, I updated a little bit about how I would know, next time, that this is among the experiences I’m watching for. I started with the squirming-out-from-under-a-problem feeling, but the longer I watched myself thinking and feeling in daily life, the more a sensation of discontinuity in imagination seemed like the bright shining landmark indicating that “the important thing about fabricated options lives here”.

“Discontinuity in imagination” wore a few different phenomenological guises.

One was “sliding”, like slipping down a wet hill over a patch of algae. I’d be imagining something, then I’d imagine the next part of the thing, and in between I did not so much “concretely imagine” as “slide” or “slip over”.

Sometimes it felt like “skipping” or “tripping”, catching my shoe on a root and falling blankly for a moment before resuming my gait as before.

Sometimes it felt like a hollowness, or an empty space. In fact, in most cases where I took the time to watch closely after snapping my fingers, I discovered an emptiness behind some kind of superficial imagining. Something like a facade.

I happen to have recorded my thoughts in a log entry as I observed an experience that involved both hollow-facade-ness and sliding. Here's an excerpt that goes into detail on that experience, with apologies for its length; this is optimized for me talking to myself more than for being nice to readers.

i can easily ask myself, "what's the fairy tale fantasy world thing you want here?" and it's an obvious question, a question that completely fits the feeling of this mental space, and the answer is also obvious: i want to stay up late relaxing and fucking around and acting like i have all the time in the world and not trying to do anything in particular at all, and i also want to wake up in the morning refreshed awake alive alert enthusiastic. what i want is for there to be an additional six hours tonight, just inserted between the usual hours of 22:00 and 23:00.

there's a click when i describe the extra hours. there's even a flow or a gravity toward filling in the details. the clock would say "22:59", then it would say "22:60", then "22:61", and it would just keep going like that until "22:419". 

the filling in of details responds somewhat to little concerns i have, hints that something is wrong: after imagining "22:419", i feel a tug, and when i follow the tug i think about "other people". i try out a couple of things. "maybe they're all asleep and they don't notice?" but no, many of them are still awake. i try again: "a time bubble. there's a bubble around just my room where six extra hours happen tonight." and as i think that, there's a relaxation, and then a part of me that's watching all of this is laughing as though at a puppy trying to carry a too-long stick through a narrow doorway.

it's very interesting, though, this relaxation after "time bubble". it's actually a space bubble, right? it's a bubble of space in which time is different, whatever that might mean. my calling it a time bubble hilights how not-even-a-concept this is. it's like an Old West movie set with just the fronts of the buildings and nothing behind them. i can't describe the architecture of the time bubble because it's not even a building.

yet the facade is convincing enough that some part of me reacts as though it's arrived in town and now it can drink the last of the water in its canteen because there's obviously a saloon right there on the corner. "it's ok, there's a time bubble. let's move on to the next question."

what's that like from the inside? it's like painting, and also like pong. a concern hurtles toward the fantasy of "six extra hours", and in response to it i throw some paint up on just the right part of the canvas to block it. there aren't any details yet, but that's ok, because i only need to block the concern on the surface, i only need to convince it to turn away. “what about other people? time bubble!” and if the concern doesn't immediately turn away, that's also all right, because i have no shortage of paint. the more time i spend thinking about it, the more detail i can imagine.

and perhaps the most interesting thing, here, is the way that this sort of imagining feels like discovery. answers like this come from the depths of the association network, from the source of lateral thinking and mnemonic skill. it's not slow deliberate reasoned thought whose pieces you can easily see. the feeling of discovery is accurate, in that i'm discovering how my mind is shaped.

but if i watch closely, there's a sliding. i feel the "discovery", and i sneakily, slippingly infer that i'm discovering the shape of the external world. i slip-slidingly infer that i'm learning about how time bubbles work.

discovering the external world feels different. knowing what feels different in discovering the external world vs discovering the shape of your own association network may be the way to recognize that you are painting in response to the pong balls, rather than noticing when the pong balls bounce off of pre-existing walls.

painting—or, perhaps i should say "fabrication", because this i think is the precise cognitive activity at the heart of "fabricated options"—has a characteristic *lightness*. there is no inertia. there is no resistance, no need for exertion or endurance. there is only dream-like ease. it's like gymnastics in super low gravity. it's like Minecraft in creative mode. it's like casting "accio solution" in the Harry Potter universe and watching things fly across the room right into your hands.


6. Field Work, Part 2

Here are some other times when I snapped my fingers.


How about I eat the chocolate chip cookie but not the calories?

This one’s classic. I tried to combine the good parts of two options while leaving out the bad. The fabrication revealed itself most memorably in the way it felt to answer my own probing questions: 

How will I avoid eating the calories? 

They’ll leak out first.

How will I cause this leaking?

I’ll break the cookie in half.

What do calories look like as they leak out?

Little golden musical notes.

This had the easy, automatic quality of facade-painting I’d identified before. “Accio solution!”


My toes should not touch each other. 

I caught a hint of “confused imagining” while wearing regular socks instead of toe socks. I usually wear toe socks because I don’t like my toes touching each other. 

It took me a while to figure out why “my toes touching” caused me to snap my fingers (I'm deliberately loose/generous with the finger-snapping rather than limiting myself to finger-snaps I already know I can justify), but I did eventually zero in on it. I was trying to force my toes not to stick together by imagining (incessantly) that although they touched, the skin experienced no friction. (“See, world? Here’s an existence proof, right here in my imagination. Update!”)

There was a blankness and a tripping-over feeling between “imagining toes touching” and “imagining no friction between them”. If you’d asked me, “How is there no friction?” I wouldn’t have been able to immediately respond with something like “the skin lacks texture”. I was skipping over that part. I would only have given you, “I don’t know how there’s no friction, there just isn’t!”

It’s not a central example of a “fabricated option”, but it’s the same mental patterns played out at a pretty low sensory level. I suspect that many instances of “frustrated should” involve something like this.


“I’ll just be fine.”

This was the first time that what I’ve so far learned from this study seemed to assert itself and cause a substantially different outcome from the default.

I was in San Francisco on my way to a doctor's appointment. There were a lot of sounds. The parking garage elevator said "Eeerrrrr! Eeerrrrr! Eeerrrrr!" in a terribly grating voice. The cars whooshed and honked. The crowds of strangers shouted to each other. It was all horribly chaotic and unfamiliar. I could feel myself closing off internally, contracting as though preparing to huddle down into a ball on the floor in some concrete corner.

But I had a mission. Not only did I want to make it to the appointment, but I wanted to talk with the doctor, to ask him questions and to understand what he said.

There was a moment of gathering forces. A moment of forcefully moving the outer shell of my mind into a strong protective shape. And with the gathering, I started to say to myself, as though declaring or proclaiming, "I'll just be fine." I began to decide that instead of being overwhelmed, I would "just" "be fine".

I say "began", because I didn't get very far.

I noticed something that indicated an instance of this thing I've been studying. It wasn't the word "just" that stood out, even though that could have tipped me off. 

Instead, it was the discontinuity. There was a feeling like tripping over uneven flagstones. It came from the way the outer shell of my mind had shifted, without more central parts having followed. It was a feeling that reminded me of "pulling one over on myself".

I paused, to pay attention to what was happening. 

I checked whether I really believed I could "just be fine", and the answer was "sort of". I knew that I definitely could do something with my mind that would allow me to walk through the city, and get all the way through my appointment, while barely even noticing my own distress, let alone exhibiting it outwardly. I could do something that would allow me to cross the street safely, to ask the questions that mattered, and to understand the responses.

The facade I'd begun to paint, though, left out a crucial component of this "deciding to be fine": the price. The neurotypical mask is heavy, and there is always a price for wearing it. I pay in exhaustion and future dysregulation.

I did actually decide to pay that price, in this case. But the decision I ended up making was "pay the price for 'being fine'". “Just be fine" was never a real option.


7. No Conclusion; Current Direction

The question I was left with, after the experience in San Francisco, was "Why did I try to deceive myself like this in the first place?"

If I hadn't caught myself in the middle of fabricating an option, I would still have paid the price for "being fine" in downtown San Francisco. I would even have paid a higher price than necessary, because I would have been relatively profligate with my masking, instead of having turned my conscious attention to the tradeoffs. So what was the point of fabrication?

I’m not sure yet, but I have a pretty confident guess.

I was already terribly overwhelmed when this happened, and I was desperate for a way out. Desperate enough that I was willing to hallucinate a temporary escape. The hallucination couldn't actually cause me to escape the situation, but it could let that one desperate time slice experience a piece of relief.

My main conclusion so far from my dalliance with fabricated options is that fabrication is a response to feeling trapped. That’s where it comes from, and what it’s made of. It’s a strategy for getting out of a too-tight possibility space, and its downfall is the way it moves only in imagination without staying in touch with the real world.

I suspect that if I'm going to get things right around "fabricated options" in the long run, I will need a handful of alternative strategies to directly address the sources of the distress. My brain engages in fabrication for a reason. If I want it to do something different, I’ll need to address that reason in some other way. The causal need isn't just [LW · GW] going to stop being a causal need.

I would bet that the most successful alternative strategies will be space-making.  Things like:

If I find myself making up fake options to escape the bind I seem to be in, I think it’s likely I’ve focused so much of my attention on that one particular bind that I’m no longer aware of the ways I’m not trapped. If I make myself more aware of the spaces through which I am free to move, I might find I’m less desperate to hallucinate an escape, and more comfortable with patiently investigating the true shape of the world.


Appendix [LW(p) · GW(p)]

14 comments

Comments sorted by top scores.

comment by Kaj_Sotala · 2023-05-18T19:41:16.525Z · LW(p) · GW(p)

Well this was an interesting experience to re-read (and I had apparently totally forgotten about having read this once before, since I was surprised by my comment from two years back in the comment section, after having missed the note in the beginning of this being republished)

Notes from while reading this:

I began to do and feel something I call “trying to squirm out from under the problem”.

Oh, this feels familiar. Emphasis on feels. There's a reaction I've sometimes noticed my mind doing, where it feels like I'm moving towards something unpleasant like a train on tracks and there's a Thing at the end of the tracks. And I don't want to move towards it, I don't want to see it, and I squirm and try to look anywhere else, but that just makes it worse. But no no no I do not want to see it. There's some sensation of someone having grabbed my other leg and pulling me inexorably towards it, and I'm trying to squirm away from their grip and desperately look elsewhere while I'm being pulled towards my doom, and the act of trying to look away sends waves of wrongness through me and feels horrible and I want to throw up.

That's the extreme version, there are also milder versions that are much less in intensity but have the same qualitative shape.

I wonder if that's the same thing as described here. Seems like it could be, at least.

“Discontinuity in imagination” wore a few different phenomenological guises.

Hmm I've certainly noticed discontinuities in thought at times, but I'm not sure if I've noticed the same kinds of ones as described here. None of these descriptions clearly resonate, but maybe there's something to it... I'll note it and move on.

How about I eat the chocolate chip cookie but not the calories?

Around this point in the post I've been thinking a bit about possible candidates, and...

I have some tennis balls in my room, because my therapist told me to get some and then do a daily exercise where I practice balancing while I have one tennis ball under my other foot. Today's the first day I'm supposed to do it, but I haven't been doing it because I've been feeling sleep-deprived and also been busy all day, so I've been putting it "until I'd have the time".

I had the thought of doing that exercise while reading this post, and there was a flash like... momentary discomfort and then relief. I didn't catch as many details as you, but I think the concept that flashed in my mind was something like "I can just not do it today" combined with a sense of there being no downsides to that. I don't think I'd have noticed that if this post hadn't primed me for it, but once my attention had been pointed to me, that sense of "no downsides" jumped out as conspicuous. As if it's not just "I didn't think of there being a downside to skipping an exercise my therapist had assigned to me", but rather "the downside had been actively left out", somehow. And that felt refreshing, like a flavor of green spring.

I paused to write this down before continuing to read the post, then the example of toes not touching each other reminded me about shoes. There's something off about the way I walk, that seems to break my shoes pretty quickly. As a result, I often need to buy new shoes. But often I keep putting it off and just walking in broken shoes for a while. I think I caught a glimpse of that "no downsides" thing there too, like I could just... avoid the hassle of buying new shoes, and then not suffer any of the discomfort of walking around in broken shoes anyway.

[one other potential match omitted for being a little embarrassing]

(got to the point of the "I'll just be fine" example)

I feel like my mind is matching some other potential examples against the pattern and there's something interesting happening. Something like looking for things that would have that sense of relief and lack of a downside, but instead of finding that, I find things that have discomfort place of that relief. A party I went to yesterday, only to remember there that I don't like these kinds of parties very much. (Even though I did mostly enjoy it.) It lacks that element of relief, but was there something suppressed with that too? Hmm.

Now the search got to the point of the way I think about AI risk and it seems like my different parts are on pretty different pages with regard to how seriously they take AI risk. Some poking at that. (I have no idea of whether my mind is actually doing something "real" or if I'm just placebo-ing myself into thinking that this kicked off some generalized search process, but it's interesting so I'll write it down anyway and see where it goes.) I feel a bit worried about the thought of really integrating all of my thoughts on AI risk, feels like that could be destabilizing, but we'll see.

Now the focus shifted to the way I don't always prepare in advance for things that I'm supposed to do, and kind of pretend I don't need to. The feeling is something like... I see different calendar entries I have, and it feels like each of them is kind of like a lid that's holding something unpleasant in place, and there's some unpleasant gas now leaking through those lids. That hadn't been properly detected before. Like the search algorithm locked onto the sense of unpleasantness and is now looking for places where some of that leaks through blocks.

(sometime later)

Now it kinda stopped and I don't notice anything more about it. Maybe I imagined the thing. I still have that mental image of my calendar entries as blocking something unpleasant that's under them, though.

I should do that exercise that my therapist assigned me to do, now.

Replies from: Kaj_Sotala
comment by Kaj_Sotala · 2023-05-18T20:20:30.584Z · LW(p) · GW(p)

(I did the exercise)

comment by LoganStrohl (BrienneYudkowsky) · 2021-11-12T06:07:16.225Z · LW(p) · GW(p)

Appendix

The answers to the questions below, and indeed even the questions themselves, are just my current best guesses. I’m a long way yet from having mastered fabrication. Still, they might at least be useful pointers.


Which situations are the crucial ones for promoting this hypothesis to attention? 

I should wonder whether I’ve fabricated an option if

  • I was sad and frustrated, then felt relief and a sudden opening of the space around me, as though the wall of a dungeon cell had fallen away to reveal a sunny fresh-air meadow.
  • My preferred alternative involves “just the good things” “without the bad things”.
  • The word “just” has come out of my mouth, or passed through my internal monologue.
  • Things seem really easy and obvious over here, and why does everybody else have to make things so difficult?
  • “If only…”, “Why can’t everyone…”, “Clearly we should…”


How do I actually move my mind to recognize that the option I’m considering might be fabricated?

If I have fabricated whatever option I’m considering, it’s likely that I have conflated “map” and “territory”. To take the fabrication hypothesis seriously, I’ll need to make a move that extricates me from that tangle. I could

  • say to myself, “There’s a way the world is. I think the world is one way, but it might be another.”
  • remember a time when things were not what they seemed. (My own reference experience is a stick that seems bent when sticking up out of water.)
  • name one plausible candidate for a crux [LW · GW], or at least notice what it’s like to try.


Once I’ve posed the hypothesis, how do I confirm or deny it?

Try to imagine the option playing out in detail. Go slow. Watch for a sensation of slipping, skipping, or blankness. Try to add additional detail in those places, and notice what it feels like to do so. 

If there’s a lot of blankness, and if you seem to shift into an easy neutral gear (or to enter creative mode, or to cast “accio solution”) while you fill in the details, this is evidence of fabrication.

If there’s a sustained feeling of grounded continuity, and you find yourself wanting to Google things or otherwise check the outside world when you try to fill in details, this is evidence against fabrication.


When I’ve successfully identified a fabricated option, how can I prevent the usual harm?

Notice that you feel constrained, and create space where you can. I suspect this alone will dramatically mitigate the damage. Fabrication comes from a kind of desperation, and people tend to be rash when they’re desperate.

The Litany of Gendlin [? · GW] may also help, if that’s something that works for you. Fabrication is a way of trying to force reality to be a certain way just by imagining hard enough. Remind yourself that reality is mind-independent. 

Look for a crux. I predict that the struggle of doing this with a fabricated option will get your wheels back in contact with the pavement.


How can I stop making this kind of mistake in the first place?

First, snap your fingers when you think you may have fabricated an option some time recently. Next, snap your fingers when think you may have fabricated an option just moments ago. Then, snap your fingers when you suspect you’re in the middle of fabricated an option. Finally, snap your fingers when you think you may be about to fabricate an option.

comment by [DEACTIVATED] Duncan Sabien (Duncan_Sabien) · 2021-11-12T20:58:34.488Z · LW(p) · GW(p)

I note as a not-very-object-level response that reading this report has caused me to ditch a draft of an upcoming post, and replace it with a much more bottom-up, from-the-inside-out version.  I'm doing this because it seems like it'll make that essay stronger, and would not have got there without being exposed to this.

comment by Kaj_Sotala · 2021-11-12T11:49:37.009Z · LW(p) · GW(p)

Top surgery is scheduled? Ugh, I don’t like it. 

Maybe I could cancel top surgery? No, that doesn’t work, because then I’d keep having boobs. 

How about I go through with top surgery then? No, that’s no good either, because then I’d have to experience surgery and recovery. 

Well, let’s see… Maybe… Maybe I could cancel top surgery? 

*sigh* Still no, for the same reasons as last time. [...]

What if [good parts of first option, and good parts of second option], but not [bad parts of either option]?

This seems quite similar to my model of how craving (in the Buddhist sense) works and how it causes suffering, in that craving looks like it creates constraints for what features reality should contain, and the mind then tries to use its existing knowledge in order to come up with a plan of action that fulfills all the constraints - even if that turns out to be impossible. I have a more detailed writeup here [LW · GW], which includes a similar "thrashing around" example.

In a sense, "what if [good parts of first option, and good parts of second option], but not [bad parts of either option]" is a good question to ask, since it seems similar in spirit to what one is trying to do when goal factoring [? · GW]: start from the assumption that such an option does exist, and then carry out a search for achieving it. Of course, the "assume that such an option does exist" is a bad idea in cases where no such option does exist.

Still, in some situations it can make sense to just assign very high prior probability on the option existing, even if all the evidence would seem to go against this possibility. The most obvious scenario is one where all the non-fabricated options involve dying. If giving up means death, then in the worst case pursuing a policy of "I am going to assume there's a way out and keep looking for it until I find one" just means that you'll die (which would have been the case anyway), but in the best case it lets you find that one very-low probability chance of making it out alive. 

In Shut up and do the impossible! [LW · GW], Eliezer also argues for pursuing things like AI alignment even when it seems impossible.

That logic would seem to help explain why brains sometimes put a probability on an option existing that is proportional to how important it is for us to achieve that goal (or put in more Buddhist terms, proportional to how much craving there is). If you crave something enough, the strength of that craving will make you believe that the option for getting the thing exists, so as to motivate you to keep looking at it. But that also means that after your brain has fabricated the option, you might stop looking. Once you assign a high enough prior probability to having the option, you feel like you already have it - and if someone points out that you don't, that may feel like them taking it away from you, causing a need to lash back at them.

From my earlier writeup:

Recall that under the [predictive processing] framework, goals happen because a part of the brain assumes that they will happen, after which it changes reality to make that belief true. So focusing really hard on a craving for X makes it feel like X will become true, because the craving is literally rewriting an aspect of my subjective reality to make me think that X will become true.

When I focus hard on the craving, I am temporarily guiding my attention away from the parts of my mind which are pointing out the obstacles in the way of X coming true. That is, those parts have less of a chance to incorporate their constraints into the plan that my brain is trying to develop. This momentarily reduces the motion away from this plan, making it seem more plausible that the desired outcome will in fact become real.

Conversely, letting go of this craving, may feel like it is literally making the undesired outcome more real, rather than like I am coming more to terms with reality. This is most obvious in cases where one has a craving for an outcome that is impossible for certain, such as in the case of grieving about a friend’s death. Even after it is certain that someone is dead, there may still be persistent thoughts of if only I had done X, with an implicit additional flavor of if I just want to have done X really hard, things will change, and I can’t stop focusing on this possibility because my friend needs to be alive.

In this form, craving may lead to all kinds of rationalization [? · GW] and biased reasoning: a part of your mind is literally making you believe that X is true, because it wants you to find a strategy where X is true. This hallucinated belief may constrain all of your plans and models about the world in the same sense as getting direct sensory evidence about X being true would constrain your brain’s models. For example, if I have a very strong urge to believe that someone is interested in me, then this may cause me to interpret any of his words and expressions in a way compatible with this belief, regardless of how implausible and far-spread [? · GW] of a distortion this requires.

comment by Jasnah Kholin (Jasnah_Kholin) · 2023-05-09T13:11:32.293Z · LW(p) · GW(p)

this is extremely good post. it example and illustrate the sort of mental-moves i believe is needed for rational thinking, of the variety of "know thyself". those things are even harder then normal to communicate, and i find this post manage to do that, and manage to give me useful information, and give me example of how such introspection can happen. i really impressed!

Replies from: CuriousMeta, BrienneYudkowsky
comment by CuriousMeta · 2023-05-25T14:06:23.430Z · LW(p) · GW(p)

Seconded (after working with this concept-handle for a day). This here seems to be the exact key for (dis)solving the way my brain executes self-deception (clinging, attachment, addiction,).

(I'm noticing that in writing this, my brain is fabricating an option that has all the self-work results I envision, without any work required)

comment by LoganStrohl (BrienneYudkowsky) · 2023-05-09T17:56:44.174Z · LW(p) · GW(p)

Related: Intro to Naturalism [? · GW] and especially The Nuts and Bolts of Naturalism (still coming out, a few posts not yet published).

Replies from: Jasnah_Kholin
comment by Jasnah Kholin (Jasnah_Kholin) · 2023-05-09T19:34:56.490Z · LW(p) · GW(p)

I already have them in my reading list, but after that post i plan to epub them and read them soon. 

comment by localdeity · 2021-11-12T07:14:53.563Z · LW(p) · GW(p)

There were a lot of sounds. The parking garage elevator said "Eeerrrrr! Eeerrrrr! Eeerrrrr!" in a terribly grating voice. The cars whooshed and honked. The crowds of strangers shouted to each other. It was all horribly chaotic and unfamiliar. I could feel myself closing off internally, contracting as though preparing to huddle down into a ball on the floor in some concrete corner.

I'm sure you've thought of this already, but nevertheless, my brain suggests using earplugs.  I keep a supply in my backpack, and use them at louder parties and mass transit; the brand I use is flesh-colored, so people are unlikely to notice them (although I don't think I'd care if they did).  I can generally hear speech well enough through them to have a conversation, although my hearing may be unusually good; if that weren't the case, then I think I'd try reducing their strength by (a) putting them in only partway, (b) cutting them or poking holes in them, or (c) finding a low-strength brand.

I suspect that having them in your pocket, as an option, provides reassurance even when you don't end up using them.

Replies from: Duncan_Sabien
comment by [DEACTIVATED] Duncan Sabien (Duncan_Sabien) · 2021-11-12T07:30:15.751Z · LW(p) · GW(p)

They've actually got them on a tiny little holster on their belt.

Replies from: BrienneYudkowsky
comment by LoganStrohl (BrienneYudkowsky) · 2021-11-12T18:04:41.346Z · LW(p) · GW(p)

I was wearing them at the time in fact.

comment by CuriousMeta · 2023-05-24T13:13:24.344Z · LW(p) · GW(p)

I find that [letting go of the (im)possible worlds where I'm not trapped] helps reframe/dissolve the feeling of trappedness. 

However, that kind of letting go often feels like paying a large price. E.g. in case of sensory overload it can feel like giving up on having any sense of control over reality/sensory-input whatsoever.

comment by Οἰφαισλής Τύραννος (o-faislis-tyrannos) · 2023-07-05T09:58:24.618Z · LW(p) · GW(p)

As someone born as male who registered as female this year, I believe that one of the ways in which it is possible to negate reality in the most radical manner is to act as if you can truly change your sex. 

The map indicates that I'm a woman, but the territory won't be changed by that. I'm a man, no matter what all the documents say. 

It would be unhealthy to pretend otherwise.