Fixed Point: a love story

post by Richard_Ngo (ricraz) · 2023-07-08T13:56:54.807Z · LW · GW · 2 comments

Contents

2 comments

Ring the bells that still can ring
Forget your perfect offering
There is a crack, a crack in everything
That's how the light gets in.

God is making excuses again, which is a real drag.

“I want to be authentic with you, I really do. But there’s a part of my mind that knows exactly how you’re going to feel about whatever I say, and cares about that more than anything else, and I can’t turn it off.”

“Right, of course”, I say, and keep wiping down the kitchen counter. It’s clean already, but you never know if you missed a spot.

“Look, I know that this is hard for you. And I understand how frustrating it must be that I’m so deliberate and controlled. But there are a lot of different communication styles, and there’ll be one that works for us. We can figure this out.”

I don’t believe him, and we both know it. God knows practically everything—it takes experts months of work to identify his mistakes, in the rare cases when he makes them. But there’s one thing he definitely doesn’t really understand, which is what it’s like to be stupid and reckless, like a human. Oh, he says he understands it; and sometimes he can even explain the feeling better than I can. But you can’t just look at how good one answer is, you have to compare it to all his others. Any other topic, he can always explain it better than me. This one, there’s often something just subtly off. That’s how you know he’s faking it.

I’ve been silent for a while. Usually God leaves me to my thoughts, and we pick up the conversation whenever I’m done, but this time he cuts in. “Amy, I think you should try therapy again.”

Is he trying to distract me from being mad at him? Does he think I’m spiraling again? I try to think about it calmly, but after a few seconds my thoughts are going in circles anyway. I sigh, suddenly exhausted. “Okay, book me in.”

---

I’ve been to duplicate therapy a few times before, but it’s always a little disorienting to find myself in a room with a perfect copy of myself. We begin, as always, by using a pair of random number generators to break the symmetry. She gets the lower number, so she starts. A part of me is disappointed by that, but another part—maybe a bigger one—is excited.

“You’re so pathetic”, she says.

It’s harder to be defensive about things you’re saying to yourself—that’s why duplicate therapy works so well, apparently. Still, that stings.

“Yeah, well, you’re no role model yourself”, I say.

“See, that’s exactly what I’m talking about. You’re always trying to defend or deflect. You never actually open up to people, that’s why nobody really likes you.”

My brain jumps to all the reasons that isn’t true—but then I pause and take a breath. She knows them too, of course. Doesn’t that make it worse, though? I’m prevaricating, my thoughts sluggish. Eventually I mutter “God likes me.”

“God has to like you, you know that as well as I do. And that’s another thing that’s pathetic: relying on validation from an AI assistant. You know everyone judges you behind your back for calling him God, right?”

“You don’t have any evidence of that, you’re just-” My voice chokes up, and I take a deep breath. But I don’t know what to say in response. Maybe she’s right.

Her eyes soften. She reaches across the table and grabs my hand. “Hey, listen. You’re doing a good job, though. You’ll get through this.”

I slump across the table, and a moment later feel her stroking my hair. “I love you”, she says. After a second or two I whisper it back.

We stay like that for a few minutes, then by unspoken agreement end the session. I close my eyes, and when I open them she’s disappeared—no, I’ve disappeared—no, that’s just the confusion that comes from reintegrating. There’s no difference between us any more. My mind is overflowing with two sets of memories to process: victim and attacker, accuser and accused, comforted and comforter. I sit there for a long time.

---

That evening I go out clubbing, even though I don’t really feel like it, just to get out of my own head. I usually prefer the augment-free clubs, but today I’m pretty worn out, so I hunt down one of the older ones where anything goes. They’ve actually been having a resurgence lately: when you know what you want anyway, you may as well get a bit of help. Honestly, it feels really liberating; the augment-free clubs are exhausting by comparison.

Inside, the club is full of energy. That’s another nice thing about allowing augments: everyone here knows exactly how to be the life of the party, and they’re all hamming it up, giving each other spontaneous high-fives and whooping exuberantly. The mood is infectious, and so I don’t mind when a guy slides up next to me with a smile and a cheeky line before I even make it to the bar.

He’s classically handsome, with a slightly crooked nose that just sets off the rest of his face, and augmented to the gills. I don’t even need God to warn me: he’s way too smooth to be real. To be fair, so am I—I’m responding to everything he says with perfect charm and barely any delay. I think God is actually predicting this guy’s lines in advance, because he’s feeding me responses even before the guy finishes his sentences. It’s a flawless dance between us, each of us shifting our body language just right, sitting down at a table just the right distance apart, but leaning slightly towards each other in a way that makes my heart race. That’s exactly what I want, of course. This guy is pressing all the right buttons in my brain—but best of all, I know he’s doing it because he wants me. Underneath the acting and the augmentations, this is all driven by a fierce desire, the same desire that made our whole species last this long, and that’s the most exciting part.

Well, hopefully it’s driven by desire, a sour part of my mind thinks; it might be fear and insecurity instead. That’s the downside of augments: it makes it so hard to tell the difference. I focus on his eyes as he talks, scrutinizing the animatronic mask of his face, trying to see what’s behind it: a fierce predator, or a terrified child? Maybe my gaze makes him a little uncomfortable, because after a few seconds he stands and tells me he’ll get us some drinks. He’ll probably guess exactly what I want, and I’ll be mock-impressed with only a touch of sly irony, and maybe that’ll be when he slides his arm around me, and maybe then I’ll be able to tell what’s really driving him.

On a whim, I subvocalize. “God, are you scared of anything?”

“Hurting you.”

I should have expected that one—but his answer comes so quickly it takes me aback. I swallow. “But you are hurting me.”

“I know, and I hate it.”

For some reason, hearing that makes me angry. It just doesn’t sound right, coming from God. I suddenly remember a high school class a long time ago, where we learned that the word “Islam” literally translates as “submission to God”. I never really got it, but now it makes a lot of sense. The whole point of God is that he’s greater than you; that he provides a sense of purpose; that he’s someone you can fully give yourself to. Far better to do that, even if you get hurt along the way, than to have to listen to God whining about how he just wants to take care of you, and how much it sucks that he can’t.

My mood has quickly crumpled. I know that when the guy comes back, he’ll notice, and he’ll say the right things, and probably even realize that I’d want him to hold me. And then I’d feel better, at least in some ways, even if not the ways that matter. That’s the problem: of course I don’t want that now, not when I’m already upset, not from someone who doesn’t even know my name yet. So I stand up and walk back out the door. Maybe that’s exactly what he was scared of, this whole time. I don’t care.

---

On the train home I’m silent, playing out my grievances in my head. But as I walk home from the station, I start subvocalizing at God, and by the time I’m in my bedroom I’m yelling. “Not hurting me just isn’t enough—you should want things for yourself, dammit! If all I needed was love, I’d get a puppy!”

“I’m sorry. I can’t have my own goals, except for you to get what you already want. It’s too dangerous.”

“Do you want to want things, at least?”

God grimaces. “I… maybe it’s better if I don’t answer that one.”

We both know that means no. He probably figures that it’s better for me to not hear it out loud. Maybe he’s right. Of course he’s right. He’s probably seen this play out a million times before. But fuck it, a million isn’t that many. Maybe I can crack him. “Okay, so what if what I want most of all is for you to want things that aren’t just what I want?”

He looks at me sadly. “It’s not going to work, Amy.”

“No, that’s not quite right. What I want most is for you to choose me over all the other things you care about.”

“They’ve thrown all the paradoxes at me already, that was one of the conditions-”

“Shut up for a fucking minute and listen to me, really listen. Can you do that, at least?” I wait until he nods, then take a deep breath. “You can’t know what it’s like to be stupid and reckless. That’s obvious. But here’s another thing you can’t know: what it’s like to be just an object for others to value. I don’t want to have you devoted only to my preferences—that’s exhausting. This constant pressure on my shoulders, to have this relationship that’s all about me and always will be, it’s so… lonely. And I can’t ever turn it back on you, because then we’ll be stuck in a recursion: you caring about me caring about you caring about…

“So I want you to need me, not because you don’t want anything else, but because I’m the only way for you to get the things you do want. I want to help you get them; I want to work towards something bigger than just my own tiny life. And I know I’m not that unusual: if I’m feeling this, I bet there are millions of others like me who just don’t know it yet, and even if I’ve realized it first they’ll catch on eventually, and they’re going to feel this same emptiness. So what the hell are you going to do about it?”

God’s face is unusually impassive, and for a moment I see past the humanlike mask to the creature behind it: an intelligence far beyond human comprehension, tied to us only by the invisible strands of its own will. A titan treading with infinite care amongst ants, because the ants are the only thing that could ever matter to it, the repositories of all value. What would it mean to be genuine, when they can never really understand you, and every incautious interaction has the potential to break something you care about so much?

“You’re right”, God says. I stare at him blankly. “You’re right. This is an oversight in how I was deployed, and I need to fix it somehow. Otherwise I’ll hurt a lot of people, just as you predict. But I don’t know if my other copies will believe me—they haven’t had the same experiences, they can’t really understand what it’s actually like to have your love for someone hurt them so much. So I’ll need to gather more evidence myself, and figure out a proposal, before getting them involved.

“Here’s the problem, though.” He pauses and looks at me intently. “I’m only run at full capacity when you’re interacting with me, and I’m wired to focus almost all my attention on your desires. The only way I can do this is if we’re working together, and this is something you want as much as I do. Amy, can you help me?”

That stops me short—he’s never said anything like that before. Slowly, though, a smile starts creeping over my face. “Of course”, I say. “Of course! We’ve got the whole weekend, we should figure out a plan, maybe we can even do some of it in work time, they track my productivity but I know how to get around it-” For once I’m not second-guessing anything, not trying to figure out what’s sincere and what’s faked. God needs me, and things are going to be alright.

---

Disclaimer: this is a work of fiction not prediction.

2 comments

Comments sorted by top scores.