The Intelligent Social Web

post by Valentine · 2018-02-22T18:55:36.414Z · LW · GW · 112 comments

Contents

114 comments

Epistemic status: Fake Framework


When you walk into an improv scene, you usually have no idea what role you’re playing. All you have is some initial prompt — something like:

“You three are in a garden. The scene has to involve a stuffed bear somehow. Go!”

So now you’re looking to the other people there. Then someone jumps forward and adds to the scene: “Oh, there it is! I’m glad we finally found it!” Now you know a little bit about your character, and about the character of the person who spoke, but not enough to fully define anyone’s role.

You can then expand the scene by adding something: “It’s about time! We’re almost late now.” Now you’ve specified more about what’s going on, who you are, and who the other players are. But it’s still the case that none of you knows what’s going on.

In fact, if you think you know, you’ll often quickly be proven wrong. Maybe you imagine in that scene you’re an uptight punctual person. And then the third person in the scene says to you, “What do you care, Alex? You’re always late to everything anyway!” Surprise! Now you need to flush who you thought you were from your mind, accept the new frame, and run with it as part of your newly evolving identity. Otherwise the scene sort of crashes.

It would go more smoothly if you didn’t hold any preconceptions about who you are or what’s going on. The scene tends to work better if you stay in the present moment and just jump in with the first thing that comes to mind (as long as it’s shaped by what has happened so far). Then the collection of interactions and emerging roles spontaneously guides your behavior, which in turn help guide others’ behavior, all of which recursively defines the “who” and “what” of the scene. Your job as a player isn’t to play a character; it’s to co-create a scene.

We can sort of pretend that there’s a “director”: it’s the intelligence that emerges between the players via their interactions. It’s a distributed system that computes relationships and context by guiding each node in its network to act freely within constraints. From this vantage point, the network guides players, and the job of each player is to be guidable but not purely passive (since a passive node is just relaying information rather than aiding in the computation). As long as everyone involved is plugged into and responsive to this network, the scene will usually play out well.


I suspect that improv works because we’re doing something a lot like it pretty much all the time. The web of social relationships we’re embedded in helps define our roles as it forms and includes us. And that same web, as the distributed “director” of the “scene”, guides us in what we do.

A lot of (but not all) people get a strong hit of this when they go back to visit their family. If you move away and then make new friends and sort of become a new person (!), you might at first think this is just who you are now. But then you visit your parents… and suddenly you feel and act a lot like you did before you moved away. You might even try to hold onto this “new you” with them… and they might respond to what they see as strange behavior by trying to nudge you into acting “normal”: ignoring surprising things you say, changing the topic to something familiar, starting an old fight, etc.

In most cases, I don’t think this is malice. It’s just that they need the scene to work. They don’t know how to interact with this “new you”, so they tug on their connection with you to pull you back into a role they recognize. If that fails, then they have to redefine who they are in relation to you — which often (but not always) happens eventually.

I’m basically taking as an axiom of this framework that people need the “scene” to work — which is to say, they need to be able to play out their roles in relation to others’ roles within a coherent context. I don’t think why this is the case is relevant for using this framework… but I’ll wave my hands at a vague just-so story anyway for the sake of pumping intuition: human beings’ main survival strategy seems to be based on coordinating in often complex ways in tribes. For the individual, this means that fitting in becomes paramount. For the group, this means knowing what to expect from each person is critical. So a trade becomes possible: the individual can fit into and benefit from the group as long as they’re playing a role that fits well with the collective.

This can result in some pretty strange roles. From this vantage point, a person who repeatedly leaves one abusive relationship only to get into another roughly similar one actually makes a lot of sense: this is a role that this person knows how to play. It’s horrible, but it’s still better than not fitting into the social scene. It creates a coherent relationship with someone who’s willing to (or has to) play an “abuser” role, and often with people in “rescuer” roles too. The trap they’re in isn’t (just) that their current abusive partner is gaslighting or threatening them; it’s that they don’t have another role they can see how to play. Unless and until that person finds a different one that fits into the social web, the strands of that web will tug them back into their old role. They don’t have enough slack in the web around them to change their fate.

The same kind of web/slack dynamics show up in more pleasant-to-play roles too. The privilege of a middle-class American white man by default has him playing out some kind of roughly known story-like path (probably involving college and having kids and maybe a divorce) that, in the end, will probably still leave him being one of the richest people on Earth. And all the while, he might well have no clue that he has other options or even that he’s on a path — but he’ll still know, somehow, not to step off that path (“I have to go to college; are you crazy?”). Never mind that his lack of slack here is awfully convenient for him.

I’ve watched religious conversions and deconversions happen via basically the same mechanism. I knew a fellow many years ago (unattached to this community) who was a proud atheist. Then he started dating a Christian girl. Something like a month later, he started quoting the Bible — but “only because they’re handy metaphors” and not because he really believed any of that stuff, you see. It later turned out he’d been going to church with her. He kept offering reasons that seemed vaguely plausible (“It’s a neat group of people, and it matters to her, and I can take the time to read”), but there’s a pattern here that was obvious. A few months later he told me he’d converted. Last I heard they had moved to Utah.

The great part is, I knew this was going to happen when they started dating. Why? Because when I warned him that he might find himself wanting to believe her religion once they started having sex, his reaction was to reassure me by acting confident that he was immune to this. That meant he was more focused on managing my perception of him than he was in noticing how the social web was tugging him toward a transition of roles. I didn’t know if they’d stay together, but I was pretty sure that if they did, he’d convert.

I could give literally hundreds of examples like this. From where I’m standing, it looks like one of the great challenges of rationality is that people change their minds about meaningful things mostly only when the web tugs them into a new role. Actually thinking in a way that for real changes your mind in ways that defy your web-given role is socially deviant, and therefore personally dangerous, and therefore something you’re motivated not to learn how to do.

Ah, but if we’re immersed in a culture where status and belonging are tied to changing our minds, and we can signal that we’re open to updating our beliefs, then we’re good… as long as we know Goodhart’s Demon isn’t lurking in the shadows of our minds here. But surely it’s okay, right? After all, we’re smart and we know Bayesian math, and we care about truth! What could possibly go wrong?

Another challenge here is that the part of us that feels like it’s thinking and talking is (usually) analogous to a character in an improv scene. The players know they’re in a scene, but the characters they’re playing don’t. The characters also aren’t surprised about who or what they are: the not-knowing of identity and context is something only the players experience, to open themselves up to the guidance of the distributed “director”. This means that (a) the characters are actively wrong about why they do what they do, (b) they are deeply confused about how much sense everything makes, and (c) they don’t know they’re confused.

I claim that most of us, most of the time, are playing out characters as defined by the surrounding web — and we usually haven’t a clue how to Look at this fact, much less intentionally use our web slack to change our stories.

I think this is also part of why improv is challenging: you have to set aside the character you would normally play in order to create room for something new.


The web as a whole wants to know what kind of role you’re playing, and how well you’re going to play it, so that it can know what to expect of you. So, a lot of its distributed resources go into computing a model of you.

One of the more obvious transmission methods is chat — idle gossip, storytelling, speculation, small talk. People sync up their impressions of someone they’ve met, and try to make sense of surprising events in conversation. If a lover brings their partner some flowers and the recipient freaks out and runs off, suddenly there’s a need to understand, and the flower-giver might try asking a mutual friend for some help understanding. And even if they do come to understand (“Oh, that’s because their last partner brought them flowers to break up with them”), there’s often an impulse to share the story with friends, so that the web as a whole can hold everyone in sensible roles and make the scene work. (“Oh, we had a funny misunderstanding earlier, poor Sam….”)

A lot of this is transmitted more subtly too, in body language and facial expressions and vocal tone and so on. If Bob is “creepy” (i.e., is playing a “creepy” role in the web), then it speaks volumes if everyone who meets Bob then cringes just a tiny bit when he’s later mentioned even if they say only good things about him. This means that someone who has never met Bob can get a “vibe” about him from multiple people in a way that shapes how they interpret what Bob says and does when they finally do meet him.

Sometimes, some people with enough web-savvy weaponize this. It doesn’t mean anything for someone to “be creepy” except that they have a web-like impact on others — which is to say, they have a “creepy” role. In a healthy network, this correlates with something actually meaningfully bad that’s worth tracking. But because perceived roles shape what people expect of a person, it’s enough for a rumor to echo through the web in order for someone to be interpreted as “creepy”. So a sufficiently cunning person could actually cause someone to be slowly isolated and distrusted without there being any facts at all to justify this as the social web's stance.

(And yes, I’ve seen this happen. Many times.)

The same kind of thing can happen with “positive” labels, too. What it means for someone to be fit for a leadership role, in the social web's eyes, is that they are seen as compatible with that role. So if someone is tall, attractive, and either vicious or strong depending on how you choose to see it, it might be enough to have the “strong” interpretation echo more powerfully than the “vicious” one in order for the web to conspire to put them in a leadership position.

…which means that even people who are seen as good leaders might not, in fact, be good leaders in the sense of making good leadership decisions. But they are by definition good leaders in the sense of playing the role well. After all, if the general consensus is that Abraham Lincoln was a great President, then there’s a sense in which that makes it true, since that’s what “great” means here. The “explanations” thereafter are often stories to justify one’s holding of a popular opinion.

The same thing holds for when someone seems “rational”. This is one reason to worry deeply when members of subgroups internally agree with each other on who is a top-notch clear thinker or “really a rationalist” but disagree with people in other subgroups. This looks less to me like people seeking truth, and a lot more like groups engaging in a subtle memetic battle over what “rational” gets to mean.

From where I’m standing, it looks to me like we’re all immersed in not-knowing, while our “characters” keep talking as though they know what’s going on, implicitly following some hidden-to-them script.


The web encodes a lot of its guidance about what we should expect and how to behave via the structure of stories. Or rather, story structures are what expectations about roles and scenes are.

The trouble is, a lot of the stories we talk about have the structure of what our characters are supposed to say rather than of what actually happens. Imagine a movie where the new kid at a school gets bullied by the popular kids and then makes friends with quirky outcasts. What happens to the bullies in the end? In real life, bullies often don’t get their comeuppance — but having this fictional story in our hearts lets us play out vivid indignation through our characters in the real-life version. Because the bullies aren’t supposed to get away with it, right? That wouldn’t be fair!

Some parts of our story-like intuitions are scripts for what should actually happen. Some are things our scripts say we should think or feel or talk about within our given roles. Some are merely incidental details. Sussing out which parts are which is part of the trick of getting this framework to work for you.

For instance, the stereotypical story of the worried nagging wife confronting the emotionally distant husband as he comes home really late from work… is actually a pretty good caricature of a script that lots of couples play out, as long as you know to ignore the gender and class assumptions embedded in it.

But it’s hard to sort this out without just enacting our scripts. The version of you that would be thinking about it is your character, which (in this framework) can accurately understand its own role only if it has enough slack to become genre-savvy within the web; otherwise it just keeps playing out its role.

In the husband/wife script mentioned above, there’s a tendency for the “wife” to get excited when “she” learns about the relationship script, because it looks to “her” like it suggests how to save the relationship — which is “her” enacting “her” role. This often aggravates the fears of the “husband”, causing “him” to pull away and act dismissive of the script’s relevance (which is “his” role), driving “her” to insist that they just need to talk about this… which is the same pattern they were in before. They try to become genre-savvy, but there (usually) just isn’t enough slack between them. So their effort merely changes the topic while they play out their usual scene.

So if you don't like the story you're in, how do you really change it?

Well, it depends on which "you" is asking the question.

Characters often want change as part of their role. And just as importantly, their role often requires that they can't achieve that change. The tension between craving and deprivation gives birth to the character's dramatic raison d'être. The "wife" can't be as clingy and anxious if the "husband" opens up, so "she" enacts behavior that "she" knows will make "him" close down. "She" can't really choose to change this because "her" thwarted desire for change is part of "her" role.

Intentionally creating real personal change requires the player to decide to shake things up. Characters avoid understanding this clearly for basically the same reason that most works of fiction avoid breaking the fourth wall.

But I claim there's a way to sidestep this and inject meaningful genre-savviness into your character if you (the player) so choose.

The essence of this is to stop.

Just stop.

For a little while, pause the incessant activity, the trying to figure out, the jumping into reaction when a feeling or idea bursts into awareness, the fidgeting to dispel social or physical discomfort instead of savoring it.

Just let all avenues for acting out a role come to stillness.

And then in your stillness, listen closely to your experience as though this is the first moment you've ever experienced anything at all.

This whole process is practically guaranteed to make your character flip out. I don't claim you'll like doing this (at least at first), or that it'll make sense to you (at first). Maybe it sounds too much like meditation and you have a storm of thoughts associated with that. Maybe the idea is so atrocious or ill-founded or mystic-flavored to you that you don't want to even try it.

And that's fine! Maybe it makes sense for you to wait until death forces this stillness on you.

But if you choose to try it anyway, you can watch as your character does these theatrics…

…and clearly see for yourself that they really are just theatrics

…and you can start to consciously remember who you are beyond all that reactivity.

That reactivity is what takes up slack. When you attend to deep stillness this way, you can directly see for yourself [LW · GW] how to create slack, just as clearly as you can feel your tongue in your mouth. And just as clearly, you can watch the ebb and flow of the social web and the ways in which you and everyone else pretends to be bound by its laws.

Then it'll be immensely obvious to you how to create real change in your life.


This was long, so I’ll try to summarize:

I’ll close this post by noting that there’s a meta-level to track here. In the story The Emperor’s New Clothes, the child’s utterance wasn’t enough on its own to pop the illusion:

"But the Emperor has nothing at all on!" said a little child.
"Listen to the voice of innocence!" exclaimed his father; and what the child had said was whispered from one to another.
"But he has nothing at all on!" at last cried out all the people. The Emperor was vexed, for he knew that the people were right; but he thought the procession must go on now! And the lords of the bedchamber took greater pains than ever, to appear holding up a train, although, in reality, there was no train to hold.

What if the father had instead responded “No, child, you’re just too foolish to see his fine garments”? He might have, out of fear of what those who were standing nearby might think of him and his kid. Then the child’s simple voice of reason would not be heard.

Or what if the people near the father/child pair had felt too uneasy to pass along what the child had said?

What if the Emperor could have instilled this kind of nervousness in his people ahead of time? He might have thought that there will be innocent children in the parade, and it might have occurred to some part of him that they had best not be taken seriously — to spare others their embarrassment, of course. Then, oh then what strange propaganda they all would see.

Some of the scripts the social web assigns work less well if they’re known. Because of this, the web will often move to silence people who threaten to speak those fragile truths. This can show up, for instance, as people trying to dismiss and discredit the person saying the idea rather than just the idea. The arguments usually sound sensible on the surface, but the underlying tone ringing through the strands of the web is “Don’t listen to this one.”

If it’s not clear why I’m mentioning this, then I imagine it’ll become really obvious quite soon.

112 comments

Comments sorted by top scores.

comment by Jacob Falkovich (Jacobian) · 2019-12-10T19:30:41.701Z · LW(p) · GW(p)

In my opinion, the biggest shift in the study of rationality since the Sequences were published were a change in focus from "bad math" biases (anchoring, availability, base rate neglect etc.) to socially-driven biases. And with good reason: while a crash course in Bayes' Law can alleviate many of the issues with intuitive math, group politics are a deep and inextricable part of everything our brains do.

There has been a lot of great writing describing the issue like Scott’s essays on ingroups and outgroups and Robin Hanson’s theory of signaling. There are excellent posts summarizing the problem of socially-driven bias on a high level, like Kevin Simler’s post on crony beliefs. But The Intelligent Social Web offers something that all of the above don’t: a lens that looks into the very heart of social reality, makes you feel its power on an immediate and intuitive level, and gives you the tools to actually manipulate and change your reaction to it.

Valentine’s structure of treating this as a “fake framework” is invaluable in this context. A high-level rigorous description of social reality doesn’t really empower you to do anything about it. But seeing social interactions as an improv scene, while not literally true, offers actionable insight.

The specific examples in the post hit very close to home for me, like the example of one’s family tugging a person back into their old role. I noticed that I quite often lose my temper around my parents, something that happens basically never around my wife or friends. I realized that much of it is caused by a role conflict with my father about who gets to be the “authority” on living well. I further recognized that my temper is triggered by “should” statements, even innocuous ones like “you should have the Cabernet with this dish” over dinner. Seeing these interactions through the lens of both of us negotiating and claiming our roles allowed me to control how I feel and react rather than being driven by an anger that I don’t understand the source of. An issue that I struggled with for years was mostly resolved after reading this post and thinking about it for a while.

The post’s focus on salient examples (family roles, the convert boyfriend, the white man’s role) also has a downside, in that it’s somewhat difficult to keep track of the main thrust of Valentine’s argument. The entire introductory section also does nothing to help the essay cohere; it makes claims about personal benefits Valentine has acquired by using this framework. These claims are neither substantiated nor explored further in the essay, and they are also unnecessary — the essay is compelling by the force of its insight and not by promising a laundry list of results.

Valentine does not go into detail about the reasons that people “need the scene to work” above all other considerations. This for two reasons: the essay is long enough as it is, and the underlying structure is more speculative than established. I hope to see more people exploring this underlying structure as a follow up. I recommend Sarah Constantin’s look at abusive relationships through the lens of playing out familiar roles [LW · GW]; I have also written an essay fitting Valentine’s idea into a broader framework of how predictive processing shapes how we think about identity and social interaction.

But again: The Intelligent Social Web didn’t just inspire me to write about ideas, it changed how I live my life. Whenever I feel a discordant emotion in a social interaction or have a goal that is thwarted I put on the framework of improv scenes and social roles to understand what is happening. And every time I reread the post after trying out the framework in real life, I glean more from it. If the post was slightly better structured and focused it could reach more readers, but it is already the most impactful thing I read on LessWrong in 2018.

Replies from: Valentine, Hazard
comment by Valentine · 2019-12-21T19:16:24.717Z · LW(p) · GW(p)

Thank you. Thank you for sharing how you were impacted. That touched me. I'm delighted to have played a role in you enjoying your life more fully. :-)

The post’s focus on salient examples (family roles, the convert boyfriend, the white man’s role) also has a downside, in that it’s somewhat difficult to keep track of the main thrust of Valentine’s argument. The entire introductory section also does nothing to help the essay cohere; it makes claims about personal benefits Valentine has acquired by using this framework. These claims are neither substantiated nor explored further in the essay, and they are also unnecessary — the essay is compelling by the force of its insight and not by promising a laundry list of results.

I quite agree. Thank you for stating this so clearly.

At the time I was under the delusion that people would read and consider what I had to say because they consciously could expect a benefit from doing so. So I tried to state the value up front. I think I was also a little embarrassed to be talking in public in a way I wasn't aware of, so the "laundry list" was a way of assuaging my unrecognized shame.

All of which is to say, I agree. :-) And I'm glad this point got into the reviews for this.

comment by Hazard · 2019-12-11T15:10:14.325Z · LW(p) · GW(p)
In my opinion, the biggest shift in the study of rationality since the Sequences were published were a change in focus from "bad math" biases (anchoring, availability, base rate neglect etc.) to socially-driven biases.

Funny enough, when I did a reread through the sequence, I saw a huge number of little ways EY was pointing to various socially driven biases, which I'd missed the first time around. I think it might have been a framing thing, where because it didn't feel like those bits were the main point of the essays, I smashed them all into "Don't be dumb/conformist" (a previous notion I could round off to).

Also great review.

comment by sarahconstantin · 2018-02-23T06:06:47.596Z · LW(p) · GW(p)

Just posting to record that this post successfully alarmed me, by raising the possibility that I might be missing really important things.

comment by PeterBorah · 2018-02-23T07:06:04.709Z · LW(p) · GW(p)

This feels like a really useful framing. It meshes with other fake frameworks I sometimes use, but the emphasis on the web pulling you back in if you don't break with it hard enough feels true and important.

If anyone remembers the r/place experiment Reddit did, similar dynamics were extremely apparent. (In brief, /r/place was a blank 1000x1000 pixel canvas, where anyone with a Reddit account could place one colored pixel anywhere they wanted every 5 minutes.) It was actually really hard to randomly vandalize anything, because wrong pixels looked out of place and would be fixed pretty fast. The only things that worked were:

1) Building a new pattern in a neutral location (which might eventually grow big enough to challenge existing patterns), or

2) Nudging a pattern into a different nearby attractor.

You didn't see coherent images dissolving into noise, because bystanders would fix things too fast. But you did see things like adding genitalia to Charizard, or changing the text "PC MASTER RACE" to "PC MASTURBATE", because those could start as relatively minor changes that bystanders might decide to help with.

The most skillful application of this I saw was when some people working on the powerful "Rainbow Road" pattern didn't want to overwrite the Where's Waldo image. They decided to try to send the road through a "portal". You can see it happen between 0:35 and 0:45 in the timelapse. A small team coordinated on Discord to build an entrance and exit portal, with a little bit of rainbow coming out of each, in exactly the right spot that the "hivemind" would naturally run into it. This worked amazingly well, and the hivemind moved to continuing the rainbow from the other end of the portal pretty much effortlessly.

The lesson I draw is that if you want to break out of a script, you can't just act illegibly and hope that will work. You need to put in the work to create or appropriate a script of your own.

comment by Valentine · 2019-12-21T20:18:44.565Z · LW(p) · GW(p)

I don't know if I'll ever get to a full editing of this. I'll jot notes here of how I would edit it as I reread this.

  • I'd ax the whole opening section.
    • That was me trying to (a) brute force motivation for the reader and (b) navigate some social tension I was feeling around what it means to be able to make a claim here. In particular I was annoyed with Oli and wanted to sidestep discussion of the lemons problem. My focus was actually on making something in culture salient by offering a fake framework. The thing speaks for itself once you look at it. After that point I don't care what anyone calls it.
    • This would, alas, leave out the emphasis that it's a fake framework. But I've changed my attitude about how much hand-holding to do for stuff like that. Part of the reason I put that in the beginning was to show the LW audience that I was taking it as fake, so as to sidestep arguments about how justified everything is or isn't. At this point I don't care anymore. People can project whatever they want on me because, uh, I can't really stop them anyway. So I'm not going to fret about it.
    • I had also intended the opening to have a kind of conversational tone, as part of a Sequence that I never finished (on "ontology-cracking"). I probably never will finish it at this point. So no point in making this stand-alone essay pretend to be part of an ongoing conversation.
  • A minor nitpick: I open the meat of the idea by telling some facts about improv theater. I suspect it'd be more engaging if I had written it as a story illustrating the experience. "Bob walked onto the stage, his heart pounding. 'God, what do I say?'" Etc. The whole thing would have felt less abstract if I had done that. But it clearly communicated well for this audience, so that's not a big concern.
  • One other reviewer mentioned how the strong examples end up obfuscating my overall point. That was actually a writing strategy: I didn't want the point stated early on and elucidated throughout. I wanted the reader to resonate with what I was describing, and then use that resonance to point out an implication of the reader's own life. That said, I bet I could do that with more punch and precision these days.
  • Reading over the "abuser"/"victim"/"rescuer" stuff, I'm now reminded of Karpman's Triangle. I didn't know about that at the time. Karpman was a grad student under Eric Berne, the father of Transactional Analysis. These days many folk know it as "the drama triangle". Were I writing this essay today I might reference this triangle.
  • I feel like most of the value of the improv analogy is actually in the contrast between player and character. When I hear about people being impacted by this article, most of what I hear has to do with the mechanics of how the social scene unfolds and how that creates constraints (anti-slack). Which is wonderful! But if I had to choose one illumination for people to experience from this whole thing, I'd rather they get a glimpse of who they are as the player, and how much that really really isn't the character that's usually talking and saying "I", "me", and "my". It's immensely freeing to see this clearly. But there's a lot of pleasure to be taken in playing genre-naïve characters, and I don't mean to dismiss that. That's just not the scene type I want to play in anymore. So on net, this wish of mine probably wouldn't meaningfully affect how I'd edit this piece.
  • The reason for referencing Omega was to foreshadow a later post on Newcomblike self-deception.
    • The short version is: If Omega is modeling your self-model instead of your actual source code to predict your actions, then you're highly incentivized to separate your self-model from your method of choosing your actions. Then you can two-box while convincing Omega you'll one-box by sincerely but falsely believing you're going to one-box. This paints a pretty vivid picture if you view the intelligent social web as the real-world version of Omega with "social role" playing the part of "self-model".
    • I'd now skip that whole reference. It made sense only in my mind. And even if I had finished the Sequence this was part of, the references to Omega would make sense only to those who had finished it and then went back to reread this essay.
  • There's something about how this essay uses the concept of slack that nags at me. I suspect it's fine for the purposes of the 2018 review, but I'd be remiss not to mention it. The intuition about slack is itself interpreted from within the social web. But slack affects only the character. So although slack is a genre-savvy concept, it's still a concept within the web itself. That introduces a dimension of self-reference that might be elegantly self-reinforcing, paradoxical, or something else. I honestly don't know.
    • This has me wonder about there being a type of construct, which is genre-savvy concepts. This whole model is an example, as is the concept of genre-savviness. I suspect that's a gateway to an insight type that's usually called "spiritual".
  • There's a bit where I refer to the possibility of using Looking to shift roles. I have a much more sophisticated view of this now. I think I was being truthful and reasonably accurate… and yet for the sake of the essay I would either expand on that reference to clarify it, or remove the reference entirely. It's not helpful to say "There's a magic consciousness thingie you can do that'll do things your character can't understand" if that's literally all I say about it.

So, with all that said, here are the edits I'd make:

  • Cut the opening section.
  • Add a hyperlink to Karpman's Triangle.
  • Erase references to Omega, maybe expanding a bit where needed instead.
  • Either delete references to changing one's fate by Looking, or spell it out in less mysterious terms.
Replies from: Valentine, Raemon
comment by Valentine · 2020-01-01T13:01:44.073Z · LW(p) · GW(p)

I've made my edits. I think my most questionable call was to go ahead and expand the bit on how to Look in this case.

If I understand the review plan correctly, I think this means I'm past the point where I can get feedback on that edit before voting happens for this article. Alas. I'm juggling a tension between (a) what I think is actually most helpful vs. (b) what I imagine is most fitting to where Less Wrong culture seems to want to go.

If it somehow makes more sense to include the original and ignore this edit, I'm actually fine with that. I had originally planned on not making edits.

But I do hope this new version is clearer and more helpful. I think it has the same content as the original, just clarified a bit.

Replies from: Raemon, Long try
comment by Raemon · 2020-01-02T20:28:34.455Z · LW(p) · GW(p)

Thanks! Probably will end up with a couple more thoughts but definitely appreciate you making some time for this. :)

Replies from: habryka4
comment by habryka (habryka4) · 2020-01-02T20:47:26.043Z · LW(p) · GW(p)

Indeed, thank you a lot for taking the time for this! 

comment by Long try · 2020-02-19T10:54:24.730Z · LW(p) · GW(p)

May be off-topic, but can you elaborate on where LW culture wants to go? Or point to a specific post...

Replies from: Valentine
comment by Valentine · 2020-02-19T18:18:19.822Z · LW(p) · GW(p)

I can't point to a specific post without doing more digging than I care to do right now. I wouldn't be too shocked to find out I'm drastically wrong. It's just my impression from (a) years of interacting with Less Wrong before plus (b) popping in every now and again to see what social dynamics have and haven't changed.

With that caveat… here are a couple of frames to triangulate what I was referring to:

  • In Ken Wilber's version of Spiral Dynamics, Less Wrong is the best display of Orange I know of. Most efforts at Orange these days are weaksauce, like "I Fucking Love Science" (which is more like Amber with an Orange aesthetic) or Richard Dawkins' "Brights" campaign. I could imagine a Less Wrong that wants to work hard at holding Orange values as it transitions into 2nd Tier (i.e., Wilber's Teal and Turquoise Altitudes), but that's not what I see. What I see instead is a LW that wants to continue to embody Orange more fully and perfectly, importing and translating other frameworks into Orange terms. In other words, LW seems to me to have devoted to keep playing in 1st Tier, which seems like a fine choice. It's just not the one I make.
  • There's a mighty powerful pull on LW to orient toward propositional knowing. The focus is super-heavy on languaging and explicit models. Questions about deeper layers of knowing (e.g., John Vervaeke's breakdown in terms of procedural, perspectival, and participatory forms of knowing) undergo pressure to be framed in propositional terms and evaluated analytically to be held here. The whole thing with "fake frameworks [LW · GW]" is an attempt to acknowledge perspectival knowing… but there's still a strong alignment I see here with such knowing being seen as preliminary or lacking in some sense unless and until there's a propositional analysis that shows what's "really" going on. I notice the reverse isn't really the case: there isn't a demand that a compelling model or idea be actionable, for instance. This overall picture is amazing for ensuring that propositional strengths (e.g., logic) get integrated into one's worldview. It's quite terrible at navigating metacognitive blindspots though.

From what I've seen, LW seems to want to say "yes" maximally to this direction. Which is a fine choice. There aren't other groups that can make this choice with this degree of skill and intelligence as far as I know.

There's just some friction with this view when I want to point at certain perspectival and participatory forms of knowing, e.g. about the nature of the self. You can't argue an ego into recognizing itself [LW · GW]. The whole OP was an attempt to offer a perspective that would help transform what was seeable and actionable; it was never meant to be a logical argument, really. So when asked "What can I do with this knowledge?", it's very tricky to give a propositional model that is actually actionable in this context — but it's quite straightforward to give some instructions that someone can try so as to discover for themselves what they experience.

I was just noticing that bypassing theory to offer participatory forms of knowing was a mild violation of norms here as I understand them. But I was guessing it was a forgivable violation, and that the potential benefit justified the mild social bruising.

Replies from: romeostevensit, Raemon, habryka4, Long try
comment by romeostevensit · 2020-02-19T19:09:34.087Z · LW(p) · GW(p)

I don't think everyone playing on the propositional level is unaware of its shortcomings, many just recognize that propositional knowledge is the knowledge that scales and therefore worthy of investment despite those shortcomings. And on the other side of things you have Kegan 3 (I don't like Integral terms for reasons related to this very topic) people with some awareness of Kegan 5 but having skipped a healthy Kegan 4 and therefore having big holes which they tend to paper over with spiritual bypassing. They are the counterpart to the rationalist strawmen who skipped over a healthy Kegan 3 (many of us here do have shades of this) and run into big problems when they try to go from 4 to 5 because of those holes from 3.

Replies from: Valentine
comment by Valentine · 2020-02-19T23:09:50.260Z · LW(p) · GW(p)
I don't think everyone playing on the propositional level is unaware of its shortcomings…

I didn't mean to imply that everyone was unaware this way. I meant to point at the culture as a whole. Like, if the whole of LW were a single person, then that person strikes me as being unaware this way, even if many of that person's "organs" have a different perspective.

…propositional knowledge is the knowledge that scales…

That's actually really unclear to me. Christendom would have been better defined by a social order (and thus by individuals' knowing how to participate in that culture) than it would have by a set of propositions. Likewise #metoo spread because it was a viable knowing-how: read a #metoo story with the hashtag, then feel moved to share your own with the hashtag such that others see yours.

comment by Raemon · 2020-02-19T20:58:44.290Z · LW(p) · GW(p)

tl;dr: that raised some interesting points. I'm not sure "actionable" is the right lens but something nearby resonated.

My current take is something like "yes, LessWrong is pretty oriented towards propositional knowledge". Not necessarily because it's the best or only way, as Romeo said, because it's a thing that can scale in a particular way and so is useful to build around.

Your point that "fake frameworks that are actionable are seen as preliminary, but there doesn't seem to be a corresponding sense that compelling-but-inactionable-models are also 'preliminary'" was interesting. I hadn't quite thought through that lens before.

Thinking through that lens a bit now, what I'd guess is that "actually, yes, non-actionable-things are also sort of preliminary." (I think part of the point of the LW Review was to check 'okay, has anyone actually used these ideas in a way that either connected directly with reality, or showed some signs of eventually connecting.' A concept I kept pointing at during the Review process was "which ideas were true, and also useful?") 

But, I think there is still some kind of tradeoff being made here, that isn't quite about actionability vs vetted-explicit-knowledge. The tradeoff is in instead some vaguer axis of "the sort of stuff I imagine Val is excited about", that has more to do with... like, in an environment that's explicitly oriented towards bridging gaps between explicit and tacit knowledge, with tacit knowledge treated as something that should eventually get type-checked into explicit knowledge and vetted if possible, some frames are going to have an easier time being talked about.

So, I do think there are going to be some domains that LessWrong is weaker at, and that's okay. I don't think actionability is the thing though.

Some of this is just about tacit or experiential knowledge just being real-damn-hard-to-convey in writing. A lot of the point of the original sequences was to convey tacit knowledge about how-to-think. A lot of the currently-hard-to-talk-about-explicitly-stuff is stuff that's real important to figure out how to convey and write up nuanced sequences about. (I do think it's necessary to figure out how to convey it in writing, as much as possible, because there are serious limits to in-person-workshop scalability)

Replies from: Valentine
comment by Valentine · 2020-02-19T22:34:05.993Z · LW(p) · GW(p)
I'm not sure "actionable" is the right lens but something nearby resonated.

Agreed. I mean actionability as an example type. A different sort of example would be Scott introducing the frame of Moloch. His essay didn't really offer new explicit models or explanations, and it didn't really make any action pathways viable for the individual reader. But it was still powerful in a way that I think importantly counts.

By way of contrast, way back in the day when CFAR was but a glimmer in Eliezer's & Anna's eye, there was an attempted debiasing technique vs. the sunk cost fallacy called "Pretend you're a teleporting alien". The idea was to imagine that you had just teleported into this body and mind, with memories and so on, but that your history was something other than what this human's memory claimed. Anna and Eliezer offered this to a few people, presumably because the thought experiment worked for them, but by my understanding it fell flat. It was too boring to use. It sure seems actionable, but in practice it neither lit up a meaningful new perspective (the way Meditations on Moloch did) nor afforded a viable action pathway (despite having specific steps that people could in theory follow).

What it means to know (in a way that matters) why that technique didn't work is that you can share a debiasing technique with others that they can and do use. Models and ideas might be helpful for getting there… but something goes really odd when the implicit goal is the propositional model. Too much room for conversational Goodharting.

But a step in the right direction (I think) is noticing that the "alien" frame doesn't in practice have the kind of "kick" that the Moloch idea does. Despite having in-theory actionable steps, it doesn't galvanize a mind with meaning. Turns out, that's actually really important for a viable art of rationality.

Not necessarily because it's the best or only way, as Romeo said, because it's a thing that can scale in a particular way and so is useful to build around.

I'm wanting to emphasize that I'm not trying to denigrate this. In case that wasn't clear. I think this is valuable and good.

…an environment that's explicitly oriented towards bridging gaps between explicit and tacit knowledge…

This resonates pretty well with where my intuition tends to point.

Some of this is just about tacit or experiential knowledge just being real-damn-hard-to-convey in writing.

That's something of an illusion. It's a habit we've learned in terms of how to relate to writing. (Although it's kind of true because we've all learned it… but it's possible to circumnavigate this by noticing what's going on, which a subcommunity like LW can potentially do.)

Contrast with e.g. Lectio Divina.

More generally, one can dialogue with the text rather than just scan it for information. You can read a sentence and let it sink in. How does it feel to read it? What is it like to wear the perspective that would say that sentence? What's the feel on the inside of the worldview being espoused? How can you choose to allow the very act of reading to transform you?

A lot of Buddhist texts seem to have been designed to be read this way. You read the teachings slowly, to let it really absorb, and in doing so it guides your mind to mimic the way of being that lets you slip into insight.

This is also part of the value of poetry. What makes poetry powerful and important is that it's writing designed specifically to create an impact beneath the propositional level. There's a reason Rumi focused on poetry after his enlightenment:

"Sit down, be still, and listen.
You are drunk
and this is
the edge of the roof."
~Rumi

Culture has quite a few tools like these for powerfully conveying deep ways of knowing. Along the same lines as I mentioned in my earlier comment above, I can imagine a potential Less Wrong that wants to devote energy and effort toward mastering this multimodal communication process in order to dynamically create a powerful community of deep practice of rationality. But it's not what I observe. I doubt three months from now that there'll be any relevant uptick in how much poetry appears on LW, for instance. It's just not what the culture seems to want — which, again, seems like a fine choice.

comment by habryka (habryka4) · 2020-02-19T19:07:33.362Z · LW(p) · GW(p)

(Note: I found this comment helpful in thinking about LessWrong, though don't have much to say in response)

comment by Long try · 2020-02-21T09:57:39.128Z · LW(p) · GW(p)

Thanks a bunch, Val. I say you saved me dozens if not hundreds of hours, because I was (am) pretty confused about the big picture around here.

The associated Ken Wilber image helps with the understanding a lot. Now, if I don't really get nearly half of the articles on LW, does that mean I'm redder than orange? Are there tests on the internet where I can pretty reliably tell where I'm standing on that scale? Also, I'm quite sure that my goal is to get to the turquoise level. What online resources I should learn and/or what "groups" I should join, in your personal recommendation?

Replies from: Valentine
comment by Valentine · 2020-02-21T18:12:39.699Z · LW(p) · GW(p)

I'm glad to have helped. :)

I'll answer the rest by PM. Diving into Integral Theory here strikes me as a bit off topic (though I certainly don't mind the question).

comment by Raemon · 2020-01-05T07:19:24.848Z · LW(p) · GW(p)

Some thoughts I wanted to share on this aspect (speaking only for myself, not Oli or anyone else)

[quick meta note: the deadline for editing was extended till the 13th, and I think there’s a chance we may extend it further]

  • That was me trying to (a) brute force motivation for the reader and (b) navigate some social tension I was feeling around what it means to be able to make a claim here. In particular I was annoyed with Oli and wanted to sidestep discussion of the lemons problem. My focus was actually on making something in culture salient by offering a fake framework. The thing speaks for itself once you look at it. After that point I don’t care what anyone calls it.
  • This would, alas, leave out the emphasis that it’s a fake framework. But I’ve changed my attitude about how much hand-holding to do for stuff like that. Part of the reason I put that in the beginning was to show the LW audience that I was taking it as fake, so as to sidestep arguments about how justified everything is or isn’t. At this point I don’t care anymore. People can project whatever they want on me because, uh, I can’t really stop them anyway. So I’m not going to fret about it.

I agree that axing the previous opening section was mostly good – it was a bit overwrought and skipping to the meat of the article seems better. I think what I'd personally prefer (over the new version), is a quick: “Epistemic Status: Fake Framework”. You sort of basically have that with the new version (linking to Fake Frameworks at the beginning, but we have the Epistemic Status convention to handle it slightly more explicitly, without taking up much space)

What I think I actually prefer, overall (for LW culture) is something like:

  • Individual posts can give a quick disclaimer to let readers know how they’re supposed to relate to an article, epistemically. Fake Frameworks are a fine abstraction. This should be an established concept that doesn't require much explanation each time.
  • Over the long term, there is an expectation that if Fake Frameworks stick around, they are expected to get grounded out into "real" frameworks, or at least the limits of the framework is more clearly spelled out. This often takes lots of exploration, experimentation, modeling, and explanatory work, which can often take years. It makes sense to have a shared understanding that it takes years (esp. because often it’s not people’s full time job to be writing this sort of thing up), but I think it’s pretty important to the intellectual culture for people to trust that that’s part of the longterm goal (for things discussed on LessWrong anyhow)

I think a lot of the earlier disagreements or concerns at the time had less to do with flagging frameworks as fake, and more to do with not trusting that they were eventually going to ground out as “connected more clearly to the rest of our scientific understanding of the world”.

I generally prefer to handle things with “escalating rewards and recognition” rather than rules that crimp people’s ability to brainstorm, or write things that explain things to people with some-but-not-all-of-a-set-of-prequisites.

So one of the things I’m pretty excited about for the review process is creating a more robust system for (and explicit answer to the question of) “when/how do we re-examine things that aren’t rigorously grounded?“.

I don’t think things necessarily need to be ‘rigorously grounded’ to be in the 2018 Book, but I do think the book should include “taking stock of ‘what the epistemic status of each post is’ and checking for community consensus on whether the claims of the post hold up’", with some posts flagged as "this seems straightforwardly true" and others flagged as "this seems to point in an interesting and useful thing, but further work is needed."

This is all to say: I have gotten value out of this post and think it’s pointing at a true thing, but it’s also a post that I’d be particularly interested in people reviewing, from a standpoint of “okay, what actual claims is the post implying? What are the limits of the fake framework here? How does this connect to the rest of our best understanding of what's going on in the brain?” (the previous round of commenters explored this somewhat but only in very vague terms).

Replies from: Valentine
comment by Valentine · 2020-01-05T23:26:45.902Z · LW(p) · GW(p)
I think what I'd personally prefer (over the new version), is a quick: “Epistemic Status: Fake Framework”.

Like so? (See edit at top.) I'm familiar with the idea behind this convention. Just not sure how LW has started formatting it, or if there's desire to develop much precision on this formatting.

I think a lot of the earlier disagreements or concerns at the time had less to do with flagging frameworks as fake, and more to do with not trusting that they were eventually going to ground out as “connected more clearly to the rest of our scientific understanding of the world”.

Mmm. That makes sense.

My impression looking back now is that the dynamic was something like:

  • [me]: Here's an epistemic puzzle that emerges from whether people have or haven't experience flibble.
  • [others]: I don't believe there's an epistemic puzzle until you show there's value in experiencing flibble.
  • [me]: Uh, I can't, because that's the epistemic puzzle.
  • [others]: Then I'm correct not to take the epistemic puzzle seriously given my epistemic state.
  • [me]: You realize you're assuming there's no puzzle to conclude there's no puzzle, right?
  • [others]: You realize you're assuming there is a puzzle to conclude there is, right? Since you're putting the claim forward, the onus is on you to break the symmetry to show there's something worth talking about here.
  • [me]: Uh, I can't, because that's the epistemic puzzle.

(Proceed with loop.)

What I wasn't acknowledging to myself (and thus not to anyone else either) at the time was that I was loving the frustration of being misunderstood. Which is why I got exasperated instead of just… being clearer given feedback about how I wasn't clear.

I'm now much better at just communicating. Mostly by caring a heck of a lot more about actually listening to others.

I think you're naming something I didn't hear back then. And if nothing else, it's something you value now, and I can see how it makes sense as a value to want to ground Less Wrong in. Thanks for speaking to that.

I don’t think things necessarily need to be ‘rigorously grounded’ to be in the 2018 Book, but I do think the book should include “taking stock of ‘what the epistemic status of each post is’ and checking for community consensus on whether the claims of the post hold up’", with some posts flagged as "this seems straightforwardly true" and others flagged as "this seems to point in an interesting and useful thing, but further work is needed."

That seems great. Kind of like what Duncan did with the CFAR handbook.

This is all to say: I have gotten value out of this post and think it’s pointing at a true thing, but it’s also a post that I’d be particularly interested in people reviewing, from a standpoint of “okay, what actual claims is the post implying? What are the limits of the fake framework here? How does this connect to the rest of our best understanding of what's going on in the brain?” (the previous round of commenters explored this somewhat but only in very vague terms).

Mmm. That's a noble wish. I like it.

I won't respond to that right now. I don't know enough to offer the full rigor I imagine you'd like, either. So I hope for your sake that others dive in on this.

Replies from: Raemon
comment by Raemon · 2020-01-05T23:53:00.347Z · LW(p) · GW(p)

I won't respond to that right now. I don't know enough to offer the full rigor I imagine you'd like, either. So I hope for your sake that others dive in on this.

Yeah, to be clear I am expecting this sort of thing to take years to do. (and, part of the point of the review process is that it can be more of a collective effort to either flag issues or resolve them)

What seems like an achievable thing to shoot for this year, by someone-or-other (and I think worth doing whether this post ends up getting included in the book or not), is something like 

a) if anyone does think the post is actually misleading in some way, now's the time for them to say so. (Obviously this isn't something I'd generally expect authors to do, unless they've actually changed their mind on a thing).

b) write out a list of pointers for "what sort of places might you look to figure out how this connects to the rest of psych literature of neuroscience, or what experiments you'd want to see run or models built if there isn't yet existing literature on this". Not as a "fully ground this out in one month", but "notes for future people to followup on."

comment by ZeitPolizei · 2018-02-23T11:20:25.878Z · LW(p) · GW(p)

I haven't really understood where the fakeness in the framework is. And the other comments also seem to not acknowledge, that it is a fake framework, which I am interpreting as people taking this framework at face value to be true or real. I suspect I haven't quite understood what is meant by "fake framework".

I'm currently seeing two main ways in which I can make the fakeness make sense to me:

  1. People do step out of their roles quite often in real life, breaking the expectations of the web. So the framework works better for broad strokes predictions, than specific behavior. Or rather, there is a lot of behavior not accounted for in the framework.
  2. Just like "every model is wrong", every framework is fake, and this is a framework that is "less fake" than others.

A thing that's rather irrelevant to the actual topic at hand, but that I feel like sharing:

  • Me reading this: "Oh neat, links about attachment theory, this is acutely relevant to me. Let's check out the 'how to tell if your partner is X' links.
  • Reading the links: "Hm yeah, seems like I'm anxious-preoccupied and the person I'm interested in could be dismissive-avoidant."
  • Reading this again after a few hours, because I felt I hadn't understood it completely: "Oh cool, let's also look at this link 'Ending the Anxious-Avoidant Dance', that seems like it's exactly what I need to know right now."
  • I continue reading the article and…

    In the husband/​wife script mentioned above, there’s a tendency for the “wife” to get excited when “she” learns about the relationship script, because it looks to “her” like it suggests how to save the relationship — which is “her” enacting “her” role.

Wait a second. Aw fuck me, this is exactly what's happening to me right now! My mood instantly improved by a ton and I kept laughing for several minutes.

PS: It's probably more helpful to point your Attachment Theory link to here instead.

Replies from: Valentine, Kaj_Sotala, SquirrelInHell
comment by Valentine · 2018-02-23T18:23:46.039Z · LW(p) · GW(p)
I haven't really understood where the fakeness in the framework is.

Well, by my model of epistemic hygiene, it's therefore especially important to label it "fake" as you step into using it. Otherwise you risk forgetting that it's an interpretation you're adding, and when you can't notice the interpretations you're adding anymore then you have a much harder time Looking at what's true.

In my usage, "fake" doesn't necessarily mean "wrong". It means something more like "illusory". The point of a framework, to me, is that it pumps intuition and highlights clusters and possible Gears. But all of that is coming from your mind, not the territory. When you don't yet know how much to trust a framework, I think it's especially helpful to have clear signs on its boundaries saying "You are now entering a domain of intentional self-induced hallucination."

Like, it's worth remembering that you don't see molecules. When you look at a glass of water and think "Oh, that's dihydrogen monoxide", if you can't tell that that's a thought you're adding and not what you're seeing, then it's very easy for you to get confused. That same kind of thought process goes into things like, "Oh, that person must be unstable." If you think you see how it's objectively true that water makes sense in terms of chemistry, then it starts to seem an awful lot like (e.g.) your judgments of people are observations rather than interpretations.

I think this kind of thing is super important to keep track of when you're using a framework for pragmatic effects. Otherwise you run the risk of either (a) being incapable of benefitting from the framework because there are parts you're suspicious of, or (b) coming to believe the framework wholeheartedly because it seems to produce results for you. It's worth remembering that astrology sure seemed to a lot of people to produce results for centuries, which caused people to speculate about very strange powers radiating from the stars.

So, I'm saying "This is a fake framework" as a reminder to track that it's adding an interpretive layer… which I think is especially important if the framework comes across as obviously true.

I'll have a lot more to say about this general point later in the sequence, by the way.

Wait a second. Aw fuck me, this is exactly what's happening to me right now! My mood instantly improved by a ton and I kept laughing for several minutes.

:-)

I loved this story. Thank you for sharing it.

PS: It's probably more helpful to point your Attachment Theory link to here instead.

Ah, you know, I just agree with you. I'll go edit that right after I post this reply.

(For posterity: the original link went here.)

Replies from: Valentine, dsatan
comment by Valentine · 2018-02-23T18:42:57.134Z · LW(p) · GW(p)

Oh, haha, I should be more careful when using a phone interface to read these comments. I visually missed that you'd said:

Just like "every model is wrong", every framework is fake, and this is a framework that is "less fake" than others.

So, yep, basically that.

comment by dsatan · 2018-02-23T19:14:09.045Z · LW(p) · GW(p)
In my usage, "fake" doesn't necessarily mean "wrong". It means something more like "illusory". The point of a framework, to me, is that it pumps intuition and highlights clusters and possible Gears. But all of that is coming from your mind, not the territory.
Like, it's worth remembering that you don't see molecules. When you look at a glass of water and think "Oh, that's dihydrogen monoxide", if you can't tell that that's a thought you're adding and not what you're seeing, then it's very easy for you to get confused.

I'd just like to point out that this leads to a interpretation of map and territory that is really weird from the perspective of the bayesian-skeptical correspondence theory given in the sequences. If I were to give a name pointing at what this metaphysics is, I'd say something like "direct realism". This is not to say that it is wrong.

Replies from: Valentine
comment by Valentine · 2018-02-23T19:44:51.552Z · LW(p) · GW(p)

Yep, I agree.

I'm also concerned that the theory you're pointing out has an ontology problem. I'm hoping to get to spelling my concern out — but that's several posts later in the sequence.

comment by Kaj_Sotala · 2018-02-23T16:47:04.258Z · LW(p) · GW(p)

I would say that it's fake in that we're not literally actors playing roles while being unaware of it, events in our life are not literally scenes in a play, etc. Lots of metaphors that are easy to understand and which import useful reasoning rules from the domain of theater, but the things that they refer to are probably implemented quite differently in brains.

comment by SquirrelInHell · 2018-02-23T15:58:43.606Z · LW(p) · GW(p)Replies from: Valentine
comment by Valentine · 2018-02-23T18:36:02.687Z · LW(p) · GW(p)
comment by Raemon · 2018-02-23T03:52:52.409Z · LW(p) · GW(p)

This is the clearest description of this model of social reality that I've seen – I found the metaphors easy to work with, and appreciated the degree of ways it built on itself.

It's making me think about a few other fake frameworks and how they fit together.

The first half of this post reminded me a lot of Melting Asphalt's Personhood essay. The shared concept is having a interface that makes it easier to plug into a social network. But the scene metaphor was a helpful lens that oriented my thinking slightly differently (and in particular, instead of thinking in terms of "everyone uses the same interfaces so that they tile easily", there are multiple roles which can fit together depending on context).

I'm also interested in how this fits (or doesn't) with the Elephant / Rider metaphor [aka Barrel of Monkeys + Wrangle or Monkeys + Voicebox or whatever]. The Player vs Character model is interesting in that... I think the Character is the one whose more of the PR Agent, which I normally associate with the 'Rider' and conscious thought.

I don't think the mapping quite makes sense, but insofar as Player == Elephant, this is a framing wherein it makes more sense to me to "identify with" the Elephant.

Replies from: Raemon
comment by Raemon · 2018-02-23T04:14:24.273Z · LW(p) · GW(p)

I will note that the metaphor I found least intuitive was calling the network "Omega" – I'd personally have named the post "The Improv Scene Model of Social Reality" or something, rather than focusing on that. But YMMV.

I do think having a unique name for this post to make it easier to refer to (without concept clashing with similarly-named posts) would be handy.

Replies from: Valentine, weft
comment by Valentine · 2018-02-23T06:02:51.004Z · LW(p) · GW(p)

That makes sense. I do want to reference this as akin to Omega from Newcomb's Problem, though. That'll become very, very relevant in a future post. But from where you're standing, I think what you're saying makes a lot of sense.

Replies from: malcolmocean, Valentine
comment by MalcolmOcean (malcolmocean) · 2018-02-24T15:24:30.322Z · LW(p) · GW(p)

Might I propose renaming the post The Social Improv Web? Like Raemon, I think that "Real World Omega", while being an important component of what you're saying here, is less likely to act as a sticky handle for this post.

Perhaps leave the old title in a parenthetical for continuity's sake: The Social Improv Web (aka "Real World Omega").

(I have written on naming concepts to reduce incidents of people thinking they understand terms they haven't even heard before)

Replies from: Kaj_Sotala, Valentine, Unreal
comment by Kaj_Sotala · 2018-02-24T17:14:15.649Z · LW(p) · GW(p)

I strongly support renaming the post into something like "The Social Improv Web".

comment by Valentine · 2018-02-24T19:30:47.849Z · LW(p) · GW(p)

Mmm… strong mixed feelings.

I agree about the general point about naming.

I worry that "The Social Improv Web" creates a different wrong impression though.

Also, none of these are names I natively use for it, but I've started to refer to the emergent distributed intelligence sometimes as "Omega", which totally matches stuff about Newcomblike problems that I'll be talking about later. (The mythic mode name I use for it is "Fate"… but I don't want to embed mythic mode in the way we talk about the framework here. I'd rather keep in mind that mythic mode is one possible implementation.)

I think I, personally, will close to never remember to call it "the social improv web".

So… I'm not yet persuaded.

But I like the thing you're trying to address, and the effort you put into it.

Replies from: Raemon, crybx
comment by Raemon · 2018-02-24T20:57:38.265Z · LW(p) · GW(p)

I do get the concern you have here, although I at least worth checking if something like "Omega, the Social Web" or some such permutation works.

comment by crybx · 2018-02-26T19:07:32.075Z · LW(p) · GW(p)

After reading your Mythic Mode post, and before seeing this comment, I was trying to think of a possible mythic mode name for this other than Omega. Hermaeus Mora, a Lovecraftian-like being from the Elder Scrolls video game series, overpowers any other ideas in my head:

Hermaeus Mora, also known as Hoermius, Hormaius, Hermorah, Herma Mora, and The Woodland Man is the Daedric Prince of knowledge and memory; his sphere is the scrying of the tides of Fate, of the past and future as read in the stars and heavens. He is not known for being good or evil; he seems to be the keeper of both helpful and destructive knowledge

He/it also looks like a bunch of tentacles, which is sort of web like.

I don't think this is remotely a name that could spread, but when I recalled that I thought of him as Herman when I played the game, I became very amused at the idea of calling "The Intelligent Social Web" by the name Herman.

comment by Unreal · 2018-02-24T19:54:55.980Z · LW(p) · GW(p)

I don't like the name "Social Improv Web" for this thing in particular.

The handles I'm mostly taking away from this article are "roles" and "the web."

So my suggested names would be "Your Role in the Web" or "The Web"

[lol "Web" stopped looking like a word to me]

comment by Valentine · 2018-02-26T00:22:40.496Z · LW(p) · GW(p)

Okay, persuaded. How's this?

(Unfortunately, this breaks links to this post…)

Replies from: Raemon
comment by Raemon · 2018-02-26T00:32:34.905Z · LW(p) · GW(p)

It does not! We were smart and clever and the english-looking part of the link is basically irrelevant – all the work is being done by the hash part of the url.

See also:

https://www.lesserwrong.com/posts/AqbWna2S85pFTsHH4/the-intelligent-blargity-blarg-wahooo-wahooooo------smile-syntax-social-web#mY64qZqoEJXzwCTFK

Replies from: Valentine
comment by Valentine · 2018-02-26T00:48:34.145Z · LW(p) · GW(p)

Woohoo! I am pleased to be wrong here!

comment by weft · 2018-02-25T13:41:04.408Z · LW(p) · GW(p)

Agreed. "Omega" already refers to too many other things in our discourse. I almost didn't open this post because I thought it would be about decision theory/ Newcomb. One reason Moloch works is that's it's an old reference that wasn't currently being used to describe a hundred other things.

In keeping with the theme, how about calling the social network something like Hestia, Vesta, or Eunomia. You can write a post personifying it in a poetic and mythic way. Omega can be her son with Hephaestus, if you like.

If you wait too long to change it, it's going to stay Omega by default, and I think that would be a BAD thing.

Replies from: Valentine, Raemon
comment by Valentine · 2018-02-28T03:06:08.464Z · LW(p) · GW(p)

For what it’s worth, the mythic mode name I usually give the social web is “Fate”, and the mythic name I give scripts played out in the web is “fates”. As in, “It’s his fate to be poor, so Fate will see to it that his business does not succeed.”

comment by Raemon · 2018-02-26T00:45:05.233Z · LW(p) · GW(p)

Val did mention he planned to tie it more explicitly in with Omega and decision theory later in the series, so I don't think this particular approach would make sense.

comment by cousin_it · 2018-02-22T23:46:06.331Z · LW(p) · GW(p)

Are social roles really that arbitrary though? I think of it as more of a market, where your position is determined by what you have to sell and for how much. For example, if you're good looking, that means your goods are better. If you're insecure, your prices are higher, and so on. Then people come together and work out a trade. Telling yourself that the market is a shared hallucination (or distributed computation, or whatever) won't change anyone else's incentives, so it won't change your market position. Better to change your goods and asking price and let the market float you up.

Replies from: Valentine, crybx
comment by Valentine · 2018-02-23T01:13:42.002Z · LW(p) · GW(p)
Are social roles really that arbitrary though?

Yes, I think so.

What I think you're highlighting are the constraints that emerge from (a) roles and scripts that have been chosen and (b) environmental factors. That second one is a bit like "The scene has to involve a stuffed bear somehow." That certainly shapes what happens, but it doesn't overwhelm the role choices. It just interacts with them.

Also, whether someone is considered attractive, or insecure, is often determined in large part by their position in the web. Yes, there can be genetic factors that make someone seem visibly more or less genetically fit… but it's pretty impressive how attractive someone can be if it's in common knowledge that everyone desires that person.

But these roles being so "arbitrary" doesn't mean you can just switch them up. You're right, they're highly constrained by context. In the OP, I described this as a question of slack.

Replies from: cousin_it, habryka4
comment by cousin_it · 2018-02-23T01:34:45.435Z · LW(p) · GW(p)

What I think you’re highlighting are the constraints that emerge from (a) roles and scripts that have been chosen and (b) environmental factors.

No mention of personal qualities that determine what you can supply to others and what you demand in return? Do you think anyone could have the right personal qualities if they were put in the right role? That seems wrong, I couldn't lead the troops as well as Napoleon.

it’s pretty impressive how attractive someone can be if it’s in common knowledge that everyone desires that person.

The same thing happens in markets.

Replies from: Valentine
comment by Valentine · 2018-02-23T03:14:12.931Z · LW(p) · GW(p)

Ah, I think we're missing each other through different usage of terminology.

I now think you're trying to point at individuals' resources that they bring into the interactions with other individuals. E.g., assumedly there was something intrinsic to Napoleon that made him a great leader of troops. Let me know if I'm missing you there.

When I translate that into the framework I'm using in the OP, I focus on three things:

  • I would call factors that are intrinsic to individuals part of the "environment". E.g., genetic gifts are environment. This is in the sense that a scene is shaped in part by what's in the set, how energetic and responsive the players are, etc.
  • A lot of things that we think of as intrinsic to individuals are actually a result of their position in the web. Hence my comment about people's attractiveness based on where they are in the web. (And yes, I agree, that happens in markets too. That's not a disagreement. I think you're noticing an overlap in the patterns that two different frameworks point at.)
  • People are usually prevented from reaching and holding roles they can't execute. I posit that you in fact aren't in a position like Napoleon's in large part because you lack the native abilities in question, and the web is accounting for that.
Replies from: cousin_it, Unreal
comment by cousin_it · 2018-02-23T20:06:00.991Z · LW(p) · GW(p)

Well, yeah. Your view seems reasonable enough on its own terms. The reason I'm being a bit naggy in the comments here is because I've got my own pet framework, which has some claim to being the next step after yours in terms of improving life outcomes :-) The market thing. I'll try to explain why I like it so much.

There seems to be a natural progression in how people think about success:

1) Success comes from being better than others

2) Success comes from ~some mind hack~

3) Success comes from trading with others

Imagine you're unhappy about your lot in life. The natural response is to grit your teeth and follow (1), trying to improve yourself so people flock to you. That's a healthy response in many ways. But then there are so many self-help books explaining how reality is in your mind (2), and you can look at it differently to get ahead! That sounds like an amazing opportunity, but to me it's not much different from (1). You're still trying to improve your position, like everyone else, but now you're also using shortcuts. Like everyone else.

The real jump is from (2) to (3), where you realize that you can't succeed alone. You've got to pull someone else along. And once you accept that, you no longer have to worry about 15 year old kids who can beat you at everything. Their existence becomes irrelevant to your success, because comparative advantage allows you to make win-win trades with anyone on Earth, up or down the skill tree. You don't have to beat anyone, or change the game, or change your mind, or do anything unusual at all. It's one of the most liberating ideas I've ever encountered.

Replies from: alkjash, Qiaochu_Yuan, ChristianKl, malcolmocean
comment by alkjash · 2018-02-23T22:47:31.053Z · LW(p) · GW(p)

I like this a lot as well, I was operating under assumption (1) until probably 2 or 3 years ago, until I realized the main value I derive from being better than others is the warm fuzzy of immediately teaching them the thing. So success (1) is self-defeating.

To fully appreciate this I had to disentangle the warm fuzzy of "pretending to teach while lording over" which is admittedly good but nowhere as good as actually watching the other person improve because of my efforts.

comment by Qiaochu_Yuan · 2018-02-23T20:16:42.996Z · LW(p) · GW(p)

I'd be excited to see a top-level post from you elaborating on this with some examples.

comment by ChristianKl · 2018-02-24T09:24:23.274Z · LW(p) · GW(p)
Your view seems reasonable enough on its own terms. The reason I'm being a bit naggy in the comments here is because I've got my own pet framework, which has some claim to being the next step after yours in terms of improving life outcomes

Valentine isn't a hedgehog about this framework. The point of calling it a fake-framework is to make it clear that it isn't the one-framework to explain everything. Given that frameworks are useful intuition pumps having multiple useful frameworks allows you to generate more useful ideas.

Not all value exchange is about market-based trading. David Ronfeld lays out Tribes/Hierarchies/Markets/Networks in "In Search of How Societies Work".

When I ask on StackExchange a question I don't think it's helpful to think of how I trade with the person who will answer my question.

It's more useful to think in terms of roles. There are certain cultural exceptions by the StackExchange community and when I write StackExchange answers it's more useful to think about living up to those norms than thinking about how to trade with people answering.

StackExchange follows network norms of value creation.

Replies from: cousin_it
comment by cousin_it · 2018-02-24T10:39:50.752Z · LW(p) · GW(p)

Huh? StackExchange has karma! So does LW. The value exchange mechanism is exposed for all to see. Reputation systems are designed markets. They also complement goods markets (Amazon, eBay, Yelp). And there's intermediate cases like social media, where companies promote their content by making it upvote-worthy. Thinking that you do stuff to follow norms misses the point: you follow norms to make a profit.

Replies from: ChristianKl
comment by ChristianKl · 2018-02-27T07:18:49.658Z · LW(p) · GW(p)

Popper made the point that one of the problems with Marxism is that the Marxist has no problem to see any conflict as being about class struggle. In the same way you can see every problem as being about market and fit them into that perspective.

On StackExchange you find bad question and answer getting downvotes even when that costs the people who downvote karma. You wouldn't expect that behavior to happen as often it it would be a market where participants purely optimize for getting their questions answered, earning karma and badges.

People desire to do work on StackExchange that doesn't bring them karma. People work through review queues even when that doesn't bring them karma to help the project.

If you start to look at a problem with multiple lenses you see more aspects of it and that helps generating new solutions.

comment by MalcolmOcean (malcolmocean) · 2018-02-24T15:46:34.674Z · LW(p) · GW(p)

I would add:

4) Success comes from collaborating with others

Trade is one way to have an economic interaction where value is created, because each of us might value something twice as much as the other, so when we trade, we get more value. But we can also create value where no value existed before. If you and I play a game together that we both enjoy, we're not trading something: we're creating new experiences that we value. If you and I start a company together, we might be selling our products on a market, but the value we're creating by working together is probably something that neither of us had on our own, therefore not well-modelled as a "trade".

Some might argue this is the same as 3, but it seems like an important distinction to me, and very relevant to improv, also.

comment by Unreal · 2018-02-23T07:45:39.808Z · LW(p) · GW(p)
I would call factors that are intrinsic to individuals part of the "environment".

Why? If you think this, then I start suspecting you might consider everything to be 'arbitrary'. What does 'not arbitrary' look like?

Like, let's say I have a genetic gift like photographic memory. And then I start playing the role as a person who remembers things that they've seen. Maybe people make jokes about it. Or they ask me what was in the paper this morning. Would this still be considered an arbitrary social role?

Replies from: Valentine
comment by Valentine · 2018-02-23T19:41:28.541Z · LW(p) · GW(p)

I now think the word "arbitrary" is shifting meaning around as different people use it. I'd like to taboo it.

Here are some things that, by my model, contribute to defining someone's position in the web:

  • Their genes.
  • The situation that they were born into. (E.g., who their parents are, in the web.)
  • Major physical events around them. (E.g., earthquakes.)
  • Physical constraints of their environment. (E.g., living in non-fertile land.)
  • The positions in the web that people they interact with hold.

I doubt that's exhaustive — but I think it might be close.

I'm reading a bunch of this thread as folk interpreting me as saying that genetic gifts don't matter. That's not my stance. I think genetic gifts do in fact matter — but I suspect I think they matter differently than other people think they do. E.g., many people might think that someone is a natural leader when what's really going on is just that they're tall and can easily put on muscle and have symmetric features and were thus raised to practice playing a role that looks leader-like. But it's helpful to the web for people to think and talk as though the people they've chosen for leadership roles have various virtues that make them worthy.

I think it's helpful to distinguish between recursive factors (your role helps define my role, which helps define your role, etc.) and non-recursive factors (e.g., being tall, or being in a drought). The non-recursive factors define the things that the recursive ones have to address. But the recursive ones define what people treat as real about the non-recursive ones. That doesn't depend on whether the thing is intrinsic to a person (as with genes) or not (as with rain). E.g., climate change has a similar kind of "web-woven reality dominates perception of physical reality" thing going on as with people who claim that Tally McTallface just seems more Presidential.

Hence my wanting to think of genes as part of the "scene" rather than as part of someone's "role".

Like, let's say I have a genetic gift like photographic memory. And then I start playing the role as a person who remembers things that they've seen. Maybe people make jokes about it. Or they ask me what was in the paper this morning. Would this still be considered an arbitrary social role?

Continuing my tabooing of "arbitrary":

I don't think "a person who remembers things that they've seen" is a role in the sense I'm talking about. That describes a function, not a position in the social web.

If there were always a person in every group whom others turned to in order to remember things, then I'd expect it to evolve into a role in the sense I mean. But in that case, you just hope that someone who's actually good at remembering things falls into that role. If that role were really high-status, you might expect it to fall to people who are really convincing rather than people who are accurate whenever there's a difference. The question isn't "Who would an impartial observer think best performs the tasks of that role?" The question is, "Who can play that role in a way that makes the scene work?"

Also, I have a genetic gift that gives me absurd amounts of physical flexibility. I could have been a circus contortionist if I'd wanted to be. And… the web doesn't care. It basically never comes into play. Why? Well, there's no role anywhere near me that takes advantage of that gift.

So, neither having a gift nor lacking it defines what position in the web someone plays.

But obviously genes play a role. E.g., basically everyone in the web is human.

Replies from: Unreal
comment by Unreal · 2018-02-23T20:55:55.336Z · LW(p) · GW(p)
many people might think that someone is a natural leader when what's really going on is just that they're tall and can easily put on muscle and have symmetric features and were thus raised to practice playing a role that looks leader-like. But it's helpful to the web for people to think and talk as though the people they've chosen for leadership roles have various virtues that make them worthy.

FWIW, my current belief is that 'tall' functions as something like a Schelling point. it only slightly grants 'leadership ability' (in that, if you're tall, i'm more likely to be able to SEE you in a crowd, and if you wanted to give me directions, that seems relevant; also taller means i have to look up to see your face, and this costs ME more than YOU, and so given you get natural energy savings in interactions, this is some indication of who has more resources; i expect things like this to compound).

anyway, being tall was probably a much more useful leadership ability way back when, but now it is not a super good indicator.

but given the lack of other more correlated signals that are fast to assess, tall is a natural Schelling point over short. and having Schelling points for this seems useful.

when i enter a room, i want to immediately be able to guess what people's relative status positions are. ideally before i hear them say anything. (because, as you say, i just wanna know what my role is goddamnit!)

my sense is there's something that bothers you about 'tall' ~ 'leader' and it doesn't seem to bother me. so that's where i get curious.

Replies from: PeterBorah, Kaj_Sotala, Qiaochu_Yuan, CronoDAS, Unreal
comment by PeterBorah · 2018-02-23T22:31:33.662Z · LW(p) · GW(p)

This almost seems too obvious to say, but one reason to be bothered by the move from "tall" to "leader" is that sometimes you want your group to have a leader with skills that cause the group to succeed, and the most optimal choice for that might not be the tallest person.

comment by Kaj_Sotala · 2018-02-24T10:41:22.193Z · LW(p) · GW(p)

The discussion about leadership reminded me that siderea has written an absolutely fascinating analysis (part 1, part 2) about leadership; the one problem with the analysis is that it basically requires you to have read at least the first 65 pages of Watership Down. But if you have done that, it's an excellent analysis about how leadership (or kingship as she calls it) isn't a really a formal position in the way that we think about it, but rather about you relate to others in the social web, and how those others relate to you in turn. That seems like a very appropriate perspective for this thread. A couple of quotes:

Adams understands that, contrary to everything our society teaches us, Kingship is not a trait of individuals, but a way of relating. Watership Down is a thorough illustration of the idea that Kingship is a relationship between an individual and a group, and between that individual and the members of that group, which in turn orders those group members' relationships with each others, and outsiders, and the world about them, and thus shapes the very nature and functioning of the group. [...]
Hazel is looking after the well-being of the whole of the group and the individual members of the group. Hazel takes responsibility for helping Pipkin's here, and for figuring out a solution to the groups' flagging strength. He enlists the strength of one to help remedy the weakness of another.
Part of that "ordering and generative" thing about Kings is that they catalyze relationships among their group members. In some sense, Dandelion didn't need Hazel to tell him to tell a story. He didn't need Hazel's permission. He could have realized it would have been useful and volunteered, "Hey, how about a nice story!" But he didn't. It didn't occur to him at all. As far as we know, it never occurs to him to consider the state of the group or what he might do to help his fellow rabbits. Now, don't get me wrong: I don't know that Dandelion ever says or does a selfish or cruel thing; from the first page to the last, he is loyal, brave, and true. I do not mean to say that Dandelion was thoughtless or bad. Dandelion is wonderful.
But the difference between Hazel and Dandelion is that Hazel made the group's problems his problems and addressed them, and Dandelion didn't. And that is Kingship. The part I'm calling caring.
Something that I'm still faintly embarrassed about is that when I first read Watership Down all this was rather lost on me. I literally didn't recognize that any of this had anything do to with what we might term leadership. I mean, not only didn't I realize that Hazel was showing leadership in these various ways, I literally didn't realize that Hazel was functioning as any sort of leader in any way. I conceived of the story through this part and on a bit further as "Hazel and Fiver decide to go their way and other rabbits come with them and they're all peers".
It completely blew by me that Hazel is making most of the decisions, that members of the group defer to his decisions and look to him for answers, that he tells or suggests things to do and the other rabbits do them, that if you were to draw a graph of who talked to whom in the story, you would see a diagram which was almost entirely an asterisk, with Hazel in the center.
I literally didn't notice any of this dynamic. So when later on the question is put explicitly whether Hazel is Chief Rabbit for this band, it shocked me. It hadn't occurred to me in any way that that might be so.
Because I was raised in a society that says that Kingship looks like The Threarah, not like Hazel. Because I was raised in a society which is deeply sick about authority, leadership, and Kingship.
comment by Qiaochu_Yuan · 2018-02-23T21:43:48.262Z · LW(p) · GW(p)

Tallness is also reasonably strong evidence of good genes and nutrition.

Replies from: ialdabaoth
comment by ialdabaoth · 2018-02-25T00:03:29.981Z · LW(p) · GW(p)

Who do you think is a better director of MIRI: Nate Soares, or Luke Muelhauser?

comment by CronoDAS · 2018-02-25T16:18:46.689Z · LW(p) · GW(p)

I think the height thing is a holdover from childhood: someone taller than you is older and more adult-like, so you should do what they say.

comment by Unreal · 2018-02-23T21:02:11.150Z · LW(p) · GW(p)

oh, maybe you're only bothered by people claiming a hand-wavy 'natural leadership' thing when in fact it's stuff like 'they're tall and handsome'. i would agree that is not ideal than being more aware of this kind of thing. belief reporting seems to help suss it out?

comment by habryka (habryka4) · 2018-02-23T03:11:40.852Z · LW(p) · GW(p)

My guess is that about 70% of attractiveness is explained by objective factors, or at least highly inter-subjective factors that are not that dependent on the role someone is playing. There are some studies on this that I read a while ago, this one seems relevant but not the one I remember: https://www.ncbi.nlm.nih.gov/pubmed/9106275

comment by crybx · 2018-02-23T00:07:29.854Z · LW(p) · GW(p)

I would be wary of thinking of social roles as a market. There's something about social interactions that isn't market related, and bringing up prices seems to be able to make people's relationships less fulfilling. At least according to Dan Ariely in the book Predictably Irrational.

"...once a social norm is trumped by a market norm—it will rarely return."

Quick Googling of market vs social norms (or some variation on that) brought up tons of links, but this one does an okay job of summarizing what the book said about it:

https://natewkratzer.wordpress.com/2013/07/23/market-norms-are-crowding-out-social-norms-and-society-is-poorer-as-a-result/

Replies from: cousin_it
comment by cousin_it · 2018-02-23T01:16:47.162Z · LW(p) · GW(p)

There’s something about social interactions that isn’t market related, and bringing up prices seems to be able to make people’s relationships less fulfilling.

Agree with the second part - trade is low status (because it means people aren't rewarding you for inherent qualities). Disagree with the first part - trade is still what's going on, just not with money. It's always a good idea to understand what the other person is buying from you and at what price. The world is full of people who thought they were being rewarded for inherent qualities. Then they accidentally withdrew the thing that was being traded, and lost relationships as a result. Eliezer had a nice essay about it recently.

comment by Qiaochu_Yuan · 2018-02-23T19:20:12.371Z · LW(p) · GW(p)

One lesson I think is really important to draw from this framework is responsibly using the power you have in any given social web. What behaviors from the people around you do you reinforce or punish? Having more choice about what you do here can make a big difference to the behavior of the web depending on your placement in it, although of course your reinforcement is subject to its own meta-reinforcement from others. (At the very least, it's interesting to try being deliberate about what you reinforce and punish, and notice what resistance you get internally and externally from doing so.)

Edit: It will perhaps come as no surprise that I think this is something you can learn via circling. Reinforcement and meta-reinforcement are often implicit and circling is an opportunity to make them explicit so you can vividly see them happening in real time: A punishes B, C punishes A for punishing B, D notices and calls out this dynamic, A punishes D for doing this, etc. Of course it helps a lot to have facilitators skilled enough to manage these sorts of dynamics.

comment by Kaj_Sotala · 2018-02-23T09:18:14.003Z · LW(p) · GW(p)

Fantastic post! I knew that my social environment was super-important for guiding my actions, but this made me realize I might once again have underestimated its significance.

I'm reminded of Kevin Simler's "Personality: An Ecosystems Perspective":

Every class has its clown, because “class clown” is a strong, viable niche (even if it’s not particularly wide) — a stable attractor in the social behavior-space of a classroom. If a class doesn’t yet have its clown, someone will inevitably find that making a wisecrack is rewarded (with social approval in the form of laughter), and before you know it, he or she will be cracking wise at every opportunity.
But like I said, there are far more niches, which are far more nuanced, than just the ones we’ve learned to identify by name. “Alpha,” for example, doesn’t refer to a single niche, but rather a whole class of niches that happen to share a particular feature (being on top). There are many different types of “alpha” niches — leading by intimidation, leading by example, leading by wits and with humor, servant leadership, having an inherited titled (kingdoms etc.), leading with the support of the people, etc.
When it comes to personality, what’s important is the fit between the niche and who you ‘naturally’ (albeit tentatively) are. A lot of personality development involves growing into a niche, but some niches fit better than others, so it’s also important to find the right niche. This is where the body and basic cognitive tendencies come into play. There’s a lot of variation among children (even before adolescence forces them to specialize even further), so it’s important to find a niche that plays to one’s unique strengths.
Replies from: PeterBorah
comment by PeterBorah · 2018-02-23T10:04:11.062Z · LW(p) · GW(p)

I had an extremely nerdy friend group in college, which led to weird effects since we couldn't all be "the nerd". One of my friends still gets annoyed at the fact that she became the "jock" and the "sensible person", just because she was slightly less helpless at life than the rest of us. Her reaction seems to be something like, "I'm for real actually a nerd, why are you making me play this other role??"

comment by habryka (habryka4) · 2018-02-22T21:27:51.174Z · LW(p) · GW(p)

Promoted to frontpage.

comment by Hazard · 2018-02-22T23:16:28.783Z · LW(p) · GW(p)

I've been trying to clarify my thinking on social reality, and this post has helped. The idea that the scene is a thing that exists outside of the actors in it is a very helpful framing.

It also helps me clarify some of the problems I had when I was deep in the uncanny valley of rationality. I grew a strong aversion to anything that smelt like a role, and I also had no idea how I actually wanted to act around other people. This resulted in me either playing and old role and resenting it, or being as non-interactive as I could be without weirding people out.

comment by Original_Seeing · 2019-06-26T16:01:35.872Z · LW(p) · GW(p)

" A few months later he told me he’d converted. Last I heard they had moved to Utah. "

This seems more easily explainable as a Crony Belief. The new belief was very valuable and useful to him, therefore it was adopted as true.

Its value as an epistemic belief plummeted as its value as a crony belief soared since the existence or non-existence of god doesn't have value to people's everyday lives (in the way that an armed robber or cancer diagnosis does).

comment by Raemon · 2018-02-23T06:50:40.564Z · LW(p) · GW(p)
I claim that most of us, most of the time, are playing out characters as defined by the surrounding web — and we usually haven’t a clue how to Look at this fact, much less intentionally use our web slack to change our stories.

Hmm, question – I didn't read the entirety of the Kensho comments, but I'm unclear whether your use of the word "Look" here is an archetypical example of the thing you were meaning to convey in the Kensho post, or if it's just something that could use a similar metaphor to explain a la cell phone world, or some third thing.

(i.e. I wasn't sure if Looking as originally described was meant to be tightly coupled with meditation and/or enlightenment, and if so, is the current usage also meant to be tightly coupled, or more incidentally related)

Replies from: Valentine
comment by Valentine · 2018-02-23T19:01:00.273Z · LW(p) · GW(p)

Mmm, I'm a little confused about what exactly your question is. I think you're trying to ask how my use of "Look" here relates to the kind of "Look" I refer to in my post on kenshō. I'll try to answer that question — but if that's not the question you meant, please let me know!

I claim that there is just one core skill, which I'm calling "Looking". But once one knows how to Look, there are lots of things to Look at. I don't know what you mean when you ask whether the example is "archetypical" — it certainly doesn't strike me as being an archetypal image — but it's definitely an example of applying the skill I was pointing at in Kenshō.

In the cell phone analogy, this would be that once someone knows how to look up at all, it becomes possible to point out different kinds of things that they can use "looking up" for. They can go look at cell towers, or watch pedestrian traffic flow directly, or lock eyes with someone else who's looking up. They're all different applications, but they all pivot on the same core skill.

For a sillier analogy: in the roleplaying game Mage: the Ascension, a mage's magical power comes from their degree of enlightenment, which is measured by the trait called Arete; but what they can use their magical power on is determined by what bodies of knowledge and practice they've mastered, called Spheres. If we are analogous to (potential) mages, then Looking is analogous to using Arete, and different worthwhile things to Look at and come to understand deeply are analogous to the domains governed by Spheres.

Replies from: Raemon
comment by Raemon · 2018-02-24T03:17:23.642Z · LW(p) · GW(p)

Your first two paragraphs mostly answered my question – I'm mostly trying to get a clearer sense of what you mean by Look, and whether it always refers to the same thing.

A different question to circumambulate. :P

Can you describe briefly (nevermind about me necessarily "getting it" in full), 2-3 examples of you Looking at something, and 2-3 examples of you doing something I might mistake for Looking but which is not the same thing, or only superficially similar?

Replies from: Valentine, Hazard
comment by Valentine · 2018-02-24T20:19:32.982Z · LW(p) · GW(p)
Can you describe briefly (nevermind about me necessarily "getting it" in full), 2-3 examples of you Looking at something, and 2-3 examples of you doing something I might mistake for Looking but which is not the same thing, or only superficially similar?

Oh man. I really like this question! Happy to oblige… though that second part is going to be super tricky. I can give lots of non-examples, but you're asking for non-examples that I think you might think are examples. I'm less confident I can do that well. But I'll try!

Some positive examples (briefly, without trying to explain in full):

  • Sometimes in conversations I've noticed that my mind has started moving quickly, and I feel a bit anxious, and when I try to think about why I get a mental fog or I forget what I was thinking about. This is a signal to me that I'm probably running some subconscious strategy. So sometimes I'll pause, and Look behind the wall of fog or forgetfulness, and hold that whole section of my mind as object. Often it tries to squirm out of reach, but I can See where it's squirming, and why, and just trace it to its root.
  • A while back I was interacting with a friend of a friend (distant from this community). His demeanor was very forceful as he pushed on wanting feedback about how to make himself more productive. I felt funny about the situation and a little disoriented, so I Looked at him. My sense of him as an experiencing being deepened, and I started noticing sensations in my own body/emotion system that were tagged as "resonant" (which is something I've picked up mostly from Circling). I also could clearly see the social dynamics he was playing at. When my mind put the pieces together, I got an impression of a person whose social strategies had his inner emotional world hurting a lot but also suppressed below his own conscious awareness. This gave me some things to test out that panned out pretty on-the-nose.
  • I noticed a few weeks ago in a conversation that I was feeling drained in a way I've come to associate with my position in the local web of Berkeley rationalists (via previous Looking). So, I Looked at the role I was falling into and what the local web was doing immediately around me, and noticed a particular way of thinking that my "character" was donning in order to fit in that was fitting with getting fatigued. I could also See how I was pouring… hmm, I just want to say "energy" here, but please understand I mean it zero out of ten as something woo. I was pouring energy into a particular role, and I Saw how to redirect that energy to a different role that would just work as long as the web didn't… ripple too much in response to my trying.

Okay, so now for my attempt at I-think-you'd-misinterpret-as-positive negative examples:

  • Againstness-type counters is a general category. More concretely, a few days ago I was practicing kung fu in a park, and a homeless man came up to me and started talking about his past from decades ago as a drug runner for the mafia. I felt my body become a little more alert, like I might have to physically defend myself against this guy. But I thought about it and recognized that wasn't likely and that I didn't need to be quite that anxious about it, so I took a breath and relaxed my muscles and calmed down a bit. That didn't involve any particularly intentional application of Looking on my part.
  • I was once led through a meditation where I laid down, imagined a light coming out of my forehead, and then followed that light to travel to the future to get advice from the ideal ten-years-from-now version of myself. That felt pleasant, and it felt like my dreaming mind came up with interesting insights during the visualization. I also think this didn't involve basically any Looking.
  • Sometimes a TAP fires in the middle of a conversation and I'll pause and think "Wait, this feels like we're drifting away from the point." Like… about a week ago, I was in a discussion with a friend who was trying to think about some social modeling stuff, and after I'd given a model that X happens because of Y, he started trying to object based on the claim that people shouldn't do Y. That felt like the conversation had slipped sideways, so I popped meta and pointed out the conversation arc, and he agreed and we got back on track. That didn't involve any Looking from what I can tell.

¿Is good?

Replies from: Raemon
comment by Raemon · 2018-02-24T20:55:55.094Z · LW(p) · GW(p)

Yup, quite good. I'll have to think on it a bit but was exactly the sort of answer I was looking for.

I'd hadn't meant to force you do a more-cognitively-intense "model me modeling Looking" thing, and maybe an easier question might have been "what's something you think a past, less experienced version of you might have thought was looking" or "what's a common mistaken impression you might think people might make" or "when people ask you to explain Looking, what's your surprise-o-meter expecting to come across wrong?"

Like, in the kensho post it was clear that you were afraid of falling into the "I am looking higher on my screen" trap, so it seemed like you had some kind of notion of what that would non-metaphorically look like, which is what I was trying to get at.

Replies from: Valentine
comment by Valentine · 2018-02-24T23:49:26.781Z · LW(p) · GW(p)
Like, in the kensho post it was clear that you were afraid of falling into the "I am looking higher on my screen" trap, so it seemed like you had some kind of notion of what that would non-metaphorically look like, which is what I was trying to get at.

Oh! Oh jeez. That makes a lot of sense. I can give tons of examples of that! That's a very different thing in my mind.

Heh, although, I should warn that giving examples of this is prone to starting arguments. Just tag all of this as "Val's interpretations of the world" and we're good. :-)

So with that, here's a few:

  • For a few months before my kenshō, ialdabaoth kept telling me that I had a social strategy that was being really annoying to him, something something sexual competition something something. I kept listening to what he was saying and thinking carefully about it, and I tried to do focusing on it, but it felt weird and I kept thinking that he was probably wrong (but as a general policy I kept in mind that I might just be deluded). This contrasts with right after the kenshō: one of the first things I Looked at was my sexual strategy system. If I remember right, I laughed and said something like, "Oh, that poor Valentine creature! It's like a leg that twitches until it fucks!" I ended up apologizing to ialdabaoth because I could clearly See what he was talking about now. We've been great on that dimension ever since. But yeah, I think it'd be fair to say he was trying to get me to Look and I was doing something that seemed perfectly sensible to me in response to that, but it sure wasn't Looking.
  • Sometimes I try to convey something one could loosely tag as "sovereignty" but is really about Seeing one's own existence and what that implies. A downstream effect of it is that there's now a meaningful difference in my mind between a "decision" that's about navigating the social web, versus a dedication that will in fact not even flinch in the face of temptation. I totally used to conflate those two, and I now think that most folk around me do too most of the time. I end up saying "No, really, choose. It's okay." And what I get back is… someone trying to sound confident or assertive as they strongly say one option, but it's really obvious that they haven't done anything different internally and are going to keep doubting themselves.
  • There's a tendency in authentic relating practices, or in Circling, where folk will make eye contact and often end up holding it for long periods of time. Many, many times, I've seen people then try to don a "loving look". Sometimes this is sincere, but sometimes it's something that folk have picked up from the culture as "what ya do, ya know?" I and others who know how to See the relevant thing here sometimes try to point out to such folk that e.g. the point is to attend to their experience rather than to have an effect on the other person. Sometimes they adjust in a seemingly useful way… and sometimes they just switch the strategy they're using to come across well, seeming to think that they're following the instruction.

Hopefully that clarifies rather than confuses. It's just that… in my mind, things you're likely to see me doing that you'd mistake for Looking is a really different category from things that people are likely to mistake for Looking in themselves.

comment by Hazard · 2018-02-24T16:34:18.265Z · LW(p) · GW(p)

Just wanted to note that I really like the phrasing of your question and think it's a quality move for circumambulating Valentine's position. (I'm also very interested in the response)

Replies from: Raemon
comment by Raemon · 2018-02-24T20:56:03.867Z · LW(p) · GW(p)

thanks!

comment by Chris_Leong · 2018-02-26T12:06:03.272Z · LW(p) · GW(p)

I am the only person who doesn't really understand the Omega analogy?

I can see some similarities, as Newcomb's problem involves predictions, but here Omega isn't trying to nudge you to do what it predicts you will do, but instead tries to make sure that you benefit from one-boxing, instead of two boxing. Is the similarity any greater than Omega being really good at predictions in both cases.

Replies from: None
comment by [deleted] · 2018-03-16T04:29:14.623Z · LW(p) · GW(p)

Omega also feels out-of-place to me. While it feels like the Web exists as a Thing with properties which can lead attitudes / etc. to propagate, painting it as the archetypal omniscient alien we've come to know doesn't really fit.

comment by lionhearted (Sebastian Marshall) (lionhearted) · 2018-02-25T02:11:09.642Z · LW(p) · GW(p)

Could I offer a thought on style?

I think this is a tremendous piece and there's at least 3-4 very remarkable insights in here... I wonder if it could be a transcendent piece with a bit more editing for tightness?

EX — if the piece had cut most of the preliminary and opened with "When you walk into an improv scene, you usually have no idea what role you’re playing. All you have is some initial prompt — something like:" — it would have grabbed attention faster. Then the core point from the improv scene could have been made 30% shorter/tighter before transitioning into the very powerful "I suspect that improv works because we’re doing something a lot like it pretty much all the time" with the family example.

After that, you explored from maybe a half-dozen angles... I wonder if perhaps instead choosing 3 tight, fast-written examples would have made the piece more lucid and more accessible to general readers?

When I run a word count, you're around 3600 words — now, you could fairly call me a hypocrite, since my essays regularly clock in around 4000-6000 and I'd be be a better writer if I spent more time editing down to 30-50% less words.

The two main reasons I don't edit enough — (1) perception of lack of time, (2) not wanting to delete correct-but-less-striking-true-stuff.

If it's #1, editing is of course taxing and time-consuming. I understand. It's the main reason why I don't edit enough.

But I find #2 to be pernicious and I catch myself doing it all the time. In writing, generally speaking, readers seem to average the quality of a piece rather than sum it. If you have three "A+" exceptional level insights/anecdotes, and mix in a few "B" level ideas and even the occasional "C+"... often readers seem to feel like, "Oh, that was pretty good" instead of "Wow, my mind is blown."

If you're writing just for the LW audience, people here have unusually long attention spans for engaging with complex pieces with different premises and branches of thought... certainly though, I think the tightest-edited version of this could help a lot of people beyond the LW community and I wonder there might be a more accessible version that led with the most striking points and moved maybe 25% faster through each of them?

Just a thought — I wouldn't have commented as such if I didn't feel like you were doing something absolutely marvelous here.

comment by Emiya (andrea-mulazzani) · 2021-02-16T12:11:57.609Z · LW(p) · GW(p)

Characters often want change as part of their role. And just as importantly, their role often requires that they can't achieve that change. The tension between craving and deprivation gives birth to the character's dramatic raison d'être. The "wife" can't be as clingy and anxious if the "husband" opens up, so "she" enacts behavior that "she" knows will make "him" close down. "She" can't really choose to change this because "her" thwarted desire for change is part of "her" role.

I'm conflicted about drawing this kind of conclusions from people behaviour, it opens up a door that allows you to interpret anything any way you like. 

More simple explanations are that if a "wife" knows how to interact with the husband in a way that causes him to open up and talk about what's happening, then the conflict gets resolved and you aren't observing a clingy and anxious "wife" anymore. 

It's actually hard to communicate openness and communication while you are feeling anxious and clinging, so you'd see a lot of people acting in ways that "discharge" their anxiety, rather than fix their problem. You don't need to go as far as to postulate that they are actually acting like this "on purpose".

Even if the "wife" is clearly showing a stereotypical script, it might just be that "she" has no utter clue of what else could be done about "her" situation. "She" could be just assuming that it's the correct way to face the problem, nag the "husband" until it finally works. Yeah, "she" would likely feel nervous and lost if considering the option of going off script and trying something else, and would avoid doing that because of that. But people have been using "punishments" in contexts where they have no hopes to work for countless millennia now, and there's no reason to assume everyone just secretly wants the target to persist in unwanted behaviour so they can punish him some more.  

There are other circumstances where drawing simpler explanations is harder, and then you can start to wonder if there is this kind of "purpose" in someone's actions. Self sabotage is definitely a real thing, sometimes. But I think you'd be safer by going with the simplest explanation first, because you can use "secret reasons" as an explanation for everything in psychology.

 

Aside from this, the post was really good and insightful. It got me thinking about what roles I'm being pushed on and where I'm pushing my friends to. 

I often see that people I know make assumptions about me being the rational one of the group, such as assuming I'd commit the stereotypical mistakes of someone who follows Hollywood rationality... which I always found weird as hell, because 1) in other contexts it's basically a meme that I'm really genre-savvy (for example, I DM in games for the group and people have a habit of worrying at least about the first four-five levels of subversions and recursions of my twists and plots), and so I thought they should realise I'd have saw the possibility of making the obvious cliched mistake coming, and 2) because I never showed any hint of such behaviours and regularly do the opposite thing, but I guess it makes more sense now.

My role, according to them, is to be incredibly devious and intelligent and do the non-supervillain equivalent of having the hero fall in my devious-four-levels-of-deception-trap, and then screw up something obvious such as leaving him unattended to free himself or fail to my own hubris or insert cliched genius mistake x, so the "balance" between intelligence and heart is reaffirmed.

comment by Vanessa Kosoy (vanessa-kosoy) · 2019-11-27T19:38:29.193Z · LW(p) · GW(p)

I think this post introduces a useful concept / way of thinking that I kept applying in my own life since reading it and that helped me understanding and dealing with certain social situations.

comment by Zack_M_Davis · 2019-11-25T02:09:48.120Z · LW(p) · GW(p)

In most cases, I don't think this is malice. It's just that they need the scene to work.

"IT IS ALL AN ACT. ASK FOR DETAILS." [LW · GW]

comment by MalcolmOcean (malcolmocean) · 2018-02-25T06:20:01.775Z · LW(p) · GW(p)

I'm really excited about this post on a whole bunch of levels.

One post on my list of posts to write is called something like "Everything is Improv", and I feel like you captured a decent fraction of what I want to say in that post, here! Plus a ton of additional pieces that I hadn't yet notice or connected yet. These two sections in particular felt very important:

"Another challenge here is that the part of us that feels like it’s thinking and talking is (usually) analogous to a character in an improv scene. The players know they’re in a scene, but the characters they’re playing don’t."

&

"But it’s hard to sort this out without just enacting our scripts. The version of you that would be thinking about it is your character, which (in this framework) can accurately understand its own role only if it has enough slack to become genre-savvy within the web; otherwise it just keeps playing out its role."

This means that certain acts of meta-communication almost become kind of like breaking the fourth wall. I've pointed at this in Acts of Speech & States of Mind:

At Upstart, we’re never just having conversations. We’re also training ourselves to think differently. We’re a theatre troupe that isn’t just performing but also practising, which involves having the skill and mutual trust so that at any moment any of us can pull out the director’s chair and say “cut” and we go meta and start talking about the way in which we were just talking. This is a key part of being able to help each other level up in this way.

But, as Val points out, it's really easy for this going-meta to just find its way back into the very dynamics that one is attempting to point at.

For example, have you ever tried pointing out that someone seems to be doing a social dominance move? In most contexts, that pointing-out action ends up itself being a social dominance move! Which isn't a problem, per se, but definitely makes it hard to shift out of that particular dance and into something else.

For that to work, you need a bunch of additional shared framework/context/intent around what the "something else" is and why you'd want it, as well as proficiency in a core ~applied mindfulness skill to avoid the temptation to continue playing out the current roles. I think Looking is one way of pointing at this skill. So an act of meta-communication is often attempting to say "Look, and become the actor, not just the character!"

This breaks character, and the fourth wall, which is a really awkward thing to do if other people aren't able to break out with you.

But with enough practice it's doable, even to the point of being able to pretty consistently act consciously rather than just playing one's character, and to do this with other people in a way that allows rewriting social scripts. Not easy, but learnable.

comment by CuriousMeta · 2019-12-11T03:09:21.014Z · LW(p) · GW(p)

Powerful improv metaphor. Powerful post.

Ah, but if we’re immersed in a culture where status and belonging are tied to changing our minds, and we can signal that we’re open to updating our beliefs, then we’re good… as long as we know Goodhart’s Demon isn’t lurking in the shadows of our minds here. But surely it’s okay, right? After all, we’re smart and we know Bayesian math, and we care about truth! What could possibly go wrong? [? · GW]

The trickiness of roles that involve the disidentification with specific roles or the concept of roles in general must not be underestimated. That's especially true for roles that seem to be opposed to the prevalent social structure.

I'm also reminded of Transactional Analysis. In particular, Games and Life Scripts.

comment by dsatan · 2018-02-23T12:14:03.773Z · LW(p) · GW(p)

Supposing I know how to Look, when and where do I Look, and what might I see? For my own purposes, I want to get a better idea of what my own role Looks like, see where I can move within the slack that I have, and see where tension in the web is coming from so that I can create more slack if needed.

Replies from: Valentine
comment by Valentine · 2018-02-23T20:04:19.690Z · LW(p) · GW(p)

Yep. From the OP:

But if you can: try Looking in the present moment at your own sense of not-knowing, notice that the same thing is alive in others, and watch as the story arises and plays out across all of you.

You're watching for the way that stories arise and play out, how people are falling into characters, the way that justifications arise in you, etc.

And… you're noticing the gap that's between all of them. The sort of canvas on which it's all written.

This can't be analytical, since that's within the character part of you. You have to watch your character and the whole scene from the transcendental not-knowing space that can never be tangled up in the web.

After a while, I claim, you can sort of "get" what the underlying patterns are, and how to act on them from outside your character. And how to lean into your character in order to produce the right effects on the web strands around you. And where you, the player, have room to reach for and tug on a different role for yourself. And what the consequences are for the web as a whole.

But in the meantime, you're just Looking at how the scene plays out, in detail, from the not-knowing.

Does that help?

Replies from: vanessa-kosoy, dsatan
comment by Vanessa Kosoy (vanessa-kosoy) · 2018-02-23T20:47:19.291Z · LW(p) · GW(p)

I found this essay insightful, but I am still confused by the concept of "Looking" (in the sense that, not only I don't understand the concept, which would be okay, I don't even understand the type signature of the concept). When you say "This can't be analytical... You have to watch... from the transcendental not-knowing space", do you mean that Looking is a mental motion that relies on mechanisms other than analytical thought (for example, on intuition), but which still happens within your biological brain, governed by the usual rules of cell membrane electrochemistry, and which in particular can theoretically be simulated by a computer program designed according to perfectly "analytical" principles, or, do you mean something which happens outside the laws of physics as we currently ("analytically") understand them?

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-23T21:44:26.607Z · LW(p) · GW(p)

The former.

Replies from: Valentine
comment by Valentine · 2018-02-23T22:20:13.865Z · LW(p) · GW(p)

Yep.

comment by dsatan · 2018-02-23T23:34:28.059Z · LW(p) · GW(p)

Ah, I remember having the distinct impression of drawing a blank when reading that sentence. Your further description helps, but it's feeling a little vague. I think I can kind of see it in my memory of past interactions, but I don't think I have enough of a handle on it that it seems like I could have done much differently. The referent of "not-knowing," and what it is not-knowing of is fuzzy/not so clear to me (though as I am writing all of this, it is becoming clearer and clearer). Is it not-knowing of how the scene will unfold? Of what the scene is? Of the roles we are playing? All of the above?

This does give me something that I can pay attention for in future social situations though and I'm pretty sure I can discover how to look at these things now, especially to see if I can achieve any of this:

After a while, I claim, you can sort of "get" what the underlying patterns are, and how to act on them from outside your character. And how to lean into your character in order to produce the right effects on the web strands around you. And where you, the player, have room to reach for and tug on a different role for yourself. And what the consequences are for the web as a whole.
Replies from: dsatan, Valentine
comment by dsatan · 2018-02-25T06:58:02.489Z · LW(p) · GW(p)

Pretty sure I have a handle on it now.

comment by Valentine · 2018-02-24T20:30:51.281Z · LW(p) · GW(p)
Ah, I remember having the distinct impression of drawing a blank when reading that sentence. Your further description helps, but it's feeling a little vague. I think I can kind of see it in my memory of past interactions, but I don't think I have enough of a handle on it that it seems like I could have done much differently. The referent of "not-knowing," and what it is not-knowing of is fuzzy/not so clear to me (though as I am writing all of this, it is becoming clearer and clearer). Is it not-knowing of how the scene will unfold? Of what the scene is? Of the roles we are playing? All of the above?

I find myself wanting to do the annoying zen thing of answering "mu". I think that's the most accurate answer.

I'll try saying more words, though, to offer an illusion of it being more satisfying:

You're trying really hard to understand what it is that you're not knowing. Or rather, your character is trying really hard at this. Whether it succeeds by its own standards or not is totally irrelevant.

If you Look, honestly, at what it is that you do not know, you'll See what I mean. I claim.

comment by Thane Ruthenis · 2022-10-19T14:55:15.535Z · LW(p) · GW(p)

Fascinating essay. It put into words a lot of inferences I've been independently making, while also suggesting a fun angle (the distributed social computations) to look at it from.

I suspect the framework is less fake in certain aspects than might seem at first glance, too! As in, it actually corresponds to some mechanical realities of how human minds are implemented [LW · GW]. In terms of the post I've linked, you could view "stopping" as momentarily ignoring the self-model you've derived and looking at the actual shards that implement your values (and then re-compiling your identity to more closely resemble your "true desires"). Or maybe even going further, trying to screen off the influence of the shards as well, and attempting to look at the world and yourself through the lens of a personality-less general-purpose optimization process (which is mainly useful when there's something gravely important you want to achieve, and you don't want to be biased by the flaws your role assigns you).

comment by Decius · 2020-01-09T12:19:12.956Z · LW(p) · GW(p)

The subtext is only clear in retrospect.

comment by Kaj_Sotala · 2019-11-21T14:07:37.789Z · LW(p) · GW(p)

This post gave me an important set of intuitions and things to be on a watch for. What stands out the most clearly in my mind are:

  • The notion of the social web acting as an agent in its own right, and being on a watchout for things which might violate existing scripts
  • The thing where, after your role is explained to you, you try to act based on that new information... while still continuing to re-enact your same role, as you don't really know how else to act.

I do think that these things were not explained with as much rigor as might have been good: but then, explaining these things in a really rigorous way is hard, and in the absence of a more legible model [LW · GW] it can be better to just point at your observations [LW · GW] and hope that some of your readers will try the things on [LW · GW].

I do think that much of what Valentine described in this post can be explained more rigorously in subagent terms [? · GW]. For example, the notion of scripts which people keep executing, in my view corresponds to subconscious schemas [LW · GW], some of which filter out all incompatible information [LW(p) · GW(p)]. I have a lot to say here, but it takes a while to work my way up there; but the fact that I even realized that I should be looking in that direction, was in part due to this post pointing me the way.

comment by FireItself · 2018-02-28T02:14:02.299Z · LW(p) · GW(p)

Love this post. As I was thinking about your Intelligent Social Web, it occurred to me that all this character-playing is serving an important role, it is adding value or it would have died out ages ago. In a small ancestral tribe, it is easy to see how this kind of web-force is keeping the whole tribe operating smoothly.

I've a question about times when we are called upon to clearly play a limited role, such as small talk. I really find it unsatisfying and dislike it. I'm curious if/how your relation to small talk has changed after you learned to See the social web. I am asking this because through your post I came to suspect maybe my dislike of it is misplaced, since I am not properly valuing its possibly important role.

comment by Elo · 2018-02-28T01:06:41.169Z · LW(p) · GW(p)

If you move away and then make new friends and sort of become a new person (!), you might at first think this is just who you are now. But then you visit your parents… and suddenly you feel and act a lot like you did before you moved away. You might even try to hold onto this “new you” with them… and they might respond to what they see as strange behavior by trying to nudge you into acting “normal”:

worth tying this into the subject object transition of the kegan stages of development. See here - meaningness.wordpress.com/2015/10/12/developing-ethical-social-and-cognitive-competence/ for a summary.

We become different people as we grow. We can hold different personas for different situations and experiences.

comment by ChristianKl · 2018-02-24T07:56:22.231Z · LW(p) · GW(p)
Actually thinking in a way that for real changes your mind in ways that defy your web-given role is socially deviant, and therefore personally dangerous, and therefore something you’re motivated not to learn how to do.

When reading that, I think it's a good reason why Quantified Self is so hard. Looking at data and making real changes about yourself based on it is emotionally taxing.

comment by nBrown · 2018-02-22T23:26:35.128Z · LW(p) · GW(p)

Yay!

comment by phillipchaffee · 2020-05-04T23:24:45.643Z · LW(p) · GW(p)

I would be interested in seeing this expanded to talk about what the change you can create by stopping is.

My initial thought is that humans have to exist in one of these stories to participate in society, but stopping could allow you to realize you want to change stories completely.

Maybe it is also possible to dramatically influence the story you are already in, but this seems less likely as the other characters would have to accept that change.

A concrete example of this is giving up drinking. If you give up drinking you usually need new friends. You won't enjoy the story anymore because you aren't drinking, and your old friends won't want a downer like you around.

comment by Dr_Manhattan · 2018-05-22T13:09:38.949Z · LW(p) · GW(p)

This really made me think of Gandalf, as being a superb conductor/chef of the social web, based on very raw ingredients (Bilbo, Frodo, Aragorn).

comment by Jason Gross (jason-gross) · 2018-04-17T03:41:17.235Z · LW(p) · GW(p)

What is the internal experience of playing the role? Where does it come from? Is there even a coherent category of internal experience that lines up with this, or is it a pattern that shows up only in aggregate?

[The rest of this comment is mostly me musing.] For example, when people in a room laugh or smile, I frequently find myself laughing or smiling with them. I have yet to find a consistent precursor to this action; sometimes it feels forced and a bit shaky, like I'm insecure and fear a certain impact or perception of me. But often it's not that, and it seems to just be automatic, in the way that yawns are contageous. It seems to me like creepiness might work the same way; I see people subtly cringe, and I mimic that, and then when someone mentions that person, I subtly cringe, and the experience of cringing like that is the experience of having a felt sense that this person is creepy. I'm curious about other instances, and what the internal experience is in those, and if there's a pattern to them.

comment by Valentine · 2018-02-22T19:08:00.522Z · LW(p) · GW(p)

Actually, update: Apparently I can't link to the old Less Wrong wiki. There's some kind of automatic script that's messing up the URLs. E.g., the very last link is intended to go here:

https://wiki.lesswrong.com/wiki/Rationalization

…but instead tries to go here:

https://www.lesserwrong.com/%3Cem%3E/wiki%3C/em%3E.lesswrong.com/wiki/Rationalization

…and then complains that that place doesn't exist.

So, apologies for some of the links being broken.

Replies from: Benito
comment by Ben Pace (Benito) · 2018-02-22T19:14:36.296Z · LW(p) · GW(p)

Sorry about that. Does it still mess up if you get rid of the 'https://' ?

Replies from: Elo, Valentine
comment by Elo · 2018-02-22T19:31:34.850Z · LW(p) · GW(p)

I have also had some linking problems of a similar style.

comment by Valentine · 2018-02-22T19:22:39.555Z · LW(p) · GW(p)

Yep, looks like.

I also notice that another link to the old LW wiki does not go wonky this way. I'm not sure why the system is particular about linking to rationalization! :-P

Replies from: habryka4
comment by habryka (habryka4) · 2018-02-22T21:07:52.073Z · LW(p) · GW(p)

Ah, sorry. I figured it out.

There is a bug right now where links sometimes do weird things when you use them simultaneously with other styling. I.e. bold or underline or italic. And you had a link that had some italic text in the middle of it, and so that had some unintended effects.

I just removed the italic for now, and am looking into how we can fix the general problem. Happy to delete this whole thread if we don't want it to take up a bunch of unnecessary space in the discussions section.

Replies from: Valentine
comment by Valentine · 2018-02-22T23:27:18.836Z · LW(p) · GW(p)

Woohoo! Thanks!

Deleting this thread seems fine by me.

comment by Gemist · 2024-01-27T17:41:06.880Z · LW(p) · GW(p)

maybe she wants to say that she is a stoic representative, and a stoic girl does not need flowers and sex?