Shoulder Advisors 101

post by Duncan_Sabien · 2021-10-09T05:30:57.372Z · LW · GW · 102 comments

Contents

  Why would I want this?
  Selection criteria: emulability and usefulness
  Improving the effectiveness of the council
  Downloading yourself
  Recap & Conclusion
None
102 comments

Motivation for post: As a former CFAR instructor, longtime teacher, and rationality pundit, I find myself giving lots of advice in lots of different contexts.  I also try to check in from time to time to find out which bits of advice actually proved helpful to people.  Over the years, I've heard from a genuinely surprising number of people that my (offhand, very basic, not especially insightful) thoughts on "shoulder advisors" were quite useful to them, and remained useful over time.  So: a primer.


"There's a copy of me inside your head?" Hermione asked.

"Of course there is!" Harry said. The boy suddenly looked a bit more vulnerable. "You mean there isn't a copy of me living in your head?"

There was, she realized; and not only that, it talked in Harry's exact voice.

"It's rather unnerving now that I think about it," said Hermione. "I do have a copy of you living in my head. It's talking to me right now using your voice, arguing how this is perfectly normal."

"Good," Harry said seriously. "I mean, I don't see how people could be friends without that."


The term "shoulder advisor" comes from the cartoon trope of a character attempting to make a decision while a tiny angel whispers in one ear and a tiny devil whispers in the other.


Many people have multiple shoulder advisors.  Some, no doubt, carry a literal metaphorical angel and devil around with them.  Others may sometimes hear the whispers of some of their favorite beloved fictional characters.  It's quite common in my experience for people to have shoulder copies of their parents, or their best friends, or their romantic partners, or particularly impactful teachers or bosses or mentors.

This is not schizophrenia (though for all I know it may use some of the same hardware, or may be a low-key, non-pathological version of schizophrenia in the same way that a healthy self-preservation instinct could be thought of as a low-key, non-pathological version of a phobia or an anxiety disorder). 

Rather, there is simply some kind of subroutine in the brain of most humans that is capable of taking in training data and learning what a given person (or character, or archetype) would say, in a given situation. It's predictive software, likely evolved in response to the need to model other chimps in the ancestral environment, and strongly selected for due to the fact that being able to model those other chimps accurately generally paid off big.

It's important to be clear that the experience of "hearing the voices" actually happens, in many people. This is not a metaphor, and it is not hyperbole or exaggeration. I'm not saying that people tend to hallucinate actual sounds—that probably would be schizophrenia.  But in the same way that most people "hear" their own thoughts, people also "hear" the voice of their dad (or "see" his facial expression), offering thoughts or advice or reacting in real time to the current situation.

"I was going to complain about having to type with my thumbs to text you, and how I'd rather just use email or Slack, but my shoulder Malo popped up to say 'Duncan, you have a Mac. Just use Messages with your keyboard.'"

"My mental copy of Jack is currently freaking out a bit about how toxic and unhealthy this sounds."

"I notice my inner Nate is betting this project will fail."

"I can hear my mom reminding me to take jam tarts when jam tarts are offered."

(Note that you don't need to "demand" that your advisor communicate in words! Often it's both easier and also just as useful to simply let them be present—to "see" their facial expressions and body language, imagine their nonverbal reactions, let yourself be aware of and attentive to them in the same way that you (likely) are aware of or attentive to other actual humans in the same room as you.  Think of how, for instance, someone at a party might say something that causes your eyes to dart over to a friend, to see their reaction—you can do the same thing with your simulated friend.)

If you already have this experience: you can curate and improve your council of shoulder advisors, and this post will give you some pointers on how.  If you do not already have this experience: you can most likely learn how to, if you want, and even a weak or limited or unreliable version of the skill has proven valuable for people.


Why would I want this?

In essence: good shoulder advisors allow you to be (at least marginally) smarter and more creative than you-by-yourself are capable of being.

I don't have a rigorous or technically valid explanation as to why, but it is a straightforwardly observable fact that, for many people, their shoulder advisors occasionally offer thoughts and insights that the people literally would not have thought of, otherwise.  Novel ideas, useful perspective shifts, apt criticisms of one's own actions or intentions, that sort of thing.  It's generally well-understood that "two heads are better than one," especially in times when one is stuck or uncertain, and shoulder advisors can be genuinely almost as good.

("One-point-seven heads are better than one.")

Having the right shoulder advisor "show up" at the right moment can be every bit as impactful as having an actual friend or mentor in the room.  And since shoulder advisors take up zero space and can be called upon at any hour and can include people you could never actually call upon in real life (such as Master Yoda or President Obama or Dwight K. Shrute or Mister Rogers or any number of Lannisters), even small improvements in:

... can be tremendously valuable.  My own cast of shoulder advisors have:

... not to mention that having robust copies of my actual friends and colleagues has much better equipped me to interact with those friends and colleagues, by giving me a head-start on how they'll respond to any number of things.


Selection criteria: emulability and usefulness

Step one, acquire shoulder advisors.  Step two, use them skillfully.

This section is for step one.  In order to use shoulder advisors, you have to have shoulder advisors, and whether you're building up a whole shoulder council for the first time or just trying to expand and curate an existing ensemble, some appointees are going to prove much more valuable than others.

Assume you had no preexisting council, and were brainstorming a list of possible advisors with the intent to winnow it down.  You might try writing down four or five names for each of the following categories:

Once in possession of a list of ~40 names, I claim the next step is to filter it based on the presence of two qualities: emulability and usefulness.

Emulability is the degree to which your brain can, or could likely learn to, successfully boot up a copy of this person and "just push play" on it, such that the copy in a sense "runs itself."  Authors sometimes talk about their characters "coming to life," and producing their own dialogue or wresting the story in an unexpected direction or even verbally arguing with the author inside their head—this is high emulability.  You want the sense that you're not making up or imagining what the person would say, via an act of explicit concentration, but rather that it's just auto-completing in the same way that a catch phrase or advertising slogan auto-completes.

In practice, emulability is often immediately obvious; you can just pluck a name off the list, imagine them sitting beside you (or reading over your shoulder, or lounging on the other side of the room) and just see how they react to what's happening to you right this second, and the claims that they hear me making.

(This is what happens to Hermione above, as soon as she bothers to check.  If attempting to bring someone into your current physical surroundings doesn't work, you could also try imagining specific scenarios, like throwing a water balloon at someone or showing up late to a thing, and see if your shoulder candidate has a characteristic response.)

In the event that this kind of imagination is not yet easy for you, though, there are a couple of qualities you can use to assess the emulation potential of a given shoulder-person, before putting in a bunch of effort.

The first of these is total training data.  People you've interacted with 100x more than average will tend to be more emulable just because you've absorbed more instances of "X happened, and they responded with Y."

(Note that as far as your brain is concerned, it makes zero difference whether the person under observation is real or fictional.  I've seen more of Miles Vorkosigan's reactions to a wide variety of stimuli than I have of many of my actual coworkers.)

The second major component is something like uniqueness or quirkiness or internal consistency.  If someone has a very specific vibe, it's easy to vividly imagine their particular responses.  Ditto if someone has strong opinions, or narrow special interests.

("I saw this video of a rocket launch and immediately thought of you, but then I got this mental image of your face looking very unimpressed, actually, and I genuinely wasn't sure why.  What does real-you have to say?")

Boring(-to-you), quiet, unopinionated, and "normal" people are thus quite hard to emulate, but that's okay because even if you could emulate them, you wouldn't get much out of them most of the time.  You're looking for the kind of people who have the potential to change your course—to think of things you wouldn't, make suggestions that aren't obvious, say the things you need to hear.

Which brings us to our second major filter: usefulness.

When I ran through the brainstorming list above, pretending that I'd never had any shoulder advisors at all, I got about 40 names, and when I filtered for emulability, I had maybe a dozen left.

Predictably, on that list were "Mom," "Dad," and "Ender Wiggin."  But if I were actually creating a council of shoulder advisors from scratch, I wouldn't necessarily want Mom or Dad or Ender to be on it.  I grew up with all three of those people having a deep influence on me—their perspectives and philosophies are already largely baked into "my whole deal," and not the sort of thing I need help keeping in the forefront.

Similarly, I don't really need more Tyler Durden or Mad-Eye Moody; I think I'm doing pretty okay on cantankerous pessimism and niche charisma.

Instead, a far more interesting person to have on my shoulder is one who can remind me of virtues I don't have down pat.  One who can snap me out of my normal patterns, cause me to smack my own forehead and mutter a rueful "of course."

For me, that list looked more like my friend Matthew from high school, who is soft-spoken and charitable and the-sort-of-Christian-the-Jesus-depicted-in-the-Bible-would-actually-like, and Jean-Luc Picard of the starship Enterprise, and an old colleague from Seattle, and the comedian Dave Chappelle.  These people were not only emulable but also truly different from me, which meant that if I could successfully add them to my shoulders, they would have the potential to catch things my regular algorithms would miss.

By "usefulness," then, what I am trying to gesture at is "I suspect my life would benefit from small, well-timed injections of this person's way-of-being."  If you are (according to yourself) too timid and hesitant, then you might look for people who are avatars of boldness, or who tend to be encouraging and supportive and make you feel confident, or who are eccentric and surprising.  If you are (according to yourself) too reckless and unreliable, then you might benefit from shoulder advisors who are avatars of caution, or who tend to pipe up with nervous hesitations, or who are good at noticing the little details before they turn into big problems.

(And if you don't know what your flaws are, or how best to go about improving yourself according-to-your-own-values, then maybe you're looking for people who are generally insightful and clear, or who are good at turning uncertainty into concrete and actionable suggestions, or who are (perhaps) somewhat scathing and unafraid to utter harsh truths.)


Improving the effectiveness of the council

Taking as given that you have some number of shoulder advisors who are either active or who you intend to start consulting, what next?

The key value of a good shoulder advisor is that they say the thing you need to hear, at the moment you need to hear it.  It doesn't take much to tip a tough decision from one direction to the other, or to start (or break) an affective spiral or chain of if-then behaviors.  A shoulder advisor is a specific instantiation of the general wish "if only I'd thought of X before Y happened"—you're trying to make it more likely that you will, in fact, remember X, especially where X is something not particularly native to your current way of doing things.

Taking the second part first, there are two ways to make sure that you hear from your shoulder advisors at the critical moment:

... there's a little bit of magic in both of these; I'm more telling you where to put your effort and not how that effort should look.  A full attempt to lay out how to build habits-of-mind goes beyond the limits of this introductory primer.

By far, though, it's the second strategy that I and others have found disproportionately impactful.  Explicit, intentional checks can only ever cover a small fraction of the times when people could really use a little extra insight.

However, doing the explicit thing is a good way to bootstrap to the automatic version, especially if you set aside five minutes to do a one-time brainstorm on "when do I wish my shoulder advisors would show up?"  Note that you can make a limited commitment, and that almost any amount of explicit practice will pay off, on the margin—if it sounds like too much to do five checks a day for three months, try doing one check per day for one week (or whatever).

(As with exercise, the best plan is one you'll actually follow through on, not one which sounds virtuous and doesn't work out.  Also, for the record, that line was literally just delivered to me by my shoulder Eli Tyre.)

A couple of tips, as you explore this space:

Once you've got a cast of characters who are willing to show up at all (or at least one solid imaginary friend), then you can worry about nudging their contributions in an actually useful direction.

My favorite techniquelet here is to refer back to the source material.  It's amazing how quickly the human brain will update its model of another human, if you actually go back and check.

"Hey, Nate, I was wrestling with [decision] yesterday, and my shoulder Nate thought the key consideration was [blah]."

"Lol.  I mean, yeah, but actually there's a much more important consideration, which is [blah]."

(Yes, my shoulder Nate actually says the word "lol" out loud, like it rhymes with "doll."  He does this because the real Nate does this, and my brain recorded it.)

If at all possible (especially in the early days), get your real advisors to not only correct your shoulder advisor's core thoughts and ideas, but to flesh out why they think what they think, and where your shoulder copy went wrong/what it doesn't seem to understand.

If your shoulder advisor is fictional, this is somewhat harder to do, but a good substitute is to write down a draft of their first contribution, then review it a day or two later with a critical eye.  Even moreso than copies of real people, your fictional shoulder advisors are free to mutate in whatever direction is useful for you.

(One thing I've had fun with is pitting them against each other—not by simulating an argument directly, where I imagine two sides of a debate, but rather by having both of them fight to convince me, or by having each of them arguing their conflicting judgment of the situation.  Having an optimist and a naysayer is a pretty good dynamic, and it's not hard for most human brains to pattern-match what each of those would say next, to the other.)

Ultimately, the idea is to give regular feedback to whatever part of your brain is running the emulation.  Upvotes for what works and feels true, downvotes for what doesn't, but most importantly, more training data.  It's fine if your shoulder advisor gets frustrated and impatient as you ask it to say more and more words—let it be frustrated and impatient in whatever way is characteristic for that individual, and just keep recording.


Downloading yourself

Again, the above was more a set of trailheads or threads-to-pull; there's not really a standard canon of advice here yet.  Hopefully, it's enough to get people started (and hopefully readers will leave further tips and advice in the comments).

There was one last piece of the overall picture that I wanted to touch on, at least briefly, and it's this:

You, too, can be a shoulder advisor.

My friend Nate and I both live in each other's heads, and we both furthermore have a vested interest in our mental clone copy.  Nate wants my shoulder Nate to be as good of a Nate copy as it can be; I want the same for his mental Duncan.  In part, this is for weird TDT-esque considerations, but mostly, it's just because I like my friend Nate, and he's my friend at least in part because of the impact I have on him, and if he's got a copy of me on his shoulder I can go on having that impact even when I'm not actually in the room.

You can in fact deliberately install yourself in other people's heads, if they're at all inclined to let you; some of my best lectures while at CFAR included me doing exactly this.  The key, as with developing your own shoulder council, is to focus on making yourself emulable.  Making your outputs reliably generable from inputs, having a specific and legible style or vibe.  If you've only got an hour, this usually means being pretty blunt and repetitive and keeping things simple:

"... so the one question I want you to keep asking yourself is 'do you know what you are doing and why you are doing it?'"

[5 minutes pass]

"Say it in my voice, in your head: 'Do you know what you are doing and why you are doing it?'"

[10 minutes pass]

"And what would Duncan say, at this moment?  Can you picture his face? ... that's right, he'd say 'do you know what you are doing and why you are doing it?'"

[40 minutes pass]

"... months from now, you're going to be sitting in your room, tired and frustrated, and you're going to look up at the clock, and you're going to sigh, and then you're going to hear my voice in your ear, and it's going to say—"

By this point, I get a message roughly once a quarter, from former students or former workshop participants or people who saw me at a conference or talk, letting me know that their shoulder Duncan appeared for them in a pinch, and that they were (usually) quite glad that he did.

If you have more than an hour to interact with someone, you can be a bit less cheesy than the above example, and encourage the same sort of feedback loops I described earlier, from the other side—Nate, for instance, often asks for the specific wording of his shoulder advisor, if I can remember it, and remarks on that wording as if he were disagreeing with shoulder Nate in a casual conversation, correcting and improving it.  

(It's just such little mannerisms that allow a shoulder advisor to be "really real"—to bring it to life, give it a personality separate from, and not dependent on, your brain's main central personality.  Again, I don't have a sound explanation of the mechanics, but it works.)

You can often make this happen by simply asking your friend or colleague or coworker to predict what you'll say, in response to a given question or prompt—

(Asking them to predict is in general better than asking them to guess.)

—and as icing on the cake, this has the added benefit that, not only are they refining their specific model of shoulder-you, they're also secretly practicing the general skill of booting up a shoulder advisor at all.

Speaking of which ...


Recap & Conclusion

This section is left as an exercise for the reader—try booting up a shoulder Duncan and see what parting words he has to offer, before you (hopefully) leave a comment down below.  And if your shoulder Duncan doesn't have anything at all to offer, see if anyone else feels like chiming in.

102 comments

Comments sorted by top scores.

comment by maia · 2021-10-11T20:38:02.030Z · LW(p) · GW(p)

It's also important to avoid bad shoulder advisors. I've spent several years trying to reduce the influence of miniature copies of abusive family members on my thinking.

EDIT: The most effective counter I've found for this is to 1) Notice that the thought I just had is actually coming from a bad source, 2) Remind myself that that person wanted me to believe/act that way for selfish and narcissistic reasons, and I shouldn't take their advice for the same reason I wouldn't take moral suggestions from people who go around kicking puppies.

Replies from: Benito, Duncan_Sabien, quintin-pope
comment by Ben Pace (Benito) · 2021-10-11T20:58:41.991Z · LW(p) · GW(p)

This was a significant lesson I learned between the ages of 13 and 23. I have repeatedly removed bad advisors from my shoulder.

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-11T21:30:31.601Z · LW(p) · GW(p)

I attempted to add some thoughts on how I'd go about this, but I'd love to hear a primer on your general method.

Replies from: Kaj_Sotala, Benito
comment by Kaj_Sotala · 2021-10-12T17:16:53.155Z · LW(p) · GW(p)

A model that pops up in several places (e.g. this book, this paper) is that these kinds of shoulder advisors show up as a kind of a preventative measure. If there are real people who would criticize or berate you for doing specific things, then your brain learns to predict when they would do that, and starts creating that criticism internally. That way, the inner critic may prevent you from doing the thing and thus spare you from being punished by the external critic who's being modeled.

In that case, one approach is to simply try to talk to your inner critic and ask it what it's trying to achieve and what it's afraid would happen if it didn't say the things it did. Sometimes it may be possible to get it to notice that e.g. avoiding the abusive family member's judgment isn't very important anymore, because you're no longer living with that person, getting it to ease off.

comment by Ben Pace (Benito) · 2021-10-11T22:18:07.267Z · LW(p) · GW(p)

I've done it twice, very explicitly. (Probably more implicitly.)

Here's some thoughts, tuned specifically to the cases I had.

  • The individuals would bring very 'defensible' arguments forward. And they would stubbornly refuse to change their mind in response to my improved understanding of the world. I knew they'd never change my mind, so I was always stuck debating them, I could never move on.
  • They had a certain level of status in a community (e.g. one was a public figure who sells books and gives talks) that was not status I was giving them. It wasn't like a friend I could just stop being friends with, they would continue to 'be in the public environment'.
  • It took me a long while to go from "this person seems wrong and set in their ways around ideas and norms that I do not support" to "as best I can tell, in some important ways this person does not live out virtue and I do not want to consult them when I am trying to understand the world or take action". Their arguments were always very 'defensible' in the given social context. To a significant extent I had to give up on that social context, give up being interested in getting status in that hierarchy, in order to stop caring what they had to say on an issue.
  • I suspect it helps to have an alternative social context to positively move one's mind into. Instead of repeating to myself that I shouldn't listen to person X, it helps to positively encourage myself to engage with person Y or social environment A, that's different and that these individuals were not a part of.

After a while, my mind didn't bring them into the conversation, and I also changed the conversations I was having in my mind. Much better for it, very glad to "just not care" what they thought.

Replies from: Benito, matto
comment by Ben Pace (Benito) · 2021-10-11T22:38:21.935Z · LW(p) · GW(p)

Oh, I notice that I also have done this sort of thing with a bunch of recent tv/films/content. 

There's a habit of modern content that, when it gets politicized, will "mimic argument". It will pretend to show sincere dialogue and debate, but it will fully swing the deck against one side and in favor of the other, and straightforwardly imply that the other side is unethical.

I can watch political art that I disagree with, I can even put up with good art that has bad political art inside of it, but when it attempts to distort what good faith dialogue is in order to win an argument, I just turn it off. I don't want to simulate that character/perspective or have a dialogue with them/it in my head.[1]

I can immediately think of four times I've done this with shows/content I otherwise greatly enjoyed and admire. I just don't want to learn to simulate them.

———

[1] Writing this out, I realize it's straightforward darkside epistemology.

Replies from: Gunnar_Zarncke, AllAmericanBreakfast
comment by Gunnar_Zarncke · 2021-10-12T07:52:16.443Z · LW(p) · GW(p)

Maybe that is one way how entertainment manipulates public opinion: By creating memorable (=easily emulable) characters that become shoulder 'influencers' that promote the official narrative right in the heads of the populace. 

comment by AllAmericanBreakfast · 2021-10-12T19:28:06.438Z · LW(p) · GW(p)

I would love an example, though I realize there are several reasons you might not want to put one out there!

Replies from: Benito
comment by Ben Pace (Benito) · 2021-10-12T20:19:40.862Z · LW(p) · GW(p)

Okay, because you asked AllAmericanBreakfast. Though I am not likely to follow-up discuss the specifics of each. Recent examples include the last season of Brooklyn Nine Nine and Bo Burnham’s “Inside”.

Replies from: AllAmericanBreakfast
comment by matto · 2021-10-14T19:12:34.019Z · LW(p) · GW(p)

The "before" state you describe, where you find yourself having arguments with stubborn advisors who refuse to change their minds, reminds me strongly of rumination.

I say this because it's something I'm working on to get out of. I'll sometimes find myself engaged in a pretty adversarial discussion about what boils down to my boundaries and be unable to fall asleep for a few hours. And it's usually the same cast of characters. I've found that I can consciously jump out of it by reminding myself that I'm merely burning energy without changing reality in any way. But I usually have to do this a few times before the "bad advisor" finally quiets down.

Do bad shoulder advisors feel like rumination to you?

Replies from: Benito
comment by Ben Pace (Benito) · 2021-10-14T19:17:14.418Z · LW(p) · GW(p)

That sounds right.

comment by Duncan_Sabien · 2021-10-11T21:30:09.133Z · LW(p) · GW(p)

Strong agree (and strong upvote).

Some general categories of strategy for working on that:

  • Setting up a dueling shoulder who's specifically motivated to stand in your defense, or pick apart an irrational or abusive argument, or even just remind you to take a breath and broaden your focus.
  • Preparing a mantra-of-rebuttal, which could be directly addressed to the annoying advisor ("I do not have to listen to you") or could be more general-purpose ("I will not allow toxic people to live rent-free in my thoughts").
  • Using CBT-esque self-conditioning to simply cut the thoughts off, mid-stream, until your brain gets the point.
  • Doing some kind of internal double crux to find the nugget of truth or usefulness that you do reflexively believe the advisor has to offer (e.g. "What this ghost in my head is saying is wrong but at least it is worthwhile to remember that some people think this way" or "What this ghost in my head is saying is wrong but it does remind me to care about X"), and then whenever it pops up, thanking it for that one nugget and sort of firmly closing the door.
comment by Quintin Pope (quintin-pope) · 2021-10-11T22:05:04.488Z · LW(p) · GW(p)

Possibly just the act of installing supportive shoulder advisors would be helpful. The brain only has so much capacity for shoulder advisors, so earmarking some of that for positive advisors may “clog the channel” so to speak. Bear in mind that shoulder advisors can be more abstract than is discussed here. E.g., you could have a wordless, nameless shard of pure positivity and acceptance.

Also, I expect shoulder advisors have a global positivity parameter that you may be able to influence. When a bad advisor tries to say something bad, stop them and force them to say something good instead, while imagining that the advisor truly believes the good thing. If your shoulder advisor objects to this practice, “correct” their objection and imagine them encouraging you to “remove the maladaptive cognitive pattern my irrational and unwarranted hostility represents”, or something like that.

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-11T23:41:52.336Z · LW(p) · GW(p)

(I note a prediction that most people wouldn't actually be able to make a wordless/nameless shard of pure positivity and acceptance work, and wouldn't get much out of it if you did, but also I'd be stoked to hear someone's experience with one that did work.)

Replies from: quintin-pope
comment by Quintin Pope (quintin-pope) · 2021-10-12T00:55:40.021Z · LW(p) · GW(p)

It’s definitely possible, though perhaps shoulder advisor is the wrong phrase to use at that point. Maybe it would be better to describe such a practice as a nonverbal mental ritual, rather than using an “angenty” framing.

You picture an incredibly happy crystal that blazes with light and feelings of positivity and acceptance (for this step, it may be helpful to put a cartoonish smily face on the crystal or to imagine it dancing, hugging you, etc). Then let those feelings radiate out from the crystal and into you, until you primarily feel the emotion from yourself. Allow yourself to be happy for the crystal’s happiness. Your own mood should naturally reflect that of the crystal as you lean into emulating the crystal’s radiant positivity.

It may also help to picture the crystal as being delighted to share that happiness with you. In this framing, both you and the crystal are happy to share your own joy with the other. Alternate between you sharing happiness with the crystal and the crystal sharing happiness with, both delighted by the other’s joy.

Note that visualisations of the sun, moon, a star, a glowing cloud, etc also work well for this exercise. I find that picturing the light as an ever-shifting rainbow of colors helps add some texture and adds dynamism to the crystal’s emotions. I also have difficulty holding a static image in my head for a long time, and the rainbow effect helps with that.

comment by Quintin Pope (quintin-pope) · 2021-10-09T23:06:39.495Z · LW(p) · GW(p)

Thanks for this great post!

TL;DR for my own thoughts:

  • I speculate on why shoulder advisors are useful
    • Drawing from ensemble methods in machine learning
    • Drawing from predictive processing and overfitting
  • I discuss generating a common corpus of training data for useful shoulder advisors
  • I discuss a few archetypes for useful shoulder advisors and their uses

Ensembles

Shoulder advisors seem to mirror a common machine learning technique, ensembling, which combines multiple ML models to get better overall performance than any individual model can reach. E.g., an ensemble of ERNIE models holds the current first place on the GLUE leaderboard (a metric for evaluating the general capabilities of language models). Shoulder advisors let you sort of ensemble thoughts across different personalities. Ensemble approaches are most helpful when the ensembled population is diverse and each model tends to specialize in particular types of tasks. That matches your usefulness criteria fairly well.

Predictive processing

If we extend predictive processing theory to internal personality traits, then our own personalities are generated by a predictive process, presumably one that bootstraps itself by predicting behavior and emotional reactions from family and friends in childhood ("other-prediction") before specializing in predicting/generating our own thoughts and emotions ("self-prediction"). Under this view, we use broadly similar neural circuitry for self-prediction, other-prediction, and generating shoulder advisors. Presumably, these three processes share common features in the brain, but with their own sets of neurons that specialize in each.

We spend far more time on self-prediction than on other-prediction, and have far weaker external signals while doing so. It's possible that the neural circuits specializing in self-prediction "overfit" more strongly than the circuits specializing in other-prediction. If you're repeatedly taking counterproductive actions, the negative reward signal from doing so may not be enough to push the overfit circuits out of predicting/generating that behavior.

Generating shoulder advisers, as a intermediate between self and other prediction, may counteract such overfitting by prompting your self-prediction neurons to interact more strongly with your other-prediction neurons. This allows the more general features and patterns learned by the other-prediction neurons to more easily feed into your own behavior and allows you to more easily pick up useful strategies and drop maladaptive behavior. In this view, it may be useful to continually rotate shoulder advisors, so that your self-prediction circuits receive constantly evolving feedback from your other-prediction circuits.

Training corpus

If shoulder advisors are beneficial, we should systematically aim to further improve their quality and the ease of generating them. One option would be to create a set of shoulder advisors whose personalities and mannerisms are optimized to fulfill common needs. Then, we can compile a corpus of "training data" for each advisor, meaning text describing the advisor and showing their responses to a variety of situations. Then, a person who wants to "install" a particular advisor reads the training data while having their current instantiation of the advisor predict how they'd behave in each situation supplied by the training data. 

Archetypes 

Here are a few shoulder advisor personality archetypes and associated advisor uses we might consider:

  • "The friend"
    • Traits:
      • Friendly, kind, empathetic, warm
      • Supportive, encouraging
      • Calm, equanimitous, happy
      • A deep feeling of beneficence towards you
    • Uses 
      • Emotional wellbeing
      • Relaxation
      • Promoting interpersonal empathy
      • Promoting positive self-worth
  • "The rationalist"
    • Traits:
      • Brilliant, analytical, incisive
      • Curious, widely-read, interested in knowledge
      • Quick to change mind in response to evidence
      • Quick to acknowledge mistaken cognition without undue emotional complications
    • Uses:
      • Analyzing data, problem solving, coming up with and evaluating new ideas
      • Learning new things
      • Motivation to read scientific papers
      • Actually changing your mind, recognizing mistakes
  • "The socialite"
    • Traits:
      • Friendly, sociable, outgoing, chatty
      • Empathetic, interested in others' perspectives
    • Uses:
      • Navigating social situations
      • Overcoming social awkwardness/anxiety
  • "The determinator"
    • Traits:
      • Focused, determined
      • unstoppable, absolute
      • Immense pain tolerance, little concern for own suffering
    • Uses:
      • Motivation to exercise/do chores/work
      • Pushing though unpleasantness, dealing with hardship
comment by avacad0 · 2021-10-12T06:20:46.938Z · LW(p) · GW(p)

Just a little bit of context here. I tend to be the kind of person who is generally able to cope DURING an traumatic event but then it may or may not take an emotional toll after the fact. That's context. 
Post- COVID American 6 month lock down (that's what it was for me), during 2021, I became majorly depressed and basically.... well, i guess the word would be ""broke". the best way my phsycologist and i can figure out how to explain it was that my personality fractured. not in terms of voices in my head exactly but i had a sort of ego death where i dissolved into a personal set of shoulder Advisors, demons, and other assorted folk who instead of talking to me were talking to each other. there was no me to talk to. but in more tangible terms my head couldnt cope with itself and my personality just became a set of shoulder personas.

The way i see this article, the shoulder advisor thing works because on a meta level you are in fact that smart. you can make better decisions/ notice more things/ do more things if you have a "trigger" that lets you bring that pattern into a higher level, more in control space. people are relatable in the sense that we relate to people pretty well and if we think about a certain person in conjunction with a certain trait we can capitalize that and emulate our "copy of them inside us.

But after my episode ive learnt that shoulder advisors dont have to be people. my personality didnt deconstruct into a linear set of voices; there were other things going on there as well. there was a lake, and a rock, and they participated in the "conversation" as much as any of the others. and i'm alot better now. but i still can use those voices when i need them. i can slip back into that space albeit with more control. people tell me that im very in touch with myself, almost too in touch with myself, and i think that's part of it.

apologies ahead of time if post is too personal; or too vague, or just oversharing and weird. it was good to write this out and i'd give it a coin flip that someone finds this helpful so here goes.

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-12T06:35:47.345Z · LW(p) · GW(p)

(I appreciate having it added to the conversation.)

comment by Ruby · 2021-10-11T19:41:13.667Z · LW(p) · GW(p)

Curated. This is a clear articulation of a rationality skill that I've never seen or heard explained elsewhere. Someone I'm close to gained a shoulder advisor in the last few years that was a dramatic breakthrough for them, but I never thought of it as something accessible to most people, yet this post updates me that there's a lot of value here for many (perhaps myself included).

comment by Raemon · 2021-10-09T19:51:35.612Z · LW(p) · GW(p)

Two anecdotes:

...

I recall when I was 18 years old or so, and I'd been arguing with a very religious friend throughout high school. I would rehearse arguments with him in my head, preparing for the next time we'd meet and I'd tell him all the reasons I thought his beliefs didn't make sense.

And for the first couple years of this, in the arguments in my head, I'd always say things like "have you considered point X" and imaginary-friend would say "oh, man, you're right. I am wrong." But then, eventually, I hit points where I'd say "what about X?" and then my imaginary friend would say "so? X doesn't matter, because [counterargument]".

This was a neat thing to discover about my ability to model people. (It also was relevant to the entire "does God exist?" debate – an eventually cruxy point for me is that you totally can build up simulations of people in your head, and I'd expect that to be hard to distinguish from God speaking to you)

...

More recently, I received benefit from asking my own future self for advice. (In fact, I asked multiple future selves who might evolve in different directions). One future self responded with some concrete, compassionate advice about how one of my coping mechanisms wasn't actually helping with my core goals.

comment by Kaj_Sotala · 2021-10-09T14:55:20.490Z · LW(p) · GW(p)

Just to make explicit a connection that seems obvious to me but I'm not sure how obvious it is to others: the existence of this phenomena fits nicely together with a global workspace model of the mind [LW · GW], where the brain may spawn new subroutines that plug into the workspace [LW · GW] and then learns various rules for when to activate them [? · GW], as well as fine-tuning their properties when the system becomes aware of a mismatch [? · GW] between the model's predicted outcome and what-the-target-person would actually have said.

Replies from: Gunnar_Zarncke, MondSemmel
comment by Gunnar_Zarncke · 2021-10-09T15:58:23.883Z · LW(p) · GW(p)

If that is true that would also mean that the ego could be just one of these shoulder advisors - albeit a privileged one. 

Replies from: Korz
comment by Korz · 2021-10-13T19:31:31.256Z · LW(p) · GW(p)

I would guess that there is some additional machinery involved in the ego compared to shoulder advisors (this might not contradict your description of ego as privileged shoulder advisor), as tulpas seem to be quite related to shoulder advisors while being 'closer to ego' in some sense. 
Probably this distinction is an important reason why shoulder advisors seem much less problematic from the standpoint of mental health.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-13T20:45:26.382Z · LW(p) · GW(p)

What additional machinery do you have in mind or what else makes you think that?

Replies from: Korz
comment by Korz · 2021-10-14T19:59:53.824Z · LW(p) · GW(p)

My thoughts on this are mostly from introspection. When I try to imagine a shoulder advisor in comparison to my self (note that I do not have shoulder advisors currently), there seem to be some additional properties to my self which a should advisor would not have.

Trying to get at the differences, what comes up is:

  • bodily sensations and urges are 'directly fed into and fuel (/delegate vote power to)' my self, but not shoulder advisors
  • decisions on movement likewise are directly connected to myself, while shoulder advisors are only influencing my mental dialogue/perception
  • similarly with things like 'felt responsibility for actions', 'identity' etc.

I am not sure that 'additional machinery' is the right term for these differences. My impression is 'the ego is much more strongly connected and fused with these other parts'

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-14T20:38:43.609Z · LW(p) · GW(p)

You describe it as a matter of degree and I can disagree with that. 

comment by MondSemmel · 2021-10-17T09:49:14.780Z · LW(p) · GW(p)

Scott's book review of Origin Of Consciousness In The Breakdown Of The Bicameral Mind also seems related.

comment by Ben Pace (Benito) · 2021-10-11T22:07:30.973Z · LW(p) · GW(p)

Two shoulder advisors I have found helpful this year:

  • Daniel Filan: has a knack for asking 'trivial' or 'obvious' questions in really situations that end up really opening up or clarifying what's happening. (It's part of why he's a good podcast interviewer at AXRP.) I've started to anticipate in my conversations where Daniel would ask a question, and asked it myself, and gotten great results out of it.
  • Gimli, son of Glóin, from the Lord of the Rings: I read the trilogy for the first time in January, and in many situations where I've been tempted to have long drawn-out passive aggressive interactions with coworkers or wander into a pit of despair, I noticed what he'd do: he'd just get on with the work. This has been very helpful.
Replies from: mary-chernyshenko
comment by Mary Chernyshenko (mary-chernyshenko) · 2021-10-13T09:48:15.573Z · LW(p) · GW(p)

One of mine is Athos from 'Twenty years after'. I admire his dedication, plainness of speech, valor, level-headedness, ability to just not defend things he finds unworthy of defence even when it would be to his advantage, etc. At most, he "says" something like, "Shall we? Yes, we shall." - and this is enough.

comment by JenniferRM · 2021-10-13T23:44:46.861Z · LW(p) · GW(p)

I'm fascinating by the mechanisms here! You said:

This is not schizophrenia (though for all I know it may use some of the same hardware, or may be a low-key, non-pathological version of schizophrenia in the same way that a healthy self-preservation instinct could be thought of as a low-key, non-pathological version of a phobia or an anxiety disorder). 

This accords with my own sense. 

My current central model for schizophrenia starts with Sapoloski's evolutionary hypothesis (sorry for youtube, it works at 1.75X and I just rewatched all of it to make sure I'm not wasting too much time), which links with how cluster A personality disorders are heritable and occur more frequently in people with schizophrenic parents/cousins.  

On this model, a few of these genes make you a tribally useful shaman or successful leader, and get positive selection (hence why the individual alleles exist at non-trivial levels), but too many all at once in the same person can predispose one to fall into a dysfunctional mental attractor.  

Cluster A disorders (classicly, the 3 on the left in the diagram below) are then maybe "very mild 'schizophrenia', with enough perks to make up for the downsides" and the downside here is evolutionary... so presumably there are phenotypes that have some advantage in some environments? 

Basically, Sapolsky thinks these phenotype/niches are spiritual/cognitive roles common to hunter gatherer society (which become dysfunctional in excess).

My favorite is "Schizotypal" which has easily detected hallmarks of: eccentricity in speech and dress. Also, interestingly, in this study it fell out as the purest expression of the "A_s" heritability dimension (without much of the other dimensions).  Sauce.

So then my guess is that there are ways of using one's imagination, and eventually these tip over into clanging and auditory hallicinations of Satan telling you to kill yourself (or whatever).

I find it useful (thinking of hallucinations) to also think of afantasia and then generalize that into all the sensory domains. 

Can you "imagine" a taste? ...a kinesthetic/tactile feeling? ...a smell? ...a sound? For some of these things, can you not avoid imagining them, or are you stuck with them?

In each sensory domain the resolution could be high or low, in its own subspace or in the shared one where sense perceptions are, and so on with limits or powers... kinda like dreams (which are potentially a variation on all this)? 

I don't know that I can really imagine "the taste of cinnamon" exactly, but if I try to imagine it, a picture of cinnamon toast pops into my mind (with my childhood kitchen in the background) and my saliva glands sorta tingle and my lips purse as if in expectation of bitterness?

The key dimension here is maybe not the vividness dimension but more precisely: intrusiveness.

I personally mostly don't have intrusive stuff if I don't want it. This goes all the way to having an "inner monologue". I don't need one mostly that I can tell? If language is specifically useful, rubber ducking often helps, but I don't have to pretend (that I've noticed) that I am the duck, or I am the one talking to the duck, to get the benefit.

Sometimes I'll remember bad experiences as if in reverie, but that is pretty rare and almost entirely under my control. Sometimes I use my audio memory loop to remember or rehearse speech. Most of my thought thoughts are nonverbal (though often they have visual or kinesthetic components). Often my imagination is just silent, or has music playing.

The idea of hearing voices on a regular basis, with no off button, that were also unhelpfully critical initially seemed (once I realized that many people have this) "totally batshit fucking crazy" to me. I remember the moment I realized this might actually be common.

It happened after learning about people with afantasia, and how they sometimes report that "all that talk about pictures in the head was, I thought, just a metaphor".  That gave me sensitivity to "<X> is all just a metaphor" being a template used by people with psychologically false universal theories.

This was a lightning bolt realization I had when wandering through a bookstore grabbing random books and hit my third random self help book with some version of a bit of wisdom about how "we all call ourselves inadequate all the time in our head".  I rolled my eyes (again) at this (essentially false to my experience) metaphor... and then it struck me that maybe it IS NOT A METAPHOR for "so many people that this author thinks it is universal"?!?

After that point I started asking people about their inner "imagination in general" experiences... and I discovered seemingly highly normal people (often quite smart, and maybe with some shaman genes?) could imagine things so vividly (often sounds or sights) that it was indistinguishable to them from "real perception" except that they know it is fake because they can control it and no one else sees or hears it. It is just imagining in EITHER or BOTH of the private mental workspace or "the workspace where sensory data lands" to them.

But then ear worms could often not be banished by people. Like in general, intrusive ear worms seem to be quite common.

But supposing one's ear worms were very high quality, and had a mentally simulated source location in their visible space, and (like ear worms in general) you couldn't turn it off... then it could be annoying. A useful technique was to keep GOOD music available, not to stop having an earworm (an impossibility for some?) but rather to change the channel <3

This loops back, for me, to this assertion by you:

It's important to be clear that the experience of "hearing the voices" actually happens, in many people. This is not a metaphor, and it is not hyperbole or exaggeration. I'm not saying that people tend to hallucinate actual sounds—that probably would be schizophrenia.  But in the same way that most people "hear" their own thoughts, people also "hear" the voice of their dad (or "see" his facial expression), offering thoughts or advice or reacting in real time to the current situation.

Sapolsky's actual lecture on schizophrenia helps make clear that the full blown version of schizophrenia is just really bad, really complicated, and related to an objectively broken brain (I have re-watched the full video at 1.75X, and it is pretty good but it mostly is just linked here to give Sapolsky a second chance to talk about the subject with a different (and more boring) kind of nuance than the video I started with.)

However, breaking down the mechanisms, and the sensory imagination/hallucination/dream aspects... 

....I think the concepts "ego syntonic" and "adaptive" are very useful here.

I suspect that for tens of thousands of years, there were no books, and so things like griot songs and homeric poety were the best form of inter-generational idea transmission that existed. 

This plausibly was culturally useful enough to have evolutionary implications for making us "genetically better at culture"?

Thus, in general, I focus on adaptiveness (for what is good) and intrusiveness (for bad things that can not be controlled). Skilled controlled musical "hallucination" is probably not necessarily maladaptive.

What about other skilled controlled imagined material?

In "Personality Characteristics of Tulpamancers and Their Tulpas" they find some evidence that many pairs (mancer and mancee) reported mutual satisfaction. So that seems like maybe sometimes it isn't maladaptive? 

Or at least it is sometimes not ego dystonic?

This gets around to the point that Gunnar makes [LW(p) · GW(p)] which is that I think some people have ONE big privileged special Tulpa that they think "is themselves".

To go further, I think some people have a special Tulpa like this, which assumes and extends the assumption, in its imagined speech performance, that it is, in fact, "the person themself". 

With the "me tulpa" conceptually bound to the "actual me" symbol in an identity relationship, the tulpa's performance comes to be called "my inner monologue" and its speech (true, false, bullshit, funny, or whatever) is NOT treated as "merely an inner monologue from an imaginary friend, named after and inspired by my total self, whose shape and actions are generated according to a huge number of criteria, many of which have nothing to do with epistemology".

Instead the person just thinks, roughly "my inner monologue just IS my thought and so my inner monologue is the totality of my cognition". Maybe?

If this is true, then the whole Inner Simulator part of the CFAR curriculum is perhaps a way to get these people (without having to do an enormous extended explanation of what they might be routing around) to tap into the mental powers that are not normally recruited by their Self Tulpa, but which do exist in their brain...

...with the practical upshot of helpfully using parts of their brain that their Self Tulpa often does not regularly use (and which are perhaps less liable to the rounding errors and half-truths inherent in mere speech).

Ben Pace has an interesting sequence of comments [LW(p) · GW(p)] (click and read up and down (it is not just the one comment)) about removing shoulder advisors that are problematic. I'm not sure if it would be safe to apply Ben's techniques to the removal of the central shoulder adviser? Deciding that a Self Tulpa doesn't deserve social status (while at the same time not making clear that the real person does, because everyone deserves care, but also because the tulpa isn't the totality of the real person, and is just a tulpa, and so on) seems like it could have bad side effects if not done very very carefully.

I think Buddhism has techniques here... but also i think maybe some of these techniques were designed by psychos trying to herd a bunch of teenagers into submissive participation in armies as soldiers? So: approach with caution?

And in the meantime... maybe tulpas are great! Maybe the more tulpas the merrier?

comment by Kaj_Sotala · 2021-10-09T17:59:41.746Z · LW(p) · GW(p)

One practice I've been doing a bit since January [LW(p) · GW(p)] has been something called Ideal Parent Figure Protocol, which includes guided meditations (e.g.) for imagining yourself as a child with the kinds of idealized parents who are always perfectly supportive and understanding and available, to correct for any emotional lacks created by the ways in which your real parents were just human and non-perfect. One of the parts of the practice is something called "microhits", which basically means making your ideal parents your shoulder advisors so that they'll be available for emotional support whenever you need it. (I haven't gotten this very strongly, but I've heard people say it's really powerful if you do get it to work.)

comment by Luke Sallmen (luke-sallmen) · 2021-10-13T04:05:05.981Z · LW(p) · GW(p)

Two additional beneficial outcomes of attempting to boot up / improve shoulder advisors:

  1. You are more likely to really pay attention during interactions with them. You get better at interactions in general if you are quite focused. And others tend to notice and react positively when they see that you really care about what they are saying.
  2. You learn to be more curious about others' thoughts, actions, and backgrounds. This can help you be more empathic, and can also help shine light on your own motivations and influences.

 

Anecdotal, but the friends of mine who simulate people / hear others' voices tend to be among the most thoughtful and socially-buttery-smooth people I know. 

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-13T05:16:58.502Z · LW(p) · GW(p)

True in my experience as well, and in the experience of at least a couple of the people I've talked explicitly to about this.

comment by Gunnar_Zarncke · 2021-10-09T18:42:00.044Z · LW(p) · GW(p)

For some time, I have tried to create mental models of other people to improve my social skills, better tune my communication, and predict reactions. It was always very difficult to put myself in other people's shoes. For a long time, I didn't understand what that even was supposed to mean. Your instructions have been extremely helpful and I managed to bring up boot up (at this point one piped up and said "call it 'boot up'") some advisors easily.

It is easiest to let the advisors just be there and act without speaking. Smiling, noding, moving.

Emulating by ex-wife - at least her speech - is relatively easy. I'm not sure what to make of it; we are well cooperating co-parents, maybe there will be parenting advice that will simplify things.  

Among the advisors that I brought up the most vivid were instances of myself. The first tries were rather boring because the copies didn't speak up but just thought and observed. Only when I changed the frame to include thoughts did this lead to some cool interaction. The copy was disoriented at first, explored the environment, asked for access to senses, and brought up nested instances at which point I decided to write up the experience. 

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-09T18:57:59.190Z · LW(p) · GW(p)

Thanks very much for this comment.  I've added in a paragraph (ctrl+f "Note that you don't") to emphasize the point about not requiring them to speak.

comment by Gunnar_Zarncke · 2021-10-09T18:20:41.186Z · LW(p) · GW(p)

Many people have multiple shoulder advisors. ...  It's quite common in my experience for people to have shoulder copies of their parents, or their best friends, ...

Is this really the case? 

I recently discussed inner monologue with a friend a he was doubtful whether this was real or just something made up. He didn't doubt that people could imagine something like it but not as something in the way you describe here. I can relate to his view because I also don't have a persistent inner monologue - though I can bring it up. 

If you asked people neutrally what goes on in their mind I guess the results would look a lot like Galton's experiments about mental imagery (mentioned e.g. in the SCC post Generalizing From One Example [LW · GW]): Some would say nothing, some would answer like my friend, most would report some inner monologue, and a some would have a lot of inner voices. 

How solid is your evidence? 

 

Replies from: Duncan_Sabien, Kaj_Sotala
comment by Duncan_Sabien · 2021-10-09T19:06:59.392Z · LW(p) · GW(p)

It's definitely the case that many people do (and also definitely the case that many people don't!).

My evidence is:

  • My own personal experience
  • Explicit conversations about the shoulder advisor concept, including phenomenological reports of what it's like, with a large number of people including Eliezer Yudkowsky, Nate Soares, Damon Pourtahmaseb-Sasi, Renshin Lee, Eli Tyre, Logan Brienne Strohl, Malo Bourgon, Jack Carroll, and Eric Zolayvar.
  • Instances in fiction and pop culture (the angel-and-devil trope, HJPEV's cast of mental models in HPMOR, Bella seeing Edward in Twilight, a lot of references to "hearing someone else's voice at a critical moment" or "an image of someone's face popping to mind" in any number of works)
  • Some pop-evo-psych and bicameral mind stuff that sort of post-dicts it

I started developing (and sharing) my explicit concepts of shoulder advisors well after encountering the stuff about some people not having visual imagery, or some people not having inner monologues, or some people not having a persistent sense of self across time, or Scott Alexander's "Different Worlds," and so forth.  Like, from the start I've been quite aware of the fact that this won't be universal.  But in my probing, a) something like half the people I talk to resonate with this from the start/already have similar experiences or habits, and b) of the remaining half, a majority take to it pretty quickly/can pick up a nascent version of the skill in five minutes.

That, of course, in no way invalidates the possibly-as-many-as-25-percent-of-people whose minds are simply running a different OS.

Replies from: Gunnar_Zarncke, Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-10T19:22:36.403Z · LW(p) · GW(p)

Update: I polled some people and for what it's worth this is the result:

  • has dialogs and trialogues, that are different shards of their personality and chime in often unasked (but not like an advisor)
  • can easily emulate other people and how they will react and what they would say (it sounded more like in the form of conceptese)  
  • can easily visualize other people and what they would say (it sounded more like in a video recording with lower person fidelity)
  • has minimal inner monologue, but can bring up limited shoulder advisors (me)
  • has almost no inner monologue
  • has no inner monologue, aspects of their personality take control unasked and without negotiation 
  • has no inner monologue and is skeptical such a thing exists at all 

Kids' answers (ages 10 to 15):

  • I construct the other person from parts and can imagine what they would say but not how they would look.
  • I can easily imagine what other persons would say. 
  • The better I know another person the better I know what they would say or do (it sounded less like an imagined person and more like 'just knowing').
Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-10T19:37:00.530Z · LW(p) · GW(p)

Can you share something about the wording of your question/prompt?  I might want to ask other people using the same prompt for consistency.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-10T20:13:03.729Z · LW(p) · GW(p)

It happened relatively organically in a board-game round. Discussion moved to how to deal with politics at the job, and you have to anticipate people's reactions, and I said that it happens in management all the time. And I think I asked: "How do you do that?" and as a follow-up question: "Do you have some kind of inner monologue?"

With my kids, I didn't assume familiarity with such a concept and used a situation they are familiar with. A setup like: "You have a lot of friends and tell them a joke, can you tell how they will react? What happens in your brain/mind when you do that?" And as follow-ups: "Can you imagine your friend saying that?" "Can you hear what tone he uses or what emotions he shows?" (We have talked about mental imagery often before so I know they are familiar with that)

While it was not intentional with the adults I prefer this pattern. Kids (at least mine) often don't like direct questions but will happily tag along if my topic relates to their situation. It also avoids leading questions. And if I offer answer options I provide a wide variety of alternatives with cues that there could be more. 

So my proposal is to describe a situation where modeling another person is a natural choice. And then ask "How do you do that (in your mind)?" and only in follow-up questions ask for inner mono/dialogue or mental imagery.  

comment by Gunnar_Zarncke · 2021-10-09T19:34:22.836Z · LW(p) · GW(p)

Thank you for sharing your epistemic status. I think this and other questions regarding consciousness deserve more study. It is relevant for AI research, on LW see e.g. Nonperson predicates [LW · GW]. This was also mentioned by Sam Altman in the recent meetup. I intend to write a post on some testable predictions about consciousness and I hope to get some into the next SSC/ACX survey. 

comment by Kaj_Sotala · 2021-10-09T18:30:03.960Z · LW(p) · GW(p)

I mostly don't have inner monologue either, but I also don't feel like that's an requirement for having shoulder advisors. Shoulder advisors can talk in conceptese, in my experience.

(I don't have shoulder copies in the strong and vivid sense that Duncan describes, but I sometimes get vague feelings that seem like they are the outputs of the same kind of process. E.g. reading an article and thinking "I feel like Duncan would be interested in this", seeing a particular plush toy and thinking "my friend S would find this one adorable", or preparing for a fraught conversation with someone and doing a simulation in my head of what I'd expect them to say in response to various things that I could say. Also I'm getting a weak sense of what you might reply to this comment, now. :-)  )

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-09T19:05:45.342Z · LW(p) · GW(p)

Agree. Esp. the

Shoulder advisors can talk in conceptese, in my experience.

Which worked best for interacting with a copy of myself. See my other comment.

comment by hath · 2021-10-11T20:40:28.438Z · LW(p) · GW(p)

My own personal experience following this post: I don't have enough training data for most of the people I'd like to emulate. When I think of the people I know irl that is like to learn from, I've spent about ten hours 1-on-1 with each of them; not enough to have a solid mental model of what advice they may give. At the same time, part of why I value their advice is that I can't predict it; they have wisdom and experience that I don't. Often, I'll ask them for advice and be surprised by their answer. When I tried to create a shoulder advisor of one of them, it didn't work; I just didn't know enough about them to accurately understand what they were thinking in a certain situation.

Still a great post, though; just didn't work for a specific use case of mine.

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-11T21:33:20.881Z · LW(p) · GW(p)

That makes sense.

I (weakly) predict that building a shoulder advisor or two out of less-useful-but-more-emulable people might be worth it via giving you the skill of emulating to the available max?  Such that, finding emulation in general a little easier and more familiar, you might be able to try again with the actual higher-value targets?

FWIW, I have been genuinely surprised by advice from shoulder advisors that I could not predict; in a very real sense, that's the primary claim of the post.  You don't have to have the same wisdom and experience to spin up a mental chatbot that will sometimes be able to mimic the pattern well enough to produce something novel and useful.  If having a good shoulder advisor required being as wise and experienced as the real person, I don't think I would have felt this post was worth writing.

Replies from: hath
comment by hath · 2021-10-11T23:11:43.887Z · LW(p) · GW(p)

I (weakly) predict that building a shoulder advisor or two out of less-useful-but-more-emulable people might be worth it via giving you the skill of emulating to the available max? Such that, finding emulation in general a little easier and more familiar, you might be able to try again with the actual higher-value targets?

That makes sense, I'll give it a try.

FWIW, I have been genuinely surprised by advice from shoulder advisors that I could not predict;

Ah, I see.

comment by CitizenTen · 2021-10-10T02:46:39.582Z · LW(p) · GW(p)

On the having a shoulder Duncan, I explicitly tried to do this upon listening to you being interviewed by Spencer Greenberg.  Sadly, digital Duncan doesn't exist in large enough quantities to emulated.  So for those of us that don't happen to live (I assume) in the Bay Area, can Duncan increase how many podcast or video's he's on?  Having a shoulder Duncan sounds really useful to a large number of people you interact with and enlarging that pool purposely seems to be a pretty good idea. Food for thought?

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-10T03:22:31.454Z · LW(p) · GW(p)

My best talk

My longest talk

My Harvard talk

My badly-in-need-of-updating website which happens to have a "writing" tab F U L L of stuff

You can find a couple others on Youtube by searching "Duncan Sabien."

Also you can get people to invite me to podcasts and I'll often say yes. =P

comment by moridinamael · 2021-10-15T15:35:30.447Z · LW(p) · GW(p)

This phenomenon is also why we have the term "role model." Successful examples of people similar to us are extremely valuable, and it is in fact very difficult to succeed without such examples.

comment by bfinn · 2021-10-13T23:03:19.691Z · LW(p) · GW(p)

Great post. Two comments:

The popular saying "What would Jesus do?" suggests many devout Christians use Jesus precisely as a shoulder advisor - no doubt frequently & with intense seriousness. Hence they may well have useful insights into the technique.

Also:

It's important to be clear that the experience of "hearing the voices" actually happens, in many people. This is not a metaphor, and it is not hyperbole or exaggeration. I'm not saying that people tend to hallucinate actual sounds—that probably would be schizophrenia.

I read a book about Jaynes' bicameral mind theory which mentioned research in the 1980s into shoulder advisers (in effect) of institutionalised paraplegics. It seems many paraplegics, who couldn't move since birth, and I think couldn't speak either, had frequent auditory hallucinations of a person, often known to them (eg a parent), who would regularly talk to them, providing commentary, advice, comfort or criticism. They would hear this as a seemingly real, external voice (but not visible). Similar I assume to schizophrenics hearing voices. The book gave this as evidence of remnants of the bicameral mind in modern humans.

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-14T03:24:31.576Z · LW(p) · GW(p)

Yeah, the bicameral mind stuff was definitely in my thoughts as I explored this concept explicitly over the past few years.

comment by frankybegs · 2021-10-10T18:04:21.715Z · LW(p) · GW(p)

"it is a straightforwardly observable fact that, for many people, their shoulder advisors occasionally offer thoughts and insights that the people literally would not have thought of, otherwise."

How can this be observable, let alone straightforwardly?

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-10T19:40:52.373Z · LW(p) · GW(p)

+1 for noticing the sketchy wording; thanks for the chance to clarify.

I intended to convey that the fact is straightforwardly observable, not the phenomenon itself.  It's the sort of thing you can confirm is true by just going and asking a bunch of people, as I have done and as Gunnar reports doing in another comment.

You are correct that I can't literally observe this happening inside any heads other than my own (but it is indeed observable inside my own head, e.g. by adding a new shoulder advisor and suddenly and consistently getting a whole class of insights that I never got previously).

comment by romeostevensit · 2021-10-11T04:22:17.909Z · LW(p) · GW(p)

I think this is part of the reason for the popularity of the Meditations of Marcus Aurelius.

Replies from: mary-chernyshenko
comment by Mary Chernyshenko (mary-chernyshenko) · 2021-10-11T18:12:33.219Z · LW(p) · GW(p)

And just the good old absolutely trustworthy Roman army guy who does what he has to do, sings as he whittles his arrows, looking up to smile cheekily at his dejected friend and wiggles his sun-bleached eyebrows with an outrageous joke that makes one actually forget the reason they approached him.

Perhaps not the most helpful character, but.

comment by Gunnar_Zarncke · 2021-10-11T00:37:49.416Z · LW(p) · GW(p)

A related (ADDED: but more intrusive and risky) technique is called Tulpa and was discussed earlier in this post: How Effective are Tulpas? [LW · GW

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-11T01:38:46.002Z · LW(p) · GW(p)

I am comfortable with having a comment making the (objectively correct) claim that tulpas are related in a narrow and technical sense.  They seem to be playing in a similar space, are probably using the same mental architecture, etc.

I would not be comfortable with having a comment leaving the (unjustified, imo) impression that tulpas are substantively similar.

Like, it may be that tulpas get a bad rap, but from what I know of them, they're much more like inventing a shoulder advisor and then ceding control to it entirely because you think it can run your life better than the core you, and that's way more extreme and requires a lot more assumptions to justify than the thing I'm recommending with shoulder advisors.  Their common-use definition is a thing that feels risky in a way that shoulder advisors do not, and feels like it requires warnings that I don't think shoulder advisors require.

EDIT: Also, afaik tuplas are much more built-from-the-ground-up, rather than being keyed into a set of recorded experiences from either real people or detailed fictional characters.  Having to ground out a mental construct in either actual reality or plausible near-reality seems like a big safeguard.

Even if I'm wrong about what tulpas really are in practice: to the extent that my understanding and my brief description above matches other people's general impression, I want to be pretty firm that that thing is not closely related to shoulder advisors in spirit.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-11T07:49:59.720Z · LW(p) · GW(p)

Well, the way you want them to be is different and less risky but as you point out: They likely run on the same mental architecture. My shoulder advisor was emulated smart enough to ask for access to senses, which was fun to play around with and felt a bit like 'ceding control'. It is probably a good idea to make sure that you create them really as advisors otherwise the downvoted points [LW(p) · GW(p)] might apply.  

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-11T08:17:31.852Z · LW(p) · GW(p)

The connection to Tulpas is close enough that curious smart people will hit on them if you want it or not. For example, it was the second comment in Kajs reshare.

Replies from: Raven
comment by Raven · 2021-10-16T19:56:26.184Z · LW(p) · GW(p)

Seconding this, I noticed the parallels the moment OP started talking about advisors injecting comments.

Tulpas can be instantiated from fictional characters, and these are called fictives or soulbonds. And it's not about ceding control, I think that's more of a did thing where some alters will hide because they're traumatized and afraid.

I suspect that feeding an advisor attention (by talking to it a lot) will help it grow into a tulpa/alter. But I'm not sure, my advisors aren't even at the level described in the post. I have to invoke them deliberately, and they disintegrate as soon as I'm done talking. On the other hand, I don't spend hours conversing with them every day, which everyone seems to agree is part of instantiating a tulpa.

comment by dregntael · 2021-10-10T07:33:56.601Z · LW(p) · GW(p)

Thank you for writing this, it really got me thinking. I'm one of those people who don't really have a firm cast of shoulder advisors. In fact, when I saw this appear in fiction (and in particular in HPMOR), I kind of assumed it was just a convenient narrative device and not something real people actually do. I suppose I should read HPMOR again and try to figure out what other blatantly obvious advice I've missed.

This does seem like a extremely useful skill to have, so I'd like to practice it if possible. I just tried to imagine one of these shoulder advisors and I do get an image, but it is very blurry and seems to be "shifting" between different states. I wonder if this is something other people have also experienced, and whether there are any tricks to get a more stable image? Perhaps a fictional person would be easier to simulate since they tend to be not as complex as real human beings. Either way I will continue to try in the coming week and report back here if I get any results.

Replies from: Kaj_Sotala
comment by Kaj_Sotala · 2021-10-10T08:17:42.868Z · LW(p) · GW(p)

I think that getting a stable visual appearance isn't very important; I'm sure it can make things more realistic if you do get it, but the most essential thing is getting a hold of the person's felt sense [LW · GW], as well as a feeling of how they would react to different things. If you get a sense of "ah in this situation, they would say X", then it doesn't matter if you're unable to clearly visualize them.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-10T15:10:59.130Z · LW(p) · GW(p)

I can bring up the idea of a shoulder adviser easily. I can also situate it in the room. It has emotions. But it is not visual. I don't 'see' the emotions or the advisor. At best it is a sketch. That's normal for me, my visual imagination is 'sketchy' (see Generalizing From One Example [LW · GW]).

comment by Samuel Shadrach (samuel-shadrach) · 2021-10-09T14:57:29.033Z · LW(p) · GW(p)

Interesting concept but idk if this works for myself personally.

Hypotheses:

 - I don't know anyone deeply enough to predict their responses or perspectives with high accuracy

 - I don't care about anyone's perspectives on my decisions enough to want to model them (even though I admit they might be useful)

 - Modelling other people this way takes too much energy

Replies from: Gunnar_Zarncke, Duncan_Sabien, Raven
comment by Gunnar_Zarncke · 2021-10-09T16:00:33.565Z · LW(p) · GW(p)
  • Modelling other people this way takes too much energy

This sounds like you use another way of modeling people. Can you share some ideas about how? It might be interesting to compare the differences.  

Replies from: samuel-shadrach
comment by Samuel Shadrach (samuel-shadrach) · 2021-10-10T13:49:15.294Z · LW(p) · GW(p)

I'm honestly not sure.

I remember having full-blown conversations with people I'm close to in my head, but they feel more like a fear-driven response where I'm having to defend myself against them, rather than working with them on anything. Or maybe I'm just anxious and I'm working out the courage before I ask them irl (thing being asked could be very mundane, or could be important).

Another reason to model people would be to figure out who find annoying, interesting, worth befriending, etc. I feel like that happens very automatically - first I experience the annoyance or interesting thing in conversation irl, then I make a mental note of it. So the next time they come up in my mind, I can reason about such and such thing I found annoying, and what underlying personality trait or ideological difference it indicates. So I reason more in my own words describing them, rather than "their words".

I can't remember any other reason I would model someone.

If it helps, I'm 20 and have struggled with social interactions in the past, possibly I will just develop this skill over time.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-10T14:35:13.826Z · LW(p) · GW(p)

I remember having full-blown conversations with people I'm close to in my head, but they feel more like a fear-driven response where I'm having to defend myself against them

I wouldn't be surprised if this mode were the most frequent one. I remember more people mentioning dialogs like this than in the form of advisors. I also have memories close to this though the mental dialogues I went through were less wordy but more hypothetical arguments back and forth (I guess what Kaj called conceptese).

But it proves that your brain can do the emulation. If it works for conflict, it should also work for advice. Though, as you say, it may require practice. 

If it helps, I'm 20 and have struggled with social interactions in the past, possibly I will just develop this skill over time.

I can relate. I struggled too and avoided social except with close friends. Only much later, around 40, when I hit limits of what I could achieve as a single software developer, did I practice it more. Of course, it's more effort now, but on the other hand, I also have more mental tools that I can through at it. And more social capital to apply.

Replies from: samuel-shadrach
comment by Samuel Shadrach (samuel-shadrach) · 2021-10-10T16:29:36.864Z · LW(p) · GW(p)

Interesting. Did you have shoulder advisors when you were younger?

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-10T19:02:58.773Z · LW(p) · GW(p)

No, as I wrote elsewhere in this thread-forest, I don't have much inner monologue, no imaginary friend, little inner conflict in general (which I attribute to a sane childhood), and I think mostly in concepts.  

comment by Duncan_Sabien · 2021-10-09T17:56:53.166Z · LW(p) · GW(p)

I would be surprised-but-not-shocked if you can't make this work in a fundamental sense.  Human variation is actually pretty wide (e.g. diachronics vs. episodics, some people having no inner monologue, synesthesia, some people having no visual imagery), so there are no doubt many thousands of people out in the world that just do not have this capacity, but an overwhelming majority of people should find this pretty easy on my models.

Which makes my first hypothesis "something about the way I've tried to explain this was clumsy, and I've given you a wrong impression, and you've correctly identified the-thing-you're-imagining as not very workable but it's not the thing I was hoping to convey."

In particular, the words "predict" and "accuracy" from hypothesis 1 seem like they might indicate that I've nudged you in the wrong direction.  There's a quote I like a lot that goes something like "History deals with what happened.  Fiction deals with what happens."

So, in a story, "but it really happened that way once!" is not sufficient justification; stories have to pass muster as feeling plausible in a way that actual reality doesn't.

Similarly, there's a huge range of things a specific person might actually say, based on a truly staggering number of factors.  The thing a shoulder advisor does is not "predict with accuracy" but rather "emulate with verisimilitude."  I don't use my shoulder advisors to guess people's exact reactions, so much as to remind myself of their typical response.  You don't have to know them deeply enough to know what they'd say, just to get a sense of "oh, they'd say something like this isn't specific enough, I've got holes in my plan, blah blah blah," and over time this simulation gets sharper and sharper and more specific (while still just being one possible example plucked out of a very wide space).

Hypothesis 2 would surprise me, but again, human variation.  You do you, in the world where H2 holds.  =P

As for hypothesis 3, I again think I must have just said things that led you down the wrong path.  There's a bit of effort in remembering to check with a shoulder advisor at all, at first—in having the mental discipline to bother to do the move.  But if you're finding it effortful in the moment—if the act of having a shoulder advisor in the room with you feels like something you're having to try to do, or work at, then you're either doing something very different from what I do or your mind is shaped very differently from most of those I've run into.

Which is possible!  Would be surprising, but not shocking.

I think my advice would be to grab your most-likely-candidate for useful shoulder advisory, and have that person read this very comment I've written here.  Like, imagine them reading these words, and see where they scoff or laugh or shake their head or furrow their brow or get really excited or whatever.

Replies from: samuel-shadrach, Gunnar_Zarncke
comment by Samuel Shadrach (samuel-shadrach) · 2021-10-10T14:02:39.121Z · LW(p) · GW(p)

Thanks for the reply!

I feel uncomfortable making my shoulder advisors make incorrect predictions, and I do value correctness. I feel I have a natural tendency to dismiss people as boring or unintelligent, and therefore I counteract this with a conscious effort to remind myself that people have more to offer than I think they do. And this attitude has helped me. So if I try modelling a response of someone I care about and consider smart, I know there's a good chance their response will be something I haven't guessed already, and I don't want to force myself to guess incorrectly or typify them. If absolutely pressed, I could enumerate categories of thoughts or something along with probabilities - but that would be a conscious effort. I can however predict emotional response like - will this person even bother to read a wall of text, will they be annoyed I sent it to them, will they read and reply something smart. (To answer your suggestion in last para)

 

re: hypothesis 3 yep it definitely sounded like what you were suggesting was intended to be low effort, hence I mentioned that it feels high effort to me.

re: hypothesis 2, if I may elaborate: it's not that I don't care, it's just that the caring is at a conscious level, and therefore if I must model someone because of this care, it feels like effort. To take a slightly different example: I have learnt the "double diamond" design thinking process, I think it is useful and yet I will need to expend energy applying it to a problem, it isn't my natural way of approaching a problem.

comment by Gunnar_Zarncke · 2021-10-09T18:47:46.551Z · LW(p) · GW(p)

Samuel's Hypothesis 2:

  - I don't care about anyone's perspectives on my decisions enough to want to model them

Duncan:

Hypothesis 2 would surprise me, but again, human variation.

As anecdata: I can very much relate to Samuel's point. It describes the younger me pretty well. It is only later that I came around to see this as useful.

comment by Raven · 2021-10-10T03:45:10.515Z · LW(p) · GW(p)

Modelling other people this way takes too much energy

OP is basically describing low grade plurality/tulpamancy and aptitude for that is highly variable.

comment by Flawed Spiral (flawed-spiral) · 2021-10-12T14:28:27.051Z · LW(p) · GW(p)

Oh, I just realized this post is new, and not one of the 8 year old posts I keep coming across and want to participate in.

I don't think this technique works for me, at all. For several reasons. And reading through the comments section, I feel like other people can't imagine my failure case (e.g. the people surveying others about whether they do / can do this - "Can you imagine your friend saying this?")

I'm visually aphantasic, I cannot conjure a visual image at all except in some phases of sleep, or sometimes under extreme tiredness while 'awake' (I hypothesize those mental states are closer to sleep than normal waking awareness anyway). This means the 'visual image' part falls apart instantly. I can, however consciously control my auditory perceptions with extremely high fidelity.

This means I can easily make a voice in my head say anything in a specific person's voice, but that can only happen with my conscious input. These voices are purely puppets, or more accurately, just 'voice filters' I can put on my 'internal voice'.

I believe I've never 'heard' anything other than what actually happened or came directly from (or through) my internal train of thought, with rare exceptions of simple sounds hallucinated when about to fall asleep, so it's rather hard for me to imagine an internal voice that is not THE internal voice.

That all was the first reason, another one is that I don't think I can model people accurately enough to do this even consciously, let alone subconsciously. I don't have a way to tell "what would my mom/dad/Alastor Moody say?" - for fictional characters I can only produce the most stereotypical sound bites associated with them if I don't make them say anything specifically, for Moody it's "Constant Vigilance", for Princess Celestia it's "I'm so proud of you", but for Albus Dumbledore or Twilight Sparkle it's just a thoughtful hum in their voice - I at most have a "How would they say this?" component in my head that lets me reword something I want (them) to say into their speech pattern and voice.

While writing this comment I did a few experiments with the mental 'voices' I have. If I force them to produce speech, I hear nonspecific speech. That is, I hear that there are words being spoken, breaks between words and sentences, and so on, but none of the actual words. Sort of like hearing some person you recognize in a crowd, but you can't make out what they're saying.

I also reflected on "mental constructs that are not me" - I don't think I'm able to form these anymore at all. Even the few dreams I have and recall are empty of other characters (even if there are 'people'). I used to have other characters in dreams in my childhood, but even then it was at best one antagonist + a crowd.

Replies from: flawed-spiral
comment by Flawed Spiral (flawed-spiral) · 2021-10-13T07:00:29.001Z · LW(p) · GW(p)

On more self reflection, and reading a bunch of posts in the subagents tag, and looking into tulpas again, I believe I just don't have the mental architecture for this kind of thing.

I hypothesize that this skill requires thinking of yourself as a personality, but I don't see myself that way. I see my 'self' as my central attention. I don't believe I have software subagents, I believe I have hardware submodules with some configuration overlays at most. There's no 'default personality'. I switch how I interact with people based on context fluidly.

I don't believe my architecture is flexible enough for this, as I can't even write fictional characters that are not a version of my mind with masked knowledge and hidden or exaggerated traits (and otherwise with a manipulated context). I don't argue with people in my head, I don't do 'what ifs' of conversations, I don't imagine what a person would say in some situation.

Replies from: Kaj_Sotala, Duncan_Sabien
comment by Kaj_Sotala · 2021-10-13T18:37:39.439Z · LW(p) · GW(p)

I don't believe I have software subagents, I believe I have hardware submodules with some configuration overlays at most. There's no 'default personality'. I switch how I interact with people based on context fluidly.

Interesting - if someone told me only that they have no default personality and that they switch how they interact with people fluidly based on context, I'd assume them to naturally view their mind in terms of subagents. Since "no default personality" sounds to me like "no unitary self, just different subagents that get activated based on the context".

How do you define the difference between software subagents and hardware submodules?

Replies from: flawed-spiral, Gunnar_Zarncke
comment by Flawed Spiral (flawed-spiral) · 2021-10-13T23:01:23.686Z · LW(p) · GW(p)

How do you define the difference between software subagents and hardware submodules?

I (feel like I) have a good understanding of what exactly which major 'parts' of my brain are capable of. I know what I can do by language modeling, I know what I can do and when with my visual cortex, auditory cortex, kinesthetic sense, motor cortex, working memory, I know what I can do with my spatial awareness. I can consciously focus on most of these parts and affect their workings, mostly by bringing them into attention. I'm also aware of, but not in direct conscious control of some other parts of the brain which makes me somewhat aware of things such as Ugh Fields. (I don't have names for all such submodules, only the most obvious ones that seem to match up to things I've read.)

These all are completely non-agenty, even if they can somewhat work independently of my central attention. Say, when you're not focusing on sound, do you view your auditory cortex as a sub-agent when it brings to your attention the fact a loud sound just happened? I don't. I also don't see these parts as separate from 'me', and I don't communicate with them in any way except with raw attention.

I should also clarify that despite my description, I don't mean whatever controls executive function when I say "central attention". I mean the part of the brain that controls the importance that affects what stays and what's replaced on the 'main bus' that other parts of the brain dump data on.

My interpretation of software subagents is that people can install a (possibly pseudo-)personality that runs in certain parts of the brain, or at least interfaces with them while your central attention is elsewhere. Importantly, it's able to use various mental resources without it coming to your central attention. This interpretation is likely wrong, as I have no experience with this other than reading people's posts on the internet, which loses a lot of detail.

Regarding "no default personality" and "no unitary self". I don't think that's the case (unless I misunderstood the term), I do have a self, I see my mind's central attention as my core self, and the rest of my brain as components that allow me to do things that make up the whole self, the externally visible person that I am. I see some of those components as more important (various long term memory, language; some higher abstraction parts of the visual cortex) or less important (the actual learned personality-like behaviors I assume in various situations, my routines, common knowledge) to preserving the 'whole' self and its values.

I believe the most important parts of self-hood happen in the center of attention, things that come to attention have a disproportionate effect on decision making, memory formation, and basically everything else that's important in the brain.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-14T01:14:47.684Z · LW(p) · GW(p)

That again sounds pretty close to how I would have described me earlier (and would mostly still describe). You didn't write about your emotional balance but if were is again like me it would involve few excited states and fewer conflicted or what's called negative states and emotions as e.g., anger. If so, that would confirm my assumption that getting to such a uniform and stable state requires a certain environment. An environment that has little need for the developing brain and mind to overfit. No forced adaptations to environmental risks like loss of caretakers or life. But also an environment rich in information and with exploring. 

Replies from: flawed-spiral
comment by Flawed Spiral (flawed-spiral) · 2021-10-14T09:15:42.504Z · LW(p) · GW(p)

True, I didn't mention emotional balance since it usually plays little role in my daily life. I used to have issues with managing extreme emotions in early childhood that I solved by both avoidance, and 'dimming' them to the point they are mostly manageable. I avoid anger in daily life because it was always unproductive for me in the past, and is incompatible with the social strategies I use nowadays (which I picked because I suck at social 'tactics').

The environment you describe matches up well, but not perfectly with the one I grew up in. I guess you can use that as a confirmation.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-14T09:46:32.328Z · LW(p) · GW(p)

Thanks for sharing. I can relate even to the exceptions and resulting strategies.

comment by Gunnar_Zarncke · 2021-10-13T21:13:32.322Z · LW(p) · GW(p)

I can relate very well to Flawed Spiral. Before I looked more into meditation and got a higher resolution awareness of inner processes, I could have written a description like him. For me, it was not so much "no unitary self" as rather a tiny self. I didn't have much of an inner monologue. And I could easily relate to Paul Graham's appeal to Keep Your Identity Small. A small self doesn't mean low complexity. For me, it was a high alignment of inner processes and values as well as a rich and updatable world model. 

Replies from: flawed-spiral
comment by Flawed Spiral (flawed-spiral) · 2021-10-13T23:34:02.296Z · LW(p) · GW(p)

I personally see an inner monologue as as much of a tool as any other part of my brain. The inner monologue, as a tight coupling of auditory and linguistic processing, is rather helpful for performing some kinds of thought, for extending working memory (the auditory loop is an extremely easy place to store small amounts of nearly arbitrary data in the immediate term, and you can abuse your language processing to store moderate amounts of linguistic data in the short term as long as you're able to retrace a path of thought through it).

I do find that I don't have a constantly running narrative of my thoughts and what I'm doing, even if I remember having one in the past. I still use internal monologue to trigger parts of my brain for things like planning, or for enhancing myself in some task as described in the earlier paragraph, but most of the time my inner monologue is inactive.

I do agree with Keep Your Identity Small, I seem to have been doing that, or something very similar, automatically from a certain point in my mid teens. This does have a side effect that I never really feel like part of most groups, which is both good and bad, as it allows me to exit groups or communities easily, and for example, permanently 'shed' online identities that I decide I can't use anymore for whatever reason (like sharing too much info that's reasonably correlatable to another identity or real life).

I'm curious what kinds of meditation you've looked into. My go-to form of meditation is focusing attention on my body, in any position, with or without muscle relaxation or increasing blood flow.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-14T01:01:43.872Z · LW(p) · GW(p)

The inner monologue, as a tight coupling of auditory and linguistic processing, is rather helpful for performing some kinds of thought, for extending working memory

Same. 

I still use internal monologue to trigger parts of my brain for things like planning

Or for recalling previous conversations or rehearsing speech (though that also falls under extended working memory).

This does have a side effect that I never really feel like part of most groups.

Same here. And that, together with what you wrote earlier ("I don't think I can model people"), leads to less of a felt connection to other people - in both directions: It makes us harder to model.

It is why I have tried to pick up skills in that direction. The way our mind has developed makes it harder - but I think if we succeed, more fluid.

I'm curious what kinds of meditation you've looked into. 

Since being a teen, I have done a lot of self-introspection. Meditation looked suspicious to me for a long time. I knew about its benefits though the same is said about religion. I was delighted to find a non-dogmatic introduction on LessWrong though I'm not such which one of the many under the tag meditation [? · GW] it was. Probably one by Kaj.  I tried the breathing exercise, and it was effortless. Same with other exercises. I had trouble locating emotions in the body and was skeptical, guessing it being illusory (same trouble Duncan has). I attended a 10-day silent Vipassana meditation retreat two years ago organized by an LWer and billed as non-dogmatic and open to individual needs. It worked out incredibly well. The teacher (Julia Harfensteller) provided a lot of exercises and cues from multiple directions. The resolution of my introspection increased immensely. At the end, I gained access to my emotions - previously, they had been so well-regulated subconsciously as to be almost invisible. In the weeks after, I went thru big parts of The Mind Illuminated (see e.g. here [LW · GW]). 

Things I did:

  • Breathing meditation (decompose the sensations of the breath)
  • Decomposing visual perception ('unseeing' shapes, forms, motions, faces)
  • Noticing beginnings and ends of thoughts down to pre-thoughts.
  • Noticing bodily sensations, itches, bodily posture with high resolution.
  • Noticing and naming emotions and noticing bodily correlates.
  • Regulating emotions up and down e.g joy.
  • Tuning brain modules up and down, e.g. awareness of physical space, social space, senses, thoughts.
  • Fun mental experiments like running two though trains in parallel (or rather interleaved).
  • Deconstruct consciousness.

When I write decompose or deconstruct, I mean it in a sense that includes an intuition like in math when you can solve equations without thinking because your practice has pushed most of the work into the subconscious (System 1 if you want) and made it automatic, effortless.

Replies from: flawed-spiral
comment by Flawed Spiral (flawed-spiral) · 2021-10-14T09:32:42.322Z · LW(p) · GW(p)

Or for recalling previous conversations or rehearsing speech

I overlooked the obvious, yes, I do that too, of course. However, less of the rehearsing speech part, and more of looking for concrete words for concepts in the moment. I do believe I would improve the fluidity of my speech by rehearsing, I'm not sure that kind of practice is aligned with my values.

Most of your meditation description sounds fascinating, it seems mostly like practicing the skill I already have to strengthen the connection between direct sensations and conscious attention. The only parts that I've never consciously done before are regulating emotions up, and paying attention in general while in emotional states.

I still find backtracking through thoughts difficult, and am not completely successful. I think the way I practice is not particularly effective, but I would like to improve.

I'm not sure I'd be willing to go to a meditation retreat, I'd have to re-evaluate quite a few things to consider actually going.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-14T10:29:28.579Z · LW(p) · GW(p)

I think the two big advantages of a retreat are

  • a lot of time in one large chunk to improve the mind (introspect, meditate, or something). As with programming some things you can only do if you go deeper and deeper in one run (extreme maker schedule).
  • tight feedback loops with the teacher and other practitioners hopefully at about the same level. 

Both interrelate. But with your specific profile and experience level, I think it will be difficult to find a suitable retreat. It might work better to work closely with a meditation practitioner that you click with.

comment by Duncan_Sabien · 2021-10-13T17:03:27.500Z · LW(p) · GW(p)

FWIW, my experience of writing fiction is very much "they are versions of me with masked knowledge and hidden or exaggerated traits."

comment by Mary Chernyshenko (mary-chernyshenko) · 2021-10-11T18:33:10.033Z · LW(p) · GW(p)

One of mine is a (real actually) gander whose name is Жирний (Fatty). The fictional him is more optimistic / not-depressed than me; a generous, unsophisticated, slightly egotistical, proud of his flock, kinda shy, but of times wordy, magnificent bird. He just never hesitates to be kind to others when my self-image would invent a carload of reasons not to.

comment by Gunnar_Zarncke · 2021-10-09T20:14:03.071Z · LW(p) · GW(p)

You might want to add a disclaimer when not to try this or what the risks are. Meditation risks have been discussed on LW [LW(p) · GW(p)] before. I don't want to overblow this, I think it is pretty safe. But it is an experimental meditation practice and you could link to the risks section of SSC Book Review The Mind Illuminated or offer to help if anything unexpected happens.  

Also, your mileage may vary. You couldn't sell me on all the nice points you listed - having a lot of time to work on it I arrived at a stable inner life via other means. What sold me on your practice was your aside 

... not to mention that having robust copies of my actual friends and colleagues has much better equipped me to interact with those friends and colleagues

Replies from: quintin-pope
comment by Quintin Pope (quintin-pope) · 2021-10-10T05:39:11.983Z · LW(p) · GW(p)

On the one hand, I agree that potential side effects are important. Shoulder advisors seem very similar to tulpas, and mental health disorders are very common (~50% or so) in the tulpamancy community. Though this paper argues that this is because mental health issues cause people to be drawn towards tulpamancy, and that tulpamancy can benefit those with mental illnesses. Of course, people who had negative experiences with tulpas would likely leave the community and not be available for surveys.

On the other hand, shoulder advisors and tulpas are fundamentally exercises in prediction. Your perception of the practice influences the results you’ll get. If you create shoulder advisors with the assumption that they’ll go wrong immediately, your odds of a beneficial outcome drop. It’s thus important not to over-emphasize potential negatives.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-10T15:11:53.771Z · LW(p) · GW(p)

Agree. 

comment by Richard Horvath · 2021-10-09T17:03:04.831Z · LW(p) · GW(p)

I love how Jean-Luc Picard was selected to be one of your advisors. He is also among my best candidates :).

 

What is the largest number of advisers you have known people to actively use? I am a bit reluctant to cut it down to four or five.

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-09T17:45:04.715Z · LW(p) · GW(p)

At any one time (i.e. in any one specific situation) the most I've ever seen anyone juggle is about six.  Like, I have sometimes actively booted up 5-7 advisors over the course of a ten-minute introspection, and have seen others do the same, and sometimes those advisors "talk to" each other directly without me feeling like the Duncan-personality is doing anything other than watching.

But as for my overall cast of shoulder advisors—it's well over thirty?  Essentially anyone I get to know past a certain level becomes emulable, and many many people might pop onto my shoulder only once or twice a year, or only once ever.  But there are at least thirty people (maybe fifteen real and fifteen fictional) who I regularly emulate.

Recommendations to cut things down were solely for the purpose of "if you've never done this/have no experience, don't try to do too much at once."

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-09T20:28:54.569Z · LW(p) · GW(p)

the most I've ever seen anyone juggle is about six.

I am curious about the effort it takes or the impact it has on resolution, or the time needed. My predictions would be 

  • no two advisors can talk at the same time but take turns (but can interrupt)
  • effort or at least time scales mostly linearly
  • Dunbar's number applies
Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-09T20:58:44.843Z · LW(p) · GW(p)

Definitely I've never had more than one "voice" going at a time, even if there are two or three voices interrupting and taking over in rapid succession.  This accords with what I've heard from other people, though I haven't specifically asked about whether people ever hear two-or-more.

I think in a given three-second span I can track something like the "mood" of at least three different perspectives or advisors at once?  The same way that (if my own experience generalizes) you can be in a room with ten people, and someone says something, and you immediately have a visceral feeling about how several of the other people will react.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-09T21:15:05.616Z · LW(p) · GW(p)

you can be in a room with ten people, and someone says something, and you immediately have a visceral feeling about how several of the other people will react.

Since a recent meditation retreat, I can notice the mood of a group of people - but that isn't the same as having a specific feeling about all individual reactions and aggregating over it. If you can do the latter I'm impressed.

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-09T21:53:04.959Z · LW(p) · GW(p)

It's usually far from "all."  It's more like, in this room with ten people who I know reasonably well, any given development will tend to provoke specific, identifiable reactions from my mental model of 1-4 of them. If ten minutes go by, my attention will land on almost everyone at some point, and I'll have clear intuitions for almost everyone at some point, but in any given moment some will be much more salient than others.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-09T22:17:14.844Z · LW(p) · GW(p)

That sounds like what experienced managers I know seem to be capable of, though they wouldn't phrase it in terms of advisors.

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-09T22:20:52.204Z · LW(p) · GW(p)

Yeah, at this point we're drifting away from the concept of "a curated mental model" and into just "general mental impressions," or something.  But they're contiguous concepts, in my head—they're just different in something like vividness or intensity or clarity.  I can "turn my focus" toward what's just a whispering impression, and it becomes more of what I'm thinking of as a fully-fledged emulation, or I can shove an emulation into the back of my mind and ignore it and it subsides into just being a tickle of awareness.

Replies from: Gunnar_Zarncke
comment by Gunnar_Zarncke · 2021-10-09T23:29:25.517Z · LW(p) · GW(p)

I can shove an emulation into the back of my mind and ignore it and it subsides into just being a tickle of awareness.

Yes. That rhymes well. I can better describe what goes on with these emulations now. 

My primary mental mode is thinking in concepts. Probably related to what Kaj called conceptese. I struggled to find concepts for people; some intermediate structure was missing. With your shoulder advisors, I have found it. And I can apply all the machinery that I have for concepts. 

With meditation practice, I got better at noticing thoughts forming or concepts activating. Following your instructions, memories of friends are brought up, and let babble [? · GW] - mostly their expressions. Some do feel easier - have higher emulability. There is quick back and forth between your suggestions and own experimentation. A list is updated with results. First spontaneous activations happen - just fleeting sentences. Like an idea coming up. 

My prediction is that this will go the same way many concepts and skills do with me: After a short time of being very concrete, they will stop standing on their own but become part of my overall way of being. Thus less like a person advising but more like just knowing what is going on (your "general mental impressions"). At least if it goes efficiently, otherwise it will remain as a tool used consciously but rarely. 

comment by BeanSprugget · 2021-10-16T20:54:29.434Z · LW(p) · GW(p)

This makes me think "tulpamancy-lite". Not that that's a bad thing - perhaps it's like a safer tulpamancy. Some thoughts:

(It's just such little mannerisms that allow a shoulder advisor to be "really real"—to bring it to life, give it a personality separate from, and not dependent on, your brain's main central personality. Again, I don't have a sound explanation of the mechanics, but it works.) Would it be useful to have a shoulder-advisor not constrained by having to relate to a real example? Or perhaps, without that link it will just tend to become more and more like you, or otherwise drift into some territory outside reasonable personality-space. Although, authors seem to be able to write stories just fine.

How hard would it be to just create a shoulder-advisor from scratch? - I mean, people can do that with tulpas, and shoulder-advisors definitely seem like less work than tulpas (no need for hallucinating them).

Do shoulder-advisors have moral value?

I find that I get self-conscious when in public without close friends, and I've wanted (but have had neither the time nor motivation) to create a tulpa, with the idea that it will make everything a lot less stressful. They wouldn't really have any actual input on anything, but just sort of raise my baseline positivity and self-esteem. If there were two of me, I'd be a lot less afraid of doing anything.

Replies from: Duncan_Sabien
comment by Duncan_Sabien · 2021-10-17T02:11:29.548Z · LW(p) · GW(p)

I have created shoulder advisors from scratch, most notably a pair of characters from a novel that I'm writing who like to show up in opposition to each other.  Not sure about any of the other points.