Unrolling social metacognition: Three levels of meta are not enough.

post by Academian · 2018-08-25T12:00:00.000Z · score: 157 (68 votes) · LW · GW · 29 comments

Contents

  I. Conceptual introduction, by example 
    Warning: reading this first section (I) will require a fair bit of symbolic reasoning/thinking, so you might find it tiring and prefer to skip to later sections.  A better writing of this section would do more work in between these symbolic reasoning bits to distill things out and make them easier to digest.  
    Scale 2: multiple events, six levels of meta
    (This is the end of the tiring symbolic reasoning section.)
  II. Higher levels of meta
  III. People don't usually unroll things this way.   Why?
  IV. A fruitful application
  V. How generalizable is this 'unrolling' technique?
  VI. Relation to "miasma" and "hype"
  VII. Apology
29 comments

Disclaimer: This post was written time-boxed to 2 hours because I think LessWrong can still understand and improve upon it; please don't judge me harshly for it.

Summary: I am generally dismayed that many people seem to think or assume that only three levels of social metacognition matter ("Alex knows that Bailey knows that Charlie knows X"), or otherwise seem generally averse to unrolling those levels. This post is intended to point out (1) how the higher levels systematically get distilled and chunked into smaller working memory elements through social learning, which leads to emotional tracking of phenomena at 6 levels of meta and higher, and (2) what I think this means about how to approach conflict resolution.

Epistemic status: don't take my word for it; conceptual points intended to be fairly self evident upon reflection; actual techniques not backed up by systematic empirical research and might not generalize to other humans; all content very much validated by my personal experiences with talking to people about feelings in real life.

Related Reading: Duncan Sabien on Common knowledge & Miasma; Ben Pace on The Costly Coordination Mechanism of Common Knowledge [LW · GW]


I. Conceptual introduction, by example

Here's how higher levels of social metacognition get distilled down and represented in emotions that end up tracking them (if poorly). Each feeling in the example below will be followed by an unrolling of the actual event or events it is implicitly tracking or referring to.

Warning: reading this first section (I) will require a fair bit of symbolic reasoning/thinking, so you might find it tiring and prefer to skip to later sections. A better writing of this section would do more work in between these symbolic reasoning bits to distill things out and make them easier to digest.

Scale 1: One event, four levels of meta (yes, we're starting with four)

1.1) Alex leaves out the milk for 5 minutes

1.2) Bailey observes (1.1), and feels it was bad.

Unrolling of referents: Bailey felt that Alex leaving out the milk was bad.

1.3) Alex observes (1.2), and feels judged.

Unrolling of referents: Alex felt that Bailey felt that Alex leaving out the milk was bad.

1.4) Alex reflects on feeling judged, doesn't like it, and concludes that Bailey is "a downer".

Unrolling of referents: Alex felt it was bad that Alex felt that Bailey felt that Alex leaving out the milk was bad.

Notice that the unrollings look and sound very different from the distillations. That's in large part because the unrolling is not our native format for storing social metacognition; it's stored via concepts like "feeling judged" or "being a downer". However, to the extent that the feeling "Bailey is a downer" is tracking something in reality, it's tracking things that track things that track things that track reality: in this case, milk spoilage.

(An aside: notice also that 1.4 involves Alex's feelings about Alex's feelings. Some people wouldn't call that an extra level of social metacognition, and would just combine it all together into "Alex's feelings". However, I'm separating those layers for two reasons: (1) the separation in counting won't affect my conclusion that the total number of levels being implicitly tracked greatly exceeds three, and (2) I think it's especially important to note when people have feelings about their own feelings, as that can lead to circular definitions in what their feelings are tracking; but that's a topic for another day.)

Scale 2: multiple events, six levels of meta

I'll start the numbering at 4 here:

2.4) Multiple similar Scale 1 events happen Alex does something X, and ends up feeling that Bailey was "a downer" about it.

Partial unrolling of referents: Alex feels that Bailey is often a downer

Complete unrolling of referents: Alex felt it was bad that Alex felt that Bailey felt that Alex doing X was bad, for multiple values of X.

2.5) Charlie observes Alex treating Bailey like "a downer", thinks this is baseless, and feels Alex is "a snob".

Partial unrolling of referents: Charlie felt it was bad that Alex felt that Bailey is a downer.

2.6) Bailey observes Charlie's opposition to Alex's snobbiness, feels socially included by Charlie, and concludes that Charlie is "protective & welcoming."

Partial unrolling of referents: Bailey felt it was good that Charlie felt it was bad that Alex was feeling that Bailey was often a downer.

Complete unrolling of referents: Bailey felt it was good that Charlie felt it was bad that Alex felt it was bad that Alex felt that Bailey felt it was bad that Alex did X, for multiple values of X.

If you count the (+/-) signs implicit in the "good" and "bad" judgements here, they suggest that Charlie is implicitly condemning Alex's multiple "X" behaviors (perhaps among other things, such as Alex's style social of delivery). Charlie might not intend or even be aware of this effect, and it might take explicit work and discourse for the group to untangle and notice. What could result from the group reflecting upon this together? Well, it's not hard at all to imagine that Charlie, upon haring about the milk and other situations (values of "X"), might conclude that Alex's behavior with the milk was reasonable, and that "Alex was right to think that Bailey was being a downer". In turn, this could burst Bailey's bubble of social support, and result in Bailey having a change of heart about how critical to be of others.

The particular consequences here of course a hypothetical, and things could go very differently depending on the details; it's just meant to illustrate how "changes of heart" can propagate through unrollings of social metacognition.

ETA (8/27): Note there is an important distinction here between implicit and explicit metacognition. Bailey alone is not (necessarily) loading up a 5-layer-deep cognitive model of what's going on, all at once. Rather, the layers are distributed across people, whence the term "social metacognition". However, six levels of metacognition really are needed for someone to aiming to ground out out all these feelings in "object-level" reality (i.e., non-mental phenomena like milk spoilage).

(This is the end of the tiring symbolic reasoning section.)


II. Higher levels of meta

Without going into further explicit detail, I hope you an see the pattern. Levels of social metacognition get distilled into simple, repeated concepts like "feeling judged", "being a downer", "being a snob", "being welcoming", and so on. To the extent that these distilled concepts behave in a somewhat systematic manner in relation to reality, they have some tendency to be actually tracking things that are actually happening. It's not uncommon for me to observe six levels of social metacognition in a given disagreement or conflict, which is why I chose six for this post.

III. People don't usually unroll things this way. Why?

Unfortunately, I think a lot of people aren't aware that it it even makes sense to try to ground out these sorts of social meta cognition in more explicit terms to be reasoned and disagreed about. I think this is because it takes a lot of working memory slots to do, such that you basically need a shared piece of paper, whiteboard, or a shared Google doc to do it reasonably and collaboratively (rather than just slinging hard-to-unpack negative judgements at each other, adversarially, either in person or on the internet). However, I've resolved conflicts through co-writing and co-diagramming relevant levels of social metacognition many times now, and found it to be very enlightening almost every time it a way that directly benefitted the "social situation". I've found it's best if the shared writing medium is used for distillation mechanism, and is augmented by actual real-time conversation over the creation of the document.

IV. A fruitful application

For instance, in the past week, Alex (anonymized) felt judged by me for a thing I noticed Alex doing. I said, (a) "Don't worry, I don't think you did anything bad", but Alex didn't find this reassuring. To check, I asked "Do you feel like I feel like you did something bad?" and Alex said "No". This ran up against my explicit models of people feeling judged that had fit well with past unrolling of the concept. So, I broke out a Google doc (in person) and started unrolling stuff. The situation was more complicated than described above, so the doc gave us mental space to explore other ideas for resolution. We eventually looped back to my question (a) above, and Alex said "Huh, yeah, I think I do feel that you feel that I did something bad." Once that awareness existed, I responded "Cool! Well guess what? I don't think you did anything bad.", and this time, it resonated with Alex and Alex no longer felt judged. I then apologized with "Also, I'm sorry you felt judged. Given that I didn't actually feel you were doing something bad, this was a mistake on my part, and I'm sorry."
This further cleared things up.

This whole process took about 15 minutes. In retrospect it might seem like we could have jumped straight to this solution by me saying "I'm sorry I made you feel judged", but that wasn't an available strategy ex ante, for two reasons:

(1) Sometimes I really am judging someone, and I'm okay with them feeling judged, because I do in fact think they did something wrong. As a result of this willingness in myself and others, it's not always believable to say "Sorry, I wish I hadn't made you feel judged". Indeed, to many this feels like a platitude. But, by actually going through the work of actually unrolling whether or not I thought Alex did a bad thing, and the other details of what was going on between us, we established enough shared clarity about the situation that we managed to "get on the same page" what whether a bad thing was done, who thought or didn't think that, and who miscommunicated or didn't miscommunicate about it.

(2) There were many other things going on that the Google doc helped to organize and sift through without getting us lost. Without that functionality, I don't think we would have been able to hone in on the particular narrative resolution above.

V. How generalizable is this 'unrolling' technique?

The application (IV) above not an isolated incident. I've founding co-writing and co-drawing to be extremely valuable in settling social disagreements and conflicts on at least 30 occasions now, with at least 7 different people, of varying degrees of inclination toward explicit symbolic reasoning. I imagine some inclination is necessary, but much less than I would I have expected perviously. For instance, I've used this sort of unrolling heuristic fruitfully in numerous conversations with folks close to me who

(1) didn't go to college or otherwise study a symbolic discipline like math or linguistics, but who

(2) were generally open minded enough to be willing to try out a "weird conflict resolution technique I'm experimenting with" where we sat down together and tried unpacked our feelings in explicit terms in a common medium (usually a Google doc).

I'll defer to the finding of the broader community here to see if others can make this sort of thing work usefully.

VI. Relation to "miasma" and "hype"

The concept of "miasma" that Duncan is gesturing at in Common Knowledge and Miasma feels like a real social phenomenon to me, succinctly definable as "negative ungrounded social metacognition". There is such a thing as positive ungrounded social metacognition, as well, which I think is normally called "hype", at least in Silicon Valley. I think both hype and miasma are failures of group coordination, and both are costly to resolve, along the lines pointed out by Ben in The Costly Coordination Mechanism of Common Knowledge [LW · GW]. However, the communicative costs of resolving these problems can be significantly decreased if people are aware of what they are. Both require creating and sharing of ideas in places of common view, like writing blog posts that a lot of people see each other commenting on, or holding meetings that a lot of people can see each other attending, or for complex topics, sitting together and co-authoring a document.

VII. Apology

I'm sorry I put very little effort into the pedagogy of this post, due to having too little time to write it. Hopefully it will be of some value anyway, due to much better posts having been written and circulated on common knowledge recently, and due to the general intellectual health of LessWrong appearing, to me, to be able to absorb mediocrely-explained ideas and flesh them out into better ones. My sense is that the culture here has been trying to move towards people not waiting until an idea is perfectly elaborated before starting to talk about it, so to the extent these ideas might be valuable, I'm punting to the community to do more elaboration and/or distillation of them. Indeed, wishing not to be a part of a "common knowledge breakdown" problem is one reason I time-boxed two hours to write this post instead of waiting to improve it.

29 comments

Comments sorted by top scores.

comment by Benito · 2018-08-27T22:17:19.046Z · score: 34 (8 votes) · LW · GW

I've curated this post, for these three reasons (starting with the most important):

  • It lays out the implicit goings-on in a great many social interactions, and it does so with a simple, technical explanation.
  • The main point is not one I've seen written up explicitly before, and seems to fit in well with my models of the world. It personally gave me a much greater affordance to notice when some social interaction / conflict is not has more than three meta levels to it, and requires the use of external working memory [LW · GW].
  • It was quite readable and relatively short. It especially laid out useful metadata on the context - I feel I have a good sense of why you wrote the post and what caused you to believe the content of the post - without getting in the way of the core point of the post.

My thoughts on extensions and improvements:

  • I'd be interested to read more concrete accounts of people using this when resolving interpersonal conflicts/confusions in their lives, and what particular heuristics helped them.
  • Seeing people figure out how it affects the points+conclusions of other writings on this topic (e.g. the ones you link to by Duncan and myself). (Alas, I have opinions on this that do not fit in this small margin.)

---

Personally, this post managed to describe levels of social meta in common knowledge that I had run away scared from trying to explain in my post on the topic (I just quoted someone else's explanation at length). I spent a while trying to think about how to explain its involvement in one-on-one social interactions, and had not come up with anything as concrete and clear (if a little cognitively taxing) as this one.

Also, you mention that you think that the community probably would prefer this quick post than no post at all, and that you think others will be able and happy to think about it + test it for themselves if it seems important. This seems correct to me - when you only have the time for a two-hour post, writing it and giving this context at the head of a post is great, can be very valuable, and I'm really glad you did it. Generally it seems both that folks are very happy you wrote this MVP version, and that they will try to test it out. (To those of you who do try it out, writing your experiences up in a LW post can be really helpful to others!)

comment by Benito · 2018-08-27T23:36:30.176Z · score: 23 (8 votes) · LW · GW

Oh, one other thought I had when reading this post: It seems to me that, when trying to come up with good explanations, people reliably don't consider ones that don't fit nicely into working memory. (Some would even say that the very definition of a good explanation is compressing a hypothesis to fit into fewer working memory slots.) But I notice that here it lead to people actually not considering true hypotheses (that more than three meta levels are being used). Perhaps this is merely another point in favour of the 'use external working memory' argument, but I can't help but feel there is some more specific heuristic it recommends about the class of questions where we're likely to make this mistake.

comment by elityre · 2018-09-05T18:08:16.614Z · score: 19 (5 votes) · LW · GW

I’m intrigued by the explicit unrolling in contrast to circling. I wonder how much circling is an instance of developing overpowered tools on weird partly-orthogonal dimensions (like embodiment) because you haven’t yet discovered the basic simple structure of the domain.

  • Like, a person might have a bunch of cobbled together hacks and heuristics (including things about narrative, and chunking next actions, and discipline) for maintaining their productivity. But a crisp understanding of the relevant psychology makes “maintaining productivity” a simple and mostly effortless thing to do.
  • Or a person who spends years doing complicated math without paper. They will discover all kinds of tricks for doing mental computation, and they might get really good at these tricks, and building that skill might even have benefits in other domains. But at the end of the day, all of that training is blown out of the water as soon as they have paper. Paper makes the thing they were training hard to do easy.

To what extent is Circling working hard to train capacities that are being used as workarounds for limited working memory and insufficient theoretical understanding the structure of human interaction?

(This is a real question. My guess is, “some, but less than 30%”.)

A lot of my strategies for dealing with situations of this sort are circling-y, and I feel like a lot of that is superfluous. If I had a better theoretical understanding, I could do the thing with much more efficiency.

For instance, I exert a lot of effort to be attuned to the other person in general and to be picking up subtle signs from them, and tracking where they’re at. If had a more correct theoretical understanding, a better ontology, I would only need to be tracking the few things that it turns out are actually relevant.

Since humans don’t know what those factors are, now, people are skilled at this sort of interaction insofar as they can track everything that’s happening with the other person, and as a result, also capture the few things that are relevant to the underlying structure.

I suspect that others disagree strongly with me here.

(Crossposted from here)

comment by binary_doge · 2018-08-28T00:41:56.376Z · score: 15 (4 votes) · LW · GW

Thanks for the read (honestly, noticed some very interesting points IMO) but I kind of fail to understand what exactly is your claim about the method you introduced.

Are you saying that it is a good model representation of social interaction? If so I would partially agree. Its cool that the model captures all the mental steps all the participants are making (if you bother to completely unroll everything), but it's not computationally superior to saying that: things like calling someone "a downer" are general beliefs that rely on a varying empirical basis, and should be checked for their verity.

In other words - instead of saying that "Bailey is a downer" means "Alex felt it was bad that Alex felt that Bailey felt that Alex doing X was bad, for multiple values of X" you can say that it means "Alex believes he observed Bailey to act as 'a downer' a sufficient amount of times" and unroll that (ask Alex why does he think Bailey's a downer, what does being a downer mean or what does he base her being one on) only if necessary. Most people would understand this explanation more intuitively, in my experience. Some might even find it trivial, cause, for example, if you call someone irresponsible more than once, most of the times you are aware that you think he has acted "irresponsibly" a sufficient amount of times, even if you don't phrase it that exact way. And this explanation is not inferior to yours in the theoretical sense, it doesn't supply less data, and it does seem a little more cost effective, explanation-wise.

Are you saying that it is a good method for practical conflict resolution? It very well may be, but your experience only teaches us that after engaging in a cooperative activity with Alex for a while, he understood that he felt judged and was more inclined to believe you weren't judging him. Psychologically, engaging in a safe activity with someone, even your captor in a hypothetical hostage situation, will diffuse tension and humanize conflicting parties in each other's eyes. It could be that you could have played a short card game with Alex and he would have been more cooperative afterwards all the same.

Even if we claim that analyzing the problem is emotionally helpful in itself, the analysis doesn't have to be all that rigorous, coherent and complete. Many conflict resolution therapy methods focus on giving all parties in a conflict the opportunity to feel heard, which makes it easier to reach emotional catharsis, and therefore agreement. But feeling heard is only a equal to or lesser amount of understanding than being completely, logically understood. Therapy methods such as these may (or may not) be more effective in achieving results, or may be just less effort-demanding.

In order to establish that this method is particularly effective, we either have to get some experimental data showing it gets better outcomes in some criteria, or explain what it has going for it that other methods unequivocally don't.

Anyway, I enjoyed reading this and would be glad if you posted some of your refined conclusions in the future :)

comment by sampe · 2018-08-27T08:46:39.742Z · score: 15 (5 votes) · LW · GW

Alex felt it was bad that Alex felt that Bailey felt that Alex leaving out the milk was bad.

I want to point out that not all instances of the word "felt" mean the same thing here, and I want to split the two meanings into new words for clarity. I think this has consequences about what it means to have feelings about one's own feelings.

Bailey felt that Alex leaving out the milk was bad.

In this case, "feeling" stands for "evaluating". Bailey gives a low value to Alex's action, and in some sense, maybe even to Alex. E.g. Alex is dumb, or Alex's actions are suboptimal.

Alex felt that Bailey felt that Alex leaving out the milk was bad.

In this case, the top level "feeling" stands for "sensing". This is a complex phenomenon:

  • X happens inside someone's mind;
  • the person holding X produces some observable effect Y (e.g. they make a facial expression);
  • the part of your brain specialized in this kind of work observes Y, infers X from it (you can't observe X!), and produces a conscious sensation Z (e.g. feeling judged);
  • you consciously observe Z in yourself.

Alex felt it was bad that Alex felt that Bailey felt that Alex leaving out the milk was bad.

The top level "feeling" stands again for "evaluating". After observing Z in yourself, you give it a low value. But Z is a proxy for X given that X is true (you may be wrong inferring X from Y). Giving a low value to Z roughly means "I wish it was not the case that X", but it's difficult to recognize as such because I think it happens very rapidly and as a complex mental sensation rather than a sentence in your head. I posit that all feelings about one's own feelings are of this kind, and I'm interested in hearing counterexamples.

So, to recap:

Alex evaluated it was bad that Alex sensed that Bailey evaluated that Alex leaving out the milk was bad.

I think the unrolling is clearer this way.

comment by Liskantope · 2018-08-26T08:35:53.687Z · score: 15 (6 votes) · LW · GW

A lot of excellent points in this post. I particularly like the one about having feelings about one's own feelings, which is something I think I've always understood on some level but perhaps the first time I thought about it consciously was from Seth Rogan's line in 40-Year-Old Virgin: "[Y]our depression is boring me for one thing, and it's actually making me a little depressed, which is then in turn making me more depressed that you're actually affecting my mood."

I think posts like this prove a more general point: we humans are thinking on many levels of meta, all the time, and insistence on pulling the levels apart to examine separately isn't "complicating things"; it's just a way of seeing more clearly what's already in our line of vision.

comment by lahwran · 2018-08-26T00:39:32.161Z · score: 14 (5 votes) · LW · GW

Thank you so much for sharing this concept with me a few years ago, it's still in the top ten of largest coherent boosts another human gave me.

I want to mention, for citation reasons - I am pretty sure a large amount of the discussion about this topic as an explicit, defineable human-common-knowledge came into my friend group through you, despite that I and others have written posts using the concept since you shared it, and that perhaps it was just your retelling of an existing thing. I really appreciate it.

comment by shminux · 2018-08-25T22:16:16.501Z · score: 9 (3 votes) · LW · GW

Very interesting analysis, thank you! This unrolling technique seems to be quite useful, at least in some situations, where, as you said people are "generally open minded enough to be willing to try out a "weird conflict resolution technique I'm experimenting with"".

One comment I have is that this is not what I understand metacognition is like, thinking about thinking about thinking ... . This appears to me more like a linear chain of inferences than a stack of them. So, in a comp-sci terms, you go through the list, rather than unroll a stack. The difference is that to get to "Bailey observes (1.1), and feels it was bad," one does not have to pop every single level from the stack, but can actually go through the chain of inferences back and forth from any element. Or at least that is what it seems like to me.

As for your point that people shy away from posting "mediocrely-explained ideas", I agree, especially if the ideas are not in the mainstream. *looks at the backlog of a dozen drafts* nope, totally not what I am doing.

comment by quanticle · 2018-08-26T01:55:25.181Z · score: 17 (5 votes) · LW · GW

As for your point that people shy away from posting “mediocrely-explained ideas”, I agree, especially if the ideas are not in the mainstream.

This might not be a bad thing. Idea inoculation is a known concept in sociology, and it can make sense to not post an explanation of something if you think that it will cause people who have heard your explanation to discount future versions of that idea.

To quote Nietzsche:

There are terrible people who, instead of solving a problem, bungle it and make it more difficult for all who come after. Whoever can't hit the nail on the head should, please, not hit at all.

comment by c0rw1n · 2018-08-25T21:11:53.396Z · score: 6 (2 votes) · LW · GW

Thank you so much for writing this! I remember reading a tumblr post that explained the main point a while back and could never find it again -because tumblr is an unsearchable memory hole- and kept needing to link it to people who got stuck on taking Eliezer's joking one-liner seriously.

comment by Vladimir_Nesov · 2018-08-26T06:10:48.496Z · score: 5 (3 votes) · LW · GW

I think the key point is that situations where A thinks that B thinks that C thinks that X don't require both A and B to be purely guessing. Instead, C might have told B they think X, and A might have observed that conversation. It feels to me that when people point at the difficulty with high levels of meta, they mean the difficulty of guessing through multiple levels. The examples in this post involve just observing what's going on, which shows how it's often easy to form such states of knowledge in practice, if you don't have to guess.

I don't particularly agree about relevance of the chunking words, as the examples are only specific models of their meaning, the meaning itself can't be unrolled into a deep explicit chain of meta, it's more of a forest of possibilities than a chain, which is unlike the usual setting of "many levels of meta". It does turn into more of a chain when there's enough data, but then the words themselves become only a small part of the relevant state of knowledge.

comment by Said Achmiz (SaidAchmiz) · 2018-08-25T23:03:30.074Z · score: 5 (3 votes) · LW · GW

1.4) Alex reflects on feeling judged, doesn’t like it, and concludes that Bailey is “a downer”.

Unrolling of referents: Alex felt it was bad that Alex felt that Bailey felt that Alex leaving out the milk was bad.

There seems to be some ambiguity here, which I found very confusing on a first read-through, and which I’m still not sure I’m interpreting in a way that makes any sense.

Do you mean:

“Alex felt bad, due to the fact that Alex felt that Bailey felt …”

Or:

“Alex felt it was bad that Alex felt that Bailey felt …”

In other words, is it that Alex dislikes being made to feel judged? Or, is it that Alex views his own response (feeling judged) to be somehow un-virtuous?

Edit: Also, how/why does Alex conclude that Bailey is “a downer”? How does this follow from the rest of it? I don’t get this part.

Edit2: I also don’t get this part:

2.5) Charlie observes Alex treating Bailey like “a downer”, thinks this is baseless, and feels Alex is “a snob”.

A snob? What? I don’t understand how anything like the usual meanings of the concept of “being a snob” can make this reaction make sense.

Something strange is going on with your word/concept usage here, and I’m more and more perplexed about what it might be.

comment by ESRogs · 2018-08-26T23:19:25.009Z · score: 6 (4 votes) · LW · GW
In other words, is it that Alex dislikes being made to feel judged? Or, is it that Alex views his own response (feeling judged) to be somehow un-virtuous?

Pretty sure it's just that Alex dislikes feeling judged, rather than an evaluation of whether his own response was virtuous.

comment by Said Achmiz (SaidAchmiz) · 2018-08-27T00:41:03.974Z · score: 0 (5 votes) · LW · GW

In that case, the unrolling makes no sense whatever.

comment by maladroit · 2018-08-27T02:06:29.959Z · score: 1 (1 votes) · LW · GW
1.4) Alex reflects on feeling judged, doesn't like it, and concludes that Bailey is "a downer".
Unrolling of referents: Alex felt it was bad that Alex felt that Bailey felt that Alex leaving out the milk was bad.
2.4) Multiple similar Scale 1 events happen Alex does something X, and ends up feeling that Bailey was "a downer" about it.
Complete unrolling of referents: Alex felt it was bad that Alex felt that Bailey felt that Alex doing X was bad, for multiple values of X.

In the first example, there's [Alex feels that Baily [feels that leaving out the milk is bad]]

Then Alex feels that that^ is bad, which probably isn't 'I feel bad about having this understanding of the situation' but more like 'I disapprove of this thing that I understand'. So Alex is grumpy or angry or frustrated or disappointed when looking at [their belief that Bailey thinks their actiosn are bad]

I'm guessing the second example follows the same pattern. There's [Alex has a general feel that Baily [has a general feel that Alex is doing bad stuff] where the word felt that Critch wrote is like a belief but isn't necessarily explicit or justified.

So then you get Alex felt bad about the fact that, from Alex's perspective, Bailey keeps ETC.

I think that's different from both of the options you had in your top comment, but it's closer to your due to the fact option. There's a thing happening, and Alex doesn't like it (is sad, irritated, mad, whatever, which Critch tagged as just 'bad')

For your first edit, the downer thing reads to me as being more like 'Baily is a bummer or a party pooper or likely to have emotional reactions to things that I Alex think are too small or too trivial to deserve such a reaction.'

For your second edit, the snob thing reads to me as 'Charlie sees Alex judging Bailey as a person over Baily's judgements of Alex, and furthermore this pattern-matches for Charlie to a kind of snobbishness, E.G. Alex is judging Baily for not living up to Alex's standards of what should or should not trigger emotion or what one should or should not do in response to emotion, and maybe Alex is being hypocritical too since they're being emotional in response to that.'

comment by ESRogs · 2018-08-28T10:02:35.923Z · score: 2 (1 votes) · LW · GW
So then you get Alex felt bad about the fact that, from Alex's perspective, Bailey keeps ETC.

What does ETC mean here? I thought maybe you meant "etc." but I can't get the sentence to make sense in my head when I read it that way.

comment by maladroit · 2018-08-28T22:48:53.023Z · score: 3 (2 votes) · LW · GW

I meant etcetera.

Alex felt bad about the fact that, from Alex's perspective, Baily keeps having bad feels about actions that Alex is taking (things analagous to leaving out the milk)

comment by Said Achmiz (SaidAchmiz) · 2018-08-27T03:26:45.533Z · score: -8 (4 votes) · LW · GW

I don’t think I have ever met any people who think or behave like what you describe, though. Is all of this really supposed to be a description of actual people? Or what?

comment by maladroit · 2018-08-27T05:17:16.238Z · score: 21 (13 votes) · LW · GW
Is all of this really supposed to be a description of actual people? Or what?

FYI this reads as incredulity-as-attack, when you might have instead written 'I'm surprised, because as far as I know, none of the people I've ever met think or behave like that' which would have contained the same critique but come across as 3x less dick

edit: it seems in the spirit of the thread to unroll a little more. Your first comment seemed to say something like 'I'm confused by the words you used,' which when I read it I felt good about that. But then when it was explained enough so that 'I'm confused' was no longer a concern, you immediately switched tacks to be complaining about something else, without acknowledging that your first concern had been addressed or that your second concern was a brand-new complaint.

That made me think that you just didn't like this, and were going to find reasons to keep not liking it. I had a model that neither of your explicit complaints thus far was your true complaint, and that made me feel like you were being uncooperative and so I also felt less-inclined-to-cooperate. Then I checked your words again to see whether they were more compatible with "trying to understand" or with "trying to socially dominate" and if I had to bet I would bet on the latter, and that's where my comment came from.

I guess it's possible I'm missing or misinterpreting stuff. But now you have something to work with if you want it.

comment by strangepoop · 2018-08-30T07:39:30.115Z · score: 5 (5 votes) · LW · GW

Somehow this comment was really inspiring! I'm glad this exchange happened, so maybe I should upvote grandparent too? :P

BTW,

incredulity-as-attack

[not] acknowledging that your first concern had been addressed

We have terms for these! They are, respectively, stonewalling and logical rudeness.

I'm still split on how I feel about jargon, and of course it's good that you didn't use any here, but it does give the concepts you describe some legitimacy (for better or worse). Legitimacy helps especially in cases where such expressions are dismissed as over-reactions unique to you, and are thus assumed to be your responsibility to fix, by some implicit jargon-efficiency argument ("if this were a thing to be concerned about, we'd have a name for it!").

comment by Elo · 2018-08-25T22:10:07.605Z · score: 5 (5 votes) · LW · GW

In extrapolating the writing time on this post, the votes on this post (53) in a few hours since writing and the method itself...

I suspect you have communication abilities that are above the average human and there's more to this method than you are putting into words in the first instance.

For example, hearing the "yes" when someone said "no" to feeling judged.

That's great for you, but unfortunate for anyone replicating. Do you have any comments on this problem?

comment by lahwran · 2018-08-26T00:44:06.072Z · score: 1 (1 votes) · LW · GW

I also find the specifics of the method unclear. When he shared it in a lightning talk a few years ago, the point that humans model each other recursively like this was the useful part for me.

comment by Stag · 2018-08-26T09:41:21.501Z · score: 4 (3 votes) · LW · GW

The complete unrolling of 2.5 (and thus 2.6) feel off if they are placed in the same chain of meta-reasoning. Specifically, Charlie doesn't seem like she's reacting to any chains at all, just the object-level aspect of Alex pegging Bailey as a downer. I can see how more layers of meta can arise in general, but in situations like these where a third person arrives after some events have already unfolded doesn't feel like it fits that model very well - is the claim that Charlie does a subconscious tree search for various values of X that might have caused such a chain of interactions, and then draws conclusions about the baselessness of the 'downer' brand based on that?

It seems that a large subset of issues in situations like these but perhaps more grave is that Bailey does indeed do 2.6 exactly as stated, except it's based on a non-existing chain in 2.5, leading to a quagmire of false understanding.

comment by ESRogs · 2018-08-26T23:24:52.320Z · score: 6 (4 votes) · LW · GW
Specifically, Charlie doesn't seem like she's reacting to any chains at all, just the object-level aspect of Alex pegging Bailey as a downer.

I sort of agree. But in the cases where Charlie knows what happened, you might expect that their evaluation of whether Alex was right to conclude that Bailey is a downer might depend on the full chain of events.

comment by Jirachi · 2018-08-28T16:34:22.650Z · score: 3 (2 votes) · LW · GW

This post seems to be helpful, and it is nice to see that meta-level reasoning is something that happens in ordinary situations, though the exact number of levels of meta reasoning isn't particularly clear.

A point of ambiguity here seems to be when you are supposed to use this as a conflict resolution strategy, though I suspect this may be due to an imperfect understanding of this as you present it. Mostly it's just that my intuition appears to say that this isn't a fully generalized system for resolving miscommunications given willingness to go along with it.

comment by drossbucket · 2018-08-26T10:58:22.580Z · score: 3 (2 votes) · LW · GW

Side note, but I really appreciated the bolded sentences marking the start and end of the 'tiring symbolic reasoning' section.

I normally give up on posts on this sort of topic precisely because I can see that I'm getting into an unknown amount of unpleasant mental effort holding all the "he said she said she said"s in my head at once. This time I could quickly gauge how much of that stuff there was, and it looked manageable, so I persevered.

comment by ryan_b · 2018-09-05T20:39:12.899Z · score: 2 (1 votes) · LW · GW

From the example it looks to me like the disagreement happens immediately, but I am not sure; I may be misreading this because of the ambiguity of the word "bad" in combination with the reality-tracking of the milk spoilage and Alex's feeling judged. What I would expect to see in unrolling the referents for each part is this:

1.1) Alex leaves out the milk for 5 minutes

1.2) Bailey observes (1.1), and feels it was bad.

Unrolling of referents: Bailey felt that Alex leaving out the milk was bad.

1.3) Alex observes (1.2), and feels judged.

Unrolling of referents: Alex felt that Bailey felt that Alex is a bad person for leaving out the milk.

Bolded the difference. This might have been within the compass of the intended meaning, but the value of unrolling the referents is not clear unless it reveals a disagreement somehow, and by my reading it doesn't appear that there is any disagreement in the initial example.

When you gave your real-life example, it appeared that you noticed a likely source of disagreement, and then used the unrolling trick to get the other party to notice that the disagreement was where you already thought it was. Have any of the cases where you used this method so far been ones of mutual bafflement?

comment by kimberchoi · 2018-09-11T16:11:06.952Z · score: 1 (3 votes) · LW · GW

I also agree that things should be written even if the idea may not be perfect, but that imperfection is what helps improve and become a clearer model. And, Bravo on at least the first polemic to kinda support your idea about the levels of metacognition.

AND OMG does this really happen? cuz i think now i can conjecture as to why i was a social pariah. Are feelings the primary mover for most homo sapiens? I thought it might be reason but i guess this is a very specific circumstance and the parties involved do prioritize feelings and assumptions before reason? When we go around acting motivated by feelings, judgings, and assessings how others view us doesnt seem to fit the criteria for distinct layers of metacognitive states. Perhaps if one were to ask why is there judgement in the first place and in the second do i care enough to act on it? At the very least ask, 'do u think that was bad?"

I dont even understand why all these feelings are being volleyed around when they dont have to be. Are these intrinsic cognitive acts or an extrinsic ones? Is this a first person layer or a third person layer or is it the layer of the author's metacognition?

I don't think that some of your feelings (the examples' feelings) count as different levels of metacognition. The first person would still run into a self reference recursion, e.g. the feelings of their feelings and thoughts on their thoughts ad inf.. The only person for whom perhaps more levels of metacognition exists is the third person omniscient. I think that these may prove to be factors when developing ideas about levels and layers of metacognition and perhaps as an actor in this scenario, ironically peel away and simplify the layers.

I hope that this heuristic is not the norm for our kind in society; The amount of hours wasted must be staggering.

comment by Kaj_Sotala · 2018-09-21T18:36:09.634Z · score: 3 (1 votes) · LW · GW
Are feelings the primary mover for most homo sapiens?

Yes. In fact, there's some lines of research suggesting that feelings are the primary mover for all homo sapiens, which seems plausible to me; people who think of themselves as "not feeling-driven" often seem to just have a negative emotional reaction to their stereotype of feeling-driven decision-making, and a positive emotional reaction to what they think of as non-feeling-based decision-making.