Knottiness

post by abramdemski · 2023-01-02T22:13:12.752Z · LW · GW · 4 comments

Contents

4 comments

(low-effort post)

Let's say there's a bunch of factors that influence how well you're able to articulate yourself in a specific domain of your life. 

One factor is simply having the language to express yourself. Another factor is having the practiced perceptual acuity of getting your bearings in that situation -- noticing what's going on in high resolution. 

A third factor could be your ability to apply general rationality skills to this specific domain, such as noticing confusion, noticing sunk cost fallacy, etc etc etc.

A fourth factor is the social equilibrium around communication, in this specific area of your life. For example, communication about pain might be interpreted as a request for help, which has social costs. You could be in an equilibrium where you don't communicate about a lot of types of pain, because you don't want to imply that type of weakness and need for help. Or whatever. 

The social equilibrium is just one factor here, but it's a particularly important one, because often our explicit verbal reasoning is heavily shaped by the social equilibrium, even when we are thinking silently to ourselves. Because social equilibria can be tied to specific areas of life, this means we can get tied up in knots in specific areas of our thinking, even if we possess general cognitive skills for untying those knots. The need to participate in social equilibria creates bucket errors [? · GW], cognitive blind spots [LW · GW], and "separate magisteria". 

As a simple example, admitting that you're not enjoying playing a board game with friends might be socially conflated with a number of things, such as not liking the people you're with, wanting to go home, etc. You might make a calculated decision to play anyway, and not voice your preferences. Over time this decision might become habit, and this might blunt your ability to notice whether you are having a good time with board games.

For example, when asking yourself a question about what board games you enjoy, you might subconsciously substitute it with other questions, like which board games you end up playing the most, or which games other people you like seem to enjoy.

This is even worse because other people might also be stuck in knotty equilibria; so, you might be making the trade-off for nothing.

I'm not pointing this out as part of an argument for people to notice their own preferences in social situations. Indeed, questions of what we enjoy, prefer, want, & desire might be some of the most knotted questions! A direct approach here seems like it could easily go wrong.

(And if you choose a "direct" approach, maybe the more reliable approach would be to say things immediately available to you (ie, observations rather than inferences) -- this helps to escape the frame.)

Sometimes we're stuck in a pattern of behavior due to stale information, which our blind spot stops us from updating. But sometimes we're stuck in that behavior due to a very real trade-off which still applies. This is the critical question determining whether a Gordian approach to knot-untying (ie, violently cut through the knot by doing something different) applies, vs a more gentle and patient approach.

It seems to me like a ton of weird signaling equilibria could be unknotted if only there were another way to communicate the important things. 

For example, imagine you don't want to go to a close friend's party, so you lie and come up with an "important work event" you have to go to. You're presumably worried about them feeling unloved, if you just honestly say that you didn't want to go to the party. If you could have communicated that love in a different way, then perhaps your attendance at the party would not feel so important that you had to lie about it. (See Separation of Concerns [LW · GW] for more thoughts in this direction.)

Anyway, what I wanted to convey was this mental image I had of (a) a sort of "epistemic competence factor", (b) they way it can vary per-domain even if your epistemic competence is very high overall, especially because of the way it can get mixed up with social signaling equilibria. 

So we have this general sort of job of not only improving our core epistemic competence, but increasing the number of different areas in our lives to which we are able to apply this competence. 

4 comments

Comments sorted by top scores.