Taking it Private: Short Circuiting Demon Threads (working example)

Thanks for writing this. I think the bullets under "things I learned" is a reasonable description of the points I was trying to make.

I would describe my current overall position somewhat differently - it's not that I think people can't be poorly calibrated in this direction, but I do think "intuition designed for Dunbar numbers therefore useless in the modern world" is way too big of a pendulum swing. My model here - Benquo was saying this in the original demon thread post and I agree - is that the social intuitions relevant to demon threads are mostly about what is or is not entering common knowledge, and my sense is that these intuitions actually scale reasonably well with the size of a social group.

Because while it's true that modern social groups can be much larger than Dunbar it's also correspondingly true that we have much better social technologies designed to produce common knowledge among these larger groups, such as highly visible blog posts, community events and speeches given at them, etc. People have a lot of experience with these social technologies and basically understand their effects on common knowledge, e.g. most people can basically keep track of what their local social norms are, what they are and aren't allowed to say and wear, etc. and are sensitive to changes in these things.

I agree that the weirder your goals are the more you need to think about other things in addition to social reality, but that by no means allows you to just forget about social reality.

A report of my own recent experience taking a conversation to private chat:

As proposed in this subthread, zulupineapple and I took to a private chat (IRC via my private webchat server) to continue our recent argument about utility functions.

We chatted continuously for just under 3 hours. We attempted to cover all or most of the points brought up in the LW thread where the disagreement occurred, and did indeed touch on most of them. Our conversation was fairly civil, and I did not experience it as heated or emotional at any point, nor did it seem like zulupineapple did either (he can post his own evaluation as a response, of course).

Unfortunately, I can’t say that we resolved any of our disagreements. As far as I can tell, we made minimal progress in reaching any true understanding of each other’s points (which is to say, I started out believing zulupineapple to be wrong and/or confused in certain specific ways, and I still believe him to be wrong and/or confused in basically all of those ways; presumably he has a symmetric opinion of my views). Nor does it seem like we reached any agreement on any of the claims on which we disagreed. Neither of us was able to convince the other of anything.

On the object level of our actual disagreement, this exercise can only be called a failure. (I have some meta-level thoughts, which I may post about separately at some point.)

(As a side note, from a technical perspective, taking the conversation to private chat was quite easy, thanks to the aforesaid private webchat server which I already had in place. The only logistical difficulty was in scheduling a time that worked for both of us, as we are in very different time zones; however, we found a mutually acceptable block of time without too much difficulty.)

This is sort of a reply to Said (who asked 'what if they don't want to take it private?'), as well as zulupineapple (who asked 'why would taking it private help?'). But my impression from the overall thread here is that I skipped some important prequesites for this post to make sense. It seemed useful to spell them out in a comment separate from the ongoing conversation.

The "take it private" suggestion is predicated an approach of "make sure the conversation is collaborative, rather than working at cross purposes'. I think it's easier to do that one-on-one than in public. Going private is meant to reduce the "difficulty setting" of maintaining a collaborative conversation.

My short answer to "what if they don't want to take it private" is to credibly show that you are fundamentally on the same side, trying to figure out the same thing together, and that it's worth both of your time to hash this out.

If that made sense, you can stop reading. If not, here's some background pieces of my model:

1. When working at cross purposes, conversations make less progress than if the people are trying to help each other accomplish the same goal. (for a metaphor, imagine two people tugging ropes attached to a boulder in opposite directions - neither will move it far. If you can get them tugging it in the same direction they'd move much faster. If they can't tug in exactly the same direction, maybe they can at least find an angle 30 degrees apart, instead of 180 degrees).
2. There's different ways to work at cross purposes - perhaps you are literal political adversaries, perhaps you're confusedly approaching different aspects of a problem from different angles for different reasons (i.e. one person's focused on "Is A true?" and the other is focused "if A, then B, and relationship between A->B is interesting whether or not A is true"). Or someone might be at cross purposes with themselves and not sure what their goals are.
3. If you are genuinely confused, annoyed, scared or angry, it's easier to stay (or get into) a collaborative mindset by cultivating curiosity (both about your partner and the subject matter) and empathy - this both reminds you that they're a real person who's also trying to do things that make sense, and can keep you oriented around figuring stuff out rather than winning. (This may be the most non-obvious / magical-seeming step, and I don't think I can describe it succinctly. If this is the sticking point I'll try to explain more)
4. If you and your main conversation partner are having a productive conversation, but it's about something that people tend to get annoyed, confused, scared or angry about, there's a risk that other people might derail the thread - asking questions about unrelated topics, making sarcastic barbs that gets everyone on edge, etc. Taking the conversation private can be a way to avoid extra people doing that, and/or avoid wasting everyone's time on a conversation that's more complicated than it needed to be.
5. If you are your partner are struggling to have a productive conversation (because you are annoyed or angry or confused at each other), but are both trying to figure something out together, then random bystanders interjecting might make it harder to cultivate curiosity/empathy/understanding/respect for each other to retain the good faith that the conversation is worth having, or even simply focusing on the topic at hand. So going private may help avoid bystanders making an confrontational conversation worse.
6. In public, words often carry tribal overtones whether you want them to or not - if someone criticizes me in front of other people, there's a possibility that they're trying to make me look bad (or, unintentionally making me look bad whether they're trying to or not). So I feel a need to defend myself, or to make sure that I have a clever comeback or something. Whereas in private, there's no audience, so it's easier to make criticisms that are just about the object level criticism.

Given all this... if you're trying to take a conversation private with this particular goal in mind, then a prequisite is for both people to feel like the private conversation is an option offerred in good faith - that if you both spent a bunch of time talking privately, you'd both be putting effort in that'd make it worth the time. This requires showcasing that you are trying to be on the same side.

I think the best tools to communicate this (while also practicing skills that are useful regardless) are trying pass ideological turing tests, demonstrate helpfulness, and cultivating curiosity and empathy (I think this post by Brienne is a useful introduction).

If you've made a habit of summarizing what you think another person's viewpoint is well enough that they think they could have written the comment, they'll have an easier time believing that you're on enough of the same page to collaboratively seek the truth.

If you don't understand the other person's viewpoint, and you trust them that they're at least talking about something worth your time to try an understand, that's when curiosity, and helpfulness empathy come in. Try your best to understand, show them the work you've done to try to understand. (This is sort of like, when asking a question on Stack Overflow, it's good to show what steps you've taken to solve the problem yourself - it both shows that you're not just asking them to do your job/homework for you, and it also helps orient them around what you've already tried, so they can explain better).

This all presumes that this is something you want - if you don't think this is actually worth the time, it may well be the right call to bow out. (although if you want to preserve a reputation as someone worth talking about difficult things with, you may want to make an effort to have the other person feel respected as you leave)

If the goal is for conversations to be making epistemic progress, with the caveat that individual people have additional goals as well (such as obtaining or maintaining high status within their peer group), and Demon Threads “aren’t important” in the sense that they help neither of these goals, then it seems the solution would simply be better tricks participants in a discussion can use in order to notice when these are happening or likely to happen. But I think it’s pretty hard to actually measure how much status is up for grabs in a given conversation. I don’t think it’s literally zero - I remember who said what in a conversation and if they did or didn’t have important insights - but it’s definitely possible that different people come in with different weightings of importance of epistemic progress vs. being seen as intelligent or insightful. The key to the stickiness and energy-vacuum nature of the demon threads, I think, is that if social stakes are involved, they are probably zero-sum, or at least seen that way.

I have personally noticed that many of the candidate “Demon” threads contain a lot of specific phrases that sort of give away that social stakes are involved, and that there could be benefits to tabooing some of these phrases. To give some examples:

• “You’re being uncharitable.”
• “Arguing in bad faith.”
• “Sincere / insincere.”
• “This sounds hostile” (or other comments about tone or intent).

These phrases are usually but not always negative, as they can be used in a positive sense (i.e. charitable, good faith, etc.) but even in this case they are more often used to show support for a certain side, cheerleading, and so on. Generally, they have the characteristic of making a claim about or describing your opponent’s motives. How often is it actually necessary or useful to make such claims?

In the vast majority of situations, it is next to impossible to know the true motives of your debate partner or other conversation participants, and even in the best case scenario, poor models will be involved (combined with the fact that the internet tends to make this even more difficult). In addition, an important aspect of status games is that it is necessary to hide the fact that a status game is being played. Being “high-status” means that you are perceived as making an insightful and relevant point at the most opportune time. If someone in a conversation is being perceived as making status moves, that is equivalent to being perceived as low status. That means that the above phrases turn into weapons. They contain no epistemically useful information, and they are only being used to make the interaction zero-sum. Why would someone deliberately choose to make an interaction zero-sum? That’s a harder question, but my guess would be that it is a more aggressive tactic to get someone to back down from their position, or just our innate political instincts assuming the interaction is already zero-sum.

There is no need for any conversation to be zero-sum, necessarily. Even conversations where a participant is shown to be incorrect can lead to new insights, and so status benefits could even be conferred on the “losers” of these conversations. This isn’t denying social reality, it just means that it is generally a bad idea to make assumptions about someone else’s intent during a conversation, especially negative assumptions. I have seen these assumptions lead to a more productive discussion literally zero times.

So additional steps I might want to add:

1. Notice if you have any assumptions or models about your conversation partner’s intents. If yes - just throw them out. Even positive ones won’t really be useful, negative ones will be actively harmful.
2. Notice your own intents. It’s not wrong to want to gain some status from the interactions. But if you feel that if your partner wins, you lose, ask yourself why. Taking the conversation private might help, but you might also care about your status in the eyes of your partner, in which case turning the discussion private might not change this. Would a different framing or context allow you both to win?

I notice the term "demon thread" feels a bit too loaded for me to actually use. It is also somewhat misleading given there is such a thing as a benign demon thread.

It's probably too late in the game to alter the term though.

Step 1. Make it easy and common to take a conversation private if someone is feeling annoyed/​threatened/​angry/​etc

Do people actually agree to offers to do this, if they are not already personally acquainted with their interlocutor? My experience is that they do not.

How do you propose to deal with people refusing or ignoring offers to take a conversation private?

I would appreciate an elaboration or restatement of "ii. Avoid bundling normative claims with descriptive claims." - I felt like I was understanding what you were saying but then "My point was more like: Arguing on the internet about the relative status of things is not effective altruism" felt like a nonsequitur, so I suspect I was misunderstanding the entire section.

"Is it a demon thread?" is more about "there is some weird force compelling more and more people to argue, raising tensions" than it's about "the argument is counterproductive or going in circles"

I had assumed it was both. Frankly, I don't care at all about the first kind of demon threads. If you entered a demon thread where nothing is actually being discussed, and ended up wasting your time and energy, that's your own fault. On the other hand, when people with good intentions start discussing a meaningful topic, but then end up going in circles and become increasingly frustrated, that is a problem that needs solving. To be fair, most demon threads may belong to both kinds. However, I feel that your solution mostly just helps with the first kind.

By the way, do you think there are demon threads going on in Kenshō that we could study?

Also, it just struck me that I don't understand, in what way is your disagreement with Qiaochu demonic? I don't feel that compelled to join, and I don't really see you going in circles. Maybe you resolved the thread too soon, before it became a good example?