On Automoderation

post by Zvi · 2017-03-19T02:56:58.000Z · LW · GW · 0 comments

Follow Up to: Trio Walks, Duo Talks

Response to: Automoderation (Ferocious Truth)

Hat Tip: Automoderation (Agenty Duck)

In my post Trio Walks, Duo Talks I laid out the thesis that the key to a high-quality discussion with more than two participants is to organize the group such that it takes on many of the features of a two-person conversation, because the back and forth between two parties allows complex ideas and paths to be considered and the discussion kept on track, whereas conversations not so organized render this extraordinarily difficult. One implication of this is that a good interaction relies on situational inequality. At the end, I did note that group discussions sometimes need to be run with an emphasis on equality, so everyone can contribute and feel included, but that the results tend to be disappointing.

The Columbus rationalists have developed a system designed to maximize what is good about an equality-based conversation. The system is well-thought out, well-presented by J Thomas Moros and has many elements that seem correct to me, so please do read the whole thing. I especially appreciate the spirit in which it is offered:

This post was written in the hope that other rationalist communities will find it useful and spread and improve the system. Take whatever parts seem useful and apply them as seems best.

Write-ups of such systems are valuable even if the systems themselves prove not to be, because they provide insight into what people are thinking about and trying. I want to see more like this!

Their write-up starts out with its justifying thesis that equality is the path to truth:

Group discussions can easily become unwieldy. Certain individuals have a tendency to dominate the conversation, limiting others’ ability to contribute. When the goal is collaborative truth seeking, this is counter productive. Those who are excluded from the discussion may be exactly the ones with the necessary insights to advance the dialogue toward the truth.

…

Automoderation embodies a wait rather than an interrupt culture. Interrupt culture may be fine for a causal fun conversation, but collaborative truth seeking is aided by a wait culture.

There are many bold claims here that I do not agree with, but I approve of stating such assumptions up front and explicitly even when justification is not provided, as this makes it clear where the system is coming from. Automoderation presumes that given a person’s inclusion at all, that person should be presumed an equal contributor, and that some people dominating the conversation is bad. The Duo Talks system implicitly assumes the opposite, and says that some people (temporarily) dominating the conversation is often actively good! While there is the risk that this will cause important insights to be missed, those who feel they have sufficiently important insights can still speak up, and trying to balance the scales seems far more likely to effectively cause important insights to be missed or never thought of in the first place; the prior over locations of valuable insights is highly uneven in the best of times, and often becomes more uneven with the discovery of some of those insights rather than more equal.

Automoderation also assumes that wait culture is better for collaborative truth seeking than interrupt culture. It even gives ‘promoting wait culture’ as one of its advantages. If you had to pick one of these two approaches in its purest form, without trying to fix the associated problems, I would be inclined to mostly agree that wait culture is the way to go. Waiting is key to a quality discussion, to allow people to gather and to finish their thoughts. The problems with interruptions, and interrupt culture, are at the center of the Duo Talks system. Automoderation says that the solution to this is for everyone to wait their turn. Duo Talks responds instead that the solution is to protect circumstantially key participants from interruption, both by raising the bar for interruption and by giving them a way to ‘recover’ from that interruption and still get where they were going. What Duo Talks does not want to do is stifle action and lose access to a lot of implicit communication and knowledge by making all of this too explicit, because one of the big purposes of Duo Talks is to increase the bandwidth of the conversation. Automoderation attempts to compensate for this with innovative hand signals and a rigid priority order.

Automoderation seems on first reading to be an excellent iteration on an equality-based, wait-based discussion system, and has components less formal systems may want to steal.

Automoderation: My Summary

The fundamental idea of Automoderation is that by default discussion goes in a circle, giving everyone an equal opportunity to speak, but with priority order for special types of statements. The rest are some extra hand gestures to allow more explicitness in silent communication, and some special case logic to keep priority at the proper place in the circle when questions are being asked.

Here is the list, with the bottom (Meta) having top priority, and the top (Change Topic) having low priority:

The extra hand gestures are thumbs up/down for approval/disapproval, the OK sign, which means you are paying attention and interested, and the palm down hand pointing towards the speaker while fingers are squiggled, which here means “I feel you” or “this resonates with me.” That last one deserves its own post, perhaps in combination with a few other similar signals, as it keeps having interestingly different meanings to different people, and indeed was on my list of ‘topics that want to be posts.’

My basic judgment is that having an explicit system for giving priority to Meta Points and Clarifying Questions is clearly a good innovation, whereas Probing Question as its own category (which they are considering eliminating as not useful) illustrates what is bad about this type of system/discussion. It is generally true that Meta Points (as in, ‘we need to open a window’) are allowed priority to interrupt discussion if a reasonable time to do so exists, and it seems good to have a signal that you have such a point. The same goes for clarifying questions. If something is unclear it usually needs to be cleared up right away, if it is going to be cleared up at all. That does not mean that every term or idea that is unclear to any participant needs to be clarified, but if someone decides it is necessary, right away is best. Again having a hand signal is valuable. I would even go further than the original design, and say that these are priority interrupts, with the person speaking getting control of the floor back afterwards. If there is an opening, the point shouldn’t wait. If there is no opening, the signal then tells the person talking to stop, at the end of their current sentence if possible, and resume after resolution.

I also mentioned the basic raising of one’s hand in Duo Talks. One of the advantages given for the Automoderation system is the ability of someone in a non-automoderated conversation to raise their hand to indicate they want to speak and are not being given an opportunity. I think this is just good. It sends the message ‘I have something to say and I feel it is important that I say it, but I am not being given a natural opportunity, so please place me in the queue as soon as possible even though we don’t normally have a queue at all.’ This gives those who are otherwise being shut out of an interrupt equilibrium the chance to speak when they need to, without giving up on the advantages of a naturally flowing conversation.

Change Topic, which they are considering turning into a signal rather than a request to speak, I see as worryingly explicit for something so inherently rude. The existence of a binary signal could effectively crowd out implicit signals and force people to send the explicit signal, since the lack of the explicit signal is evidence and once you have an explicit signal there is less motivation to pay attention for the implicit ones. This then results in an awkward decision (since I am strongly guessing the explicit signal is often socially costly, even if there is a convention that it isn’t, but other dynamics push for sending the signal quickly), and forces people to either give the signal with a low threshold, in which case weak opinions get to dominate strong opinions, or give the signal with a high threshold and often have a null signal crowd out a useful one. There is also the question of what counts as a change in topic. I do see the value in communicating this desire explicitly, especially in groups where implicit signals are unreliable, so I am somewhat sympathetic.

To me, Probing Question illustrates the core issue with the whole system. Probing Question is given lower priority than Expand, where Expand is pretty much anything someone wants to say that is relevant to the subject at hand. That means that probing questions only get asked when no one wants to make a comment. Back and forth exchanges are considered bad (again, the mindset of equal time as inherently valuable) so they have the lowest priority. In practice, that means probing questions rarely get asked and Columbus may eliminate the category to save on complexity costs.

It would be hard for me to disagree more with this perspective. Probing questions and back and forth discussion are the hallmark of a high quality conversation. The Duo Talks system essentially maximizes for probing questions assuming good things will then happen. The whole goal is to dig deep. If no one is asking probing questions, my experience says the chance of discovering new and important things goes way down. If it was practical, I would go so far as to give probing questions priority over expansions, but due to how people would respond to that incentive, this is not viable.

Also illustrative is this, which is listed as one of the system’s advantages:

Many raised hands and other signals can indicate to a speaker that others wish to speak and it may be a good idea to bring their current remarks to a close to allow others to speak. This is made more palatable by the knowledge they will have another chance to speak when it comes around the circle again.

If I see many hands raised, all of whom will be given the chance to speak before I can speak again, I have no expectation that I will be able to speak again before the topic has effectively mutated enough to move past what I want to say, and I would be inclined to say more rather than less if I feel my point is important. If only one hand is raised, that can actually makes me more willing to yield rather than less. Many hands raised does indeed give the message ‘get on with it’ but in a noisy and potentially bad way; I worry that the people who will be cut off this way are those who are saying something more valuable and interesting – they gave everyone else an idea!

The Automoderation write-up does acknowledge the biggest disadvantages of its system, even if we disagree on their importance; first on the list of disadvantages is the inability to enable back and forth. The suggestion is that a moderator can sometimes help with this, essentially saying that one needs to know when to stop using the system, which seems right except that I see wanting back and forth as the default rather than the exception. On similar notes, it is observed that topic drift is difficult to prevent.

I find the sizing and related applicability notes at the end to be especially interesting:

• Automoderation breaks down in large groups. While it has been used with some success in groups of 15 to 20 that was only because not many participants actually wished to speak. Had all of them wished to speak, it likely would not have gone well.

• Automoderation is probably best suited for groups of 5 to 10 people.

• Automoderation is best suited for groups where all members are cognizant of the degree to which they can genuinely contribute to the discussion.

These size estimates match my experiences and intuitions for equality-based discussions. You can have a group of 10 and give everyone a fair chance to speak, have most of them actually speak, and end up with something worthwhile. A group of 15 is pushing it and 20 is impossible, unless the bulk of the participants are silent. It is great when we hold a discussion meetup in New York and get 20 people, I wish we could do that reliably, and at least half of the 20 are reliably silent any given night with that kind of turnout, but it still feels overwhelming. Of course, if everyone understands that you talk once a year, you can sort of do 400, but that is clearly not a participatory format.

The last note is worth pondering as well, since it is a key to and a central problem for all good discussions. Often the right thing to say in a conversation is nothing, or very little, or at least not too much. I definitely struggle with recognizing this and acting upon it, and am grateful to those friends who remind me of this from time to time. The other side of the coin, how to make sure others are made aware when they are talking too much or out of their depth, is hard. At best, it is an advanced social skill. Automoderation makes this problem harder rather than easier, by setting an implicit expectation that everyone can contribute equally. It also raises the question: If everyone knows how much they should be contributing, do we need an explicit framework at all?

While this post contains a lot of disagreement with Automoderation and its underlying ideas, that is because I feel the system is sufficiently worthy and well thought out as to be worthy of this level of attention and engagement. As such things go, it is pretty freaking cool, and I don’t want the tone of the details to give the wrong impression on this front. I definitely intend to steal a bunch of the good stuff.

P.S. A side note: There is an underlying pattern here that by reducing bandwidth and flexibility in exchange for clearer rules and the ability to maximize a few explicit target metrics, one wins when the target metrics describe the important thing, but when the metrics miss something, or keep you tied to a local maxima, you lose, and you can lose hard. I keep seeing similar things everywhere lately, and am working on making the central point but finding it difficult. Goodheart’s Law is an even worse problem than you can think it is.

0 comments

Comments sorted by top scores.