Dangers of steelmanning / principle of charity

To me, charitable reading and steelmanning are rather different, though related.

To read charitably is to skip over, rather than use for your own rhetorical advantage, things in your interlocutor's words like ambiguity, awkwardness, slips of tongue, inessential mistakes. On the freeway of discussion, charitable reading is the great smoother-over of cracks and bumps of "I didn't mean it like that" and "that's not what it says". It is always a way towards a meeting of the minds, towards understanding better What That Person Really Wanted To Say - but nothing beyond that. If you're not sure whether something is a charitable reading, ask yourself if the interlocutor would agree - or would have agreed when you're arguing with a text whose author is absent or dead - that this is what they really meant to say.

I prefer "charitable reading" and not "the principle of charity" because the latter might be applied very broadly. We might assume all kinds of things about the interlocutor's words acting out of what we perceive as charity. For example, "let's pretend you never said that" in response to a really stupid or vile statement might strike many people as an application of the principle of charity, but it is clearly not a charitable reading. And that's good - it's really a different sort of thing, whether desirable or not.

Steelmanning, on the other hand, is all about changing the argument against your position to a stronger one against your position. The "against your position" part is left out of some good explanations in other comments here, but I think it's crucial. Steelmanning is not a courtesy or a service to my interlocutor. It is a service to me. It is my attempt to build the strongest case I can against my position, so I can shatter it or see it survive the challenge. The interlocutor might not agree, if I were to ask them, that my steelmanned argument is really stronger than theirs; that's no matter. I'm not doing it for them, I'm doing it for myself.

When you look at it like this, there should be no danger of confusing the steelmanned argument with the interlocutor's original one. The steelmanned argument is properly yours, it is based on the original argument but should not be attributed to the interlocutor even rhetorically. There's no benefit to the conversation from doing that. You're not doing anyone a favor by pretending they said something they didn't.

In a conversation, live or close to live, charitable reading is always the appropriate and virtuous thing to do, but steelmanning your interlocutor's argument might not be. It often is appropriate, but that isn't a given. Remember, the steelmanned argument is your creation and is meant for you, you owe it to yourself to test your beliefs with it, but not necessarily in the context of this conversation. Not because concealing it is an easier way to victory, but rather because what's steelmanned for you might not be steelmanned or even interesting to your interlocutor. Their argument said A, and you may have found a way to strengthen it further to say B, but they might not want to claim B, to defend B, to agree that B is stronger than A. That said, if you do think the steelmanned argument would be useful to them, by all means introduce it, but explicitly as your own. Some phrases that are commonly said in such cases would be: "I see your point here, and I would even add ... but still, I would disagree...", or "You could also say that...", or you can propose a back-and-forth: "I think this is wrong because of... You might want to reply that... But to that, I would say..." In all these cases, the interlocutor is free to agree or disagree with your explicitly introduced steelman.

Now, going to the example in the post, where the ancient Roman chooses to interpret a progressive argument for increasing welfare as "really" carrying between lines the ancient Roman rationale. He is not doing a charitable reading of his interlocutor's words - they would definitely not agree that this is what they meant to say. And he is not steelmanning anything either, because he hasn't strengthened an argument against his own position; rather, he fortified his existing beliefs by manufacturing another fake confirmation. If he were to modify the progressive's argument in some way that would make it harder for him to interpret it in the ancient-Roman sense, that would be steelmanning.

To sum up:

• Charitable reading is always done for the sake of the discussion, to improve its usefulness, to reduce noise, and to avoid conscious or unwitting misrepresentation. It should never introduce anything to the argument that its original owner wouldn't have recognized as what they said. It's always a good idea.
• Steelmanning is always done for your own sake. It always says something new that the original owner of the argument didn't think of or at least didn't say. When put back into the discussion, it should be introduced explicitly as your words. Steelmanning is usually a good idea whenever something important to you is being discussed. Steelmanning every trivial thing is tedious and silly; you're doing it for youself, so you get to decide what should be steelmanned.

I'm reminded of Bret Victor's recent comment on reading Latour:

It’s tempting to judge what you read: "I agree with these statements, and I disagree with those." However, a great thinker who has spent decades on an unusual line of thought cannot induce their context into your head in a few pages. It’s almost certainly the case that you don’t fully understand their statements. Instead, you can say: "I have now learned that there exists a worldview in which all of these statements are consistent." And if it feels worthwhile, you can make a genuine effort to understand that entire worldview. You don't have to adopt it. Just make it available to yourself, so you can make connections to it when it's needed.

That, to me, is a principle of charity well applied. I wouldn't at all say that steelmanning is a stronger form of that -- a rationalist trying to steelman Latour would be like your Roman trying to steelman progressivism. Steelmanning is about constructing what you see as stronger versions of an argument, while the principle of charity is about trying to get into your interlocutor's head under the assumption that whatever they're saying or doing seems reasonable and right to them. The latter is much harder and rarer, in my experience, although that's not to say the former isn't more valuable in some situations.

You describe some real problems with steelmen. I think a first-order defense against them is just to ask whether your interlocutor agrees with your steelman or not.

Quoth Yvain:

I no longer try to steelman BETA-MEALR [Ban Everything That Anyone Might Experience And Later Regret] arguments as utilitarian. When I do, I just end up yelling at my interlocutor, asking how she could possibly get her calculations so wrong, only for her to reasonably protest that she wasn’t make any calculations and what am I even talking about?

I consider steelmanning to be a safeguard against reversing stupidity, especially in political contexts. If my opponent says X, I am likely to say non-X for many bad reasons such as: my opponent defends X using arguments I disagree with.

But if you show me that X can also be defended using arguments I would agree with, then I will be less likely to automatically throw X away, and I will be more able to consider X on its own merits.

Steelmanning is good for understanding "X can also be defended by good arguments", and is dangerous because it provides a bad model of my opponent. (Unless my original model was so bad that the steelmanning didn't make it worse; which wouldn't be completely unexpected in a political debate.)

In your example, the time-travelling Roman would get a completely bad idea about why Obama wants to increase unemployment benefits. But he would get a useful insight about why he might want to support increasing unemployment benefits. It's bad for modelling Obama, it's good for thinking about possible consequences of the unemployment benefits. (If you were an Annona-worshipping Roman, you would want to realize she would be happy about the unemployment benefits.)

Analogically, steelmanning Reactionary ideas is bad for modelling Reactionaries, but it could be good to think about some topics they raise. It may help you identify parts of their set of ideas which may have value for you, outside of the original set.

But this difference is really not obvious, and people are likely to get it wrong. Perhaps it would be good to say it explicitly when steelmanning anything: "This is not why those people defend their ideas, but it's why someone else might find the same ideas meaningful."

The Wikipedia formulation is "write for your enemy", i.e. state their position sufficiently well that they would accept you have stated their position sufficiently well. This is useful in that they are generally present and will let you know in no uncertain terms if you've failed to achieve this. This is only a guideline, as unreasonable opponents do exist and the stretch is so that you can write a better neutral article.

But why didn't A just frame her argument in objective, consequentialist terms? Do we assume that what she wrote was sort of a telephone-game approximation of what was originally a highly logical consequentialist argument? If so where can I find that argument? And if not, why are we assuming that A is a crypto-consequentialist when she probably isn't?

I never thought that steelmanning implied necessarily assuming that A would agree with the steelmanned version. If A says something that seems to have a reasonable point behind it but is expressed badly, then yes, in that case the steelmanned version can be something that they'd agree with. But they might also say something that was obviously wrong and not worth engaging with - but which nonetheless sparked an idea about something that was more reasonable, and which might be interesting to discuss.

In either case, we've replaced a bad argument with a better one that seems worth considering and discussing. Whether or not A really intended the argument to be understood like that doesn't matter that much.

To take a more concrete example, in What Data Generated That Thought?, I wrote:

All outcomes are correlated with causes; most statements are evidence of something. Michael Vassar once gave the example of a tribe of people who thought that faeries existed, lived in a nearby forest, and you could see them once you became old enough. It later turned out that the tribe had a hereditary eye disease which caused them to see things from the corners of their eyes once they got old. The tribe's theory of what was going on was wrong, but it was still based on some true data about the real world. A scientifically minded person could have figured out what was going on, by being sufficiently curious about the data that generated that belief.

If the person giving the original argument is the tribe, the original argument is "faeries exist", and the steelmanned argument is "these people carry the genes for a hereditary eye disease", then our steelmanned version certainly isn't what the tribe originally intended. But what does it matter? Steelmanning their argument still gave us potentially useful information.

There's another way it can go wrong:

"You claim X, which sounds pretty bizarre to me so I'll charitably assume you meant a weaker version X' that fits in my worldview, and I'll forget that you originally claimed an argument for X."

Excellent points. I've never been a huge fan of steelmanning. A couple more:

1. People talk as if steelmanning is inherently a virtue, but in practice they're selective about what they steelman. You won't see many steelmannings of Young Earth Creationism around these parts--or even plain vanilla theism. If people are going to steelman, it would be nice for them to be more up-front about why they chose to steelman this particular argument (or when they're telling someone else "hey why aren't you steelmanning the person you're attacking," be upfront about what that particular argument deserves steelmanning.

2. If you choose which arguments to steelman more or less at random, or for bad reasons, it seems like it's a violation of privileging the hypothesis.

Steelmanning is optimal * when looking for true beliefs about the world **, as long as you're aware that the source of the argument only provided a weaker form of the argument ***.

* In an environment without any resource constraints, which unfortunately never is the case. Still, if you got time on your hands and nothing else to do ...

** Arguments in their maximally persuasive form have more potential to shift your beliefs in the correct direction. Neglecting a potential strong form of an argument is tantamount to ignoring evidence.

*** So steelmanning the cold fusion crackpot's argument may have brought you to firmly believe in cold fusion, that's fine as long as you don't forget that the crackpot still believes in the right conclusion for the wrong reasons (the weak form of the argument), and is as such still a crackpot.

I endorse all of these problems as real. Too much steelmanning blinds to the reality of the rather incompetent civilization and malfunctioning species we live in.

"The dole was there because it made the emperor more popular" and that is the same reason it exists today. Charitable social policies exist primarily to buy votes. Take Head Start as one of many, many examples of failed programs: http://nypost.com/2010/01/28/head-start-a-tragic-waste-of-money/. $166 billion wasted on a program that is demonstrably no help. It seems to be a dismal failure, but continues to exist, because it sounds good and gets votes. The reason why there are so many seemingly failed government programs, is because those programs that are still around are "successful" in the sense that they bought votes for their political proponents. It is like evolutionary biology. Programs that buy votes for politicians "survive", because politicians who support those programs get voted into office. Programs that don't buy votes for politicians don't survive, because there is no one in office to support them.

Switch to Main?

Great article, I hadn't heard this argument before but I think it's a good point. I'll also mention that I think the Ideological Turing Test does a good job of combating some of your worries here, although of course has its own dangers.

I suspect there's a difference between steelmanning as in removing unnecessary assumptions or context, and steelmanning as in completely changing the logical foundation of the argument, just retaining the bottom line proposition, as our Roman seems to be doing. They are both valid, with the second one more vulnerable to the problems you mention, especially #1.

Also, either way, it would be a mistake to take the steelmanned argument and attribute it back to the source of the original argument. This seems to be the cause of your problems #2 and #4, maybe also #3 to the degree it's an actual problem, and isn't really necessary for steelmanning as I understand it. If you want to understand the person, or the culture, then you should mine their raw utterances for all the information you can get. But if you are looking to believe truth, then it helps to strengthen your opposition to the point where they can offer your current beliefs a proper fight, since you can't depend on the opposition doing that for you.

I think to some degree there is a difference between the principle of charity and steelmanning. I'd guess your problems lie more with the principle of charity, but maybe I shouldn't be too charitable in this case?

I think it would be useful to identify subcategories of what people mean by steelmanning and then see if we can approve some of those.

Very good, although I have heard similar arguments (though less elaborated) in conversation. The principle of charity (or steelmanning - never heard that term before) certainly is important, but sometimes it just goes too far. At one seminar I used to attend the seminar leader used to "re-interpret" the most confused and illogical argument, saying - "did you mean so and so", to which the interpretee of course invariably gratefully responded yes (though of course he never in his life had come up with such an interesting argument). The whole thing was a bit of a comedy...the problem was, though, that most people didn't see what was happening (including both the re-interpreter and the re-interpretee), so people got a false impression the re-interpretee was much less confused than he really was.

One wonders if incidents like these contribute to the false impression that academic abilities are roughly equal, something that I discuss here: http://lesswrong.com/lw/jhy/division_of_cognitive_labour_in_accordance_with/

Another thing that's seldom pointed out is that we should use the principle of charity selectively. Some people have a strong track-record for saying interesting things, so if there is something they've said that we think is confused, wicked or false, we should take another look at it and see if we can make more sense of it. Other people have a very poor track-record, and with them it is rather the other way around: confused, wicked and/or false claims is the normal, and we should rather take another look at our interpretation if it tells us they've said something deep and interesting.

Taking arguments more seriously than you possibly should. I feel like I see all the time on rationalist communities people say stuff like "this argument by A sort of makes sense, you just need to frame it in objective, consequentialist terms like blah blah blah blah blah" and then follow with what looks to me like a completely original thought that I've never seen before.

Rather than - or at least in addition to - being a bug, this strikes me as one of charity's features. Most arguments are, indeed, neither original nor very good. Inasmuch as you can substitute them for more original and/or coherent claims, then so much the better, I say.

I thought this was in Main when I promoted it. Any reason not to move it to Main?

As someone who steelmans and interprets other people charitably a lot, I hadn't thought of the problems this could cause. I've managed to change my mind about a lot of things in the past few years; I wonder how much of this is because I didn't have any beliefs I held very strongly before, and don't hold many of my current beliefs all that strongly either.

Original thoughts arriving in the form of difficult-to-express intuitions is a common phenomenon. Early analytic philosophers were struggling with the right way to express the intuition that "greater(7, 5) = true" and "lesser(5, 7) = true" represent the same fact. Now we know that the correct answer is to derive both as consequences of the same abstract model of the relevant entities (such as natural numbers) whose existence is to an extent independent of the language used to describe it. The function of the model is to take the way a language is used and tie it to a specific semantic structure. It is possible to construct an isomorphic model using a different set of linguistic definitions, a different language.

When we are dealing with more subjective models than numbers, I think most people grope around trying to pinpoint which of a set of models that are identical on certain points best suits the criteria satisfying their background mental makeup for the meaning they want to express. George Orwell's advice to let the meaning pick the words rather than words the meaning might be related to this game of carefully picking a meaning first, and then searching for the words that best express that meaning. So while I agree with some of the points raised in the main post, I would suggest that other people not using the same words to make the same argument as you is not a great reason to think it is incompatible with yours in the places that matter to each of you.

It might be worthwhile to examine where two models we're comparing diverge in objective consequences that are important, and where divergence is only a matter of subjective language use. The question of prime significance in the Roman example for me is not whether charity is for the glory of the emperor or for the betterment of the poor, but whether the Roman is in favor of undertaking charity in such a way that appearance of imperial glory is optimized or such that the effects of the social safety net are optimized.

If he says that imperial glory demands the optimization of the social safety net, then our differences might amount to a fight over words, at least over this bit of argumentation, from my side. He may attach importance and place emphasis elsewhere, but unless I abstract away some background information by defining criteria to identify local isomorphisms, no two arguments made by two people might ever be the same. If he says that he wants to optimize imperial glory irrespective of its effects on the poor, then I have bigger things to worry about. It's wrong for me to conflate my argument with his in that case, because the premise that the Roman is making the same argument as me is false in the areas that matter to me. In fact, owing to my misunderstanding, the stronger reasons I provide for "his argument" will in any case be irrelevant to the argument the Roman is actually making.

As long as you exercise careful discernment in determining whether or not differences are meaningful to the argument being made or quibbles over words, and to identify where the differences come from, which parts of the argument are important to each of you and why you disagree, I don't see where the problem is. Once all these provisos and disclaimers are out of the way, feel free to be inspired by the Roman's argument for charity to invent your original Age of Enlightenment argument for charity.

The Roman might even respond with, "Yes, finally you're starting to see sense. Charity is a good thing like I maintained all along. You still don't appreciate the true importance of imperial glory, but you'll come around to my way of thinking eventually." Or if he wants to convince you that charity is a good thing within your system of ethics, if only because he thinks it's important for you to broadcast American glory for some reason, or if he just wanted to challenge your worldview as an impartial critic by asking why you don't support charity if helping others is so important to you, then your own argument might be just the thing he was looking for, irrespective of the alienness of his worldview.

Does that make sense?

It seems to me there are too separate issues.

1) Do you act like other people actually SAID the better argument (or interpretation of that argument) that you can put in his mouth?

2) Do you suggest the better alternative in debates and discussions of the idea before arguing against it.

2 is certainly a good idea while all the problems come from item 1. Indeed, I would suggest that both parties do best when everyone ACTS LIKE OTHER PEOPLE SAID WHATEVER YOU JUDGE TO BE MOST LIKELY THEY ACTUALLY INTENDED TO SAY. So you don't don't then on misspeaking nor do you pretend they argued for some straw-man position. However, everyone benefits the most when they learn why what they actually argued wasn't right,(especially if you offer a patched version when available).

This way people actually learn when they make erroneous arguments but the best arguments on each side are still addressed.

Indeed, I think a huge reason for the lack of useful progress in philosophy is too much charity.

People charitably assume that if they don't fully understand something (and aren't themselves an expert in the area) the person advancing the notion is likely contributing something of value that you just don't understand yet.

This is much of the reason for the continued existence of continental philosophy drivel like claims that set theory entails morality or the deeply confused erudite crap in Being and Time. Anyone who isn't actually an expert in this kind of philosophy feels it would be uncharitable (or at least seem uncharitable) to get up and denounce it as psuedo-philosophical mumbo-jumbo it is. It may seem harmless but the existence of this kind of stuff within the boundaries of philosophy means that less extreme but also wrong views are also not cut out.

Charity is more directly harmful within analytic (logic/math based philosophy as opposed to continental nonsense) philosophy where people frequently make the naive assumption that various theories, e.g., the definite description theory of reference and the baptismal naming theory of reference, are somehow either right or wrong and argue for these positions just as they would argue for the claims about the fundamental theory of physics. Yet, more sophisticated philosophers have frequently realized this entire naive realism viewpoint is flawed. There isn't a real thing meaning, just speech and writing, and thus these theories can only be taken as theoretical tools that help provide a useful framework for organizing patterns observed in speech acts and despite their incompatible assumptions can both be useful as approximations.

Unfortunately, I have observed time and time again that in situations like this the insight isn't passed on since it would be uncharitable to assume the philosophers who publish in this manner aren't really just debating which is a better approximation to help organize patterns in speech/writing.

Similarly charity stops people from being called out when they continue to wrestle in print with problems (surprise quiz etc..) that have a clear correct solution that was given decades ago since it would be uncharitable to assume (as it true) they simply don't have a good grip on the way mathematics can be applied or fails to apply to real world situations.

wrong thread

Not sure if I properly understood the original post - apologies if I'm just restating points already made, but I see it like this.

Whatever it consists of, it's pretty much the definition of rationality that it increases expected utility. Assuming that the intermediate objective of a rationalist technique like steelmanning is to bring us closer to the truth, then there are 2 trivial cases where steelmanning is not rational:

(1) When the truth has low utility. (If a lion starts chasing me, I will temporarily abandon my attempt to find periodicity in the digits of pi.)

(2) When the expected impact of the resulting update to my estimate of what is true is negligible.

No doubt, there is need for some skill to estimate when such cases hold.

This is a very nice post that highlights an important issue that I hadn't previously been fully conscious of.

Thank you for writing that.

To me steelmanning sometimes feels like rationalizing, only instead of rationalizing your position, you rationalize your opponent's. It might be still useful, though.

steelmanning is really just for winning arguments extra hard in the view of your rational audience/cheerleaders.