Important fact about how people evaluate sets of arguments

Any time you get a data point about X, you get to update both on X and on the process that generated the data point.  If you get several data points in a row, then as your view of the data-generating process changes you have re-evaluate all of the data it gave it you earlier.  Examples:

• If somebody gives me a strong-sounding argument for X and several weak-sounding arguments for X, I'm usually less persuaded than if I just heard a strong-sounding argument for X.  The weak-sounding arguments are evidence that the person I'm talking to can't evaluate arguments well, so it's relatively more likely that the strong-sounding argument has a flaw that I just haven't spotted.
• If somebody gives me a strong-sounding argument for X and several reasonable-but-not-as-strong arguments against X, I'm more persuaded than just by the strong argument for X.  This is because the arguments against X are evidence that the data-generating process isn't filtered (there's an old Zack_M_Davis post about this but I can't find it).  But this only works to the extent that the arguments against X seem like real arguments and not strawmen: weak-enough arguments against X make me less persuaded again, because they're evidence of a deceptive data-generating process.
• If I know someone wants to persuade me of X, I mostly update less on their arguments than I would if they were indifferent, because I expect them to filter and misrepresent the data (but this one is tricky: sometimes the strong arguments are hard to find, and only the enthusiasts will bother).
• If I hear many arguments for X that seem very similar I don't update very much after the first one, since I suspect that all the arguments are secretly correlated.
• On social media the strongest evidence is often false, because false claims can be better optimized for virality.  If I hear lots of different data points of similar strength, I'll update more strongly on each individual data point.

None of this is cheap to compute; there are a bunch of subtle, clashing considerations.  So if we don't have a lot of time, should we use the sum, or the average, or what?  Equivalently: what prior should we have over data-generating processes?  Here's how I think about it:

Sum: Use this when you think your data points are independent, and not filtered in any particular way -- or if you think you can precisely account for conditional dependence, selection, and so on.  Ideal, but sometimes impractical and too expensive to use all the time.

Max: Useful when your main concern is noise.  Probably what I use the most in my ordinary life.  The idea is that most of the data I get doesn't pertain to X at all, and the data that is about X is both subject to large random distortions and probably secretly correlated in a way that I can't quantify very well.  Nevertheless, if X is true you should expect to see signs of it, here and there, and tracking the max leaves you open to that evidence without having to worry about double-updating.  As a bonus, it's very memory efficient: you only have to remember the strongest data favoring X and the strongest data disfavoring it, and can forget all the rest.

Average: What I use when I'm evaluating an attempt at persuasion from someone I don't know well.  Averaging is a lousy way to evaluate arguments but a pretty-good-for-how-cheap-it-is way to evaluate argument-generating processes.  Data points that aren't arguments probably shouldn't ever be averaged.

Min: I don't think this one has any legitimate use at all.  Lots of data points are only very weakly about X, even when X is true.

All of these heuristics have cases where they abjectly fail, and none of them work well when your adversary is smarter than you are.

This doesn't seem to be what I or the people I regularly interact with do... I wish people would give some examples or link to conversations where this is happening.

My own silly counter-model is that people take the sum, but the later terms of the sum only get added if the running total stays above some level of plausibility. This accounts for idea inoculation (where people stop listening to arguments for something because they have already heard of an absurd version of the idea). It also explains the effect Ronny mentions about how "you may very quickly find that everyone perceives the anti-T-ers as being much more reasonable": people stopped listening to the popular-and-low-quality arguments in favor of T.

I could have sworn that there was an LW comment or post from back in the day (prob 2014 or earlier) where someone argued this same point that Ronny is making — that people tend to judge a set of arguments (or a piece of writing in general?) by its average quality rather than peak quality. I've had that as a cached hypothesis/belief since then.

Just tried to search for it but came up empty. Curious if anyone else remembers or can find the post/comment.

Perhaps we can use a model in which an argument’s strength is a linear combination of its max, sum, and mean:

Strength = A x Max + B x Sum + C x Mean

So then the question becomes how to assign the weights.

In theory you could obtain an empirical result by assigning values to the strength, max, sum and mean for various beliefs you hold and their competing beliefs you do not hold, then finding the global coefficient values that best account for the numbers you put in. You could also try this with nonlinear models. If you could get people you consider worthy of emulation to take this study, perhaps you would find that there’s a model and set of parameters that best explains their collective approach to decision-making, which you could then adopt for yourself.

Need to specify context and define more precisely what it means to judge an argument. This post seems to vacillate between high-bandwidth mutual truth-seeking discussions and “social movement”, which is more of a simple broadcast-of belief model, with little room for debate or consideration of arguments.

To the direct question, rationality does not naively aggregate arguments by strength or quantity. It demands decomposing the arguments into evidence, and de-duplicating the components which overlap/correlate to get an update. And what people typically do depends entirely on the people and situations you consider to be typical.

This largely feels true.

But if someone is disposed to believe P because of strong argument X, the existence of weak arguments for X doesn't feel like it dissuades them.

There's a related phenomenon where--separate from what people believe and how they evaluate arguments--your adversaries will draw attention to the most objectionable things you say, and a movement's adversaries will draw attention to the most objectionable things a member of the movement says.

I think it really depends on the situation. Ideally, you'd take the best argument on offer for both positions, but this assumes arguments for both positions are equally easy for you to find (with help from third parties, not necessarily optimizing [well] for you making good decisions). I think in practice I try to infer what the blind-spots and spin-incentives [LW · GW] of the arguments I hear are, and try to think about what world we'd have to live in [? · GW] in order for these lines of arguments to be the ones which I end up hearing about via these sources.

Never do I do any kind of averaging or maximizing thing, and although what I said above sounds more complicated than saying "average!" or "maximize!", it mostly just runs in the background, on autopilot, at this point, so it doesn't take all that much extra time to implement. So I think its a false-dichotomy.

In some sense, one strong argument seems like it should defeat a bunch of weak arguments, but this assumes you're in a situation you never actually find yourself in in real life [LW · GW]^[1]. In reality, once you have one strong argument and a bunch of weak arguments, now begins the process of seeing how far you can take the weak arguments and turn them into strong arguments (either by thinking them yourself or seeking out people who seem to be convinced by the weak-to-you versions of the arguments). And if you can't do this, you should evaluate how likely you think it is you can make one of those weaker arguments stronger (either by some learned heuristics about what sorts of weak arguments are shadows of stronger ones, or looking at their advocates, or those who've been convinced, or the incentives involved, etc.).