Why is "Argument Mapping" Not More Common in EA/Rationality (And What Objections Should I Address in a Post on the Topic?)
post by HarrisonDurland · 2022-12-23T21:58:18.870Z · LW · GW · 2 commentsThis is a question post.
Contents
Answers 4 TAG 3 Slider 3 Garak None 2 comments
I recently published an article in the Georgetown Security Studies Review (GSSR) on the use of "argument management systems" (e.g., Kialo) for the complex debates that arise in fields where it's often impractical to resolve disagreements through standard empirical methods (e.g., RCTs). I've long been confused why this method of discussion is not more widely supported in EA and Rationalist circles, and am considering adapting my GSSR article to AI policy/safety research (or Longtermism more generally, which various people have criticized for being too speculative/theoretical rather than being based on empirical tests) and posting it here. However, before doing that I would love to get a sense of people's reasons for skepticism or apathy towards such methods, so I could potentially address them in the post.
For what it's worth, I have seen Leverage Research's report on the topic, and I am aware of the criticism that "argument mapping" (in some formats) is overly formal and too complicated. (I plan to respond to these points)
In short, I expect my argument to be fairly similar to what I laid out in my GSSR article: the way we currently present arguments (i.e., predominantly through prose/paragraph text) seems rife with points of failure and inefficiencies, especially given how debates are often not linear but rather branch and have cross-cutting points. In fields like international relations and peace/conflict studies I've repeatedly encountered instances where people fail to (seriously) address existing counterarguments, and more generally it is hard for audiences to determine who has or hasn't addressed counterarguments. In contrast, I think that making one's arguments more explicit and keeping track of the arguments in a more-searchable and more-permanent format than "memory" or prose would help to mitigate some of these problems.
To me, better methods of argumentation seems like a natural extension of norms that promote statistics/experimental methods in science, but thus far I've found the EA/Rationalist communities fairly lukewarm towards the ideas (even if they are more receptive on average than the general public).
Answers
Argument mapping systems exist , in the sense that code for them exists, but have repeatedly suffered from lack of quality content.
When I was considering whether to try to make one happen the main reason was that users would need to treat it like learning a new language. That makes it rather high barriers and high payoff.
The other thing was that it would be murky whether "lets just gather more data / user activity" is done enough or will never get to a critical point. Narrowness makes it so that it actually gets used but makes it less exciting as a thing to have.
I really like the idea of such an argument mapping system. In the early days of wikipedia I thought it might be a natural extension of the wikipedia project to have a debate-plattform attatched to it. So its interesting for me to see that a platform like Kialo exists - I didnt know about it before your post, thanks.
I hold the same opinion as you do, that many arguments are very complex and when you try to get a good impression of the state of the debate it is a complicated task, because its hard to find good, exhaustive and well structured debates. And a platform that systematizes this important task is very valueable for people and society in my opinion.
2 comments
Comments sorted by top scores.
comment by Unnamed · 2022-12-24T18:34:25.596Z · LW(p) · GW(p)
The thing I'd most want to see in an article in favor of argument mapping is examples of well-mapped arguments.
My sense (as of several years ago, which is the last time I thought much about this) is that the idea of argument mapping sounds promising, but it hasn't worked well in practice. The various attempts to create argument mapping software & communities haven't led to arguments being captured well.
It could also be interesting to see smaller/narrower examples, e.g. of a single claim within an argument rather than of a whole well-mapped argument. Especially if you can highlight a problem that many argument mapping projects have, give a small/narrow example of it, and then talk about how your preferred approach to argument mapping tries to do better, with a small/narrow example of what that looks like.
comment by johnswentworth · 2022-12-24T18:31:36.222Z · LW(p) · GW(p)
I view this question in the same cluster as:
- Why do we use text to specify programs? Why not some other interface/data structure?
- Why do we use text to write papers or blog posts? Why not some other interface/data structure?
The key relevant property of "text", here, is the serial format - i.e. it's naturally represented as a list of characters, one after another, as opposed to e.g. a tree or DAG or graph or photo or something more unusual. (Presumably the serial structure of text is itself inherited from the serial structure of speech.)
For all these questions, it indeed seems that some better interface/data structure should exist. The problem is that people trying to take on this sort of project usually just pick some obvious data structure (like e.g. a tree of claims and subclaims, in the case of argument management), without carefully examining the structure of both the visible content and peoples' thoughts in the domain, figuring out what data structure naturally fits that content/thoughts, experimenting and iterating on the data structure choice, etc. And then we end up with an interface/data structure which still needs about as much shoehorning as text in order to express what we want, but users have to relearn how to use it. Whereas a "natural" data structure would, ideally, support all the things people want in fairly obvious and intuitive ways.