The End of Bullshit at the hands of Critical Rationalism

It's a nice dream, and I would excited if you could do it, but I don't think it is possible given the reality of the modern sociopolitical situation. What I think you don't appreciate is that at the end of the day, most people really really don't care about building a better world. They care about promoting their own status, defeating their enemies, and justifying their hatreds.

To justify this claim, I'll cite a few Arthur Chu Facebook quotes on the subject of LessWrong and rationality:

Arthur Chu For the peanut gallery -- what people are deliberately dredging up is that I hate the Less Wrong/"rationalist" community precisely because of its "We are nerdy white guys, here to tell you why you are wrong" culture and its intense defensiveness and insecurity (sorry, "immune system") at being called on being a haven for bullshit, including stuff like "Stop saying nerdy white guy, that is its OWN FORM OF RACISM"!

Arthur Chu Oh, and if you don't even know what Less Wrong is, it's basically a nerdy white guy religion that started out as a bunch of people gathering donations to freeze themselves until a Computer AI Jesus can be built and create utopia. And if you've ever heard of it but don't give all your money to it Computer AI Jesus will rebuild you in the future and put you in Computer AI Hell. It's spun off into a whole bunch of other shit since but it's not a group of people that really has any business lecturing people on who is and isn't "sane".

Arthur Chu is especially willing to voice his prejudices, but my strong suspicion is that most other people think the same way. So even if you set up a completely objective and rational truth-finding web site, it would simply be attacked and destroyed by political actors for being a racist religion or for being run by nerdy white guys or whatever.

Argumentative cheaters thrive because their arguments aren't properly scrutinized.

This statement does not pass the "fact check".

People have been repeatedly shown to believe what they want to believe for various reasons including status, affiliation, cognitive dissonance, convenience and many others. They happily overlook and downplay the " fallacies, half-lies, evasions" by the home team while emphasizing those of the opponents/enemies.

The factcheck.org site hardly made a dent in the misrepresentations, and is rarely mentioned as an impartial fact checking site (I do not know whether it is one).

A better question to ask is "how to make people care for accuracy and impartiality?" Eliezer's approach was Hanson's OB-inspired "raising the sanity waterline", eventually evolving into CFAR, with a limited success so far. Maybe there are other options, who knows.

1. deductive fallacies are useful inductive arguments. eg ad hominems - as gussied up under terms like 'conflict of interest' and 'risk of bias' - are excellent tools for evaluating studies.
2. factchecking organizations have been, and still are, being tried; and such criticism forms regular columns in newspapers. Have you noticed it helping?
3. On Bullshit defines bullshit as making claims without caring whether they're true.
4. Politics is not about policy.

This is kind of a tangent to the subject, but seeing someone bring up Critical Rationalism on Less Wrong still brings up some pretty massive negative associations for me. By far the majority of all the mentions of Critical Rationalism on this site, due to monomania and prolific rate of posting, come from the author of the most downvoted post ever to appear on Main, and possibly the most disruptive user ever to frequent the site.

I'm thinking that the website should be strongly devoted to neutrality or objectivity, as is Wikipedia.

In Wikipedia part of being objective means to accurately report what's generally known about a topic and not engaging in original research. I think you will have a hard time to judge fallacies without people engaging in something like original research. Fox news has it's own brand of fair & balanced with isn't exactly the same thing that most people think of when they hear the phrase.

What kind of objectivity do you want for your website?

The status quo bias of one person is another person's Chesterton's fence.

In a better coordinated world in which more people cared about truth there would exist certification organizations that, for a fee, would read an article and if the article met certain standards would issue an "honest argument" certification that could be displayed on the article. Having such a certification would, ideally, attract more viewers giving the author more advertising revenue which, also ideally, would more than pay for the cost of certification.

Monolithic vs subjective As pointed out it's hard to gather everyones input to a single result. Rather than have a single fallacy / not fallacy rating have each user be able to express (and own) whether a statement is fallacious. In a usual case it would have the result of "95.4% people think this is a false dictonomy". However there is valuable information on cross-correlating what arguments pass which evaluator. You could have functionality to "ignore all evaluators that think this is a fair argument". People could also profilate themselfs as being quality evaluators. There is a problem/feature where the standard of the evaluator need not be rigour. You could for example have a profilic evaluator for each major political leaning. Or you could aggregate the information by cross-referencing proclaimed political identity ie "65% of self-identified democrats think this argument is fair"

Applicability vs context Being able to target already produced texts means there would be wide applicability. However I am a little concerned on selection effects on what makes it as a "thing to scrutinize". This kind of thing would be effective about small isolated arguments. However politicians that fit their arguments to fit the situation they are presented in could be wrongly presented in being judged outside of that speech situation. Maybe they know that there are better / more valid arguments for their position but choose to utter those they know their audience can relate to. Bringing those arguments under a close scrutiny would be to partly miss the point. I guess part of the idea would be to apply pressure to always use arguments that could pass harsher standards? However I can see many downsides to that. I would rather have all the arguments to be processed to be explicitly (re)created in the context of the website. Then it would be clear that everybody involved respects the clean play attitude and that the arguments are meant to be elaborate and precise. This could mean that only the core and essential points would be covered. That is, it would not be a witch hunt to harass other medias but be an internal matter.

explicitness vs summary score I would have each argument input in a special language/notation that forces every argument to be explicit and computer readable. The arguments would not be prose but collections and networks of semantic tokens. This would provide human language independence ie french and english users would render the tokens in their language but they would be manipulating the same exact ones when one makes a claim in french it would be accessible to the english user too. With the guarantee of computer readableness you could things like compare the axioms of two users and point where contradict, at such a point a discussion is possible. You could then track how often did those discussion shift opinions and which arguments were effective at which populations / belief bases. This could easily be rendered a tool for anti-knowledge seeking testing which manipulations work the best. If such a reduction is not done the meaning of any end result will be a bit nebulous. Its meaning would depend on the process by which it is produced and it would mask approval of a group in the guise of numeric inarguable data. If the vision of what the "clean play" consist off it could be useful but I doubt there is a single axis that would be so critically important to track. I would rather have metrics that tell stuff but don't give a conclusion than reach a conclusion I am not sure what it tells.

I'm thinking that the website should be strongly devoted to neutrality or objectivity, as is Wikipedia.

The problem is that wikipedia isn't that good at finding the truth about controversial topics.

Such a website would probably be a good idea-- you don't need to have everyone using it, you just need a good-sized audience. Snopes and polifact are doing alright.

To my mind, the interesting question is choosing a manageable task that will make it easy for the site to grow.

This idea really isn't that original. Philosopher, journalists, and others have done this for quite some time, and this has, I'm confident, had significant positive effects on the public debate. Religious people were forced to improve their arguments as a result of philosophical criticism from Hume and others, for instance. Also, even if you might think that the political debate is bad, imagine how bad it would have been if there were no journalists reporting relatively objectively on politics.

My suggestion is thus just to make existing efforts at objective reporting and criticism more systematic and comprehensive: it's not a qualitative leap, but just a quantitative shift. I don't see why that would be impossible in principle (though it may be hard in practice) or why that would not have any effect, given how much our present institutions for objective criticism actually has achieved (in my view).

I have a very good friend who has taught collegiate level debate for forty years. Just before he retired, he did an experiment where he and his students would actually do what you are proposing here and point out, list, highlight and rebut the various forms of argumentative cheating on Facebook and Twitter and see what happened. The result? Of the thirty students in one class, seventeen were banned from groups and had their friends list drop to below 50 people. The remaining students found that there followers dropped, that they received less over all views, and that they became targets of more and more abusive comments with threads that would begin as discussions and quickly de-escalate to that all time internet favorite the argumentum ad hominum. That pattern held good across eight classes and 240 students (give or take). I grant that one professors off the cuff experiment does not equal solid research but it is perhaps a bit disheartening.

And that leaves out both cognitive dissonance and the backfire effect. No one wants to be proven wrong, and many will fight for their worldviews and the facts that fit them, even facts that are not facts.

And given that people also frequently admit to not using the (biased but) existent research tools that are already out there why would they bother to search for and use a research tool designed for something like this? I can see such a site rapidly being used as a weapon by one side of an argument and being shouted down as "fake news" by the other.

Also, just as an aside, I am a student at Duke University- School of Medicine and we are absolutely FORBIDDEN to use Wikipedia or any other crowd-sourced knowledge base as a source in our own researches. As one of the administrators pointed out, at least eleven of the professorial staff and hundreds of the students contributed to Wiki at any given time and it was a common hobby to go rewrite a rival's work.

There have been many proposals like this before. My favorite idea (which I cannot recall the name of right now) was a browser plugin that would overlay annotations onto arbitrary webpages. People could make it highlight certain questionable bits of text, link to opposing viewpoints or data, and discuss with each other whether the thing was accurate. Imagine a wiki talk page, but for every conceivable site.

Pointing out fallacies in others is generally less useful than finding them in yourself. When you see someone committing a fallacy ask if you've made any similar errors.

The picture covers some text.