Comment by LiorSuchoy on How did academia ensure papers were correct in the early 20th Century? · 2018-12-30T02:15:40.307Z · LW · GW

Hi

I am new to this website so please excuse me if I diverge from the rules of discourse.

I think there is an inaccurate assumption in this text. Even though scientific ideas become official through publishing in scientific journals it is not the only platform to discuss science. I will suggest here other mechanisms to root out errors and discuss scientific ideas based on my experience as a researcher in Earth Sciences. I know other fields use different methods though, to my understanding, these methods are fundamentally similar.

Since Earth Sciences suffer from chronicle lack of evidence Earth scientists are bound to come up with contradicting ideas. Most of these ideas are rooted out in the initial stage of fitting the model to existing evidence. The rest are then presented not in papers but as presentations in seminars, workshops and conferences. Since most of the problems are only handled by a small amount of researchers these meetings are very effective in highlighting all the existing models of the discussed issue. Naturally basic flaws will rise at that stage. Only afterwards ideas tend to be submitted to publication. Much can be written about the peer review system but if it is done well it will also point out weak claims and flaws in logic. Though, due to the aforementioned stages, it is usually focused on technical details or gaps in background and references rather than the baseline logic. Lastly, a published work is not the bottom line of an argument. Other papers may be published with other models or new evidence that will make older publications irrelevant. Since those papers will have to go through all the previous stages as well it is likely that they will result in either new agreed baseline or agreed disagreement that requires further evidence. In both cases it is unlikely that the argument will generate further publications (and thus may explain your findings). The exception is when new evidence emerge in a case of agreed disagreement. I find it likely that all sides of the argument will try to use their models to explain the new evidence with varying degrees of success. Normally when a model stop fitting the evidence it is simply ignored in future publications. So it may still appear to exist as a possible explanation but it is effectively not there. In most cases to find the agreed models for a problem researchers turn to review papers. These are meant to present the issue, all possible explanations and the strong and weak points of each model.

So, to sum things up, I think the process you are looking for is the one done under less official interactions. Theories are confronted in meetings and such. Less accurate theories are simply ignored in future discourse.

There are clear disadvantages to this system. The main one is that it is clearly sensitive to manipulation by strong people on key positions in their field. A theory can be ignored not because it is wrong but because its publisher is not popular. This may be the strongest argument to why publications that are agreed to be wrong should not be changed or commented after the fact.

I hope this makes sense and answers your question.

Lior