Information-generating research projects

I think that one incredibly important task is to give impartial and reliable information about the quality of information. Not even scientific peer-review is a sufficiently good procedure. For this reason, the Meta-Research Innovation Centre at Stanford has started a:

“journal watch” to monitor scientific publishers’ work and to shame laggards into better behaviour.

To do these kinds of evaluations you of course need expertise. But other kinds of evaluations require little but intelligence, good judgment, an impartial mindset and persistence. For instance, we crave for better and more impartial evaluations of the validity of statements made by governments, politicians, companies, and other influential organizations. I have a long-term goal of doing something along those lines. I think rationalists could contribute a lot here.

I have my suspicions about the second point: "Can be done by people who aren't subject matter experts." The whole concept of doing research projects in areas about which one knows little doesn't strike me as a good idea. GIGO is a real problem not solvable by fancy number manipulations.

I do like cross-disciplinary raids and excursions, but they involve bringing the tools, techniques, and insights of one discipline to another field and not just amateurs breezing in.

I think the idea of trying to get people who aren't experts to aggregate and analyze information is an excellent one. I think one good thing to do is to think about what kinds of analyses/aggregations that aren't done at present you could do. What does science/Wikipedia/Google etc not cover?

I think one major problem is that when there is a conflict of interest, no one provides us with impartial, reliable information. That is why I suggest that there should be a place for reliable and impartial criticism of statements. But one could also think of more positive suggestions; for instance for how to take advantage of information technology in a socially optimal fashion. I think that at the moment, special interest groups are blocking us from making optimal use of the new technology. For instance, in my field, the academia, there is a massive resistance to massive online courses because academics fear it will lead to unemployment (which I don't think is true, by the way, but that is beside the point), even though these massive online courses could (or that's what people hope at least) provide students with better education at a lower price.

My point is that in cases where a certain policy P maximizes the total utility, but a certain alternative policy Q leads to much better outcomes to a special interest group, and just a little bit worse outcome to everyone else, Q will often chosen (even though there are no good reasons for it) because the special interest group will care so much more about the issue than everyone else. Hence it's important that someone takes responsibility to work for the common good.

To a certain extent, I think there are think tanks that does the latter, though, but it would be good if one could involve more people in this (for instance on fora or wikis).