Confusion about science and technology

My tentative theory is that there's a lot of knowledge that's less formal than science in engineering, manufacturing, and the practice of medicine which makes it possible to get work done, and some fairly effective methods of filtering information that comes from science.

Honestly? Not really.

There are some filters, true. If you work in biochem, for example, you will end up trying to follow someone's instructions in a lab setting, and you eventually learn better than to read certain open access publishers and that certain numbers are red flags. I've personally tried to use someone's image processing code from a research paper, and found that it wouldn't even compile in the version of MatLab they claimed to use, and wouldn't have worked even if it did compile. That ended up being a simple and correctable error, but there are far less forgivable horror stories in the machine learning field. The Schon scandal was discovered when folk tried to replicate things in the lab, and it just didn't work. There have been some efforts to computationally model natural data sets versus fraudulent ones, although I've not yet seen any serious implementation.

There are some structural issues that make the fields of experimental psychology and experimental sociology a little more prone to unreliability and fraud, but they're not as unique to those fields as we'd like. The biggest thing is the time cost of replicating experiments and natural variance in test subjects, and that's an issue in practical medicinal science and even engineering, too.

We didn't find out about Stapel's fraud for years, even though most of his works required no greater research tools than bored undergrads. Vioxx had a test case of eighty million people, which it didn't really need given that the heart disease risks showed up in vastly smaller groups. It took a decade and a half for the fraud in the Wakefield study to finally be retracted. And it's impossible to measure the degree of fraud or error we don't catch.

The practical difference is that we've gotten the larger tests cases, often at the cost of significant amounts of sweat and blood. Sometimes for much lesser sins than fraud or irreproducibility. Where we didn't think through every possible mechanical flaw or didn't test the materials completely enough, bridges collapsed with people on them. Where a simple math error occurs, over a hundred million USD goes exactly the wrong way. And we learn.

The only meaningful filter is reality. Peer review will forgive a great many errors. Physics does not.

I know this may come off as a "no true scotsman" argument, but this is a bit different- bear with me. consider christianity (yes, I'm bringing religion into this, sort of...) in the beginning, we have a single leader preaching a set of morals that is (arguably) correct from a utilitarian standpoint, and calling all who follow that set "christians" by so doing, he created what Influence: Science and Practice would call "the trappings of morality" ...so basically, fast-forward a few hundered years, and we have people who think they can do whatever they like and it'll be morally right, so long as they wear a cross doing it. parallel to the current situation: we set up science- a set of rules that will always result in truth, if followed. by so doing, we created the trappings of right-ness. fast forward to now, and we have a bunch of people who think they can decide whatever they want, and it'll be right, so long as they wear a labcoat while doing it. understand, that's a bit of metaphor, in truth, these "scientists" (scoff) simply learned the rules of science by rote without really understanding what they mean. to them, reproducible results is just something nice to have as part of the ritual of science, instead of something completely necessary to get the right answer

...all of this stuff I said, by the way, is said in one of the core sequences, but I'm not sure which. I may reply to myself later with the link to the sequence in question.

1. Science is amazingly reliable compared to any other method of generating knowledge about the world.

2. It's misleading to talk about science in general in this context, as different sciences have different amount of "kicking back" going on. The more the world kicks back, and the faster it happens, the easier it is for scientists to falsify & iterate. Technology is based on sciences with some of the most testable hypotheses. This doesn't mean (necessarily) that scientific articles are more reliable in those areas, but the unreliable ones are detected faster.

3. We happen to live in a universe where physical interactions are highly local, and forces generally fall off very quickly with distance. On the typical scales we build things at, the physical laws are also very close to deterministic and can be modeled with high accuracy by simplified models with few entities (e.g. looking at a body as a point particle with mass at its center of mass). There's an anthropic argument about why this should be so somewhere in Feynman, I forget where. Taken all together, this allows very high reproducibility, which enables rapid development of technology by iterated experiments.

4. Indeed, engineering is distinct from science and contains vast lore all of its own that's not reducible to basic science.

5. I feel like I've just written a bunch of trivialities.

Science is a funnel of filters; there's no single point - not publishing, not peer review, not cite counts - that reliably distingushes true hypotheses from false ones. But taken together, it works.

Have you tried to test your idea? Are engineers better than scientists at predicting which results will turn out to be true?

In the short term, science may not be all that reliable, but keep in mind that we know that this particular data is unreliable because replication attempts have rendered it highly suspect.

In some areas, science is self correcting over a much longer time horizon than others. For fundamental physics, a comparably easy to correct misapprehension would be unlikely to persist for so long (in a modern day context.) In medicine, it might last much longer (because there are so many hypotheses to evaluate relative to the available research funds and manpower, and there's often inadequate communication between scientists who research stuff and doctors who implement it.) Psychology isn't the absolute bottom of the heap in terms of speed of correction, but it's well down there.

In a nutshell, it's simply this: Stuff that people actually care about are more likely to be under intense scrutiny and thus more likely correct.

If you sift through the chemical literature, I guarantee you will find scores of papers that talk about compounds and synthesis methods that are false or bogus. It's not really important; no one ever cares about them enough to actually investigate the problems. Remember that passing peer review doesn't mean, "There is no doubt about this work." It just means, "Not obviously wrong; here it is exposed to the world and everyone is invited to look at it more deeply."

It's much the same in even more rigorous fields like math and physics. I have personally found serious errors in at least two math papers that have been published in respectable journals. Neither of them had any citations, otherwise people would have found the errors sooner. I was the first citation.

On the other hand, papers that have thousands of citations are more likely to be correct, simply because more people have looked at them.

Pure science seems to be primarily a hypothesis generating mechanism. "Hey look at this thing we found! It might be cool and it might also be nothing." Engineering (or applied science) is a product generating mechanism. "So we tried a bunch of stuff, and now we've got this Somewhat Reliably Working Box. It does This Action when you push a button!"

Science is a self-correcting mechanism. If I try to base my research on a paper that I can't replicate, I'd drop it and base the research on something else. So the reproducibility problem is not lethal.

I'm not involved in any science fields so for all I know this is a thing that exists, but if it is, it isn't discussed much: perhaps some scientific fields (or even all of them?) need an incentive for refuting other peoples' experiments. As far as I understand it, many experiments only ever get reproduced by a 3rd party when somebody needs it in order to build on their own hypothesis. So in other words, "so-and-so validated hypothesis X1 via this experiment. I have made hypothesis X2 which is predicated on X1's validity, so I'll reproduce the experiment before moving forward".

What if there was a journal dedicated to publishing research papers whose goal is purely to invalidate prior experiments? Or even more extreme, a Nobel prize in Invalidation? Could some fields be made more reliable if more people were put to the task of reproducing experiments?

My tentative theory is that you don't need to be reliable to create progress. Exploratory behavior isn't supposed to e reliable - it's just supposed to succeed every once in a while. After the initial success, the reliable folks can take over and keep going until they hit a wall again.

There's an easy way to test this. Just pick a recent technology for which 1) you don't know the history 2) you don't really know how it works. Look around your room (or your head) and find any random example of recent, remarkable technology...and then check wikipedia. You've got to go roughly recent enough that not everyone was using them before 1990.

Prediction (before doing it): The pattern will emerge that the first crucial step was done by a scientist, and then later it was expanded into something useful by others.

I picked Pacemaker, LCD screen and contact lense. Reading these pages, I judge that the first, key breakthrough which made contacts and LCD possible were done by scientists. (In the case of the contact lens, the tech was there but impracticle until science made a breakthrough.). Pacemakers seems to be largely non-scientists.

Just keep picking various technologies at random and see the history to get a sense for the extent and nature of science's contribution to technology. Don't look at the reliability of new science.

Edit: Come to think of it, cardiac pacemaker might actually not be recent enough...

You appear to be getting at the concept of Tacit Knowledge.

Collins showed [10] that a particular laser (The ppTEA laser) was designed in America and the idea, with specific assistance from the designers, was gradually propagated to various other universities world-wide. However, in the early days, even when specific instructions were sent, other labs failed to replicate the laser, it only being made to work in each case following a visit to or from the originating lab or very close contact and dialogue. It became clear that while the originators could clearly make the laser work, they did not know exactly what it was that they were doing to make it work, and so could not articulate or specify it by means of monologue articles and specifications. But a cooperative process of dialogue enabled the tacit knowledge to be transferred.

some fairly effective methods of filtering information that comes from science.

Sure, there is a fairly effective method called empirical testing. You build/make/create it and you see if it works in the real world.

Results from science tend to come as well-defined inside a narrow context and subject to specific assumptions and constraints. Reality is much messier so the scientific results, even if "true", might (and often do) turn out to have no practical value.