Too Much Effort | Too Little Evidence

When lacking evidence, the testing process is difficult, weird and lengthy - and in light of the 'saturation' mentioned in [5.1] - I claim that, in most cases, the cost-benefit analysis will result in the decision to ignore the claim.

And I think that this inarguably the correct thing to do, unless you have some way of filtering out the false claims.

From the point of view of someone who has a true claim but doesn't have evidence for it and can't easily convince someone else, you're right that this approach is frustrating. But if I were to relax my standards, the odds are that I wouldn't start with your true claim, but start working my way through a bunch of other false claims instead.

Evidence, in the general sense of "some way of filtering out the false claims", can take on many forms. For example, I can choose to try out lucid dreaming, not because I've found scientific evidence that it works, but because it's presented to me by someone from a community with a good track record of finding weird things that work. Or maybe the person explaining lucid dreaming to me is scrupulously honest and knows me very well, so that when they tell me "this is a real effect and has effects you'll find worth the cost of trying it out", I believe them.

Lucid dreaming is actually an interesting one where I always have to start an introduction to it with "It has been scientifically proven, I am not crazy or part of a cult.". Even then I sometimes get sceptical responses. In most cases, I don't think I would be able to communicate it at all if it had not been scientifically proven. Meditation is another one that is highly stigmatised because of its associations with (in many cases demonstrably) crazy claims and.. "thrown out with the bath water" though this seems to be slowly changing as more and more studies are showing its benefits.

These are two instances where scientific evidence has surfaced so it is easy to talk about. They are good as indicative examples. The post is about experiences that (assuming they exist) have not yet enter the area discoverable by our current scientific tools.

But you could always be missing something. Something can be true even though all the evidence looks bad. But something can also be true if there is no evidence (either good or bad). Something can be true even if the only evidence is fraudulent. Something can be true even if the only evidence for it is that you asked your 6 year old child for an example of science and he told you it.

Why don't you wonder about any of them being true, instead of just wondering about the case where wishful thinking comes into play?

"But since we live in a world where people constantly offer anecdotal evidence to support their claims of unexplained subjective experiences rational people tend to ignore them."

Stanley Jaki tells this story:

Laplace shouted, "We have had enough such myths," when his fellow academician Marc-Auguste Pictet urged, in the full hearing of the Académie des Sciences, that attention be given to the report about a huge meteor shower that fell at L'Aigle, near Paris, on April 26, 1803.

I presume this would be an example of Laplace being rational and ignoring this evidence, in your view. In my view, it shows that people trying to be rational sometimes fail to be rational, and one case of this is by ignoring weak evidence, when weak evidence is still evidence. Obviously you do not assume that everything is necessarily correct: but the alternative is to take it as weak evidence, rather than ignoring it.

But (if my reasoning is correct) the fact is that a real method can work before there is enough evidence to support it. My post attempts to bring to our attention that this will make it really hard to discover certain experiences assuming that they exist.

Discounting the evidence doesn't actually make it any harder for us to discover those experiences. If we don't want to lose out on such things, then we should try some practices which we assign low probability, to see which ones work. Assigning low probability isn't what makes this hard -- what makes this hard is the large number of similarly-goofy-sounding things which we have to choose from, not knowing which ones will work. Assigning a more accurate probability just allows us to make a more accurate cost-benefit analysis in choosing how much of our time to spend on such things. The actual amount of effort it takes to achieve the results (in cases where results are real) doesn't change with the level of rationality of our beliefs.

MrMind's point is not "trust is central", but "prior beliefs are central". What person B believes about the world and has learned through many years of observation and analysis does and should color their reaction to A's claim.

"there is an abundance of extreme strong evidence (experimental and mathematical)" means that we find the story that somebody actually performed some kind of interaction with the universe to hold the belief (very) plausible. Contrast this with faked results where somebody types up a scientific paper but actually forges or doesn't do the experiments claimed. One of the main methods of "busting" it is doing it ourselfs ie replication.

There are crises where research communities spend time and effort on the assumption that a scientific paper holds true. We could say that if this "fandom" does not involve replication then their methodology is something other than science and thus they are not scientists. However the enthusiasim or the time spent on the paper by itbelief doesn't make it any more scientific or reliable. If the "founding paper" and derivative concepts and systems are forgeries it taints the whole body of activity to be epistemologically of low quality even if the "derivating steps" (work done after the forged paper) were logically sound.

However "what knowledge you should be a fan of?" is not a scientific question. Given all the hypotheses out these there is no right or wrong choice what to research. The focus is more that if you decide to target some field it would be proper to produce knowledge rather than nonsense. "If I knew exactly what I was doing it would not be called research, now would it?". There can be no before the fact guarantees what the outcome will be.

Asking whether you should bother to figure something out totally misses the epistemology of it. Someone who sees inherent value in knowledge will see moderate pain to attain it. But typically this is not the only relevant motivator. In the limit where this is the only motivation there is never a question of "hey we could do this to try figure out X" that would be answered in the "don't do it" variety. Such a system of morales could for example think that whether it is moral to ever have leisure as that time could be used for more research or how much more than fullitme one should be a researcher and how much one can push overtime to to not risk being burned out and not being able to function as a researcher in the future.

A hybrid model that has other interests than knowledge can and will often say "it would be nice to know but its too expensive - that knowledge is not worth the 3 lives lost those resources could be alternatively used for to save", "no we can't do human expeirments as the information is not worth the suffering of the test subjects" (Nazis get here a science boost as they are not slowed down by ethics boards). However sometimes it is legit to enter patients into a double blind experiement where 50% of patients will go untreated (or with only placebo) as the knowledge attained can then be used to for the benefit of other patients ("suffering" metric comes as net positive despite having positive and negative components). So large and important tool knowledge gains can offset other considerations. But asking the reverse of "how small a knowledge gain can be assumed to be overridden by other considerations" can't not really be answered without knowledge of what would override it. Or we can safely say that there is no knowledge small enough that it wouldn't be valuable if obtainable freely.

I wrote http://lesswrong.com/r/discussion/lw/oe0/predictionbased_medicine_pbm/ to suggest a change on a broader level.

When it comes to the personal level I'm however at the moment not aware of a good LW post that speaks explicitly about trust.