What is wrong with "Traditional Rationality"?

By that same logic, I could say "Popper is in the non-quantitative tradition, which is mainstream (in contrast to Bayesian epistemology)". Reflecting one aspect of the mainstream, even a particularly important one, is still not sufficient for actually being mainstream.

Uncertainty is an unavoidable aspect of the human condition.

So in the very first sentence, the authors' have revealed a low opinion of humans. They think humans have a condition, although they don't explain what it is, only that uncertainty is part of it.

Um, I think you are possibly taking a poetic remark too seriously. If they had said "uncertainty is part of everyday life" would you have objected?

So inference and judgement are governed by heuristics, genetic in origin (though this is just implied and the authors do nothing to address it).

Heuristics are not necessarily genetic. They can be learned. I see nothing in their paper that implies that they were genetic, and having read a fair amount of what both T & K wrote, there's no indication that I saw that they strongly thought that any of these heuristics were genetic.

It's not that humans come up with explanations and solve problems, it's not that we are universal knowledge creators, it's that we use heuristics handed down to us from our genes and we must be alerted to biases in them in order to correct them, otherwise we make systematic errors. So, again, a low opinion of humans. And we don't do induction - as Popper and others such as Deutsch have explained, induction is impossible, it's not a way we reason.

Ok. This confuses me. Let's says that humans use genetic heuristics, how is that a low opinion? Moreover, how does that prevent us from being universal knowledge creators? You also seem to be confused about whether or not something is a good epistemology being related to whether or not a given entity uses it. Whether humans use induction and whether induction is a good epistemological approach are distinct questions.

This seems close to, if anything, Christian apologists saying how if humans don't have souls then everything is meaningless. Do you see the connection here? Just because humans have flaws doesn't make humans terrible things. We've split the atom. We've gone to the Moon. We understand the subtle behavior of the prime numbers. We can look back billions of years in time to the birth of the universe. How does thinking we have flaws mean one has a low opinion of humans?

I'm curious, when a psychologist finds a new form of optical illusion, do you discount it in the same way? Does caring about that or looking for those constitute a low opinion of humans?

Our problems, of course, were constructed to elicit conjunction errors, and they do not provide an unbiased estimate of the prevalence of these errors.

So they admit bias

That's a tortured reading of the sentence. The point is that they wanted to see if humans engaged in conjunction errors. So they constructed situations where, if humans were using the representativeness heuristic or similar systems the errors would be likely to show up. This is, from the perspective of Popper in LScD, a good experimental protocol, since if it didn't happen, it would be a serious blow to the idea that humans use a representativeness heuristic to estimate likelyhood. They aren't admitting "bias"- their point is that since their experimental constructions were designed to maximize the opportunity for a representativeness heuristic to show up, they aren't a good estimate for how likely these errors are to occur in the wild.

Yes. It is based on inductivist assumptions about how people think, as the quote above illustrates. They disregard the importance of explanations and they think humans do probabilistic reasoning using in-born heuristics and that these are universal.

So it seems to me that you are essentially saying that you disagree with their experimental evidence on philosophical grounds. If your evidence disagrees with your philosophy the solution is not to deny the evidence.

Do you agree with his claim that ""Probability estimate" is a technical term which we can't expect people to know? Do you agree with his implicit claim that this should apply even to highly educated people who work as foreign policy experts?

Do you think foreign policy experts use probabilities rather than explanations?

In some contexts, yes. For example, foreign policy experts working with economists or financial institutions sometimes will make probability estimates for them to work with. But let's say they never do. How is that at all relevant to the questions at hand? Do you really think that the idea of estimating a probability is so strange and technical that highly educated individuals shouldn't be expected to be able to understand what is being asked of them? And yet you think that Tversky had a low opinion of humans? Moreover, even if they did have trouble understanding what was meant, do you expect that would cause all the apparent bias to go by sheer coincidence just as one would expect given the conjunction fallacy?

It's not an agenda in the sense of a political agenda (though it does have some connections to political ideas), nor a conspiracy, nor a consciously intended and promoted agenda.

But, they have a bunch of unconscious ideas -- a particular worldview -- which informs how they approach their research and, because they do not use the rigor of science which prevents such things, their worldview/agenda biases all their results.

The proper rigor of science includes things like describing the experimental procedure in your paper so mistakes can be criticized and it can be repeated without introducing unintended changes, and having a "sources of error" section where you discuss all the ways your research might be wrong. When you leave out standard parts of science like those, and other more subtle ones, you get unscientific results. The scientific method, as Feynman explained, is our knowledge about how not to fool ourselves (i.e. it prevents our conclusions from being based on our biases). When you don't use it, you get wrong, useless and biased results by default.

One of the ways these paper goes wrong is it doesn't pay enough attention to the correct interpretation of the data. Even if the data was not itself biased -- which they openly admit it is -- their interpretation would be A) problematic and B) not argued for by the data itself (interpretations of data never are argued for by the data itself, but must be considered as a separate and philosophical issue!)

If you try enough, you can get people to make mistakes. I agree with that much. But what mistake are the people making? That's not obvious, but the authors don't seriously discuss the matter. For example, how much of the mistake people are making is due to miscommunication -- that they read the question they are asked as having a meaning a bit different than the literal meaning the researchers consider the one true meaning? The possibility that the entire phenomenon they were observing, or part of it, is an aspect of communication not biases about probability is simply not addressed. Many other issues of interpretation of the results aren't addressed either.

They simply interpret the experimental data in a way in line with their biases and unconscious agendas, and then claim that empirical science has supported their conclusions.

Do you have a link to Hanson? Taking Children Seriously is trickier than it appears, so new people often mistakenly think it is similar to ideas they heard before or were thinking before.

Cool that we all like Feynman. But Feynman was in the Popperian tradition, so I don't see how Yudkowsky could be building on Feynman when he says he is "dethroning Popper". Can you point me to a place in the sequences where Feynman is discussed?

He merely points out that it is not enough to prevent many people, including himself, from believing stupid things

Yes, it is easy to fool ourselves, as Feynman said. That's why you need a philosophy that focuses on finding errors and correcting them, as Popperism does. You're always going to make mistakes - the truth is not obvious after all - but it is through our mistakes that we progress, so be relentless in uncovering mistakes, make your mistakes fast, and celebrate them!

If Yudkowsky hates Popper and likes Feynman, then either

1) he likes some individual, narrow aspects of feynman (perfectly fine and unobjectionable)

2) he hasn't understood what feynman is about

3) i have misunderstood feynman, badly

Agree so far? Do you think it's number 1?

No, he invented it. Read Popper's The World of Parmenides. (BTW, Popper took the trouble to learn ancient Greek)