When does an insight count as evidence?

Insight has to count as evidence when you are dealing with finite computational agents.

Now we know that either P=NP is proveable from the axioms of set theory, or its converse is

This isn't to be said casually; it would be a huge result if you could prove it, and very different from the case of neither being provable. Most things that are true about the natural numbers are not provable in any given set of axioms.

I mention this because I read that and woke up and said "What? Really?" and then read the following parenthetical and was disappointed. I suggest editing the text. If we don't know anything in particular about the relation of P=NP to set theory, it shouldn't be said.

What purpose are you after with this query? It sounds dangerously much like a semantic discussion, though I may be failing to see something obvious.

"Wrong", you say, "no new information has been injected here, I have simply pointed out how to reason rationally.

I'm not sure if this line makes sense. If somebody points out the correct way to interpret some piece of evidence, then that correct way of interpreting it is information. Procedural knowledge is knowledge, just as much as declarative.

Putting it in another way: if you were writing a computer program to do something, you might hard-code into it some way of doing things, or you might build some sort of search algorithm that would let it find the appropriate way of doing things. Here, hard-coding corresponds to a friend telling you how something should be interpreted, and the program discovering itself corresponds to a person discovering it herself. If you hard-code it, you are still adding extra lines of code into the program - that is, adding information.

I think this post makes an excellent point, and brings to light the aspect of Bayesianism that always made me uncomfortable.

Everyone knows we are not really rational agents; we do not compute terribly fast or accurately (as Morendil states), we are often unaware of our underlying motivations and assumptions, and even those we know about are often fuzzy, contradictory and idealistic.

As such, I think we have different ways of reasoning about things, making decisions, assigning preferences, holding and overcoming inconsistencies, etc.. While it is certainly useful to have a science of quantitative rationality, I doubt we think that way at all... and if we tried, we would quickly run into the qualitative, irrational ramparts of our minds.

Perhaps a Fuzzy Bayesianism would be handy: something that can handle uncertainty, ambivalence and apathy in any of its objects. Something where we don't need to put in numbers where numbers would be a lie.

Doing research in biology, I can assure you that the more decimal places of accuracy I see, the more I doubt its reliability.

But at what difficulty does it become rational to approximate, or do a meta-analysis?

It depends on how much resources you choose to, or can afford to, devote to the question.

Say I have only a few seconds to ponder the product 538 time 347, and give a probability assignment for its being larger than 150,000; for its being larger than 190,000; and for its being larger than 240,000.

My abilities are such that in a limited time, I can reach definite conclusions about the two extreme values but not about the third; I'd have to be content with "ooh, about fifty-fifty". Given more time (or a calculator!) I can reach certainty.

If we're talking about bounded rationality rather than pure reason, the definition "identical states of knowledge" needs to be extended to "identical states of knowledge and comparable expenditure of resources".

Alternatively you need to revise your definition of "irrational" to admit degrees. Someone who can compute the exact number faster than I can is perhaps more rational than I am, but my 1:1 probability for the middle number does not make me "irrational" in an absolute sense, compared to someone with a calculator.

I wouldn't call our friend "irrational", though it may be appropriate to call them lazy.

In fact, the discomfort some people feel at hearing the words "irrational" or "rational" bandied about can perhaps be linked to the failure of some rationalists to attend to the distinction between bounded rationality and pure reason...

ETA:

the more stupid the person, the more difficult it would be to claim that they were, in fact, irrational.

So ? You already have a label "stupid" which is descriptive of an upper bound on the resources applied by the agent in question to the investigation at hand. What additional purpose would the label "irrational" serve ?

Insight doesn't exactly "count as evidence". Rather, when you acquire insight, you improve the evidence-strength of the best algorithm available for assigning hypothesis probabilities.

Initially, your best hypothesis-weighting algorithms are "ask an expert" and "use intuition".

If I give you the insight to prove some conclusion mathematically, then the increased evidence comes from the fact that you can now use the "find a proof" algorithm. And that algorithm is more entangled with the problem structure than anything else you had before.

when, if ever, does an insight count as evidence?

I suspect you use the term "insight" to describe something that I would classify as a hypothesis rather than observation (evidence is a particular kind of observation, yes?).

Consider Pythagoras' theorem and an agent without any knowledge of it. If you provide the agent with the length of the legs of a right-angled triangle and ask for the length of the hypotenuse, it will use some other algorithm/heuristic to reach an answer (probably draw and measure a similar triangle).

Now you suggest the theorem to the agent. This suggestion is in itself evidence for the theorem, if for no other reason then because P(hypothesis H | H is mentioned) > P(H | H is not mentioned). Once H steals some of the probability from competing hypotheses, the agent looks for more evidence and updates it's map.

Was his first answer "rational"? I believe it was rational enough. I also think it is a type error to compare hypotheses and evidence.

If you define "rational" as applying the best heuristic you have, you still need a heuristic for choosing a heuristic to use (i.e. read wikipedia, ask expert, become expert, and so on). If you define it as achieving maximum utility, well, then it's pretty subjective (but can still be measured). I'd go for the latter.

P.S. Can Occam's razor (or any formal presentation of it) be classified as a hypothesis? Evidence for such could be any observation of a simpler hypotheses turning out to be a better one, similar for evidence against. If that is true, then you needn't dual-wield the sword of Bayes and Occam's razor; all you need is one big Bayesian blade.

You make the common error of viewing the answer as binary. A proper rationalist would be assigning probabilities, not making a binary decision. In the one-in-a-million example, your friend thinks he has the right answer, and it is because he thinks he is very probably right that he is irrational. In the P=NP example, I do not have any such certainty.

I would imagine that a review of P=NP in the manner you describe probably wouldn't push me too far from 50/50 in either direction. If it pushed me to 95/5, I'd have to discredit my own analysis, since people who are much better at math than I am have put a lot more thought into it than I have, and they still disagree.

Now, imagine someone comes up with an insight so good that all mathematicians agree P=NP. That would obviously change my certainty, even if I couldn't understand the insight. I would go from a rationally calibrated ~50/50 to a rationally calibrated ~99.99/.01 or something close. Thus, that insight certainly would be evidence.

That said, you do raise an interesting issue about meta-uncertainty that I'm still mulling over.

ETA: P=NP was a very hypothetical example about which I know pretty much nothing. I also forgot the fun property of mathematics that you can have the right answer with near certainty, but no one cares if you can't prove it formally. My actual point was about thinking answers are inherently binary. The mistake of irrational actor who has ineffective tools seems to be his confidence in his wrong answer, not the wrong answer itself.

(please note that this is my first post)

I found the phrasing in terms of evidence to be somewhat confusing in this case. I think there is some equivocating on "rationality" here and that is the root of the problem.

For P=NP, (if it or its inverse is provable) a perfect Bayesian machine will (dis)prove it eventually. This is an absolute rationality; straight rational information processing without any heuristics or biases or anything. In this sense it is "irrational" to not be able to (dis)prove P=NP ever.

But in the sense of "is this a worthwhile application of my bounded resources" rationality, for most people the answer is no. One can reasonably expect a human claiming to be "rational" to be able to correctly solve one-in-a-million-illness, but not to have (or even be able to) go through the process of solving P=NP. In terms of fulfillingone's utility function, solving P=NP given your processing power is most likely not the most fulfilling choice (except for some computer scientists).

So we can say this person is taking the best trade-off between accuracy and work for P=NP because it requires a large amount of work, but not for one-in-a-million-illness because learning Bayes rule is very little work.

(please note that this is my first post)

I found the phrasing in terms of evidence to be somewhat confusing in this case. I think there is some equivocating on "rationality" here and that is the root of the problem.

For P=NP, (if it or its inverse is provable) a perfect Bayesian machine will (dis)prove it eventually. This is an absolute rationality; straight rational information processing without any heuristics or biases or anything. In this sense it is "irrational" to not be able to (dis)prove P=NP ever.

But in the sense of "is this a worthwhile application of my bounded resources" rationality, for most people the answer is no. One can reasonably expect a human claiming to be "rational" to be able to correctly solve one-in-a-million-illness, but not to have (or even be able to) go through the process of solving P=NP. In terms of fulfillingone's utility function, solving P=NP given your processing power is most likely not the most fulfilling choice (except for some computer scientists).

So we can say this person is taking the best trade-off between accuracy and work for P=NP because it requires a large amount of work, but not for one-in-a-million-illness because learning Bayes rule is very little work.

I don't think the two examples you gave are technically that different. Someone giving an "intuitive" answer to the diagnostic question is basically ignoring half the data; likewise, someone looking for answer to P=NP using a popularity survey is ignoring all other data (e.g. the actual math).

The difference is whether you know what data you are basing your evaluation on and whether you know you have ignored some. When you can correctly state what your probability is conditional on, you are presenting evidence.