When to scream "Error!"

Say I'm in circumstance X and believe Y, and you're in X' and believe Y'..

I can try to understand what evidence you have for Y' -- that is, try to understand the subset of your experience of X' that is relevant to believing Y'. I can then use that understanding as additional evidence that informs my estimates of the likelihood of Y and Y'.

Or I can decide that I don't feel like putting that much effort into the question. Which is perfectly valid... as you say, this comes up all the time, and one has to prioritize.

In that second case, I can take various shortcuts.

Indeed, many such shortcuts are wired into my brain, and they aren't necessarily bad ones -- though of course experienced deceivers are used to subverting them. Others can be learned, either implicitly or explicitly. Some are so unreliable in the modern world I do best to _un_learn them.

What I try not to do is fool myself into thinking I've actually evaluated the situation when I've merely taken a shortcut. If I'm dismissing Y' and reaffirming my belief in Y without doing the analysis, the alarm goes off to remind me that no, that's not justified. Y is my current belief, and I'm choosing not to investigate Y' because I've got better things to do, and that's really all I can legitimately say.

You seem to have interpreted my question to mean "When do you scream 'Error!' when you shouldn't?". While your response contains valuable answers to that question, what I had in mind was building a (nearly) comprehensive list of indicators that you are currently being an effective rationalist, so that when you don't notice any good indicator, you can scream "Error!".

I treat not screaming "Error!" as the default state, and I think that you should at least pay attention to problem indicators even if you otherwise feel like you're grokking well, as doing so might help to prevent you from going down the wrong mental path.

I am not dismissing your approach. I am saying do both.

Do you have examples of white list states, where you should be more prone to screaming "Error!" if you don't have many/any of them?

My first example is that you are pursuing a worthwhile goal, which is an attempt at refining Anna's "use curiosity" heuristic. (Notice how even though Anna presented it as lack of curiosity is a negative indicator, I see it as curiosity is a positive indicator.)

Admittedly, I don't have a lot of personal experience with healthy stupidity.

I'm a fairly bright guy, by human standards, and most of my experience with genuine stupidity stems from a period of my life where I was recovering from stroke-induced brain damage, which is at least arguably a form of unhealthiness.

That said, I think everyone is familiar with the effects of fatigue on cognition. It's harder to hold onto a train of thought, it's harder to account for emotional bias, it's harder to keep things in memory. When I get tired, I get stupid. It's a kind of mental fog.

When I was recovering, I felt like that all the time.

Partly, of course, that's because I was genuinely tired a lot of the time, in large part because my blood pressure was being artificially lowered and I was greying out a lot.

But even when I wasn't tired -- during those brief blessed windows after waking up and before I started physical therapy, for example -- there was still that characteristic mental fog, all the time.

That's what I was thinking about, when I wrote it.

That said, I have no idea whether healthy stupidity feels at all like that.

This is the child. Downvote it for karma balance iff you don't want Dorrika's post to be top-level.

Psst. If this comment's parent and its grandparent have different absolute karma counts, someone is doing the poll wrong -- the karma is supposed to balance out.