Rationality: Appreciating Cognitive Algorithms

Eliezer is attacking a particular usage of the word "true." That point is well taken. Further, I appreciate his explicit linking of his thoughts into the larger philosophical debate.

But I am unaware of any philosophical movement that uses "true" the way Eliezer attacks. The sentence I quote could have made the same point (and been more accurate) if postmodernist/relativism was omitted entirely. What purpose do you think including the label had? In particular, why was the label (inaccurately) applied to a position that Eliezer just demonstrated was false?

The sky is black about half the time

If you count navy as blue rather than as black, that happens more rarely than “half the time”. (I'd say “10% of the time” as I have that number cached in my mind as the duty cycle of fluorescence detectors for ultra-high-energy cosmic rays.) You know, the moon.

and it's pretty common for it to be white, too.

And when that happens, in places where electric lighting is widely used, it tends to become orange (not quite -- does that colour have a name?) during the night!

When the sky is white, it's not the sky; it's clouds blocking the sky. When the sky is black it's just too dark to see the sky. At least that was my intuition before I knew that the sky wasn't some conventionally blue object. I guess its a question of word usage whether the projective meaning of "blue" which is something like "looks blue under good lighting conditions" should still be applied when it's not caused by reflectance. Though it's not blue from all directions is it?

It's something that everybody has quick access to. Another version would be "things fall", which is better but also only works on a planet and with objects denser than air for example. It would be ideal to have some unchanging reference object that we can make statements about, instead we have something that everyone has seen and they can say "I have seen that, it was pretty much blue"

You're right. I think the lesson we should take from all this complexity is to remember that the wording of a sentence is relevant to more than just it's truth conditions. Language does a lot more than state facts and ask questions.

I agree with Xachariah's view of semantics. I think that the first 'I believe' does imply a different meta level of belief (often associated with a different reason for believing). His example does a good job of showing how someone can drill down many levels, but the distinction in the first level might be made more clear by considering a more concretely defined belief:

"We're lost" -- "I'm you're jungle leader, and I don't have a clue where we are any more."

"I believe we're lost" -- "I'm not leading this expedition. I didn't expect to have a clue where we were going, but it doesn't seem to me like anyone else knows where we are going either."

--

"Sarah won state science fair her senior year of high school" -- "I attended the fair and witnessed her win it."

"I believe that Sarah won state science fair her senior year of high school" -- "She says she did, and she's the best experimentalist I've ever met."

"I believe that I believe that Sarah won state science fair her senior year of high school" -- "She says she did, and I don't believe for one second that she'd make that sort of thing up. That said, she's not, so far as I can tell, particularly good at science, and it shocks me that she might somehow have been able to win."

--

"Parachuting isn't all it's cracked up to be." -- "I've gone parachuting, and frankly, I've gotten bigger adrenaline rushes playing poker."

"I don't believe parachuting's all it's cracked up to be." -- "I haven't gone parachuting. There's no way I would spend $600 for a 4 minute experience when I can't imagine that it's enough fun to justify that."

Without the 'I believe,' what I tend to be saying is, I trust the map because I drew it and I drew it carefully. With the 'I believe,' I tend to be saying I trust this map because I trust it's source even though I didn't actually create it myself. In the case of the parachuting, I don't know where the map comes from, it's just the one I have.

Placing additional "I believe"s in front of a statement changes what part of the statement you have confidence in.

The statement 'I believe God exists' usually does mean that someone places confidence in eir community's ability to determine if God exists or not rather than placing confidence in the statement itself. Most of the religious people I know would say 'God exists' rather than 'I believe God exists' and most of them believe that they have directly experienced God in some way. However, most of them would say 'I believe the Bible is true' rather than 'the Bible is true' -- and when pressed for why they believe that, they tend to say something along the lines of "I cannot believe that God would allow his people to be generally wrong about something that important" or something else that asserts that their confidence is in their community's ability to determine that 'the Bible is true' rather than their confidence being in the Bible itself. I don't know if this is a very localized phenomenon or not since all of the people I've had this conversation with belong to the same community. It's how I would tend to use the word 'believe' too, but I grew up in this community, so I probably tend to use a lot of words the same way as the people in this community do.

In Xachariah's example the certainty/uncertainty is being placed on the definition of 'believe' at each step past the first one, so the way that the the statement is changing is significantly different in the second and third application of 'I believe' than it is in the first. The science fair example applies the 'I believe' pretty much the same way twice.

When I say "Sarah won science fair," I'm claiming that all of the uncertainty lies in my ability to measure and accurately record the event. Her older sister is really good at science too; it's possible that I'm getting the two confused but I very strongly remember it being Sarah who won. On the other hand, I'm extremely confident that I wouldn't give myself the wrong map intentionally -- I have no reason to want to convince myself that Sarah is better at science than she actually is.

That source of uncertainty essentially vanishes when the source of my information becomes Sarah herself. I now have a new source of uncertainty though because she does have a reason to convince me that she is better at science than she actually is. However, I trust the map because it agrees with what I'd expect it to be. I'd still think she was telling the truth about this if she lied to me about other things.

In the third case, I'm once again extremely confident that Sarah won science fair. She told me she did, and she tells the truth. What she's told me does not at all agree with my expectations; I don't really place confidence in the map, I place confidence a great deal of confidence in Sarah's ability to create an accurate map, and I place a great deal of confidence in her having given me an accurate map. The map seems preposterous to me, but I still think it's accurate, so when someone asks me if I believe that Sarah won science fair, I wince and I say "I believe that I believe that Sarah won science fair" and everyone knows what I mean. My statement isn't really "Sarah won science fair." It's "Sarah doesn't lie. Sarah says she won science fair. Therefore, Sarah won science fair." If I later find out that Sarah isn't quite as honest as I think she is, this is the first thing she's told me that I'll stop believing. Unless that happens, I'll continue to believe that she won.

Also, "I would expect X".

I took it as literary allusion in a place where such may not have been suited to something in a literalist genre. Which may count as a mistake.

Why not just say "The democrats are more likely than not to win the next election?"

I don't expect my brain to be internally consistent, and I consider all thoughts it thinks my thoughts because the alternative seems more dissociative than necessary

What does it mean to consider these thoughts "your" thoughts, what does this ownership signify? Your brain produced them; what else is there to say? Endorsement of correctness doesn't need to relate to personal identity.

Isn't that exactly the question we often ask juries to consider in, for example, liability lawsuits? "It was rational for the defendant to assume (or conclude under the circumstances) X, but in fact not-X, because of Y or because they got really unlucky, and therefore we find them not liable."

I will happily concede that juries do a poor job accounting for hindsight bias, and hold defendants to low standards of rationality, but it seems to me that the usual question is something like "even though not-X, was it rational to believe X?"

Whether "reasonable" and "rational" really mean the same thing in this case is open, but I submit that it is the same question as whether juries hold defendants to reasonable standards of rationality.

Yes, when we talk about something having systematic effects, we mean it tends probabilistically to have those effects.

Typically, when I want to discuss probabilistic relations I'll use a probabilistic word, like "probably" or "tends to" or "correlates with." When I use the word "systematically," I typically want to imply a causal relationship. Taking Eliezer's old example, if I put a pebble in the bucket when a sheep leaves the fold and take a pebble out of the bucket when a sheep returns to the fold, I've created a causal system, which will have the systematic effect of letting me know how many sheep are outside the fold by checking the level of the bucket. Whether the system is deterministic or stochastic doesn't make much difference for how I think about the graph connecting the nodes, though it will change the underlying mathematics.

Now, I'll note my answer is very different from Eliezer's, and I suspect that's because "tends probabilistically to" is a simpler concept than a causal system; I might be trying to explain addition using multiplication.

How about "if you try this many times, it will usually work." I'm not sure you can taboo 'usually' (or 'systematically'). It seems to be one way to invoke a rather fundamental-seeming process of abstracting from specific cases to general categories about which you can then form summarizing beliefs. If you ask someone to taboo something too basic, the best they can do is to rephrase and hope you'll get what basic thing they were referring to.

Whenever I see semantic dissection in major posts, I always worry that language is just too messy, just a towering stack of cards, and wonder why there isn't more discussion about why we use English when the language doesn't seem optimal for science and seeking Truth. Obviously it's rational for those who already speak English to continue using it for lack of an immediately available, preferable alternative, but I don't see much discussing this fact, arguing whether we should start over, etc. Admittedly, I haven't yet grokked the "ways words can be wrong" sequence.

My previous post on the topic.

Help me get over my linguistic-existential dread by refuting (or accepting) the statement "To develop optimal rationality skills, the first step should be to redesign our linguistic operating system."

if someone made a serious attempt to unpack "systematically"

Since you seem to understand that there's an equivocation, wouldn't it be easier to just state up front what the two different meanings are supposed to be?

I'm still not sure what you're trying to point out here. Can you be more explicit/specific?

Hm.
Are you asking "how could I tell that someone else isn't being rational?" or "how could I communicate to someone else that they aren't being rational in such a way that they'd benefit from it?" or something else?

I speak as i find

As a long-time RW regular, I thought it was pretty accurate. Everyone thinks they're rational, particularly the infuriatingly hard-of-thinking. RW is Internet television and an enjoyable waste of your time at best. With useful bits. (This is approximately how I treat LW as well, of course.)