Comments for "Rationality"

The only legitimate purpose of words having definitions in the first place is to let two people communicate - the purpose of attaching certain syllables to certain concepts is to help transport meanings from one mind to another.

Not quite, I think. I frequently use words in my own thinking, and have much reason to think I'm not alone in this, and if you really think that's illegitimate then I'd be interested to know why. (I bet you don't.) And it sure seems like I make use of the fact that (some) words have definitions when I do that.

Instrumental rationality: achieving your values. Not necessarily "your values" in the sense of being selfish values or unshared values: "your values" means anything you care about. The art of choosing actions that steer the future toward outcomes ranked higher in your preferences. On LW we sometimes refer to this as "winning".

In my opinion, Wikipedia puts things much better here:

Rationality is a central principle in artificial intelligence, where a rational agent is specifically defined as an agent which always chooses the action which maximises its expected performance, given all of the knowledge it currently possesses.

• http://en.wikipedia.org/wiki/Rationality

The advantage wikipedia has is that it is talking about expected performance on the basis of the available information, not about actual performance. That emphasis is correct - rationality is (or should be) defined in terms of whether operations performed on the available information constitute correct use of the tools of induction and deduction - and should not depend on whether the information the agent has is accurate or useful.

This has been discussed many times: there is a distinction between trying to win and winning.

That's a great analysis, and it should bring more clarity to our discussions if we can all agree on that or modify it as necessary until we basically agree.

One thing I am wondering about though is that the analysis seems to present the two subspecies of rationality -- epistemic and instrumental -- as being on equal footing, or as somehow equally fundamental. (That's my reading based on labeling them as 1 and 2 and not indicating that either is more fundamental.)

It seems to me though that instrumental rationality is what we really, really mean by rationality, and what you have referred to as epistemic rationality is one particular (astoundingly powerful) technique of instrumental rationality.

There are always multiple ways of mapping the territory. We have no way of deciding between them other than in terms of instrumental rationality, by choosing the one that seems most useful for achieving our values.... We might call that most useful map truth or corresponding to reality, but it only acquires that status via instrumental rationality. Epistemic rationality thus depends on instrumental rationality, but the converse is not true.

Due to the recent flurry of arguments about "defining rationality", and that my karma has risen over 20, I was considering writing a post on the same topic as yours Eliezer. There's sort of a "darn it" feeling that you beat me to the punch, but I'm also glad that you did, because your writing is much more clear and elegant than mine. Plus your linking to your past posts on OB on the subject is much more comprehensive than anything I could have accomplished.

My one comment is that I noticed you never used the terms "descriptivism" and "prescriptivism" while I almost always do when talking about e.g. the absurdity of thinking that the contents of "the" dictionary (as if there is only one dictionary, or as if all dictionaries are in perfect agreement with each other) determines the meaning of the words. Are you intentionally avoiding these terms, or simply don't find them useful, or some other reason?

Eliezer: "Similarly, if you find yourself saying "The rational thing to do is X, but the right thing to do is Y" then you are almost certainly using one of the words "rational" or "right" in a way that a huge chunk of readers won't agree with. In this case - or in any other case where controversy threatens - you should substitute more specific language: "The self-benefiting thing to do is to run away, but I hope I would at least try to drag the girl off the railroad tracks""

Yes. Rational does not equal "sensible" or "putting self first".

So can we be rational in arguing about morality? If I decide that human life has value, I can argue from that prior, rationally, that it is Right to try and drag the girl off the railroad tracks.

I believe that human life has value, even though that is not a completely rigorous, defined statement of my belief about human life. I doubt I have the words to fully express my beliefs about the value of human life.

It is possible that I generalise "human life has value" from my own selfish needs, I do not like being alone for too long, I would have to adjust and learn a great deal before I could survive without Society.

So I believe that for me to believe "human life has value" is Right, or at least permissible, but not necessarily Rational (epistemic or instrumental) in itself, though I can take it as an axiom, and argue rationally based upon it.

Or if my belief that "human life has value" derives rationally from "I will base my values on my own selfish needs" which derives from "I want to survive": in "I want to survive" there is a Want, which is not derived rationally from anything.

I'm new to LessWrong so pardon me if my question has an obvious answer or seems silly, and please let me know if there are flaws in my reasoning.

"The self-benefiting thing to do is to run away, but I hope I would at least try to drag the girl off the railroad tracks." In this context, is the rational choice different for different people?

I believe that rationality can mean different things to different people because people have different moral compasses- a set of values they believe is right or are the most important to them. If I place another human's life in high enough regard that I would try to drag the girl off the railroad tracks, and that instinct is greater than my need to run away (self preservation), then I will end up trying to help the girl. If my priorities are reversed and I value self preservation more, I will likely run away.

Now what if I'm the sort of person who values self preservation more and hence am likely to run away, but I WANT to be the kind of person who would stop and help the girl? I'm making a conscious effort to be selfless in life. What is the rational choice for me then? I understand that if I were to actually be in such a situation, I would not have the time to logically make up my mind about what to do but would simply act on instinct, but I'm still interested in understanding what the rational choice would be in that situation.

Suppose I have a blind man and a sighted man. The blind man has a cataract that could be repaired with surgery. I tell them that a jar contains a very large number of pebbles, all of them are either green or blue, and there are twice as many of one color than the other. I pull out a random pebble, which turns out to be green, and show it to both the men. I ask them to write down what they think the probability is that the next pebble I pull out will be green. The sighted man writes 51%; the blind man writes 50%. Who is more rational?

Thanks Eleizer,

I needed appropriate names to specify which of those two I was referring to. I'll be sure to use 'epistemic' or 'instrumental' when the context demands. That'll save many a distracting explanatory sentence.

When reading a textbook or technical work, I frequently use marginalia to comment on the work. I find it a useful tool to increase reading comprehension, force me to organize my thoughts, and allow me to return to that part of the book years later to use it as a reference. I'm reading through The Sequences, but since it is in digital form I am unable to make use of my usual practice. Instead, I intend to leave several comments such as this in the appropriate discussion threads. I initially was using a word document, but have found it tedious to constantly transfer between computers and devices. If these comments and notifications are objectionable to anyone, I'll switch back. The FAQ says that it is worthwhile to comment on ancient posts and long-dead threads, so I took that as encouragement. I only have a couple tangential comments on this particular piece, but I expect most future marginalia to generally be much more extensive.

It’s worrying that people will falsely guess P("Bill plays jazz") < P("Bill plays jazz" & "Bill is accountant"). If these were the profiles for two murder suspects, the jury could easily make a very bad judgment call. However, we are evolutionarily wired to be good at social profiling. I suspect that the error here is that people are reading this problem as if it was the type that we are good at. When all you have is a hammer, everything looks like a nail. For example, they might read the problem to mean P(A) < P(A + B), where + indicates “and/or” (logical disjunction). This would also explain why people tend to believe “if A implies B, than B implies A” (that correlation implies causation).

Heh: the royal "we".