Rationality Quotes August 2014

Hollywood is filled with feel-good messages about how robotic logic is no match for fuzzy, warm, human irrationality, and how the power of love will overcome pesky obstacles such as a malevolent superintelligent computer. Unfortunately there isn’t a great deal of cause to think this is the case, any more than there is that noble gorillas can defeat evil human poachers with the power of chest-beating and the ability to use rudimentary tools.

From the British Newspaper 'The Telegraph', and their article on Nick Bostrom's awesome new book 'Superintelligence'.

I just thought it was a great analogy. Nice to see AI as an X-Risk in the mainstream media too.

A good rule of thumb to ask yourself in all situations is, “If not now, then when?” Many people delay important habits, work and goals for some hypothetical future. But the future quickly becomes the present and nothing will have changed.

Scott Young

"I want information. I want to understand you. To understand what exactly I'm fighting. You can help me."
"I obviously won't."
"I will kill you if you don't help me. I'm not bluffing, Broadwings. I will kill you and you will die alone and unseen, and frankly you are far too intelligent to simply believe that the stories of ancestral halls are true. You will die and that will probably be it, and nobody will ever know if you talked or not—not that conversing with an enemy in a war you don't support is dishonorable in the first place."
"You'll let me leave if I stonewall, because you don't want to set a precedent of murdering surrendered officers."
"We'll see. Would you like another cup?"
"No."
Derpy smiled deviously. "You know, in that last battle? We didn't fly our cannon up there to the cliffs. Nope. We had Earth ponies drag them. Earth ponies are capable of astounding physical feats, you know. We're probably going to be using more mobility in our artillery deployment going forward, now that they've demonstrated how effective the concept is."
"...why did you tell me that? What would drive you to tell me that?"
"I'll ask again before I continue. Would you like to assist me, Broadwings?"
"I am a gryphon. Telling me your plans will do nothing to change that. I will not barter secrets."
She leaned back, gesturing with a hoof as she talked. "My biggest strengths are that I understand the way crowds think and that I am good at thinking up unexpected ways to solve simple problems. My army's biggest weakness is that my soldiers are inexperienced, and that unexpected developments have an inordinate effect on their morale. Also, my infantry will never be able to stand against a sustained lion charge, so I have to keep finding ways to nullify that disadvantage, and frankly I won't be able to forever."
"I don't understand. What are you doing, Mare? Why are you--"
"--my personal biggest weaknesses," she continued, her smile now malicious, "are my struggles with morality, identity, and my desire to be loved. There's also my relationship with the stallion Macintosh Apple, who is usually called Big Macintosh, with whom I spend upwards of ten hours a day, and on whom I am completely emotionally dependent. If he were to be killed, I'd probably fall apart emotionally. I also have a daughter named Dinky—not by him, mind you—who is in the Southmarch, and who I am very, very guilty about abandoning. If anything were to happen to her I might kill myself. Do you understand yet, Broadwings?"
"Mare, this is insanity. I cannot--"
"--All right then, we'll continue. I also have in this camp Sweetie Belle, Apple Bloom, and Scootaloo, three little fillies, though they're growing quite quickly now. Sweetie Belle is the writer of many propaganda songs, Apple Bloom is Big Mac's sister, who he protects like a daughter, and I believe Scootaloo has no special importance but the other two would defend her to the death. They would be quite easy to kill as well. Do you understand yet?"
"Mare! Are you mad?! Do you have any idea how dangerous it is to tell me these things? Aren't you afraid I would tell--"
"--Good," she nodded. "You're beginning to understand. Let's see. My logistics framework right now is nonexistent. I'm entirely reliant on local villages bringing me food and materiel, and on capturing food and materiel meant for your armies. My army is nowhere near as mobile as it appears, since it can only operate in areas where I have established relationships with each particular village. A bit of simple recon work would let you figure out where I can and cannot go. Do you understand yet?"
Broadwings' eyes opened and his pupils shrank with dawning recognition. "...If I came back to my army, I would use this to defeat you. If I told any other gryphon, they would use it to defeat you. You...you have..."
"Yes. I have sealed your fate; you will not see your home. I can't let you leave now. I absolutely can't. I can now either kill you or keep you prisoner until this war is over—and I don't keep useless prisoners. It's now out of my hooves. One or the other. You pick."

~emkajii, Equestria: Total War

A man is walking on the moon with his eyes turned up toward space And the bright blue world that watches him reflected on his face. The whole world sees the hero there and the module crew also. But few can see the guiding team that guards him from below.

Here's a health to the man who walked the moon, and the module crew above, And the team that watches from the sky with worry, joy, and love. To all who blazed the sky-trail come raise your glasses 'round; And a health to the unknown heroes, too, who never left the ground.

Here's a health to the ship's designers, and the welders of her seams, And all who man the radar-scan to watch our dawning dreams. For all the unknown heroes, sing out to every shore: "What makes one step a giant leap is all the steps before".

Leslie Fish, musically praising the Hufflepuff virtues.

Surgeons finally did upgrade their antiseptic standards at the end of the nineteenth century. But, as is often the case with new ideas, the effort required deeper changes than anyone had anticipated. In their blood-slick, viscera-encrusted black coats, surgeons had seen themselves as warriors doing hemorrhagic battle with little more than their bare hands. A few pioneering Germans, however, seized on the idea of the surgeon as scientist. They traded in their black coats for pristine laboratory whites, refashioned their operating rooms to achieve the exacting sterility of a bacteriological lab, and embraced anatomic precision over speed.

The key message to teach surgeons, it turned out, was not how to stop germs but how to think like a laboratory scientist. Young physicians from America and elsewhere who went to Germany to study with its surgical luminaries became fervent converts to their thinking and their standards. They returned as apostles not only for the use of antiseptic practice (to kill germs) but also for the much more exacting demands of aseptic practice (to prevent germs), such as wearing sterile gloves, gowns, hats, and masks. Proselytizing through their own students and colleagues, they finally spread the ideas worldwide.

-Atul Gawande

That's why I'm skeptical of people who look at some catastrophic failure of a complex system and say, "Wow, the odds of this happening are astronomical. Five different safety systems had to fail simultaneously!" What they don't realize is that one or two of those systems are failing all the time, and it's up to the other three systems to prevent the failure from turning into a disaster.

-- Raymond Chen

It was a gamble: would people really take time out of their busy lives to answer other people’s questions, for nothing more than fake internet points and bragging rights?

It turns out that people will do anything for fake internet points.

Just kidding. At best, the points, and the gamification, and the focused structure of the site did little more than encourage people to keep doing what they were already doing. People came because they wanted to help other people, because they needed to learn something new, or because they wanted to show off the clever way they’d solved a problem.

...

An incredible number of people jumped at the chance to help a stranger

-- Jay Hanlon, Five year retrospective on StackOverflow

The amount of energy necessary to refute bullshit is an order of magnitude bigger than to produce it.

-- Alberto Brandolini (via David Brin)

is consciousness more like the weather, or is it more like multiplication?

Scott Aaronson

More context:

a perfect simulation of the weather doesn’t make it rain—at least, not in our world. On the other hand, a perfect simulation of multiplying two numbers does multiply the numbers: there’s no difference at all between multiplication and a “simulation of multiplication.” Likewise, a perfect simulation of a good argument is a good argument, a perfect simulation of a sidesplitting joke is a sidesplitting joke, etc.

Maybe the hardware substrate is relevant after all. But [...] I think the burden is firmly on those of us who suspect so, to explain what about the hardware matters and why. Post-Turing, no one gets to treat consciousness’s dependence on particular hardware as “obvious”—especially if they never even explain what it is about that hardware that makes a difference.

But if that were the case, then moral philosophers - who reason about ethical principles all day long - should be more virtuous than other people. Are they? The philosopher Eric Schwitzgebel tried to find out. He used surveys and more surreptitious methods to measure how often moral philosophers give to charity, vote, call their mothers, donate blood, donate organs, clean up after themselves at philosophy conferences, and respond to emails purportedly from students. And in none of these ways are moral philosophers better than other philosophers or professors in other fields.

Schwitzgebel even scrounged up the missing-book lists from dozens of libraries and found that academic books on ethics, which are presumably mostly borrowed by ethicists, are more likely to be stolen or just never returned than books in other areas of philosophy. In other words, expertise in moral reasoning does not seem to improve moral behavior, and it might even make it worse (perhaps by making the rider more skilled at post hoc justification). Schwitzgebel still has yet to find a single measure on which moral philosophers behave better than other philosophers.

• Jonathon Haidt, discussing the idea that ethical reasoning causes good behaviour, in his book 'The Righteous Mind'.

I found the book-stealing thing quite funny, although I imagine that some of the results described could be explained by popularity; if more people get into / like ethics, then there are more people who might steal library books, more antisocial people who don't respond to emails, etc. This hasn't been demonstrated to my knowledge though, and I'm otherwise inclined to believe that people who spend their days thinking about ethics in the abstract, are simply better at coming up with rationales for their instinctive feelings. Joshua Greene says rights are an example of this, where we need a dictum against whatever our emotions are telling us is despicable, even though we can't find any utilitarian justification for it.

We try things. Occasionally they even work.

Parson Gotti

A good argument is like a piece of technology. Few of us will ever invent a new piece of technology, and on any given day it’s unlikely that we’ll adopt one. Nevertheless, the world we inhabit is defined by technological change. Likewise, I believe that the world we inhabit is a product of good moral arguments. It’s hard to catch someone in the midst of reasoned moral persuasion, and harder still to observe the genesis of a good argument. But I believe that without our capacity for moral reasoning, the world would be a very different place.

-Joshua Greene, “Moral Tribes”, Endnotes

Come back with your shield - or on it.

Our kind might not be able to cooperate, but the Spartans certainly could. The Spartans were masters of hoplite phalanx warfare where often every individual would have been better off running away but collectively everyone was better off if none ran away than if all did. The above quote is what Plutarch says Spartan mothers would tell their sons before battle. (Because shields were heavy if you were going to run away you would drop it, and coming back on your shield meant you were dead.) Spreading memes to overcome collective action problems is civilization level rational.

This seems like an elegant and funny take on Ben Franklin's wisdom.

Walter Sobchak: "Am I wrong?"

The Dude: "No you're not wrong."

Walter Sobchak: "Am I wrong?"

The Dude: "You're not wrong Walter. You're just an asshole."

-The Big Lebowski, Directed by Joel Coen and Ethan Coen, 1998

Most of the time what we do is what we do most of the time.

-Daniel Willingham, Why Don't Students Like School. The point is that, quite often the reason we're doing something is that that's what we're used to doing in that situation.

Note: He attributes the quote to some other psychologists.

Sometimes the biggest disasters aren't noticed at all -- no one's around to write horror stories.

Vernor Vinge, A Fire Upon the Deep

Most of the time he asked questions. His questions were very good, and if you tried to answer them intelligently, you found yourself saying excellent things that you did not know you knew, and that you had not, in fact, known before. He had "educed" them from you by his question. His classes were literally "education" - they brought things out of you, they made your mind produce its own explicit ideas.

Thomas Merton, about professor Mark Van Doren

After describing

blind certainty, a close-mindedness that amounts to an imprisonment so total that the prisoner doesn't even know he's locked up.

David Foster Wallace continues

The point here is that I think this is one part of what teaching me how to think is really supposed to mean. To be just a little less arrogant. To have just a little critical awareness about myself and my certainties. Because a huge percentage of the stuff that I tend to be automatically certain of is, it turns out, totally wrong and deluded. I have learned this the hard way, as I predict you will, too.

"Science alone of all the subjects contains within itself the lesson of the danger of belief in the infallibility of the greatest teachers of the preceding generation." -Richard Feynman

Few people are capable of expressing with equanimity opinions which differ from the prejudices of their social environment. Most people are even incapable of forming such opinions.

Albert Einstein

There is a real joy in doing mathematics, in learning ways of thinking that explain and organize and simplify. One can feel this joy discovering new mathematics, rediscovering old mathematics, learning a way of thinking from a person or text, or finding a new way to explain or to view an old mathematical structure.

This inner motivation might lead us to think that we do mathematics solely for its own sake. That’s not true: the social setting is extremely important. We are inspired by other people, we seek appreciation by other people, and we like to help other people solve their mathematical problems.

-- William Thurston

The entire essay is a beautiful discussion of success and failure in practicing the art of mathematics. Changing the things that need to be changed, much of it applies to practicing the art of rationality.

The power is not in the choice of metaphor, it is in the ability to shift among metaphors. Teaching people this other metaphor [...] but not leaving them with the flexibility to move freely in and out is not having enabled them at all.

-- Kent Pitman

"We must not criticize an idiom [...] because it is not yet well known and is, therefore, less strongly connected with our sensory reactions and less plausible than is another, more 'common' idiom. Superficial criticisms of this kind, which have been elevated into an entire 'philosophy', abound in discussions of the mind-body problem. Philosophers who want to introduce and to test new views thus find themselves faced not with arguments, which they could most likely answer, but with an impenetrable stone wall of well-entrenched reactions. This is not at all different from the attitude of people ignorant of foreign languages, who feel that a certain colour is much better described by 'red' than by 'rosso'.

Paul Feyerabend, Against Method, 4th Edition, p. 59.

Language exists only on the surface of our consciousness. The great human struggles are played out in silence and in the ability to express oneself.

Franz Xavier Kroetz

In the fields of observation, chance favours only the prepared mind.

Louis Pasteur.

Challenge my assumption, not my conclusion, and do it with new evidence, instead of trying to twist the old stuff.

"The Originist", by Orson Scott Card

I believe the first part is frequently good advice. The second half is good, but not quite as good-- there still may be good new angles on old evidence.

Two mares, each convinced she was standing firmly on The Shores Of Rationality, stared helplessly into The Sea Of Confusion and despaired over their inability to ever rescue the friend helplessly floundering within.

A vivid description of inferential distance from Twilight's Escort Service.

Edit: It's from a comedy that relies on misunderstandings; Twilight chooses the word "escort" to advertise her teleportation abilities. If you don't enjoy awkwardness-based comedies, I recommend you stay away. The actual quote is about explaining a value difference.

'Deep pragmatism' is Joshua Greene's name for 'utilitarianism'.

Today we, some of us, defend the rights of gays and women with great conviction. But before we could do it with feeling, before our feelings felt like “rights,” someone had to do it with thinking. I’m a deep pragmatist, and a liberal, because I believe in this kind of progress and that our work is not yet done.

Joshua Greene, “Moral Tribes"

"Just as eating against one’s will is injurious to health, so studying without a liking for it spoils the memory, and it retains nothing it takes in." -Da Vinci

I'm starting a new 30 day challenge: the month of no "should." Instead of tediously working down a list of all the little chores and errands that I "should" be doing, I'll work to listen to what that little voice inside me wants to do. I think it will be interesting.

— Matt Cutts

Never let your sense of morals get in the way of doing what's right.

-Isaac Asimov

I've realized that I had started noticing and mitigating trivial inconveniences some time after reading the Yvain's post. Something as simple as leaving the door open or taking cookies from the the wrapper and placing them in a bowl, (or supporting form auto-fill, or placing the button (physical or virtual) you want the user to press right there in front if you are a developer) makes a difference in the "feature" being used (e.g. cookies being eaten).

Up next: figure out a way to use fewer parenthesis (including nested ones (yes, I've heard of commas)).

Most of the time what we do is what we do most of the time.

• Daniel Willingham, Why Don't Students Like School. The point is that, quite often the reason we're doing something is that that's what we're used to doing in that situation.

Note: He attributes the quote to some other psychologists.

Believe nothing, no matter where you read it, or who said it, no matter if I have said it, unless it agrees with your own reason and common sense.

-Buddha

But the more central point is that trying to explain or predict [institutional] behavior idealistically, in terms of things called "values" or moral fortitude, is foolish. It's magical thinking. "One party believes in..." Institutions don't have beliefs. They have incentives.

• Internet commenter

There is, to the [Slytherin adept], only one reality governing everything from quarks to galaxies. Humans have no special place within it. Any idea predicated on the special status of the human — such as justice, fairness, equality, talent — is raw material for a theater of mediated realities that can be created via subtraction of conflicting evidence, polishing and masking.

• Venkatesh Rao, The Gervais principle

society should not be looking for ways to maintain privacy. It should be looking for ways to make privacy unnecessary. We will never be free until we lose our unnecessary secrets and discover we are better off without them.

Scott Adams

(Please read the link for context before commenting on the quote alone)

Any adequate spiritual system has to account for all of reality. Drinking is part of reality, therefore drinking is spiritual.

-a Vedic (Hindu) philosophy society in a pub.

This strikes me as rationality of the "taking joy in the mealy real" sort from the unliklyest of places. Either that, or a good way to rationalise getting drunk.

Material, adj. Having an actual existence, as distinguished from an imaginary one. Important

• Ambrose Bierce, The Enlarged Devil's Dictionary, Compiled and Edited by Ernest J. Hopkins, p. 194