Rationality Quotes July 2013

If posting things said on lesswrong or OB or from HPMOR aren't in scope, it seems a little odd things said in HPMOR discussion on a forum run by you that doesn't happen to be those two is.

The idea of the rule is to not have this thread be an echo chamber for LessWrong and Yudkowsky quotes. As a sister site, Overcoming Bias falls under the same logic (though I think, given that the origin of LessWrong in OvecomingBias constantly becomes more distant in time, I wouldn't mind that rule getting relaxed for OvercomingBias more recent entries.)

But either way, I haven't seen that many lesswrong members participate in "hpmor/reddit" or that many hpmor/reddit members participate in lesswrong, so I think it makes sense to NOT ban hpmor/reddit quotes from this thread...

I don't think Eliezer runs r/HPMOR/ ...

You're misreading the quote. The intention is on the part of the person who designed the gun, not the person who's trying to fire it.

Assuming we're the stupidest possible biological species capable of starting a technological civilization seems almost (though not quite) as wrong as asserting we're the smartest such. In both cases we're generalizing from a sample size of one.

Well, considering it took us thousands to hundreds of thousands of years (depending on whether you buy that certain, more chronologically recent adaptations played a significant role) to start developing the rudiments of technological civilization, after evolving all the biological assets of intelligence that we have now, I think it's pretty fair to infer that we're not that far above the minimum bar.

A species whose intelligence was far in excess of that necessary to be capable of technological civilization could probably have produced individuals capable of kickstarting the process in every generation once they found themselves in an environment capable of supporting it. By that measure, we as a species proved quite resoundingly lacking.

Assuming we're the stupidest possible biological species capable of starting a technological civilization seems almost (though not quite) as wrong as asserting we're the smartest such.

I agree they're both very wrong, but I don't think the levels of wrongness are as close as you suggest. The former sounds much, much wronger to me. We're much more likely to be close to the dumb end than close to the smart end.

It's not that we've seen a large number of biological species capable of starting a technological civilization and we're dumbest of them. It's that we've seen a large number of biological species incapable of starting a technological civilization and we're only slightly smarter than any of them.

We know we're at the boundary of being able to start a technological civilization because we can do it and we've seen biological species within an epsilon neighborhood that cannot.

To which I made roughly the same comment then that I did this time.

It seems like your comment misses the point of the Unix philosophy, which is that the designers do not undertake to know in advance exactly which user actions are "stupid" and which are "clever". Unix is supposed to be a solid framework in which you can do things; figuring out what's stupid and what's clever is left to the user. It is an expression of fundamental designer trust in the user.

A car can be driven on the road, or it can go onto a sidewalk and kill pedestrians and the driver. In comparison, a train or trolley can't easily go off its rails.

Is a car misdesigned because it is an open-ended, unconstrained tool? Not necessarily; you must weigh the costs of other possibilities against their benefits.

Unix is deliberately built as a general open-ended collection of tools. It enables many more things to be done than other systems which start by presuming a list of things the user wants to do. And some of the things it enables are mostly harmful, although they too are sometimes useful. It's not misdesigned; it makes a design tradeoff that is the correct one in some situations, and not in others.

I think the chief obstacle to preventing stupidity without preventing cleverness is that there are clever ideas you haven't thought of yet that sound stupid. There's also the fact that what is stupid under one set of circumstances is clever under others.

Suppose I was designing a new car radiator, and I decided that I wanted to prevent idiots from, say, filling it with motor oil, so I built a system which would prevent the addition of anything other than water or antifreeze. Then suppose the radiator sprang a leak. At this point, the owner of the car might want to use the old shade-tree mechanic's trick of putting an egg in the radiator that will flow to the leak and produce a plug (or, more intelligently, a synthetic compound that does it more effectively and with less likely damage to the machinery) ... but they can't, because of the thing I did to prevent stupidity.

These two concepts do not contradict each other. Unix can allow "doing stupid things" AND at the same time make them "difficult to impossible". So, the conclusion that Unix was misdesigned is not correct, at least not basing on your definition

When the right thing can be made obvious and easy, it generally does so. It also makes the wrong thing difficult. However, short of giving it AI, there is no way for Unix to tell a clever thing from a wrong thing, so it lets you do both, but only if you're logged in as root.

Being able to design stupid things is an important skill for any designer. Steering away from it tends to reduce your process to cached thoughts.

You can still hammer your thumb with a hammer. That doesn't make it a badly designed tool for driving nails.

In unix, all you have to refrain from doing is rm -rf / and you will be fine.

Thank you! Or as Google Translate has it, "Scholarship has slightly into the leaves, without carrying fruit."

It's entirely consistent to believe that the level of something is too high and has been too high for a long time, yet to not oppose it in principle.

It may be consistent, but is it common? Especially in political debates?

The standard version is that in looking at the stars we realize our own insignificance. Apart from the sheer non-sequitur from "of comparatively small dimensions" to "insignificant" (to whom?!), such tropes may serve as a sort of moral anaesthetic: "Taking the Hubble View, does it really, fundamentally matter if I steal money from my investors?"

The general problem is that of making leaps from empty empirical facts to (almost certainly mistaken or self-serving) moral conclusions.

Truth.

Correct.

Yes I did read the book.

This is factually false.

What exactly was claimed to be a fact and how do you know it's false?

Primitive life is pretty great if you have low population density

Um. Really? What do you call primitive life, then? Does it include contemporary medicine, for example?

Me, I'll take air conditioning, indoor plumbing, mosquito control, and antibiotics any day...

Spending 9 hours a day 5 days a week sitting in a cubicle staring at a monitor and typing in numbers is horrible in its own ways, which the author prefers to accomodate and ignore.

I 100% agree with this. As a kid, I used to daydream about going and living by myself in the wilderness, partly because sitting in a classroom all day was so awful. (The other aspect is that I didn't like people much when I was 10). I've compromised by finding a job where I don't have to sit down and type numbers into a computer...at least, not much. Also I like people a lot more now.

I have a sneaking suspicion that's not what the OP meant by "Nature."

Because the consequences of losing are so terrible, people tend to avoid serious fighting if they can. Being hunted - a far more likely state - is decidedly un-fun.

It's actually from the prologue of a romance novel, and the narrator is a werewolf.

You're using this remarkable set of interacting or interdependent components of interlinked hypertext documents in a global system of interconnected computer networks powered by a flow of electric charge to whine about a rationality quote! How quaint.

Well, there's the scenario where one person does both the engineering and the drinks, but only has a limited amount of effort in his job to exert, and he chooses to devote all of that effort to engineering the plane and only a tiny portion of it to ensuring the quality of the drinks. That scenario is obviously absurd.

But if you slightly modify that, person->company and effort->money, that's pretty much what's going on. The company has a limited amount of money to spend, and spending most of it on engineering and almost nothing on drinks has similar dynamics to a single worker who's choosing to spend all his time on engineering and almost nothing on drinks. Even if the company internally contains several workers and the engineer and the drink maintainer are different people.

back to the original quote for a bit...Dresden actually complains quite a bit. But after dealing with flaming monkey poo (literally), a white court vampire as a friend, using a cleaning spell to deal with some giant scorpions, and who knows how many dead bodies (some of which were animated)....drinks seem really, really shallow to him. Not to mention he's trying hard not to think too much about how, if he lets his magic the least bit off the leash, it will crash the plane. (something about complicated technology seems to override the rule "cannot accomplish what you don't believe in accomplishing")

Moving back to real life, someone is willing to complain about the drinks while someone else is being mugged.

Furthermore, If the person's REAL complaint is about the unpleasant security measures, cramped seats, and air pressure changes, complaining about the drinks, even if the complaint gets the drinks to improve, will not really optimize much.

Nonetheless it is important to have a firm grasp on the progress we have already attained. It's easy to go from "we haven't made any real progress" to "real progress is impossible". And so we should acknowledge the achievements we have made to date, while always striving to build on them.

You're right that it would indeed be a mistake to say "things are already great, let's stop here". But then, "things are really awful, so let's get better" doesn't sound quite right either. The attitude I would lean towards, and which I think is compatible with the quote, is "things are already pretty awesome, how could we make them even more awesome?".

I don't think the comparison is to complaining about very bad things happening elsewhere, it's more like "we've got it so much easier than our forebears, why do people still complain about misspellings on the internet? They should be grateful they have an internet."

One fallacy is that the person who says sort of thing fails to realize that complaining about complaining is still complaining.

I'm not saying we should settle for anything. Certainly not.

But to forget the awesomeness that already exists is a mistake with consequences. When looking at the big picture, it's important to realize that our current tradjectory is upwards. When planning for something like space travel, it's important to remember that air travel sounded just as crazy a hundred years ago. And when thinking about thinking, it's worth remembering that this same effect will hit whatever awesome thing we think of next.

It seems comprehensibly big. It would take between three and four years to walk around the Earth, walking for a sustainable number of hours at a reasonable pace every day, if you could walk around it in a straight line.

I think that shorting out effect is what is meant by "mind-bogglingly".

People have walked from yellowstone to the grand canyon. I couldn't do it myself, but I can read their accounts and understand them.

Earth is big, but our minds are amazed, not boggled. It's with the galaxy that we just start thinking "system error".

Oz: you have prior probabilities for theories A, B, and C; you hear a new theory D that you hadn't previously considered, and you recalculate the influence of previous evidence to see how much credence you should give D.

This seems slightly off both in terms of what (the writer intends us to infer) is going on in Oz's head, and what ought to be going on. First, it seems that Oz may have considered vampires or other supernatural explanations, but dismissed them using the absurdity heuristic, or perhaps what we can call the "Masquerade heuristic" - that's where people who live in a fictional world full of actual vampires and demons and whatnot nevertheless heurise as though they lived in ours. (Aside: Is 'heurise' a reasonable verbing of "use heuristics?") Upon hearing that his friends take the theory seriously (plus perhaps whatever context caused them to make these remarks) he reconsiders without the absurdity penalty.

Second, what should be going on is that Oz has theories A, B, C with probabilities adding up to 1-epsilon, where epsilon is the summed probability of "All those explanations which I haven't had time to explicitly consider as theories". Just because he's never explicitly formulated D and formally assigned a probability to it, doesn't mean it doesn't have an implicit one. Once it is picked out of hypothesis space, he can detach it from the other previously unconsidered theories, formally assign an initial probability much smaller than epsilon, and update from there. Of course this is not realistic as a matter of human psychology, but what I'm arguing is that "I never thought of theory X before" does not actually demonstrate that "Oh yeah, theory X makes a lot of sense" is not a Bayesian update. It just means that the updater hasn't had the processing power to fully enumerate the space of available theories.

Does Oz already know that he's a werewolf at this point? That would seem to bring "vampires exist" into the realm of plausible hypotheses.

Discworld is set in a time roughly parallel to the late 1700s or early 1800s. Medicine didn't really work, and livestock were significant capital.

That makes it cheaper for people to argue against the information than to argue for it.

Not necessarily. It makes it cheaper for people to argue against whatever slim fraction of the information they can put up as a strawman without risking their own tongues. But it's hard to put up a real argument against an opposition that you can't really even quote.

And that increases that chance that people will finally consider him a liar.

Not if that strawman is easily blown away by whatever samizdat eventually conveys the full information.

Yvain explains some of the mechanisms better than I could in points 5 through 7 here:

http://squid314.livejournal.com/333353.html

The effectiveness of silencing someone really depends on how common such silencing is for a given regime. For example, if a regime silences all critics (regardless of whether they tell the truth or lie) an individual act of censorship doesn't carry any information about whether the censored info was true or false.

On the other hand, tons of claims are made against the US government every day, and no action is taken against almost all of them. If the government suddenly acted to silence one conspiracy theorist, far more attention would be paid to his claims, and the action would likely backfire.

How is that quote a spoiler? Also, how long does a work need to be out before spoilers are no longer an issue? Is it ok if I tell you that Macbeth dies at the end?

Does it actually help?

Yes. One option is to use it as a memorable trigger- "Oh, I'm making mistake X, like the proverb"- and then amend behavior. (This is one of the reasons why it's worth trying to word proverbs as memorably as possible- rhyming helps quite a bit. If your actions you want to jigger, then do not fail to set a trigger! Sometimes it works better than others.)

A superior option is, upon seeing the maxim, to contemplate it fully, and plan out now how it could be avoided in some way, and then practice that offline.

In general, though, de Montaigne is highlighting the general thrust of Less Wrong. Knowing the ways in which people in general make mistakes is most useful to you if you use that knowledge to prevent yourself from making that mistake, and a general mistake people make is to not do that!

any new idea will be treated as a variant of something the listener has already thought of or heard.

From a Bayesian point of view, this is as it must be. People have priors and will assess anything new as a diff (of log-odds) from those priors. Even understanding what you are saying, before considering whether to update towards it, is subject to this. You will always be understood as saying whatever interpretation of your words is the least surprising to your audience.

BTW, this is standard in natural language processing (which is what a lot of Schank's AI work was in). When a sentence is ambiguous, choose the least surprising interpretation, the one containing the least information relative to your current knowledge.

The narrower your audience's priors, the more of a struggle it will be for them to hear you; the narrower your priors, the more you will struggle to hear them.

Having shown how Schank's Law is but an instance of Bayesian inference, I trust you will all find it acceptably unsurprising. :)

[A]ny new idea will be treated as a variant of something the listener has already thought of or heard.

This does raise the question of how anyone learns anything in the first place. :)

A less cynical take on this is that people compare what they hear to their previous experience (stored in compressed form) and accept or reject it depending on how well it matches.

The convention I was taught is that "This... and that" is quoting someone who pauses after 'This', while "This [...] and that" indicates that I elided something. This seems to me both useful and clear.

Also, I'd try to avoid ellipses in a quote unless you are in fact leaving something out. I suspect here you just meant it to reflect the doctor's speech pattern, but it's a bit confusing.

"Word... Word" has a different meaning to "Word ... word". The usage in this comment would not confuse many and replacing the ellipsis with a period would change the meaning of the quote. As it happens most sources I can find don't include the ellipsis so the addition would be a mistake. So I agree with you in this instance and agree in general with a slight modification to "adding ellipses to a quote".

Your comment, with which I agree, inspired me to post this quote.

You need to make a stronger case. Your neurobabble is the result of trying to do philosophy after science rather than of trying to do science without philosophizing. The Copenhagen interpretation (which is, by the by, still in vogue outside of LessWrong, please stop the groupthink) allowed people to get on with their science instead of getting bogged down in its bewildering philosophical implications. Ignoring philosophy was the right thing to do. So I see both your examples as proofs that we should do science without any philosophy.

No, it’s absence of evidence if you notice that we have ready access to high-resolution videos of innumerable rare events and elusive animals.

And yet the best footage of UFOs, ghosts, and Bigfoot still consists of some blurry, hazy, shiny, or dark blob smeared somewhere in a couple of frames of a shaky video at the absolute limit of the camera, exactly the same as forty years ago. Which is exactly what you’d expect to see if these were in fact normal things and optical artifacts that are perfectly explainable when they’re actually close enough to see.

You think Randall Monroe is making this mistake?

Well, "magic" is in common speech shorhand for "I have no idea how this works and I don't think anyone else has either." Science in common speech is "someone somewhere smarter than me knows how this works." (barring No One Knows what Sciece doesn't Know).

The problem with calling things magic is that it serves as a Semantic Stopsign.

And I must paraphrase Sam Huges Ra, a story in which "magic" is a newly discovered branch of physics: Calling magic 'magic' was an incredibly bad decision in the first place.

In your life, salsa dancing ability is definitely not the sole metric you wish to be optimizing for.

Things you presumably want to optimize might be something like personal happiness, bettering the world or wherever you find meaning.

If one truly wanted to drop resources into optimizing salsa ability, I'd imagine filming the dance floor from a few cellphones every week, uploading the video to youtube and paying a few experts on a salsa forum to give the dancers a rating and feedback would give a somewhat valid metric that you could go about tracking, quantifying and optimizing.

But I presume that that is not the primary goal of most salsa-goers. I guess that people go to salsa dancing nights because they are fun, good exercise and you get to socialize with a group of guys and girls who want to dance with girls and guys.

Can you try tracking happiness? Sure, why not. Have a prompt to record happiness appear at random intervals, or write a journal to note big highs or lows. Then questions like "do things like salsa increase my happiness more than things like video games" or whatever become addressable in a slightly more informed way.

I agree with you that your mind should not be on contrived proxy goals while you are salsa dancing. Better to be enjoying the salsa. But I disagree with the implication that because many metrics are tangential to the 'true' goal, careful measurement is flawed. It it still the fun/happiness that you care about, just now you are doing a smarter job of tracking it.

On the other hand, if I'm empathic, if I'm in a happy mood and get outside of my head I'm more likely to have success in making woman enjoy dancing with me.

And there's your measurement! (But then the school I graduated from taught quantum mechanics instead of Taylorism, so I may have an unusually expansive idea of what constitutes a measurement.)

Umm, is it me being sleepy, or did he get P(I picked up a seashell) and P(I'm near the ocean) mixed up in the equation? P(near the ocean | evidence) shouldn't be inversely proportional to P(near the ocean).

Well spotted. Bayes rule is p(A | B) = p(B | A) * p(A) / p(B). This cartoon sees to mixed up p(A) and p(B) just as you note.

Shouldn't that be: "All you get if you are good at digging holes is a bigger shovel."?

Wikipedia has a couple of citations giving it to Healey, although that is hardly definitive. The First Law of Holes has its own subsection on his Wikipedia page.

If wishes were fishes, we'd all cast nets.

Gurney Halleck in Dune by Frank Herbert

Ahem..

I think that you may have misunderstood the point that I was trying to make. I am not advocating excessive caution. Rather, I value self-knowledge and knowledge of the environment and the people you interact with in that environment. Obviously, a certain amount of margin of error should be included in any decision making.

It has been my experience as a driving instructor that most pupils are entirely too cautious especially on faster roads where going too slowly may cause a following vehicle to attempt an unsafe overtaking manoeuvre

There could be a hedonic treadmill effect: as you get richer, you get more things, but eventually you get used to it and it stops being better than your old life. But you still don't want to give your wealth away, because you have gotten used to having more stuff, and you're not sure that you would get used back to your old way of life the way you get used to your new one.

My superficial knowledge of Seneca and the stoics doesn't allow me to debate the premise fully. It does tell me that the argument that it is better can be debated. That people prefer to be rich does not make it better.

An aside: A rich man that gives away his wealth is not equal to a person that is poor from the start or has lost his riches. The person that gives it away, keeps his connections, earns respect and, generally, is in a position re-earn a fortune.

Rich can be worse than poor, knowledge can be worse than ignorance, sickness can be better than health, and death can be better than life. But none of these are the way to bet.

It is also worth considering the relevant causal graph. Wealth --> Happiness allows of such exceptions. But what do they look like in terms of the causal graph Wealth --> Happiness <-- Character? If someone can't handle a sudden accession of money, is it the money or their personal failings that should be blamed? If you see a friend in that situation, do you advise them to get rid of their money or learn to handle it better?

A lot of what looks like time discounting is really uncertainty discounting.

Cowen is explicitly discussing time discounting. As he writes, "Should we—because of positive discounting—not give them enough fuel to make a safe landing?" (emphasis added) There may of course be other reasons for treating these people differently, including uncertainty about the long-term future, but Cowen is not focusing on these reasons here.

You're right that Cowen got it backwards, but you're wrong about this:

Acceleration is what causes the opposite, e.g. turning the spaceship around to come back

Acceleration is not the cause. The reason the astronauts age less is that the path they follow through space-time corresponds to a smaller proper time than the path followed by people who remain on the Earth, and the proper time along a path is what a clock following that path measures. So it's a geometrical fact about the difference between the two paths that causes the asymmetrical aging, not the acceleration of the astronauts.

To make this obvious, it is possible to set up a scenario where another group of astronauts leaves Earth and then returns, accelerating the exact same amount as the first group, but following a path with larger proper time. This second group of astronauts will age more than the first group, even though the accelerations involved were the same.

A lot of elementary presentations of relativity identify acceleration as the relevant factor in twin paradox type cases, but this is wrong (or, more charitably, not entirely right).

if you decline to condemn them to death, how are they different from other “residents” in the distant future?

Probably because some are more real and others are less so.

Can you explain in more detail what you mean by this?

Well, it was a very good heuristic up until photography and than television were invented.

This predates television entirely and photography all but entirely.

Whatever the actual rule is, the next time it should be spelled out explicitly.

I think we're looking at the wrong kind of criticism. Like, the kind of criticism you can make with almost equal ease of results that will and won't turn out to replicate later.

As you know, I agree with you that Roberts is incorrigibly biased, and I liked your earlier post on this. But I think we can be critical in the sense you have in mind, and still try to cultivate the attitude that I take Roberts to be hinting at. Perhaps this is not very clear in the passage I chose to quote though.

Thanks.

There's a saying about India, "Whatever you can rightly say about India, the opposite is also true."

Very true, although where the USA is now is really not "the Constitution" simpliciter, so much as "the Constitution + all case law."

cough 30 second case cough

Sunk Cost? Also, Tsuyoku Naritai - we can do so much better with the knowledge currently available to us.

It's a quote against which one can test their rationality, maybe?

• When someone died or when it was made has no relevance; only its merit in guiding a government is relevant.
• Their moral and political views don't matter either, unless contained in the present US Constitution; this seems like argumentum ad hominem at first glance, but one needs to check the claim before evaluating its persuasiveness.
• One must argue that knowledge of modern America confers enough of a benefit to forming a working governmental body that scrapping and rewriting the entire U.S. Constitution is preferable to the amendment process.

In an effort to steelman: perhaps the Professor meant to indicate that with the advent of the internet, a representative democracy is no longer the most effective means of running a government by the people, for the people, and of the people. If he was feeling radical, he may have been hinting at how political science has developed as a discipline since the Enlightenment era when the principles founding the U.S. government were theorised; perhaps the best solution is a flexible one, able to adapt to the political system most effective at running an efficient government while still remaining resistant to tyranny. Exempli gratia a futarchy for four years, some form of crypto-direct democracy for eight years, a modified version of Finland's government for ten years, etcetera.

I'd guess the quotee wouldn't call generic space opera "science fiction" either. I sure wouldn't, myself.

Thanks, retracted.

Timeless decision theory, what I understand of it, bears a remarkable resemblance to Kant's Categorical Imperative. I'm re-reading Kant right now (it's been half a decade), but my primary recollection was that the categorical imperative boiled down to "make decisions not on your own behalf, but as though you decided for all rational agents in your situation."

Some related criticisms of EDT are weirdly reminiscent of Kant's critiques of other moral systems based on predicting the outcome of your actions. "Weirdly reminiscent of" rather than "reinventing" intentionally, but I try not to be too quick to dismiss older thinkers.

AI workers in the 1960s should have read Wittgenstein's discussion of games to understand a key problem with building symbolic logic systems that have an atomic symbol correspond to each dictionary word.

Can you elaborate on this? It sounds fascinating. I confess I can't make heads or tails of Wittgenstein.

Moral philosophy in general is under-appreciated in FAI discussion in this community.

LW Metaethics Sequence : Solving Actual Moral Dilemmas as Inventing Peano Arithmetic : Inventing Artificial Intelligence. In short, an important and insightful first step. Hardly a conclusive resolution of the outstanding issues.

But if we want Friendly AI, we need to be able to tell it how to resolve moral disputes somehow. I have no idea if recent moral philosophy (post-1980) has the solutions, but I feel that even folks around here underestimate the severity of the problems implied by the Orthogonality Thesis.

The emphasis I used was in the original, but I agree that it would work better with the emphasis on "think."

(I don't think the cat experiments are very conclusive here. As far as I know, the functions that have been identified in the early visual system are things like edge detection and motion detection. But such functions are used for video compression. So not only could a radio set perform them in principle, an ordinary digital TV set already does.)

I don't think this is quite where the analogy was. The brain's information-processing features you describe seem to be analogous to the radio's volume and clarity... it seems Eagleman was trying to compare the radio's content not to the brain's content, but to consciousness or something. At least, that's the best steelmanning attempt I've got.

Fair enough.

I appreciate the comment on pronunciation; dictionary.com and wiktionary disagree with me, but I got the pronunciation from Google. I assume you're correct.

I think that it's reasonable to say that this quotes list should be restricted to quotes that help you become more rational, but if so, there should be some indication that this is the goal. Despite that, the idea applies to anyone, one's self included, who persists in some action shown conclusively to be incorrect (Such as pronouncing nuclear "Nookular"). There may be a reason to do so, but that does not change the idea. It is attributed to either Erasmus, or his contemporary, Richard Pace.

Wikipedia has the story:

"The term originates from a story about a priest who misread sumpsimus [latin:"we have received"] as mumpsimus. After being told about his mistake he stated that he had been using mumpsimus for a number of years and was not about to change, saying "I've got so used to using the word mumpsimus that I'll just go on saying it that way.""

Certainly: I thought it was funny.