Rationality Quotes February 2013

I think the Brecht quote is somewhat misleading. The problem is not that not enough people want/demand goodness, the problem is that it is too easy to profit by cheating without getting caught.

Then try to make it politically profitable to help sustain those changes you make. Make it so painfully obvious that the only reason to remove those changes would be for one's unethical gain that no politician would ever do so. The problem then though, is that people end up just not caring enough.

Couldn't I also set up the system to try to exclude the wrong people from ever getting power?

Not if having power over others turns the right people into the wrong people.

Couldn't I also set up the system

No. No-one can set up the system. The most that anyone can do is introduce a new piece into the game, pieces like Google, or Wikipedia, or Wikileaks.

I think the spirit of the quote is that instead of counting on anyone to be a both benevolent and effective ruler, or counting on voters to recognize such things, design the political environment so that that will happen naturally, even when an office is occupied by a corrupt or ineffective person.

The nice thing about working with incentives is that they're pretty stable relative to political leanings. I'd expect a given person's perceptions of politicians' level of corruption or incompetence or any other negative adjective you can think of to depend almost entirely on party affiliation, but you can actually leverage that to get changes in incentive structures passed: just frame it as necessary to curb the excesses of those guys over there, you know, the ones you hate.

And in any case the quote works just as well for the governed. As anyone who's ever moderated a large forum can tell you, playing with incentives works almost embarrassingly well and quickly compared to working on sympathy or respect for authority. Of course, it's also harder to do.

If you have corrupt politicians, blame the voters. The politicians did not vote themselves into the office. (Unless they own the vote-counting machines factory.) I guess the quote suggests that "making it politically profitable for the wrong people to do the right things", whatever precisely that means, could still be easier than replacing the whole population of voters; or at least the majority of them.

Agreed. It's too easy to pander to a base that doesn't expect you to be good, just deliver a few things... things that matter a great deal less than the cumulative effect of having the right people in charge.

The story appears to be apocryphal. I've heard many versions of it associated with various famous scientists. The source quoted is a collection of jokes, with very low veracity. Additionally, there are no independent versions of the story anywhere on Google. By the way, the quoted date of Sommerfeld's death is also incorrect. I wonder if there even were (unpowered) ceiling fans in Munich's trolleys during that time.

Perhaps because pressure is (approximately) constant, for every molecule going into the car, one must leave it (on average)?

Unfortunately, this seems to be the default way humans do things.

That whole testing sequence is one of the best examples in film of how to distinguish what's expected of you from what's actually a good idea.

(Or in that specific case, what seems to be expected of you.)

And they ship the characters the fans want.

And no bugs.

Most people, when giving advice, don't optimize for maximal usefulness. They optimize for something like maximal apparent-insight or maximal signaling-wisdom or maximal mind-blowing, which are a priori all very different goals. So you shouldn't expect that incredibly useful advice sounds like incredibly insightful, wise, or mind-blowing advice in general. There's probably a lot of incredibly useful advice that no one gives because it sounds too obvious and you don't get to look cool by giving it. One such piece of advice I received recently was "plan things."

Well, a lot basic rationality literally seems to be about doing what is almost obvious but is hard to do because of bugs in your cognitive architecture. This reminds me of the following quote by Elon Musk in an interview where he was asked what he would say to new start-up founders:

Try to get together a group of people to do something useful. This may seem like an obvious thing, but often people will organize into a company that doesn't produce anything useful.

And by the same author:

That's Jeff Atwood. The quote is from Joel Spolsky. While the two both work together on Stack Exchange, they're different individuals.

Yep.

Not sure what an "accurate emotion" would mean, feel like some sort of domain error. (e.g. a blue sound.)

Contrasting "accurate beliefs and useful emotions" with "useful beliefs and accurate emotions" would probably make a good exercise for a novice rationalist.

This is addressed by several Sequence posts, e.g. Why truth? And..., Dark Side Epistemology, and Focus Your Uncertainty.

Beliefs shoulder the burden of having to reflect the territory, while emotions don't. (Although many people seem to have beliefs that could be secretly encoding heuristics that, if they thought about it, they could just be executing anyway, e.g. believing that people are nice could be secretly encoding a heuristic to be nice to people, which you could just do anyway. This is one kind of not-really-anticipation-controlling belief that doesn't seem to be addressed by the Sequences.)

If useful doesn't equal accurate then you have biased your map.

The most useful beliefs to have are almost always accurate ones so in almost all situations useful=accurate. But most people have an innate desire to bias their map in a way that harms them over the long-run. Restated, most people have harmful emotional urges that do their damage by causing them to have inaccurate maps that "feel" useful but really are not. Drilling into yourself the value of having an accurate map in part by changing your emotions to make accuracy a short-term emotional urge will cause you to ultimately have more useful beliefs than if you have the short-term emotional urge of having useful beliefs.

A Bayesian super-intelligence could go for both useful beliefs and emotions. But given the limitations of the human brain I'm better off programming the emotional part of mine to look for accuracy in beliefs rather than usefulness.

A useful belief is an accurate one. It is, however, easy to believe a belief is useful without testing its veracity. Therefore it is optimal to test for accuracy in beliefs, as opposed to querying one's belief in its usefulness.

Conversely, why not both accurate beliefs and emotions?

Let useful come into play when choosing your actions. This can include framing your emotions - but if you just go around changing your emotions to whatever's useful, you're not being yourself.

Agreed. The idea that I should be paying attention to and then hacking my emotions is not something I learned from the Sequences but from the CFAR workshop. In general, though, the Sequences are more concerned with epistemic than instrumental rationality, and emotion-hacking is mostly an instrumental technique (although it is also epistemically valuable to notice and then stop your brain from flinching away from certain thoughts).

Indeed, accurate emotions appear a better description. Consider killing someone might free up many opportunities, and would only have the consequence of bettering many lives; the useful emotion would be happiness at the opportunity to forever end that person's continued generation and spread of negative utility. Regardless of whether the accurate emotion might yield the same result, I'd trust the decisions of they who emote accurately, for though I know not whither hacking for emotional usefulness leads, a change of values for the disutility of others I strongly suspect.

It's the point of the problem, anyway.

SInnett is probably a very good designer, but the battery design was outsourced.

Shoes aren't just about positional social status, are they? (I mean, the difference between a $20 pair of shoes and a $300 pair of shoes mostly is, but the difference between a $20 pair of shoes and no shoes at all isn't, is it?)

I think people just accidentally conflate keeping problems in perspective with the idea that the existence of bigger problems makes the small problems negligible and therefore equivalent to non-problems.

I've seen this happen with positive things too; sometimes you won't mind repeatedly doing small favors for someone and they start acting like you not minding means the favor is equivalent to doing nothing from your perspective, which is frustrating when your small but non-zero effort goes unacknowledged.

It's sort of like approximating sinθ as 0 for small angles. ^_^

I think both your comment and the quote are forgetting the instrumental purpose of crying and/or feeling bad.

But if you look at it other way, then pointing out unpleasant facts about other people's condition (that don't apply to us) is equivalent to pointing out good facts about our condition, which should make us feel better, as it leaves us in a better world than we started out in.

What's an actual utility?

What we call "getting perspective" on a situation is basically a name for updating your baseline expectation for how reality "ought to be" at the present moment.

I agree that resetting your baseline is often important if you think that your lack of shoes is a soul-crushing awfulness. This quote is mainly arguing against the attitude that says "you have feet therefore your shoe problem is a non-problem, don't even bother feeling bad or working on it". It's comparatively very minor, but it should be fixed just like any other problem. This quote is arguing against resetting your baseline to the point where minor problems get no attention at all.

The kind I mean is the kind where reality feels awful or crushingly disappointing if it's not the way it "ought" to be

“need”

Also from the review:

A pacemaker malfunction isn't automatically fatal. In most cases the patient's heart will still beat, although with an abnormal rhythm. The severity of a pacemaker problem depends on the type of malfunction as well as the severity of the patient's condition. EM interference can cause problems, but major problems are rare considering the amount of EM interference pacemaker patients are exposed to. Pacemakers are designed to minimize these problems. It's hard to believe that dozens of pacemaker patients with various heart conditions and different makes and models of pacemakers would simultaneously die from microwave exposure.

If I recall correctly, he also pointed out that the fact they had invited two experts on magnetic fields was also a strong clue.

... I had not known about red buttons on SMBC.

roll d20... success on 'resist re-binge' check.

Umm... how do I use the red button on a mobile device? (I also have this problem with xkcd.)

If your sock does not have feathers or duck-shit on it, then it is somewhat more likely that it has not been sat on by a duck.

My shoe doesn't look like a duck in my closet, but it also doesn't look like the absence of a duck in my closet.

I'm not sure I understand this. Do you mean that the way your shoe looks is not evidence for the presence or absence of a duck somewhere in your closet?

I think the original quote was meant to imply that as long as your shoe doesn't have the properties that differentiate ducks from non-ducks then your shoe possesses the absence of duck properties and should be assumed to be a non-duck. In other words, for a given object each property must have a binary value for duckness and when all properties have non-duckness values, you should conclude that the object as a whole has a non-duckness property.

I get confused by too many negatives and ducks.

Fixed.

I. J. Good's original, which I've somewhat abridged, explicitly specifies that there are no competitors who cause visible losses/gains after the invention is rejected.

The success of Market-Based Management / Koch Industries appears to be due at least in part to their focus on NPV at the managerial level. You get stories like (from memory, and thus subject to fuzz) the manager of a refining plant selling the land the plant was on to a casino which was moving to the area, which he was rewarded for doing because the land the plant was on was more valuable to the casino than the company, even after factoring in the time lost because the plant was shut down and relocated. The corporate culture (and pay incentive structure) rewarded that sort of lateral use of resources, whereas a culture which compartmentalized people and departments would have balked at the lost time and disruption.

Wikipedia may ultimately have to do one of two things, or both:

1) Provide better structure for alternate versions of contested ideas

2) Construct a practically effective demarcation between strictly factual domains, and anything more interpretive.

Such a demarcation will always be challenged; I don't see any way around that, but I'd also insist that it's necessary for our sanity. Supposed it was possible, maybe using a browser with links to a database, to try to "brand" (or give the underwriters seal of approval to) those pages that provided straightforward factual assertions, and unretouched photographs, and scans of original source texts, such as all newspapers of which a copy still exists), and to promote the idea that the respectability of any interpretive or ethical claim consists very largely in its groundedness in showing links to the "smells like a fact" zone.

I imagine that depends on what we're willing to consider a "person-shadow".

Any thoughts on what your minimum standard for such a thing would be?

For example, I suspect that if we're ever able to construct artificial minds in a parameterized way at all (as opposed to merely replicating an existing mind as a black box), it won't prove too difficult thereafter, given access to all my writings and history and whatnot, to create a mind that identifies itself as "Dave" and acts in many of the same ways I would have acted in similar situations.

I don't know if that would be a worthwhile thing to do. If so, it would presumably only be worthwhile for what amount to entertainment purposes... people who enjoy interacting with me might enjoy interacting with such a mind in my absence.

Relevant: Greg Egan, "Steve Fever".

How would this encourage them to actually value logic and evidence instead of just appearing to do so?

You can't reason someone out of a position they didn't reason themselves into.

I don't think this is empirically true, though. Suppose I believe strongly that violent crime rates are soaring in my country (Canada), largely because I hear people talking about "crime being on the rise" all the time, and because I hear about murders on the news. I did not reason myself into this position, in other words.

Then you show me some statistics, and I change my mind.

In general, I think a supermajority of our starting opinions (priors, essentially) are held for reasons that would not pass muster as 'rational,' even if we were being generous with that word. This is partly because we have to internalize a lot of things in our youth and we can't afford to vet everything our parents/friends/culture say to us. But the epistemic justification for the starting opinions may be terrible, and yet that doesn't mean we're incapable of having our minds changed.

No one really believes God will protect them from harm...

I have some friends who do... At least insofar as things like "I don't have to worry about finances because God is watching over me, so I won't bother trying to keep a balanced budget." Then again, being financially irresponsible (a behaviour I find extremely hard to understand and sympathize with) seems to be common-ish, and not just among people who think God will take care of their problems.

Sadly, that only works on a natural-selection basis, so the ethics boards forbid us from doing this. If they never see anyone actually failing to survive, they won't change their behavior.

If you threaten someone in their survival they are likely to get emotional. That's not the best mental state to apply logic.

Suicide bombers don't suddenly start believing in reason just before they are send out to kill themselves.

Soldiers in trenches who fear for their lives on the other hand do often start to pray. Maybe there are a few atheists in foxholes, but that state seems to promote religiousness.

They beat you up. People who haven't specialized in logic and evidence have not therefore been idle.

And in some instances that would likely be what we call logic or evidence.

Haha, didn't occur to me until I read your comment, so there's one data point for you.

It's not just you. It was comprehensible but annoying for approximately the first 10 minutes, and then it became completely muddy. I hope there's a transcript somewhere.

I wouldn't think so, but I wasn't expecting five upvotes on my comment saying so, either. Maybe we really are that pedantic.

This is only incidentally anti-deathist, though; its substance has more to do with popular reactions to controversial ideas. Which doesn't seem all that shiningly rational to me either, but perhaps I'm missing something.

Or perhaps there are more criteria (aesthetic, informational, other) by which these quotes may be judged than whether they are anti-death or not.

And that other quote is neither ASoIaF nor TV series, it's a misquotation.

Three things, in no particular order:

• I seem to recall that, in some obscure language, each noun has an agency level and in a sentence the most agenty noun is the subject by default, unless the verb is specially inflected to show otherwise: for example, “[dog] [bite] [man]” would mean ‘a man bit a dog’, regardless of word order, because the noun “[man]” has higher agency than “[dog]”.

• Would you sooner see a tiger chasing a man, or a man running away from a tiger? If the former, it's not just the fact that butterflies are not human, it's the fact that the butterflies are small.

• I think that, at least in the case of the lion, it would also depend on whether the two of them are moving towards the left side or the right side of my visual field. I heard that in _The Great Wave off Kanagawa_ the boats are intended to look more agenty than the wave, but for Western people it will typically look like the other way round (due to Western languages being written from left to right), and for a Westerner to get the right effect they'd have to look at the picture in a mirror. (It works for me, at least.)

Don't good hunters have good mental models of their prey? I mean I get that you're thinking that it wouldn't help to feel sympathy for animals of other species. But it would help in many cases to have empathy, and to see things from the other animal's perspective.

The whole quote:

If you're not making quantitative predictions, you're probably doing it wrong, or you're probably not doing it as well as you can. That's sort of become kind of critical to how we operate. You have to predict in advance. Anybody can explain anything after the fact, and it has to be quantitative or you're not being serious about how you're approaching the problem.

The problems you face might not require a serious approach; without more information, I can't say.

78.544%.

Not necessarily. You don't punish people based on their likelihood of being guilty but based on severity of their crime.

If torture is used as tool to gain information instead of being used to punish it's even more questionable whether the likelihood of being guilty correlates with the severity of the torture. The fact that someone decides to torture to get more information suggests that they have an insuffienct amount of information.

If there a 50% chance that a person has information that can prevent a nuclear explosion, you can argue that it's ethical to torture to get that information.

After the bomb has exploded and you know for certain who did the crime, there not much need to torture anyone.

An interrigator that tortures is more likely to get false confession that implicate innocents. If he then goes and tortures those innocents, you see that people who torture are more likely to punish innocents than people who don't.

I can't help but wonder if they really expect, in 50 years time, that we will live in a world of strictly hetrosexual marriages.

Here are a few things that have at one time or another been considered "obviously inevitable":

• The spread of enlightened dictatorship on the Prussian model.

• The spread of eugenics.

• The control of the world economy by "rational" central planners.

My point is that you appear to be overestimating how well you can predict the future.

What are they possibly hoping to achieve?

I don't think you really believe this argument. In particular if the success of something you opposed seemed inevitable, you'd still oppose it.

Maybe that reasoning isn't the best way to decide to actively do a thing, but it surely counts towards the cessation of resistance to a thing.

What I think is happening is that you support the "inevitable" outcome but are getting frustrated that the opposition just won't go away like they're "supposed" to.

I can't help but wonder if they really expect, in 50 years time, that we will live in a world of strictly hetrosexual marriages. What are they possibly hoping to achieve?

Being elected at some point in the next 3 years. They aren't trying to achieve anything related to homosexual marriages. They don't care.

FWIW, 20 years ago (when my now-husband and I first got together) I expected that I would live in a world of strictly heterosexual marriages all my life.
That didn't incline me to cease my opposition to that world.
So I can empathize with someone who expects to live in a world of increasing marriage equality but doesn't allow that expectation to alter their opposition to that world.

Good observation. I will accept your correction: It's only weird if it doesn't work, and it doesn't work unless you're in Stevie Wonder's presence

That interpretation didn't even occur to me, possibly because I read the whole poem instead of the bit I quoted (and maybe I quoted the wrong bit). Here is the whole thing (it's short). I always feel a bit awkward arguing about how I interpreted a poem, so maybe this will resolve the issue?

(Incidentally, am I the only one mildly annoyed by how people seem to think of "rationality quotes" as "anti-deathism quotes"? The position may be rational, but it is not remotely related to rationality.)

http://en.wikipedia.org/wiki/Executive_dysfunction

In most cases 'executive dysfunction' covers the same territory as 'adult ADHD', but it can also be the outcome of some kinds of brain damage.

This makes me wonder how much my writing skills would improve if I retyped excellently written essays for a while.

Agreed. I'm actually not sure if what I should take away from that introduction is "This material seems easy but isn't, so go through everything carefully even if you think you understand it" or the opposite: "If this book seems easy, it's not advanced enough for you and you already know everything; so read something else instead."

To me, it seems like a horribly hostile approach to teaching people, which comes across as saying, "In order to learn anything from me, you must abase yourself before me." Which is to say, "I am incapable of conveying useful information to anyone who does not present abject submission to me."

But then, it's possible that I'm just hearing Severus Snape (or the class of lousy teachers he is an imitation of) in the "so you think you know everything?" bullshit.

I think the quote's main function is to warn those who don't know anything about programming of a kind of person they're likely to encounter on their journey (people who know everything and think their preferences are very right), and to give them some confidence to resist these people. It also drives home the point that people who know how to program already won't get much out of the book. I quoted it because it addresses a common failure mode of very intelligent and skilled people.

In case you'd be interested in a second opinion: I made it through twenty-one exercises of Learn Ruby the Hard Way a couple months ago, got bored, and have retained almost none of it. I'm probably not the target audience, but that doesn't bother me so much; on the other hand, if I'm not retaining stuff after faithfully going through Hard Way's copybook approach to language acquisition, that doesn't speak well for its efficacy among people who are. Unless for some reason programming experience makes me less likely to retain new languages? But that's (a) counterintuitive, and (b) contrary to data I've seen for natural languages, at least.

In any case, I don't think I'll be returning to the series.

Python is just a programming language. Insofar as it can be said to have a personality, that personality is an accommodating and inoffensive one. The community is pretty good, too; the concentration of assholes is unremarkable, and places like /r/learnpython are quick to help out beginners with questions.

My understanding having completed parts of it is that it's aimed at someone who doesn't know what a programming language is. If you do know, you're probably better off with another book (and you're also probably better off with something other than Python, but that's my personal opinion clashing with Python's opinions).

I thought that unlikely, because it's from last week's XKCD What If?

Maybe Randall has said it before (or borrowed it from someone else).

That seems unlikely; the quote above was only posted about three weeks ago and nothing about Icarus turns up in a search. Can anyone find a duplicate?

Authors are deliberately excluded from all this, on the grounds that they're so in love with what's inside the book that they don't understand what the cover stuff is for. Which is advertising.

The purpose of cover art is not to show the reader what's inside the book.

It's to get his attention from across the bookstore and get him to pick the book up in the first place.

Half-naked women and muscular barbarians are very good for getting teenaged readers to at least take a look. Black and red are good, too. And spiffy hardware, like spaceships. Cut-out covers, foil, blood, all that stuff--it gets attention, and the art and marketing people really don't give a damn whether it agrees with what's inside the book.

The cover gets you to pick up the book and read the blurbs; the blurbs are supposed to convince you to actually buy it. The blurb writer doesn't care any more about accuracy than the art director did; his job is to sell the book, period. One way to do that is to skim through the book and pick out all the most lurid details.

So all this is done without the author's interference. The author might put up a fuss about the half-naked women, since everyone in the story is ninety years old and wearing dirty bathrobes the whole time. The author might object to having his sentimental tale of old age cover-blurbed, "Shocking Love Secrets of the Ancients!" Who wants to waste time arguing with him? Better to shut him out and deliver the package as a fait accompli.

-- Lawrence Watt-Evans

(Except when it's a novel and the text on the back cover spoilers events from the middle of the book or later which I would have preferred to not read until the right time.)

There's something to that, but it's not as if Varian's Microeconomic Analysis is going to have the cover of Spice and Wolf 1.

Probably purely true for some books, but as someone who buys thousands of books a year, my impression is they are very likely to reveal who they think their readers will be (hence a lot of covers say "stay away" to me), and just occasionally they can show a startling streak of originality. E.g. the board designs (there may be no dustjacket) on Dave Eggers' books are uniquely artistic in my opinion, and in this case since he has been seriously into graphics, I don't think it's any accident. You might think "Maybe this book is written by a bold and original person" and IMHO you'd be right. Also, the cover design of The Curious Incident of the Dog in the Night-Time by Mark Haddon kind of sent a message on my wavelength and it was not misleading (for me).

This misses the point a bit due to an equivocation on "assume". In ordinary discourse, it usually means "assume for the purpose of action until you encounter contrary evidence". That's very different from the scientist's hypothetical assumptions that are made in order to figure out what follows from a hypothesis.

I believe that this statement, while correct, misses the point of preemptive debiasing. Yvain said it better.

I don't consider it, I assume it.

But "dumb" and "ignorant" are not points on a line, they are relative positions.

To quote this bloke at a climbing gym I used to frequent "We all suck at our own level".

Or better yet, assume nothing

You always assume things, whether you are aware of it or not. At least by making your assumptions explicit and conscious, you have a better chance of noticing when they are wrong. And assuming "that people are dumb, ignorant, and making mistakes" is a common default subconscious failure mode.

It's a joke, which I assume is intended for a mostly non-physicist audience.

Not at all. It means you don't know about the real mistakes you make (so you can't fix them), and you spend resources trying to fix something that's not really broken.

I take the quote to be normative rather than descriptive. Science is not free from motivated cognition, but that's a bug, not a feature.

if most researchers' backgrounds are likely to introduce implicit biases toward certain conclusions on certain topics, then taking a close look at the experimental structure to rule out such bias isn't merely a good political sop but is actually good science in its own right.

Unfortunately, since the people deciding which papers to take a closer look at tend to have the same biases as most scientists, the papers that actually get examined closely are the ones going against common biases.

If travelling faster than light is possible,
I desire to believe that travelling faster than light is possible;
If travelling faster than light is impossible,
I desire to believe that travelling faster than light is impossible;
Let me not become attached to beliefs I may not want.

That's the thing, the science wasn't good or bad, it was the to decision to give the results to certain people that held that quality of good/bad. And it was very, very bad. But the process of looking at the world, wondering how it works, then figuring out how it works, and then making it work the way you desire, that process carries with it no intrinsic moral qualities.

I think nuclear weapons have a chance of killing a large number of people but are very unlikely to kill the human race.

What happens if you apply the same epistomological standards to claims that someone is racist that you apply to claims from science?

In the Wikipedia article that desribes him he's quoted as suggesting that the US should have retaliated 9/11 with nuclear bombs. That a non-scientific racist position.

Huh, what definition of "racist" are you using here? Would you describe von Neumann's proposal for a pre-emtive nuclear strike on the USSR as "racist"?

He published some material that's widely considered as racist in Psychology Today. I don't see why "racist" is no valid word to describe his conclusions.

I'm not sure what you mean by "racist", however is your claim supposed to be that this somehow implies that the conclusion is false/less likely? You may want to practice repeating the Litany of Tarski.

and the human mind loves to find patterns even when the probabilities of the pattern being a rule are low. Coincidences are correlation.

I thought it was pretty good in its own way, although I expected (coming at it from Shalizi) much more math & science than it actually had.

I am only about one-third of the way through, but it is definitely a good book thus far.

I would not personally buy it, since I only purchase fiction that I am certain I will read more than once, but it is definitely worth reading.

Give a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.

Well, there are lots of cultists running around trying to summon an Elder God. This will almost certainly end in disaster. The options we have to fight this are: a) We can try to stop all Elder-God-summoning related program activities or b) We can try to get there first and summon a Friendly Elder God.

Both a) and b) are almost impossibly difficult and I find it hard to decide which is less impossible.

I suppose it does, in as objective a measure something like 'harmony' is.

I think I parsed that quote less along the lines of 'dude, you hardly know any math and so I won't love you' and more along the lines of 'dude, you seem to have the same taste for movies and music and we can have a conversation -- I love (hanging out with) you'.

The former has an objective measure and thus one can speak of definite growth while the latter is subjective.

"Love you just as you are" doesn't imply "hate for you to change".

After all, you are changing.

To the extent that you already know that nuclear power plants are basically safe they clearly do not apply as an analogy here. Reasoning from them like this is an error.

Yes, but you can say that because you have the independent evidence that nuclear power plants are workable, beyond the mere say-so of a couple of scientists. You don't have that kind of evidence for AI safety.

Also, this:

Non-Friendly AI is no Elder God. It kills you, at worst.

... is not a given. What makes you think that the worst it would do is kill you, when killing is not the worst thing humans do to each other?

Does this mean something different than "Truth doesn't have a moral valence"?

My primary interpretation was "attaching yourself to arguments obstructs your ability to seek the truth." If you are interested in the truth, it does not matter if you or your interlocutor is wrong or right; it matters what the truth is.

Another interpretation is "is-thinking leads to accuracy, should-thinking leads to delusion."

A third interpretation is "moralistic thinking degrades morals." I don't consider that interpretation interesting enough to agree or disagree with it.

Maybe it's a reference to the idea that you need something more important than The Truth, so that you keep testing/refining your answer when you think you've got to the truth.

i'm going to reply to the quote as if it means "Truth doesn't have a moral valence" and rebuttal that truth should be held more sacred then morals rather then simply outside of it. For example if there are two cases and case 1 leads to a morally "better" (in quotes because the word better is really a black box) outcome then case 2 but case 1 leads to hiding the truth (including hiding from it yourself) then I would have to think very specifically about it. In short I abide by the rule "That which can be destroyed by the Truth should be" but am weary that this breaks down practically in many situations. So when presented with a scenario where i would be tempted to break this principle for the "greater good" or the "morally better case" I would think long and hard about whether it is a rationalization or that i did not expend the mental effort to come up with a better third alternative.

It's ... ahem ... non-canon. A different faction.

I thought it interesting that the near-inverse of a useful rationality quote can still be a useful rationality quote.

It's not from the TV series either.

The TV series quote would be this: "There is only one God. And his name is Death. And there is only thing we say to Death: 'Not today'."

Basically the grandparent post seems to be just a quote from memory, combining bits and pieces from both places, accurate to neither.

I could've sworn it was from both of them, and, thus, from the books originally...

One can make either true statements or false statements about reality.

One can do these two things, but not to the exclusion of alternatives. One can make statements which are confused or nonsensical, that are not even false.

In any case, a statement doesn't have inherent truth value outside the way it's interpreted by the people who hear it. The statement that "If a tree falls in the forest, it does not make a sound" is true or false depending on the meanings understood by the audience and the person uttering it. It's entirely possible to convey false understandings by making statements which omit relevant information. To refuse to call a statement which is deliberately tailored to make its audience believe falsehoods a lie is using a distinction in an unhelpful way.