Great Product. Lousy Marketing.

To have this attitude, you need a strong presumption of your own superiority. Instead of engaging them in a conversation where you can both better discover the truth, with you also remaining open to info they may offer, you decide this is war and "all is fair in love and war." You know what is true and what is good for the world and you've decided it is important enough to the world for them to believe your truth that you will force it upon them in any way you can. No doubt this is a possible situation, and there is possible evidence that could reasonably convince you that this is your situation. But do pause and consider whether your confidence might be overconfidence, biased by arrogance, and also consider the consequences of their hearing that this is in fact your attitude toward them.

About the use of dark arts, and the magic paragraph... Maybe there's an important difference between:

• turning someone into an extreme rationalist who would see they had been "hacked" and be grateful for it.
• giving someone correct opinions but otherwise have them remain irrational and ignorant of having been "hacked".

Let's imagine for a moment, the most convincing paragraph ever written. It was truly a world-wonder of persuasion - it converted fundamentalist Christians into atheists, suicide bombers into diplomats, and Ann Coulter-4-President supporters into Less Wrong sycophants.

Would it produce people who could explain the differences between Pascal's Wager and the expected utility argument for cryonics and why they should produce different answers? Or who could accept Many Worlds without thinking that they make all available decisions "equally"? Or who could reject the theory that physical processes affect epiphenomenal consciousness but not the other way around on the basis that people physically report their conscious experience? Will they be able to systematically get similarly difficult questions right?

Or would it produce atheists that can believe in homeopathy or astrology?

Someone believes that the Singularity is a sack of absurdities and is sad about the fact that we're all deluded into believing it. To help pry us loose from our little cult, they post an argument that's really persuasive on a gut level: A photoshopped picture of me killing Meredith Kercher. (Shortly after, I am arrested by the Italian police.)

Is there a moral difference between that and what you're proposing besides, "Oh, but they're wrong and we're right?" If so, what do you think it is?

We actually label persuasive strategies that can be used to market our true ideas as "dark arts".

The linked page specifies that the "dark arts" specifically take advantage of biases for persuasion. So it's a bit misleading to say "We actually label persuasive strategies that can be used…", because we do not label all strategies as such. Our goal should be to snap people out of their biases, so that they can naturally accept anything that turns out to be true. That could be taken as a "persuasive strategy", but it is not a dark one.

I'd favour attempts to develop bias-busting techniques intended to be used on general audiences. (Have there been any discussions about this — developing, as it were, a Defense Against the Dark Arts curriculum?) I would oppose attempts to evangelize our conclusions to general audiences without imparting to them the underlying framework of rationality that would allow them to independently discover or at least verify these conclusions. Using sophistry to persuade someone of a truth isn't much better than using the same tricks to persuade them of a falsehood, and even if we ignore moral issues, it is nearly useless, because if a person ends up with better ideas but all the same biases, their heads can later just as easily be filled with whole new sets of bad ideas by other Dark Arts practitioners.

Edit: I'd like to add that I don't mean that I believe biases can be countered with Pure Reason — which, I suppose, is what makes them bisaes in the first place. As the saying goes, we can't reason people of what they were never reasoned into. Debiasing techniques will not consist solely of rational arguments based on evidence, because the goal is to get people to the point where they can accept such things in the first place. But that does not mean that we have to, or ought to, resort to actively bad reasoning. (Especially to someone with an anti-reason worldview — serious anti-epistemology — invalid logical arguments don't work any better than valid ones anyway.) That, I think, is something that the prominent members of the "New Atheist" movement (it's not all that new, I know) are getting right (for the most part). This movement is unapologetic, it's emotional, it can be abrasive and offensive at times, but it's not dishonest. As one example of that, see, for instance, PZ Myers strongly criticizing a recent study which, in part, appears to show a positive correlation between IQ and atheism. He didn't have to criticize it. He could have been among those "patting [them]selves on the back". But he sided with intellectual honesty over embracing a potentially flawed argumentative tool. If we want to spread rationality, we should be thinking along the same lines.

"shut up and get rich" is the only strategy I see working for rationalists TBH.

The post and comments -- all very interesting so far -- tend to assume that a rationalist's goal in conversation with someone else is always a first-order one, where success and failure are represented only by whether the interlocutor changes his/her mind with respect to rationality or truth-seeking. There might be a limited category of cases where this assumption isn't good.

For example, if there were a young-earth creationist who was also a fourth-grade teacher, and who was in the habit of subtly undermining scientific truth with talk of "controversies" and the like, I think we could assume that he was erecting serious roadblocks to his students' future progress as rationalists. (I think I need not spell out the ways in which that might be true.) Now suppose I were able to persuade him that something like evolution by natural selection took place, even if I did so by some fundamentally dishonest means (by citing certain Bible verses, say), and in a way that led to no increase in his overall ability to think rationally about anything. If the result were that he kept his damn mouth shut about bogus scientific controversies in the presence of impressionable children, thereby not impeding their future progress as rationalists, the broad cause of rationality might well have been helped along.

Generalizing a bit, it seems rarely preferable to allow someone to continue believing something false that they have no justification for, instead of convincing them to believe something true that they have no justification for, in cases where -- as the OP seems to suggest -- those are the only two realistic options. To say we must never consider using some "dark arts" in the service of rationality at a level or two of remove seems somehow ... deontologist.

I guess I'm for persuasion, think the ends justify in this case. Otherwise you're all bringing knives to a gunfight and crying foul when you get shot down. Could there be a degree of "sour grapes" here resulting from finding one's self inexplicably unpersuasive next to a velvet tongued dummy? Are we sure we eschew the tactics of rhetoric because they're wrong? Is it even fair to describe "dark arts" as wrong?

I say rather that speech is meant to be persuasive. Better speech would then be more persuasive. As such persuasion backed by truth should be sturdier than mere naked truth, which is only palatable to connoisseurs who like their facts without sauce. Have I mixed enough metaphors for you there?

I think thought, hence communication, is highly model based, so you can be compelling with a metaphor that "fits" and one that's well chosen would fit very well indeed. Then the topic under discussion gets a piggyback ride from the metaphor and the stipulated conclusion from the model is transferred to the new topic. (there must be an established name for that sort of trick?) But what if it's in defense of the truth? I say it's ok. To go back to the first metaphor: you gotta fight fire with fire; facts burn too easily.

The very definition of the "dark arts" is those arts of persuasion that don't particularly correlate well to truth. So if such a super-persuasive paragraph existed, there's no reason to expect it to be one that favoured our beliefs over those of Ann Coulter. If we abandon those persuasive techniques that favour truth, we throw away the one advantage we believe we have over the many others that wish to persuade.

Also, don't throw around boo lights like "almost religiously entrenched" without evidence.

I use self-promotion techniques that I know are more flair and feeling than rational points to convince people of how good I am at certain things; I then follow through by showing that I am, in fact, as good or better than my marketing material showed me to be. This combination works very well, and I do not feel deceptive for properly calibrating their expectations before I am able to provide true evidence.

I suppose I'm more concerned with how well they're calibrated (beliefs) than with their methods of calibration (rationality); the latter provides far more benefits, but is also far more difficult to change.

This post seems predicated on the notion that we've established a gospel to preach, and we're just not preaching it. What's the gospel? "Rationality"? "Truth"? The Sequences? I don't buy it. There's no finished product here to sell.

I'm reminded of conversations I've had with people who express deep frustration with Obama's inability to cajole Congress into moving quickly on health care reform, but don't have a good answer to the question, "Why do you think Congress will come up with a workable solution to the problem?"

If you persuade people with bad reasons-- and if people are persuaded more by eloquence than they would have been by the substance of the argument, they are persuaded for bad reasons -- in the end they will be worse off than before with respect to truth, even if they change their minds about some particular errors.

You shouldn't mistake the convention that people aren't supposed to use the Dark Arts here on LessWrong, to mean that the LW community rejects the Dark Arts.

I've spoken with dozens of people who are involved with SIAI or read/post on LessWrong. I would not characterize them as being slow to embrace the Dark Arts when compared with American intellectuals in general.

I like the metaphor of the peacenik wanting to rid the world of violence by suggesting that police not use weapons. Let's elaborate on the analogy between Dark Arts and violence.

Tit For Tat is a common policy for trying to control violence. One obvious and much lamented flaw in the strategy is that it tends to foster cycles of violence, with each side claiming that the other side started it and that "they" use more vicious tactics.

To get past the problems of biased measurement of proportional response and so on, and thereby break the cycle of violence, at least one side has to undertake to dramatically de-escalate - while still providing a deterrent to unrestricted use of force. The last point is essential. Absolute pacifism may well work brilliantly as a response to some provocateurs, but not all.

I like to summarize my views on violence by saying that the only thing worse than two armies fighting a war is one army fighting a war. If rationalists foreswear the Dark Arts, the result will be one army fighting a war. And we'll be the targeted civilians. Well, not quite. Reason is roughly comparable to small arms; the Dark Arts are heavy artillery and airpower. Or so I fear. If anyone has links to research on the relative persuasive powers of reason and unreason, it would certainly help clarify the issue.

It's tough to market existential risk because you end up sounding like a crazy doomsday prophet if the sell is too hard. Most people don't like to think about it and don't like being reminded that the world could suddenly end, unless it's a religion reminding them.

The desire to persuade people isn't necessarily rational, especially when it comes to "enlightening" people on the superiority of rational thought. I think a truly rational person's allegiance should always rest in truth. Truth, in it self, is a very powerful notion that doesn't need the help of manipulative persuasive tactics to inspire people.

I think persuasive techniques can be adapted to help discover the truth, as long as the parties involved completely respect each other and are willing to ask questions that help the other better articulate their perspective--they have to ultimately be on the same side and trust that there is rational thought behind each other's statements, even if it's not apparent at first.

I like the metaphor of the peacenik wanting to rid the world of violence by suggesting that police not use weapons. Let's elaborate on the analogy between Dark Arts and violence.

Tit For Tat is a common policy for trying to control violence. One obvious and much lamented flaw in the strategy is that it tends to foster cycles of violence, with each side claiming that the other side started it and that "they" use more vicious tactics.

To get past the problems of biased measurement of proportional response and so on, and thereby break the cycle of violence, at least one side has to undertake to dramatically de-escalate - while still providing a deterrent to unrestricted use of force. The last point is essential. Absolute pacifism may well work brilliantly as a response to some provocateurs, but not all.

I like to summarize my views on violence by saying that the only thing worse than two armies fighting a war is one army fighting a war. If rationalists foreswear the Dark Arts, the result will be one army fighting a war. And we'll be the targeted civilians. Well, not quite. Reason is roughly comparable to small arms; the Dark Arts are heavy artillery and airpower. Or so I fear. If anyone has links to research on the relative persuasive powers of reason and unreason, it would certainly help clarify the issue.

This may have some connection to the akrasia discussions-- what are non-destructive methods (or even beneficial methods) for getting from thought to action? I'm assuming that thinking well is as much an action as physical movement-- there's some difference since a lot of people have preferences and talent for one or the other-- but subjectively, I find there's an overlap.

Is there a qualitative difference between the methods that work to change your own behavior and methods to change other people's?