Follow-up to: Every Cause Wants To Be A Cult, Cultish Countercultishness
One of the classic demonstrations of the Fundamental Attribution Error is the 'quiz study' of Ross, Amabile, and Steinmetz (1977). In the study, subjects were randomly assigned to either ask or answer questions in quiz show style, and were observed by other subjects who were asked to rate them for competence/knowledge. Even knowing that the assignments were random did not prevent the raters from rating the questioners higher than the answerers. Of course, when we rate individuals highly the affect heuristic comes into play, and if we're not careful that can lead to a super-happy death spiral of reverence. Students can revere teachers or science popularizers (even devotion to Richard Dawkins can get a bit extreme at his busy web forum) simply because the former only interact with the latter in domains where the students know less. This is certainly a problem with blogging, where the blogger chooses to post in domains of expertise.
Specifically, Eliezer's writing at Overcoming Bias has provided nice introductions to many standard concepts and arguments from philosophy, economics, and psychology: the philosophical compatibilist account of free will, utility functions, standard biases, and much more. These are great concepts, and many commenters report that they have been greatly influenced by their introductions to them at Overcoming Bias, but the psychological default will be to overrate the messenger. This danger is particularly great in light of his writing style, and when the fact that a point is already extant in the literature, and is either being relayed or reinvented, isn't noted. To address a few cases of the latter: Gary Drescher covered much of the content of Eliezer's Overcoming Bias posts (mostly very well), from timeless physics to Newcomb's problems to quantum mechanics, in a book back in May 2006, while Eliezer's irrealist meta-ethics would be very familiar to modern philosophers like Don Loeb or Josh Greene, and isn't so far from the 18th century philosopher David Hume.
If you're feeling a tendency to cultish hero-worship, reading such independent prior analyses is a noncultish way to diffuse it, and the history of science suggests that this procedure will be applicable to almost anyone you're tempted to revere. Wallace invented the idea of evolution through natural selection independently of Darwin, and Leibniz and Newton independently developed calculus. With respect to our other host, Hans Moravec came up with the probabilistic Simulation Argument long before Nick Bostrom became known for reinventing it (possibly with forgotten influence from reading the book, or its influence on interlocutors). When we post here we can make an effort to find and explicitly acknowledge such influences or independent discoveries, to recognize the contributions of Rational We, as well as Me.
Even if you resist revering the messenger, a well-written piece that purports to summarize a field can leave you ignorant of your ignorance. If you only read the National Review or The Nation you will pick up a lot of political knowledge, including knowledge about the other party/ideology, at least enough to score well on political science surveys. However, that very knowledge means that missing pieces favoring the other side can be more easily ignored: someone might not believe that the other side is made up of Evil Mutants with no reasons at all, and might be tempted to investigate, but ideological media can provide reasons that are plausible yet not so plausible as to be tempting to their audience. For a truth-seeker, beware of explanations of the speaker's opponents.
This sort of intentional slanting and misplaced trust is less common in more academic sources, but it does occur. For instance, top philosophers of science have been caught failing to beware of Stephen J. Gould, copying his citations and misrepresentations of work by Arthur Jensen without having read either the work in question or the more scrupulous treatments in the writings of Jensen's leading scientific opponents, the excellent James Flynn and Richard Nisbett. More often, space constraints mean that a work will spend more words and detail on the view being advanced (Near) than on those rejected (Far), and limited knowledge of the rejected views will lead to omissions. Without reading the major alternative views to those of the one who introduced you to a field in their own words or, even better, neutral textbooks, you will underrate opposing views.
What do LW contributors recommend as the best articulations of alternative views to OB/LW majorities or received wisdom, or neutral sources to put them in context? I'll offer David Chalmers' The Conscious Mind for reductionism, this article on theistic modal realism for the theistic (not Biblical) Problem of Evil, and David Cutler's Your Money or Your Life for the average (not marginal) value of medical spending. Across the board, the Stanford Encyclopedia of Philosophy is a great neutral resource for philosophical posts.
Ross, L. D., Amabile, T. M. & Steinmetz, J. L. (1977). Social roles, social control, and biases in social-perceptual processes. Journal of Personality and Social Psychology, 35, 485-494.
Comments sorted by top scores.
comment by gjm ·
2009-03-22T22:40:34.126Z · LW(p) · GW(p)
The article on theistic modal realism is ingenious. (One-sentence summary: God's options when creating should be thought of as ensembles of worlds, and most likely he'd create every world that's worth creating, so the mere fact that ours is far from optimal isn't strong evidence that it didn't arise by divine creation.)
I don't find the TMR hypothesis terribly plausible in itself -- my own intuitions about what a supremely good and powerful being would do don't match Kraay's -- but of course a proponent of TMR could always just reject my intuitions as I'd reject theirs.
However, I think the TMR hypothesis should be strongly rejected on empirical grounds.
It is notable -- and this is one element of a typical instance of the Argument From Evil -- that our world appears to be governed by a bunch of very strict laws, which it obeys with great precision in ways that make substantial divine intervention almost impossible. It seems that there are many many many more possible worlds in which this property fails than in which it holds, simply because the more scope there is for intervention the more ways there are for things to happen. Therefore, unless the sort of lawlikeness we observe is so extraordinarily valuable that tiny changes in it make a world far less likely to be worth creating, we should expect that "most" worlds in the TMR ensemble would be much less lawlike than ours: e.g., we might expect prayers to be commonly answered in clearly detectable ways. So how come we're in such an atypical world?
Generalizing: I think we should expect that for most measures of goodness X, worlds with higher values of X should be dramatically more numerous in the TMR ensemble unless increasing X reduces the number/measure of possible worlds much more drastically than for most other choices of X. (Because when you increase X, you get the chance to reduce Y or Z or ... a bit. More choices.) Therefore, we should expect that for measures of goodness X where "better" doesn't imply "much more constrained" most worlds (hence, in particular, ours, with high probability) should have values of X that are close to optimal, or at least far from marginally acceptable. This doesn't seem to be true.
It seems to me that counter-arguments to these are likely to be basically the same as counter-arguments to the original argument from evil.
The other thing about TMR is that it undermines any version of theism that expects God to behave as if he cares about us. If TMR is right then, any time God has the option of doing something to make your life better, then he forks the universe vastly many ways and tries out every possible option (including lots of ways of doing nothing, and even ways of deliberately making things worse for you) apart from ones that make the whole universe not worth while. As mentioned above, it seems to me that this should make us expect that visible divine intervention should be pretty common, but in any case it's not terribly inspiring. A bit like having a "friend" who, any time she interacts with you, rolls dice and chooses a random way of behaving subject only to the constraint that it doesn't cause the extinction of all human life. Similarly, you've got no reason to trust any alleged divine revelation unless its wrongness would be so awful as to make the world not worth creating. (These arguments are again closely parallel to ones that come up with the ordinary argument from evil, in response to responses that basically take the form of radical skepticism.)
Replies from: Alexei
↑ comment by Alexei ·
2010-08-06T22:02:04.090Z · LW(p) · GW(p)
Granted that god exists and cares about us and he can change the world, even in tiny aspects, it's very likely god will use those small aspects as a base to create the perfect world (kind of like AI FOOM). It follows that any world where god has some kind of minimum control will converge to the perfect world. Given that we are not in the perfect world, we can assume god does not have the minimum level of control.
comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) ·
2009-03-21T23:57:30.478Z · LW(p) · GW(p)
EDIT: I agree with your conclusion, but...
(Checks Don Loeb reference.)
While, unsurprisingly, we end up adding to the same normality, I would not say that these folks have the same metaethics I do. Certainly Greene's paper title "The Terrible, Horrible, No Good, Very Bad Truth About Morality" was enough to tell me that he probably didn't have exactly the same metaethics and interpretation I did. I would not feel at all comfortable describing myself as a "moral irrealist" on the basis of what I've seen so far.
Drescher one-boxes on Newcomb's Problem, but doesn't seem to have invented quite the same decision theory I have.
I don't think Nick ever claimed to have invented the Simulation Argument - he would probably be quite willing to credit Moravec.
On many other things, I have tried to use standard terminology where I actually agree with standard theories, and provide a reference or two. Where I am knowingly being just a messenger, I do usually try to convey that. But you may be reading too much into certain similarities that also have important points of difference or further development.
EDIT2: I occasionally notice the problem you point to, and write a blog post telling people to read more textbooks. Probably this is not enough. I'll try to reach a higher standard in any canonicalized versions.
Replies from: thomblake, steven0461, CarlShulman, Roko
↑ comment by thomblake ·
2009-04-02T14:12:49.633Z · LW(p) · GW(p)
I think the biggest issue here is your tendency to not cite sources other than yourself, which is an immediate turn-off to academics. To an academic, it suggests the following questions (amongst others): If your ideas are so good, why hasn't anyone else thought of them? Doesn't anyone else have an opinion on this - do you have a response to their arguments? Are you actually doing work in your field without having read enough to cite those who agree or disagree with you?
(I know this isn't a new issue, but it seems it bears repeating.)
Replies from: wedrifid, TheAncientGeek
↑ comment by wedrifid ·
2010-08-09T02:36:39.569Z · LW(p) · GW(p)
Other questions that are implicitly asked:
- Why are you not signalling in group status?
- Why are you not signalling alliance with me or my allies by inventing excuses to refer to us?
- Are you an outsider trying to claim our territory in cognitive space?
- Are you talking about topics that are reserved for those with higher status in our group than we assign you?
↑ comment by TheAncientGeek ·
2014-05-02T15:51:03.308Z · LW(p) · GW(p)
This point could count against any amateur philosopher.
What is more pertinent: why insist you are doing better than the professionals? You should assume you are making ,mistakes and reinvemtimg wheels.
Why not learn the standard jargon? You may not have the time or inclination to learn the whole subject, but the jargon is the most .valuable thing to learn, because it enables you to communicate with professinals who can help you. If you are able to admit to yourself that, as an amateur, you might need help.
There are some failure modes that arepart and parcel of being an amateur, and some further ones that take you into crank territory.
↑ comment by steven0461 ·
2009-03-22T01:06:48.571Z · LW(p) · GW(p)
I took a look at Greene's dissertation when Roko started pushing it, but I don't think Greene's views are much like Eliezer's at all. Specifically he doesn't seem to emphasize what Eliezer calls the "subjectively objective" quality of morality, or the fact that people may be mistaken as to what their morality says. Correct me if I'm wrong.
I agree with the rest of the original post.
Replies from: CarlShulman
↑ comment by CarlShulman ·
2009-03-22T01:41:43.470Z · LW(p) · GW(p)
I agree about the difference of emphasis but I don't think they have a major substantive disagreement on those issues. You can check with Owain Evans, who knows him.
↑ comment by CarlShulman ·
2009-03-22T00:04:28.002Z · LW(p) · GW(p)
Greene doesn't really think it's horrible, just that people mistakenly think it's horrible and recoil from irrealism about XML 'rightness tags' on actions because they think it would mean that they should start robbing and murdering. Nick does acknowledge Moravec on his website now, after being informed about it (he wasn't aware before that).
Perhaps I shouldn't have covered both being a messenger and acknowledgment of related independent work in the same post.
Replies from: Roko
↑ comment by Roko ·
2009-03-22T00:29:53.381Z · LW(p) · GW(p)
Greene doesn't really think it's horrible, just that people mistakenly think it's horrible and recoil from irrealism
yes, I detected a hint of irony in the title. The thesis is that it isn't actually that horrible, rather that people don't want to face up to the truth, and it is because of this somewhat irrational fear that even considering the possibility of antirealism is avoided.
↑ comment by Roko ·
2009-03-22T00:26:04.424Z · LW(p) · GW(p)
Certainly Greene's paper title "The Terrible, Horrible, No Good, Very Bad Truth About Morality" was enough to tell me that he probably didn't have exactly the same metaethics and interpretation I did.
Have you read it? It takes about a day and a half to read, and I think that he points out an error with the position that you took in the "p-right" etc discussions on OB. Would it be off topic for me to do a post on this?
Other than that, he takes the same position you do. I recommend that you read his dissertation, and then email him to discuss the application of this set of ideas to transhumanism/singularity. He would probably be interested.
Replies from: Eliezer_Yudkowsky
↑ comment by Eliezer Yudkowsky (Eliezer_Yudkowsky) ·
2009-03-22T00:37:36.719Z · LW(p) · GW(p)
In this case, I'd actually say email me first with a quickie description.
Replies from: CarlShulman
↑ comment by CarlShulman ·
2009-03-22T00:46:11.553Z · LW(p) · GW(p)
Roko exaggerates. It's only 377 pages and written in an accessible style.
It summarizes the ethical literature on moral realism, and takes the irrealist view that XML tags on actions don't exist, and that even if they did exist we wouldn't care about them. It then goes into the psychology literature (Greene does experimental philosophy, e.g. finding that people misinterpret utility as having diminishing marginal utility in contravention to experimental instructions), e.g. Haidt's work on social intuitionism, to explain why it is that we think there are these moral properties 'out there' when there aren't any. Lastly, he argues that we can get on with pursuing our concerns (reasoning about conflicts between our concerns, implications, instrumental questions, etc), but suggests that awareness of the absence of XML tags can help us to better understand and deal with those with differing moral views.
Replies from: Eliezer_Yudkowsky, PhilGoetz
↑ comment by PhilGoetz ·
2009-03-22T01:17:47.352Z · LW(p) · GW(p)
XML tags on actions don't exist, and that even if they did exist we wouldn't care about them.
Replies from: Vladimir_Nesov
↑ comment by Vladimir_Nesov ·
2009-03-22T01:26:52.375Z · LW(p) · GW(p)
There is no objective truth about which actions are the right ones, no valuation inherent in the actions themselves. And even if there was, even if you could build a right-a-meter and check which actions are good, you won't care about what it says, since it's still you that draws the judgment.