Posts

Comments

Comment by Demosthenes on You don't need Kant · 2009-04-02T20:35:58.407Z · LW · GW

There's a proverb I failed to Google, which runs something like, "Once someone is known to be a liar, you might as well listen to the whistling of the wind." You wouldn't want others to expect you to lie, if you have something important to say to them; and this issue cannot be wholly decoupled from the issue of whether you actually tell the truth. If you'll lie when the fate of the world is at stake, and others can guess that fact about you, then, at the moment when the fate of the world is at stake, that's the moment when your words become the whistling of the wind.

-from Eliezer's quoted article Here

I don't know if you read the entire body of my comment bringing up Kant, but it rests on asking if there was a similarity in Eliezer's argument and Kants with a question mark at the end.

Both Eliezer and Kant seem to think that this abstract thing called "trust" suffers when individuals choose to lie for their own purposes. Both of them suggest that individuals who believe this would benefit from adopting a maxim that they should not lie.

Eliezer states in the comments that you can lie to people who aren't part of your community of rational or potentially rational individuals.

Kant says that you can't lie to people, even if they aren't part of your club.

You don't need the CI to reach either of these conclusions; the comment points out that you could do this on Utilitarian grounds. Utilitarian reasoning might even support Kants "don't like to anyone ever" over Eliezer's conceptions.

As for arguing Kant leading to a dead end, there is plenty of contemporary philosophy that still uses a lot of Kant and even NPOV Wikipedia has a section detailing Kant in contemporary philosophy.

Also, I think most people on here agree that Kant was wrong. In more ways than one. Thus debating Kant is pretty much a dead end.

I am always of the mind that saying that someone's assumptions are wrong doesn't lead to their argument having no value ever for any future discussion. In this particular case we got to use a Kantian thought experiment to talk about what looks like a variation on Kantian logic. I'm sorry I used the K word.

The idea of everyone on LW believing that Kant was almost totally wrong and that we should completely discard him is a little unsettling to me. There is a much larger community out there that accepts elements of Kant's arguments and methods and still applies them; I would again push a Robin Hanson line by suggesting that most rationalists are elsewhere and we should work harder to find them.

Comment by Demosthenes on You don't need Kant · 2009-04-02T16:52:21.602Z · LW · GW

In the Robin Hanson tradition, whenever I think that I have figured out a flaw in Kant's reasoning, I halt, recognize that he lived until he was 79 and spent everyday of his life thinking about these sorts of things and taking long walks. It is good to question him, but also to be humble and research any extant rebuttals to one's own argument.

There is a good overview of Kant here: http://www.trinity.edu/cbrown/intro/Kant_ethics.html

and more at the Stanford Encyclopedia of Philosophy.

Kant had a peculiar obsession with what rational and reasoning actors would choose to do and what would happen if all people say rationality and reason as definitive tools. Why there is so much resistance to delving into Kant in the Less Wrong community is beyond me.

Comment by Demosthenes on You don't need Kant · 2009-04-02T03:10:31.962Z · LW · GW

I was just a little put off that you used me as an example of pulling Kant in when he doesn't apply: I took some care to keep Kant within Kant's domain and ask for specifics about how EY's OB position differed.

Most of your post is dedicated to refuting Kant's assumptions... that would have answer part of my questions in the other post ... but does it necessarily follow that he is pulled in to make one's opponents into straw men?

The discussion could have used some Kant and I am really do not agree that he does not apply.

In regard to why one shouldn't bring him up, you seem to suggest that Plato really was right after all:

“Shall we, then, thus lightly suffer our children to listen to any chance stories fashioned by any chance teachers and so to take into their minds opinions for the most part contrary to those that we shall think it desirable for them to hold when they are grown up?” “By no manner of means will we allow it.” “We must begin, then, it seems, by a censorship [377c] over our storymakers, and what they do well we must pass and what not, reject. And the stories on the accepted list we will induce nurses and mothers to tell to the children and so shape their souls by these stories far rather than their bodies by their hands. But most of the stories they now tell we must reject.”

Who are we protecting at LW? I think everyone here can tell the difference between EY, The Black Belt Bayesian and Kant on most issues, but from time to time I like a little clarification.

Comment by Demosthenes on You don't need Kant · 2009-04-01T23:49:23.180Z · LW · GW

Did anyone say that they believed in Kant?

Actual comment thread (with context intact!): (http://lesswrong.com/lw/6w/degrees_of_radical_honesty/4jn?context=1#4jn)

We were talking about never lying; I copied a quotation from Constant's critique of Kant (they were explicitly discussing a version of the "Tell-the-Murderer" thought experiment) and then summarized Kant's negative response to Constant.

I'm not really sure why one wouldn't bring it up? We had two different conceptions of why you shouldn't lie in the main post. Eliezer's sounded a lot like Kant's, but then he said that you don't have to include everyone in the group of people you would never lie to. Kant specifically addresses this argument.

Next step....

Bring up Kant.

I would probably just add a comment that you disagree with Kant's assumptions to the chain and go on to state that this makes his argument of no use to us in the modern day and age.

I threw in the point that you could reconstruct his arguments as utilitarian critiques in the hope that someone might not just discount Kant and be done with it...c'est la vie.

For interested partires, there's plenty more out there (http://plato.stanford.edu/entries/lying-definition/), but Kant is still relevant.

Comment by Demosthenes on The Benefits of Rationality? · 2009-04-01T23:28:09.159Z · LW · GW

Mysticism and random decision making are both acceptable and highly successful methods of making decisions; most of human history has relied on those two... we still rely on them. If you are a consequentialist, you can ignore the process and just rate the outcome; who cares why nice hair is correlated with success -it just is! Why does democracy work?

What makes rationalism worth the time is probably your regard for the process itself or for its outcomes. If its the outcomes then you might want to consider other options; following your biases and desires almost blindly works out pretty well for most people.

Comment by Demosthenes on The Benefits of Rationality? · 2009-04-01T13:19:12.953Z · LW · GW

logi:

That's possible and probably partially accurate; if there were more posts taking the form "I believe X because..." on Less Wrong, I might be more open to the idea that people are doing that.

Ciphergoth:

Also, this would be a terrible community to signal truth-seeking in, considering how entrenched the "rationality as win" metaphor is. As I mentioned in the hair example, I think a lot more people here are signaling a burning interest in real-world application than really have one.

I just wanted to get Yvain's opinion about how much value from posting on Less Wrong was coming from signaling. Yvain suggested that this was not his or her main goal and that LW would be a uniquely poor place to attempt it. I personally doubt both of those points, but I was hoping to get some clarification since the comments about signaling and the nature of truth-seeking don't seem to be part of a system of beliefs.

Are you worried that signaling truth-seeking is legitimate enough?

Comment by Demosthenes on Degrees of Radical Honesty · 2009-04-01T03:58:40.807Z · LW · GW

"To tell the truth is a duty, but is a duty only with regard to the man who has a right to the truth."

Kant disagrees and seems to warn that the principle of truth telling is universal; you can't go around deciding who has a right to truth and who does not. Furthermore, he suggests that your lie could have terrible unforeseen consequences.

Lie to the Nazis who you feel "don't deserve the truth" and then they end up treating everyone on the rest of the block like liars and sending all sorts of people to the concentration camps or outright killing them because its not worth trying to ferret out truth etc..etc..etc...

Eliezer:

When I was reading through your other article I thought the "fate of the world" part suggested that not lying should be the basis for a universalizable duty like Kant's. The existence of a future "Fate of the world" event makes it seem like you are getting at the same unforeseen consequences point as The Big K -is this accurate?

I am concerned that deciding on who is rational enough to treat honestly is a slippery slope.

Personally, this seems like a point where you take a metaphysical stand, rationally work your way through the options within your axiomatic system and then apply your rational program to the choice at hand. I am more utilitarian than Kant, but it is not hard to ignore "proximity" and come up with a cost/benefit calculation that agrees with him.

Comment by Demosthenes on Degrees of Radical Honesty · 2009-04-01T00:24:29.967Z · LW · GW

If you'll lie when the fate of the world is at stake, and others can guess that fact about you, then, at the moment when the fate of the world is at stake, that's the moment when your words become the whistling of the wind.

Is it correct to interpret this as similar to Pascal's Wager? The possibility of a fate-of-the-world moment is very low but the payout for being an honest fellow in this case is huge?

Comment by Demosthenes on Most Rationalists Are Elsewhere · 2009-03-31T22:45:08.514Z · LW · GW

I would like to see more people who practice rationality and assumption questioning in other disciplices: women's studies, public policy, art and literature. I took a lot of literary philosophy classes back in the day and read quite a few post-modern critiques that mirror what I see on Less Wrong.

Almost every post-modern analysis depends on questioning how someone framed their subject and proceeds to recommend different assumptions; surely people with these backgrounds have examples to offer outside of game theory and psychology.

It would also be good to see some legal types. Lawyers competing in front of Judges who then make decisions that affect people's lives must certainly have put a little thought toward the roles of rationality and persuasion in truth seeking. Even if you don't care for lawyers, you have to wonder how judges proceed.

Maybe we should invade other forums and lead the discussions back here?

EDIT ( In regard to that OB post on female perspectives, its interesting that Robin Hanson of all people wasn't more humble about his potential lack of knowledge in a new field when his post got a poor response! Goes to show how important other perspectives are to this project)

Comment by Demosthenes on The Benefits of Rationality? · 2009-03-31T22:38:14.158Z · LW · GW

And you must enjoy the signal value you a little bit! You aren't keeping your Less Wrong postings in your diary under lock and key!

Comment by Demosthenes on The Benefits of Rationality? · 2009-03-31T22:35:00.187Z · LW · GW

Yvain:

Do you really believe that you engage in Truth-Seeking for utilitarian reasons? I get the impression that you don't really believe that.

Would you be willing to to enter a computer simulation where you got to investigate higher math puzzles (or metaphysics) with no applications? Spend your days in a fantastic and never-ending Truth-Seeking project (we'll throw great sex, food and housing into the holodeck for you as well)?

I liked this better at the beginning when you were prodding people who say that they see rationalism as a means to an end! You seem to be going back to consequentialism!

I don't believe that rationalists WIN because I don't believe that winning WINS

Comment by Demosthenes on The Benefits of Rationality? · 2009-03-31T16:24:21.357Z · LW · GW

This debate has already played out in attacking and defending Pragmatism.

A lot of the rubrics by which to judge whether or not rationalism wins or whether or not rationalism is an end in itself involve assigning meaning and value on a very abstract level. Eliezer's posts outline a reductionist, materialist standpoint with some strong beliefs about following the links of causality. Rationalism follows, but rationalism isn't going to prove itself true.

Deciding that rationalism is the best answer for your axiomatic belief system requires taking a metaphysical stand; I think that if you are looking for a definite metaphysical reason that you should practice rationalism, then you are interested in something that the practice of rationalism is not going to help much.

Comment by Demosthenes on Framing Effects in Anthropology · 2009-03-30T00:48:54.903Z · LW · GW

The Nacirema are actually just gaseous meat sacks:

http://www.youtube.com/watch?v=gaFZTAOb7IE

Comment by Demosthenes on Church vs. Taskforce · 2009-03-29T18:48:17.105Z · LW · GW

That was my point; I was making a dig on the goals of argumentative atheists looking for a support group vs people who might want to advance rationalist goals

Comment by Demosthenes on Rationality: Common Interest of Many Causes · 2009-03-29T18:30:56.446Z · LW · GW

In a nutshell, it might be cool to make a website and organization that promotes data collections and debate.

Rationalism requires access to high quality empirical evidence. Holding your hypotheses up to constantly changing data is a major theme of this site.

We can only rationally discuss our hypotheses and beliefs when we have something to test and the quality of datasets floating around on the internet is often low or inaccessible.

A good rationalist project might be to highlight resources for empirical evidence, run "data debates" where experts attack and defend each others datasets; a wiki for best-practices in data collection, or a wiki for navigating popular issues through good datasets (try as a nonexpert to find what studies on taxation and inequality are best and you can end up running in circles).

I think you would want to tailor this kind of project toward non experts, giving people (especially journalists) a good starting place for finding meaningful, well-collected data that can form a good jump point for rational analysis.

A project like this also leaves the door open to many interpretations and many goals, so it isn't necessarily cutting down on the number of voices out there.

I would also be interested in cataloging failed attempts. More and more I have been trying to look at survivorship biases (http://en.wikipedia.org/wiki/Survivorship_bias) behind all my beliefs.

Are there any good examples of projects like this in existence? Maybe we can leverage the community here to throw our weight behind one.

Comment by Demosthenes on Church vs. Taskforce · 2009-03-29T04:04:29.584Z · LW · GW

Yvain is spot on; secular service organization already exist and function. I have occasionally attended some meetings at a Rotary club and it usually involves eating, a list of ongoing activities, community highlights and recognition of visiting members.

What is special about the way a rationalist helps people? Maybe starting a program to fund probability and philosophy of science classes in the community?

Law school sounds like the best option for finding fellow argumentative atheists.

Comment by Demosthenes on Defense Against The Dark Arts: Case Study #1 · 2009-03-29T03:35:12.014Z · LW · GW

In his youth, Steve Jobs went to India to be enlightened. After seeing that the nation claiming to be the source of this great spiritual knowledge was full of hunger, ignorance, squalor, poverty, prejudice, and disease, he came back and said that the East should look to the West for enlightenment.

....or maybe the quotation and by extension the entire comment were meant to suggest that traditionally materialist concerns like sanitation, wealth and longevity are more deserving of the title enlightenment and than our categorizing of enlightenment to only mean the spirit is not entirely accurate. Expressing wonder at reductionist, material understanding of the universe shouldn't be new to this crowd. Expressing value judgements do not a dark art make.

...or maybe it meant to ignore all Indian claims to enlightenment....

There is a lot of nonsense on OB and LW about separating content from style; the occasional attempts to translate into positivist verifiable claims or examples of Dark Arts often say more about the person doing the translating than illuminating the text for the reader.

Yvain obviously interpreted this in a very specific way. Yvain has a good basis for asking Phil to clarify the issues. These sorts of things are more valuable as discussions and instead it was turned into a broadcast.

This is not a criticism, but just a suggestion that the world of give-and-take, persuasion and rebuttal can be a lot more valuable than posting an instantiation of meaning for the comment that is highly suspect at best.

Comment by Demosthenes on Bogus Pipeline, Bona Fide Pipeline · 2009-03-24T15:09:13.145Z · LW · GW

If anyone wants to do some background reading on the test before commenting, this paper addresses some of the common criticisms:

http://www.psy.utexas.edu/psy/FACULTY/Markman/jpsp01.pdf

"How do indirect measures of evaluation work? Evaluating the inference of prejudice in the IAT"

Comment by Demosthenes on Playing Video Games In Shuffle Mode · 2009-03-23T22:20:25.857Z · LW · GW

Yes! something like a table of contents?

The Tag Cloud is a good way to start, but once you generate 10 posts a day for too long, the tag cloud is no longer a useful navigation tool

Something like this maybe: http://drupal.org/project/hypergraph

Drupal can also automatically generate "related content" based on whatever criteria you define as important or manually entered links. Adding more and more blocks to the page might not be good for efficiency, but providing more diverse paths to explore the content on these sites would be great.

In the long run, the more crosslinking there is, the easier it will be to visualize the stronger nodes and the easier it will become to find highly cited posts. At this point, good posts get even more citation. Good navigation is the critical first step.

I spend a lot of time these days fishing through older posts on Overcoming Bias, looking for something to read, but it is definitely not set up as a repository of knowledge.

Comment by Demosthenes on Cached Selves · 2009-03-23T19:58:15.962Z · LW · GW

Has anyone brought up this study by Bruner and Potter (1964) before? I think it would relate to intertemporal beliefs and how we sometimes perceive them to be more sound than they really are:

http://www.ahs.uwaterloo.ca/~kin356/bpdemo.htm

In this demonstration, you will see nine different pictures. The pictures will get clearer and clearer. Make a guess as to what is being shown for each of the pictures, and write down your guess. Note the number of the picture where you were first able to recognize what was being shown. Then go backwards - press the "BACK" button on the browser - and see at which point you can no longer identify the picture. Are your "ascending" and "decending" points the same?

========

IF YOU HAVE TRIED THE STUDY:

Pictures of common objects, coming slowly into focus, were viewed by adult observers. Recognition was delayed when subjects first viewed the pictures out of focus. The greater or more prolonged the initial blur, the slower the eventual recognition. Interference may be accounted for partly by the difficulty of rejecting incorrect hypotheses based on substandard cues.

It would be interesting to think of your intertemporal frame of mind as discontinuous and running at 24 frames per second (like a film). Maybe your consciousness gives your sense of beliefs a false sense of flowing like a movie from one time state to the next.

Comment by Demosthenes on Cached Selves · 2009-03-23T16:35:29.907Z · LW · GW

It may be time for a good Style vs Content Debate; first commenter to scream false dilemma gets a prize

Comment by Demosthenes on When Truth Isn't Enough · 2009-03-22T23:15:04.522Z · LW · GW

Speaking of themes; I guess one thing that bothered me about this post (many of your other posts are very good) is that this post doesn't seem to serve a point; questioning assumptions is often brought up on OB and LW and asking others to be more precise in describing their assumptions is also very common. Any connection to positivism here seems very tenuous; criticizing positivism has little or no impact on soft-positivism.

I feel that there are many other OB and LW posts that would address this issue more effectively and it might be better to just make this page an index of them as opposed to a full post.

That said, there is no way to easily recognize a lot of the themes here and LW in particular runs the risk of just becoming a repository of the same things over and over again.

Are we really refining the art of human rationality here?

Comment by Demosthenes on Cached Selves · 2009-03-22T22:57:23.903Z · LW · GW

This is the first post I've seen that seems to really fit Less Wrong's mission of "refining the art of human rationality"

This post clearly spells out some issues, links them to research and presents possible solutions. I hope that more posts in the future take this form.

This post also nicely outlines the problem of one's ability to really doubt themselves constantly at the appropriate level. I think these two points present a big challenge to the mission of leading a rational life:

3c. Reframe your past behavior as having occurred in a different context, and as not bearing on today’s decisions. Or add context cues to trick your brain into regarding today's decision as belonging to a different category than past decisions. This is, for example, part of how conversion experiences can help people change their behavior. (For a cheap hack, try traveling.)

3d. More specifically, visualize your life as something you just inherited from someone else; ignore sunk words as you would aspire to ignore sunk costs.

How someone could do enough compartmentalizing of their identity to pull of either of these tasks escapes me.

Comment by Demosthenes on When Truth Isn't Enough · 2009-03-22T22:19:18.793Z · LW · GW

This post just appeared at Language log and has this to say:

Language Log could devote a thousand posts to the project of underlining and elaborating the ways in which grammar does not protect us against misunderstanding the sound of an uttered name, and logic does not protect us against what we say having double meaning. Come to think of it, the thousand posts may already have been written: there are over 5,500 old posts searchable here and already over 1,200 new ones on the present server searchable here.

http://languagelog.ldc.upenn.edu/nll/?p=1258

Asking others to clarify their assumptions is good, but translating them into positivist language might not be as precise as it sounds

Comment by Demosthenes on Why Our Kind Can't Cooperate · 2009-03-21T04:17:23.881Z · LW · GW

I think your caution is warranted, the fact that you can see the other people in the synagogue who don't stand up could be very hurtful to the nonparticipants. Highlighting individual donors or small groups is a good way to show public support without giving away to much information about your membership's participation as a whole.

If you are interested in more rigorous studies (we did ours in excel), you might want to try Dean Karlan's "Does Price Matter in Charitable Giving? Evidence from a Large-Scale Natural Field Experiment " http://karlan.yale.edu/p/MatchingGrant.pdf

I will try to dig up some other papers online

Comment by Demosthenes on Rationalist Poetry Fans, Unite! · 2009-03-20T18:10:37.621Z · LW · GW

Putting a double space after each line should break it; I had the same problem with poetry.

test
lines
here

Comment by Demosthenes on Why Our Kind Can't Cooperate · 2009-03-20T16:22:36.410Z · LW · GW

I've worked for a number of non profits and in analysis of our direct mailings, we would get a better response from a mailing that included one of two things

  1. A single testimonial mentioning the amount that some person gave
  2. Some sort of comment about the group average (listeners are making pledges of $150 this season)

This is one of the reasons that some types of nonprofits choose to create levels of giving; my guess is that it is gaming these common level of giving ideas by creating artificial norms of participation. Note You can base your levels on actual evidence and not just round numbers! (plus inflation, right?)

We also generally found that people respond well to the idea of a matching donation (which is rational since your gift is now worth more).

I do believe that anonymous fund raising removes information about community participation that is very valuable to potential donors. Part of making a donation is responding the signal that you are not the only one sending a check to a hopeless office somewhere.

Anonymous polls might be a good idea, but especially among rational types, you might want the individual testimony: you get to see some of the reasoning!

I think the synagogue in the story picked up on these ideas and used them effectively. But the nice thing about raising money through direct mailing and the internet is that you can run experiments!

Comment by Demosthenes on Issues, Bugs, and Requested Features · 2009-03-20T16:10:14.799Z · LW · GW

I think other people have said it, but Slashdot has one of the best commenting structures around.

Different values for different categories (funny, insightful etc....), anonymous posting, reputation, very clear thread structure. All sorts of fun stuff

Comment by Demosthenes on How to Not Lose an Argument · 2009-03-20T03:54:56.326Z · LW · GW

American Rhetoric is an incredible site and there are some real gems that aspire to rational persuasion with some flair.

http://www.americanrhetoric.com/top100speechesall.html

Malcolm X's "Ballot or the Bullet" navigates the fact that he is black, widely regarded as dangerous and Muslim all at once while urging people to put these things aside and think about his plans and their outcomes. He does a first rate job of tailoring his rhetoric to increase your emotional desire to think and not react.

Milton Friedman is another person to watch live. He gets a bad rap from people who don't watch him, but if you have ever heard him speak, he excels at softly thinking through positions with the listener while still taking bold positions.

http://www.youtube.com/watch?v=JfdRpyfEmBE&feature=player_embedded

They said it best in Howard Stern Private Parts:

"I want to hear what he'll say next."

Gladwell is probably the best for presenting fact and figures in speeches. You might not like his numbers or rigor, but his presentatoon methods are top notch, often focused on challenging his listeners beliefs with academic research:

http://video.google.com/videoplay?docid=6204900041349106153&ei=YRTDSerTM4qIqwLbn9yuCw&q=Gladwell&hl=en

Comment by Demosthenes on Rationalist Fiction · 2009-03-19T14:57:30.100Z · LW · GW

"If You Give a Mouse a Cookie"

A good primer on chaos theory for youth?

"Rendezvous with Rama"

Why have a plot when gradual discovery with expository dialogue will do.

"Contact" -Sagan

More scientific method

"The Diamond Age" -Stephenson

This book even has long discussions of computing

"Sideways Stories from Wayside School"

Anything with jokes is going to be about logic at some point.

"Of Human Bondage" -Maugham

This book has a famous scene where the Phillip goes to Paris to study art; you get the impression that he isn't very good at painting and as time goes on he starts to recognize that his fellow students are not great painters either. After two years, he gradually builds up the courage to have one of his instructors look at all of his work and let him know whether or not he can achieve his goal of becoming great painter. After he receives a negative verdict he commits to a new life plan.

Comment by Demosthenes on The "Spot the Fakes" Test · 2009-03-17T17:30:31.156Z · LW · GW

Setting up this sort of experiment, especially in regard to poetry or other humanities topics, seems to be the overwhelming barrier.

We can take at face value that "Malley's" poems were created from phrases of a limited length selected at random (whatever that really means in this case) and then arranged in a random manner.

This setup would allow us to say that some modernist critics cannot distinguish a modernist poem written by a single person (although with possible allusions and cribbings) from one constructed with phrases less than a specified length from a specific pool of literature.

From what I have found on the affair, it is hard to see if there was much experimental design at all (a criticism that Sokal can share in):

"So, in a series of mischievous creative fugues, they gleaned lines from here and there, even from the American Armed Forces guide to mosquito infestation, and put it together in what they perceived to be a brilliant imitation of the new poetic genre. They dubbed the poet Ern Malley and to avoid the publishers seeking contact with him, they said that, like Keats, he had died young. They then invented his sister, Ethel, who “discovered” the poetry and decided to send it to Harris to judge it for literary merit." -(http://www.ernmalley.com/text_only.html)

In this specific case, we are stuck with two people who seemed to intentionally create a spoof of modernist poetry which is not a terrible representation of the genre. For a progressive journal to publishing something that was designed to make a strong attempt at passing as modernist poetry using the new technique of collage seems completely appropriate.

Does this seem like an adequate control poem for an experiment of this sort:

====

Night Piece

The swung torch scatters seeds
In the umbelliferous dark
And a frog makes guttural comment
On the naked and trespassing
Nymph of the lake.

The symbols were evident,
Though on park-gates
The iron birds looked disapproval
With rusty invidious beaks.

Among the water-lilies
A splash - white foam in the dark!
And you lay sobbing then
Upon my trembling intuitive arm

I suspect that a randomly generated poem from a large amount of source material would look significantly different. I tried out some google poem generators (which are probably not acceptable for this sort of experiment either), and the results weren't as nice http://shawnrider.com/google/index.php?query=modernism&Submit=generate+poem

In the end, problems with authorship and creation by collage are two of the widely recognized features of modernist poetry http://en.wikipedia.org/wiki/Modernist_poetry_in_English. The hoax seems to prove that some of modernist poetry's techniques are indeed effective.

I think your point about intentionally created spoofs like Nathicana coming out as good poetry drives home the point that these sorts of parodies aren't necessarily a good example of control poem construction.

Making these sorts of critiques brings in the distinction between being rational vs rationalizing http://www.overcomingbias.com/2007/09/rationalization.html. If you already have a point you want to prove and proceed to construct a method whereby you'll prove it, it isn't truly rational. If you spend a long time working on experimental design and becoming curious about how these methods (structural analysis of myths or modernist poetry) succeed/fail vs a random smattering of words and ideas, then you can build some rational knowledge on the matter.

While I like the idea of the spot the fakes test, I think it would be difficult to come up with good examples where the experimental design really leads to interesting conclusions with the scope of the project.