Two Truths and a Lie

It's an interesting experiment, and probably a good teaching exercise under controlled conditions to teach people about falsificationism, but real theories are too complex and theories about human behavior are way too complex.

Take the "slam dunk" theory of evolution. If "Some people and animals are homosexual" was in there, I'd pick that as the lie without even looking at the other two (well, if I didn't already know). There are some okay explanations of how homosexuality might fit into evolution, but they're not the sort of thing most people would start thinking about unless they already knew homosexuality existed.

(Another example: plate tectonics and "Hawaii, right smack in the middle of a huge plate, is full of volcanoes".)

I like the cuteness of turning an old parlor game into a theory-test. But I suspect a more direct and effective test would be to take one true fact, invert it, and then ask your test subject which statement fits their theory better. (I always try to do that to myself when I'm fitting my own pet theory to a new fact I've just heard, but it's hard once I already know which one is true.)

Other advantages of this test over the original one proposed in the post: (1) You don't have to go to the trouble of thinking up fake data (a problematic endeavor, because there is some art to coming up with a realistic-sounding false fact -- and also because you actually have to do some research to make sure that you didn't generate a true fact by accident). (2) Your test subject only has a 1 in 2 shot at guessing right by chance, as opposed to a 2 in 3 shot.

I think you oversell the usefulness of this test, both because of how hard it is to make predictions about unrepeatable "experiments" that don't include value-judgments and because of how easy it is to game the statements - imagine:

(A) the false statement to be selected to be false for extraneous reasons and (B) for the proponent of the Big Idea to argue (A) when it isn't true.

Let's say my friend and I are doing this test. His Big Idea is signaling; my task is to construct three statements.

1) Men who want to mate spend a lot of money. (Signaling resources!) 2) Women who want to mate volunteer. (Signaling nurturing!) 3) Children often share with each other, unprompted, while young. (Signaling cooperation to parents!)

Well, obviously #3 isn't right because of other concerns - it turns out competing for, and hoarding, resources has been evolutionarily more successful than signaling social fitness. Does that mean signaling as an idea isn't useful? No; it wrongly explained (3) for a valid reason. (3) is false for reasons unrelated to signaling.

This activity doesn't sound terribly promising to me, but it DOES sound like the sort of thing that is easy enough to try that people should try it rather than just criticizing it. One of the great insights of the Enlightenment is that math-guys are more averse to doing actual experiments to test their ideas than they should be if they are trying to understand the world and thus to win.

Points for pjeby's comment once again, by the way.

I don't think it's anywhere as easy or as effective as you seem to suggest. In the land of theories so informal that there is a nontrivial question of whether they are trivial, one can get a hint of which statements are correct by other means and then rationalize these hints using the Great Theory. If it's not possible to tell which statements are correct, one can argue that the questions are too tough, and any theory has limits to its application.

Try this with Natural Selection and you will find that it can explain just about any animal behavior, even fake animal behavior. What should be the take away lesson from this?

I expect that, in practice, the theory holders will demure to offer answers. They will instead say, "Those aren't the kinds of questions that my theory is designed to answer. My theory answers questions like . . . ". They will then suggest questions that are either vague, impossible to answer empirically, or whose answers they know.

That's my theory, and I'm sticken' to it :).

No, you shouldn't make broad inferences about human behaviour without any data because they are consistent with evolution, unless your application of the theory of evolution is so precise and well-informed that you can consistently pass the Two-Truths-and-a-Lie Test.

This sentence could have beneficially ended after "behavior," "data," or "evolution." The last clause seems to be begging the question - why am I assuming the Two-Truths-and-a-Lie Test is so valuable? Shouldn't the test itself be put to some kind of test to prove its worth?

OK, so my favorite man-with-a-hammer du jour is the "everyone does everything for selfish reasons" view of the world. If you give money to charity, you do it for the fuzzy feeling, not because you are altruistic.

What would you propose as the three factual claims to test this? I'm having a hard time figuring any that would be a useful discriminant.

Thinking about this a bit, it seems most useful to assert negative factual claims, ie: "X never happens".

Merely stylistic, I think, but I'd avoid "It has been said that...". Aside from being inappropriate passive voice (Who said it?), it has that weird feel of invoking ancient wisdom. That's only cute when Eliezer does it.

A modification for this game: come up with a list of 3 (or more) propositions, which are randomly false/true. This way the predictive theory can fail or succeed in a more obvious way.

I think this is cross-validation for tests. There have been several posts on Occam's Razor as a way to find correct theories, but this is the first I have seen on cross-validation.

In machine learning and statistics, a researcher often is trying to find a good predictor for some data and they often have some "training data" on which they can use to select the predictor from a class of potential predictors. Often one has more than one predictor that performs well on the training data so the question is how else can one choose an appropriate predictor.

One way to handle the problem is to use only a class of "simple predictors" (I'm fudging details!) and then use the best one: that's Occam's razor. Theorists like this approach and usually attach the word "information" to it. The other "practitioner" approach is use a bigger class of predictors where you tune some of the parameters on one part of the data and tune other parameters (often hyper-parameters if you know the jargon) on a separate part of the data. That's the cross-validation approach.

There's some results on the asymptotic equivalence of the two approaches. But, what's cool about this post is that I think it offers a way to apply cross-validation to an area where I have never heard it discussed (I think, in part, because its the method of the practitioner and not so much the theorist--there are exceptions of course!)

this is why I am not a libertarian. I believe in liberty in general, but it is just one more tool to getting what we want and should be treated accordingly.

but you need liberty to have wants

no. thou art physics remember? postulating magical preferences that materialize from the ether is silly. your wants are shaped by forces beyond your control. balking at a few more restrictions in cases where there is clearly a net good is silly.

This reminds me of David Deutsch's talk "A new way to explain explanation" (oddly enough also reposted here)

Seems to be related to David Deutsch - A new way to explain explanation