Fake Causality

I just wanted to say that this is the best damn blog I've read. The high level of regular, insightful, quality updates is stunning. Reading this blog, I feel like I've not just accumulated knowledge, but processes I can apply to continue to refine my understanding of how I think and how I accumulate further knowledge.

I am honestly surprised, with all the work the contributors do in another realms, that you are able to maintain this high level of quality output on a blog.

Recently I have been continuing my self-education in ontology and epistemology. Some sources are more rigorous than others. Reading Rand, for example, shows an author who seems to utilize "phlogiston" like mechanics to describe her ethical solutions to moral problems. Explanations that use convincing, but unbounded turns of phrase instead of a meaningful process of explanation. It can be very challenging to read and process new data and also maintain a lack of bias (or at least an awareness of bias, that can be accounted for and challenged). It requires a very high level of active, conscious information processing. Rereading, working exercises, and thinking through what a person is saying and why they are saying it. This blog has provided me lots of new tools to improve my methods of critical thinking.

Rock on.

I feel like I've not just accumulated knowledge, but processes I can apply to continue to refine my understanding of how I think and how I accumulate further knowledge.

You've warmed my heart for the day.

Great post and I agree with Brandon. Eliezer, I recommend you admin a message board (I've been recommending an overcomingbias message board for a while) but I think in particular you'd thrive in that environment due to your high posting volume and multiple threads of daily interest. I think you're a bit constrained intellectually, pedagogically, and speculatively by this format.

I think I've said this before, but there is some defense that can be made for the phlogiston theorists. Phlogiston is like an absence of oxygen in modern combustion theory. The falsifiable prediction that caused phlogiston to be abandoned was that phlogiston would have mass, whereas an absence of oxygen (what it was in reality) does not.

Could evolution be a fake explanation in that it doesn’t predict anything? I’m no creationist but what your explaining in regards to phlogiston seems to have a lot of similarity to evolution. Seems to me like no matter what the data is you can put the tag of evolution on it. Now I’m no expert on evolution so don’t flame me. Just a question on how evolution is different.

What TGGP said. Also, would an AI really be better at determining the falsifiability of a theory? It seems to me that, given a particular theory, an algorithm for determining the set of testable predictions thereof isn't going to be easy to optimize. How does the AI prove that one algorithm is better than another? Test it against a set of random theories?

C of A, TalkOrigins addresses your argument.

Phlogiston is not necessarily a bad thing. Concepts are utilized in reasoning to reduce and structure search space. Concepts can be placed in correspondence with multitude of contexts, selecting a branch with required properties, which correlate with its usage. In this case active 'phlogiston' concept correlates with presence of fire. Unifying all processes that exhibit fire under this tag can help in development of induction contexts. Process of this refinement includes examination of protocols which include 'phlogiston' concept. It's just not a causal model, which can rigorously predict nontrivial results through deduction.

Eliezer, we need more posts from you elucidating the importance of optimizing science, etc., as opposed to the current, functional elements of it. In my opinion people are wasting significant comment time responding to each of your posts by saying "hey, such-and-such that you criticized actually has some functionality".

An analogous principle operates in rigorous probabilistic reasoning about causality. ... We count each piece of evidence exactly once; no update message ever "bounces" back and forth. The exact algorithm may be found in Judea Pearl's classic "Probabilistic Reasoning ...

Actually, Pearl's algorithm only works for a tree of cause/effects. For non-trees it is provably hard, and it remains an open question how best to update. I actually need a good non-tree method without predictable errors for combinatorial market scoring rules.

In response to Hopefully Anonymous, I think there is a real difference between unfalsifiable pseudosciences and genuine scientific theories (both correct and incorrect). Coming up with methods to distinguish the two will be helpful for us in doing science. It is easy in hindsight to say how obviously wrong something is, it is another to understand why it is wrong and whether its wrongness could have been detected then with the information available as this could assist us later when we do not have all the information we would wish to.

Robin: Yes indeed. If you can find a cutset for the tree, or cluster a manageable set of variables, all is well and good. I suspect this is what happens with most real-life causal models.

But in general, finding a good non-tree method is not just NP-hard but AI-complete. It is the problem of modeling reality itself.

Robin Hanson said: "Actually, Pearl's algorithm only works for a tree of cause/effects. For non-trees it is provably hard, and it remains an open question how best to update. I actually need a good non-tree method without predictable errors for combinatorial market scoring rules."

To be even more precise, Pearl's belief propagation algorithm works for the so-called 'poly-tree graphs,' which are directed acyclic graphs without undirected cycles (e.g., cycles which show up if you drop directionality). The state of the art for exact inference in bayesian networks are various junction tree based algorithms (essentially you run an algorithm similar to belief propagation on a graph where you force cycles out by merging nodes). For large intractable networks people resort to approximating what they are interested in by sampling. Of course there are lots of approaches to this problem: bayesian network inference is a huge industry.

Very interesting. In computer networking, we deal with this same information problem, and the solution (not sending the information from the forward node back to the forward node) is referred to as Split Horizon.

Suppose that Node A can reach Network 1 directly - in one hop. So he tells his neighbor, Node B, "I can get to Network 1 in one hop!". Node B records "okay, I can get there in two hops then." The worry is that when Node A loses his connection to Network 1, he asks Node B how to get there, and Node B says "don't worry, I can get there in two hops!". This causes Node A to hand his traffic to Node B, who promptly turns it around and hands it back, and thus a loop is created. The solution, split horizon, is exactly as you say here: when you learn a piece of information, record which direction you learned it, and do not advertise that information back in that direction.

Thanks for the link Davis but it does not address the issue that is brought up in the original post. The examples given in your link were "retrodictions". To quote the original post...

“Thanks to hindsight bias, it's also not enough to check how well your theory "predicts" facts you already know. You've got to predict for tomorrow, not yesterday. It's the only way a messy human mind can be guaranteed of sending a pure forward message.”

I’m not arguing that evolution is pseudoscience. I’m just saying that evolution as an explanation could makes us think we understand more than we really do. Again I am no creationist, the data clearly does not fit the creationist explanation.

Another suberb post. I learn so much from your writings.

Is phlogiston theory so much worse than dark matter? Both are place-holders for our ignorance, but neither are completely mysterious, nor do they prevent further questions or investigation into their true nature. If people had an excellent phenomenological understanding of oxygen, but called it phlogiston and didn't know about atoms or molecules, I wouldn't discount that. Similarly, it can be very useful to use partial, vague and not-completely-satisfactory models, like dark matter.

I'm not sure this that this is fair to phlogiston or the scientists who worked with it. In fact, phlogiston theory made predictions. For example, substances in general were supposed to lose mass when combusting. The fact that metals gain mass when rusting was a data point that was actively against what was predicted by phlogiston theory. The theory also helped scientists see general patterns. So even if the term had been a placeholder, it allowed scientists to see that combustion, rust and metabolism were all ultimately linked procedures. The very fact that these shared patterns and that phlogiston predicted changes in mass (and that it failed to predict the behavior of air rich in oxygen and carbon dioxide (although those terms were not used until late)) in the wrong direction helped lead to the rejection of phlogiston theory. There are classical examples of theories with zero or close to zero predictive value. But phlogiston is not one of them.

Edit: Said lost mass when meant gain mass. Metals gain mass when rusting they don't lose mass. Phlogiston said they should lose mass but they actually gain mass. Also, fixed some grammar issues.

Maybe phlogiston is also magic, thus witches were imbued with it. This might explain some things.

So.. How precisely would I go about doing this? I mean, let's say I really thought that phlogiston was the reason fire was hot and bright when it burns. Something that today, we know to be untrue. But if I really thought it was true, and I decided to test my hypothesis, how would I go about proving it false?

What I think the point is about, is that if I already believe that phlogiston was the reason fire is hot and bright, and I observe fire being both hot and bright, then I think this proves that phlogiston is the reason fire is hot and bright. When actually, that's pointless because I'd have to prove that phlogiston is indeed the reason fire is hot and bright, not the other way around. Am I right? Or did I just end up confusing myself even more, because I'm not entirely sure that what I said is correct and/or makes any sense. O_o

I really enjoyed reading this post, especially its connection with the Pearl's belief propagation algorithm in bayesian networks.Thank you Eliezer!

This is a great layperson explanation of the belief propagation algorithm.

However, the phlogiston example doesn't show how this algorithm is improperly implemented in humans. To show this, you need an example of incorrect beliefs drawn from a correct model, i.e. good input to the algorithm resulting in bad output. The phlogiston model was clearly incorrect. As other commenters have pointed out, contemporary scientists were painfully aware of this, and have eventually abandoned the model. Bad output from bad input doesn't demonstrate a bug in implementation, certainly not the specific bug you mentioned:

we don't keep rigorously separate books for the backward-message and forward-message

Such a defect would probably not even allow a mouse to be as intelligent as it is.

Sadly, we humans can't rewrite our own code, the way a properly designed AI could.

Sure we can!
In fact, we can't stop rewriting our own code.

When you use the word "code" to describe humans, you take a certain degree of semantic liberty. So we first need to understand what is meant by "code" in this context.

In artificial computing machines, code is nothing more than a state of a chunk of memory hardware that causes the computation hardware to operate in a certain way (for a certain input). Only a tiny subset of the possible states of any chunk of memory hardware are "executable", i.e. don't cause the computation hardware to reach a certain "failure" state. This gives us an almost clear-cut distinction between (executable) code and (non-executable) data, under the assumption that data is very unlikely to be executable by chance. Given the correct design, a machine can write code to its memory and then execute it.

In humans, the distinction between memory hardware and computation hardware is unclear, if it exists at all. Moreover, it's unclear how to apply the above distinction between code and data: what is a human's "failure" state? I guess a state of the brain (containing both memory and computation hardware, until and unless we can ever separate the two) can be said to be "executable" if, placed in a certain environment, the person doesn't go and die.

It follows that any change that the brain does to its own state, which then affects its computation, to the result of not dying, is, in fact, "rewriting its own code". This, of course, happens all the time and (perhaps ironically) cannot be stopped without killing the brain.

In a wider loop, we also have drugs, medications and, eventually, gene therapy. But that's more similar to a robot reaching for the keyboard (or the screwdriver).

Another interesting (and sad) example: during the conversation between Deepak Chopra and Richard Dawkins here, Deepak Chopra used the words "quantum leap" as an "explanation" for the origin of language, the origin of life, jumps in the fissile record, etc.

Edit: Finally he claimed it was a metaphor.

There's a fair amount of hindsight bias going on with this critique of phlogiston. Phlogiston sounds plausible on the surface. It's a reasonable conjecture to make given the knowledge at the time and certainly worth investigation. Is it really any less absurd to postulate some mystery substance in the air that essentially plays the same role? If they'd chosen the latter, we'd be lauding them for their foresight.

It's perfectly feasible to draw up tables of the presumed phlogiston content of various materials and use this to deduce the mass of the residue after complete combustion. You could even predict how much of a material would burn if placed in a closed container. Phlogiston or oxygen, you get the same tables and the same empirical laws--before anyone discovered the latter. It's a lot easier to come up with empirical laws than it is to explain the underlying mechanism.

Is anyone satisfied with a physical theory content to give empirical laws and which makes no comment on underlying mechanisms? Gravity was used because it worked, but I doubt anyone was comfortable with the action at a distance. The Copenhagen Interpretation is an active refusal to engage in any kind of speculation as to why quantum mechanics "really" works, and routinely gets slammed on this site.

Does Phlogiston make the fire hot the same way CO2 makes the climate change?

The phlogiston theory gets a bad rap. I 100% agree with the idea that theories need to make constraints on our anticipations, but i think you're taking for granted all the constraints phlogiston makes.

The phlogiston theory is basically a baby step towards empiricism and materialism. Is it possible that our modern perspective causes us to take these things for granted to the point that the steps phlogiston ads aren't noticed? In another essay you talk about walking through the history of science, trying to imagine being in the perspective of someone taken in by a new theory, and i found that practice particularly instructive here. I came up with a number of ways in which this theory DOES constrain anticipation. Seeing these predictions may make it easier to help raise new predictions for existing theories, as well as suggest that theories don't need to be rigorous and mathematical in order to constrain the space of anticipations.

The phlogiston theory says "there is no magic here, fire is caused by some physical property of the substances involved in it". By modern standards this does nothing to constrain anticipation further, but from a space of total ignorance about what fire is and how it works, the phlogiston theory rules out such things as:

• performing the correct incantation can make the difference between something catching and not catching fire
• If some elements catch fire in one location, changing the location of those elements, or the time of day, or the time of year that the experiment is performed, shouldn't make it easier or harder to start a fire. The material conditions are the only element that matters when determining whether something will catch fire.
• If a jar placed over the candle caused the candle to go out, because the air is 'saturated with phlogiston', then placing a new candle under the same jar should result in the new candle also going out . As long as the air under the jar hasn't been swapped out, if it was 'saturated with phlogiston' before we changed the candle, it should remain 'saturated with phlogiston' after the candle.

The last example is particularly instructive, because the phrase "saturated with phlogiston" is correct as long as we interpret it to mean "no longer containing sufficient oxygen." That is a correct prediction based on the same mechanism as our current (extremely predictive) understanding of what makes fires go out. It's that the phlogiston model just got the language upside down and backwards, and mistakes the absence of fuel for the presence of something that inhibits the reaction. They did call oxygen "dephlogisticated air", and so again, the theory says "this stuff is flammable, wherever it goes, whatever the time of day, or whatever incantation or prayer you say over it" - which is correct, but so obviously true that we perhaps aren't seeing it as constraining anticipation.

From my understanding of the history of science, it's possible that the phlogiston theory constrained the hypothesis space enough to get people to search for strictly material-based explanations of phenomena like fire. In this sense, a belief that "there is a truth, and our models can come closer to it over time" also constrains anticipation, because it says what you won't experience: a search for truth that involves gathering evidence over time, and refining models, which never get better at predicting experience.

Is a model still useful if it only constrains the space of hypotheses that are likely to pan out with predictive models, rather than constraining the space of empirical observations?