The Strong Occam's Razor

gets out the ladder and climbs up to the scoreboard

5 posts without a tasteless and unnecessary torture reference

replaces the 5 with a 0

climbs back down

Years ago, before coming up with even crazier ideas, Wei Dai invented a concept that I named UDASSA. One way to think of the idea is that the universe actually consists of an infinite number of Universal Turing Machines running all possible programs. Some of these programs "simulate" or even "create" virtual universes with conscious entities in them. We are those entities.

Generally, different programs can produce the same output; and even programs that produce different output can have identical subsets of their output that may include conscious entities. So we live in more than one program's output. There is no meaning to the question of what program our observable universe is actually running. We are present in the outputs of all programs that can produce our experiences, including the Odin one.

Probability enters the picture if we consider that a UTM program of n bits is being run in 1/2^n of the UTMs (because 1/2^n of all infinite bit strings will start with that n bit string). That means that most of our instances are present in the outputs of relatively short programs. The Odin program is much longer (we will assume) than one without him, so the overwhelming majority of our copies are in universes without Odin. Probabilistically, we can bet that it's overwhelmingly likely that Odin does not exist.

If two theories imply different invisibles, they shouldn't be considered equivalent. That no evidence can tell them apart, and still they are not equal, is explained by them having different priors. But if two theories are logically (agent-provably, rather) equivalent, this is different, as the invisibles they imply and priors measuring them are also the same.

The thing about positivism is it pretends to be a down-to-earth common-sense philosophy, and then the more you think about it the more it turns into this crazy surrealist madhouse. So we can't measure parallel universes and there's no fact of the matter as to whether they exist. But people in parallel universes can measure them, but there's no fact of the matter whether these people exist, and there's a fact of the matter whether these universes exist if and only if these people exist to measure them, so there's no fact of the matter whether there is a fact of the matter whether these universes exist. And meanwhile to the extent that these people exist, some of them claim there is no fact of the matter as to whether we exist, so really there's not one positivism, but there's a positivism for every quantum world. And in each of these positivisms, which worlds it's meaningful to talk about the existence of gets determined by some random process many times each second. And the question of whether other people in the same universe as you exist is meaningless too, because it makes no predictions that differ from those of the interpretation that other people are all well-disguised walruses who act exactly like people. And if you make an identical copy of yourself, you could end up being either one of them, so there's a 50% chance that there's a fact of the matter whether each continuation is a walrus. Etc.

This theory makes exactly the same observational predictions as your current best theory of physics, so it lies in the same equivalence class and you should give it the same credence.

You're blurring an important distinction between two types of equivalence:

• Empirical equivalence, where two program-theories give the same predictions on all currently known empirical observations.
• Formal equivalence, where two program-theories give identical predictions on all theoretically possible configurations, and this can be proved mathematically.

If two theories are only empirically equivalent, you use the complexity penalty and prefer the simpler one. If the theories are formally equivalent, you don't bother trying to tell them apart. If you don't know which equivalence relation holds, you sit down and start doing math.

Boxes proofed against all direct and indirect observation, potential for observation mixed with concrete practicality of such observation, strictly-worse choices, morality... one would be hard-pressed to muddle your thought experiment more than that.

Let's try to make it a little more straightforward: assume that there exists a certain amount of physical space which falls outside our past light cone. Do you think it is equally likely that it contains galaxies and that it contains unicorns? More importantly, do you think the preceding question means anything?

I'm skeptical of the idea that the hypothesis "Odin created physics as we know it" would actually make no additional predictions over the hypothesis . I'm tempted to say that as a last resort we could distinguish between them by generating situations like "Omega asks you to bet on whether Odin exists, and will evaluate you using a logarithmic scoring rule and will penalize you that many utilons", though at this point maybe it is unjustified to invoke Omega without explaining how she knows these things.

But what do you think of some of the examples given in "No Logical Positivist I"?

On August 1st 2008 at midnight Greenwich time, a one-foot sphere of chocolate cake spontaneously formed in the center of the Sun; and then, in the natural course of events, this Boltzmann Cake almost instantly dissolved.

I would say that this hypothesis is meaningful and almost certainly false. Not that it is "meaningless". Even though I cannot think of any possible experimental test that would discriminate between its being true, and its being false.

Would that be in the same equivalence class as its negation?

no matter how seductive it sounds, skipping over the is-ought distinction is not permitted

Yeah, some of us are still not convinced on that one.

Speaking of which, does anyone actually have something resembling a proof of this? People just seem to cast it about flippantly.

Is the Hulatioamiltonian formulation of classical mechanics apriori more probable than the Lagrangian formn?

They are both derivable from the same source, Newtonian mechanics plus the ZF Set theory. They are equivalent and therefore equally probable.

The shortest possible version of them all - mutually equivalent theories - is the measure how (equally) probable are they.

My favourite justification of the Occam razor is that even if two theories are equivalent in their explicit predictions, the simpler one is usually more likely to inspire correct generalisations. The reason may be that the more complicated the theory is, the more arbitrary constraints it puts on our thinking, and those constraints can prevent us from seeing the correct more general theory. For example, some versions of aether theory can be made eqivalent to special relativity, but the assumptions of absolute space and time make it nearly impossible to discover something equivalent to general relativity, starting from aether.

I personally think Occam's razor is more about describing what you know. If two theories are equally good in their explanatory value, but one has some extra bells and whistles added on, you have to ask what basis you have for deciding to prefer the bells and whistles over the no bells and whistles version.

Since both theories are in fact equally good in their predictions, you have no grounds for preferring one over the other. You are in fact ignorant of which theory is the correct one. However, the simplest one is the one that comes closest to describing the state of your knowledge. The more complicated ones add extra bits that can only really be described as speculations, not knowledge at all, because all the extra bits of 'information' in that theory are not based on any data at all.

Perhaps a more complicated theory is true? Perhaps. But which one of the many many many more complicated theories should you pick? You have no evidence on which to make this choice.

Equally, one shouldn't be too doctrinaire about it. We don't know the simplest explanation is correct - we simply know it's the best way of describing what we know so far. If there are several similar theories of almost equal explanatory weight, there are grounds for reasonable agnosticism even if there is one that's a little 'lighter' than the others.

Adding premises like "Odin created everything" makes a theory less probable and also happens to make it longer; this is the entire reason why we intuitively agree with Occam's Razor in penalizing longer theories. Unfortunately, this answer seems to be based on a concept of "truth" granted from above - but what do differing degrees of truth actually mean, when two theories make exactly the same predictions?

and

Imagine you have the option to put a human being in a sealed box where they will be tortured for 50 years and then incinerated. No observational evidence will ever leave the box... Now consider the following physical theory: as soon as you seal the box, our laws of physics will make a localized exception and the human will spontaneously vanish. This theory makes exactly the same observational predictions as your current best theory of physics, so it lies in the same equivalence class and you should give them the same credence.

Why do we only care about observational predictions and not all "in principle" predictions? (Serious question, not rhetorical). My intuition is that in the first quote "Physics" and "Odin created physics" aren't in the same equivalence class because the latter makes an additional prediction: the existence of Odin. Similarly in the second quote, there are differing predictions about what is happening inside the box even if they are physically impossible to test, so I would put the two theories in different equivalence classes. I would do the same for MWI and Copenhagen quantum mechanics.

This is pure intuition on my part. I haven't had a chance to take a look at the math of K-complexity and the like, so I might just be missing something relatively basic.

Theories that require additional premises are less likely to be true, according to the eternal laws of probability ... Unfortunately, this answer seems to be based on a concept of "truth" granted from above - but what do differing degrees of truth actually mean, when two theories make exactly the same predictions?

Reading this and going back to my post to work out what I was thinking, I have a sort-of clarification for the issue in the quote. The original argument was that, before experiencing the universe, all premises are a priori equally likely, so we can generate as many hypotheses as we like out of them. Then, after experience (a posteriori) some premises are extremely likely (shorthand: true) and some are extremely unlikely (shorthand: false). Now, in our advanced position, there are a large number of false premises, a small number of true premises, and an unknown number of premises we haven't had experiences about. We can now generate as many "prediction-equivalent" theories as we like by combining the unknown premises with the true ones. As long as we avoid the false premises, all our hypotheses will be based on true premises, and premises which we have not yet checked. To refine that argument, it is the conjunction of specifically these unknown premises that might weaken the hypothesis. Therefore, we ought to include as few of these as-yet-untested premises in our hypothesis, in order to reduce the chances of it being wrong.

Now, in the case of two theories making the same prediction: I suggest that it is possible to look at an unknown premise and decide whether we can check it. In this sense, if it is checkable, we can view it as a prediction: the hypothesis includes premise C such that if C is true then the hypothesis is true and if C is false the hypothesis is false. In other words, the hypothesis makes a prediction that C is true. If it is uncheckable, though, we don't use the word prediction. This is the discussion that Matt_Simpson and cousin_it are having down below. If both theories make the same predictions because one says A B C and D, and the other says A B C E F G and H, (A and B are true, C is unknown-testable, D through H are unknown-untestable), then the theories are still distinguishable, still different, but they make the same predictions. In this specific case, we should in principle prefer the first theory, because it has one one fault-line and the second has four: even though we don't think we can test any of these fault-lines, the laws of probability still apply. So this is what degrees of truth mean when all theories make the same predictions.

Now, what about two different theories that say A B C and D? I'm not familiar with the physics, but it appears that Hamiltonian and Lagrangian systems are in this scenario: both say A B C and D, in different but equivalent ways. I haven't had enough time to think to bake up an answer for this, but I suspect it is similar to how you can express the same truth-table with different combinations of premises and operators. The question that stumps me is: in logic, we don't care about the operators except for how they twist the premises. In physics, we actually do care about the operators, to an extent: we give them names like "the mechanism of x" and "the underlying reality". So it seems to me that saying Lagrangian and Hamiltonian are different but equivalent is saying that two different logical formulations with the same truth-table are different but equivalent, except in physics we feel the difference actually matters.

I wonder if this can't be considered more pragmatically? There was a passage in the MIT Encyclopedia of Cognitive Sciences in the Logic entry that seems relevant:

Johnson-Laird and Byrne (1991) have argued that postulating more imagelike MENTAL MODELS make better predictions about the way people actually reason. Their proposal, applied to our sample argument, might well help to explain the difference in difficulty in the various inferences mentioned earlier, because it is easier to visualize “some people” and “at least three people” than it is to visualize “most people.” Cognitive scientists have recently been exploring computational models of reasoning with diagrams. Logicians, with the notable exceptions of Euler, Venn, and Peirce, have until the past decade paid scant attention to spatial forms of representation, but this is beginning to change (Hammer 1995).

This made me think a bit differently about how we might choose between two abstract models with the same explanatory power. It seems that the rational thing to do is to choose the one that allows you to reason the most fluently so as to minimize the likelihood of fallacious reasoning.

In fact, it seems that we should expect the cognitive sciences to provide clues about how we could adjust formal systems with the view of easy of understanding and technical fluency when reasoning about/with them.

Taking this view; assuming we had finished physics, all the future work would be about tweaking the formalisms toward the most intuitive possible ones with respect to the knowledge we have of human reasoning. What would be important is that they be as easy to understand as possible. That way we could hope to ensure more efficiency in technological development as well as better general understanding among the public.

What if Tegmark's multiverse is true? All the equivalent formulations of reality would "exist" as mathematical structures, and if there's nothing to differentiate between them, it seems that all we can do is point to appropriate equivalence class in which "we" exist.

However, the unreachable tortured man scenario suggests that it may be useful to split that class anyway. I don't know much about Solomonoff prior - does it make sense now to build a probability distribution over the equivalence class and say what is the probability mass of its part that contains the man?

Theories that require additional premises are less likely to be true, according to the eternal laws of probability. Adding premises like "Odin created everything" makes a theory less probable and also happens to make it longer; this is the entire reason why we intuitively agree with Occam's Razor in penalizing longer theories. Unfortunately, this answer seems to be based on a concept of "truth" granted from above -

Not to me it doesn't. (Though I may not understand what you mean by "truth" here.) Bayesian probability theory as I've come to understand it deals with maps directly and with the territory only indirectly. It purports to describe how logic, or the laws of thought, apply to uncertainty. So we can describe in some detail how these laws demand a lesser probability for a compound hypothesis, without ever mentioning the reality you want this hypothesis to address. In that sense the math doesn't care about the content of your theories.

(I started to add a silly technical nitpick for something else on the site. Suffice it to say that numerical probabilities serve as maps of other maps, or measures of the trust that a 'rational' mind would have in those maps.)

So should we trust our 'logical' map-evaluating software in this respect? Well, it seems to work so far. As I understand it our mindless evolution created the basics by trial and error, after which we created more mathematics by similar methods. (We twisted our mathematical intuitions into strange shapes like "the square root of minus one" and kept whatever we found a use for.) Bayes' Theorem as we know it emerged from this process. So we can imagine discovering that probability or logic in general has misled us in some fundamental way. Perhaps the most complex possible theory is always correct and we just can't imagine said theory (or, indeed, a way for it to exist). But our internal software tells us not to expect this. ^_^

Another intriguing answer came from JGWeissman. Apparently, as we learn new physics, we tend to discard inconvenient versions of old formalisms. So electromagnetic potentials turn out to be "more true" than electromagnetic fields because they carry over to quantum mechanics much better. I like this answer because it seems to be very well-informed!

I don't like this explanation- while potentials are useful calculation tools both macroscopically and quantum mechanically, fields have unique values whereas potentials have non-unique values. It's not clear to me how to compare those two benefits and decide if one is "more true."

The alternative way to look at it: if you only knew E&M, would you talk in terms of four-vector potentials or in terms of fields? Most of the calculations for complicated problems are easier with potentials (particularly for magnetism), but the target is generally coming up with the fields from those potentials. Similarly, most calculations in QM are easier with the potentials (I've never seen them done with fields, but I imagine it must be possible- you can do classical mechanics with or without Hamiltonians), but the target is wavefunctions or expectation values.

So it's not clear to me what it means to choose potentials over fields, or vice versa. The potentials are a calculation trick, the fields are real, just like in QM the potentials are a calculation trick, and the wavefunction is real. They're complementary, not competing.

Isn't this comment a shorter version of this post Belief in the Implied Invisible?

I just added to the post.

Your thought experiment of the person in the sealed torture box ignores the question of what evidence I have to believe that such a box exists and what evidence I have that the physical theory you've outlined is true (in the thought experiment).

The fact that a theory makes the same predictions as some other theory is irrelevant if I don't have good reason for thinking the theory might be true in the first place. The problem with "Odin created physics" is that I have no good reasons to believe in the existence of Norse gods and that the universe was created by one of them, just like I have no good reasons to believe that a localized exception to physics will occur for some particular box.

To phrase my point a little differently, I think we have to consider the genesis of the theory. If the theory exists because somebody appended "because God made it so" to some other existing theory, then it doesn't matter that it makes the same predictions -- it fails because there are no good reasons for thinking it might be the case. We must consider why the theory makes the predictions it does. "Odin created physics" makes the particular predictions it does for no other reason than that it was specifically designed to make exactly the same predictions (and no other predictions).

I like this argument. But in this case I think there's another argument that doesn't rely on morality so much.

Your belief that the two theories in question will always make the same predictions is conditional on the box being perfectly sealed, and the universe continuing to expand forever. There's a small chance that these things are not true, and if that turns out to be the case, you may or may not expect to see the guy again, depending on what physical theory you believe in.

I think Matt Simpson is getting at this when he talks about counterfactual predictions.

You could counter this by inventing the physical theory that says that the guy in the box disappears, but if somehow you ever recover the box and open it, then the guy will reappear as if he had been there all along. But if we're going that far we may as well posit the theory that the Moon doesn't exist even though we can see it. The definition of existence should involve morality.

In other words. You can't make a theory less/more probable just by expressing it differently, with more/less words.

Only the shortest known formulation counts.

Doesn't the human inside qualify as an observer? For all we know, WE outside the box could be the ones tortured for 50 years and then incinerated once the button is pushed.

It probably has no consequences in reality, because no matter how seductive it sounds, skipping over the is-ought distinction is not permitted.

What, we can't just assume for now that torture is bad without getting into metaethics?

I don't think that argument is even valid. After all, I have the option of putting a human in a box. If I do, one hypothesis states that the human will be tortured and then killed. The other hypothesis states that the human will "vanish"; it's not precisely clear what "vanish" means here, but I'm going to assume that since this state is supposed to be identical in my experience to the state in the first hypothesis, the human will no longer exist. (Alternative explanations, such as the human being transported to another universe which I can never reach, are even more outlandish.)

In either case, I am permanently removing a human from our society. On that basis alone, in the absence of more specific information, I choose not to take this option.

I think you will have to come up with a scenario where 'the action coupled with the more complicated explanation' is more attractive than both 'the action with the simpler explanation' and 'no action' in order to make this argument.

Very nice summary!