Li and Vitanyi's bad scholarship

post by alanf · 2020-09-05T08:07:02.851Z · LW · GW · 11 comments

Contents

11 comments

In An Introduction to Kolmogorov Complexity and Its Applications 4th edition Li and Vitanyi claim that Solomonoff Induction solves the problem of induction. In Section 5.1.4 they write:

The philosopher D. Hume (1711–1776) argued that true induction is impossible because we can reach conclusions only by using known data and methods. Therefore, the conclusion is logically already contained in the start configuration. Consequently, the only form of induction possible is deduction. Philosophers have tried to find a way out of this deterministic conundrum by appealing to probabilistic reasoning such as using Bayes’s rule. One problem with this is where the prior probability one uses has to come from. Unsatisfactory solutions have been proposed by philosophers such as R. Carnap (1891–1970) and K.R. Popper.

What Hume actually wrote about induction was (Section IV, Part II, pp. 16-17):

Should it be said that, from a number of uniform experiments, we infer a connexion between the sensible qualities and the secret powers; this, I must confess, seems the same difficulty, couched in different terms. The question still recurs, on what process of argument this inference is founded? Where is the medium, the interposing ideas, which join propositions so very wide of each other? It is confessed that the colour, consistence, and other sensible qualities of bread appear not, of themselves, to have any connexion with the secret powers of nourishment and support. For otherwise we could infer these secret powers from the first appearance of these sensible qualities, without the aid of experience; contrary to the sentiment of all philosophers, and contrary to plain matter of fact. Here, then, is our natural state of ignorance with regard to the powers and influence of all objects. How is this remedied by experience? It only shows us a number of uniform effects, resulting from certain objects, and teaches us that those particular objects, at that particular time, were endowed with such powers and forces. When a new object, endowed with similar sensible qualities, is produced, we expect similar powers and forces, and look for a like effect. From a body of like colour and consistence with bread we expect like nourishment and support. But this surely is a step or progress of the mind, which wants to be explained. When a man says, I have found, in all past instances, such sensible qualities conjoined with such secret powers: And when he says, Similar sensible qualities will always be conjoined with similar secret powers, he is not guilty of a tautology, nor are these propositions in any respect the same. You say that the one proposition is an inference from the other. But you must confess that the inference is not intuitive; neither is it demonstrative: Of what nature is it, then? To say it is experimental, is begging the question. For all inferences from experience suppose, as their foundation, that the future will resemble the past, and that similar powers will be conjoined with similar sensible qualities. If there be any suspicion that the course of nature may change, and that the past may be no rule for the future, all experience becomes useless, and can give rise to no inference or conclusion. It is impossible, therefore, that any arguments from experience can prove this resemblance of the past to the future; since all these arguments are founded on the supposition of that resemblance. Let the course of things be allowed hitherto ever so regular; that alone, without some new argument or inference, proves not that, for the future, it will continue so. In vain do you pretend to have learned the nature of bodies from your past experience. Their secret nature, and consequently all their effects and influence, may change, without any change in their sensible qualities. This happens sometimes, and with regard to some objects: Why may it not happen always, and with regard to all objects? What logic, what process or argument secures you against this supposition? My practice, you say, refutes my doubts. But you mistake the purport of my question. As an agent, I am quite satisfied in the point; but as a philosopher, who has some share of curiosity, I will not say scepticism, I want to learn the foundation of this inference. No reading, no enquiry has yet been able to remove my difficulty, or give me satisfaction in a matter of such importance. Can I do better than propose the difficulty to the public, even though, perhaps, I have small hopes of obtaining a solution? We shall at least, by this means, be sensible of our ignorance, if we do not augment our knowledge.

What Hume wrote isn’t that we can only use known data and methods. Rather, he said that no argument can prove that the future will resemble the past. So drawing conclusions about what will happen in the future from past data is illogical. He didn’t say that the only possible form of induction is deduction.

In addition, the future always resembles the past in some respects and not others, so saying the future resembles the past is irrelevant to creating and assessing ideas.

Popper wasn't trying to solve the problem of how to make Bayesian induction work. He claimed that induction was impossible, not that he had a way of making it work by finding the right prior (Realism and the Aim of Science, Chapter I, Section 3, I):

It seems that almost everybody believes in induction; believes, that is, that we learn by the repetition of observations. Even Hume, in spite of his great discovery that a natural law can neither be established nor made ‘probable’ by induction, continued to believe firmly that animals and men do learn through repetition: through repeated observations as well as through the formation of habits, or the strengthening of habits, by repetition. And he upheld the theory that induction, though rationally indefensible and resulting in nothing better than unreasoned belief, was nevertheless reliable in the main—more reliable and useful at any rate than reason and the processes of reasoning; and that ‘experience’ was thus the unreasoned result of a (more or less passive) accumulation of observations.

As against all this, I happen to believe that in fact we never draw inductive inferences, or make use of what are now called ‘inductive procedures’. Rather, we always discover regularities by the essentially different method of trial and error, of conjecture and refutation, or of learning from our mistakes; a method which makes the discovery of regularities much more interesting than Hume thought.

Li and Vitanyi want us to think they can solve the problem of induction, but they can’t even summarise the arguments against their position accurately.

11 comments

Comments sorted by top scores.