Link: Strong Inference

Is artificial intelligence, rightly considered, an empirical science? If not, what is it?

Most of the empirical sciences you mentioned involve understanding things that already exist, rather than bringing new things into existence.

This sounds like a disguised query. What are you really asking about AI? Separate the definition from the implications; what's an "empirical science", what further properties does the definition imply about entities it describes, and does AI need those properties to accomplish its goal?

(I'd describe it more as a research program that draws from several empirical sciences (cognitive science, etc.) and sometimes motivates advances in those fields, in the same way that nuclear physics is an empirical science, while the projects to create nuclear weaponry were not empirical sciences in themselves but were research goals drawing heavily from scientific knowledge.)

Why can't AI researchers formulate and test theories the way high-energy physicists do?

Within limited domains, they do. Face recognition, for instance. You are correct that it's not a solved problem, but any new theory of face recognition, or in general any proposed computational model of the visual cortex, can be trivially tested: have it look through pictures and recognize faces, and compare it to how well untrained humans can recognize faces.

Also, reiterating a comment I made a couple days ago on a similar statement of yours:

I am not, of course, against mathematics per se. But the reason math is used in physics is because it describes reality. All too often in AI and computer vision, math seems to be used because it's impressive.

I'd find it much more impressive if you could do anything useful in AI or computer vision without math.

You seem to be jumping from "AI isn't a hard science like physics because we don't know exactly what we're trying to mathematically model" (since it studies minds-in-general rather than just the minds we can directly do experiments on) to "math isn't useful in AI or computer vision". Maybe computer vision can benefit from Riemannian manifolds, maybe it can't, maybe it's a privileged hypothesis that we shouldn't even be bothering to ask about, but do you really expect that any technical solution will be non-mathematical?

My gut feeling is that the top-level post doesn't give a nice summary of what 'strong inference' actually is, so here's a snippet from Platt's paper that does:

Strong inference consists of applying the following steps to every problem in science, formally and explicitly and regularly:

1) Devising alternative hypotheses;

2) Devising a crucial experiment (or several of them), with alternative possible outcomes, each of which will, as nearly as possible, exclude one or more of the hypotheses;

3) Carrying out the experiment so as to get a clean result;

1') Recycling the procedure, making subhypotheses or sequential hypotheses to refine the problems that remain; and so on.

It is like climbing a tree.

I would like to observe that you can divide sciences as follows:

1) physical sciences (Physics and Chemistry are models): investigating the fundamentals using the old idea of the scientific method, ie hypothesis to experiment to accepting or rejecting hypothesis. It is a predict and test method.

2) historic sciences (Biology, Geology, Cosmology are models but not Biochemistry and Biophysics which go it the first group); uses the theories of physical science to create historic theories (like evolution or plate tectonics) about how things became what they are. It is only rarely that a hypothesis can be directly tested by experiment and the method sort of proceeds as observation/taxonomy to hypothesis explaining bodies of observations to simple experiments to prove that proposed processes are possible. It is a collect data and try to organize into a integrated story sort of method and very inductive.

3) inventive Science (Engineering, Medicine are examples): uses the theories of physical and historic sciences to create useful and/or profitable things. Here the method is to identify a problem then look for solutions and test proposed solutions in tests/trials.

Associated with these sciences are theoretical areas (Mathematics, Information Theory are examples): These do not test theories and are not in the business of predicting and testing against reality. Their deductive rather then inductive. They create theoretic structures that are logically robust and can be used by the other sciences as powerful tools.

I have left out the social sciences, history proper, linguistics, anthropology and economics because it is not clear to me that they are even sciences and they do not fit the mold of mathematics either. But they are (like the others) large communal scholarships and I am not putting them down when I say they may not be sciences.

My definition of a science is a communal scholarship that

a) is not secretive but public using peer reviewed publication (or its equivalent) in enough detail that the work could be repeated,

b) is concerned with understanding material physical reality and doing so by testing theories in experiments or their equivalents ie no magical or untestable explanations,

c) accepts the consensus of convinced scientists in the appropriate field rather than an authority as a measure of truth. The method by which the scientists are convinced may be anything in principle, but scientists are not likely to be convinced if the math has mistakes, logic is faulty, experiments are without controls, supernatural reasons are used etc. etc.

AI would definitely fall into the inventive sciences. An appropriate method would be to identify a problem, invent solutions using knowledge of physical and historic science and the tools of math etc., see if the solutions work in systematic tests. None of these is simple. To identify a problem, you need to have a vision of what is the end point success and a proposed path to get there. Vision does not come cheap. Invention of solutions is a creative process. Testing takes as much skill and systematic, clear thinking as any other experimental-ish procedure.

How many of us write down our alternatives and crucial experiments every day, focusing on the exclusion of a hypothesis?

I'm reminded of Gawande on the importance of using checklists in medicine.

To me, the major questions are a) is Platt's thesis correct b) how much further has this process proceeded since 1964. c) do any or most applied sciences currently do the sort of science Platt advocates d) if not does this imply the possibility for a soft-takeoff via singleton through the reinvention of science?

Re: Is artificial intelligence, rightly considered, an empirical science? If not, what is it?

Not exactly science: more technology and engineering.

Are there any particular examples of fields where "overmathematizing" demonstrably slowed down research?

This seems like your standard physics bias. That is, if what these scientists are doing doesn't look exactly like what physicists are doing, what they are dong isn't really science.

Come on guys, this stuff should have died off in the 1960s. Evolutionary biology, microeconomics, and artificial intelligence cannot and should not try to be physics. The very nature of the subject matter prevents it from being so.

I'm amazed that this paper is from '64 and so are other very interesting ones I have been reading lately. There are so many valuable gems from the past yet they seem to be never taught and quickly forgotten.

Why can't AI researchers formulate and test theories the way high-energy physicists do?

But surely they do? Every proposal for a way of doing AI (I'm reading this as AGI here) is a hypothesis about how an AI could be created, and the proposers' failure to create an AI is the refutation of that hypothesis. Science as normal. Talk of physics envy is just an excuse for failure. The problem with excusing failure is that it leaves you with failure, when the task is to succeed.

I think there is a science of intelligence which (in my opinion) is closely related to computation, biology, and production functions (in the economic sense). The difficulty is that there is much debate as to what constitutes intelligence: there aren't any easily definable results in the field of intelligence nor are there clear definitions.

There is also the engineering side: this is to create an intelligence. The engineering is driven by a vague sense of what an AI should be, and one builds theories to construct concrete subproblems and give a framework for developing solutions.

Either way this is very different than astrophysics where one is attempting to: say, explain the motions of the heavenly sphere: which have a regularity, simplicity, and clarity to them that is lacking in any formulation of the AI problem.

I would say that AI researchers do formulate theories about how to solve particular engineering problems for AI systems, and then they test them out by programming them (hopefully). I suppose I count, and that's certainly what I and my colleagues do. Most papers in my fields of interest (machine learning and speech recognition) usually include an "experiments" section. I think that when you know a bit more about the actually problems AI people are solving you'll find that quite a bit of progress has been achieved since the 1960's.

Excellent link - thank you!

You misquoted:

I think the usual factors

should be:

I think the usual explanations