The difficulty in predicting AI, in three lines

To summarise the argument further.

"A lot of people talk rubbish about AI. Therefore most existing predictions are not very certain."

That doesn't in itself mean that it's hard to predict AI - merely that there are many existing predictions which aren't that good. Whether we could do better if we (to take the given example) used the scientific method isn't something the argument covers.

I don't see how the third proposition follows from the first two.

The social sciences are contentious, their predictions questionable.

And yet social sciences use the scientific method; mathematics doesn't

Hence statements involving math should be treated as less certain than any prediction in the social sciences.

I've started giving AI timelines as between 10 years and 100 years.

That implicitly assumes that there aren't reasons why social sciences are contentious which don't also apply to AI predictions, but I don't think that's terribly unreasonable (EDIT: where by “I don't think that's terribly unreasonable” I mean that the reasons why the social sciences are contentious despite using the scientific method that I can think off the top of my head would also kind-of apply to AI predictions).

And yet social sciences use the scientific method; AI predictions generally don't.

Can you please clarify this point?

Yes, people say all sorts of unjustified stuff about AI as if their musings were true, out of excitement and carelessness. But the line of thought in the post is ultimately destructive because it sets low expectations for no good reason.

To use the scientific method just means to make falsifiable predictions. So any arbitrary hypothesis counts, no matter how outlandish, so long as it's predictive. On the other hand, you don't need to use science in order to reason, and since "human-level AI" is not available for experimental study, we can only reason about it. But it's a pretty sure thing that such an AI will think that 1+1 equals 2...

There are no details here e.g. about the methodologies used to produce futurological predictions of the "time until X", or about the premises employed in reasoning about AI dispositions and capabilities; and that means there's no argument about the degree of reliability or usefulness that can be obtained when reasoning about AI; just the bare assertion, "not even as good as the worst of social science". Also, there's no consideration of the power of intention. A lot of the important statements in LW's AI futurology are about designing an AI to have desired properties.

A working AI probably needs to duplicate thousands of individual systems found in the human mind. Whether we get there by scanning a brain for 4 years and 1 million electron beams working in parallel, or we have thousands of programming teams develop each subsystem, this is not going to be cheap.

You don't get there by accident - evolution did it, but it took millions of years, with each subsystem being developed to build upon previous ones.

Have you heard anything about some massive corporation or government getting ready to drop a few tril on an all out effort?

No, and the current discussions are how there are not enough common resources to pay for current needs. There isn't enough money to fund large militaries and to pay all of the expenses for the elderly and fix the roads and do everything else as it is. Money has to be borrowed from more successful economies, which just makes the fiscal crisis worse in the future.

Also, no corporation can justify spending more money than any company on the planet actually has to develop something that no one has ever done before and thus seems likely to fail.

Having read the brain emulation roadmap, and articles on how modern neural networks can model individual subsystems in the human mind successfully, this does not seem like a problem that we have to wait another 100 years to solve. The human race might be able to do it in 20 years if they started today and put the needed resources into the problem.

But it isn't going to happen, and predictions of success can't really be made until the actual process is actually started. It could be 10 years from now, it could be 200, before the actual effort is initiated. On the plus side, as time goes on, the cost to do this does go down to an extent. The total "bill of materials" for the hardware goes down with every year with Moore's law. Better software techniques make it more likely that such a huge project could be developed and not be so buggy it wouldn't run at all. But, in 30 years from now, it will still be a difficult and expensive endeavor needing a lot of resources.