Why the singularity is hard and won't be happening on schedule

post by DavidPlumpton · 2011-10-13T19:51:33.063Z · LW · GW · Legacy · 29 comments

Here's a great article by Paul Allen about why the singularity won't happen anytime soon. Basically a lot of the things we do are just not amenable to awesome looking exponential graphs.



Comments sorted by top scores.

comment by lessdazed · 2011-10-13T21:28:54.052Z · LW(p) · GW(p)

I don't blame you for saying "the singularity" rather than "a singularity", as that was in the title of the linked post. However, I think it is bad mental hygiene and wish you hadn't said "the singularity" in the title of your post.

I do blame you for introducing "won't be happening on schedule". As far as I can tell, more commenters here believe in a God of the Bible than believe in Kurzweil's schedule.

Replies from: Normal_Anomaly
comment by Normal_Anomaly · 2011-10-15T15:38:55.730Z · LW(p) · GW(p)

As far as I can tell, more commenters here believe in a God of the Bible than believe in Kurzweil's schedule.

I wish I could upvote this twice: one for being a good use of hyperbole to put the point across, and once for not really being hyperbole.

comment by Desrtopa · 2011-10-13T22:12:59.003Z · LW(p) · GW(p)

I don't put much stock in the specifics of Kurzweil's schedule, but I'm skeptical of the relevance of some of Allen's objections. I agree with the first comment on the article, that he's too focused on replication of human intelligence as a prerequisite for a singularity.

Existing AI may be pretty brittle, but I suspect that by the time we can create AI that can, to use his example, infer that a tightrope walker would have excellent balance, Strong AI is likely not to be far off.

comment by [deleted] · 2011-10-14T01:18:53.513Z · LW(p) · GW(p)

Robin Hanson addressed this same thing today, but found faults in Allen's reasoning. I made a comment there too but I don't know if it has posted yet, so I included it below:

The connectomics project between Harvard and MIT is a particular place where a useful approximate link between specific technologies and ability to emulate brains may be calculable in the short term.

FWIW, I am studying in this course this semester and I am working on some research that uses connectomics to provide plausible complexity bounds on some brain operations, for the purpose of arguing against Ian Parberry’s analysis and conclusion that human cognitive resources are prohibitively difficult to emulate without better abstract knowledge of cognitive science.

There is other recent evidence that suggests an algorithmic approach to brain activity will at the very least give us short term access to replicating certain specific human cognitive functionality. I think this is a case where Leo Breiman’s distinction between “the two cultures” of data analysis is pretty apt.

Replies from: hairyfigment, shminux
comment by hairyfigment · 2011-10-14T08:54:03.161Z · LW(p) · GW(p)

Perhaps you can answer this: does Hanson start from a factually wrong claim about sufficient conditions for brain emulation? Would we need to know the strength of the connections within the brain, as this link claims, and does this pose more difficult problems?

Replies from: None
comment by [deleted] · 2011-10-14T16:59:32.438Z · LW(p) · GW(p)

I am not a brain scientist myself. I think that knowing strength of connection as well as plasticity are both very important. In fact, I think that plasticity considerations are one of the main things that Penrose correctly addresses in The Emperor's New MInd. However, from a graph-theoretic and machine learning point of view, this does not strike me as intractable. Just yesterday I witnessed some new results in the connectomics project in which they can essentially (95+% accuracy on training data sets) reconstruct the wiring diagram of non-trivial volumes of a mouse brain (10^(-7) m^3) in less than 5 minutes using Gibbs sampling methods.

The current problem is getting enough resolution to make the accuracy much higher than 95%. No engineer involved with the project believes this will be difficult to accomplish in a ~10 year time span. The next detail will be imaging at that resolution in a video of a functioning brain and they are already discussing ways to achieve this too. I agree there are some real difficulties in understanding neurotransmitter functionality. But nothing suggests it will be prohibitively difficult for engineering in the next 25 years... at least that's my opinion if we are to just brute force directly store detailed videos of neural activities at the 250 nm resolution level. If we make any breakthrough whatsoever about abstraction and principles in neural physics that allow us to discard some of that brute force data resolution, it will only make the problem easier and less expensive.

Replies from: Normal_Anomaly
comment by Normal_Anomaly · 2011-10-15T15:28:19.703Z · LW(p) · GW(p)

(95+% accuracy on training data sets)

I have only very limited knowledge in this area, so I could be misreading you. But doesn't "in training data sets" mean that the process had been developed using that specific data? That could mean that you have a program really good at reconstructing that piece of mouse brain, but not at reconstructing mouse brain in general. We had this problem in the last research project I worked on, where we'd use a gene expression data set to predict mood in bipolar subjects. We had to test the predictions on a separate data set from the one used in development to make sure it wasn't overfit to the training data. Is the same thing the case for your work, or am I misunderstanding your use of "training data"?

Replies from: None
comment by [deleted] · 2011-10-15T16:37:36.651Z · LW(p) · GW(p)

It is a good insight to notice that this is a potential problem, which is generally referred to as a generalization error. If you train a classifier or compute a regression on some data, there is always a chance that when you are given new data, it will perform poorly because of unforeseen larger-scale patterns that were poorly represented in the training data.

However, the scientists performing this work as also aware of this. This is why algorithmic learning theory, like machine learning methods, is so successful. You can derive tight bounds on generalization error. The process you refer to with the gene expression -- testing on additional labeled data to see that you are not overfitting and that your parameters give good predictive power -- is called cross-validation, and it's definitely a huge part of the connectomics project.

You might enjoy this paper by Leo Breiman, which talks about this exact distinction between merely fitting data vs. algorithmic data analysis. Many statisticians are still stuck believing that it is good to assume underlying analytic models for nature and then use goodness-of-fit tests to determine which underlying models are best. This is a categorically bad way to analyze data except in some special cases. Algorithmic data analysis instead uses cross-validation to measure accuracy and seeks to model the data formation process algorithmically rather than generatively.

Most computer scientists are not even aware of this distinction because the algorithmic approach (usually through machine learning) is the only one they have ever even been taught.

Replies from: Normal_Anomaly
comment by Normal_Anomaly · 2011-10-15T17:07:41.536Z · LW(p) · GW(p)

Thanks for the response and the paper link. I'm confident that the connectomics project does use cross-validation. I'm just wondering, is the 95+% accuracy you mentioned on the training data or the test data?

Replies from: None
comment by [deleted] · 2011-10-15T19:38:19.590Z · LW(p) · GW(p)

It is from cross validation. The training data is for building their procedure, and then the procedure is applied to testing data that was kept separate from the data used to train.

Replies from: Normal_Anomaly
comment by Normal_Anomaly · 2011-10-15T22:47:39.294Z · LW(p) · GW(p)

I see. Good for them! Thanks for the info.

comment by shminux · 2011-10-14T02:38:59.139Z · LW(p) · GW(p)

Hanson being pro-singularity is such a high-probability prior, it contributes nothing to anyone's estimate of the likelihood of a singularity. And, surely enough, his arguments are the same as usual: Moor's law, planes vs. birds, leave out the details etc.

Personally, I like the smbc take on the Moor's law (very mildly NSFW).

On a semi-related note, I envy you taking a course from Scott Aaronson :)

Replies from: dlthomas, shminux
comment by dlthomas · 2011-10-14T18:52:27.352Z · LW(p) · GW(p)

"Of course he would argue for X, therefore you shouldn't listen to his arguments for X" is a fully general counterargument. The fact that he is making arguments for X shouldn't cause much updating; the content of those arguments may very well. I realize that's not precisely the point you are making.

If the arguments are already known to us, then they shouldn't cause any updating if we are perfect logicians. However, it should be noted that this wasn't a general post on the likelihood of the singularity, but a response to the other article (the bird metaphor was theirs, not Hanson's). He was pointing out how already known arguments de-fang the arguments presented: if Hanson's objections are correct, then if you were doing much updating because of the new arguments presented by Paul Allen, you should undo most of it.

Edited to add:

I should note that this is only applicable when the existing argument undermines the other argument, not when it simply overwhelms the new argument, or you get back to double-counting problems.

Replies from: shminux
comment by shminux · 2011-10-14T18:57:07.522Z · LW(p) · GW(p)

if Hanson's objections are correct, then if you were doing much updating because of the new arguments presented by Paul Allen, you should undo most of it.

Aha, that makes sense.

comment by shminux · 2011-10-14T17:48:00.121Z · LW(p) · GW(p)

Hmm, I wonder if I pressed some hot button, given the silent disapproval.

Replies from: hairyfigment
comment by hairyfigment · 2011-10-14T19:17:40.983Z · LW(p) · GW(p)

I upvoted the grandparent to -1 at first, for that reason. But I reversed this at 0 because I saw you imputed the bird argument to him and he didn't use it (nor would it make sense for him to do so).

Also, I don't know what Othello has to do with Moore's Law.

Replies from: shminux
comment by shminux · 2011-10-14T19:50:17.832Z · LW(p) · GW(p)

On planes and birds:

No doubt a detailed enough emulation of bird body motions would in fact fly. It is true that a century ago our ability to create detailed bird body simulations was far less than our ability to infer abstract principles of flight. So we abstracted, and built planes, not bird emulations. But this hardly implies that brains must be understood abstractly before they can be emulated.


I don't know what Othello has to do with Moore's Law.

what Othello?

Replies from: dlthomas, pedanterrific
comment by dlthomas · 2011-10-14T22:59:45.847Z · LW(p) · GW(p)

Othello was a Moor. Moore proposed the law. Up a couple, you confused the two.

comment by pedanterrific · 2011-10-14T22:47:17.958Z · LW(p) · GW(p)

I think the lesson here is that in the set of [webcomics with unpronounceable four-letter names] LW leans more toward xkcd than smbc.

For one thing, if Munroe made that joke you know he'd have actually done the math.

Replies from: shminux
comment by shminux · 2011-10-15T01:22:38.242Z · LW(p) · GW(p)

Too bad he stopped being funny a few years back, while smbc is consistently great and covers the LW-relevant themes in a lot of detail, though I won't risk linking it here.

Replies from: Normal_Anomaly
comment by Normal_Anomaly · 2011-10-15T15:35:27.998Z · LW(p) · GW(p)

YMMV. I think xkcd is consistently amusing, but smbc is a mix of good LW relevance and pointless penis humor. Go ahead and link any relevant smbcs you find in future. The one linked above doesn't really have anything to do with the argument it's mocking, which in turn isn't being endorsed by anyone on here. That's probably why you got downvoted, though I don't know firsthand.

comment by Lapsed_Lurker · 2011-10-13T22:53:41.614Z · LW(p) · GW(p)


"The human brain is really complicated, so I don't think we'll be able to do AI any time soon, because we need 'exponential' increases in software - also 'complexity brake'" (whatever those mean)

[edit] Changed TL:DR to Summary, which I should have used (or similar) in the first place.

Replies from: jasonmcdowell, moridinamael
comment by jasonmcdowell · 2011-10-14T01:30:30.532Z · LW(p) · GW(p)

Thanks, I loathe 'tl;dr'.

comment by moridinamael · 2011-10-14T02:28:31.296Z · LW(p) · GW(p)

I stopped reading when I reached the word "complexity."

Replies from: shminux
comment by shminux · 2011-10-14T04:21:52.277Z · LW(p) · GW(p)

From your link:

But concepts are not useful or useless of themselves. Only usages are correct or incorrect.

In this particular case they are going into some detail to explain what they mean.

comment by lavalamp · 2011-10-13T21:02:00.913Z · LW(p) · GW(p)

It's hard to tell the difference between the very first little bit of an exponential curve and a linear one.

Replies from: billswift
comment by billswift · 2011-10-14T05:32:57.816Z · LW(p) · GW(p)

And a logistic curve is even harder before it starts slowing. I think the various arguments for superhuman AI are far more significant than the projected curves.

Replies from: timtyler
comment by timtyler · 2011-10-14T12:44:16.940Z · LW(p) · GW(p)

The "logistic growth curve" charge, as voiced by John Wilkins:

First of all it occurs to me that people who expect the Singularity to occur simply do not get the logistic growth curve.

Replies from: billswift
comment by billswift · 2011-10-14T17:09:50.912Z · LW(p) · GW(p)

I didn't mean it as a "charge", I was simply pointing out that until and unless it hits an inflection point there is simply no way of knowing which it is (unless there is a known limitation to growth, like the speed of light). Which is one of the reasons that I consider the arguments for superhuman AI far more significant than plotting curves.