An inflection point for probability estimates of the AI takeoff?

This sounds like a probability search problem in which you don't know for sure there exists anything to find - the hope function.

I worked through this in #lesswrong with nialo. It's interesting to work with various versions of this. For example, suppose you had a uniform distribution for AI's creation over 2000-2100, and you believe its creation 90% possible. It is of course now 2011, so how much do you believe it is possible now given its failure to appear between 2000 and now? We could write that in Haskell as let fai x = (100-x) / ((100 / 0.9) - x) in fai 11 which evaluates to ~0.889 - so one's faith hasn't been much damaged.

One of the interesting things is how slowly one's credence in AI being possible declines. If you run the function fai 50*, it's 81%. fai 90** = 47%! But then by fai 98 it has suddenly shrunk to 15% and so on for fai 99 = 8%, and fai 100 is of course 0% (since now one has disproven the possibility).

* no AI by 2050

** no AI by 2090, etc.

EDIT: Part of the interestingness is that one of the common criticisms of AI is 'look at them, they were wrong about AI being possible in 19xx, how sad and pathetic that they still think it's possible!' The hope function shows that unless one is highly confident about AI showing up in the early part of a time range, the failure of AI to show up ought to damage one's belief only a little bit.

That blog post is also interesting from a mind projection fallacy viewpoint:

"What I found most interesting was, the study provides evidence that people seem to reason as though probabilities were physical properties of matter. In the example with the desk with the eight drawers and an 80% chance a letter is in the desk, many people reasoned as though “80% chance-of-letter” was a fundamental property of the furniture, up there with properties like weight, mass, and density.

Many reasoned that the odds the desk has the letter, stay 80% throughout the fruitless search. Thus, they reasoned, it would still be 80%, even if they searched seven drawers and found no letter. And these were people with some education about probability! One problem is people were tending to overcompensate to avoid falling into the Gambler’s Fallacy. They were educated, well-learned people, and they knew that the probability of a fair coin falling heads remains 50%, no matter how many times in a row heads have already been rolled. They seemed to generalize this to the letter search. There’s an important difference, though: the coin flips are independent of each other. The drawer searches are not.

In a followup study, when the modified questions were posed, with two extra “locked” drawers and a 100% initial probability of a letter, miraculously the respondents’ answers showed dramatic improvement. Even though, formally, the exercises were isomorphic."

2k/2k1/2002/... 2040  2041       2042         2043             2044              2045                2046                   2047                    2048                          2049                              2050 etc.

AI can beat humans at chess, autonomously generate functional genomics hypotheses, discover laws of physics on its own, create original, modern music, identify pedestrians and avoid collisions, answer questions posed in natural language, transcribe and translate spoken language, recognize images of handwritten, typewritten or printed text, produce human speech, traverse difficult terrain...

There seems to be a lot of progress in computer science but it doesn't tell us much about the probability, let alone timescale, of artificial general intelligence undergoing explosive recursive self-improvement. Do we even know what evidence we are looking for, would we recognize it?

How can we tell when we know enough to build a seed AI that can sprout superhuman skills, that are not hardcoded, as diverse as social engineering, from within a box? How do we even tell such a thing is possible in principle? What evidence could convince someone that such a thing is possible or impossible?

Just imagine you emulated a grown up human mind and it wanted to become a pick up artist, how would it do that with an Internet connection? It would need some sort of avatar at least and then wait for the environment to provide a lot of feedback.

So even if we're talking about the emulation of a grown up mind it will be really hard to acquire some capabilities. Then how is the emulation of a human toddler going to acquire those skills? Even worse, how is some sort of abstract AGI going to do it that misses all of the hard coded capabilities of a human toddler?

Can we even attempt to imagine what is wrong about a boxed emulation of a human toddler that makes it unable to become a master of social engineering in a very short time?

Can we imagine what is missing that would enable one of the existing expert systems to quickly evolve vastly superhuman capabilities in its narrow area of expertise?

If we are completely clueless about how a seed AI with the potential of becoming superhuman intelligent could be possible, how could we possible update our probability estimates by looking at technological progress? There is a lot of technological progress, even in the field of AI, but it doesn't seem to tell us much about AI going FOOM.

There is another question in this context. What would increase (decrease) the probability of the S event? Clearly the absence of another Earth despite of the Kepler's observations - increases it. What about Watson success? The fact we have no 10 Ghz processors?

Which are those relevant facts or events, which do influence? Can we say in a year or two - yes, since Wolframalpha manages it's own code, a self optimizer is highly likely in the next 10 years?

I think, one cannot do those estimations without pondering these.

I'd like to suggest a way to organize our thinking about this, but it doesn't quite directly bear on your question. Your question is: how should our confidence in the singularity ever occurring change as time goes on? A related and easier question is: if we grant that a singularity is 100% likely to occur eventually, how much danger should we feel at different times in the future? I'm ready to drop some jargon about this easier question: I think we should be considering the relative probability

(1) P(still alive at time t + 1 | still alive at time t)

which is the ratio P(still alive at time t+1) / P(still alive at time t) by Bayes theorem. By taking the logarithm and considering a small unit of time, this is approximately

(2) exp(- pdf(t) / (1 - cdf(t)))

where pdf and cdf are the probability density function and cumulative density function of "when a singularity will occur." I have seen this expression (2) called the "failure rate" or "hazard rate" of the distribution. Anyway and in particular you can compute the distribution from the hazard rate.

For instance, you might think (1) is constant, meaning something like "an attack could come at any time, but we have no reason to expect one time over another." In that case you are dealing with an exponential distribution. Or you might think that (1) is low now but will asymptotically approach some constant as time goes on, in which case you might be dealing with a gamma distribution. These gamma distributions are interesting in that they have a peak or mode) in the future, which is what I at first thought you might be getting at by "inflection point."

I find it impossible to predict without knowing specifics about the future scenario. As we get closer to creating an AI, we are almost guaranteed to find out more difficulties associated with it.

Maybe in 10 years we will find some unforeseen problem that we have no idea how to resolve, in which case of course my probability estimate would significantly drop.

Or, if we have not seen any significant progress in the field, I predict my estimate would remain constant for the first 30 years, then decrease every year progress is not being made.

If there is a continuous stream of progress that doesn't also reveal huge new barriers, then I don't believe it would ever go down. But, I find it hard to imagine any scenario that presents continual progress, doesn't show any major roadblocks, yet still has not managed to develop AI more than 200 years form now.

That is not called an "inflection point". It is called a "maximum

There seem to be two separate questions, when we will have artificial intelligence that approximates human intelligence, and when an AI takeoff will occur. If we get the first and the second doesn't happen shortly thereafter then we should strongly reduce our estimates for the second happening at all. But the second cannot happen until the first has happened. Moreover, if the two are extremely tied up (that is they will likely occur very close to each other), then the only reason we haven't observed such an event already might be that such an event is likely to wipe out the species that triggers it, and there's just survivorship bias. (This seems to be the sort of anthropic reasoning that just makes me feel very confused so I'm not sure this is a reasonable observation.)

I would say that if we don't have human-like AI in the next fifty years, and there's no obvious temporary barrier preventing technological improvement (i.e. global collapse of civilization or at least enough bad stuff to prevent almost any research) then I'd start seriously thinking that people like Penrose have a point. Note that doesn't mean that there's anything like a soul (Penrose's ideas for example suggest that an appropriately designed quantum computer could mimic a conscious entity), although that idea might also need to be on the table. I don't consider any of those sorts of hypotheses at all likely right now, but I'd say 50 years is about where the good rationalist should recognize their confusion.