Bloggingheads: Yudkowsky and Aaronson talk about AI and Many-worlds

Upvoted, but it wasn't nearly as fascinating as I'd hoped, because it was all on our home turf. Eliezer reiterated familiar OB/LW arguments, Aaronson fought a rearguard action without saying anything game-changing. Supporting link for the first (and most interesting to me) disagreement: Aaronson's "The Singularity Is Far".

At one point in the dialog, Scott raises what I think is a valid objection to the "nine people in the basement" picture of FAI's development. He points out that it's not how science progresses, and so not how he expects this novel development to happen.

If we consider FAI as a mathematical problem that requires a substantial depth of understanding beyond what's already there to get right, any isolated effort becomes likely hopeless. Mathematical progress is a global effort. I can sorta expect a basement scenario if most of the required math happens to be already developed, and so the remaining challenge is to find the relevant math, assemble it in the right way, and see the answer. But that doesn't sound very likely.

Alternatively, a "team in the basement" could wait for the right breakthrough in the mainstream mathematics, and, being prepared, to apply it faster than anyone else to the problem. This seems more realistic, but may require the mainstream to know what to look for. Which involves playing with existential risk.

Well, that was interesting, if a little bland. I think the main problem was that Scott is the kind of guy who likes to find points of agreement more than points of disagreement, which works fine for everyday life, but not so well for this kind of debate.

By the way, I noticed that this was "sponsored" by the Templeton Foundation, which I and many other people who care about the truth find deeply repulsive.

I liked the discussion, especially the final part on the many world interpretation (MWI).

I had the impression that Eliezer had a better understanding of quantum mechanics (QM), however I found one of his remarks very misleading (and it also confused Scott rightly): Eliezer seemed to argue that MWI somehow resolves the difficulty of unifying QM with general relativity (GR) by resolving non-locality.

It is true that non-locality is resolved by Everett's interpretation, but the real problem with QM+GR is that the renormalization of the gravity wave function does not seem to work out mathematically. At least not in a straightforward manner. However MWI requires gravity to be quantized and therefore forced the physicists to come up with a more elaborate solution.

Anyways, I agree with Eliezer on the other arguments in favor of MWI (linearity, locality, objectivity, etc...), but think that making overreaching remarks rendered his position at least a bit suspect for no good reason.

To be fair: MWI has its own technical quirks (e.g. choice of basis, explanation of probabilities,...) but they don't seem to be as fundamental as those of the classical interpretation. However the discussion would have been more interesting if Scott could have brought up those points rather than the purely philosophical issues.

I picked up a copy of Jaynes off of ebay for a good price ($35.98). There are 2 copies left in that auction. Someone here might be interested:

http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=280380684353

No need to vote this comment up or down.

I note that the Born probabilities were claimed to have been derived from decision theory for the MWI in 2007 by Wallace and Deutsch:

“Probabilities used to be regarded as the biggest -problem for Everett, but ironically, they are now its most powerful success” - David Deutsch.

"In a September 2007 conference David Wallace reported on what is claimed to be a proof by Deutsch and himself of the Born Rule starting from Everettian assumptions. The status of these arguments remains highly controversial."

• http://en.wikipedia.org/wiki/Many-worlds_interpretation

Robot ant: http://www.youtube.com/watch?v=0jyBiECoS3Q

I would say real ants are currently waaay ahead of robot ant controllers.

On the other hand - like EY says - there's a whole bunch of things that we can do which ants can't. So it is not trivial to compare.

Thumbs up to Eliezer Yudkowsky for getting around to giving some actual timescales. They are incredibly vague timescales - but it is still a tricky thing to estimate the difficulty of - so that's OK, I guess.

This is gonna be so awesome, I just took a moment to try to make sure I'm not dreaming.

On the issue of many-world, I must just be slow because I can't see how it is "obviously" correct. It certainly seems both self consistent and consistent with observation, but I don't see how this in particular puts it so far ahead of other ways of understanding QM as to be the default view. If anyone knows of a really good summary for somebody who's actually studied physics on why MWI is so great (and sadly, Eliezer's posts here and on overcomingbias don't do it for me) I would greatly appreciate the pointer.

In particular, two things that I have a hard time wrapping my head around are: -If multiple worlds really are "splitting" from our own how is this accomplished without serious violations of mass and energy conservation. (I'm sure somebody has treated this somewhere since it's so basic, but I've never seen it.) -Even assuming everything else is fine, the actual mechanism for which world diverge has to be spelled out. (Maybe it is somewhere, if so please help me end my ignorance.)

I'll admit that I haven't actually spent a great deal of time considering the issue, but I've never come across answers to basic questions of this sort.

Dennett and Hofstadter have "extremely large" estimates of the time to intelligent machines as well. I expect such estimates will prove to be wrong - but it is true that we don't know much about the size of the target in the search space - or how rough that space is - so almost any estimate is defensible.

Time symmetry is probably not a big selling point of the classical formulation of the MWI. What with all those worlds in the future that don't exist in the past.

OK - no information is created or destroyed - so it's technically reversible - but that's not quite the same thing as temporal symmetry.

It would be better if it were formulated so there were lots of worlds in the past too. You don't lose anything that way - AFAICS.

The discussion got a bit sidetracked around about when EY asked something like:

If you are assuming that you can give the machine one value and have it stable, why assume that there are all these other values coming into it which you can't control.

...about 27 minutes in.

Scott said something about that being how humans work. That could be expanded on a bit:

In biology, it's hard to build values in explicitly, since the genes have limited control over the brain - since the brain is a big self-organising system. It's as though the genes can determine the initial developmental trajectory - but then there's the wind to deal with.

If machine intelligence turns out to work much like that, then we may have similar difficulties building in machine values. If we can find a way of getting the machines to absorb values from surrounding agents, then that might save a lot of trouble.

Humans get many of their values from surrounding humans - via human culture. Were it not for that we would be like our cannibal ancestors from 1MY ago. Conscience and guilt are some of the mechanisms used to absorb those values. Evolution built those in - rather than all the details of the values of human society. It would have been technically difficult to build those in - and the result would have been inflexible. Instead it built a learning machine - and allowed the details of the values of human society to be one of the things learned.

Machine intelligence is quite likely to work along those lines if it is built on a connectionist model - where the brain grows from a simple initial state. There, we can't easily wire in the details of particular values - since it is so hard to understand the details of what is going on. However, we can wire in some gross values - pain, suffering, irritation, etc. Guilt is basically a way of applying negative reinforcement to past actions. It's a fairly primitive value - the kind that it is easier to build in.

I'm not sure the halved doubling time for quantum computers is right.

Maybe I'm not getting into the spirit of accepting the proposed counterfactuals - but is quantum computer performance doubling regularly at all? It seems more as though it is jammed up against decoherence problems already.

Scott cites the Doomsday Argument in his "The Singularity Is Far":

http://scottaaronson.com/blog/?p=346

Surely that is a mistake. The Doomsday Argument may suggest that the days of humans like us may be numbered, but doesn't say much more than that - in particular it can't be used to argue against a long and rich future filled with angelic manifestations. So: it is poor evidence against a relatively near era of transcension.

Am I missing something here? EY and SA were discussing the advance of computer technology, the end of Moore's rule-of-thumb, quantum computing, BIg Blue, etc. It seems to me that AI is an epistemological problem not an issue of more computing power. Getting Big Blue to go down all the possible branches is not really intelligence at all. Don't we need a theory of knowledge first? I'm new here so this has probably already been discussed but what about freewill? How do AI researchers address that issue?

I'm with SA on the MWI of QM. I think EY is throwing the scientific baby out with the physics bath water. It seems to me that the MWI is committing the mind projection fallacy or the fallacy of the primacy of consciousness. I also agree with whoever said (paraphrased) that all these interpretations of QM just differ on where they hide the contradictions... they are all unsatisfactory and it will take a genius to figure it out.