[Link] John Baez interviews Eliezer

Interesting interview. Possibly a dumb question though, but... in the interview Eliezer said that as soon as they actually solved all the reflective decision theory stuff, they could code the thing.

But... don't they also have to actually solve all the tricky "exactly how to properly implement the CEV idea in the first place" thing? ie, even given they've got a way to provably maintain the utility function under self modification, they still have to give it the right utility function. (Or did I just misunderstand that interview answer?)

(Or has SIAI actually made really good progress on that to the point that the reflective decision theory side of things is the big roadblock now?)

The impression on the amount and level of early-age training a serious mathematical competence requires I get from that is somewhat terrifying. On the other hand, I do appreciate pointing out the difference between appreciating feats of maximally difficult virtuoso mathematics and using mathematics as a helpful framework for describing things in an exact and unambiguous way.

The next installment will presumably be next week. Does anyone know if this is correct? Also, how many weeks will this run for?

Gah . . . the comments on that interview. People are being wrong on the internet and it won't even help to correct them because the issues are too complicated and multiple responses will just make me look fanatical.

I think there must be more coming; it's short, ends abruptly, and opens with:

This week I’ll start an interview

I intend to plunge into the decision theory of self-modifying decision systems and never look back. (And finish the decision theory and implement it and run the AI, at which point, if all goes well, we Win.) (This Week’s Finds (Week 311))

...

After all, if you had the complete decision process, you could run it as an AI, and I'd be coding it up right now. (Eliezer_Yudkowsky 12 October 2009 06:19:28PM)

Can this be interpreted the way that Eliezer Yudkowsky believes that he himself, or the SIAI, will not only define friendliness but actually implement it and run a fooming AI to take over the universe? If they really believe that and if it is likely that they can succeed, I still think that even given a very low probability of them being dishonest one should seriously consider how it can be guaranteed that the AI they run is actually friendly. Let me ask you people who believe that the SIAI can succeed, are you not worried at all about unfriendly humans? You just trust their words? That's really weird. If I don't misunderstand what he is saying in those two quotes above, or if he isn't joking, he's actually saying that he'll run a fooming AI.