John Baez Interviews with Eliezer (Parts 2 and 3)

Baez replies with "Ably argued!" and presumably returns to his daily pursuits.

Please don't assume that this interview with Yudkowsky, or indeed any of the interviews I'm carrying out on This Week's Finds, are having no effect on my activity. Last summer I decided to quit my "daily pursuits" (n-category theory, quantum gravity and the like), and I began interviewing people to help figure out my new life. I interviewed Yudkowsky last fall, and it helped me decide that I should not be doing "environmentalism" in the customary sense. It only makes sense for me to be doing something new and different. But since I've spent 40 years getting good at math, it should involve math.

If you look at what I've been doing since then, you'll see that it's new and different, and it involves math. If you're curious about what it actually is, or whether it's something worth doing, please ask over at week313. I'm actually a bit disappointed at how little discussion of these issues has occurred so far in that conversation.

This made me laugh aloud. Two points here

(a) Baez may take what Eliezer has to say into consideration despite not updating his beliefs immediately.

(b) Baez has written elsewhere that he's risk averse with respect to charity. Thus, he may reject expected value theory based utilitarianism.

For PR purposes "AI or environmentalism" strikes me as being just about the worst framing possible. I wish it could have been avoided somehow.

A much more helpful framing is to see unfriendly AI as pollution with brains.

Baez replies with "Ably argued!" and presumably returns to his daily pursuits.

One of the best comments ever :-)

I have been asking a lot of well-known and influential people lately about their opinion regarding risks from AI and the best possible way to benefit humanity by contributing money to a charity. Most of them basically take the same stance as John Baez, they accept the arguments but continue to ignore them. Sadly only a few gave their permission to publish their answers. And some people, for example Tyler Cowen, just refered to books they have written (e.g. 'Discover Your Inner Economist') that I haven't read so I am not sure about their opinion at all. The only detectable pattern was that quite a few mentioned the same charity, Médecins sans frontières. People as diverse as John Baez, Douglas Hofstadter and Greg Egan said that Médecins sans frontières would be a good choice. Here for example is the reply from Greg Egan who is one of the few people that gave their permission to publish their answer:

I asked:

What would you do with $100,000 if it were given to you on the condition that you donate it to a charity of your choice?

Greg Egan replied:

I would spend a week or two investigating a range of research programs in tropical medicine, to see if there was a project that might benefit significantly from a donation of that size. If I could not identify a clear opportunity for an impact there, I would donate the money to Médecins Sans Frontières, a charity I currently support to a much smaller degree, as I am reasonably confident that the money would be well spent by them.

I do not believe there is an "existential threat" from AI, and while I don't doubt that AI research will lead to some benefits to humanity in the long run, I expect that the timing and degree of those benefits would be negligibly affected by an investment of $100,000.

Wow. For some reason, I found these interviews far more convincing than EY's previous writings on these topics.

I thought exactly the same! After reading all of the interviews I am actually much more convinced that he is right about risks from AI.

What do you (and 'everyone else familiar with computers') mean by determinism?

The hardware and the software. Think of a provably correct compiler.

The main relevant paragraph in this interview is the one in part 2 whose first sentence is "The catastrophic sort of error, the sort you can’t recover from, is an error in modifying your own source code."

From the context, I think what EY means is that the AI must be structured so that all changes to source code can be proved safe-to-the-goal-system before being implemented.

On the other hand, I'm not sure why EY calls that "deterministic" rather than using another adjective.

Does the worse than random essay help to explain?