My hour-long interview with Yudkowsky on "Becoming a Rationalist"

Mostly stuff that's familiar if you've read enough of Eliezer's articles, but this bit was interesting:

LUKE: Well Eliezer one last question. I know you have been talking about writing a book for quite a while and a lot of people will be curious to know how that’s coming along.

ELIEZER: So, I am about to finished with the first draft. The book seems to have split into two books. One is called How to Actually Change Your Mind and it is about all the biases that stop us from changing our minds. And all these little mental skills that we invent in ourselves to prevent ourselves from changing our minds and the counter skills that you need in order to defeat this self-defeating tendency and manage to actually change your mind.

It may not sound like an important problem, but if you consider that people who win Nobel prizes typically do so for managing to change their minds only once, and many of them go on to be negatively famous for being unable to change their minds again, you can see that the vision of people being able to change their minds on a routine basis like once a week or something, is actually the terrifying Utopian vision that I am sure this book will not actually bring to pass. But, it may none the less manage to decrease some of the sand in the gears of thought.

Does a transcript exist?

I don't think you need to apologize for the self-promotion. I was happy to hear it, and glad to be told about it.

Yudkowsky gives the value of boredom as an example - as he did in value is fragile.

The problem here is that boredom is bound to be one of the basic AI drives. It will emerge as an instrumental value in practically any goal-directed intelligent system - since getting stuck doing the same thing repeatedly is a common bug that prevents agents from attaining their goals. If you have to do science, make technology, build spaceships, etc - then it is important to explore, and not to the same thing all the time - and goal-directed agents will realise that.

Of course, some things never get boring - even for humans. Food, sex, etc - but for the other things, you don't need to build boredom in as a basic value, a wide class of agents will get it automatically.

So, for example, a gold mining agent won't get bored of digging holes, but they will get bored doing unprofitable scientific research. Boredom is just nature's way of telling you that you have exhausted the local opportunities - and really should be doing something else.

Anyway, we don't have to worry too much about missing out things that will arise naturally through ordinary utility maximisation in a wide range of goal-directed agents. The idea that the universe will go up the tubes unless humans reverse-engineer boredom and manually program it into their machines is not correct. A wide class of machines that simply maximise utility will get boredom automatically - along with a desire for atoms, energy and space - unless we explicitly tell them not to assign instrumental value to those things.