Meta comment on LW, as it relates to this post:

So when I checked this post in the morning, it had received a negative ten (up to that point in time) in “karma.”  When I hovered over the negative ten, I saw the message “3 votes,” so apparently three people strongly disapproved of (disagreed with?) this post. Five days ago, I received a negative eight in karma from two people. I asked for guidance and input, but none has been forthcoming (at least in the five days since then). 

I don’t mind the “negative karma” votes, in and of themselves, but it seems like such votes would be more useful if they were accompanied by explanations. Otherwise if it’s a matter of, for example, faulty logic or faulty givens, there’s no way to know this, let alone improve. I also don’t know if it’s the same people strongly disapproving from one post to the next, or different people, each with a unique reason for disapproval. 

As far as my point in this particular post (that I hope Eliezer will sit for another interview w/ Lex to cover more topics within the scope of AI safety, or at least address them in another venue like LW), I listened to the last few minutes of the interview again, and Lex said “Thank you for the fight you’re fighting. Thank you for being fearless and bold and for everything you do. I hope we get a chance to talk again and I hope you never give up.”

Then Eliezer said “You’re welcome. I do worry I didn’t address a whole lot of fundamental questions I expect people have, but you know maybe we got a little bit further and made a tiny little bit of progress and I’d say like be satisfied with that, but actually no, I think one should only be satisfied with solving the entire problem.” 

Then Lex said “To be continued....” 

In light of what Eliezer said, I wonder whether he would also “strongly disapprove” of the hope for more discussion that I expressed in my post. I also wonder whether this is the way Eliezer imagined karma would be used on LW. 

--

A side note is that millions of people listen to Lex's podcasts, so if we really want to get the most brilliant young people working on interpretability, and AI alignment problems in general, then shouldn't we want Lex to spend more time discussing these problems? Do the negative karma votes on this post mean people don't want more public discussion of AI safety?

I agree very much with your post. My perception from listening to the conversation was that none of the analogies were accurate and it seems they were both struggling a bit to understand each other, that's why a follow-up interview probably makes sense. 

Yet one way in which AI systems are more like humans than aliens, that wasn’t covered, is that humans create and teach AI systems, in the way that human parents create and teach human children. A hopeful note to this aspect of the analogy is that most human children grow up to be smarter than their elderly parents, but the children usually protect, rather than kill, their parents

I don't think that's relevant. Humans have the aforementioned brain circuitry that predisposes us to socialisation. I feel like the NN equivalent would be something designed to be more structured, with designated functional areas, than the complete blank slates we use today.

That said, I don't want the AI to stick me into a retirement home, completely deprived of agency, so that I can't bother it while it's running the world, either. That's something even human sons and daughters do, and the AI equivalent would be The Matrix.