Podcast with Divia Eden and Ronny Fernandez on the strong orthogonality thesis

post by DanielFilan · 2023-04-28T01:30:45.681Z · LW · GW · 1 comments

This is a link post for https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5saWJzeW4uY29tLzQzODA4MS9yc3M/episode/YzFlYTRlMjItMTMyOS00ZTU5LWI4YTEtYmQ1N2FlYmEyYzg3

On my side podcast, "The Filan Cabinet", I invited Ronny Fernandez and Divia Eden to talk about the strong orthogonality thesis, and whether it's true. Seems like people here might also be interested. Podcast description below, and you can listen here.

In this episode, Divia Eden and Ronny Fernandez talk about the (strong) orthogonality thesis - that arbitrarily smart intelligences can be paired with arbitrary goals, without additional complication beyond that of specifying the goal - with light prompting from me. Topics they touch on include:


Comments sorted by top scores.

comment by Adrien Sicart · 2023-05-05T18:53:51.498Z · LW(p) · GW(p)

My first post should be validated soon, and is a proof that the strong form does not hold: in some games some terminal alignment perform less than non-terminal equivalent alignment.

An hypothesis is that most goals, if they become “terminal” (“in itself”, impervious to change), prevent evolution, and mutualistic relationships with other agents.