Comment by gaspardbos on We learn long-lasting strategies to protect ourselves from danger and rejection · 2023-09-28T09:53:43.384Z · LW · GW

Dear Richard,

I stumbled upon this particular post in my initial explorations of lesswrong and researching what the best knowledge is that the community has been able to come to on the topic of "drives, intentions and pursuit of goals" as related to the non-human agency of risky AI.

Thanks for creating these sequences on fear which looks to be a well-thought-out thesis with an interesting proposal for a fear-reducing strategy. The reason I'm making this my first comment on the forum is because the topic also resonates with my personal experience, as I see it has done for others in the comments section. Therapy has helped me touch on some childhood memories and has been productive in reshaping some of my thinking and behavior for the better.

And I also think it is necessary to understand where our own goals come from if we want to align them with AI.

I think what could improve your writing and reasoning in this post, although you might balance it out with the other posts that I have not read yet, is to distinguish a bit more the doubt you have about the instrumental value of the strategy and the ontology of the fear. I think you can rightly posit that behaviors emerge out of the need for the child to adapt and you could reference (even) more sources that point to this.  Also, therapy that takes the person back to these memories to transform them is evidenced to work. 

What I am missing is an investigation into how the fear, undesirable in situations that might cognitively or intelligently be observed as non-threatening, is part of the story someone tells themselves about themselves; their self-perceived identity. Like you say, any giving goal could have at its base any give negative experience, although some might be more easily correlated through observance of their features (fear of intimacy because of abandonment or abuse, etc.), any story could be thought of by the person that makes them cope at the risk of expending rationality.

So while you do touch on these things, and probably in other posts as well, I think that by revisiting your writing and making up your mind about some things you could take out the caveats and make the post a bit more authoritative, especially from the paragraph onwards that starts with "I want to flag...".

Looking forward to your response.