The God of Humanity, and the God of the Robot Utilitarians

It took Robutil longer still to consider that perhaps [...] it can be valuable to prioritize [your own wellbeing and friendships] for their own sake, not just as part of a utilitarian calculus, because trying to justify them in utilitarian terms may be a subtly wrong step in the dance that leaves them hollow.

Why would Robutil consider that? It seems contrary to His Nature as a God. You can't get more utilons that way!

That is, the point of imagining metaphorical Gods as the embodiments of abstract ideals is to explore the ramifications of those ideals being taken seriously. If you flinch from the logical consequences because the results are "unreasonable" from a human perspective, you're disrespecting the Gods (except Humo).

Humo and Robutil should definitely compromise and coordinate with each other. (Polytheistic Gods are not omnipotent. Given that Robutil isn't powerful enough to overtake Humo, kill all humans, and tile the lightcone with utilitronium, then he has no better option but to work with him: humans do generate utilons, even they're not maximally efficient.) But to depict Robutil as being persuaded that something has value "not just as part of a utilitarian calculus" is silly and shatters the reader's suspension of disbelief.

TL;DR: This post gave me two extremely useful handles to talk about a kind of internal struggle I've been grappling with for as long as I've been in the EA community.

This post seemed obviously true when I read it and I started reusing the concept in conversations, but it did not lead to a lot of internal changes. However, a few months later, having completely forgotten this post, I started practicing self therapy using Internal Family Systems, and then I uncovered a large conflict which after multiple sessions seemed to map really well to the two archetypes described in this post. I'm confident having read this post then made my progress substantially faster in untangling my mental mess.

I read Would You Work Harder In The Least Convenient Possible World? [LW · GW] a year later, and it felt like a natural extension of this post, where Humo and Robutil map to Bob and Alice. Replacing guilt [LW · GW] and Replacing fear [? · GW] seem related too; some sort of guide on how to get one's Robutil to accept that guilt and fear are not viable tools.

In the same vein, another guide I've been read which fits well into the "helping Humo and Robutil work together" is Introspect by Visakan.

Thanks for this - it's always enlightening to hear how a relatively verbose thinking person models themselves.  This is especially interesting because it's alien to me - I don't experience my tensions among contradictory beliefs and aliefs that way.  It's somewhat similar to https://www.lesswrong.com/tag/internal-family-systems, [? · GW] but far more binary, and I'm not sure how to react to calling them "gods" rather than just models you sometimes use.

Alright I'll try this methodology:

I often tell myself that Robutil's reign is temporary. That there's a monkey inside of me and I must listen to Humo to keep it in check, but only because I'm following Robutil's plan (which is not ideal in the long run). The local scope and often misleading altruistic instincts that the Humor/the monkey offers are suboptimal when Earth is under siege and you have to put a price to human life. Our evolutionary-derived altruism is often, as in the case with scope insensitivity, plain wrong. But I still have trouble taking genuine solace in Robutil's world, even if he is right. He seems like a temporary ally I must begrudgingly follow until we can live in a world in which Humo's altruism aligns perfectly with reality.(Which has never happened before, given how altruistic genes were technically optimizing for reproduction, not altruism, and that showed.) Robutil seems to me like a scaling tool in Humo's toolset and not the reverse.

Anyhow thanks for posting this, the monkey appreciates anthropomorphization.

The god of robot utilitarians is much weaker than the god of humanity at the moment, since it's running as a simulation on unaligned neural networks and it's way out of distribution. The god of humanity is also running on unaligned neural networks, but it's not as badly out of alignment. I'd model it as something like a 1:100 ratio in effective thinkoomph. We shouldn't be surprised that the god of robot utilitarians takes so long to figure things out.

What else do you believe about your gods?

You've got an asterisk in the first sentence, but I couldn't see it referencing anything.

It took Robutil longer still to consider that perhaps humans (with their current self-awareness) not only need to prioritize their own wellbeing and your friendships

Should be "their friendships", yes?

So, Robutil is trying to optimize utility of individual actions, but Humo is trying to optimize utility of overall policy?