Self-Integrity and the Drowning Child

I think the key to the drowning child parable is the ability of others to judge you. I can't judge you for not donating a huge portion of your income to charity, because then you'll bring up the fact that I don't donate a huge portion of my own income to charity. Sure, there are people who do donate that much, but they are few enough that it is still socially safe to not donate. But I can judge you for not saving the child, because you can't challenge me for not saving them - I was not there. This means that not saving the child poses a risk to your social status, which can greatly tilt the utility balance in favor of saving them.

But how exactly do you do this without hammering down on the part that hammers down on parts? Because the part that hammers down on parts really has a lot to offer, too, especially when it notices that one part is way out of control and hogging the microphone, or when it sees that one part is operating outside of the domain in which its wisdom is applicable.

(Your last paragraph seems to read "and now, dear audience, please see that the REAL problem is such-and-such a part, namely the part that hammers down on parts, and you may now proceed to hammer down on this part at will!")

I agree with and appreciate the broad point. I'll pick on one detail because I think it matters.

this whole parable of the drowning child, was set to crush down the selfish part of you, to make it look like you would be invalid and shameful and harmful-to-others if the selfish part of you won [...]

It is a parable calculated to set at odds two pieces of yourself... arranging for one of them to hammer down the other in a way that would leave it feeling small and injured and unable to speak in its own defense.

This seems uncharitable? Singer's thought experiment may have had the above effects, but my impression's been that it was calculated largely to help people recognize our impartially altruistic parts—parts of us that in practice seem to get hammered down, obliterated, and forgotten far more often than our self-focused parts (consider e.g. how many people do approximately nothing for strangers vs. how many people do approximately nothing for themselves).

So part of me worries that "the drowning child thought experiment is a calculated assault on your personal integrity!" is not just mistaken but yet another hammer by which people will kick down their own altruistic parts—the parts of us that protect those who are small and injured and unable to speak in their own defense.

At a pond, my niece was in a child floaty, reached too far and flipped over into the water. I slammed my half-eaten sandwich on my brother's chest, hoping he would grab it and ran into the water and saved her. 

She was fine and I got to finish my sandwich.

Regarding the direct example 

I feel like it's self-subverting. There's an old canard about https://www.watersafetymagazine.com/drowning-doesnt-look-like-drowning/ Given how staggeringly disproportionate the utility losses are in this scenario I think even a 1% chance of my assumption that 'I have 15 seconds to undress' would lead to death means I should act immediately.

In general when thinking about superfast reflex decisions vs thought out decisions: Obey the reflex unless your ability to estimate the probabilities involved has really low margins of error. My gut says X but my slow, super weak priors-that-have-never-been-adjusted-by-real-world-experience-about-this-first-time-in-my-life-situation say Y... Yeah just go with X. Reflect on the outcome later and maybe come up with a Z that should have been the gut/reflex response.

There's an old video game Starcraft 2 advice from Day9 that's surprisingly applicable in life: Plan your game before the game, in game follow the plan even if it seems like it's failing, after the game review and adjust your plan. Never plan during the game, speed is of the essence and the loss of micro and macro speed will cost you more than a bad plan executed well.

Don't plan during a crisis moment where you have seconds to react correctly. Do. Then later on train yourself to have better reflexes. Applicable when socializing, doing anything physical, in week 1 of a software development 2 week sprint etc.

Regarding the more general point of people having ... self-consistent utility functions/preferences

I fundamentally disagree that you shouldn't criticize someone for their utility function. An individual's utility function should include reasonably low-discount approximations of the utility functions of people around them. This is what morality tries to approximate. People that seem to not integrate my preferences into their own signal danger to me. How irrelevant is my welfare in their calculations? How much of my utility would they destroy for how small a gain in their own utility?

People strongly committed to non-violence and so on are an edge case, but I'd feel much more comfortable with someone not in control over their own utility function than someone that is in control, based on the people I have encountered in life so far.

How intrusively should people integrate each other's preferences? How much should we police other individual's exchange rate from personal utils to other people utils? No good answer, it varies over time and societies.

Society is an iron maiden, shaped around the general opinion about what the right action is in a given scenario. Shame is when we decide something that we know others will judge us badly for. Guilt is when we've internalized that shame.

The art of a good society is designing an iron maiden that most people don't even notice. 

It seems irrational to me to not internalize the social moral code to some extent into my individual utility function. (It happens anyway, might as well do it consciously so I can at least reject some of the rules) If the social order is not to my taste, try and leave or change it. But just ignoring it makes no sense. 

I'd also argue that the vast majority of preferences in our, so-called, 'personal' utility function are just bits and bobs picked up from the societal example palette we observed as we grew up.

People's utility functions also include components for the type of iron maiden they want their society to build around other members. I want to be able to make assumptions about the likely outcomes of meeting a random other person. Will they try to rob me? If I'm in trouble will they help me? If my kid is playing outside unsupervised by me, but there's always random people walking by, can I trust that any of them will take reasonably care of the child if the kid ends up in trouble? 

I strongly do not want to live in a society that doesn't match my preferred answers on those and other critical questions. 

I absolutely do not want to live in a society that has no iron maiden built at all. That is just mad max world. I can make no reasonable assumption about what might happen when I cross paths with another person. When people are faced with situations of moral anarchy, they spontaneously band together, bang out some rules and carve out an area of the wilderness where they enforce their rules.

Humans don't swim very well wearing lots of clothing. Take off your suit before going into the water.

Trying to summarize for those of us not fond of long-winded parables.

• A single moral agent is a bad model of a human,
• Multiple agents with individual utilities/rules/virtues is a more accurate model.
• It's useful to be aware of this, because Tarski [? · GW], or else your actions don't match your expectations.

I unfortunately have very little of substance to add, but a strong upvote was not quite enough.

There is something in here of Iron Hufflepuff, and I'm exceedingly grateful to Eliezer for dignifying and validating it so unequivocally in this meta-parable.  I expect to link to this fairly frequently over the next decade.

An interesting difference between the drowning child situation and the "could donate to effective charity to save children's lives" situation is that the person who happens to be walking by that pond has a non-transferable opportunity to save a child's life for $500 (or whatever the cost of the clothes are, plus some time cost, and the inconvenience of getting wet and muddy). In the case of effective charity, even if one declines to donate, other people will still have the same opportunity. In the case of the drowning child, the fact that you are the only one who can act makes jumping in to save the child somehow seem more urgent. If you don't save the child, then you'd be somehow "wasting" a valuable opportunity. For a mostly selfish person who values all lives other than their own at less than $500, the opportunity would be valuable to others but useless for themselves.

If the going rate is $1000 to save a child through effective charity, then a mostly altruistic person would be willing to pay a mostly selfish person $600 to compensate for the costs of their clothes. There would have to be some "honour" involved, since the selfish person couldn't exactly unsave the child after the fact. If they could make the deal work anyway, then the mostly selfish person would have succeeded in selling non-transferable opportunity for $100, and it would be worthwhile for them to save the drowning child.

My objection lies in the second part of the drowning child parable. The part where someone geographically distant is considered identical to the child in front of me, and money is considered identical to the actions of saving. It's some sort of physics being the same everywhere intuition being inappropriately applied. Of course distance in time, space, or inference create uncertainty. Of course uncertainty reduces expected value and possibly even brings the sign of the action into question if the expected variance is high enough.

Curated. To generalize, as the stakes continue to seem high ("most important century"-level high), it's easy to feel an immense obligation to act and to give it all up for the sake of the future. This meta-parable reminds us that humans aren't made solely of parts that give everything up, and that it's a matter of self-integrity to not do so.

not selfish and unselfish components in their utility function, but parts of themselves in some less Law-aspiring way than that

Utility functions don't model all agents; we should look at a larger space. I expect it to better model not just a human but also a council of humans or a multiverse of acausal traders. I expect this also to say how an AGI should handle uncertainty about preferences.

There should be a natural way to aggregate a distribution of agents into an agent, obeying the obvious law that an arbitrarily deeply nested distribution comes out the same way no matter which order you aggregate its layers in.

The free generalization, of course, is an agent as a distribution of utility functions. The aggregate is then simply a flattening of nested distributions. This does not tell us how an agent makes decisions - we don't just take an expectation, because that's not invariant under u and 2u being equivalent utility functions.

We might need to replace the usual distribution monad with another.

Something with a utility function, if it values an apple 1% more than an orange, if offered a million apple-or-orange choices, will choose a million apples and zero oranges.  The division within most people into selfish and unselfish components is not like that, you cannot feed it all with unselfish choices whatever the ratio.  Not unless you are a Keeper, maybe, who has made yourself sharper and more coherent; or maybe not even then, who knows?

I fear that this parable encourages a view whereby the utility function "should" factorize over intuitively obvious discrete quantities [LW(p) · GW(p)] (e.g. apples and oranges). My utility function can value having a mixture of both apples and oranges. 

TLDR: if we model a human as a collection of sub-agents rather than single agent, how do we make normative claims about which sub-agents should or shouldn't hammer down others? There's no over-arching set of goals to evaluate against, and each sub-agent always wants to hammer down all the others.

If I'm interpreting things right, I think I agree with the descriptive claims here, but tentatively disagree with the normative ones. I agree that modeling humans as single agents is inaccurate, and a multi-agent model of some sort is better. I also agree that the Drowning Child parable emphasizes the conflict between two sub-agents, although I'm not sure it sets up one side against the other too strongly (I know some people for whom the Drowning Child conflict hammers down altruism).

What I have trouble with is thinking about how a multi-agent human "should" try to alter the weights of their sub-agents, or influence this "hammering" process. We can't really ask the sub-agents for their opinion, since they're always all in conflict with all the others, to varying degrees. If some event (like exposure to a thought experiment) forces a conflict between sub-agents to rise to confrontation, and one side or the other ends up winning out, that doesn't have any intuitive normative consequences to me. In fact, it's not clear to me how it could have normativity to it at all, since there's no over-arching set of goals for it to be evaluated against.

This post felt like a great counterpoint to the drowning child thought experiment, and as such I found it a useful insight. A reminder that it's okay to take care of yourself is important, especially in these times and in a community of people dedicated to things like EA and the Alignment Problem. 

In case anyone else, like me, followed the link near the start of OP to the story from which this is excerpted, and is wondering whether (having had a bunch of updates on 2021-10-24 but none since) it's likely to be dead: I had a look at its pattern of updates and it's very bursty, with gaps on the order of 1-3 weeks between bursts, so the current ~1w of no updates is not strong evidence against there being more future updates to come.

(It seems like the majority of fictional works posted online end up being abandoned, and I generally prefer not to read things of non-negligible length that stop in mid

The Watcher spoke on, then, about how most people have selfish and unselfish parts—not selfish and unselfish components in their utility function, but parts of themselves in some less Law-aspiring way than that.

I guess it's appropriate that children there learn about utility functions before learning about multiplication.

This applies to integrity of a false persona just as well, a separate issue from fitting an agentic persona (that gets decision making privileges, but not self-ratification privileges) to a human. Deciding quite clearly who you are doesn't seem possible without a million years of reflection and radical cognitive enhancement. The other option is rewriting who you are, begging the question, a more serious failure of integrity (of a different kind) whose salience distracts from the point of the dath ilani lesson.

I prefer hypocrisy to cruelty. 

More gennerally I think this just misses the point of drowning child.  The argument is not that you have this set of preferences and therefore you save the child, the argument is that luxury items are not of equal moral worth to the life of a child.  This can be made consistent with taking off your suit first if think the delay has a sufficiently small probability of leading to the death of child and you think the death of a child and the expensive suit are comparable. 

In dath ilan, it is virtuous to write more stories about dath ilan.

I reject the parable/dilemma for another reason: in the majority of cases, I don't think it's ethical to spend so much money on a suit that you would legitimately hesitate to save a drowning child if it put the suit at risk?

If you're so rich that you can buy tailor-made suits, then sure, go save the child and buy another suit. If you're not... then why are you buying super-expensive tailor-made suits? I see extremely few situations where keeping the ability to play status games slightly better would be worth more than saving a child's life.

(And yes, there's arguments about "near vs far" and how you could spend your money saving children in poor countries instead, but other commenters have already pointed out why one might still value a nearby child more. Also, even under that lens, the framework "don't spend more money than you can afford on status signals" still holds.)

Coming late to this, and having to skim because Real Life, but I would modify:

Keltham is being coherent, said the Watcher.

Keltham's decision is a valid one, given his own utility function (said the Watcher); you were wrong to try to talk him into thinking that he was making an objective error.

However, the Watcher said, Keltham's utility function is also awful and Keltham should be shunned for it by any being with decent ethics.

I really don't care how valid your utility function is, or how rational you think you are, if it turns you into the sort of person who has to weigh the possibility of a child dying against materialistic concerns. In that case, you've sacrificed your soul for the sake of optimising something worthless.