Lies Told To Children

I liked this, but I observe that I don't think it would have gotten nearly as much karma if it had been written by someone without a reputation already.

...which suggests that we're maybe doing something wrong?

Or is this in fact more informative, because we know that it came from Eliezer?

I often wonder what effect the idea of Santa Claus has on children's cognitive development. We have created a vast conspiracy to deceive young children, spanning every possible layer of authority: your parents are in on it, your teachers are in on it, mass media and corporations are in on it, even the government is in on it. All working together to support the idea that a man from the North Pole is going to fly around the world and bring you presents.

As a child gets older, maybe pieces of the story start to not make sense. The child expresses a little doubt, but is quickly reassured. The child may think, what's more likely: that my doubts are misplaced, or that everyone in the world I've been taught to trust is wrong or lying to me?

Then finally the child's ability to suspend reason is pushed to its breaking point. Yes, everyone else is really is wrong. My own comprehension of the world is a more reliable source of truth than what I've been instructed to believe. The child finally confronts their parents. "You're right," the child is told. "You've passed the test. You're on the inside now. Don't tell your little sister."

Don't tag "dath ilan", please; tags are shown at top and that's a humongous spoiler.

If there is actually such a thing as Security, the best thing to do is whatever you think will keep you (or other people) alive. Mouthing off to a security officer about your actual reason for saving the redhead is behavior that is suboptimal because it would lead to your death. So testing kids to "see if they'll reject conformity" is actually testing kids to see if they'll pointlessly risk their life.

Furthermore, the belief that every sapient being is precious is something that the child was told. In other words, the child cannot claim "well, nobody reasonably argued for the exception" because nobody argued for the rule either!

And I don't see why the child doesn't say "you lied to me about Security. How do I know you're not lying to me about (insert every random thing about the society that the kid might find slightly questionable)." For that matter, the kid could think that "there's no security" is itself a lie and that the whole thing is a Hundred Flowers Campaign--the actual purpose is not to test for compassion, it's to detect troublemakers by giving them a few weeks to fully reveal themselves. It's your world, which isn't actually a Hundred Flowers Campaign, but how does the kid know, now that he's been lied to at least once?

Quite a good story. But I think at this point I would quite like Eliezer to make some sort of statement about to what degree he endorses Dath Ilan, ethically speaking.  As a fictional setting it's a great machine for fleshing out thought experiments, of course, but it seems downright dystopian in many ways. 

(I mean, the fact that they're cryopreserving everyone and have AGI under control means they're morally "preferable" to Earth, but that's sort of a cheat. For example, you could design an alt. history where the world is ruled by a victorious Third Reich, but where Hitler got super into cryopreservation in his old age, and poured a lot of resources and authority into getting the populations under his control to accept it too. Probably in the long run that world is "preferable" to the world where the Allies win but billions more brains rot — but it's still not much of a utopia in any useful sense.)

Of course, Eliezer previously stuck the non-consensual-sex thing in Three Worlds Collide, as an attempt to simulate the “future societies will likely trivialize things we consider unthinkable and there's no way to tell what” effect. I suspect — hope? — some of the ickier parts of Dath Ilan are in the same boat. 

However, there are also many elements of Dath Ilan that are obviously supposed to come across as straightforwardly aspirational. And I would rather like some clarity on what bits are meant to ring how, both to get a fuller experience out of reading the existing Dath Ilan stuff and any further entries, and to improve my model of Eliezer's utility function. 

…Oh, by the way, I'm posting this here, because I hadn't previously realized Eliezer was going to keep returning to Dath Ilan, but the central premise of this one isn't really what sticks out to me the most, "are we sure we want a world like this?"-wise. Mostly it was some of the stuff in the original Q&A. 

Predictions over the course of my reading:

Guessed "dath ilan" from title.

Switched prediction to "Amenta" at mention of red hair.

Switched back to "dath ilan" (with much higher certainty) at the first use of capital-C "Civilization".

I said, at the end, was that I'd better be getting paid for this, and they all laughed and said of course I was, lots of money, at least as much as my parents were getting, because children are sapient beings too.

This seems like a rather hypocritical thing to say, unless dath ilan had some clever idea for how to implement this compensation that I'm failing to see right now.

If I was a subject in this experiment, there would be no amount of money you could pay me to retroactively agree that this was a fair deal. There's just nothing money can buy that would be worth the years of deception and the hours of mortal terror. 

If it was earth it'd be different, because earth has absolutely dire problems that can be solved by money, and given enough millions, that'd take precedence over my own mental wellbeing. But absent such moral obligations, it's just not worth it for me.

So do parents surreptitiously ask their children what sum of money they'd demand as compensation for participating in a wide variety of hypothetical experiments, some real, some fake, years before they move to a town like this? Seems rather impractical and questionable, considering how young the children would be when they made their choice.

What were the lives of the red-headed children like? Were they in the know about the nature of the experiment? I suppose they must have been, but then how could such small kids be such good actors? I'm doubtful about the power of stories versus someone's experience, but I liked the story :) 

I say this because I think it relates to / could be within your same world:

One of my favorite current story ideas is to write a story where there's a girl who lives in a world where each society is running an experiment about the best way to live life. In school, she finds out that while most societies are aware of the experiment that's being run, her society is one of the few where the experiment is kept secret from the people in it.

As she grows up, she realizes how poor her society is, and how little technology they have. It improves very quickly over the years, but when she was younger they didn't even have washing machines or dishwashers.

So, she starts digging to try to figure out why it is that all their technology was so bad, and why her society started off so poor, and begins advocating for the idea that whatever experiment is being run, it should be shut down, because their quality of life is so much lesser than other societies.

Eventually, the movement she starts begins gaining momentum, and the entire society begins to revolt against being in the experiment. When she's finally able to bring her case to the main governing decision body (the council), she finds out what the experiment was:

Despite their wealth, many of the people in the other societies are depressed and feel like they have very little to live for. Is it better to live wealthy and securely without worries, or to struggle but have a purpose of overcoming those challenges?

The later chapters cover the (now older) woman fighting with her emotions. She is proud that she was able to fight for her society to raise its standards, and she feels like she's lived a good life. However, not having the wealth at first was really tough, and she's worried the council will make more societies like hers and subject more people to those tribulations she experienced when she was younger. But, because she and others in her society ended up greatly enjoying their lives despite (and because of, as the experiment showed) the hardships, the experiment was considered a success. If she likes her life overall, should she really oppose the creation of more societies like hers?

I don't understand one thing about the universe. The test and the fact that this story wind "lies-told-to-children" storytelling night, seems to indicate that most children never hear those dirty-red lies. However, if that causes those children to be raised, to learn, differently than most of the planet, this only mean that the test checks whether those children are actually good, not whether the whole planet education system raises good children. This is especially important as I can imagine and fear that seeing this discrimination for years, then having to act yourselves, has an extremely strong impact on the later behavior of the person. Hopefully a good impact, because they have lived how conformity has been able to impact them for years and how it required an external event for them to do any action helping discriminated people. It's hard for me to imagine that civilization can be sure the remaining of the society is as good without having them actually tested in this realistic way.

I've a regret that the requirement to write down thoughts only came once. Clearly, there was no way I ever could see where it goes at the time it was requested. However, later, as soon as Red was mentioned, I guessed the end, the tests. Still cried reading it. I suspect that if I didn't know who the author was, I would not have guessed, that seems to unusual for most author authors I love

There wasn't actually any such thing as Security, and if there ever was it would mean that it was time to overthrow the government immediately.

I held back tears at this part.

I think that the idea of dath ilan being better at solving racism than earth social media is really valuable (in basically every different way that dath ilan stories are valuable, which is a wide variety of extremely different reasons). It should be covered again, at projectlawful at least, but this is a huge deal, writing more of it can achieve a wide variety of goals, and it definitely isn't something we should sleep on or let die here.

This made me tear up.

I feel like Conservative Judaism does this pretty well.

In Reform Judaism, everyone is in on the joke. Which means there's no obstacle to overcome, nothing to learn. Reform Jews win no points on Lies-Told-To-Children storytelling night.

Orthodox Jews play the game too seriously. Like, you get a bunch of people who win the game, and they tend to be interesting people who honed an important skill, but far too many don't get it during their lifetimes. Which means that they have these awful ideas like "we are God's chosen people" and "the world exists so we can read ancient legal texts". And that's a crying shame.

Conservative Jews (at least some of them) are told lies, but they're expected to eventually see through the lies, and when they do, nobody disowns them, and they come out better people. That's an important formative experience, and if Conservative Judaism is teaching anything toxic, it's not too hard to remove. They get some points on Lies-Told-To-Children storytelling night.

Oh, and Happy Passover.

The child does not even ask after the red headed kids? If the red-headed kids are real, sentient beings, this experiment is going to damage near everyone involved; we know the social problems and trauma caused by the Stanford prison experiment, and that was consensual, informed and contained.

The experiment also fundamentally underestimates what it takes to resist indoctrination. If your government systematically oppresses a group, and wants you to go along to it, they will not teach you universal rights spanning across these groups and so set you up to be for the minority and against the government. The nazis certainly did not teach such things; rights were for "true Germans", with the strong implication that you needed a genetic claim and to prove yourself in ways only vaguely specified and that it would be very unwise to push. The nazi government also took great care to give you comprehensive reasons for the hate, to build a whole propaganda complex to make sense of why you should hate the jews. Resisting that is a lot harder than going "you said all sapients have rights, Jews are sapient, so what gives". Political systems that systematically abuse subgroups tend to be very clear that rights are for in-group people, not just any people. You are imagining a heroic resistance to a complete strawman. 

The story the nazis told you went something like this. (Warning, I am about to repeat vile nazi propaganda, and will increasingly shift to nazi tone, terminology and pronouns to illustrate the effect.)

Rights, according to the nazis, were neither universally for humans, nor something inalienable. They didn't like the idea of rights you just got due to your needs at all. Rights had to be deserved. They taught you that the German people - an ethnic group of ancient, noble blood, not a group you could be admitted to by virtue of a flimsy piece of paper - had a grand destiny. That that feeling inside you that you want more for humanity, that we can be better, stronger, more rational than this, that there should be progress and growth beyond all limits, daring exploration past any frontiers, that this incredible feeling and drive is shared among all German people; that because of your beauty and brilliance and bravery, you have earned something wonderful, a future grander than any you can imagine.

But that our destiny has been foiled. Foiled by the jews. Who tricked us into debt and caused the great inflation that left us all so poor, when we had worked so hard, taking all the money we had earned and devaluing it until it was better to burn it for cinder than use it to buy wood, no longer worth the paper it was printed on. Who betrayed us, and made us lose the first world war, when we were fighting so bravely, with genius tactics and incredible sacrifices that should have meant we won, but the jews stabbed us in the back, wasting the sacrifices of our brave fathers and friends. Who brought the black plague that ravaged Europe, killing one person in three across the continent. Who broke God's laws on lending, violating the rules of the bible, yet had the audacity to claim that they were the chosen people. Who stole all the good jobs, leaving them to smirk at us from the universities, and not because they were better, but because they were sneaky and corrupt, and because their crazy, ridiculous ideas were in fashion while our reason was ignored. Who had strange rites, in a strange tongue, and mutilated their children's genitals, and tortured animals to death in bloody ways. Whose genes were unclean, so that breeding with them would make your children strange and evil and sick, also; that this was hard biology, scientific, beyond opinion, simple fact, proven, known, something a sentimentalist may deny because he prefers to be politically correct, but a rationalist would not. And that they may look perfectly like real people, sound like real people, but that this is a mask, a trick, to sneak into your heart, a careful manipulation to look out for. They don't really feel. They just pretend, so you put down your guard and they can exploit you.

But the jews are crafty, and sly. Even very rational and strong Germans, when listening to the jews, have been seduced into aiding these traitors and enemies in our midst. Aiding them is as bad as being one. No true German would let this corruption onto his heart. Anyone who does, is an enemy also, to be discarded and treated as one. It is wisest to avoid all contact with jews entirely, to ward yourself against this trickery. Not just not to date them (a disgusting thing to do, you might as well fuck an animal, or drug yourself to oblivion while pregnant), or befriend them; you should not buy from them, in fact, they should not be in your city at all. All the more so because the beautiful houses they live in have been stolen from Germans, really; you are just taking back what should have been yours from birth, your rightful home and heritage. The jews should be moved away, where they cannot do harm. And put to work, for everyone knows that they don't work, that all the riches they have have been accumulated through trickery and deceit. Their books, which are filled with lies, should be burnt, unread. To allow them two way communication is dangerous. We all like to think we would be strong enough to resist jewish manipulation, but humans are hackable. Do not give them an opening. Do not listen. Keep them locked away.

The situation is dire, and needs to be addressed with maximum efficiency. We are unfortunately in a war now, a war we never wanted, and did everything to avoid, you know how much Hitler loves peace, how much he resisted it; but alas, the German people were abused so terribly and relentlessly abroad, so threatened, so in need, so unjustly treated, that we had no choice but self-defence, finally allowing our men to shoot back. But think of the first world war again - what if the jews betray us again? Germany would be truly destroyed, all our culture and knowledge, literature, science, land, all in ashes and forgotten. A horror. 

So in light of the speed and intensity of the actions needed, what should we do? The military is quite the inspiration in this regard, with the best chosen to lead, each in their place. What is important to understand is that questioning orders in the military is a terrible idea, there is not sufficient time to go back and ask your officer why, and wouldn't another thing be better. A good soldier is utterly loyal, loyal to the end. He would not betray Germany. He would not hesitate. He knows his superiors are wise, and would not lie, and would never ask something that is not utterly, completely necessary. He proves himself through his loyalty, and that is how he can become a leader himself one day.

The nazis would tell you of rats, how rat babies look so cute, and they cry so pitifully when killed, so that you might think you did not want to kill them, and would rather let them go. But people who let rats go are fools. The rats multiply. They come inside your house, they eat your food, they sleep in your bed. You softness will bring you disease and ruin. It will starve your real family and community. As a strong, rational man, you will harden your heart against the jews. You will do what needs to be done. It is hard, and you will have to be very strong to go through with this. Exterminating them will be the hardest thing you have ever done, the biggest test of your rational mind against your heart. Your reason and knowledge must prevail. They must be killed, there is no other way.

Yes, the jews might cry and beg and make all sorts of arguments. Think of the German children you are saving when killing the Jewish ones. Of the grand future of Germany that will compensate for this pain many times of over. Killing many jews at once can be very stressful, we know. We know how hard it is for someone as noble as you to do such a thing. You are very strong and brave to stand killing so many. But to lessen the burden, it is likely better to gas them, to save you the distasteful sight, and the bullets. 

If all the Germans knew what was happening in the camps, they would be upset and hurt a lot. The German children would weep to see the ugliness of such a harsh reality, necessary, but ugly all the same. And yet, it would be no less necessary, it is for their own good. If you are very strong, you will carry the knowledge of what happens here and tell noone. You will be kind to the Germans, and execute the jews in the shadows to spare the Germans the horrid sight. You will be efficient, and never sadistic, never wasteful. Only doing your duty, so another does not have to for you. Someone has to push the gas canister into the room below. You see your colleague shaking. Be his strength, and do it for him. You will make maximum use of the jews for the benefit of Germany, putting your mind to the task of recovering as many resources as possible. You will calculate how to house and feed them so we get maximum labour for the investment, not giving them unnecessary luxuries - the German children are starving and freezing, remember - but just enough so they can do the labour we need. You will use the jews to test medications and find out the limits of exposure to cold and toxins and infections and injury, leading revolutions in medicine and saving countless lives, like the hero you are.  People will ignore where this knowledge came from, but they will use it and need it. You will recycle their jewellery and clothes, down to the gold of their teeth, and from this grossness, good will come again, those clothes will warm children in need, that gold will turn into beauty. You will make a profit, which the country needs, through clever measures and brilliant leadership. You will burn the corpses to make everything clean and finished, until the Jewish problem is nothing but ash to fertilise our fields, and grow rye for our bread. Not because you are ashamed; oh no, all of this is legal, every death has its number and tidy paperwork, its stamps and signatures, everything is in order, efficiently organised, the leaders know and are so proud of you. You are saving everyone who deserves to be saved by being this strong.

***

This story was poured into you everywhere. In history, you learned how the jews had fucked us up. In math, you calculated the damage they do. In biology, you learned how their genes were terrible, their very brains deformed. After school, you had social activities all day that were directly controlled by the nazis, practicing blind obedience and violence and in-group thinking, rehearsing it until it became automatic. The radio you had blared it all day long, another Jewish atrocity discovered over and over. Other radio was jammed where possible, and where not, people who even listened to enemy news were per se traitors who should be moved into the outgroup, people you should be alert for, and report. You had no internet. The books, not just by jews, but against war, for human rights, for anything good, had been burnt. The universities and schools had been taken over. Instead, propaganda filled the books and posters and speeches, your teacher, your boss, your everyone, day and night. It was echoed by your very family, pledging that they were so loyal to the Führer that they would report a traitor, even if he were found in their own family, God forbid.

The propaganda was overt, loud, outspoken, so you knew you could be, and should be, and would be asked to be; you could not say good morning without giving the Hitler salute, giving in or standing out like a sore thumb, endangering your own family, who would be taken with you to the camps if you rebelled, including your innocent kids who understood and had done nothing. But it was also subtle and indirect. The world you were shown was consistent with jews being corrupt and evil.

It was possible to resist. People did. Everyone should have. I hope with all my fucking heart that I would have. I hope it so damn much it feels like my heart will burst.

But we know that nearly everyone thinks they would have resisted the nazis, and that very few people actually did. That should give you pause when assuming you would resist a similar movement targeting a different group.

Importantly, the people who did resist, e.g. those who actively hid jews in their flats when the raids happened and the jews knocked on their doors, generally did not resist on rational grounds. They often said they could give no rational reason at all. There have been extensive interviews with people who helped, who often say something like "The idea of not helping the man at my door, whispering "they have come to kill me; please hide me" never ever occurred to me. It would have been so obviously wrong, wrong beyond words. I did not think of the danger to my family; that way lies madness. I did not think of how we would get supplies; these were problems for the future, and they would just have to be solved then. You have to take them in. There was no choice. I had pulled him inside before a single thought had finished." 

It frightened me when I first read those interviews. Because I meanwhile realised I would absolutely have thought about this. I would have hesitated, at least for a split second, imagining my girlfriend being killed over this, thinking how the fuck will we pull off the logistics of this. After all, my house is not currently filled with Ukrainians, nor was it filled with Syrians; I protested, I mobilised, I helped out at shelters, but we discussed it and decided someone living on the coach of our tiny flat for months would drive us batshit, plus the lease did not allow it - but being in the crowded shelter will have fucked with the traumatised Ukrainians even more, and we still did not take any in. 

I can imagine a world in which I would have failed in nazi times, and this horrifies me more than I can say. It haunts me. I am desperately hoping the horror of failing like this will keep me on the right side, precisely because I can feel the pull of the abyss enough to cling to the path so hard my fingers will bleed, knowing that I could very well slip.

Even if in the real world, the security man would not have smiled and clapped. But taken you, as well, and your family, to boot.

Curated. 

>! At first, I thought about this piece as a comment on educational systems, but that's not really it. The post is about how a society attempts to audit itself and its education. Our community is not at the scale where this setyp is feasible (nor do we necessarily have the time), but it does get me thinking about what are reasonable tests that demonstrate whether we're morally and epistemologically on track. 

Growing up, as a kid, I was always told that every sapient life is precious, everything that thinks and knows itself -

Between this beginning and the title, I thought this was going someplace completely different.

Excellent short story!

Curious if anyone has any non-plot-related takeaways that might be enlightening.

When you realize that they totally have a city somewhere that's running the control trial.

This is the first thing I've read set in dath ilan and I think I'd enjoy reading more.

I see the tag wiki lists several other writings in dath ilan, would anyone here care to suggest a reading order for me?

Was this an attempt to flesh out why Dath Ilani would have low posteriors on there being a conspiracy? 

So i read a lot of dah-ilani glowfics the previous weeks, and yet, i didn't guess right. i didn't stop to put numbers on it, so i can in retrospective (and still not sure it's actually good idea to put numbers on all things). and it was 0.9 that the stroy is about kid losing trust in adults because they was told a lie, and 0.3 that after that, it turned out they should trust adults and this distrust is bad (like teens that think all drugs are not dangerous because adults exaggerate the harm of the less-harmful ones). in that situation, i was basically 50-50 divided if the Aesop is about the importance of bounded trust for the reader, that should see themselves as the kid, or that lying to children is bad, and the reader should not do that. 

i did realized it's dath illan and experiment some time after. and now i even more curious to what dath ilan would do with people like me, who see lying as Evil. not try to change them - the utilityfunction is not up to grabs.

i'm pretty sure typical minding make my attempts to do that sub-optimal. i just find it hard to imagine society when most people actually OK with that state of affairs. but my attempts at imagining trust broken and things go bad feel unrealistic, an-dath-ilani to my sense of how-dath-ilan-is. for example, this: https://pastebin.com/raw/fjpS2ZDP doesn't strike me as realistic. i expect dath ilan can use the fact the Keepers Are Trustworthy for example, to swear to a child they will never ever pull such experiments on them, and the child believe that. i expect dath ilan check in younger age how children react to that sort of thing, that is standard on dath-ilani education, and stop if they see it bad for some kid.

and yet... the utilityfunction is not up to grabs. and for some reason, this "fact" about dath ilan is somehow more bad then, for example, the places where dath ilan allow people lack of reflection so they can remain themselves and not go full Keeper.  i disagree there and find it wrong, but it strike me as difference in prioritization, when here it's look like our utilityfunctions are opposite in this small section.

i see lies and deceptions as Evil, even of sometimes it can be traded off, and society with more slack will use it to lie much less, almost never. dath ilan LIKE it's clever experiments and lies children should figure out for themselves. and i would have expected that Keepers would be the type of people that HATE such lies with fury of a thousand suns. so in the end, i remain confused, and feel like dath ilan is somewhat world-where-people-like-me-doesn't-exist. which, most Utopias just ignore uncomfortable complications, but dath ilan is much better then most. and i can't really believe dath ilan heredity-optimized to not have people that hate lying and being lied to.

so in the end, i just confused.

Really engaging! Was especially gripped at the bluffing section, and laughed out loud in shock at the reveal. Really enjoyed it :)

Seems like the real test would be to do it without the television shows?