Preference Inversion

post by Benquo · 2025-01-02T18:15:52.938Z · LW · GW · 28 comments

This is a link post for https://benjaminrosshoffman.com/preference-inversion/

Contents

31 comments

Sometimes the preferences people report or even try to demonstrate are better modeled as a political strategy and response to coercion, than as an honest report of intrinsic preferences. Modeling this correctly is important if you want to try to efficiently satisfy others' intrinsic preferences, or even your own. So I'm sharing something I wrote on the topic elsewhere.

You asked why people who "believe in" avoiding nonmarital sex so frequently engage in and report badly regretting it. Instead of responding within your frame, I'm going to lay out the interpretive framework that seems most natural to me to use for this problem, and then answer in those terms.

We can call things or actions good or bad, right or wrong, with reference to some intention that both the speaker and listener have in mind. For instance, a sturdier and sharper knife is a better one, because our uses for knives tend to converge. We can expect to be understood when we call some knives "good" and leave out "for cutting," and likewise when we call spoiled food bad without reference to a shared interest, because it harms the body of the eater, which harm we generally expect animals to try to avoid.

Moral injunctions such as "it is wrong to lie," "it is bad to steal," can diverge from the local interests of the organism being admonished, in service of a larger, convergent goal. By abstaining from some narrowly self-interested behaviors now, we preserve the necessary conditions for our needs to be met in the future, and the relation between the costs and the benefits can in principle be explained within the system of reference that judges actions as good or bad.

Not all injunctions are like this. For instance, reproduction is such a large component of inclusive fitness that it's not clear what good an organism could get to compensate it for forgoing reproduction. If, like the early Essenes or Christians, we judge sexual desire and activity to be simply bad, we cannot explain this inside the moral system in terms of an animal's rational decision to defer gratification. (This isn't an analytically certain proof, and depends on some contingent facts about apes. If ants or bees talked about something like right and wrong, or good and bad, their relation to those ideas might work very differently from ours.) Instead, we have to explain these statements from an independent system of reference, outside the one that judges reproduction to be bad. There are two things to be explained:

1 How can someone be induced to persistently endorse, promote, and act on perverted moral judgments, i.e. judgments that on net oppose rather than promote their interests as an organism?

2 How are such inducements ecologically fit? Why are they selected for and under what circumstances? Why do we see a lot of them, with lots of discernible traces in the world, rather than a negligible amount?

In some primate groups, a dominant male will punish submissive males for revealing sexual desire for the sexually mature females.[1] This is not exclusive to language-using apes, so it cannot be a mere instruction to lie - it has to be a demand to fake disinterest, i.e. to distort one's own behavior to emulate it. This is an easy to understand example of an important general fact about humans: we can be threatened into internalized preference falsification, i.e. preference inversion.

There seems to be some sexual heterogeneity here. On priors this makes sense; while women's concealed estrus allows them to consciously decide whether to conceal or reveal sexual interest, men's erections are notoriously difficult to control consciously, so adolescent men rapidly learn to deform their unconscious desires to match what their society says they ought to want. Experimental evidence confirms this; while both women and men will predominantly tend to report sexual arousal patterns that conform to social desirability, men's genital arousal patterns conform to their constructed identities much more than women's do.

Ecologically, preference inversion seems likely to persist if groups using that social technology have an advantage in recruiting their members into conflicts against other groups, and thus in winning those conflicts. This can take the form of warfighting at scale, which requires people to move towards danger with no clear self-interest in doing so. It can also take subtler forms of indirect conflict, of the sort described in The Debtors' Revolt, Moral Mazes, The Golden Notebook, The Fountainhead, etc.

The ecological success of moral perversions depends on their uneven adoption, i.e. on hypocrisy. If everyone felt an uncomplicated preference for moving towards danger, there would not be a next generation. Likewise if everyone were chaste and celibate. Submissive males in a primate group will be hoping for opportunities to supplant the dominant male, or to subvert his control. Clerics and warriors are recruited or retained through enjoying more approval than peasants for the "virtues" of asceticism and danger-seeking, but they survive through the fruits of peasants' "vicious" way of life, and in some cases have to replenish their own population by recruiting from "bad" peasants.

To generalize, if you have been coerced into participating in a perverted moral order, you are stuck with some combination of internalizing an orientation against life, and internalizing an orientation against morality, i.e. being "bad." A priest or warrior might imagine that they are possessed by a god when lying to peasants or murdering enemies, but possessed by some demon when seeking forbidden intimacy or abandoning a fight. In Freudian terms, these correspond to the superego (literally "above-me," the imagined authority to which you attribute agency for your destructive behavior) and the id (literally "it," an imagined subversive subagent with all the desires your moral frame demands that you disown).

One thing that can cause confusion here - by design - is that perverted moralities are stabler if they also enjoin nonperversely good behaviors in most cases. This causes people to attribute the good behavior to the system of threats used to enforce preference inversion, imagining that they would not be naturally inclined to love their neighbor, work diligently for things they want, and rest sometimes. Likewise, perverted moralities also forbid many genuinely bad behaviors, which primes people who must do something harmless but forbidden to accompany it with needlessly harmful forbidden behaviors, because that's what they've been taught to expect of themselves.

Some societies have norms against nonmarital sex that really do seem to function to promote marital intimacy and monogamous household formation - notably, the Amish and non-Modern Orthodox Jews. There also seems to be a less legibly distinct subset of more conventional conservative Christians who report being eager to marry and experience marital intimacy, though I am not sure how they reconcile this if at all with the New Testament. But these are not the people you are asking about.

You are asking about people whose relevant narrative center is not the positive value of marital intimacy, but the badness of sexuality, whether or not they mouth a party line endorsing the former. Many people in these types of conservative Christian cultures - more often women in my experience - report that after marriage, they have difficulty engaging in sexual behaviors, because they've learnt from childhood that sex was bad and dirty, and it's confusing for this behavior to suddenly shift from condemned to endorsed.

At this point the behavior you describe should no longer be perplexing. People who have been coerced into preference inversion cannot honestly report their own preferences or intentions as an organism. Instead, they must choose between some combination of internalized coercion, and complementary demonic possession.

This treatment of the topic is very compact. I was heavily influenced by Jessica Taylor's On Commitments to Anti-Normativity, and Friedrich Nietzsche's Genealogy of Morals.

  1. ^

    Tactical Deception and the Great Apes: Insight Into the Question of Theory of Mind, by Casey Kirkpatrick:
    Other observations of deception recorded by deWaal (1986) involved several instances in which a subordinate male courted a female by displaying his penile erection. Whenever a dominant male unexpectedly appeared, the aroused subordinate would hide his erection from the view of the approaching chimpanzee (deWaal 1986: 233; Whiten 1993: 377; Whiten & Byrne 1988: 215- 216). The chimpanzee dropped his arm, always leaving his hand to dangle between the dominant male and his erection. This was done in order to avoid a violent confrontation, which would have been inevitable had the dominant been aware of the subordinate's actions.
    [...]
    deWaal, F. 1986. "Deception in the Natural Communication of Chimpanzees". In Deception: Perspectives on Human and Non-human Deceit. Mitchell,(ed.). pp. 221-224. Albany: University of New York State.
    Whiten, A. and Richard Byrne. 1988. The Manipulation of Attention in Primate Tactical Deception. In Machiavellian Intelligence: Social Expertise and the Evolution of Intellect in Monkeys, Apes and Humans. Byrne and Whiten, (eds.). Oxford: Clarendon Press.
    Whiten, Andrew. 1993. "Evolving a Theory of Mind: the Nature of Non-Verbal Mentalism in Other Primates". In Understanding Other Minds: perspectives from Autism. Baron-Cohen, Tager-Flusberg and Cohen, (eds.). pp. 367-396. New York: Oxford University Press.

28 comments

Comments sorted by top scores.

comment by cousin_it · 2025-01-02T21:33:05.268Z · LW(p) · GW(p)

I think this is wrong. First you say that celibacy would be pushed on lower status people like peasants, then you say it would be pushed on higher status people like warriors. But actually neither happens: it's not to the group's advantage (try to explain how making peasants or warriors celibate would advantage the group - you can't), and we don't find major religions doing it either, they are pro-fertility for almost all people. Celibacy of priests is an exception, but it's small and your explanations don't work for it either.

Replies from: Benquo, Benquo
comment by Benquo · 2025-01-02T22:58:35.061Z · LW(p) · GW(p)

I don't think I made those claims. I did say that clerics are often supposed to be celibate, and warriors are generally supposed to move towards danger, in a single sentence, so I see how those claims might have been confused.

The general pattern I'm pointing out is that some scarce resources, or the approval which is a social proxy for such resources, are allocated preferentially to people who adopt an otherwise perverse preference. These systems are only sustainable with large amounts of hypocrisy, where people are on the whole "bad" rather than "good" according to the approval criteria. (Elite overproduction is when societies fail to preserve this proportion.)

The plausibility of such inversions is demonstrated by their presence in other sorts of apes where they're more clearly motivated by local incentives, as they may also have been in humans' precivilized ancestral environment. Precivilized people didn't have "peasants," but in many contexts nondominant male apes may have persistent reproductive disadvantages, i.e. some territorial apes may have tournament-style mating for males.

Replies from: cousin_it
comment by cousin_it · 2025-01-02T23:21:45.062Z · LW(p) · GW(p)

Yeah, I missed a big part of your point on that. But another part maybe I didn’t? Your post started out talking about norms against nonmarital sex. Then you jump from that to saying they’re norms against reproduction - which doesn't sound right, religious people reproduce fine. And then you say (unless I'm missing something) that they're based on hypocrisy, enabling other people to not follow these norms, which also doesn't sound right.

Replies from: Benquo
comment by Benquo · 2025-01-02T23:42:39.870Z · LW(p) · GW(p)

Successful religions don't suppress reproduction in practice. But many do maintain an explicit approval hierarchy that ranks celibacy and sexual restraint above typical sexual behavior, sometimes expressing overt disgust with sexuality. This creates a gradient of social rewards that aids group cohesion, but requires most people to be "imperfect" by design. An important failure mode is that some conscientious people try to fully internalize the explicit values, ending up with clinical symptoms of sexual aversion that persist even when officially sanctioned (e.g. in marriage).

Replies from: cousin_it
comment by cousin_it · 2025-01-03T00:38:08.273Z · LW(p) · GW(p)

But many do maintain an explicit approval hierarchy that ranks celibacy and sexual restraint above typical sexual behavior

I think we just disagree here. The Bible doesn't say married people shouldn't have sex, and no prominent Christians say that either. There are norms against nonmarital sex, and there are norms against priests having sex, but between these things you draw a connection and generalization to all people which doesn't sound right to me.

Replies from: Benquo, JenniferRM
comment by Benquo · 2025-01-03T13:31:39.579Z · LW(p) · GW(p)

OK, so we've got something like a factual disagreement. Here are some observations that would change my mind substantially:

Credible testimony from someone who'd previously been documented claiming that their variant of Christianity had inculcated in them an anti-sex attitude, that they'd been lying to normalize their non-culturally-conditioned aversion to sex.

An exposé demonstrating that many such prominently documented testimonies were fake and did not correspond to actual people making those claims.

Examples of the sort of thing I mean:

I try to find the Christian bible passages saying it's better never to marry or have sex (e.g. Matthew 19:9-12, 1 Corinthians 7), and persistently fail to find them. Or someone persuasively explains that I'm idiosyncratically misinterpreting them, and I can't find evidence of many people agreeing with me (e.g. those verses showing up when I do a Google search for "bible passages saying it's better never to marry or have sex").

A methodologically careful cross-cultural survey demonstrates that this sort of well-attested sex-aversion isn't more common in people raised in high-commitment Christian communities, than in people in other cultures with no such messages.

What would change your mind?

Replies from: cousin_it, dr_s, cousin_it
comment by cousin_it · 2025-01-03T21:25:22.885Z · LW(p) · GW(p)

For every person saying "religion gave me a hangup about sex" there will be another who says "religion led to me marrying younger" or "religion led me to have more kids in marriage". The right question is whether religion leads to more anti-reproduction attitude on average, but I can't see how that can be true when religious people have higher fertility.

Replies from: Benquo
comment by Benquo · 2025-01-03T21:54:59.715Z · LW(p) · GW(p)

This doesn’t seem to engage with the content of the post at all, or with my multiple corrections to your implausible misunderstandings, so I think this is a motivated pattern of misunderstanding and I’m done with your comments on this post.

comment by dr_s · 2025-01-04T20:53:30.327Z · LW(p) · GW(p)

I don't know about the Bible itself, but there's a long and storied tradition of self mortification and denial of corporeity in general in medieval Christian doctrine and mysticism. If we want to be cute we could call that fandom, but after a couple thousand years of it it ends up being as important as the canon text itself.

comment by cousin_it · 2025-01-03T17:42:01.257Z · LW(p) · GW(p)
comment by JenniferRM · 2025-01-03T06:23:30.974Z · LW(p) · GW(p)

There is text in the bible that strongly suggests the new testament set up celibacy as morally superior to sex within marriage. In practice, this mostly only one-shotted autists who got "yay bible" from their social group, and read the bible literally, but didn't read enough of the bible to realize that it is a self-contradicting mess.

You can "un self contradict" the bible, maybe, with enough scholarship such that people who learn the right interpretative schemes can learn about how maybe Paul's stuff shouldn't be taken as seriously as the red text, and have all the "thoughtful scholars" interpret the mess in a useful and mostly non-contradictory way...

In real life, normies just pick and choose, mostly by copying the "pick and choose" choices of people who seem successful and useful as role models, and they don't think too hard about which traditions they are following and why they're following them... but the strong "generalized anti-sex attitudes" in the bible would make a classic example for Reason As Memetic Immune Disorder [LW · GW]. They aren't used there, but they easily could be.

comment by Benquo · 2025-01-02T22:43:44.549Z · LW(p) · GW(p)Replies from: cousin_it
comment by cousin_it · 2025-01-02T23:18:52.924Z · LW(p) · GW(p)
comment by ChristianKl · 2025-01-03T17:28:29.734Z · LW(p) · GW(p)

A priest or warrior might imagine that they are possessed by a god when lying to peasants or murdering enemies, but possessed by some demon when seeking forbidden intimacy or abandoning a fight. 

Buddhist have some norm against forbidden intimacy. The way to get around them in tantra is to identify with a God while having sex and not act with the normal human identity.

Jewish tradition has similar methods where the way to get around the sinful nature of sex is about identifying with god during the act. 

comment by Dagon · 2025-01-02T22:33:26.039Z · LW(p) · GW(p)

Upvoted, but I think the model is questionable.  Start with 

better modeled as a political strategy and response to coercion, than as an honest report of intrinsic preferences.

Do you have an operational definition of "intrinsic preferences"?  I can't tell if you're just pointing out hypocrisy, or asserting that there are "real" and "illusory" preferences, or something else.  In my mind, there's a lot of deception (both self- and other-targeted) involved in understanding and communicating one's preferences, and a whole lot of speech acts related to the topic are not directly felt.  A lot of moralizing falls into this category - say what I want others to do, which isn't what I "want" to do myself.

In humans, there's a ton of inconsistency as well, which makes "preferences" a slippery concept.

Replies from: Benquo, programcrafter
comment by Benquo · 2025-01-02T22:48:27.851Z · LW(p) · GW(p)

Sometimes people profess or try to reveal a preference for X, as a response to coercive pressures that are specifically motivated by prior underlying preferences for anti-X. This is what I'm calling preference inversion. My intuition is that generally, upon reflection, people would prefer to satisfy their and others' preferences as calculated prior to such influences. I don't know whether there are other sorts of analogous distorting factors nearly all reasonable people would not like to satisfy upon reflection, but in general, I'm using the term "intrinsic preferences" to refer to whatever's left over after all such generally appealing adjustments.

Replies from: jimmy, Dagon
comment by jimmy · 2025-01-03T19:34:22.736Z · LW(p) · GW(p)

I don't think it's so simple at all.

If you start with the conclusion that sex is great, and anti-premarital sex campaigns are really just anti-you-procreating campaigns and therefore oppressive and bad, then sure. I don't think that's a fair assumption across the board (e.g. Amish as an existence proof of "something more"), but it certainly doesn't work for all preferences and it's generally not so clear.

Let's look at preference for eating lots of sweets, for example. Society tries to teach us not to eat too much sweets because it's unhealthy, and from the perspective of someone who likes eating sweets, this often feels coercive. Your explanation applied here would be that upon reflection, people will decide "Actually, eating a bunch of candy every day is great" -- and no doubt, to a degree that is true, at least with the level of reflection that people actually do.

However when I decided to eat as much sweet as I wanted, I ended up deciding that sweets were gross, except in very small amounts or as a part of extended exercise where my body actually needs the sugar. What's happening here is that society has a bit more wisdom than the candy loving kid, tries clumsily to teach the foolish kid that their ways are wrong and they'll regret it, and often ends up succeeding more in constraining behavior than integrating the values in a way that the kid can make sense of upon reflection.

So which preferences are "real"? The preference for candy or the preference for no candy and no diabetes? What you are calling "intrinsic preferences" is often just shallow preferences, which haven't yet been trained to reflect nuances of reality like "more of a good thing isn't always better" and "here's where it's good and here's where it's not good". There's preferences declared, preferences acted on, and preferences that will be regressed to in absence of guiding pressure. The declared preferences are generally going to align better with the coercive forces than the preferences that will be regressed to in absence of said pressure, but the preferences acted on can easily be more reflectively stable than those regressed to -- because all that takes is for the culture to be wiser than the individual, and the individual to not have caught up yet.

Returning to the case of nonmarital sex, of course it feels good -- just like candy feels good. There is something there that we need (namely "sex", and "calories"), but the question is over whether naïve indulgence across all contexts will result in blowing past Goodhart's warnings into more harm than good, and whether the "oppressive society" is actually forming you into a closer approximation of the reflectively sensible thing to do.

Societies pressures can end up perverted, but individual's intrinsic preferences start out perverted. Who is closer to reflectively stable, society who aggressively shames overconsumption of sweets, or the kid who wants to eat all the sweets? Society who aggressively shames nonmarital sex, or the teenager who wants to bone everyone?

As we mature, our desires change, and the degree to which reversion in absence of external pressures brings us closer to something truly reflectively stable depends on how much we've learned to separate overconsumption of sweets from appropriate consumption of sweets, and overconsumption of nonmarital sex to appropriate consumption. I think the answer depends too much on the specific (sub)culture and the specific individual at a specific time in their life to make any sweeping generalizations.
 

Replies from: Nick_Tarleton, programcrafter
comment by Nick_Tarleton · 2025-01-03T20:20:09.433Z · LW(p) · GW(p)

Let's look at preference for eating lots of sweets, for example. Society tries to teach us not to eat too much sweets because it's unhealthy, and from the perspective of someone who likes eating sweets, this often feels coercive. Your explanation applied here would be that upon reflection, people will decide "Actually, eating a bunch of candy every day is great" -- and no doubt, to a degree that is true, at least with the level of reflection that people actually do.

However when I decided to eat as much sweet as I wanted, I ended up deciding that sweets were gross, except in very small amounts or as a part of extended exercise where my body actually needs the sugar. What's happening here is that society has a bit more wisdom than the candy loving kid, tries clumsily to teach the foolish kid that their ways are wrong and they'll regret it, and often ends up succeeding more in constraining behavior than integrating the values in a way that the kid can make sense of upon reflection.

The OP addresses cases like this:

One thing that can cause confusion here - by design - is that perverted moralities are stabler if they also enjoin nonperversely good behaviors in most cases. This causes people to attribute the good behavior to the system of threats used to enforce preference inversion, imagining that they would not be naturally inclined to love their neighbor, work diligently for things they want, and rest sometimes. Likewise, perverted moralities also forbid many genuinely bad behaviors, which primes people who must do something harmless but forbidden to accompany it with needlessly harmful forbidden behaviors, because that's what they've been taught to expect of themselves.

I agree that the comment you're replying to is (narrowly) wrong (if understanding 'prior' as 'temporally prior'), because someone might socially acquire a preference not to overeat sugar before they get the chance to learn they don't want to overeat sugar. ISTM this is repaired by comparing not to '(temporally) prior preference' but something like 'reflectively stable preference absent coercive pressure'.

Replies from: jimmy, Benquo
comment by jimmy · 2025-01-03T20:49:39.805Z · LW(p) · GW(p)

The part of OP you quoted only covers part of what I'm saying. It's not just that we can be pressured into doing good things, it's also that we have no idea what our intrinsic desires will become as we learn more about they interact with each other and the world, and there is a lot of legitimate change in intrinsic preferences which are more reflectively stable upon sufficiently good reflection, but which nevertheless revert to the shallower preferences upon typical reflection because reflection is hard and people are bad at it.

"Reflectively stable in absence of coercive pressure" is very difficult to actually measure, so it's more of a hypothetical construct which is easy to get wrong -- especially since "absence of coercive pressure" can't actually exist, so we have to figure out which kinds of coercive pressure we're going to include in our hypothetical.

comment by Benquo · 2025-01-03T20:43:15.048Z · LW(p) · GW(p)

"As calculated prior" is not quite correct, "reflectively stable absent coercive pressure" is a better formulation.

comment by ProgramCrafter (programcrafter) · 2025-01-04T00:33:48.901Z · LW(p) · GW(p)

I object to the framing of society being all-wise, and instead believe that for most issues it's possible to get the benefits of both ways given some innovators on that issue. For example, visual communication was either face-to-face or heavily resource-bounded till the computer era - then there were problems of quality and price, but those have been almost fully solved in our days.
Consequently, I'd prefer "bunch of candy and no diabetes still" outcome, and there are some lines of research/ideas into how this can be done.
As for "nonmarital sex <...> will result in blowing past Goodhart's warnings into more [personal psychological, I suppose] harm than good", that seems already solved with the concept of "commitment"? The society might accept someone disregarding another person if that's done with plausible deniability like "I didn't know they would even care", and commitment often makes you promise to care about partner's feelings, solving* the particular problem in a more granular way than "couples should marry no matter what". The same thing goes with other issues.

That said, I've recently started to think that it's better to not push other people to less-socially-accepted preferences unless you have a really good case they can revert from exploration well and would be better off (and, thus, better not to push over social networks at all), since the limit point of person's preferences might shift - wicked leading to more wicked and so on - to the point person wouldn't endorse outcomes of change on reflection. I'm still considering if just noting that certain behavior is possible is a nudge significant enough to be disadvantaged (downvoted or like).

 

*I'd stop believing in that if commitment-based cultures had higher rate of partners failing on their promises to care than marriage-based; would be interested in some evidence either way.

Replies from: jimmy
comment by jimmy · 2025-01-04T20:50:03.254Z · LW(p) · GW(p)

I object to the framing of society being all-wise,

Society certainly is not all-wise, and I did not frame it as such. But it is wiser than the person who thinks "Trying heroin seems like a good idea", and then proceeds to treat heroin as if it's the most important thing in the universe.

Is it wiser than you, in some limited way in some limited context that you are unaware of? Is it less wise, in other ways? I'd bet on "both" before either.

Consequently, I'd prefer "bunch of candy and no diabetes still" outcome, and there are some lines of research/ideas into how this can be done.

This isn't the eating your cake and having it too that you think it is.

Yes, computers allow us to do things we couldn't do before, and that's great. Before, you might have to choose between meeting with Bob in the north or Richard in the south, and technology enables you to have both. Great!

The thing is, neither meeting Bob nor Richard is a "sin". It's not a "thing you will be tempted to do due to shallow preferences" where society recognizes that those preferences are shallow and predictably lead to bad outcomes. Society wasn't all up in your business decisions telling you who to meet because it didn't trust you to make the obviously right one; that was on you.

Candy gluttony, like heroin use, is a sin. It's something that society knows is bad news, but will feel like good news to individuals, because individuals are myopic and lack the bigger picture. If you had lived a million lifetimes, and thrown away your life to heroin thousands of times, heroin wouldn't be so tempting because you'd know from experience that heroin ain't great. But you haven't, so you don't, and society has some wisdom to offer individuals here.

Candy consumption is the same thing, scaled down a little bit. You're not after the calories, the micronutrients, or anything real in the candy itself. You're after how it makes you feel. You're after the feeling of getting what you want, without thought about whether you want the right thing. In other words, you're wire-heading.

Pills that reduce the consequences for "sins" -- whether candy consumption, or heroin consumption, or nonmarital sex -- can be good. If you're going to die from syphilis because you were too dumb to listen to society, having some forgiveness can certainly be a good thing, and maybe you'll learn your lesson instead of just dying.

But if you think "Syphilis is treatable!" justifies all nonmarital sex, then you're gonna need a new type of pill soon.

And if you think that once you have BC then now all nonmarital sex is justified, then you're on track for a statistically less happy marriage.

It's not that options aren't often good, or even that options which reduce consequence of sin aren't good. I'm also not arguing that antibiotics and birth control don't open up options for good nonmarital sex, or that no one is with it enough to be able to reliably find them. Some people are; maybe you're one of them.

But technology is not a good alternative to good decision making and informed values. Waiting around for technology that doesn't exist yet instead of learning more about what is good now is a mistake. You get better results by learning what is good than by relying on technological crutches, and the way that this happens will often be difficult to foresee.

As an amusing anecdote relating to this, one of the more sexually successful men I know decided that he was no longer going to have sex with anyone but his future wife -- whom he had not yet met. I called bullshit, and bet him $100. Not long after he made that bet, I saw his disinterest in non-future-wife sex turn a "I like you and would have sex with you" level of attraction into a "I will do whatever it takes to marry you" level of attraction. It's hard to even conceptualize such moves from a shallow pleasure seeking mindset, and impossible to enact them. And yet, I'm quite confident that he wouldn't have been able to marry her otherwise, and that his alternative sex life would have been much worse even from a superficial pleasure seeking perspective. It's hard to do justice to so briefly, but that was a very strong move that led to a great marriage which wouldn't have worked otherwise, and no amount of technological crutches would have gotten him to where he is today.

that seems already solved with the concept of "commitment"?

You mean like... marriage? :p

In all seriousness, I'm not taking a "in marriage only!" stance here.

The success story I give above involved sex outside marriage as an active ingredient in more than one way, and could be used to argue against a strict "in marriage only!" stance. At the same time, it demonstrates value of "in marriage only" which has been lost in what the norm has become.

He was able to thread that needle and get unusually good results because he had both respect for and an understanding of traditional "in marriage only", and a strong enough rebellious streak to not let himself be bound by forces he didn't agree with. You can't get those results without respect for traditional wisdom, and neither can you get it by becoming slave to some pastor's clumsy interpretation of them.

comment by Dagon · 2025-01-03T00:39:51.516Z · LW(p) · GW(p)

Hmm.  By "coercion", you include societal and individual judgements, not just actual direct threats.  It's still hard for me to separate (and even harder for me to privilege) "innate" preferences, over "holistic" preferences which acknowledge that there is a real advantage to existing smoothly in the current society, and include the contradictory sub-desires of thriving in a society, getting along well with allies, having fewer enemies, etc. and for the biological urges for (super)stimuli. 

Replies from: Benquo
comment by Benquo · 2025-01-03T02:42:03.506Z · LW(p) · GW(p)

Consider two different contexts in which one might negotiate tradeoffs around work. When discussing work-life balance, you can openly weigh tradeoffs between career and personal time. But when asked 'Why do you want to work at MegaCorp?' in an interview, acknowledging you're trading anything for a paycheck marks you as deviantly uncommitted. The system requires both pretense of pure dedication and practical compromises, while making that pattern itself unspeakable.

My post was about how this dynamic creates internalized preference inversion - where people become unable to even model certain tradeoffs to themselves, not just discuss them. And this isn't just social pressure - you can actually be killed or imprisoned by cops or psychiatrists for ill-defined deviancy, with much conformity driven by vaguely intuited threats to construe you as the relevant sort of deviant.

Replies from: Dagon
comment by Dagon · 2025-01-03T19:37:53.025Z · LW(p) · GW(p)

Thanks for the discussion.  I think I understand what you're pointing at, but I don't model it as an inverted preference hierarchy, or even a distinct type of preference.  Human preferences are very complicated graphs of long- and short-term intents, both rational, reflective goals and ... illegible desires.  These desires are intertwined and correlated, and change weights (and even composition) over time - sometimes intentionally, often environmentally.  

Calling it an "inversion" implies that one set is more correct or desirable than another, AND that the correct one is subverted.  I disagree with both of these things philosophically and generally, though there are specific cases where I agree for myself, and for most in the current environment.  My intuitions are specific and contextual for those cases, not generalizable.

Replies from: Benquo
comment by Benquo · 2025-01-04T14:14:21.226Z · LW(p) · GW(p)

When one preference is expressed only because its holders are extracting resources from people or mindparts with the opposite preference, that seems to me to justify assigning the self-sustaining one priority of some kind.

comment by ProgramCrafter (programcrafter) · 2025-01-03T01:27:03.698Z · LW(p) · GW(p)

Do you have an operational definition of "intrinsic preferences"?

Let me try. I believe that if copies of a person (as determined by their genotype*) would be raised in different cultures and environments, their revealed preferences would mainly be clustered around a single point, with some shifts determined by what desires society showed them as acceptable/tractable and what as unfashionable. Given proper diversity of surrounding cultures list, I'd say the cluster median indicates person's intrinsic preferences.

 

*it also seems intuitively right to discount traits that were [IVF?]optimized for, if any, as an outside influence; though, I have no strong opinion on this.

comment by Fer32dwt34r3dfsz (rodeo_flagellum) · 2025-01-04T18:11:10.302Z · LW(p) · GW(p)

I expect to post additional comments on this thread, but for now, w.r.t.

Sometimes the preferences people report or even try to demonstrate are better modeled as a political strategy and response to coercion, than as an honest report of intrinsic preferences.

has the author of this post read Private Truths, Public Lies: The Social Consequences of Preference Falsification (Kuran, 1997)? I've read but have not yet written a review of the book, so I cannot comment too critically on its value in this present conversation, but I believe the author should minimally check it out or skim its table of contents. To pull a better overview (from GoodReads) than I can provide off hand:

Preference falsification, according to the economist Timur Kuran, is the act of misrepresenting one's wants under perceived social pressures. It happens frequently in everyday life, such as when we tell the host of a dinner party that we are enjoying the food when we actually find it bland. In Private Truths, Public Lies Kuran argues convincingly that the phenomenon not only is ubiquitous but has huge social and political consequences. Drawing on diverse intellectual traditions, including those rooted in economics, psychology, sociology, and political science, Kuran provides a unified theory of how preference falsification shapes collective decisions, orients structural change, sustains social stability, distorts human knowledge, and conceals political possibilities.

A common effect of preference falsification is the preservation of widely disliked structures. Another is the conferment of an aura of stability on structures vulnerable to sudden collapse. When the support of a policy, tradition, or regime is largely contrived, a minor event may activate a bandwagon that generates massive yet unanticipated change.

In distorting public opinion, preference falsification also corrupts public discourse and, hence, human knowledge. So structures held in place by preference falsification may, if the condition lasts long enough, achieve increasingly genuine acceptance. The book demonstrates how human knowledge and social structures co-evolve in complex and imperfectly predictable ways, without any guarantee of social efficiency.

Private Truths, Public Lies uses its theoretical argument to illuminate an array of puzzling social phenomena. They include the unexpected fall of communism, the paucity, until recently, of open opposition to affirmative action in the United States, and the durability of the beliefs that have sustained India's caste system.

Replies from: Benquo
comment by Benquo · 2025-01-04T20:29:18.374Z · LW(p) · GW(p)

Thanks for the recommendation, I’ll check out that book. I was aware secondhand of the expression “preference falsification” and its meaning - related to what Bryan Caplas calles “social desirability bias.”

By coining the term “preference inversion” I’m trying to call attention to an important special case of preference falsification, where the fact that a preference has been inverted (and corresponding construction of a hypocritical or ‘bad’ majority) is part of the core mechanism, rather than an accidental cost. This is why Jessica’s idea of antinormativity is relevant; a certain sort of preference falsification has the primary function of creating a guilty conscience, rather than compelling object-level prosocial behavior.

comment by LTM · 2025-01-04T15:53:57.228Z · LW(p) · GW(p)

I think this post is interesting, although I don't particularly agree with the conclusions. I think it is helpful to think about the formation of your mind and goals - a tradition which I know goes back to Rousseau and most likely goes back further (I am not very knowledgeable on the topic).

I think a lot of the difficulty goes back to the distinction between 'real'/'intrinsic' and goals and those people proport to believe in. Looking at your example of a Christian sexual prude, nothing about their behaviour implies to me that these virtues of chastity and distance from sex are false at all. Perhaps had they been raised in the absence of society, they would think differently. But if that were the case their views on fashion would probably vastly differ also, and yet we don't consider their desire to dress a certain way to be falsified.

To be clear, I am not claiming that values cannot be gained societally. More that the issue at play is not the creation and adoption of these values from a childhood of coercive exposure, but that these values are hard to satisfy. Sexual prudishness makes it difficult to fulfil inbuilt sexual desires, and so combining both into a single person leaves them less overall satisfiable.

But this goes far beyond societal impacts. In a world of scarce resources, our many inbuilt physical desires as well as our entirely inbuilt desire to behave altruistically conflict in a similar way. It is difficult to be both feared and loved, as some people seem to want to be. Your concern might be better framed at this kind of conflict dragging down one's maximum available utility than a conflict between inbuilt and constructed desires.

Nice post! I'd be interested to see what you think about the above. 

 

As a side note, you mention that men are apparently better at aligning their sexual preferences to those of society than women. This could be evidence of men being hardwired to adopt the expectations of sexual prudishness to their core. But it could also be that men shaped those expectations to begin with through generations of societal dominance. Or a myriad of other factors which mean that for a reason entirely distinct from inbuilt preference falsification, men's desires just line up better to those of society's than women's.

comment by Forged Invariant · 2025-01-04T05:24:06.282Z · LW(p) · GW(p)

Based on the quote from Kirkpatrick, It looks like a clear example of preference falsification, but I do not see any reason to believe that it is internalized preference falsification. Did I miss how the submissive apes were internalizing the preference to not mate? The sentence "This is an easy to understand example of an important general fact about humans: we can be threatened into internalized preference falsification, i.e. preference inversion." makes me think that you intended it as an example of primates internalizing a preference falsification. It feels like the quote is only evidence that primates will be deceptive.

 

On rereading, the claim that "This is an easy to understand example of an important general fact about humans: we can be threatened into internalized preference falsification, i.e. preference inversion" seems reasonably supported by the next paragraph about male vs female arousal in humans. Maybe I just attached the claim to the wrong evidence.