AI romantic partners will harm society if they go unregulated

So on one hand I agree with the distaste towards "steer humans to make babies" attitude, but on the other, I think here it's important to notice that it's not the AI waifus companies that would be friends of your free will. You'd have companies with top research departments dedicated specifically to hack your mind by giving it an addictive product, so that you may give them money, and damn your interests (they'll coach that of course in some nominal commitment to customer satisfaction but then it's off to whaling). Their business model would literally be to make you unhappy but unable to break out because between you and potential happiness is a gulf of even worse unhappiness. That is... less than ideal, and honestly it just shouldn't be allowed.

I personally think you massively underestimate the dangers posed by such relationships. We are not talking about people living healthy well-adjusted lives but choosing not to have any intimate relationships with other humans. We're talking about a severely addictive drug, perhaps on the level of some of the most physiologically addictive substances we know of today. Think social media addiction but with the obsession and emotions we associate with a romantic crush, then multiply it by one hundred.

You don't know me, regulation certainly doesn't know me, and if I have a strong idea what I want my hypothetical family to look like since 16 or so, deliberately pushing against my strategy and agency for 14 years presumably entails certain ... costs.

I've watched two interviews recently, one with Andy Matuschak, another with Tyler Cowen. They both noted that unschooling with complete absence of control, coercion, and guidance, i.e., complete absence of paternalism, is suboptimal (for the student themselves!), exactly because humans before 25 don't have a fully developed pre-frontal cortex, which is responsible for executive control in humans.

Complete absence of paternalism, regarding eduction, gender identity, choice of partner (including AI partner!) and other important life decisions, from age 0, is a fantasy.

We only disagree about the paternalism cut-off. You think it should be 16 yo, I think it should be closer to 30 (I provided a rationale for this cut-off in the post).

"You don't know me, regulation certainly doesn't know me" -- the implication here is that people "know themselves" at 16. This is certainly not true. The vast majority of people know themselves very poorly by this age, and their personality is still actively changing. Many people don't even have a stable sexual orientation by this age (and even by their mid-20's). Hypothetically, we can make a further exception from over-30 rule: a person undergoes a psychological test and if they demonstrate that they've reached Stage 4 of adult development by Kegan, they are allowed to build relationships with AI romantic partners. However, I suspect that only very few humans reach this stage of psychological development before they are 30.

There's an appeal to investing increasingly more resources in increasingly few descendants; and if the gains of automation and AI are sufficiently well distributed, why not do so?

Under most people's populations ethics, long-termism, and various forms of utilitarianism, more "happy" conscious observers (such as humans) is better than fewer conscious observers.

What comes to resources, if some form of AI-powered full economy automation and post-scarcity will happen relatively soon, than the planet could definitely sustain tens of billions of people who are extremely happy and all their needs are met. Conversely, if this will not happen soon, it could probably only be due to an unexpected AI winter, but in this case the population reduction may lead to economy stagnation or recession, which will lead even to worse wellbeing in those "fewer"  people, despite there is a bigger share of Earth's resources at their disposal (theoretically).

Your comments in the replies seem to reject that these men are suffering. No, obviously they are.

I didn't deny that some people actually suffer (probably big portion of them are clinically depressed, though, so would "qualify" to use AI partners before 30 in my proposal), I just said that it's by no means normal to suffer if you just don't have a romantic partner, but otherwise your life is "ok". See this comment [LW(p) · GW(p)].

This perverted strategy of ameliorating symptoms of problems (such as social media, problems with (sexual) self-image and expectations, "dating market", social isolation, etc.) because it will provide a constant stream of $$$, instead of treating the root causes, is what bugs me.

I'm far from convinced that "society" (represented by payment providers) already decided to ban sex bots. It doesn't make much sense, given that payment providers do serve porn industry, OnlyFans, etc. Maybe the issue was that character.AI didn't clearly market itself as "adult startup" and didn't impose age restrictions, and providers saw potential legal risks with this?

I see https://www.evaapp.ai/ is growing, https://caryn.ai/ is growing, they aren't getting banned.

I generally worry much less about hardcore enthusiasts to develop and host open-source waifus, of course fundamentally this cannot be banned, but the potential reach of these will be an order or two orders of magnitude smaller than easy-to-use mobile app. I don't worry about society-scale effects from people using open-source waifus. 

You seem to assume monogamy in the sense that a person needs to break up with an AI partner to develop another relationship with a human. I think that would be unlikely. 

Well, you think it's unlikely, I think it will be the case for 20-80% people in AI relationships (wide bounds because I'm not an expert). How about AI romance startups proving this is at least as "unlikely" that 90+% people could "mix" human and AI romance without issue, on long-term psychological studies? FDA demands drug companies to prove long-term safety of new medications, why we don't hold technology which will obviously intrude in human psychology to the same standard?

The base rate for depression alone among 12-17-year-olds alone is 20%. A company that sells an AI partner would likely be able to optimize in a way that it helps with depression and run a study to prove it. 

You are arguing with the proposed policy which is not even here yet. I think barring clinically depressed people from AI romance is a much weaker case and I'm not ready to defend it here. And even if it would be a mistake to give depressed people access to AI partners, just allowing anyone over 18 to use AI partners is a bigger mistake, anyway, as a matter of simple logic, because "anyone" includes "depressed people".

Unfortunately, a substantial part of my own negative reaction is because all these other limitations of freedom you suggest are in fact within the Overton Window, and indeed limiting the freedom of young men between 16 and 25 naturally extrapolates to all the others.

(Not that I'm not concerned about the freedom of young men, but they're not somehow valid sacrificial lambs that the rest of us aren't.)

Women will find AI partners just as addicting and preferable to real partners as men do.

Dramatically increasing taxes for childless people.

Too-low fertility concerns me deeply. My current preferred strategy had been something financial along the lines of this proposal, but on reflection, I think I need to update.

The main reasons that I see driving the people around me to defer/eschew children are, in rough decreasing order of prevalence:

• Lack of a suitable partner.
• Lack of fertility.
• Expectation of being an incompetent parent.
• Severe untreated mental health ailments.
• Other career priorities, not compatible with child raising.

The third point above is, in my social circle, usually downstream of needing to spend too much time working, not being able to afford childcare services, not being able to afford college, etc.

These issues could be lessened by financial interventions with the net effect of offsetting the burdens of child raising (obviously, implementation details matter a great deal.)

...I had previously assumed that "expectation of incompetence" was the primary issue because it's my primary issue (in the sense of looming large in my mind). ...but having taken an inventory of the childless adults in my social network, I now see that "inability to find a suitable partner" and "infertility" are much bigger issues.

"Inability to find a suitable partner" seems risky to fix, because it looks easy to create horrible side-effects. Many in my social network are in relationships but aren't having children because they're abusive relationships. Others have successfully escaped abusive relationships and subsequently given up on finding a partner; being alone is less painful, and they expect to find another abuser with nontrivial probability if they go looking for another partner. Still others lost a suitable partner to a treatable disease, due primarily to insurance companies ending up de facto in charge of healthcare decisions, and rendering decisions without the patient's health as the priority.

I don't see a good solution. (Well, except for "don't allow insurers to drive healthcare decisions".) All attempts to "solve" this issue I see being promoted in real life seem on-net harmful. My home state of Texas thinks the best solution is to ban divorce, and we expect that to be implemented within the next year. I don't see that as being a net good, even if it may compel additional births in the short term on net. Trapping people in abusive relationships seems incredibly dystopian. I also foresee this making young women even more afraid of getting into the wrong relationship than they already are, which runs the risk of making the "finding suitable partners" problem worse in the long run.

"Lack of fertility" seems challenging to fix, but far less fraught. I was born without a uterus, and wish to bear a child if the medical technology were developed to allow me to grow a uterus. I have a sibling on life-saving medication, a side-effect of which is extremely low sperm count -- a better drug to treat the condition (or an actual cure) could resolve this issue. Multiple of my sisters-in-law have simply failed to conceive for years, due to unknown-to-me causes, and I suspect the issues are similarly fixable via continued medical innovation.

I think you distorted and caricatured a lot of what I've said.

Because it is better for the society if they suffer.

Suffering from not having a partner is not a normal psychological reaction and young men who suffer because of that should have access to psychotherapy (and we should rather work on increasing people's access to psychotherapy than to AI partners, see this comment [LW(p) · GW(p)]). It's normal to lead a productive and enjoyable life without a romantic partner.

Also, many people who would suffer because of the absence of a romantic partner probably would be depressed, and I'm not ready to defend barring depressed people from AI partners.

No one proposes taking romantic books and movies away from young women. Despite creating unrealistic expectations, etc. Apparently their frustration is not necessary to keep the civilization going, or perhaps is considered a too high price to pay.

I addressed this in the last section--social media also creates unrealistic expectations, so as porn, etc. Different tech and media (such as romantic books) have different balance of benefits to individuals and costs to the society. I don't see what bearing does this have on the AI partners policy question.

What else could increase childbirths? Banning women from studying at universities. Banning homosexual relationships. Banning contraception. Dramatically increasing taxes for childless people. -- All of this would be unacceptable today. The only group we are allowed to sacrifice for the benefit of the society are the young cishet men.

"Banning homosexual relationships" and "banning contraception" are grotesque examples that obviously fail, let me skip that.

"Dramatically increasing taxes for childless people", can actually be sensible, but indeed politically intractable (however... I'm not sure, it depends on the country and the actual amount of "dramatical" increase, especially considering that this will actually appear as increased overall tax and increased tax return on people with children, rather than as "childless tax"). This is even with line with many post-scarcity and post-labor proposals, to consider parenting and "community engagement" as actual work rather than something people just do "for free". But anyway, tractability of this or that policy (and effects of its introduction) should be part of the equation anyway, like ROI of a policy. I've discussed this in the second half of this comment [LW(p) · GW(p)].

What are we planning to give to the young men in return for taking away from them the latest technological advances? Let me guess... nothing.

How about building an equitable society where everyone can achieve happiness and find meaning in life regardless of whether they can find a sexual partner or not?

If the incels of future say "all my problems could be taken away by a click of a button, but old people made it illegal for me to click the button (despite them clicking the same button every day themselves)", they might actually have a valid point.

As I discussed in the post, that AI partners will actually be such a "magic button" which those incels won't deeply regret pressing ten or twenty years afterwards, doesn't seem obvious to me, and I think it's these startups' responsibility to demonstrate, at least demonstrate that strong negative effects don't emerge over long timelines. Why we easily accept that it takes many years (routinely 10 years) to pass new drugs through all safety tests, but don't hold psycho-technology (which AI partners essentially are) to the same standard? In the modern world, psychological health becomes ever more important.

From a hedonistic and individualistic perspective, sure, AI partners will be better for individuals. That's the point that I'm making. People will find human relationships frustrating and boring in comparison.

But people also usually don't exclusively care for themselves, part of them also cares about the society and their family lineage, and the idea that they didn't contribute to these super-systems even in itself will poison many people's experiences of their hedonistic lives. Then, if people don't get to experience AI relationships in the first place (which they may not be able to "forget"), but decide to settle for in a way inferior human relationships, but which produce more wholesome experience overall, there total life satisfaction may also be higher. I'm not claiming this will be true for all and necessarily even most people; for example, child-free people are probably less likely to find their AI relationships incomplete. But this may be true for a noticeable proportion of people, even from Western individualistic cultures, and perhaps even more so from Eastern cultures.

Also, obviously, the post is not written from the individualistic perspective. The title says "AI partners will harm society", not that it will harm the individuals. From the societal perspective, there could be a tragedy of the commons dynamic where everybody takes maximally individualistic perspective but then the whole society collapses (either in terms of the population, or epistemics, or culture).

Human relationships should be challenging. Refusing to be challenged by those around you is what creates the echo chambers we see online, where your own opinions get fed back to you, only reassuring you of what you already believe. These were created by AI recommendation algorithms whose only goal was to maximise engagement.

Why would an AI boyfriend or girlfriend be any different? They would not help you develop as a person, they would only exist to serve your desires, not to push you to improve who you are, not to teach you new perspectives, not to give you opportunities to bring others joy.

Because your AI partner does not exist in the physical world?

I mean, of course that an advanced chatbot could be a both better conversator and a better lover than most humans, but it is still an AI chatbot. Just to give an example, I would totally feel like an idiot should I ever find myself asking chatbots about their favorite dish.

[downvoted]

your legislation proposal is highly specific and also socially nonstandard - the "18 is an adult" line is, of course, essentially arbitrary and there's a reasonable case to make either way, but because 30 is very high (I'm not aware of any similarly-sweeping law that sets it above 21 in US at least), you do assume a burden of proof that, imo, is heavier than just stating that the frontal cortex stops developing at 25, then tack on five years (why?).

First of all, as our society and civilisation gets more complex, "18 is an adult" is more and more comically low and inadequate.

Second, I think a better reference class are decisions that may have irreversible consequences. E.g., the minimum age of voluntary human sterilisation is 25, 35, and even 40 years in some countries (but is apparently just 18 in the US, which is a joke).

I cannot easily find statistics of the minimum age when a single person can adopt a child, but it appears to be 30 years in the US. If the rationale behind this policy was about financial stability only, why rich, single 25 yo's cannot adopt?

I think it's better to compare entering AI relationship with these policies than with drinking alcohol or watching porn or having sex with humans (individual cases of which, for the most part, don't change human lives irreversibly, if practiced safely; and yes, it would be prudent to ban unprotected sex for unmarried people under 25, but alas, such a policy would be unenforceable).

Similarly for the remaining two criteria - to suggest that mental illnesses disqualify someone from being in a romantic relationship I think clearly requires some qualification / justification - it may or may not be the case that one might be a danger to one's partner, but again, what is the comparative likelihood?

I don't think any mental condition disqualifies person from having a human relationship, but I think it shifts the balance in the other direction. E.g., if a person has bouts of uncontrollable aggression and has a history of domestic abuse and violence, it makes much less sense to bar him from AI partners and thus compel him to find new potential human victims (although he/she is not prohibited from doing that, unless jailed).

I interpreted you as arguing that some people should be prevented from having partners, and only then should AI partners be made available to people

No, this is not what I meant, see above.

for what its worth, your comment on porn/onlyfans, etc. doesn't actually stop the whataboutism - you compare AI romance to things nominally in the same reference class, assert that they're bad too, and assert that it has no bearing on your argument that these things are not restricted in the same way. It's fine to bite the bullet and say "I stand by my reasoning above, these should be banned too", but you should argue that that is true explicitly if that's what you think. It is also not a broadly accepted fact that online-dating/porn/onlyfans/etc. constitute a net harm; a well-fleshed out argument above could lead someone to conclude this, but it weakens your argument to merely assert it in the teeth of plausible objections.

All these things are at least mildly bad for society, I think this is very uncontroversial. What is much more doubtful (including for me) is how the effects of these things on individual weigh against their effects on society. The balance may be different for different things and is also different than the respective balance for AI partners.

First, the discussion of the ban of porn is unproductive because it's completely unenforceable.

Online dating is a very complicated matter and I don't want to discuss it here, or anywhere really, especially to "justify" my position about AI partners. There are lots of people on this issue already. But what I would say for sure that the design of the currently dominant online dating systems such as Tinder is very suboptimal, just as the design of the presently dominating social media platforms. There could be healthier designs for online dating systems both for individuals and society than Tinder, but Tinder won because swiping itself is addictive (I tell from first-hand experience here; I'm addicted to swiping on Tinder).

OnlyFans I think is just cancer and should be shut down, I think (e.g., see here; although this particular screen is from Twitch and not OnlyFans, I think OnlyFans is full of this shit, too). It doesn't make individual lives better any more substantially than porn, but has more negative effects on society.

Note that I also don't suggest complete ban of AI partners; but to mostly restrict it for under-30's.

Missing argument pieces: you lack an argument for why higher fertility rates are good, but perhaps more importantly, to whom such benefits accrue (ie how much of the alleged benefit is spillover/externalities).

I thought this is such a table stakes in EA/LessWrong circles it's not worth justifying. Will MacAskill in "What We Owe The Future" have spent many pages arguing for why more people is good and procreation is good. I assumed that most readers of the post either have read this book or have absorbed this positions through other materials. Regardless, even if you don't agree with these conclusions, you can think of this as the assumption that I'm taking in the post (indeed, if we don't care about future unborn lives, why care about the society at all?)

I think you engage this somewhat when you discuss rationale for the severity of the law, but your proposal would require the deliberate denial of a right of free association to certain people - if you think this is okay, you should explicitly state the criterion of severity by which such a decision should be made.

Sorry, I don't understand what "denial of association to certain people" you are talking about here? Or you take "AI partners" as kinds of "people", as in Mitchell Porter's comment [LW(p) · GW(p)]?

If this restriction is ultimately an acceptable cost, why does the state - in particular - have the right / obligation to enforce this, as opposed to leaving it to individual good judgement / families / community / etc?

This is an extremely general regulation and political science question, not specific to the case of AI partners at all. Why don't we defer to individual good judgement/families/community all other questions, such as buying alcohol before 18 (or 21), buying and taking drugs or any other (unchecked) substances, adopting children, etc.? I think fundamentally this is because individuals (as well as their families and communities) often cannot have a good judgement, at least until a certain age. Entering a real loving relationship with an AI is a serious decision that can transform the person and their entire future life trajectory, and I think 18 yo's don't nearly have the capacity and knowledge to make such a decision rationally and consciously.

what about the former incel who is now happily married to his AI gf?

By "incel" you mean a particular subculture, or all people who are failing to find any intimacy in a long time, which is, by the way, 1/3rd of all young men in America? The "young man" from my story belongs to this wider group. Regarding this wider group, my response would be: life with just porn (but no AI partners) is not that bad that we need to rush AI partners in, and a lot of these people will find satisfying human relationships before they are 30. If they embark on AI partnerships, however, I'm afraid they could be "locked in" there and never find satisfying human relationships afterwards.

Of the polycouple that has an AI just because they think its neat?

I didn't think about this case. Off the cuff it sounds OK to me, yes.

You should be more comparative: compare your primary impacts with other possible concerns in terms of magnitude, group-of-incidence (i.e. is it harming groups to whom society has incurred some special obligation to protect?), and severity.

Didn't quite understand what do you mean here.

You also need a stronger support for some assertions on which your case hinges: why should it be so much more likely that an AI is going to fall far short of a human partner? If that's not true, how important is this compared to your points on fertility?

I think it fall short for some people and not others (see this comment [LW(p) · GW(p)]). I don't know what the relative prevalence will be. But anyway, I think it's relatively less important to the fertility point. Life without AI partner is not really suffering, after all, for most people (unless they are really depressed, feel completely unloved, worthless, etc., which I made reservations about). Incidentally, I don't that most people could make a decision to be child-free in full consciousness before they are 30, and found it surprising that the minimum age of vasectomy is 18 (in United States). But after they are 30, I think people could have full freedom to decide they are going to be child-free and live in a relationship with AI partner happily thereafter.

How should these benefits be compared to what a contralocutor would argue occurs in the best case (in likelihood, magnitude, and relative importance)?

Answering this rationally hinges on "solving ethics", which nobody has done, neither me nor my contralocutors (and is likely not possible in principle, if ethics are subjective and constructed all the way down). So ultimately, this will based on vibes-based intuitions about the relative importance of the society and the individual, which I (and my contralocutors) will then find persuasive ways to justify. But this is not the matter of rationality yet, this is a matter of politics, ultimately.

if you believe in short timelines to superintelligence

Due to serial speed advantage of AIs, superintelligence is unnecessary [LW(p) · GW(p)] for making humanity irrelevant within a few years of the first AGIs capable of autonomous unbounded research. Conversely, without such AGI, the impact on society is going to remain bounded, not overturning everything.

Agreed with the first part of your comment (about superintelligence).

On the second part, I think immediately generalising the discussion about the role of "person" in the society at large is preliminary. I think it's an extremely important discussion to have, but it doesn't weigh on whether we should ban healthy under-30's from using AI partners today.

In general, I'm not a carbon chauvinist.

Let's imagine a cyberpank scenario: a very advanced AI partner (superhuman on emotional and intellectual levels) enters a relationship with a human and they decide to have a child, with the help of donor sperm (if the human in the couple is a woman) or with the help of a donor egg and gestation in an artificial womb (if the human in the couple is a man); the sexual orientation of the human or the AI in the couple doesn't matter.

I probably wouldn't be opposed to this. But we can discuss permitting these family arrangements after all the necessary technologies (AIs and artificial wombs) have matured sufficiently. So, I'm not against human--AI relationships in principle, but I think that the current wave of AI romance startups has nothing to do with crafting meaning and societal good.

For those readers who hope to make use of AI romantic companions, I do also have some warnings:

1. You should know in a rough sense how the AI works and the ways in which it's not a human.
   1. For most current LLMs, a very important point is that they have no memory, other than the text they read in a context window. When generating each token, they "re-read" everything in the context window before predicting. None of their internal calculations are preserved when predicting the next token, everything is forgotten and the entire context window is re-read again.
   2. LLMs can be quite dumb, not always in the ways a human would expect. Some of this is to do with the wacky way we force them to generate text, see above.
   3. A human might think about you even if they're not actively talking to you, but rather just going about their day. Of course, most of the time they aren't thinking about you at all, their personality is continually developing and changing based on the events of their lives. LLMs don't go about their day or have an independent existence at all really, they're just there to respond to prompts.
   4. In the future, some of these facts may change, the AIs may become more human-like, or at least more agent-like. You should know all such details about your AI companion of choice.
2. Not your weights, not your AI GF/BF
   1. What hot new startups can give, hot new startups can take away. If you're going to have an emotional attachment to one of these things, it's only prudent to make sure your ability to run it is independent of the whims and financial fortunes of some random company. Download the weights, keep an up to date local copy of all the context the AI uses as its "memory".
   2. See point 1, knowing roughly how the thing works is helpful for this.
   3. A backup you haven't tested isn't a backup.

If the legal system says to people that it knows better than them about which images and words they should be allowed to look at because some images and words are "psychologically harmful", that's pretty suspicious. Humans tend to have pretty good psychological intuition.

This seems demonstrably wrong in the case of technology.

People who would be made happier, or helped to be kinder or more productive people by their AI partners would not get those benefits.

As I mentioned in the post as well as many comments here, how about AI partner startups demonstrating that these positive effects dominate negative effects, rather than vice-versa? Why do we hold addictive psycho-technology (which AI partner tech is, on the face of it, because "lower love" is also a form of addiction, and you cannot yet have a "higher love", i.e., shared fate with AI partners in the present wave, they are not yet learning, conscious, and are not moral patients) to a different standard than new medications?

On the margin, more people would stay in relationships they'd be better off leaving because they fear being alone and don't have the backup option of an AI relationship.

I commented on this logic here [LW(p) · GW(p)], I think this doesn't make sense. People probably mostly stay in relationships they'd be better off leaving either because there is a roller-coaster dynamic they also partially enjoy (in which case knowing there is AI waiting for them outside of the relationship probably doesn't help), or for financial reasons or reasons of convenience. People may fear be physically alone (at home, from the social perspective, etc.), but hardly many people are particularly attached to the notion of being "in a relationship" so much that the perspective of paying 20 bucks per month for their virtual AI friend will be sufficient to quiet their anxiety about leaving a human partner. I cannot exclude this, but I think there are really few people like this.

People who genuinely have no chance of ever finding a romantic relationship would not have a substitute to help make their life more tolerable.

In the policy that I proposed, I made a point for these cases. Basically, a psychotherapist could "prescribe" an AI partner to a human in such cases.

Most of all, such a policy dictates limitations to people as to who/what they should talk to. This is not a freedom that one should lightly curtail, and many countries guarantee it as a part of their constitution. If the legal system says to people that it knows better than them about which images and words they should be allowed to look at because some images and words are "psychologically harmful", that's pretty suspicious. Humans tend to have pretty good psychological intuition. And even if people happen to be wrong about what's good for them in a particular case, taking away their rights should be counted as a very large cost.

AI partners won't be "who" yet. That's a very important qualification. As soon as AIs become conscious, and/or moral patients, of course there should be no restrictions. But without that, in your passage, you can replace "AI partner" or "image" with "heroin" and nothing qualitatively changes. Forbidding building companies that distribute heroin is not "taking out freedoms", even though just taking heroin (or other drug) which you somehow obtained (found on the street, let's say) is still a freedom.

You (and/or other people in the "ban AI romantic partners" coalition) should be trying to gather much more information about how this will actually affect people. I.e. running short term experiments, running long term experiments with prediction markets on the outcomes, etc. We need to know if the harms you predict are actually real. This issue is too serious for us to run a decade long prohibition with a "guess we were wrong" at the end of it.

How do you imagine myself or any other lone concerned voice could master sufficient resources to do these experiments? All AI partners (created by different startups) will be different, moreover, and may lead to different psychological effects. So, clearly, we should adopt a general policy when it's the duty of these startups to demonstrate that they products are safe for psychology of their users, long-term. Product safety regulations are normal, they exist in food, in medicine, in many other fields. Even current nascent AI psychotherapy tools need to pass a sort of clinical trials because they are classified as sort of "medical tools" and therefore are covered by these regulations.

Somehow moving AI romance outside of this umbrella of product categories regulated for safety and taking a complete laissez-faire approach just doesn't make any sense. What if an AI partner created by a certain company really flames up misogyny in men who are using these tools? So far we even assumed, throughout these discussions, that AI partner startups will be "benign" and "good faith", but if we take these assumptions we shouldn't also regulate foods and lots of other things.

Does platonic co-parenting usually involve co-living, or it's more often that parents live separately and either take care of the child on different days, or visit each other's homes to play with kids?

I agree that it probably won't be socially acceptable to admit that you are in love with your AI partner for a time being. Therefore, the young man in my short "mainline scenario" downplays the level of intimacy that he has with his AI partner to his friends. Parents probably won't know at all, their son just "studies at college and doesn't have time for girls". Importantly, the young man may deceive even himself, not consciously perceiving their attitude towards the AI as "love", but nevertheless he may become totally uninterested in seeking romance with humans, or even watching porn (other than the videos generated with the avatar of his AI partner).

I'm not sure about what I've written above, of course, but I definitely think that the burden of proof is on AI startups, cf. this comment [LW(p) · GW(p)]. 

Maybe we understand different things by "emotional intelligence". To me, this is just the ability to correctly infer the emotional state of the interlocutor, based on the context, text messages that they send, and the pauses between messages.

I don't think this requires any breakthroughs in AI. GPT-4 is basically able to do this already, if we take out of the question the task of "baseline adjustment" (different people have different conversational styles, some are cheerful and use smile emojis profusely, others are the opposite, use smileys only to mark strong emotions) and the task of intelligent summarisation of the context of the dialogue. The latter are exactly the types of tasks I expect AI romance tech to be ironing out in the next few years.

Detecting emotions from the video of human face or the recording of their speech is in some ways even simpler, there are apparently already simple supervised ML systems that do this. But I don't expect AI partners to be in a video dialogue with users yet because I don't think video generation will become sufficiently fast for realtime video, yet. So I don't assume that AI will receive the stream of user's video and audio, either.

So, why not pay young women surrogate mother rates for children (about 100k)?  And allow the state to take primary custody so that young women can finish school/start early career/have more children.  (the father would be whoever the woman chooses and would not owe child support conditional on the woman giving up primary custody)

In general, paying people for parenting (I would emphasise this instead of pure "childbirth"), i.e., considering parenting a "job", I think is a reasonable idea and perhaps, soon this will be inevitable in developed countries with plummeting fertility rates and increasing efficiency of labour (the latter due to AI and automation).

The caveat is that the policy that you proposed will cost the government a lot of money initially, while the policy that I proposed costs nothing.

I think it'll be really hard to regulate this in any kind of meaningful way. This is much harder to regulate than drugs or alcohol.

Why do you think so? Even if open-source LLMs and other necessary AI models will develop so quickly that real-time interaction and image generation will be possible on desktops, and there will be open-source waifu projects, mobile is still controlled by app stores (Apple and Google Play) where age-based restrictions could be easily imposed. Jailbreaking/unlocking the phone (or installing an apk from Github), connecting it to a local server with AI models, and maintaining the server is so burdensome that I expect 90+% of potential users of easy-to-access AI partner apps would fall off.

This is not to mention that open-source projects developed by hardcore enthusiasts will probably cater for their specific edgy preferences and not appeal to a wide audience. E.g., an open-source project of anime waifus may optimise to trigger particular fetishes rather overall believability of the AI partner and the long-term appeal of the "relationship". Open-source developers won't be motivated to optimise the latter, unlike AI partner startups, because their lifetime customer value would directly depend on that.  

Other than fertility rate, what other harms are there, and to whom, such that it's any of society's business at all? Are you thinking of it as being like addiction, with people choosing something they (initially) think is good for them but isn't?

Yes, I discussed in this comment [LW(p) · GW(p)] how people could perceive settling for AI partners less wholesome life because they didn't pay their duty to society (regardless of whether this actually matters from your theoretical ethics point of view, if people have deeply held, culturally embedded idea about this in their head, they could be sad or unsettled for real). I don't venture to estimate how prevalent this will be, and therefore how this will weigh against net good for personal satisfaction of people who would have no issue with settling for AI partners whatsoever.

Kaj Sotala suggested [LW(p) · GW(p)] in this comment that this "duty for society" could be satisfied through platonic co-parenting. I think this is definitely interesting, could work for some people, and I think loudable that people do this, but I have doubts how widespread this practice could become. It might be that parenting and romantic involvement with the co-parent are pinned too strongly to each other in many people's minds.

First, though, I don't think the scenario you're proposing is anywhere near as bad for fertility as suggested. [...] And I imagine a lot of people would still want kids with an AI partner as well.

This is the same type of statement that many other people have made here ("people won't be that addicted to this", "people will still seek human partners even while using this thing", etc.), to all of which I reply: it should be AI romance startups' responsibility to demonstrate that the negative effect will be small, not my responsibility to prove that the effect will be huge (which I obviously couldn't do). Currently, it's all opinion versus opinion.

At least the maximum conceivable potential is huge: AI romance startups obviously would like nearly everyone to use their products (just like currently, nearly everyone watches porn, and soon nearly everyone will use general-purpose chatbots like ChatGPT). If the AI partners will be so attractive that about 20% of men are falling for them so hard that they don't even want to date any women anymore through the rest of their lives, we are talking about 7-10% of drop in fertility (less than 20% because not all of these men would counterfactually have kids anyway, also "spare" women could decide to have kids alone, some will platonically co-parent, etc.)

Plenty of real-world people and partners incapable of having biological children together have a sufficiently high desire to do so to go to great lengths to make it happen. Plenty of others want to do so but limit themselves for non-biological reasons, often financial. Society could do a lot more to facilitate them getting what they want, but doesn't.

I agree with all this, and this is all extremely sad. But this seems to be irrelevant to the question of AI partners: if there are other problems that depress fertility rate, it doesn't mean that we shouldn't deal with this upcoming problem. Moreover, while problems like from financial inequality (and precarity), reducing biological fertility of men and women due to stress and environmental pollution, etc., are big systemic problems that are very hard and very expensive to fix, it's currently relatively very cheap to prevent further potential fertility drop resulting from widespread adoption of AI partners by under-30's: just pass a regulation in major countries!

It being a genuine problem would require that no intervention succeed in turning it around [...] It would also require there not be any substantial subsets of the global population that consistently choose biological partners or lager families. 

In my ethics, more conscious observers matter today, it doesn't make a difference from a normative perspective than "later in the future" the population will recover. Also, following this logic, saving people from death (e.g., from malaria) today makes very limited sense, because all it prevents is transient few days of suffering of a person. But really I think EAs value more the "rescued years of mostly happy experience later".

Similarly, the fact that somewhere else in the world some people procreate a lot doesn't somehow make shrinking population in other parts of the world "less bad".

It being a genuine problem would require that [...] no further AI advances lead to AGI sufficient for AIs to "count" as population for the relevant purposes that make population growth desirable. [...]

Third, I think there's a lot of other downstream impacts of a world with sufficiently-good-for-this-to-be-an-issue AI romantic partners that make this less of a concern for society. [...]

I also think this cluster of arguments are not applicable. Following this logic, pretty much nothing matters if we expect world to become unrecognisably weird soon. I also expect this to happen, with very high probability, but in discussing societal impacts of AI, global health and poverty, environmental destruction, and other systemic issues, we have to take a sort of deontological stance and imagine that this weirdness won't happen. If it will happen, all bets are off, anyway. But in discussing current "mundane" global issues, we should condition that weirdness doesn't happen for some reason (which is not totally implausible: global ban on AGI development still could happen, for instance, and I would even support it).

Besides, as I noted in the beginning of my post, I think AI partners based on even today's AI capabilities (LLMs like GPT-4, text-to-speech, text-to-image, etc.) could already be used to make an extremely compelling AI partner, much more attractive than today's AI partners. It's still very early days of these products, but in a few years they will catch up.

Sorry, but this is empty rhetoric and a failure to engage with the post on the object level. How did the printing press jeopardise the reproduction of people in society? And not indirectly, through a long causal chain (such as, without the printing press, we wouldn't have the computer, without the computer we wouldn't have the internet, without the internet we wouldn't have online dating and porn and AI partners, etc.), but directly? Whereas AI partners will reduce the total fertility rate directly.

The implication of your comment is that technology couldn't be "bad" (for people, society, or any other subject), which is absurd. Technology is not ethics-neutral and could be "bad".

Are you referring to the concerns of Conrad Gessner? From Why Did First Printed Books Scare Ancient Scholars In Europe?:

Gessner’s argument against the printing press was that ordinary people could not handle so much knowledge. Gessner demanded those in power in European countries should enforce a law that regulated sales and distribution of books.

If so, I don't understand the parallel you are trying to draw. Prior to the printing press, elites had access to 100s of books, and the average person had access to none. Whereas prior to AI romantic partners, elites and "proles" both have access to human romantic partners at similar levels. Also, I don't think Gessner was arguing that the book surplus would reduce the human relationship participation rate and thus the fertility rate. If you're referring to other "smart people" of the time, who are they?

Perhaps a better analogy would be with romance novels? I understand that concerns about romance novels impacting romantic relationships arose during the 18th and 19th centuries, much later.

Aside: I was unable to find a readable copy of Conrad Gessner's argument - apparently from the preface of the Bibliotheca Universalis - so I am basing my understanding of his argument on various other sources.

Is this comment a question whether these assumptions are in my post?

Have kids when we can have AGI in 10-25 years is good and not, actually, very evil OMG what are you doing.

You don't buy that having kids [when we can have AGI soon...] is good, right? OK, I disagree with that strongly, population-wise.  Do you imply that the whole planet should stop having kids because we are approaching AGI? Seems like a surefire way to wreck the civilisation even if the AI alignment problem will appear simpler than we think or miraculously solved. Will MacAskill also argues against this position in "What We Owe The Future".

Specifically for people who directly work in AI safety (or have intellectual capacity to meaningfully contribute to AI safety and other urgent x-risk priorities, and consider doing so), this is a less clear-cut case, I agree. This is one of the reasons for which I'm personally unsure whether I should have kids.

Right social incentives can't make A LOT of people poly pretty fast.

There was no such assumption. The young man in my "mainline scenario" doesn't have this choice, alas. He has no romantic relationships with humans at all. Also, I'm afraid that AI partners will soon become sufficiently compelling that after prolong relationships with them, it will be hard for people to summon motivation to court average-looking (at best), shallow, and dull humans who don't seem to be interested in this relationship themselves.