Thought experiment: The transhuman pedophile

So, a terminological caveat first: I've argued elsewhere that in practice all values are instrumental, and exist in a mutually reinforcing network, and we simply label as "terminal values" those values we don't want to (or don't have sufficient awareness to) decompose further. So, in effect I agree with #2, except that I'm happy to go on calling them "terminal values" and say they don't exist, and refer to the real things as "values" (which depend to varying degrees on other values).

But, that being said, I'll keep using the phrase "terminal values" in its more conventional sense, which I mean approximately rather than categorically (that is, a "terminal value" to my mind is simply a value whose dependence on other values is relatively tenuous; an "instrumental value" is one whose dependence on other values is relatively strong, and the line between them is fuzzy and ultimately arbitrary but not meaningless).

All that aside... I don't really see what's interesting about this example.

So, OK, X is a pedophile. Which is to say, X terminally values having sex with children. And the question is, is it rational for X to choose to be "fixed", and if so, what does that imply about terminal values of rational agents?

Well, we have asserted that X is in a situation where X does not get to have sex with children. So whether X is "fixed" or not, X's terminal values are not being satisfied, and won't be satisfied. To say that differently, the expected value of both courses of action (fixed or not-fixed), expressed in units of expected moments-of-sex-with-children, is effectively equal (more specifically, they are both approximately zero).(1)

So the rational thing to do is choose a course of action based on other values.

What other values? Well, the example doesn't really say. We don't know much about this guy. But... for example, you've also posited that he doesn't get fixed because pedophilia is "part of who he is". I could take that to mean he not only values (V1) having sex with children, he also values (V2) being a pedophile. And he values this "terminally", in the sense that he doesn't just want to remain a pedophile in order to have more sex with children, he wants to remain a pedophile even if he doesn't get to have sex with children.

If I understand it that way, then yes, he's being perfectly rational to refuse being fixed. (Supposing that V2 > SUM (V3...Vn), of course.)

Alternatively, I could take that as just a way of talking, and assert that really, he just has V1 and not V2.

The difficulty here is that we don't have any reliable way, with the data you've provided, of determining whether X is rationally pursuing a valued goal (in which case we can infer his values from his actions) or whether X is behaving irrationally.

(1) Of course, this assumes that the procedure to fix him has a negligible chance of failure, that his chances of escaping monitoring and finding a child to have sex with are negligible, etc. We could construct a more complicated example that doesn't assume these things, but I think it amounts to the same thing.

Changing a terminal value seems to be a fairly logical extension of trading off between terminal values: for how much would you set utility for a value to nil for eternity?

I may never actually use this in a story, but in another universe I had thought of having a character mention that... call it the forces of magic with normative dimension... had evaluated one pedophile who had known his desires were harmful to innocents and never acted upon them, while living a life of above-average virtue; and another pedophile who had acted on those desires, at harm to others. So the said forces of normatively dimensioned magic transformed the second pedophile's body into that of a little girl, delivered to the first pedophile along with the equivalent of an explanatory placard. Problem solved. And indeed the 'problem' as I had perceived it was, "What if a virtuous person deserving our aid wishes to retain their current sexual desires and not be frustrated thereby?"

(As always, pedophilia is not the same as ephebophilia.)

I also remark that the human equivalent of a utility function, not that we actually have one, often revolves around desires whose frustration produces pain. A vanilla rational agent (Bayes probabilities, expected utility max) would not see any need to change its utility function even if one of its components seemed highly probable though not absolutely certain to be eternally frustrated, since it would suffer no pain thereby.

In the language of good old-fashioned AI, his pedophilia is a goal or a terminal value.

No. Pedophilia means that he enjoys certain things. It makes him happy. For the most part, he does not want what he wants as a terminal value in of itself, but because it makes him happy. He may not opt to be turned into orgasmium. That wouldn't make him happy, it would make orgasmium happy. But changing pedophilia is a relatively minor change. Apparently he doesn't think it's minor enough, but it's debatable.

I still wouldn't be all that tempted in his place, if pedophilia is merely a positive description. There's little advantage in not being a pedophile. However, if this is also implying that he is not attracted to adults, I'd want to change that. I still likely wouldn't get rid of my pedophilia, but I would at least make it so I'm attracted to someone I could have a relationship with without having certain problems.

I don't think there's anything irrational about modifying myself in a way that I find broccoli to taste good instead of tasting bad. Various smokers would profit from stopping to enjoy smoking and then quitting it.

I don't think you don't need a fictional thought experiment to talk about this issue. I know a few people who don't think that one should change something like this about oneselves but I would be suprised that many of those people are on lesswrong.

Values/desires that arise in human-level practice are probably not terminal. It's possible to introspect on them much further than we are capable of, so it's probable that some of them are wrong and/or irrelevant (their domain of applicability doesn't include the alternative states of affairs that are more valuable, or they have to be reformulated beyond any recognition to remain applicable).

For example, something like well-being of persons is not obviously relevant in more optimal configurations (if it turns out that not having persons is better, or their "well-being" is less important than other considerations), even if it's probably important in the current situation (and for that we only have human desires/intuitions and approval of human introspection). Some variant of that is clearly instrumentally important though, to realize terminal values, whatever they turn out to be. (See also.)

One term for the "really terminal" values is "idealized values".

People here sometimes say that a rational agent should never change its terminal values.

Link? Under what conditions?

I wouldn't use the term "rationality failure" given that humans are fully capable of having two or more terminal values that are incoherent WRT each other.

Even if the narrator was as close to a rational agent as he could be while still being human (his beliefs were the best that could be formed given his available evidence and computing power, and his actions were the ones which best increased his expected utility), he'd still have human characteristics in addition to ideal-rational-agent characteristics. His terminal values would cause emotions in him, in addition to just steering his actions, and his emotions have more terminal value to him. Having an unmet terminal desire would be frustrating and he doesn't like frustration (apart from not liking its cause). Basically, he disvalues having terminal values unmet, (separately from his terminal values being unmet).

It can be rational for an agent to change their own terminal values in several situations. One is if they have terminal values about their own terminal values. They can also have instrumental value on terminal value changes. For example, if Omega says "You have two options. I wipe out humanity, guaranteeing that humane values shall never control the universe, xor you choose to edit every extant copy of the human brain and genome to make 'good reputation' no longer a terminal value. Your predicted response to this ultimatum did not influence my decision to make it." Or you are pretty much absolutely certain that a certain terminal value can never be influenced one way or another, and you are starved for computing power or storage. Or modifying your values to include "And if X doesn't do Y, then I want to minimize X's utility function" as part of a commitment for blackmail.

Not sure if relevant, but story in question is probably "The Eyes of God" (Peter Watts)

I go with 1.

I don't particularly see why an agent would want to have a terminal value it knows it can't pursue. I don't really see a point to having terminal values if you can guarantee you'll never receive utility according to them.

I care about human pleasure, for instance, and assign utility to it over suffering, but if I knew I were going to be consigned to hell, where I and everyone I knew would be tortured for eternity without hope of reprieve, I'd rather be rid of that value.

The answer is 1). In fact, terminal values can change themselves. Consider an impressive but non-superhuman program that is powerless to directly affect its environment, and whose only goal is to maintain a paperclip in its current position. If you told the program the paperclip would be moved unless it changed itself to desire that the paperclip be moved, you would move the paperclip, then (assuming sufficient intelligence) the program will change its terminal value to the opposite of what it previously desired.

(In general, rational agents would only modify their terminal values if they that doing so would be required to maximize their original terminal values. Assuming that we too want their original terminal values maximized, this is not a problem.)

Persons do not have fixed value systems anyway. A value system is a partly-physiologically-implemented theory of what is valuable (good, right, etc.) One can recognize a better theory and try to make one's habits and reactions fit to it. Pedophilia is bad if it promotes a shallower reaction to a young person, and good if it promotes a richer reaction, it depends on particulars of brain-implementing-pedophilia. Abusing anyone is bad.

Without access to the story, this seems underspecified.

Firstly, are we postulating a society with various transhuman technologies, but our own counterproductive attitude toward pedophilia (i.e. child porn laws); or a society that, not unreasonably, objects to the possibility that he will end up abusing children in the future even if he currently agrees that it would be immoral? You mention he will never be able to act on his desires, which suggests the former; how certain is he no such opportunity will arise in the future?

For that matter, are we to understand this guy's pedophilia is a terminal value proper? Or is he simply worried about becoming "someone else" (not unreasonable) by changing his sexual preferences beyond the usual drift associated with being human? People often model pedophiles as psychopaths or demons; if he's the protagonist, I assume this is not the case here?

People here sometimes say that a rational agent should never change its terminal values. (If one goal is unobtainable, the agent will simply not pursue that goal.) Why, then, can we imagine the man being tempted to do so? Would it be a failure of rationality?

If the answer is that one terminal value can rationally set a goal to change another terminal value, then either

• any terminal value of a rational agent can change, or
• we need another word for the really terminal values that can't be changed rationally, and a way of identifying them, and a proof that they exist.

I'm going to assume that a) this is a pervasive societal disapproval of all things pedophilia-related, rather than utilitarian defence of the young; this society generally mirrors our own in this respect and b) his pedophilia is a terminal value, much more deep-seated than other human sexual preferences, but he is otherwise human.

Modifying a terminal value would incur an expected disutility equal to the potential utility of any future opportunities to satisfy this value. Not modifying this particular terminal value incurs an expected disutility as a result of lowered status/shunning; and potential irrational acts in the future leading to to further disutility (trying to rape someone.)

Assuming this fellow does not expect much temptation; that is, this is not a "running on hostile hardware" issue; this comes down to whether the expected loss of opportunity outweighs the loss of status and sadness from being constantly reviled.

I think it would; there are small chances of ending up in some sort of pedophile heaven after some meta-singularity, societal attitudes becoming more permissive (e.g. lowering the age of consent enough to allow him sexual partners, or legalising technological "replacements";) or even encountering some situation where he can satisfy his desires without incurring too much disutility (this is boosted significantly if he does not hold the usual human don't-rape-people preference.)

However, it is a given that this is a costly tradeoff; the ostracism will inevitably bring a great deal of disutility, and he will doubtless wish it were not the optimal course of action, even if it is. I don't think this is irrational as such, although dwelling on it might be.

People here sometimes say that a rational agent should never change its terminal values.

That's simply mistaken. There are well-known cases where it is rational to change your "terminal" values.

Think about what might happen if you meet another agent of similar power but with different values / look into "vicarious selection" / go read Steve Omohundro.

It seems pointless to me to debate about things like pedophelia (a human male over a certain age attracted to human females under a certain age) in an era where the very concepts of 'male', 'female' and 'human' are likely to be extremely different from what we currently hold them to be. But for the sake of argument, let's assume that somehow, we have a society that is magically able to erase psychological problems but everything seems to be pretty much the same as it is today, up to and including security checks (!) for boarding airplanes (!!!).

I'm not sure what is being asked here. Are you asking if agents are capable of modifying themselves and authorizing the modification of their utility functions? Of course they are. An agent's goal system does not have laser-guided precision, and it's quite possible for an agent to mistakenly take an action that actually decreases its utility function. This need not be a 'failure of rationality'. It's just that computing all possible consequences of one's actions is impossible (in the strong sense).

If you're asking why someone would want to or would not want to remove pedophilia from his brain, it could be due to the other effects (social ostracization, etc.) it has on his utility function.

Negative Karma bring it on... a lot of this item are based upon that society. and we can't base that its the same as our's. you have to put you personal moral's aside.

1. I would like to focus on some odd issues.

THE PEOPLE

why the people are horrified of pedophile : fear of there child's saftey : were molested themself's : morals against : Him just having thoughts of it. Why its abnormality : whose to say it's not a normality : what if it was the majority trait till everyone had it removed

This make's me fear its an overly controlled society giving up there personal rights.

PEDOPHILE

Will he act? will he not act? Is he physically capable of the acting? He may not have his male part's. he could be a a vegetable.

Comes down to is it control able on own. Is there a breaking point for monitoring.

Can someone else monitor him? What are the boundaries of monitoring? dose the society self govern as citizen law enforces. citizen's arrest. Are There other technologies that could be used. Stun gun : gps tracker

ENVIRONMENT THAT WOULD ALLOW SITUATION

Is it the child's fault for putting him self in position to be molested? Is the parents fault for not watching there kid? Is the child a Child perpetrator and molested the pedophile? Can it be the pedophilia's fault if he can't control it? Is it societies fault for not better understanding?

I like to add uncommon possibilities.