3. Premise three & Conclusion: AI systems can affect value change trajectories & the Value Change Problem

post by Nora_Ammann · 2023-10-26T14:38:14.916Z · LW · GW · 4 comments

Contents

  Premise three: AI systems can affect value change trajectories
  Putting things together: the Value Change Problem
    Brief excursion: Directionality of Fit
None
4 comments

In this post, I introduce the last of three premises—the claim that AI systems are (and will become increasingly) capable of affecting people’s value change trajectories. With all three premises in place, we can then go ahead articulating the Value Change Problem (VCP) in full. I will briefly recap the full account, and then give an outlook on what is yet to come in post 4 and 5, where we discuss the risks that come from failing to take VCP seriously.

Premise three: AI systems can affect value change trajectories

The third and final premise required to put together the argument for the Value Change Problem is the following: AI systems are (and will become increasingly) capable of affecting people’s value change trajectories.

I believe the case for this is relatively straightforward. In the previous post, we have seen several examples of how external factors (e.g. other individuals, societal and economic structures, technology) can influence an individual's trajectory of value change, and that they can do so in ways that may or may not be legitimate. The same is true for AI systems. 

Value change typically occurs as a result of moral reflections/deliberation, or learning of new information/making new experiences. External factors can affect these processes—e.g. by affecting what information we are exposed to, by biasing our reflection processes towards some rather than other conclusions,etc.—, thereby influencing an individual's trajectory of value change. AI systems are another such external factor capable of similar effects. Consider for example the use of AI systems in media, advertisement or education, as personal assistants, to help with learning or decision making, etc. From here, it’s not a big step to recognise that, with the continued increasing in capabilities and deployment of these systems, the overall effect AI systems might come to have over our value change trajectories.

Posts 4 and 5 will discuss all of this in more detail, including by proposing specific mechanisms by which AIs can come to affect value change trajectories, as well as the question when they are and aren’t legitimate. 

As such, I will leave discussing of the third premise and this and swiftly move on to putting together the full case for the Value Change Problem: 

Putting things together: the Value Change Problem

Let us recap the arguments so far. First, I have argued that human values are malleable rather than fixed. In defence of this claim, I have argued that humans typically undergo value change over the course of their lives; that human values are sometimes uncertain, underdetermined or open-ended, and that some ways in which humans typically deal with this involves value change; and, finally, that transformative experiences (as discussed by Paul (2014)) and aspiration (as discussed by Callard (2018)), too, represent examples of value change. 

Next, I have argued that some cases of value change can be (il)legitimate. In support of this claim, I have made an appeal to intuition by providing examples of cases of value change which I argue most people would readily accept as legitimate and illegitimate, respectively. I then strengthened the argument by proposing a plausible evaluative criteria—namely, the degree of self-determination involved in the process of value change—which lends further support and rational grounding to our earlier intuition. 

Finally, I argued that AI systems are (and will become increasingly) capable of affecting people’s value change trajectories. (While leaving some further details to posts 4 and 5.)

Putting these together, we can argue that ethical design of AI systems must be taken seriously and find ways to address the problem of (il)legitimate value change. In other words, we ought to avoid building AI systems that disrespect or exploit the malleability of human values, such as by causing illegitimate value changes or by preventing legitimate ones. I will refer to this as the ‘Value Change Problem’.

What does it mean for AI design to take the problem of (il)legitimate value change seriously?  Concretely, it means that ethical AI design has to try to i) understand the ways in which AI systems do or can cause value change, ii) understand when a case of value change is legitimate or illegitimate and iii) build systems that do not cause illegitimate value change, and permit (or enable) legitimate value change.

In the remaining two posts, I will discuss in some more depth the risks that may result from inadequately addressing the VCP. This gives raise to two types of risks: risks from causing illegitimate value change, and risks from preventing legitimate value change. For each of these I want to ask: What is the risk? What are plausible mechanisms by which these risks manifest? What are ways in which these risks manifest already today, and what are the ways in which they are likely to be exacerbated going forward, as AI systems become more advanced and more widely deployed?

In the first case—risks from causing illegitimate value change—, leading with the example of recommender systems today, I will argue that performative predictors can come to affect that which they set out to predict—among others, human values. In the second case—risks from preventing legitimate value change—, I will argue that value collapse—the idea that hyper-explication of values tends to weaken our epistemic attitudes towards the world and our values—can threaten the possibility of self-determined and open-ended value exploration and, consequently, the possibility of legitimate value change. In both cases, we should expect (unless appropriate countermeasures are taken) the same dynamic to be exacerbated—both in strength and scope—with the development of more advanced AI systems, and their increasingly pervasive deployment.

Brief excursion: Directionality of Fit

 A different way to articulate the legitimacy question I have described here is in terms of the notion of ‘Directionality of Fit’. In short, the idea is that instead of asking whether a given case of value change is (il)legitimate, we can ask which ‘direction of fit’ ought to apply. Let me explain. 

Historically, ‘directionality of fit’ (or ‘direction of fit’) was used to refer to the distinction between values and beliefs. (The idea came up (although without mentioning the specific term) in Anscombe’s Intention  (2000) and was later discussed by Searl (1985) and Humberstone (1992).) According to this view, beliefs are precisely those things which change to fit the world, while values are those things which the world should be fitted to. 

However, once one accepts the premise that values are malleable, the ‘correct’ (or desirable) direction of fit ceases to be clearly defined. It raises the question of when exactly values should be used as a template for fitting the world to them, and when it is acceptable or desirable for the world to change the values. If I never accept the world to change my values, I forgo any possibility for value replacement, development or refinement. However, as I've argued in part before and will discuss in some more detail in post 5, I might reason to consider myself morally harmed if I lose that ability to freely undergo legitimate value change. 

Finally, this lens also makes more salient the intricate connection between values and beliefs: the epistemic dimensions of value development, as well as the ways values affect our epistemic attitudes and pursuits.

4 comments

Comments sorted by top scores.