Utility is not the selection target

post by tailcalled · 2023-11-04T22:48:20.713Z · LW · GW · 1 comments

Contents

  Sometimes greenbrownness is the selection target
  Sometimes disutility is the selection target
  Sometimes the superficial appearance of utility is the selection target
  Conclusion
None
1 comment

Epistemic status: shitpost.

Suppose that you are selecting for utility. Naively, you might think that this means you are selecting for utility, but actually this is not the case.

Sometimes greenbrownness is the selection target

Military gear is sort of greenbrownish.

This occurs because the designers of military gear are selecting for keeping soldiers alive, which benefits from camouflage, which in the common environments works best if it is greenbrownish. However, it fails at this in nongreenbrownish environments. Hence, keeping soldiers alive is not the selection target of military gear.

Sometimes disutility is the selection target

In the prisoner's dilemma, the highest-utility outcome is (cooperate, cooperate).

However, utility maximizers will end up in (defect, defect), which is strictly lower utility than (cooperate, cooperate). Thus, utility maximization might have utility minimization as the selection target.

Sometimes the superficial appearance of utility is the selection target

Let's say that you see a website banner saying "You are the 1000000000th visitor to the site. Click to receive your award!". This looks like something people would say if they want to give you something, so you click the banner.

This does not end well for you, but you went into it selecting it because it looked like it would end well for you. 

Conclusion

Beware about reasoning about utility maximizers as maximizing utility. Utility maximizers may instead be maximizing many other things that are unrelated to utility, and not be maximizing utility.

1 comments

Comments sorted by top scores.