Preference conditional on circumstances and past preference satisfaction
post by Stuart_Armstrong · 2019-06-17T15:30:32.580Z · LW · GW · 1 commentsContents
1 comment
I've mentioned conditional preferences before [LW · GW]. These are preferences that are dependent on facts about the world, for example "I'd want to believe X if there are strong argument for X".
But there is another type of preference that is conditional: my tastes can vary depending on circumstances and on my past experience. For example [LW(p) · GW(p)], I might prefer to eat apples during the week and oranges on weekends. Or, because of the miracle of boredom [LW · GW], I might prefer oranges if (but only if) I've been eating apples all week so far.
What if I currently want apples, would want oranges tomorrow, but falsely believe (today) that I would want apples tomorrow? This is a known problem with "one-step hypotheticals [LW · GW]", and a strong argument in practice for assessing preferences over time rather than at a single moment .
In theory, there are meta-preferences that allow one to get this even at a single moment , such as "I want to be able to follow my different tastes at different times" or a more formalised desire for variety and exploration.
1 comments
Comments sorted by top scores.
comment by Dagon · 2019-06-17T17:47:30.570Z · LW(p) · GW(p)
I strongly suspect those meta-preferences are both critical for correct extrapolation of human values/preferences, AND are the place where we'll find a fair bit of actual inconsistency of human desires.
"I want to be able to follow my illegible whims" seems like a very common and strong meta-preference, and I haven't seen it modeled well in any discussions.