Distinctions when Discussing Utility Functions
post by ozziegooen · 2024-03-09T20:14:03.592Z · LW · GW · 7 comments7 comments
Comments sorted by top scores.
comment by Mateusz Bagiński (mateusz-baginski) · 2024-03-10T14:30:02.123Z · LW(p) · GW(p)
I like this as a step towards deconfusion/explicitness/conceptual clarity.
One concern I have is something like this: to the extent that (1) our decision-making computations are not like what you call (AFAICT) operational utility function optimization; and (2) the "native format" of our preferences/values is different from a utility function; [granted, both are empirical and testable in principle, though it may be difficult] to that extent, an estimated utility function is a practical abstraction that obscures the lower-level machinery/implementational details. Then, when our preferences change, framing it in terms of a "change in some kind of utility function" is like saying that "a body's center of mass moved" without referencing the actual body.
Joe Carlsmith discussed related issues recently in this essay.
An important [claim made]/[possibility raised] in that essay (that I think should be brought up more explicitly here) is that different "deliberation/idealization procedures" may produce very different results and never converge in the limit.
Replies from: ozziegooen↑ comment by ozziegooen · 2024-03-10T16:19:21.980Z · LW(p) · GW(p)
an estimated utility function is a practical abstraction that obscures the lower-level machinery/implementational details
I agree that this is what's happening. I probably have different intuitions regarding how big of a problem it is.
The main questions here might be something like:
- Is there any more information about the underlying system, besides its various utility function, useful for decision-making?
- If (1) is false, can we calibrate for that error when trying to approximate things with the utility function? If we just use the utility function, will be be over-confident, or just extra (and reasonably) cautious?
- In situations where we don't have models of the underlying system, can utility function estimates be better than alternatives we might have?
My quick expected answers to this:
- I think for many things, utility functions are fine. I think these are far more precise and accurate than other existing approaches that we have today (like, people intuitively guessing what's good for others) .
- I think if we do a decent job, we can just add extra uncertainty/caution to the system. I like to trust future actors here to not be obviously stupid in ways we could expect.
- As I stated before, I don't think we have better tools yet. I'm happy to see research into more work in understanding the underlying systems, but in the meantime, utility functions seem about as precise and information-rich as anything else we have.
is that different "deliberation/idealization procedures" may produce very different results and never converge in the limit.
Agreed. This is a pretty large topic, I was trying to keep this essay limited. My main recommendation here was to highlight the importance of deliberation and potential deliberation levels, in part to better discuss issues like these.
comment by Mateusz Bagiński (mateusz-baginski) · 2024-03-10T14:03:19.163Z · LW(p) · GW(p)
So, if a person spent a research effort equivalent to “1,000 hours of quality-adjusted research” on estimating their utility function, the result would be a D4 function.
Shouldn't this be "D3"? If I understand correctly: D0 is 1 hour, D1 is 10 hours, D2 is 100 hours, D3 is 1000 hours and D4 is 10,000 hours.
Replies from: ozziegooen↑ comment by ozziegooen · 2024-03-10T16:19:36.703Z · LW(p) · GW(p)
Good point, fixing!
comment by the gears to ascension (lahwran) · 2024-03-09T20:45:19.584Z · LW(p) · GW(p)
this seems confused to me. are we talking about utility functions over worldlines, which I take to be the default meaning of "utility function", or utility functions over ... something else? if something else, is it still justifiable to call it a utility function, rather than a value function?
Replies from: ozziegooen↑ comment by ozziegooen · 2024-03-09T20:54:47.975Z · LW(p) · GW(p)
Do you have a preferred distinction of value functions vs. utility functions that you like, and ideally can reference? I'm doing some investigation now, and it seems like the main difference is the context in which they are often used.
My impression is that Lesswrong typically uses the term "Utility function" to mean a more specific thing than what economists do. I.E. the examples of utility functions in economics textbooks. https://brilliant.org/wiki/utility-functions/ has examples.
They sometimes describe simple things like this simple relationship, as a "utility function".
↑ comment by the gears to ascension (lahwran) · 2024-03-09T21:13:56.559Z · LW(p) · GW(p)
actually, I'm not finding anything to back my cached assumption that utility functions over worldlines are the default, and without that I'm not sure there's a coherent difference between utility and value functions.