LessWrong 2.0 Reader

I don't actually personally agree with Bayesian decision theory anymore and am currently inclined to treat value more like an objective fact about the world than as an individual preference. The provocative position would be a Beff-like one that value = entropy, but while that is an incremental improvenent on utilitarianims/value = negentropy, it is hellish and therefore I can't endorse it fully.

Regardless of the issues with my own position, I'm confused about your worldview. Do you not have a distinction between expected and actual utility, or do you consider there to be two different kinds of changes in values? How do you model value of information? (If you do model it, that is.)

Thank you very much for the criticism. Jeremias: I genuinely appreciate it. Please allow me to provide some context before commenting. Professor W.J. Wukmir developed the orectic theory, which I used in this post to provide a theory of emotions. The man in question lived in Barcelona, where he arrived around 1960 and was stateless until his death in 1981. Unfortunately, his theories have been lost to the course of time. A few months ago, the last of his disciples passed away. I'm afraid his theory will be forgotten in some second-hand bookstore as a rarity. His disciples said that in his youth (in the early 1930s), Wukmir had intense disputes with Freud and Adler in the Vienna circle to which the three belonged. Freud saw him as a revolutionary, while Wukmir believed that Freud was unable to understand him. I apologize for writing this preview, but my purpose was only to offer light on the origins of this theory.

The critique raises some important issues that should be carefully considered. However, I believe there are some fundamental misunderstandings concerning the nature and purpose of emotions as described by Wukmir's orectic theory, which I will address. I apologize for not being more clear in explaining them. 
Emotions, according to this theory, are not just "shortcut heuristics" but also essential systems of vital orientation. Emotions are more than just quick evaluations in the absence of detail; they are complex, multidimensional evaluations that incorporate multiple aspects of an organism's internal state, external environment, and previous experiences. This valuation process takes place at all levels of an organism, from individual cells to complex brain systems. The goal here is to emphasize the relevance of the "valuation" process rather than focusing solely on the emotions involved. 

Your critique implies a distinction between emotion and cognition, which my argument explicitly rejects. According to the orectic theory, emotion and cognition are inextricably related parts of the same process of vital orientation. Every cognitive act entails emotional valuation, and every emotional response contains cognitive components. This integration is not a "ghost in the machine," but rather an essential component of how living systems process information and make decisions. This is what we require from AI: process information and make decisions.

Your claim that AI can make decisions without emotions ignores the importance of emotions in decision-making, according to orectic theory. It's probably my fault for not explaining it more concisely. Emotions are not an add-on to decision-making; they are essential to the process of analyzing possibilities and deciding courses of action. According to this viewpoint, assigning meaning and value to stimuli and prospective actions is an emotional process.

The notion that ethics can be "arbitrarily programmed" without emotional capacity misses the importance of emotions in moral judgment. In my opinion, ethical decisions are fundamentally dependent on emotional valuations of situations and the resulting outcomes. A strictly logical system of ethics without emotional components would lack the critical ability to attach meaning and value to different outcomes.
Your argument appears to presuppose a restricted definition of intelligence that focuses on information processing and decision-making in a mechanical sense. The theory I described, as well as many modern perspectives in cognitive research, advocate for a more embodied and embedded conception of intelligence that must include emotional processes. From this standpoint, a "fully functioning artificial intelligence" would have to have emotion-like mechanisms.

While it is true that more precise prompts may elicit different answers, the purpose of the GPT-4o example was to demonstrate the limitations of simply language-based AI in grasping emotional context. Human communication is highly reliant on emotional understanding that extends beyond the literal meaning of words, and this understanding is rooted in shared embodied experiences.
 

Finally, the notion that language can completely compensate for the lack of nonverbal emotional cues overlooks the embodied character of emotional cognition. In my opinion, our emotional knowledge is largely based on our bodily experiences and cannot be completely reduced to linguistic descriptions.

That is just replacing the idea of fixed values with a fixed utility function. But it is just as changeable whatever you call it.

Show me your utility function before you were born.

In Bayesian decision theory, there's the distinction between expected utility, which changes as one learns about the environment, and actual utility, which does not. Under this frame, I'd be inclined to round you off to using the words "values"/"liking"/etc. to refer to expected utility. Would you agree with that? If not, why not?

It might be tempting to round the OP off to use the word "values"/"ought" to refer to actual utility, but the details of that are kind of awkward at the edges so I would hold off on that.

The discord links seem to time out. Here's a new one: https://discord.gg/D5ECfZ47

I'll be wearing a purple 'Roboflow' hat and holding an ACX sign. If you can't find us, I'll be routinely checking the discord and my email on the day ( Linaskondrackis@gmail.com); feel free to ping me if you're around.

I like what you’re doing trying to do here. I think this is important work.

I’m a bit confused at what you mean by Layperson though? These are good for the ‘every day’ above average intelligence ‘switched on’ type of individual.

But that is not what I image a Layperson as. I interact regularly with ~100 people. (For context, I am a Drama Teacher and Trivia Host)

I thought about how many I predict could understand these examples, given 20 seconds of their attention. I thought of 10 people. The other 90% would fall into a few other categories that all end with them not being more knowledgeable after coming across the text.

But am I confused? Was that 90% not the target audience?

The core claim in this post is that our brains model the world as though there's a thing called "our values", and tries to learn about those values in the usual epistemic way.

I find that a very strange idea, as strange as Plato’s Socrates’ parallel idea that learning is not the acquisition of something new, but recollection of what one had forgotten.

If I try X, anticipating that it will be an excellent experience, and find it disappointing, I have not learned something about my values, but about X.

I have never eaten escamoles. If I try them, what I will discover is what they are like to eat. If i like them, did I always like them? That is an unheard-falling-trees question.

If I value a thing at one period of life and turn away from it later, I have not discovered something about my values. My values have changed. In the case of the teenager we call this process “maturing”. Wine maturing in a barrel is not becoming what it always was, but simply becoming, according to how the winemaker conducts the process.

But people have this persistent illusion that how they are today is how they always were and always will be, and their mood of the moment is their fundamental nature, despite the evidence of their own memory.

At a human level, the counts for each type of atom is basically always conserved too, so it's not just a question of why not momentum but also a question of why not moles of hydrogen, moles of carbon, moles of oxygen, moles of nitrogen, moles of silicon, moles of iron, etc..

I guess for momentum in particular, it seems reasonable why it wouldn't be useful in a thermodynamics-style model because things would woosh away too much (unless you're dealing with some sort of flow? Idk). A formalization or refutation of this intuition would be somewhat neat, but I would actually more wonder, could one replace the energy-first formulations of quantum mechanics with momentum-first formulations?

Why don't more people seek out and use talent scouts/headhunters? If the ghost jobs phenomenon is substantial, that's a perfect use case. Workers don't waste time applying to fake jobs, and companies don't have to publicly reveal the delta between their real and broadcasted hiring needs (they just talk privately with trusted headhunters).

Are there not enough headhunters? Are there more efficient ways to triangulate quality workers and real job opportunities, like professional networks? Are ghost jobs not that big of a deal? Do people in fact use headhunters quite a lot?

I think the simple solution is to not talk about logical tautologies and contradictions when expressing the Kolmogorov axioms for a theory of subjective Bayesianism. Instead talk about what we actually know a priori, not about tautologies which we merely could know a priori (if we were logically omniscient).

Yes, absolutely. When I apply probability theory it should represent my state of knowledge, not state of knowledge of some logically omniscient being. For me it seems such an obvious thing that I struggle to understand why it's still not a standard approach.

So are there some hidden paradoxes of such approach that I just do not see yet? Or maybe some issues with formalization of the axioms?