LessWrong 2.0 Reader

Similarly in Baba is You: when people don't have a crisp understanding of the puzzle, they tend to grasp and straws and motivatedly-reason their way into accepting sketchy sounding premises [LW · GW]. But, the true solution to a level often feels very crisp and clear and inevitable. 

A few of the scientists I’ve read about have realized their big ideas in moments of insight (e.g., Darwin for natural selection, Einstein for special relativity). My current guess about what’s going on is something like: as you attempt to understand a concept you don’t already have, you’re picking up clues about what the shape of the answer is going to look like (i.e., constraints). Once you have these constraints in place, your mind is searching for something which satisfies all of them (both explicitly and implicitly), and insight is the thing that happens when you find a solution that does.

At least, this is what it feels like for me when I play Baba is You (i.e., when I have the experience you’re describing here). I always know when a fake solution is fake, because it’s really easy to tell that it violates one of the explicit constraints the game has set out (although sometimes in desperation I try it anyway :p). But it’s immediately clear when I've landed on the right solution (even before I execute it), because all of the constraints I’ve been holding in my head get satisfied at once. I think that’s the “clicking” feeling.

Darwin’s insight about natural selection was also shaped by constraints. His time on the Beagle had led him to believe that “species gradually become modified,” but he was pretty puzzled as to how the changes were being introduced. If you imagine a beige lizard that lives in the sand, for instance, it seems pretty clear that it isn’t the lizard itself (its will) which causes its beigeness, nor is it the sand that directly causes the coloring (as in, physically causes it within the lizards lifetime). But then, how are changes introduced, if not by the organism, and not by the environment directly? He was stuck on this for awhile, when: “I can remember the very spot in the road, whilst in my carriage, when to my joy the solution occurred to me.”

There’s more going on to Darwin’s story than that, but I do think it has elements of the sort of thing you're describing here. Jeff Hawkins also describes insight as a constraint satisfaction problem pretty explicitly (I might’ve gotten this idea from him), and he experienced it when coming up with the idea of a thousand brains.

Anyway, I don’t have a strong sense of how crucial this sort of thing is to novel conceptual inquiry in general, but I do think it’s quite interesting. It seems like one of the ways that someone can go from a pre-paradigmatic grasping around for clues sort of thing to a fully formed solution.

Thinking further, a key part of it is that temperature has a tendency to mix stuff together, due to the associated microscopic kinetic energy.

with a less than fifty percent change of killing more than one billion people

Typo, 'change' should be 'chance'

predictable

damn that hit me

I don't actually personally agree with Bayesian decision theory anymore and am currently inclined to treat value more like an objective fact about the world than as an individual preference. The provocative position would be a Beff-like one that value = entropy, but while that is an incremental improvenent on utilitarianims/value = negentropy, it is hellish and therefore I can't endorse it fully.

Regardless of the issues with my own position, I'm confused about your worldview. Do you not have a distinction between expected and actual utility, or do you consider there to be two different kinds of changes in values? How do you model value of information? (If you do model it, that is.)

Thank you very much for the criticism. Jeremias: I genuinely appreciate it. Please allow me to provide some context before commenting. Professor W.J. Wukmir developed the orectic theory, which I used in this post to provide a theory of emotions. The man in question lived in Barcelona, where he arrived around 1960 and was stateless until his death in 1981. Unfortunately, his theories have been lost to the course of time. A few months ago, the last of his disciples passed away. I'm afraid his theory will be forgotten in some second-hand bookstore as a rarity. His disciples said that in his youth (in the early 1930s), Wukmir had intense disputes with Freud and Adler in the Vienna circle to which the three belonged. Freud saw him as a revolutionary, while Wukmir believed that Freud was unable to understand him. I apologize for writing this preview, but my purpose was only to offer light on the origins of this theory.

The critique raises some important issues that should be carefully considered. However, I believe there are some fundamental misunderstandings concerning the nature and purpose of emotions as described by Wukmir's orectic theory, which I will address. I apologize for not being more clear in explaining them. 
Emotions, according to this theory, are not just "shortcut heuristics" but also essential systems of vital orientation. Emotions are more than just quick evaluations in the absence of detail; they are complex, multidimensional evaluations that incorporate multiple aspects of an organism's internal state, external environment, and previous experiences. This valuation process takes place at all levels of an organism, from individual cells to complex brain systems. The goal here is to emphasize the relevance of the "valuation" process rather than focusing solely on the emotions involved. 

Your critique implies a distinction between emotion and cognition, which my argument explicitly rejects. According to the orectic theory, emotion and cognition are inextricably related parts of the same process of vital orientation. Every cognitive act entails emotional valuation, and every emotional response contains cognitive components. This integration is not a "ghost in the machine," but rather an essential component of how living systems process information and make decisions. This is what we require from AI: process information and make decisions.

Your claim that AI can make decisions without emotions ignores the importance of emotions in decision-making, according to orectic theory. It's probably my fault for not explaining it more concisely. Emotions are not an add-on to decision-making; they are essential to the process of analyzing possibilities and deciding courses of action. According to this viewpoint, assigning meaning and value to stimuli and prospective actions is an emotional process.

The notion that ethics can be "arbitrarily programmed" without emotional capacity misses the importance of emotions in moral judgment. In my opinion, ethical decisions are fundamentally dependent on emotional valuations of situations and the resulting outcomes. A strictly logical system of ethics without emotional components would lack the critical ability to attach meaning and value to different outcomes.
Your argument appears to presuppose a restricted definition of intelligence that focuses on information processing and decision-making in a mechanical sense. The theory I described, as well as many modern perspectives in cognitive research, advocate for a more embodied and embedded conception of intelligence that must include emotional processes. From this standpoint, a "fully functioning artificial intelligence" would have to have emotion-like mechanisms.

While it is true that more precise prompts may elicit different answers, the purpose of the GPT-4o example was to demonstrate the limitations of simply language-based AI in grasping emotional context. Human communication is highly reliant on emotional understanding that extends beyond the literal meaning of words, and this understanding is rooted in shared embodied experiences.
 

Finally, the notion that language can completely compensate for the lack of nonverbal emotional cues overlooks the embodied character of emotional cognition. In my opinion, our emotional knowledge is largely based on our bodily experiences and cannot be completely reduced to linguistic descriptions.

That is just replacing the idea of fixed values with a fixed utility function. But it is just as changeable whatever you call it.

Show me your utility function before you were born.

In Bayesian decision theory, there's the distinction between expected utility, which changes as one learns about the environment, and actual utility, which does not. Under this frame, I'd be inclined to round you off to using the words "values"/"liking"/etc. to refer to expected utility. Would you agree with that? If not, why not?

It might be tempting to round the OP off to use the word "values"/"ought" to refer to actual utility, but the details of that are kind of awkward at the edges so I would hold off on that.

The discord links seem to time out. Here's a new one: https://discord.gg/D5ECfZ47

I'll be wearing a purple 'Roboflow' hat and holding an ACX sign. If you can't find us, I'll be routinely checking the discord and my email on the day ( Linaskondrackis@gmail.com); feel free to ping me if you're around.

I like what you’re doing trying to do here. I think this is important work.

I’m a bit confused at what you mean by Layperson though? These are good for the ‘every day’ above average intelligence ‘switched on’ type of individual.

But that is not what I image a Layperson as. I interact regularly with ~100 people. (For context, I am a Drama Teacher and Trivia Host)

I thought about how many I predict could understand these examples, given 20 seconds of their attention. I thought of 10 people. The other 90% would fall into a few other categories that all end with them not being more knowledgeable after coming across the text.

But am I confused? Was that 90% not the target audience?