Ephemeral correspondence

It feels like there's a never-directly-claimed but oft-implied claim lurking in this essay.

The claim goes: the reason we can't consciously control our perception of the color of the sky is because if we could then human partisanship would ruin it.

The sane response, upon realizing that internal color-of-the-sky is determined not by the sky-sensors, but by a tribal monkey-mind prone to politicking and groupthink is to scream in horror and then directly re-attach the world-model-generator to reality as quickly as possible.

If you squint, and treat partisanship as an ontologically basic thing that could exert evolutionary pressure, it almost seems plausible that avoidance of partisanship failure modes might actually be the cause of the wiring of the occipital cortex :-)

However, I don't personally think that "avoiding the ability of partisanship to ruin vision" is the reason human vision is wired up so that we can't see whatever we consciously choose to see.

Part of the reason I don't believe this is that the second half of the implication is simply not universally true. I know people who report having the ability to modify their visual sensorium at will, so for them, it seems to actually be the case that they could choose to do all sorts of things to their visual world model if they put some creativity and effort into it. Also: synesthesia is a thing, and can probably be cultivated...

But even if you skip over such issues as non-central outliers...

It makes conceptual sense to me that there is probably something like a common cortical algorithm (though maybe not exactly like the precise algorithmic sketch being discussed under that name) that actually happens in the brain. Coarsely: it probably has to do with neuron metabolism and how neurons measure and affect each other. Separately from this, there are lots of processes for controlling which neurons are "near" to which other neurons.

My personal guess is that in actual brains, the process mixes sparse/bayesian/pooling/etc perception with negative feedback control... and of course "maybe other stuff too". But fundamentally I think we start with "all the computing elements potentially measuring and controlling their neighbors" and then when that causes terrible outcomes (like nearly instantaneous subconscious wireheading by organisms with 3 neurons) evolution prunes that particular failure mode out, and then iterates.

However, sometimes top down control of measurement is functional. It happens subconsciously in ancient and useful ways in our own brain, as when afferent cochlear enervation projects something like "expectations about what is to be heard" that make the cochlea differentially sensitive to inputs, effectively increasing the dynamic range in what sounds can be neurologically distinguished.

This theory predicts new wireheading failures at every new level of evolved organization. Each time you make several attempts to build variations on a new kind of measuring and optimizing process/module/layer, some of those processes will use their control elements to manipulate their perception elements, and sometimes they will do poorly rather than well, with wireheading as a large and probably dysfunctional attractor.

"Human partisanship" does seem to be an example of often-broken agency in an evolutionarily recent context (ie the context of super-Dunbar socially/verbally coordinated herds of meme-infected humans) and human partisanship does seem pretty bad... but as far as I can see, partisanship is not conceptually central here. And it isn't even the conceptually central negative thing.

The central negative thing, in my opinion, is wireheading.

I'm enjoying these posts.

you do get to decide whether or not to perceive it as a complement or an insult.

compliment

dieties

believed

But from the perspective of a mind designer, it's bonkers. The world-model-generator isn't hooked up directly to reality!

I can't help but feel that this is a "lies to children" approach; it makes great sense to me why the module that determines other people's intentions takes input from conscious control. If nothing else, it allows the brain to throw highly variable amounts of resources at the problem--if Master Control thinks it's trivial, the problem can be dropped, but if it thinks it's worthwhile, the brain can spend hours worrying about the problem and calling up memories and forming plans for how to acquire additional information. (Who to ask for advice is itself a political play, and should be handed over to the political modules for contemplation.)

That is, it seems to me that physical reality (the color of the sky) and social reality (whether or not the coworker is complimenting or insulting you) are different classes of reality, that must be perceived and manipulated in different ways.

The valuable rationality lesson seems to be acknowledging that social reality, while it may seem the same sort of real as physical reality, is a different kind. I'm reading you, though, as claiming that they're treated differently, when they should be treated the same way.

Your world model vs executive framing is interesting, but I suspect the two-system model described in Thinking Fast and Slow carves reality at the joints better. From reading Kahneman, I get the impression that it’s not so much our conscious desires to modify our world model that trip us up. In fact, the opposite appears to be true in the sense that conscious, deliberative thinking, when applied well, almost always leaves you with a better model of the world than your subconscious System 1 processes, which have a tendency to quickly generate an explanation of the evidence that seems coherent and pleasing and then stop (why keep thinking if you’ve already figured it out?)

I guess maybe your framing is a useful way to think about motivated cognition, which Kahneman doesn’t discuss as much. I think I stopped doing as much motivated cognition when I installed a process in my brain that watches my thought patterns for things that look like they might be stereotypical examples of motivated cognition and makes me anxious until I dig a little deeper. (For example, writing the previous sentence caused the process to fire because it was an instance of me claiming that I’m less biased than I used to be, and “I’m not biased the way everyone else is” matches my motivated cognition pattern recognizer.) I have a personal hunch that motivated cognition is a basic function call that your brain uses all over the place, e.g. “what company should I start?” or “what sounds appealing for dinner?” initiates the same basic sort of thought process as “why is this blog post whose conclusion I disagree with wrong?” If my hunch is correct, the best way to use motivated cognition in a lot of cases may be to run it in both directions, e.g. spend a timed minute thinking of reasons the blog post is wrong, then spend a timed minute thinking of reasons the blog post is right. I suspect you’ll generate more ideas this way than if you spend a timed two minutes in directionless mulling.

I guess if anyone wants to install this observer process in themselves, probably the most general way to do it is flag instances where you find yourself searching for evidence to support a particular conclusion.

Another story about how confirmation bias arises is through the global approach/avoid assessments generated by System 1. Under this view, it’s not so much motivated cognition that gets you… even if you set out with the intention of seeing the world the way it really is, you’ll find yourself naturally shying away from explanations that make you uncomfortable and devoting less thought to them, while devoting more thought to pleasant explanations. On a community site like Less Wrong, say, users could find themselves mysteriously procrastinating more on writing a post that disputes a popularly held view than a post which no one is likely to disagree with, even when their conscious deliberative judgements about the value of writing each post are identical. Under this view, stoic resolve to think thoughts even when they seem uncomfortable would be key. If this prediction is correct, giving people the stoic resolve to overcome aversions would help them in many areas of their life and in particular help them fight confirmation bias.

I guess another strategy would be rather than learning to overcome aversions, reframe potentially uncomfortable truths by reminding yourself that you want to know what’s true?

By the way, I’m not sure why Thinking Fast and Slow is not discussed here on Less Wrong more. It’s such a fabulous book, and compares quite favorably with the sequences in my view. I found the content overlap to be surprisingly small though; I suspect it is worth many peoples’ time to read both.

But from the perspective of a mind designer, it's bonkers. The world-model-generator isn't hooked up directly to reality! We occasionally get to choose how parts of the world-model look! We, the tribal monkeys known for self-deception and propensity to be manipulated, get a say on how the information engine builds the thing which is supposed to correspond to reality!

In theory, perception and cognition are not really different. In practice, the mental algorithms that implement "perception" are different ones, evolved a different stage of development, from the ones that implement "decisions" and your internal locus of control. Certain inferences have to be bounced back and forth between the two in order to come out right -- a mind designed a priori would just do bounded-rational cognition the whole way down, but your evolved brain has to bounce percepts and inductive biases back and forth between algorithms.

You yourself are part of the pathway between reality and your map of it, part of a fragile link between what is, and what is believed. And if you let your guard down, even for one moment, it is incredibly easy to flinch and shatter that ephemeral correspondence.

It's worse/better than that. The slash is because, well, I'm fairly sure this is a fairly intrinsic part of being a conscious being rather than a blind optimizer, so on the one hand, you've got the privilege of conscious experience, and on the other hand, it helps render you a delusional nutcase. You are the map. You are also the little monkey voice that screams about things. You are also the still, small voice who does the joined-up thinking and tries to manage all the other voices in the head, and tries to make the map correspond to the territory, but still irrevocably lives inside the map.

Welcome to life! It's sometimes fun. We're working on improving that.

One obvious reason for being able to choose to determine various parts of the map is that it contributed to survival. For example, the leader of the tribe hates you and makes a few insulting remarks. You can choose to interpret these in a fairly neutral way, or you can interpret them as they are. If you choose to interpret them as hateful and insulting, as they are, you may have a hard time not responding in a corresponding manner, and so you may end up dead. You will be better off if you can choose to interpret them in the neutral way. Or again, the leader of the tribe proclaims an obviously false religious dogma. If you can choose to accept it, things will go on as usual. If you cannot accept it, you may have a hard time pretending well enough to avoid getting killed as a heretic. Again you will be better off if you are in control of your map.

Also, I disagree that there is any rigid distinction between beliefs we can control and others we cannot (as I suggested in my post on belief in belief). We cannot generally change the visual sensation when we look at the sky. But whether or not we believe the statement, "the sky is blue," is indeed up to us, and some people will e.g. deny that the sky is blue, since it is not really colored in the same way as other things. Or someone could indeed believe that the sky is fundamentally green, if that were e.g. a religious dogma.

Good post, but I believe you have a typo:

You yourself are part of the pathway between reality and your map of it, part of a fragile link between what is, and what is beleived.

I'm not sure the seeing the sky as blue is a perfect example, since one can construct optical illusions where the "world-model generator" considers something blue even though that isn't the case.

The title makes me think of a letter, quickly burnt. :)

There is a wrong-note in the reasoning of this post that immediately started niggling at me, but it's subtle and I'm having trouble teasing out the underlying assumption. I want to say that you're taking "The purpose of consciousness is consciousness" as a given, when that is arguably false. Likewise, I want to accuse you of drawing causal arrows from consciousness to other modules of human mind design, which as far as I know is ruled out, evolutionarily speaking.

I offer this:

The "executive process" as you call it is part of the world-modeler. It is the world-modeling module that evolved in response to a very unique world-modeling challenge. There is a critical difference between sky-color and insult-vs-complement that you seem to be glossing over. A given wavelength of light always has the same properties. A given array of posture, facial expression, tone, etc. does not always map directly to the same social reality.

We can't chose to see a smile on a scowling face any more than we can chose to see a green sky, but unlike the sky, the same facial expression can mean vastly different things depending on context, because the causes underlying any given expression depends on a thing that is just as complicated as you are.

The "executive process" is how evolution solved the entirely new problem of adding other world-modelers to the world-model and that's what it does. If it is glitchy and unreliable, well, it is still very new. The very first functional wing to evolve probably wasn't all that good at producing lift, either.