Subagents, introspective awareness, and blending

post by Kaj_Sotala · 2019-03-02T12:53:47.282Z · LW · GW · 18 comments

Contents

  Robots again
  Blending
None
18 comments

In this post, I extend the model of mind that I've been building up in previous posts to explain some things about change blindness, not knowing whether you are conscious, forgetting most of your thoughts, and mistaking your thoughts and emotions as objective facts, while also connecting it with the theory in the meditation book The Mind Illuminated. (If you didn't read my previous posts, this article has been written to also work as a stand-alone piece.)

The Mind Illuminated (Amazon, SSC review), or TMI for short, presents what it calls the moments of consciousness model. According to this model, our stream of consciousness consists of a series of discrete moments, each a mental object. Under this model, there are always different “subminds” which are projecting mental objects into consciousness. At different moments, different mental objects get selected as the content of consciousness.

If you’ve read some of the previous posts in this sequence, you may recognize this as sounding familiar. We started by discussing some of the neuroscience research on consciousness [LW · GW]. There we covered the GWT/GNW theory of consciousness being a “workspace” in the brain that different brain systems project information into, and which allows them to synchronize their processing around a single piece of information. In the next post, we discussed the psychotherapy model of Internal Family Systems [LW · GW], which also conceives the mind of being composed of different parts, many of which are trying to accomplish various aims by competing to project various mental objects into consciousness. (TMI talks about subminds, IFS talks about parts, GWT/GNW just talks about different parts of the brain; for consistency’s sake, I will just use “subagent” in the rest of this post.)

At this point, we might want to look at some criticisms of this kind of a framework. Susan Blackmore has written an interesting paper called “There is no stream of consciousness”. She has several examples for why we should reject the notion of any stream of consciousness. For instance, this one:

For many years now I have been getting my students to ask themselves, as many times as possible every day “Am I conscious now?”. Typically they find the task unexpectedly hard to do; and hard to remember to do. But when they do it, it has some very odd effects. First they often report that they always seem to be conscious when they ask the question but become less and less sure about whether they were conscious a moment before. With more practice they say that asking the question itself makes them more conscious, and that they can extend this consciousness from a few seconds to perhaps a minute or two. What does this say about consciousness the rest of the time?

Just this starting exercise (we go on to various elaborations of it as the course progresses) begins to change many students’ assumptions about their own experience. In particular they become less sure that there are always contents in their stream of consciousness. How does it seem to you? It is worth deciding at the outset because this is what I am going to deny. I suggest that there is no stream of consciousness. [...]

I want to replace our familiar idea of a stream of consciousness with that of illusory backwards streams. At any time in the brain a whole lot of different things are going on. None of these is either ‘in’ or ‘out’ of consciousness, so we don’t need to explain the ‘difference’ between conscious and unconscious processing. Every so often something happens to create what seems to have been a stream. For example, we ask “Am I conscious now?”. At this point a retrospective story is concocted about what was in the stream of consciousness a moment before, together with a self who was apparently experiencing it. Of course there was neither a conscious self nor a stream, but it now seems as though there was. This process goes on all the time with new stories being concocted whenever required. At any time that we bother to look, or ask ourselves about it, it seems as though there is a stream of consciousness going on. When we don’t bother to ask, or to look, it doesn’t, but then we don’t notice so it doesn’t matter. This way the grand illusion is concocted.

This is an interesting argument. A similar example might be that when I first started doing something like track-back meditation [LW · GW] when on walks, checking what was in my mind just a second ago. I was surprised at just how many thoughts I would have while on a walk, that I would usually just totally forget about afterwards, and come back home having no recollection of 95% of them. This seems similar to Blackmore’s “was I conscious just now” question, in that when I started to check back the contents of my mind just a few seconds ago, I was frequently surprised by what I found out. (And yes, I’ve tried the “was I conscious just now” question as well, with similar results as Blackmore’s students.)

Another example that Blackmore cites is change blindness. When people are shown an image, it’s often possible to introduce unnoticed major changes into the image, as long as people are not looking at the very location of the change when it’s made. Blackmore also interprets this as well to mean that there is no stream of consciousness - we aren’t actually building up a detailed visual model of our environment, which we would then experience in our consciousness.

One might summarize this class of objections as something like, “stream-of-consciousness theories assume that there is a conscious stream of mental objects in our minds that we are aware of. However, upon investigation it often becomes apparent that we haven’t been aware of something that was supposedly in our stream of consciousness. In change blindness experiments we weren’t aware of what the changed detail actually was pre-change, and more generally we don’t even have clear awareness of whether we were conscious a moment ago.”

But on the other hand, as we reviewed earlier, there still seem to be objective experiments [LW · GW] which establish the existence of something like a “consciousness”, which holds approximately one mental object at a time.

So I would interpret Blackmore’s findings differently. I agree that answers to questions like “am I conscious right now” are constructed somewhat on the spot, in response to the question. But I don’t think that we need to reject having a stream of consciousness because of that. I think that you can be aware of something, without being aware of the fact that you were aware of it.

Robots again

For example, let’s look at a robot that has something like a global consciousness workspace. Here are the contents of its consciousness on five successive timesteps:

1. It’s raining outside
2. Battery low
3. Technological unemployment protestors are outside
4. Battery low
5. I’m now recharging my battery

Notice that at the first timestep, the robot was aware of the fact that it was raining outside; this was the fact being broadcast from consciousness to all subsystems. But at no later timestep was it conscious of the fact that at the first timestep, it had been aware of it raining outside. Assuming that no subagent happened to save this specific piece of information, then all knowledge of it was lost as soon as the content of the consciousness workspace changed.

But suppose that there is some subagent which happens to keep track of what has been happening in consciousness. In that case it may choose to make its memory of previous mind-states consciously available:

6. At time 1, there was the thought that [It’s raining outside]

Now there is a mental object in the robot’s consciousness, which encodes not only the observation of it raining outside before, but also the fact that the system was thinking of this before. That knowledge may then have further effects on the system - for example, when I became aware of how much time I spent on useless rumination while on walks, I got frustrated. And this seems to have contributed to making me ruminate less: as the system’s actions and their overall effect were metacognitively represented and made available for the system’s decision-making, this had the effect of the system adjusting its behavior to tune down activity that was deemed useless.

The Mind Illuminated calls this introspective awareness. Moments of introspective awareness are summaries of the system’s previous mental activity, with there being a dedicated subagent with the task of preparing and outputting such summaries. Usually it will only focus on tracking the specific kinds of mental states which seem important to track.

So if we ask ourselves “was I conscious just now” for the first time, that might cause the agent to output some representation of the previous state we were in. But it doesn’t have experience in answering this question, and if it’s anything like most human memory systems, it needs to have some kind of a concrete example of what exactly it is looking for. The first time we ask it, the subagent’s pattern-matcher knows that the system is presumably conscious at this instant, so it should be looking for some feature in our previous experiences which somehow resembles this moment, but it’s not quite sure of which one. And an introspective mind state, is likely to be different from the less introspective mind state that we were in a moment ago.

This has the result that on the first few times when it’s asked, the subagent may produce an uncertain answer: it’s basically asking its memory store “do my previous mind states resemble this one as judged by some unclear criteria”, which is obviously hard to answer.

With time, operating from the assumption that the system is currently conscious, the subagent may learn to find more connections between the current moment and past ones that it still happens to have in memory. Then it will report those as consciousness, and likely also focus more attention on aspects of the current experience which it has learned to consider “conscious”. This would match Blackmore’s report of “with more practice [the students] say that asking the question itself makes them more conscious, and that they can extend this consciousness from a few seconds to perhaps a minute or two”.

Similarly, this explains me not being aware of most of my thoughts, as well as change blindness. I had a stream of thoughts, but because I had not been practicing introspective awareness, there were no moments of introspective awareness making me aware of having had these thoughts. Though I was aware of the thoughts at the time, this was never re-presented in a way that would have left a memory trace.

In change blindness experiments, people might look at the same spot in a picture twice. Although they did see the contents of that spot at time 1 and were aware of them, that memory was never stored anywhere. When at time 2 they looked at the same spot and it was different, the lack of an awareness of what they saw previously means that they don’t notice the change.

Introspective awareness will be an important concept in my future posts. (Abram Demski also wrote a previous post on Track-Back Meditation [LW · GW], which is basically an exercise for introspective awareness.) Today, I’m going to talk about its relation to a concept I’ve talked about before: blending / cognitive fusion.

Blending

I’ve previously discussed “cognitive fusion” [LW · GW], as what happens when the content of a thought or emotion is experienced as an objective truth rather than a mental construct. For instance, you get angry at someone, and the emotion makes you experience them as a horrible person - and in the moment this seems just true to you, rather than being an interpretation created by your emotional reaction.

You can also fuse with more logical-type beliefs - or for that matter any beliefs - when you just treat them as unquestioned truths, without remembering the possibility that they might be wrong. In my previous post, I suggested that many forms of meditation were training the skill of intentional cognitive defusion, but I didn’t explain how exactly meditation lets you get better at defusion.

In my post about Internal Family Systems [LW · GW], I mentioned that IFS uses the term “blending” for when a subagent is sending emotional content to your consciousness, and suggested that IFS’s unblending techniques worked by associating extra content around those thoughts and emotions, allowing you to recognize them as mental objects. For instance, you might notice sensations in your body that were associated with the emotion, and let your mind generate a mental image of what the physical form of those sensations might look like. Then this set of emotions, thoughts, sensations, and visual images becomes “packaged together” in your mind, unambiguously designating it as a mental object.

My current model is that meditation works similarly, only using moments of introspective awareness as the “extra wrapper”. Suppose again that you are a robot, and the contents of your consciousness is:

1. It’s raining outside.

Note that this mental object is basically being taken as an axiomatic truth: what is in your consciousness, is that it is raining outside.

On the other hand, suppose that your consciousness contains this:

1. Sensor 62 is reporting that [it’s raining outside].

Now the mental object in your consciousness contains the origin of the belief that it’s raining. The information is made available to various subagents which have other beliefs. E.g. a subagent holding knowledge about sensors, might upon seeing this mental object, recognize the reference to “sensor 62” and output its estimate of that sensor’s reliability. The previous two mental objects could then be combined by a third subagent:

1. (Subagent A:) Sensor 62 is reporting that [it is raining outside]
2. (Subagent B:) Readings from sensor 62 are reliable 38% of the time.
3. (Subagent C:) It is raining outside with a 38% probability.

In my discussion of Consciousness and the Brain [LW · GW], I noted that one of the proposed functions of consciousness is to act as a production system, where many different subagents may identify particular mental objects and then apply various rules to transform the contents of consciousness as a result. What I’ve sketched above is exactly a sequence of production rules: at e.g. step 2, something like the rule “if sensor 62 is mentioned as an information source, output the current best estimate of sensor 62’s reliability” is applied by subagent B. Then at the third timestep, another subagent combines the observations from the previous two timesteps, and sends that into consciousness.

What was important was that the system was not representing the outside weather just as an axiomatic statement, but rather it was explicitly representing it as a fallible piece of information with a particular source.

Here’s something similar:

1. I am a bad person.
2. At t1, there was the thought that [I am a bad person].

Here, the moment of awareness is highlighting the nature of the previous thought as a thought, thus causing the system to treat it as such. If you used introspective awareness for unblending, it might go something like this:

1. Blending: you are experiencing everything that a subagent outputs as true. In this situation, there’s no introspective awareness that would highlight those outputs as being just thoughts. “My friend is a horrible person”, feels like a fact about the world.
2. Partial blending: you realize that the thoughts which you have might not be entirely true, but you still feel them emotionally and might end up acting accordingly. In this situation, there are moments of introspective awareness, but there are also enough of the original “unmarked” thoughts to also be affecting your other subagents. You feel hostile towards your friend, and realize that this may not be rationally warranted, but still end up talking in an angry tone and maybe saying things you shouldn’t.
3. Unblending: all or nearly all of the thoughts coming from a subagent are being filtered through a mechanism that wraps them inside moments of introspective awareness, such as “this subagent is thinking that X”. You know that you have a subagent which has this opinion, but none of the other subagents are treating it as a proven fact.

By training your mind to have more introspective moments of awareness, you will become capable of perceiving more and more mental objects as just that. A classic example would be all those mindfulness exercises where you stop identifying with the content of a thought, and see it as something separate from yourself. At more advanced levels, even mental objects which build up sensations such as those which make up the experience of a self may be seen as just constructed mental objects.

18 comments

Comments sorted by top scores.