Consciousness: A Compression-Based Approach

post by interstice · 2022-04-16T16:40:11.168Z · LW · GW · 14 comments

Contents

  Explanations as Compression
  Meta-Learning[4]
  Practical Application
None
14 comments

TL;DR: To your brain, "explaining things" means compressing them in terms of some smaller/already-known other thing. So the seemingly inexplicable nature of consciousness/qualia arises because qualia are primitive data elements which can't be compressed. The feeling of there nonetheless being a "problem" arises from a meta-learned heuristic that thinks everything should be compressible.

What's up with consciousness? This question has haunted philosophers and scientists for centuries, and has also been a frequent topic of discussion on this forum. A solution to this problem may have moral relevance soon, if we are going to create artificial agents which may or may not have consciousness. Thus, attempting to derive a satisfying theory seems highly desirable.

One popular approach is to abandon the goal of directly explaining consciousness and instead try to solve the 'meta-problem' of consciousness -- explaining why it is that we think there is a hard problem in the first place. This is the tactic I will take.

There have been a few attempts to solve the meta-problem, including some previously reviewed on LessWrong [LW · GW]. However, to me, none of them have felt quite satisfying because they lacked analysis of what I think of as the central thing to be explained by a meta-theory of consciousness -- the seeming inexplicability of consciousness. That is, they will explain why it is that we might have an internal model of our awareness, but they won't explain why aspects of that model feel inexplicable in terms of a physical mechanism. After all, there are other non-strictly-physical parts of our world model, such as the existence of countries, plans, or daydreams, which don't generate the same feeling of inexplicability. So, whence inexplicability? To answer this, we first need to have a model of what "explaining things" means to our brain in the first place.

Explanations as Compression

What is an explanation? On my (very rough) model of the brain^[1], there are various internal representations of things -- concepts, memories, plans, sense-data. These representations can be combined together, or transformed into each other. A representation X explains representation(s) Y, if Y can be produced from X under a transformation. The brain is constantly trying to explain complex representations in terms of simpler ones, thus producing an overall compression of its internal data. This drive towards compression is the source of most of our concepts as well as a motivator of explanation-seeking behavior in everyday life.

I assume people here are familiar with this basic idea or something like it, but here's some examples anyway. Say you are awoken by a mysterious tapping sound in the night. Confused, you wander around, following the sound of the tapping until you discover a leak in your roof. You have explained the tapping sound -- what was previously an arbitrary series of noises can be derived from your previous knowledge of the world plus the assumption that your roof has a leak, producing an overall compression of your experiences. High-level concepts such as the idea of "dog" are also produced in this way -- after encountering sufficiently many dogs, your brain notices its basic experiences with them can be re-used across instances, producing an abstract 'dog' concept with wide applicability^[2].

This basic explanation-seeking behavior also drives our quest to understand the world using science.^[3] Newtonian mechanics compresses the behavior of many physical situations. Quantum mechanics compresses more. The standard model+general relativity+standard cosmological model, taken together, form a compression of virtually everything we know of in the world to date. This mechanistic world model still ultimately serves the function of compressing our experiences, however. It is at this point that the question of the hard problem arises -- yes, we can explain everything in the physical realm, but can we truly explain the ineffable redness of red?

However, given the mechanism of 'explanation' provided, I think it's not too surprising that 'qualia' seemingly can't be explained mechanistically! The reason is that 'qualia', as representations, are simply too simple to be compressed further. They're raw data, like 0/1 in a binary stream -- the string "1" can't be compressed to anything simpler than itself; likewise, the percept 'red' is too simple and low-level to be explained in terms of anything else. Likewise, "why am I experiencing anything at all" cannot be answered because the aspect of experience that is being queried about is too simple -- that it exists in the first place. So that's it -- the brain has internal representations which it tries to compress, and 'qualia' are just incompressible representations.

Meta-Learning^[4]

"But wait", you might be thinking, "your model is missing something. If we were just compression-producing algorithms, we wouldn't think it was mysterious or weird that there are some inputs that are incompressible, we would just compress as best we could and then stop. Your model doesn't explain why we think there's a hard problem at all!"

To explain why we think there's a hard problem at all, we need to add another layer -- meta-learning. The basic idea is simple. While our brain hardware has certain native capabilities that let us do pattern-matching/compression, it also does reinforcement learning on actions that lead us to obtain better compressions, even if those actions are not immediately compressing. For example, imagine you live in a town with a library, containing an encyclopedia. If you find a weird new animal while exploring in the woods, you might learn to go to the library and check the encyclopedia, hopefully finding an explanation that links the new phenomenon and what you already know. The action of going to the library is not itself a compression, but it tends to reliably lead to forming better compressions.

On a subtler level, we can imagine there are certain internal habits of thought or frames of mind that tend to lead to producing better compressions, while not themselves directly being compressing. For instance, you might learn the habit of trying to construct a mathematical model of what you're learning about, or of trying to mentally imagine a very specific example when hearing about an abstract idea. "Materialism" can be thought of as a complex of facts and habits of thought that tend to produce good compressions -- "for any given phenomenon, a good explanation for it can be found by localizing it in time and space and applying the laws of physics".

These meta-learned habits of thought, I claim, are the source of the intuition that 'redness' ought to be explainable mechanistically(after all, everything else is) The paradoxical feeling of the hard problem arises from the conflict between this intuition and the underlying compressor's inability to compress basic percepts such as red.

As an analogy, imagine a string-compressing algorithm which uses reinforcement learning to train an 'adaptive compressor' which searches for strategies to generate better compressions. If this RL algorithm is sophisticated enough, it might learn to have an internal self-model and model of the task it's carrying out -- making strings shorter. But if its training data mainly consists of strings which are highly compressible, it might learn the heuristic that its goal should be to make any string shorter, and that this should always be possible. Such robots, if they could communicate with each other, might write essays about "the hard problem of 0", and the paradoxical impossibility of a string which somehow seems to have no possible compression!

Practical Application

"That might make sense of the feeling of paradox, although the details of the meta-learning stuff sound a bit iffy", you may be thinking. "But even if I grant that, I still don't feel like I understand qualia on a gut level. It still feels like the innate redness of red can't possibly be explained by physics, even if it can be predicted that I would feel that way. Solving the meta-problem isn't enough to solve the hard problem in a way that satisfies".

For myself, I have mostly lost the feeling that there is anything paradoxical about qualia or consciousness. But this is not because the above argument convinced me they can be explained away. Rather, it caused me to adjust my sense of how I should think about reality, my senses, and the relationship between them.

Imagine you were giving advice to the string-compressing robot which thought '0' ought to be compressible. It wouldn't be right to tell it that actually, 0 is compressible, you just need to take a meta-view of the situation which explains why you think that. Instead, you might use that meta-view to motivate it to adjust its heuristics -- it should learn to accept that some strings are incompressible, and focus its efforts on those that in fact can be compressed. Similarly, although both redness and physics still seem as real as ever to me, I've lost my sense that I should necessarily be able to explain redness in terms of physics. Physics is a framework developed to summarize our experiences, but some aspects of those experiences will always be beyond its scope. I no longer feel that there is anything confusing here(if any readers feel differently, feel free to try to re-confuse me in the comments!)

"So, your 'solution' to the hard problem is just to give up, and accept both physics and qualia as real? How does that explain anything, isn't that the naive view we are trying to overcome?" Well yeah, basically, depending on how you define and think about realness. You could frame it as 'giving up', but I think that sometimes resolving confusion about a question necessarily looks like giving up on one level while gaining understanding on another: take the halting problem for instance. "What about the implications for building AI and morality and so forth? And how does this relate to the consciousness of other people?" Those are interesting hard questions -- but they exceed the scope of this post, which is just intended to explain the inexplicability of consciousness! I will hopefully return to moral/etc. implications later.

---

1. I'm not super attached to any of the details here. But the basic structure seems likely to hold well enough for the rest of the analysis to go through. ↩︎

2. Yes, there is also social learning and other methods we use to learn concepts, but IMO those are overlays atop this more basic mechanism. ↩︎

3. There's some details here about how e.g. our brains can't natively use the standard model to predict things, we have to rely on social testimony for most of our evidence, etc., but I'm going to elide that here because I don't think it's relevant. ↩︎

4. This section is somewhat more speculative than the previous. ↩︎

14 comments

Comments sorted by top scores.