Compartmentalization as a passive phenomenon

GEB has a section on this.

In order to not compartmentalize, you need to test if your beliefs are all consistent with each other. If your beliefs are all statements in propositional logic, consistency checking becomes the Boolean Satisfiability Problem, which is NP-complete. If your beliefs are statements in predicate logic, then consistency checking becomes PSPACE-complete, which is even worse than NP-complete.

Not compartmentalizing isn't just difficult, it's basically impossible.

If understand you correctly, you are saying that most people are not knowledgeable enough about the different domains in question to make any (or judge any) cross-domain connections. This seems plausible.

I can think however of another argument that confirms this but also clarifies why on Less Wrong we think that people actively compartmentalize instead of failing to make the connection and that is selection bias. Most people on this site are scientists, programmers or other technical professions. It seems that most are also consequentialists. Not surprisingly, both these facts points to people who enjoy following a chain of logic all way to the end.

So, we tend to learn a field until we know it's basic principles. For example, if you learn about gravity, you can learn just enough so you can calculate the falling speed of an object in gravitational field or you can learn about the bending of space-time by mass. It seems rather obvious to me that the second method encourages cross-domain connections. If you don't know the basic underlying principles of the domains you can't make connections.

I also see this all the time when I teach someone how to use computers. Some people build an internal model of how a computer & programs conceptually work and are then able to use most basic programs. Others learn by memorizing each step and are looking at each program as a domain on it's own instead of generalizing across all programs.

I wonder if there'd be a difference between the survey as written (asking what a pen would do on the moon, and then offering a chance to change the answer based on Apollo astronauts) vs. a survey in which someone asked "Given that the Apollo astronauts walked on the moon, what do you think would have happened if they'd dropped a pen?"

The first method makes someone commit to a false theory, and then gives them information that challenges the theory. People could passively try to fit the astronaut datum into their current working theory, or they could actively view it as an outside attack on their position which they had to defend against. Maybe if the students had given people the information about the astronauts first, the respondents would have applied the cross-domain knowledge more successfully.

But I totally sympathize with you about the occasional virtues of compartmentalization. The worst field I've ever found for this is health and medicine. You learn that some vitamin is an antioxidant, then you learn that some disease is caused by oxidation, you make the natural assumption that the vitamin would help cure the disease, and then a study comes out saying there's no relationship at all.

But I suspect compartmentalization might actually be the natural state, the one that requires effort to overcome.

Look at it this way: what evolutionary pressure exists for NOT compartmentalizing?

From evolution's standpoint, if two of your beliefs really need to operate at the same time, then the stimuli will be present in your environment at close enough to the same time as to get them both activated, and that's good enough to work for passive consistency checking. For active consistency checking, we have simple input filters for rejecting stuff that conflicts with important signaling beliefs and whatnot.

OTOH, there's no evolutionary pressure for something that sifts through your entire brain contents, generating arbitrary scenarios where two pieces of information might conflict or produce some startlingly new and useful idea.

I think part of the problem with the moon question was that it suggested two wrong answers first. How would you have answered the question if it was just "If a pen is dropped on the moon, what will happen? Explain in one sentence."

I would have shrugged and said "It will fall down, slowly." But when I saw "float away" and "float where it is", those ideas wormed their way into my head for a few seconds before I could reject them. Just suggesting those ideas managed to mess me up, and I'm someone whose mental model of motion in space is so strong that I damn near cried with joy when I watched Planetes and saw people maneuvering in zero gravity exactly the way they're supposed to. (And it turns out I'm not the only one to have this exact same reaction. Weird.)

So, I'm thinking that the wrong multiple-choice answers are responsible for a lot of the confusion, the same way most people wouldn't interpret bumps in the night as angry ghosts unless they hear that the house is haunted.

Yes, building mental connections between domains requires well-populated maps for both of them, plus significant extra processing. It's more properly treated as a skill which needs development than a cognitive defect. In the pen-on-the-moon example, knowing that astronauts can walk around is not enough to infer that a pen will fall; you also have to know that gravity is multiplicative rather than a threshold effect. And it certainly doesn't help that most peoples' knowledge of non-Earth gravity comes entirely from television, where, since zero-gravity filming is impractical, the writers invariably come up with some sort of confusing phlebotinum (most commonly magnetic boots) to make them behave more like regular-gravity environments.

I read the question as asking about THE Moon, not "a moon". The question as written has no certain answer. If a moon is rotating fast enough, a pen held a few feet above its surface will be at orbital velocity. Above this level it will float away. The astronaut might also float away, unless he were wearing heavy boots.

Someone posted a while back that only a third of adults are capable of abstract reasoning. I've had some trouble figuring out exactly it means to go through life without abstract reasoning. The "heavy boots" response is a good example.

Without abstract reasoning, it's not possible to form the kind of theories that would let you connect the behavior of a pen and an astronaut in a gravitational field. I agree that this is an example of lack of ability, not compartmentalization. Of course, scientists are capable of abstract reasoning, so its still possible to accuse them of compartmentalizing even after considering the survey results.

For those well-versed in physics, it seems mind-numbingly bizarre to hear someone claim that the Moon's gravity isn't enough to affect a pen, but is enough to affect people wearing heavy boots. But as for some hypothetical person who hasn't studied much physics... or screw the hypotheticals - for me, this sounds wrong but not obviously and completely wrong. I mean, "the pen has less mass, so there's less stuff for gravity to affect" sounds intuitively sorta-plausible for me, because I haven't had enough exposure to formal physics to hammer in the right intuition.

Absolutely. Another piece of the puzzle required to understand whether the pen 'obviously' falls or not is, 'what kind of atmosphere does the moon have'? What fraction of people know that there is no atmosphere on the surface of the moon? (Do I really know this?? I think I just remember being told this, and despite being told, I'm not certain there's absolutely no atmosphere on the moon.)

Without detailed information about the atmosphere, you really don't know. On Earth, the pen floats in water, but doesn't float in air.

(And then you have the added problem that there's a high chance people will first recall the image of the flag blowing on the moon, which is unfortunate for physics.)

Quite convincing, thanks. I'll want to think about it more, but perhaps it would be a good idea to toss the word out the window for its active connotations.

ISTM, though, that there is a knack for for cross-domain generalization (and cross-domain mangling) of insights, that people have this knack in varying degrees, that this knack is an important component of what we call "intelligence", in the sense that if we could figure out what this knack consists of we'd have solved a good chunk of AI. Isn't this a major reason why Hofstadter, for instance, has focused so sharply on analogy-making, fluid analogies, and so on?

(This is perhaps a clue to one thing that has been puzzling me, given Eliezer's interest in AI, namely the predominance of topics such as decision theory on this blog, and the near total absence of discussion around topics such as creativity or analogy-making.)

Nitpick: "If a pen is dropped on A moon"
It doesn't specify Earth's moon. If a pen were dropped on say, Deimos, it might very well appear to do B) for a long moment ;) (Deimos is Mars' outermost moon and too small to retain a round shape. Its gravity is only 0.00256 m/s^2 and escape velocity is only 5.6 m/s. That means you could run off it.)

On the other hand, the word "dropped" effectively gives the game away. Things dropped go DOWN, not up, and they don't float in place. Would be better to say "released".

And now, back to our story...

I don't believe it was assumed that compartmentalization is something you actually "do" (make effort towards achieving). Making this explicit is welcome, but assuming the opposite to be the default view seems to be an error.

I thought about it for a moment, and almost settled on B - after all, there isn't much gravity on the moon, and a pen is so light that it might just be unaffected. I

Many people (probably more people) make the same mistake when asked 'which falls faster, the 1 kg weight or the 20 kg weight?'. I guess this illustrates why compartmentalization is useful. False beliefs that don't matter in one field can kill you in another.

I fail to understand how compartmentalization explains this. I got the answer right the first time. And I suspect most people who go it wrong did so because of the assumptions (unwarranted) they were making - meaning if they had just looked at the question and nothing else, and if they understood basic gravity, they would've got it right. But when you also try to imagine some hypothetical forces on the surface of the moon or relate to zero gravity images seen on tv etc, and if you visualize all these before you visualize the question, you'd probably get it wrong.

So my theory is that much of compartmentalization is simply because the search space is so large that people don't end up seeing that there might be a connection between two domains.

I'm not sure about this. In your examples here, people are in fact completely lacking any full understanding of gravitation and/or (I suppose) knowledge of the masses of notable celestial objects in our solar system.

Now, I have to admit that the correct answer wasn't obvious to me at first.

I up-voted just for you admitting this in your example, but lets talk about this. You knew the moon had gravity, but not much. Then you remember its enough for astronauts to walk on, so most likely enough to attract a pen as well.

Do you understand that the same thing would happen if you were standing on an asteroid? Yes you can stand on an asteroid - I wouldn't recommend walking or any movement at all without a tether but as long as you don't move you will stay on its surface. In this case, gravity would not be enough for you to walk even with "heavy boots".

But if you just release the pen (don't throw it or toss it at all please!) it will still fall. Every object with mass has gravity, and any two objects will be attracted even if its by a relatively weak force. Yes other celestial masses will exert influence but as long as this asteroid is not on a collision course with another body we can be reasonably sure that 3-4 feet from its surface, its gravity will be greater than any other body's.

If you don't understand gravitation, you can't really expect to answer the question correctly. As for the people who can't answer it correctly: lots of people didn't really care much for physics when they studied it, and so while they may have known this at one time (to pass a test) it did not really get integrated into their working knowledge. It may have been possible to dredge up the correct answer by asking the right question to trigger the right memory, but the fact is that they really just don't have this knowledge, even if the information is in their brain.

The way I solved the pen on the moon question is that I remembered the famous demonstration one of the Apollo astronauts did with a feather and hammer on the moon, and didn't think there should be a meaningful difference between those objects and a pen. I could've worked out the physics, but pattern-recognition was faster and easier. 

So my theory is that much of compartmentalization is simply because the search space is so large that people don't end up seeing that there might be a connection between two domains. Even if they do see the potential, or if it's explicitly pointed out to them, they might still not know enough about the domain in question (such as in the example of heavy boots), or they might find the proposed connection implausible.

If a person's knowledge is highly compartmentalized, and consists of these three facts:

1. A human being walked across the moon.

2. There are small rocks on the surface.

3. The moon is a planetary body.

without any educational background, would they choose the right answer?

I believe that their is a high probability that basic intuition would lead to an accurate answer.

So what went wrong, in your case? I don't think that you can attribute it to a failure of compartmentalization. It wasn't that you didn't make connections to your prior knowledge; the problem was that you made too many and that you hadn't organized your priors into a confidence hierarchy.

Confusion occurs when tenuous connections are made and lead to an over-analysis of the question. You differ from the person in the hypothetical, because you had prior knowledge of the forces involved. Connections are only helpful when they are made from strong foundational knowledge to new applications. When you are making many connections from a condition of uncertainty, to a new problem, your intuition fails. It results in the assignment of a low confidence level, to each of many connections, while ignoring basic observations or truisms.

It seems, you were confident in the areas of physics, most applicable in this situation; enumerating the atmosphere, gravity, and mass as the most influential. You attempted to remember how these forces interacted and recognized that they had dimensions to them, that you had forgotten. The connections caused your intuition to be replaced by a humbleness, and the go to answer was a balanced combination of forces. Thus the pen would float.

It is clear that you came to the problem with much more information than my hypothetical person, armed with three foundational facts. You too had those three facts, if that was all that was in your moon compartment the intuition would have been clearer.

(I apologize for the presumptions I made in referring to your thought process. This is a situation, in which, we find ourselves frequently.)

Your "wrong but not obviously and completely wrong" line made me think that the "obviously and completely" part is what makes people who are well-versed in a subject demand that everyone should know [knowledge from subject] when they hear someone express obvious-and-complete ignorance or obvious-and-complete wrongness in/of said subject. I've witnessed this a few times, and usually the thought process is something like "wow, it's unfathomable that someone should express such ignorance of something that is so obvious to me. There should clearly be a class to make sure this doesn't happen." After reading what you wrote about compartmentalized knowledge and connected knowledge, this type of situation makes much more sense.