The Territory

This comment is about something I'm confused about, and I'm sufficiently confused about it that I can't write it as a clearly-articulated question or statement. Its current state is more like a confused question that my brain in trying to untangle as I'm reading this sequence. So I'll probably meander, and the meandering probably won't come together into a clear satisfying thing by the end of this comment.

A big reason I'm interested in Logan-style naturalism is that you (Logan) frequently say things about it that resonate with ways in which I approach my own work. The most salient instance is your concept of "pre-conceptual intimacy":

In pre-conceptual intimacy, they're making a lot of fascinating observations and surprisingly quick improvements to relevant parts of life. But they're also feeling very confused and disoriented, because their pre-existing concepts around the problem just don't seem to make sense anymore, and they don't have new stories about what's going on yet either. They tend to utter phrases like, "Is memory even a thing?", "How could I ever have thought that?", and "I really have no idea what's going on with this, and it turns out I never have."

https://www.lesswrong.com/posts/EKc4RfKhPRmnLtRXn/research-facilitation-invitation [LW · GW]

There is a particular mode that I'm in when I do what I think of as my best work, and this paragraph reminds me of it. Although, hm, now that I actually have it in front of me, most of the details don't quite match… but I think that's mostly because most of the words in this paragraph are about pre-existing concepts, and when I'm in this mode, I mostly just don't pay attention to any concepts that don't currently fit.

–I think I maybe only feel particularly disoriented when I have pre-existing concepts that don't fit, but still feel like they encapsulate something important that I don't want to lose sight of?–

Anyway, when I try to correct for these things, "pre-conceptual intimacy" seems to resonate a lot with this mode that I sometimes work in.

However, an aspect of the description above that still doesn't match is that, when I'm working in this mode, I'm not in any very obvious way making any observations. It seems like most of what I'm doing is that I'm having a vague confused intuition, and I, uh. I bump them around in my head until they maybe turn into concepts that make sense, or are at least a little more like that? (Turns out that I don't actually particularly know how to describe the thing.)

It wouldn't be incorrect, I think, to say that I'm looking at two parts of my map that are inconsistent with each other, and I'm trying to make them consistent, or I'm looking at one part of my map (my confused intuition) and I try to use it to fill in a different, blank part of my map.

There are ways to increase some kinds of knowledge that largely involve staring at maps. Perhaps your own map is not clearly labeled in places, or it’s somehow inconsistent with itself, or it doesn’t match the map of an expert.

[...]

But the main thing a cartographer ought to be focused on, the vast majority of the time, is the world itself.

This seems obviously right for literal cartography. I want to talk, as part of my meandering, about how it might apply to math. I don't actually feel confused about math, I just find it a helpful example.

It seems to me that, on the one hand, (most) math research could be described as being all about staring at some parts of your map and trying to use them to fill in other parts of your map. But–

Now I mean something like, “The thing that is made of something other than my own perceptions and interpretations. The thing that resists my expectations, according to its own rules. The thing that does not care what I think, or what I have happened to imagine.”

–but on the other hand, math certainly resists my expectations according to its own rules. Moreover, when I try to do prove a math result, I am in contact with that resistance: I get feedback from the territory (of math), even though it seems like in some sense it wouldn't be incorrect to say that all I am doing is to stare at different parts of my map.

I think this is somehow an important node in my confusion (though, again, I don't actually feel confused about math): When reading your posts, I seem to have formed a, uh, story? frame?, that says that {getting feedback from the territory} is important and that it is sort of the opposite of {merely staring at your map}. So if I think of doing math as both "getting feedback from the territory" and "nothing but staring at your map", that breaks that model.

Maybe this just means that I should not think of math proofs as happening all in the map; maybe I should say that because doing math proofs gives me feedback about the way that math resists my expectations, therefore by definition it is not happening all in my map.

…I feel uncomfortable and kind of dirty about the previous paragraph, and that's after working on it for a while trying to make it less bad. As written, it seems to be saying: I have formed a picture of a constellation in my head, and now I'm looking at the sky and there is no star where my mental picture says there should be one; and I now want to know whether the real constellation instead has this nearby star or that one. Sometimes it genuinely makes sense to ask "does this way of thinking about things feel more revealing, or that one?" But in this case, it just feels like a wrong question. The real thing inside me I would like to convey is that I'm mentally lightly touching on each of the two pictures I could draw, getting in touch with how both feel somewhat right but fairly wrong, because touching on what the world looks like from these two wrong perspectives jiggers something around in my head that makes me feel that I'm a little closer to resolving my confusion.

Maybe this just means that I should not think of math proofs as happening all in the map; maybe I should say that because doing math proofs gives me feedback about the way that math resists my expectations, therefore by definition it is not happening all in my map.

But suppose I'm a physicist. I spend some of my time doing experiments, and I spend some of my time thinking about physics and about my experiments, and as I do the latter, I frequently do math. I'm not interested in studying math, the thing I want to study is physics, but math is an important tool for doing so. And when I use this tool, it resists my expectations just as it does when I do math for its own sake; once I have a formal model of some physical phenomenon, math tells me something about what to expect from my physical observations in a way that does not care what I think, or what I happened to imagine. But it feels like in the case of physics, something important is captured by saying that my experiments are making direct contact with the territory of physics, whereas the math I do is all in my map.

Worse, consider Einstein when he said that if Eddington's attempt to verify General Relativity had failed, "Then I would have felt sorry for the dear Lord. The theory is correct." [1] At this point, he had obviously done a lot of math about GR, but the math couldn't have given him that confidence that the theory was correct; given how little empirical observation [2] it was based on, it must have been {philosophical arguments slash his sense of how physics worked} that allowed him to come to this conclusion. And for him to predict so well on so little data, his philosophical intuitions must somehow have had the power to resist expectations according to their own rules, not in the way physics experiments do, but kind of close to the way mathematical calculations do. But if we tried to say that Einstein's philosophical intuitions didn't happen in the map, then… would any sort of thinking that actually does anything useful count as "happening in the map"?

[1] Each source I've checked seems to give a slightly different quote and story, which, uuuh, but anyway.

[2] (Empirical observation distinguishing it from Newtonian mechanics, I mean.)

I think that I personally have a tendency to spend a lot of time thinking about my map, and that I could, in many domains I care about, benefit from noticing a bunch of low-hanging fruit in making more direct observations. But I don't actually think that {what I consider to be my highest-quality work} to be an example of this. I mean, that work is certainly informed by intuitions I've formed in contact with present day machine-learning systems, or by doing math, or by watching my own thinking. But it's not, I think, made of contact with these things, and my contact with the territory (AGI alignment) seems to me about as tenuous as Einstein's with his. I think that the best work I do is in fact made up of thinking about what some existing parts of my map can tell me about what should be in other parts of my map.

So what am I to make of naturalism, or "Knowing the territory takes patient and direct observation"?

One story I could try to tell is that naturalism won't have much to tell me about the part of my work I consider most important. I could either say that "Knowing the territory takes patient and direct observation" is false because sometimes you can come to deep understanding of the territory just by thinking about it; or I could perhaps say that it's true, and that just thinking is useful but isn't enough to give you deep mastery of the subject; or I guess I could say that my approach to my work is just fundamentally doomed.

This story may be true. But it doesn't currently ring true, because it doesn't explain why numerous things you've said about naturalism have had such a strong resonance with my model of my work process.

When I do my thing that is in fact mostly "just thinking about it", I have a distinction in my head that I track with my felt sense, which feels as if it's tracking "whether I'm interacting with the real thing, or merely making up stories about it". In the first case, I feel like I am in contact with something that can resist my expectations (although in truth this thing is itself made of anticipations).

I very much have habits for this kind of work that rhyme with patience: I approach it with a frame of mind that "isn't expecting to find answers today", that is looking at the feeling of the thought on the tip of my tongue and pokes around in that vicinity but isn't expecting to be able to articulate that thought by any particular point in time. I look at the problem from this perspective and from that perspective, and feel successful if this shifts something in my felt sense that makes me feel a little bit less confused.

And there is an experience and sensation there that, as a felt sense, very much resonates with the idea of peeling back interpretative layers and increasing sensation at the point of contact, and with the metaphor of being "naked" in contact with the thing.

Maybe all of this is me missing you and interpreting your words in terms of something I'm familiar with. But what I actually think is going on is that your words are painting a picture of a constellation in my head that sort-of-but-not-quite matches the stars I see in the night sky; and that there is some nearby picture that does make sense of what I'm seeing; but I, like, just really don't know what that picture looks like, yet. (And so I look at it from this wrong perspective and from that one, and notice how that shifts my felt sense of it a little and makes me feel a little less confused–which is why I had to write a long meandering semi-essay in order to be able to say anything detailed about it at all.)

↑ comment by Duncan Sabien (Deactivated) (Duncan_Sabien) · 2022-02-16T19:52:02.794Z · LW(p) · GW(p)

Just a smidge of a reply, because I haven't grokked all of the above but this felt maybe useful:

Math (and certain ways of doing logic in general) has an unusual property in that it is an extremely parsimonious map?  It's a map that clings super tightly to the territory, with unusually little in the way of the problems gestured at in the image below.

So maybe that resolves the distinction between "wait, amn't I in full contact with the territory?" and "wait, amn't I also just staring at my map?"

Unlike most maps, math/logic don't throw a wide lasso around lots and lots of points; they're skeletal and precise, and so it's less the case that you will lose track of the difference between your map and the thing your map is mapping.

Maybe.

Snippet of thought.

This feels insightful.

I have experienced incremental_progress-towards-a-clearer-model_of_reality, which feels like one of the highest praise one aspiring rationalist can give another.

Fascinating. As someone who has used map and compass from almost as long as I can remember, work with maps everyday and having lived most of my life pre-cellphone, your description was also an insight into another way of looking at the world. What is obvious to one is not obvious to another and I suspect I would get quickly lost in other worlds you inhabit.

Can I recommend that you do a little reading on geomorphology? I first read a textbook on that at high school and it changed the way I looked at the world. Landscapes talked back to me. Studying geology and making geological maps massively reinforced it. You can anticipate what the landscape will do as you trasverse it. Like geological mapping, you are doing constant bayesian updates. Building a model in your head, making predictions and then a very short time later comparing it to what is in the territory itself. 

One of the drawbacks of having been a beta reader is having much less in the way of substantive commentary to offer (since it all got incorporated into the final draft).

But I will note that "confusing purpose for substance" seems like a really big deal when it comes to understanding humans, and How To Do Humaning Better.

We are really, really good at confusing purpose for substance.

A lot of what you're pointing at here reminds me of an idea I had for illustrating how the brain hemispheres work (based on Iain McGilchrist's new model, not the old debunked models from the 60s). I had an image of a comic or something depicting the two hemispheres as hiking partners, the left hemisphere (LH) with its face buried in the map and the right hemisphere (RH) looking around at the territory. And there could be a series of short stories showing how if they're not able to talk to each other they can get in various confusions, but how if they are able to coordinate effectively then the RH can help the LH update the map, and the LH's map can help make sense of the fully-detailed reality of the territory.

"Pre-conceptual intimacy" seems to point very much at the RH's way of attending to the world in general. A quote from a twitter thread of mine, paraphrasing McGilchrist: "whatever we experience comes to us first – it “presences” to us in unpreconceived freshness – in the RHem"

I've found understanding the nature of the brain hemispheres to be one of the most useful models I've taken in over the last few years, particularly for improving my relationship between map & territory and noticing how my attention affects what I experience. So I'd highly recommend it to you Logan and anyone who's interested in these questions you're exploring.

Ways Of Attending is a great short intro but costs $20 which is silly given that it's basically a long essay, like 30 pages. I've written a thorough summary of it (about half the length of the whole book) in this twitter thread. (I've also written a bit about it in this LW comment about how people are confused when they're saying "system 1" & "system 2" [LW(p) · GW(p)]) Then there's also The Master and his Emissary, his longer book with a couple thousand citations.

This elicited in me a very specific kind of joy I first experienced reading ZAMM, and for which I have made all too little time ever since. I have nothing substantial to say beyond that I find your prose delightful in the same way I find delight in Original Seeing. Thank you!

(Copying over FB reactions from while reading) There’s something that feels familiar so far. For myself and when I’m working with clients, I often encourage experiments and journaling about the experience as they go. Part of the reason is uncertainty about the result, but another part is taking the time to check your expectations against your actual experience as it’s happening.

Like, I recently felt really drained. Noticing that feeling, I could immediately say several things that had been draining, but I wouldn’t have said they were hard while I’d been doing them. I was just caught up in making things go smoothly at the time. It was only afterwards noticing that how drained I felt that made me realize the activity was very energy draining. It would have been easy to brush over the activity with “Oh, I had a fun time”, which was true. But it was also true that I had to recover afterwards.

I feel like there’s something valuable here in being able to pay the kind of attention to my experience that led me to that knowledge. But I don’t have a good vocabulary to describe it to my clients.