Oh, yes! I forgot about your original comment about this [LW(p) · GW(p)]

Iirc it's in the original boundaries sequence. Sorry I'm too lazy to look it up.

It's the part about natural bargaining points and people being able to 'go home'

Psychology may [… be …] ultimately very important, perhaps critically important

oh good, my other project is psychology, so I really hope it circles back.

I originally got into the AI safety stuff because I was trying to understand psychological boundaries properly and realized that no one did. Then I realized that many people were maybe doing the same thing when thinking about interactions between AIs and humans.

Hmm I'm confused about why people ask "where is the boundary?" in this situation. 

I visualize this situation as more of a topological map, kinda like

where the horizontal axes are kinda like physical space (more precisely: such that pieces (e.g.: your arm, your brain, web search, personal note taking app) thar are more closely connected are closer horizontally)

and the vertical axis is like "communication bandwidth". For example, the communication bandwidth (ability to sense and control) between your brain/nervous system and your arm is pretty darn high, probably higher than almost anything else in the universe.

Yeah, you could try to cut out a particular hill in that topology and say "that hill is Bob" but that's obviously cutting a lot of detail.

thanks just listened to it. This reminds me a lot of what Scott Garrabrant has been thinking about. Perhaps intentionally setting up membranes within a society so that failure/infection/etc. in one region doesn't infect every other region. They talk about the same thing but for insight in solving problems