Worried that I might already be a post-rationalist. I'm very interested in minimizing miscommunication, and helping people through the uncanny valley of rationality. Feel free to pm me about either of those things.
I'm glad you're bringing sender-receiver lit into this discussion! It's been useful for me to ground parts of my thinking. What follows is almost-a-post's worth of, "Yes, and also..."
Stable "Deception" Equilibrium
The firefly example showed how an existing signalling equilibrium can be hijacked by a predator. What once was a reliable signal becomes unreliable. As you let things settle into equilibrium, the signal of seeing a light should lose all informational content (or at least, it should not give any new information about whether or not the signal is coming from mate or predator).
Part of the what ensures this result is the totally opposed payoffs of P.rey and P.redator. In any signalling game where the payouts are zero-sum there isn't going to be an equilibrium where the signals conveys information.
More complex varied payouts can have more interesting results:
Again, at the level of the sender-receiver game this is deception, but it still feels a good bit different from what I intuitively track as deception. This might be best stated as an example of "equilibrium of ambiguous communication as a result of semi-adversarial payouts"
I would not speculate on the mental life of bees; to talk of the mental life of bacteria seems absurd; and yet signalling plays a vital biological role in both cases. -Skyrms
I want to emphasize that the sender-receiver model and Skyrms' use of "informational content" are not meant to provide an explanation of intention. Information is meant to be more basic than intent, and present in cases (like bacteria) where there seems to be no intent. Skyrms seems to be responding to some scholars who want to say "intent is what defines communication!", and like Skyrms, I'm happy to say that communication and signals seems to cover a broad class of phenomena, of which intent would be a super-specialized subset.
For my two-cents, I think that intent in human communication involves both goal-directedness and having a model of the signalling equilibrium that can be plugged into an abstract reasoning system.
In sender-receiver games, the learning of signalling strategy often happens either through replicator-dynamics or a very simple Roth-Erev reinforcement learning. These are simple mechanisms that act quite directly and don't afford any reflection on the mechanism itself. Humans can not only reliably send a signal in the presence of certain stimulus, but can also do "I'm bored, I know that if I shout 'FIRE!' Sarah is gonna jump out of her skin, and then I'll laugh at her being surprised." Another fun example is that seems to rely on being able to reason about the signalling equilibrium itself is "what would I have to text you to covertly convey I've been kidnapped?"
I think human communication is always a mix of intentional and non-intentional communication, as I explore in another post. When it comes to deception, while a lot of people seem to want to use intention to draw the boundary between "should punish" and "shouldn't punish", is see it more as a question of "what sort of optimization system is working against me?" I'm tempted to say "intentional deception is more dangerous because that means the full force of their intellect is being used to deceive you, as opposed to just their unconscious" but that wouldn't be quite right. I'm still developing thoughts on this.
Far from equilibrium
I expect it's most fruitful to think of human communication as an open system that's far from equilibrium, most of the time. Thinking of equilibrium helps me think of directions things might move, but I don't expect everyone's behavior to be "priced into" most environments.
HOLY shit! I just checked out the new concepts portion of the site that shows you all the tags. This feels like a HUGE step in the direction the LW team's vision of a place where knowledge production can actually happen.
One way I think about things. Everything that I've found in myself and close friends that looks and smells like "shoulds" is sorta sneaky. I keep on finding shoulds which seem have been absorbed from others and are less about "this is a good way to get a thing in the world that I want" and "someone said you need to follow this path and I need them to approve of me". The force I feel behind my shoulds is normally "You SCREWED if you don't!" a sort of vaguely panicy, inflexible energy. It's rarely connected to the actual good qualities of the thing I "should" be doing.
Because my shoulds normally ground out in "if I'm not this way, people won't like me", if the pressure get's turned up, following a should takes me farther and farther away from things I actually care about. Unblocking stuff often feels like transcending the panicy fear that hides behind a should. It never immediately lets me be awesome at stuff. I still need to develop a real connection to the task and how it works into the rest of my life. There's still drudgery, but it's dealt with from a calmer place.
Some sort of "combination" seems plausible for perception. Baars actually mentions "The binding problem" (how is it that disparate features combine to make a cohesive singular perception) but I couldn't see how their idea addressed it.
This is actually one of the reasons I'm interested in looking for stuff that might be the "clock time" of any sort of bottleneck. Some amount of simultaneity of perception seems to be a post production thing. The psychological refractory period relates to experiments where you see and hear something and have to respond, and one seems to block the other for a moment (I haven't investigated this in depth, so I'm not v familiar with the experimental paradigm). But there are other things that totally seem like simultaneously experience modalities of perception. I wonder what sorts of experiments would piece apart "actually happening at the same time" from "rapid concurrent switching + post production experience construction". I'm very interested in finding out.
I don't know the concrete details about what "taking on a global value" looks like, but I visualize a grid (like in Kevin Simler's going critical post) that has a few competing colors trying to spread, and it seems reasonable that you could tweak the setting of the network such that very quickly one signal dominates the entire network.
But I don't know how to actually make something like that.
If you're interested in the TIN specifically, what I got from the paper was "here's a totally plausible candidate, and from what we know about self-organization in neural networks, it could totally do this functionality".
The biggest reason to think that there's something that's winner-take-all with a global value, is to explain bottlenecks that won't go away. Intentional conscious thought seems to be very serial, and the neural turing machine model does a decent jump of showing how a global workspace is central to this. If there's no global workspace, and there's just the thalamus doing sensory gating, and routing chunks of cortex to each other, I'd expect to see a lot more multi tasking ability.
Also, this is more a property than a constraint, if global communication works by routing then everything that's routed needs to know where it's going. This makes sense for some systems, but I think part of the cool flexibility in a GNW architecture is that all of the cortex sees the contents of the GNW, and subsystems that compute with that as an input can spontaneously arise.
Example #4: Logarithms. Jeff thinks we have a reference frame for everything! Every word, every idea, every concept, everything you know has its own reference frame, in at least one of your cortical columns and probably thousands of them. Then displacement cells can encode the mathematical transformations of logarithms, and the relations between logarithms and other concepts, or something like that. I tried to sketch out an example of what he might be getting at in the next section below. Still, I found that his discussion of abstract cognition was a bit sketchier and more confusing than other things he talked about. My impression is that this is an aspect of the theory that he's still working on.
George Lakoff's whole shtick is this! He's a linguist, so he frames it in terms of metaphors; "abstract concepts are understood via metaphors to concrete spatial/sensory/movement domains". His book "Where Mathematics Comes From" is a in depth exploration of trying to show how various mathematical concepts ground out in mashups of physical metaphors.
Jeff's ideas seem like they would be the neurological backing to Lakoff's more conceptual analysis. Very cool connection!
Thank you for pointing to the basal ganglia's relation to motor control! This feels like one of those things that's obvious but I just didn't know because I haven't "broadly studied" neuroanatomy. Relatedly, if anyone knows of any resources of neuroanatomy that really dig into why we think of this or that region of the brain as being different, I'd love to hear. I know there's both a lot of "this area defs has this structure and does this thing" and "an fMRI went beep so this is the complain-about-ants part of your brain!", and I don't yet have the knowledge to tell them apart.
Connecting this with the GNW, several of the salience cues used in the model are perceptual signals, e.g. whether or not a wall or a cylinder is currently perceived. We also know that signals which get to the GNW have a massively boosted signal strength over ones that do not. So while the GNW does not "command" any particular subsystem to take action, salience cues that get into the GNW can get a significant boost, helping them win the action selection process.
This was a very helpful emphasis shift for me. Even though I wasn't conceptualizing GNW as a commander, I was still thinking of it as a "destination", probably because of all the claims about its connection to consciousness. The "signal boosting" frame feels like a much better fit. Subsystems are already plugged into the various parts of your brain that they need to be connected to; the GNW is not a universal router. It's only when you're doing Virtual Machine esque conscious thinking that it's a routing bottleneck. Other times it might look like a bottle neck, but maybe it's more "if you get a signal boost from the GNW, you're probs gonna win, and only one thing can get boosted at a time".
Hmm, yeah looks like I got PP attention backwards.
There's two layers! Predictive circuits are sorta "autonomously" creating a focus within the domain of what they predict, and then the "global" or top-down attention can either be an attentional subsystem watching the GNW, or the distributed attentional-relevancy gate around the GNW.
The pandemonium stuff is also a great model. In another comment I mentioned that I'm fuzzy on how tightly or loosely coupled different subsytems can be, and how they are organized, and I was unintentionally imagining them as quite monolithic entities.
Your GNW has an active generative model built out of lots of component models. I would say that the "tennis-match-flow" case entails little sub-sub-components asynchronously updating themselves as new information comes in—the tennis ball was over there, and now it's over here. By contrast the more typically "choppy" way of thinking involves frequently throwing out the whole manifold of generative models all at once, and activating a wholly new set of interlocking generative models. The latter (unlike the former) involves an attentional blink, because it takes some time for all the new neural codes to become active and synchronized, and in between you're in an incoherent, unstable state with mutually-contradictory generative models fighting it out.
Ahhhh this seems like an idea I was missing. I was thinking of the generative models as all being in a ready and waiting state, only ever swapping in and out of broadcasting on the GNW. But a model might take time to become active and/or do it's work. I've been very fuzzy on how generative models are arranged and organized. You pointing this out makes me think that attentional blink (or "frame rate" stuff in general) is probably rarely limited by the actual "time it takes a signal to be propogated on the GNW" and much more related to the "loading" and "activation" of the models that are doing the work.
Hmm, you did notice a point where I sorta simplified Chalmers to get the post done.
Then one could answer "but why couldn't a computer system just have this enum-like system that had all the properties which match your subjective experience, without having that subjective experience?"
This is near a question I do think is interesting. I'm starting to think there's a sliding scale of "amount of subjective experience" a thing can have. And I am very curious about "what sorts of things will and won't have X amount of subjective experience".
I guess my beef is that when it's framed as "But why does XYZ system entail qualia?" I infer that even if in the far future I had a SUPER detailed understanding of "tweak this and you get X more units of experience, if you don't have ABC any experience is impossible, LMN architecture is really helpful, but not necessary" that Chalmers would still be unimpressed and got "But why does any of this lead to qualia?"
Well, I don't actually think he'd say that. If I had that sorta detailed outline I think his mind would be blown and he'd be super excited.
But when I imagine the person who is still going "But why", I'm imagining that they must be thinking of qualia is this isolated, other, and separate thing.
I've defs got socially formed priors on what things do and don't have experience. And when I try and move past those priors and or think "we'll these priors came from somewhere, what were they originally tapping into?" I see that anyone making a judgement about this is doing so through what they could observe.
Yeps. This feels like seeing one's experience of color as all of the things it's connected to. You've got a unique set of associations to diff colors, and that makes your experience different.
What I've seen of the "hard problem of consciousness" is that it says "well yeah, but all those associations, all of what a color means to you, that's all separate from the qualia of the color", and that is the thing that I think is divorced from interesting stuff. All the things you mentioned are the interesting parts of the experience of color.
Thanks! I might want to make another post at some point that really digs into subtle differences between rationality and legibility. Because I think a lot of people's rationality is legibility. It's like the shadow side of rationality.
In high-school I read pop cogSci books like "You Are Not So Smart" and "Subliminal: How the Subconscious Mind Rules Your Behavior". I learned that "contrary to popular belief", your memory doesn't perfectly capture events like a camera would, but it's changed and reconstructed every time you remember it! So even if you think you remember something, you could be wrong! Memory is constructed, not a faithful representation of what happened! AAAAAANARCHY!!!
Wait a second, a camera doesn't perfectly capture events. Or at least, they definitely didn't when this analogy was first made. Do you remember red eye? Instead of philosophizing on the metaphysics of representation, I'm just gonna note that "X is a construct!" sorts of claims cache out in terms of "you can be wrong in ways that matter to me!".
There's something funny about loudly declaring "it's not impossible to be wrong!"
In high-school, "gender is a social construct!" was enough of a meme that it wasn't uncommon for something to be called a social construct to express that you thought it was dumb.
Me: "God, the cafeteria food sucks!"
Friend: "Cafeteria food is a social construct!"
Calling something a social construct either meant "I don't like it" or "you can't tell me what to do". That was my limited experience with the idea of social constructs. Something I didn't have experience with was the rich feminist literature describing exactly how gender is constructed, what it's effects are, and how it's been used to shape and control people for ages.
That is way more interesting to me than just the claim "if your explanation involves gender, you're wrong". Similarly, these days the cogSci I'm reading is stuff like Predictive Processing theory, which posits that all of human perception is made through a creative construction process, and more importantly it gives a detailed description of the process that does this constructing.
For me, a claim that "X is a construct" of "X isn't a 100% faithful representation" can only be interesting if there's either an account of the forces that are trying to assert otherwise, or there's an account of how the construction works.
Put another way; "you can be wrong!" is what you shout at a someone who is insisting they can't be and is trying to make things happen that you don't like. Some people need to have that shouted at them. I don't think I'm that person. If there's a convo about something being a construct, I want to jump right to the juicy parts and start exploring that!
(note: I want to extra emphasize that it can be as useful to explore "who's insisting to me that X is infallible?" as it is to explore "how is this fallible?" I've been thinking about how your sense of what's happening in your head is constructed, noticed I want to go "GUYS! Consciousness IS A CONSTRUCT!" and when I sat down to ask "Wait, who was trying to insist that it 100% isn't and that it's an infallible access into your own mind?" I got some very interesting results.)
Have you used system verilog or some other hardware description language? Your clunk model of the ripple adder looks suspiciously like verilog code I wrote to make a ripple adder in a class. I can't recall enough deets to tell how different they are, but you might gain some insights from investigating.
Thanks! In my head, I was using the model of "flip 100 coins, exact value of all coins is micro states, heads-tails count is macro state". In that model, the macro states form disjoint sets, so it's probably not a good example.
I think I get your point in abstract, but I'm struggling to form an example model that fits it. Any suggestions?
I liked this paper and summary, and was able to follow most of it except for the actual physics :)
I feel like I missed something important though:
If we are trying to judge P(B|A,I|), what's the use of knowing the entropy of state A? The thrust I got was "Give weight to possible B in accordance with their entropy, and somehow constrain that with info from A,I", but I didn't get a sense of what using A,I as constraints looked like (I expect that it would make more sense if I could do the physics examples).
I think the sentiment was "even things that look like they might only be operating at level 1, they are also operating at other levels".
The fact that the stranger responds at all to your request for the bathroom signifies an amount of "We are on the same side enough to not physically attack each other". There are places where you can ask a stranger a question and they straight up won't answer you, or won't give you a true answer.
I've been writing A LOT on twitter lately. It's been hella fun.
One thing that seems clear. Twitter threads are not the place to hash out deep disagreements start to finish. When you start multi threading, it gets chaotic real fast, and the character limit is a limiting force.
On the other side of things, it's feels great for gestating ideas, and getting lots of leads on interesting ideas.
1) Leads: It helps me increase my "known unknowns". There's a lot of topics, ideas, disciplines I see people making offhand comments about, and while it's rarely enough to piece together the whole idea, I often can pick up the type signature and know where the idea relates to other ideas I am familiar with. This is dope. Expand you anti-library
2): gestation: there's a limit to how much you can squeeze into a single tweet, but threading really helps to shotgun blast out ideas. It often ends up being less step-by-step carefully reasoned arg, and more lots of quasi-independent thoughts on the topic that you then connect. Also, I easily get 5x engagement on twitter, and other people throwing in their thoughts is really helpful.
I know Raemon and crew have mentioned trying to help with more gestation and development of ideas (without sacrificing overall rigor). post-rat-twitter / strangely-earnest-twitter feels like it's nailed the gestation part. Might be something to investigate.
It might be useful to know that I'm not that sold on a lot of singularity stuff, and the parts of rationality that have affected me the most are some of the more general thinking principles. "Look at the truth even if it hurts" / "Understanding tiny amounts of evo and evo psyche ideas" / "Here's 18 different biases, now you can tear down most people's arguments".
It was those ideas (a mix of the naive and sophisticated form of them) + my own idiosyncrasies that caused me a lot of trouble. So that's why I say "rationalist memes". I guess that if I bought more singularity stuff I might frame it as "weird but true ideas".
I found this a very useful post. It feels like a key piece in helping me think about CFAR, but also it sharpens my own sense of what stuff in "rationality" feels important to me. Namely "Helping people not have worse lives after interacting with rationalist memes"
Why is there no formula for the roots of a fifth (or higher) degree polynomial equation in terms of the coefficients of the polynomial, using only the usual algebraic operations (addition, subtraction, multiplication, division) and application of radicals (square roots, cube roots, etc)?
Which makes me wonder; would there be a formula if you used more machinery that normal stuff and radicals? What does "more than radicals" look like?
I'm noticing an even more granular version of this. Things that I might do casually (reading some blog posts) have a significant effect on what's loaded into my mind the next day. Smaller than the week level, I'm noticing a 2-3 day cycle of "the thing that was most recently in my head" and how it effects the question of "If I could work on anything rn what would it be?"
This week on Tuesday I picked Wednesday as the day I was going to write a sketch. But because of something I was thinking before going to bed, on Wednesday my head was filled with thoughts on urbex. So I switched gears, and urbex thoughts ran their course through Wednesday, and on Thursday I was ready to actually write a sketch (comedy thoughts need to be loaded for that)