Posts
Comments
I think I've been unwaveringly lifeist the whole time. My main shift has been that I think I see some value in deathist sentiment that's absent from most lifeist rhetoric I'm familiar with. I want a perspective that honors why both arise.
I did dabble with ideas around whether death is an illusion. And I still think there might be something to it. But having done so, it looks like a moving goalposts thing to me. I still don't want to die, and I don't want my loved ones to die, and I think that means something that matters.
"Is the MMA guy actually getting his arm/tibia bent in fights in ways that they wouldn't if he mixed in some Aikido?"
I don't have enough MMA experience to know with much confidence. But from what little bit of BJJ rolling I've done, my impression is yes, folk who don't know the unbendable arm trick end up struggling sometimes in ways they don't have to.
It's reflected on both sides, really. If BJJ folk really understood this unbendable joint thing, they wouldn't keep trying to bend my arm to get through the grip I have (e.g. holding their lapel on either side with each hand). They pointlessly exhaust themselves. Usually more experienced folk will switch strategies at that point. But the fact that so many of them even try suggests to me that they're used to most people they roll with not being able to do this thing.
But I don't know. Maybe I'm just unfamiliar with those arts and this tool isn't useful in those situations.
The third is that he gives clear objective rules, I do a bad job on positioning/activation on my own accord, and then he shows me how to not do that. This is the only case where The Unbendable Arm is worth anything, and it's not because the technique itself is so great but because I was so dumb to start with.
I don't know about "dumb". Maybe "ignorant", the way an infant is ignorant of how to stand, or someone unpracticed will fall over if standing on one leg with their eyes closed. It takes a while of using the body in a way it hasn't been used before in order for the new skill to click into place.
Otherwise yeah, what you're saying makes sense to me.
And it's true, I don't know whether the unbendable arm is at all novel or useful to you. It's clearly novel for most people IME. Including very practiced martial artists who haven't otherwise worked with it. But I don't know, maybe you already do something equivalent. Or maybe it's irrelevant to the things you care about.
I assume you're still up in the bay area? I'm not likely to be up there anytime soon, but if you're ever back down in socal and want to play with this stuff let me know. It sounds fun.
I bet we'd come out of it with a force vector diagram and a good way to clearly demonstrate what's going on.
Yep, still in the Bay Area. Sounds good. And yep, I agree RE force diagram etc.
"This is supposed to stay standing" seems to be implied…
Maybe by others, but I don't need it. I can do it on my back. Or mid leap (though that's harder to demo :P ). Or upside down. I think I can do it with my arm stretched behind my back, but I'm less confident of that one; I'd have to try it.
But if my upper arm is pinned in a way that keeps me from moving my elbow, then yeah, I think that breaks the technique. Although in practice most people can't pin my upper arm in the way that matters. Even if they're trying to pin my arm to the ground. They'd have to try really really hard to fix my upper arm to the ground to get the unbendable thing to falter. At least in the ways I've encountered so far.
Similarly, what are the rules on footwork?
I'm not sure what you're asking. I can do it seated. Or while doing a shoulder stand. Or while lifted off the ground in a bearhug. I don't think there are implicit rules about footwork.
Do you have a preferred video demonstration? Or can you draw the force vector diagrams? I don't doubt that you're observing something real here, but from looking through YouTube I'm not seeing what you're describing.
Boy do I relate. For whatever reason, the demos on YouTube are almost universally weirdly bad. I'm guessing Ivan found the one he linked to because I linked to it; it's the clearest short demo I'd found.
I'll see about making one sometime. It's a little tricky for me in particular to convey over video because I'm so visibly strong. But I'll give it some thought. Maybe I can ask for help from some of the bodybuilders at one of the local gyms.
Sadly I can't draw a force diagram because I honestly don't know how it works. I can almost make sense of it. The technique works perfectly well if I put my wrist on an unyielding inanimate object like a table, so I think I'm somehow transferring the downward force near the elbow into the upward force on the wrist. But I'm not at all sure how that "somehow" works. I just… do it. By "extending ki". :-P
"I want him to tense up as physically hard as possible" -- and then he has to admonish his volunteer for "losing focus" when his unbendable arm bends
This one is just awful. Just utterly dumb. It's correct that losing focus when you're learning the technique causes it to fail. But at no point in the video does he demonstrate the actual thing. They're treating it like mysterious magic — which makes sense! It's hard to do without treating it like a bit of magic at first.
I can do the unbendable arm while distracted now. It's something my body just does when I choose for it to.
"the only purpose of this is for me to experience relaxation completely on this muscle" -- with emphasis that the arm is gonna bend at least a little bit and maybe more.
Yeah, I'm with you, this isn't impressive. It's kind of sort of right ish. But his understanding of how to do it is sloppy IMO. That slop shows up in how he stumbles around.
Complete relaxation is not necessary for doing this technique. But if you're doing the technique right, tremendous relaxation is possible. So if you want to check if you're doing it right, you can try relaxing more than you would be relaxed if you were fighting to keep your arm straight.
He's right that the arm might bend a little. It's an adjustment thing. Kind of like how your knees might bend a little if you catch a falling heavy object: it's just a spring action as your body adjusts to the new incoming force. But if you're doing unbendable arm right, you can actually let the arm completely fold up and then straighten it out again while they're applying force. It's quite easy.
But my guess is that he's not referring to adjustment. I think he's making excuses for poor technique.
This one shows something closer to what you're talking about, but it's pretty clear that he's cheating the starting position by giving himself a more advantageous position the second time and having the other guy start in a less advantageous position. Which I guess kinda raises the question of what exactly is it supposed to demonstrate? Is that "cheating" or is that the entire thing being conveyed?
At a glance, it looks to me like this guy really is doing the thing I'm talking about. There might be extra stuff going on, but my impression is that if you vary that extra stuff it won't affect the power of the unbendable arm. I really don't think it's a matter of "advantageous position".
I'm not going to die on the hill of "This guy is authentic." It's just a passing impression from watching the video. That said, I might start linking to this demo instead honestly! :-D Although I do like the original guy's looseness better.
I'd be happy to do this demo with you with the "advantageous positioning" mostly however you want it to be. I say "mostly" because there are some things that'll break the technique's ability to work, and I can nearly always tell you ahead of time what those will be. (E.g., if you rigidly pin my upper arm to a stationary object like a countertop, the technique won't work.) But it's really not about the level of advantage I think you're talking about. I can do the unbendable arm on my knees with my arm pointing straight over my head. I can do it with two people, one pushing up on my wrist and the other pushing down on my elbow. I hold totally inanimate objects using this technique, like sacks of groceries, by placing a wrist on a surface like a wall or a railing. I'm pretty darn sure it's a physics trick having to do with redirecting forces somehow. It's just a little tricky to learn how to do it.
My wife's first reaction to "don't let me bend your arm" was actually to swat my hand away from her wrist, which playfully points out that we're implicitly holding back in unspecified ways for sake of the demonstration, and if we were actually trying to not let someone bend our arm we would be doing something quite different. So what are the rules, anyway? As long as the rules are kept hidden it's really easy to move the goalposts around without anyone noticing.
I'm very, very confident it doesn't work via moving goalposts.
I don't know how to delineate all the "rules". If you change the context such that you're not testing the arm's unbendability, then you won't get to experience the thing I'm talking about. There are some ways of testing unbendability that will, in fact, get even my arm to bend in defiance of my trying to use this technique. There's an amount of force that should, in theory, cause tendons/ligaments/bones to start snapping, at which point of course the arm will fold.
Maybe there are other things for me to name there. I don't know.
But if you're concerned about any hidden rules here, feel free to ask me about them. If you give me a scenario, I can tell you whether (a) it's testing the thing I'm talking about and (b) how it'd do.
E.g., if you vice grip my upper arm and use a car jack against my wrist to force my arm to bend, it absolutely will. I have no power against that setup.
E.g., if you stare really hard at my arm to try to get it to bend but you don't touch it, you won't be testing the thing I'm talking about.
E.g., if you are trying to bend my arm like in many of these demos but you surprise me by smashing my foot really hard, I might stop doing the technique and you might successfully bend my arm — not because it failed but because I stopped doing it. (Although even here, if it were somehow really dire that I demo it, I might flinch and it might falter for a moment, but I'd be able to recover it and re-extend my arm against your force.)
And as I said up above, within some sensible limits I'm very happy to demo this technique in person. I'd ask that you not smash my foot or otherwise be mean about it! And there are some tests you might want to do that I can just flat-out tell you would cause the technique to stop working, so I don't think there's much point in running those. But if you want me to, say, do it blindfolded while lying on the ground and singing the national anthem, I'd be totally happy to do that. It'll work just fine.
I think this is related to how interfaces are selected for by functionality, not accuracy. Accuracy is one thing that can make them functional sometimes. But not always!
I like Donald Hoffman's example of computer GUI desktops. Those aren't meant to reflect the actual state of anything at the hardware level. A given "file" on your "desktop" might be scattered across your hard drive for instance. The point of the desktop is to create a kind of semi-fictional interface that humans can use. It's not totally disconnected from what's going on in the computer; otherwise it wouldn't work. But in many ways it'd get worse as a usable interface if you made it more accurate.
Likewise, motivational inner speech ("I can do it! I can do it!") isn't meant to be an epistemically justified conclusion. It's a way of drumming up resources to try something. Refusing to use it because its literal denotations aren't justified is a kind of confusion.
Another example is the common thing about fixing posture by imagining a string attached to the top of your head that's suspended from far up in the sky. Obviously there's no such string. But somehow visualizing and "feeling" the string and kind of "hanging" from it can often help people rearrange their spine in a helpful way. It's much harder to give specific instructions about what shifts to make in the spine: that's just not the natural interface for making the right adjustments.
A maybe stranger example is learning to balance on one foot with your eyes closed. A lot of that is just your body getting used to using the vestibular and proprioceptive senses, and lots of quick micro adjustments, to stay upright. But AFAICT it's completely irrelevant whether you understand that. You just… try. And you keep trying. And eventually something hidden happens, and you get better at it (probably — some don't!). The interface there is this magic "trying" thing. More accurate detail about what you're trying is just noise (AFAIK)!
It's unclear to me where the boundaries of this effect are. If I don a frame of faith in Christ, does this create a world interface that makes some things available that are harder to access in some other way? How would I tell? If it affects how pleasant life feels to me, and by it how open-hearted I'm able to be with those I love, and those who don't hold that faith don't seem to be as free to open their hearts… well, it kind of looks like the faithless are the confused ones, aren't they? Kind of like a person who won't use a GUI interface because it's deceptive.
(I'm not claiming that specifically faith in Christ does this, by the way! I'm giving a hypothetical example. I also think some people in fact do have a subjective experience as though this hypothetical example is real — but that involves a lot of social complexities such as that they're supposed to experience it as true. So please take it as a purely theoretical example. Or feel free to debate whether it's accurate of course! But please don't frame it as though you're debating with me about its accuracy.)
As the author, I want to note that I'm way more skeptical of my earlier explanation.
(Accordingly, I'm less confident of my current explanation too!)
I'm pretty sure the movement cannot work if you vice grip my upper arm such that it can't move relative to my body (or to the ground). The elbow must be free to rise a bit.
Sometimes I'll demo the trick by letting people fold my arm, asking them to continue applying as much pressure as they like, warn them that I'm about to start straightening my arm, and then just do so. It's close to effortless for me. But when I do it, my whole arm goes up first. I then lower my arm again once it's straight.
The lats might be closely involved, but I don't think that's the main issue. I haven't noticed any effect whatsoever in terms of unbendable arm strength based on lat strength.
Also, once someone who's vastly weaker than me gets the "click", they're able to (sometimes) defy me regardless of how much muscular strength I put into it.
So I'm pretty sure it's some kind of leverage trick.
My best guess right now is somethingsomething redirecting force between the incoming hands into each other somethingsomething.
I use the same feeling to hold very heavy objects when I put my hand on a table. Like hanging a heavy grocery bag on my elbow with my arm straight and my palm up. I keep it suspended by letting the bag press the back of my hand into the table. IME I'm not so much fighting the bag's bending my arm as I am just… reaching. If you make the bag heavier, it presses my hand harder into the table. That's it. It's not relevantly harder for me to keep the bag suspended.
Again, no lats.
If heaven doesn't exist, believing in it, and even acting as though you want to go there, won't get you there.
I'm not so sure. My impression is that while dying, perception of time and reality can break down. It might be kind of like falling into a dream state or dropping into an intense psychedelic trip. As the subjective experience of time breaks down, each moment can stretch out until it's subjectively eternal. If at that point you have a well-developed belief in Heaven, that could very well be what opens up for you, and you could reside there "forever".
Given that it's going to feel like something as you die, it sure seems preferable that it be something utterly wonderful. Rather than (say) something horrifying as your animal terror around dying defines the thoughts and anticipations that shape the psychedelic dreamscape you "eternally" fall into. Shaping the dying experience sure seems to me like it'd require some kind of prep work.
Obviously I agree that if Heaven isn't a place that your eternal soul literally goes to, then what you believe won't get you there. Because there's no "there" to get to.
I just want to suggest that maybe that's a strawman. Depending on your disposition, you might really wish you'd developed faith in Christ or whatever as you watch your death take you. At that point "salvation" won't be theoretical, I'm guessing. It won't matter that Heaven/Hell/whatever is "just" a dying brain experience; that's not much consolation in the middle of it IME.
(This thought inspired by both strange meditative states and some horrific psychedelic experiences. Hence "IME", not "IMO". Both are a bit deceptive though: I don't mean to say that my experiences are for sure equivalent to the dying experience.)
Kudos on doing the test!
FWIW, the key thing in unbendable arm isn't about tensing only the relevant opposing muscles. It's more about redirecting incoming forces at each other instead of fighting them directly.
The real test is in dealing with someone who's way stronger than you. If you have no hope of keeping your arm straight via tricep strength.
I'm guessing that either (a) you're not much stronger than your wife or (b) she didn't click into the thing the visualization is meant to help people click into.
Seriously, the thing I mean when I point at this technique isn't a vague "energy" trick that fails upon encountering an MMA fighter or whatever. And it definitely doesn't rely on subtly deceiving people into tensing the wrong muscles. Unbendable arm is immensely demonstrable. As weird as the following might sound, I use it every day. It's easily in the top ten most useful things I learned from aikido and might be in the top three.
Oh cool! In aikido we'd call that "extending ki" (in contrast to "cutting ki", where your "mind" stops at the point of contact and your "ki" (here roughly "followthrough") abruptly halts).
Huh, I'm not aware of having deleted that post! I wonder where it went.
FWIW, after teaching a bunch of folk the unbendable arm, I've had to revise my impression in that original comment. Telling people the physics helps in a few cases, and seemed to help pretty dramatically in the first few (hence my original comment), but the variance was just way wider than I thought.
(Aside: wanting to acknowledge that the strength of my earlier comment wasn't in epistemic integrity. It's part of a communication pattern of mine that I've been examining lately.)
My impression is that people actually get to the full unbendable arm more reliably from the firehose imagery. In most cases, describing the physics does not help them find the surprising ease. They might succeed in keeping their arm straight, but it tends to be a struggle.
The firehose thing is not how I do it though! It's more like, there's an extension feeling in my limbs. When someone tries to bend my arm, if I'm already focusing on the extension feeling, they just can't bend it. If it starts to bend a little bit, I reach a little more. No effort against the force. It's more like I'm trying to reach the far wall — but just a little.
But telling people to reach doesn't seem to help as much. IME folk tend to reach by bending their spine, which seems to destabilize the whole thing. (Doesn't have to, but in practice it does.) They also switch from reaching to fighting once the pressure is on.
Things like Alexander Technique seem to help a lot more. Getting them to focus on the horizon and being peripherally aware of the pressure, instead of collapsing their awareness on the struggle at their arm.
Maybe more detail specifically about unbendable arm PCK than most need! But since my earlier comment now looks deceptive to me, I wanted to offer a correction.
I've been reflecting on this since it was posted. Coming back to it from time to time.
I just wanted to make a note saying: received. I believe I see your point, and I've been taking it in.
(I also disagree with some of the narrativemancy you employ here about me. It's difficult for me to publicly agree with anything in your comment here because of a similar mechanism that you're objecting to in the OP. I wish I didn't need to add this note. I'd rather just say "I'm hearing something meaningful in what you're saying and have been taking it in." That's the part that matters. But I can do so only if I also register that I very much disagree with your model of what I was trying to do, and in some cases I strongly disagree with your analysis of what I was in fact doing. Thankfully I can still learn from your message anyway. I just wanted to say — more to LW than to you, really — that I've heard your objection and have been taking the truth I can find in it seriously.)
It seems to me that there are arguments to be made in both directions.[1] It's not clear to me just yet which stance is correct. Maybe yours is! I don't know.
My point is that it's understandable for intelligent people to suspect that there isn't enough data available yet to produce ASI on the current approach. You might disagree, and maybe your disagreement is even correct, but I don't think the situation is so vividly clear that it's incomprehensible why many people wouldn't be persuaded.
- ^
As a quick gesture at the point: as far as I know, all the data LLMs are processing has already gone through a processing filter, namely humans. We produced all the tokens they took in as training data. A newborn, even blind, doesn't have this limitation — and I'd expect a newborn that was given this limitation somehow very much could have stunted intelligence! I think the analog would be less like a blind newborn and more like a numb one, without tactile or proprioceptive senses.
I think the issue is that the form of the data matters a lot. ChatGPT basically just works with symbols as far as I know. How many symbols are there for it to eat? Are there enough to give the same depth of understanding that a human gets from processing spatial info for instance?
I don't think this is a hard limit for AI in general. But I can totally understand why there's skepticism about there being enough digital data right now to produce ASI on the current approach. I'd expect the AIs would need to develop different kinds of data collection, like using robots as sense organs for instance.
Nah. I could have been more precise while spitballing. (For some definition of "could". I don't know how to be freeform playful/creative while carefully scrutinizing each detail of what I say for precision and accuracy.) But I meant all that more in the spirit of "Huh, I wonder if evolution did something like this and we've been assuming it didn't. That'd make some sense of why some of our social reform efforts go wonky in these particular ways."
I don't know what would be good for us to collectively do. I aspire to have good questions. Good answers would be nice, but I think those mostly fall out of seeking and pondering good questions.
It's very disconcerting to read "I notice my brain does extra work when I talk with women... wouldn't it be easier if society were radically altered so that I didn't have to talk with women?" Like, what? And there's no way you or anyone else can become more rational about this? This barrier to ideal communication with 50% of people is insurmountable? It's worth giving up on this one? Hello?
I was not proposing that.
This was fun to look back on a year later.
I like how thoroughly I owned it. How it's clearly wild speculation that tries to weave together some questionable observations. I also like the questions it raises. I remember how the basic hypothesis — that sexual signaling maybe evolved largely to aim at one's own sex in the human social context — helped me to notice some interesting questions I hadn't picked up on before. (E.g., why does slut-shaming seem to more target women who doll up for the male gaze instead of women who are actually pretty openly DTF when there's a difference? Is my perception off, or is this really a trend, and if so then why?)
I notice that a year later I'm simply unconvinced of the basic idea. I don't know why. It might be that there's something true that (some part of me thinks) works best if it goes unrecognized, so I need to blind myself to it. Or, maybe it's just malarky. I don't know!
The main thing, I think, is that on the whole the framework doesn't feel wholesome. It's not totally icky to me. Just… off? Like, sure, respecting people and accounting for sexual dynamics is important. There are probably built-in tendencies along these lines. Those tendencies seem interesting to examine and explore. But despite my attempt to be descriptive rather than normative in this post, there's a kind of implication that trying to be attractive to the opposite sex is defecting on culture. That implication seems kind of weird to me now.
To the extent that there's a glitchiness in trying to be attractive to others, I now suspect it's more to do with Goodhart drift. A guy who's just living his life and thus ends up attractive is less likely to be creating a hollow illusion than some other guy who chooses what to do based on what he thinks will get him a girl. As just one example type! It's a very general point about trying to affect how people see you, instead of just doing stuff and letting people see the truth: the former tends to invoke Goodhart, and thus people are often distrustful of it. But that doesn't mean it doesn't work sometimes anyway! Or even that it's bad.
The bonobo analogy was unfortunate. It was a good illustration, except that it was confusing. I mention the problem in one of the comments. The fact that bonobos specifically use sex is irrelevant to why I named them. The point was that when there's a potentially scarce resource introduced, the tribe starts by affirming their within-tribe bonds instead of leaping into competition for the resource. (Or at least so I'm told!) The fact that they do so with sex wasn't relevant to the analogy. Since the rest of the post was about sexual dynamics, I think that particular choice of analogy was unnecessarily confusing.
All that said: I like the kind of wild-yet-owned speculation this post is made of. It's fun and thought-provoking. I find it stimulating just to reread!
I'd love to see something that massages the various observations here into a better explanation. In particular, there's nothing meaty here that really tackles why there's an asymmetry between men and women in terms of intrasex popularity. (Women swoon over men who are (a) beloved and trusted by other men and/or (b) highly desired by other women. Men don't seem to care whether a woman has lots of girlfriends who love her and are all over the place when it comes to lots of other men wanting her.) The loose model doesn't predict it and can't even neatly explain it. Or at least I don't see how!
I honestly don't know how much, if any, further thinking or insight or experimentation arose from this post. It might have just amounted to a passing entertainment. It affected some of my thinking and actions in the first half of this year, but I don't think the ideas have meaningfully come to mind since roughly the summer.
I've changed how I want to talk about all this stuff quite a bit since 2018. I don't talk about "Looking" for the most part anymore. Not for carefully thought-out reasons. I just don't like the feel of trying to describe this stuff that way anymore. It feels over-reified and too… prideful. Not just about me, but as in, the framework seems to imply a skill or capacity a person has or doesn't have, and that it's better if they have it. I no longer think that's how grace works.
With that caveat: yes probably? I'm guessing Focusing can help. I'm not sure though!
My impression these days is that the kenshō "insight" is basically what you recognize when you stop restricting your perceptions with your cognitive frames. The tricky part is that what I just said is a cognitive frame, so the conceptual mind can take what I just said and claim to have some kind of understanding of the kenshō thing. But it can't. It literally cannot understand it. It's like an LLM with a pure text interface talking about truly appreciating visual art: it might give some amazing and even helpful analysis, but it doesn't have the right type of input or processing to see a painting at all.
Focusing might help by giving the system a way of orienting to things in a non-conceptual way. The conceptual mind can still create frames during and after, but they're only kind of helpful to the Focusing process, so the conceptual mind can't lead the Focusing effort.
"You are not the king of your brain. You are the creepy guy standing next to the king going, 'A most judicious choice, sire'."
(Here I'm hinting at viewing "you" as the identity structure that lives within the conceptual mind — what some spiritual/mystical places sometimes mean when they say "ego".)
So, that's my guess. In short, I'd guess yes? But I really don't know.
You ask a good question. I have a lot of thoughts about it. Different answers at different levels. Like, what should a civilization do vs. what should a parent do vs. what should a teacher do? Different answers.
The overall theme, though, is to remove coercion and appeal to native fascination instead. If you have something of value to the student to offer, then in practice there's a way to either (a) show the student that value or (b) earn the student's trust that you're tracking what they care about such that when you say "Trust me" they know there's something good there even if they can't see it for themselves just yet.
If you're aiming to be a teacher… well, it's tricky because last I checked, the systems you're embedded in impose mandatory coercion. You have to cover certain topics, often in a certain order, within a certain window of time, etc. Especially since "No Child Left Behind" tied funding to test scores. And parents get mad and start rattling sabres if their kids come back from math class with a bunch of weird stuff the parents don't recognize. Although maybe that was just the Boomers.
But that said! There are clever ways of working within these social constraints. If you can do that, the overall thrust for a teacher is to prioritize being curious about how the students are thinking rather than on getting them to understand certain concepts.
The lion's share of work for a really good math teacher is in identifying zinger questions. You have to see how a student is thinking about a problem, and follow their contours of reasoning, and notice where it's going to run them into trouble. You could just tell them about the trouble, but it's far more effective to ask them to explain something or figure out something that will lead them right to the paradox spot.
After a while you'll probably develop a really rich repertoire of such questions. And maybe more preciously, you'll be familiar with a vast library of thinking styles that students actually use in the parts of math that you teach. This is what the education literature refers to as "pedagogical content knowledge" or "PCK" (which is where the CFAR class on "Seeking PCK" came from).
That's my main answer. Two other points worth mentioning:
- By the time they're in high school, the basis of their math trauma will probably have already formed. You're not likely to be the tipping point into horror for them.
- Math trauma is way more reversible than most people realize.
So don't worry about that part too much. Just zoom in on what you love about the subject, stay in contact with the kids' wonder, and aim to be a guide facilitating their exploration of what fascinates them. I think good things follow pretty naturally from that.
Would you say the same of most other class subjects?
I was homeschooled and then studied math education, so I'm not sure. But my passing impression is (a) yes, it applies to most methods of teaching in schools regardless of subject; but (b) math taught this way is particularly damaging.
I want to emphasize that this is my impression. I'm also not entirely sure why math seems to be more damaging. I have guesses. I just observe that e.g. literature hatred or music phobia aren't nearly as prevalent as math trauma is. Best as I can tell.
I ask because, with the exceptions of reading and persuasive writing, I don't think that any conventional school subject is more applicable to the average person's life than grade-school math.
Yes, people can get through life with an astonishing ignorance of mathematics, but they can get through life with an even more astonishing ignorance of social studies, literature, and the sciences.
Well, sure. But people will also pick up the math they need as they need it for the most part. That's true of most subjects really.
I didn't learn to read in school. I went to kindergarten before being homeschooled, and they were teaching us the alphabet and some basic words, but I could already read books by then. I learned to read because I wanted to read.
There's something very weird in our cultural groundwater around what teaching is. It's like we start with a prescription of subjects and then default to coercion to get students to "know" those subjects. Why? If it's relevant to their lives, we could learn to point out the connection in a way that feels alive to them. If we can't do that, then what makes us so sure that it's relevant for them?
Do you have a different philosophy of education, a different ranking of subjects' importance, or something else?
Yeah I do. I think the most imporant function of widespread education is to make good citizens. Which is to say, children put through an education system need to come out of it better able to engage with the system that runs their civilization, including the education process for the next generation.
In the United States, I think that puts civics as the most important subject. It's really key that citizens understand how their government works, what the checks and balances are, how jury nullification works, what forms of corruption actually do arise even within the current system, etc. Otherwise they don't know how to participate in the government that's supposedly "by the people, for the people". This is vastly more important than learning math they don't naturally pick up in their day-to-day lives.
I think the two things you named are really good though. I wish public education had those as real goals! That'd be nice. I think (1) happens sort of despite the education methods, and (2) happens more through other cultural channels than it does through formal public education. Just my impression.
To me this is exciting. I deduced that the mental architecture you're describing should be possible. It's extremely cool to hear someone just name it as a lived experience. Like, what would a mind that's actually systematically free of Newcomblike self-deception have to be like, assuming the hostile telepaths problem is real? This is one possible solution. Assuming I haven't misunderstood what you're describing!
Ah yeah, I think "gaining independence" is a better descriptor of (what I meant by) that solution type.
A few examples:
- Framing kids as "disruptive" or "inattentive" or otherwise having the wrong nature if they feel disengaged. This is after informing them what they're going to study without consulting what's relevant or interesting to them, and then using social power to require them to study those things. But the problem is supposedly the student, not the system.
- Claiming that they'll need these math tools later in life, and that this justifies adults pressuring the kids to learn those skills now. (This is more bullshit-flavored than gaslight-flavored, but I think they're psychological neighbors.)
- Pretending that because a word problem touches on a topic kids care about, the math is relevant to what the kids like about that topic.
- Insisting that forcing kids to take math classes is for their own good, and if the kids don't see why or don't agree, then they should believe the adults over their own sense of things.
It makes me so angry. It's perfectly antithetical to the essence of math as I see it.
In broad strokes I agree with you. Here I was sharing my observation of four cases where a friend was involved this way. One case might have been miscommunication but it doesn't seem likely to me. The other three definitely weren't. In one of those I personally knew the guy; I liked him, but he was also emotionally very unstable and definitely not a safe father. I don't think the abuse was physical in any of those four cases.
! I'm genuinely impressed if you wrote this post without having a mental frame for the concepts drawn from LDT.
Thanks. :)
And thanks for explaining. I'm not sure what "quasi-Kantian" or "quasi-Rawlsian" mean, and I'm not sure which piece of Eliezer's material you're gesturing toward, so I think I'm missing some key steps of reasoning.
But on the whole, yeah, I mean defensive power rather than offensive. The offensive stuff is relevant only to the extent that it works for defense. At least that's how it seems to me! I haven't thought about it very carefully. But the whole point is, what could make me safe if a hostile telepath discovers a truth in me? The "build power" family of solutions is based on neutralizing the relevance of the "hostile" part.
I think you're saying something more sophisticated than this. I'm not entirely sure what it is. Like here you say:
Basically, you have to control things orthogonal to your position in the lineup, to robustly improve your algorithm for negotiating with others.
I'm not sure what "the lineup" refers to, so I don't know what it means for something to be orthogonal to my position in it.
I think I follow and agree with what you're saying if I just reason in terms of "setting up arms races is bad, all else being equal".
Or to be more precise, if I take the dangers of adaptive entropy seriously and I view "create adaptive entropy to get ahead" as a confused pseudo-solution. It might be that that's my LDT-like framework.
I like this way of expressing it. Thanks for sharing.
I think it's the same core thing I was pointing at in "We're already in AI takeoff", only it goes in the opposite direction for metaphors. I was arguing that it's right to view memes as alive for the same reason we view trees and cats as alive. Grey seems to be arguing to set aside the question and just look at the function. Same intent, opposite approaches.
I think David Deutsch's article "The Evolution of Culture" is masterful at describing this approach to memetics.
(Though maybe I should say that the therapist needs to either experience unconditional positive regard toward the client, or successfully deceive themselves and the client into thinking that they do. Heh.)
I mean, technically they don't even need to deceive themselves. They can be consciously judgy as f**k as long as they can mask it effectively. Psychopaths might make for amazing therapists in this one way!
I think the word "power" might be creating some confusion here.
I mean something pretty specific and very practical. I'm not sure how to precisely define it, but here are some examples:
- If someone threatens to freak out at you if you disagree with them, and you tend to get overwhelmed and panic when the freak out at you, then they have a kind of power over you. Building power here probably looks like learning to experience them freaking out without you getting overwhelmed.
- If someone pays for your rent and food but might stop if they get any hint that you're gay, it might not be safe to even ask yourself honestly whether you are. You build power here by getting an income, or a source of rent and food, that doesn't depend on the hostile telepathic benefactor.
- If your lover gets turned on by you politically agreeing with them and turned off by disagreement, you might find your political views drifting toward theirs for "unrelated" reasons. One way to build power here is to get other access to sex. Another is to diminish your libido. Another is to break up with them. (Not saying any of these are a great idea. I'm just naming what the solution of "building power" might look like here.)
I'm not familiar with LDT. I can't comment on that part. Sorry if that means what I just said misses your point.
The fact that Bob has this policy in the first place is more likely when he's being self-deceptive.
I don't know if that's true. It might be. But some possible counterpoints:
- People can distrust systems that demand they check. "You have nothing to fear if you have nothing to hide" can get a response of "No" even from people who don't have anything to hide.
- If someone subconsciously thinks they can pull off the illusion of honestly looking while in fact finding nothing, they become more likely to choose to look because they're self-deceiving.
- Someone with a policy of not looking might be better at making their own self-deception unnecessary.
…more often it will be the result of Bob noticing that he's the sort of person who might have something to hide.
Sure, that way of deciding doesn't work.
Likewise, if you're inclined to decide you're going to dig into possible sources of self-deception because you think it's unlikely that you have any, then you can't do this trick.
The hypothetical respect for any self-deception that might be there needs to be unconditional on its existence. Otherwise, for the reason you say, it doesn't work as well.
(…with some caveats about how people are imperfect telepaths, so some fuzz in implementation here is in practice fine.)
That said, I think you're right in that if Omega-C is looking only at the choice of whether to look or not, then yes, Omega-C would be right to take the choice as evidence of a deception.
But the whole point is that Omega-C can read what conscious processes you're using, and can see that you're deciding for a glomerizing reason.
That's why why you choose what you do matters so much here. Not just what you choose.
It's a general rule that if E is strong evidence for X, then ~E is at least weak evidence for ~X.
Conservation of expected evidence is what makes looking relevant. It's not what makes deciding to look relevant.
If I decide to appease Omega-C by looking, and then I find that I'm self-deceiving, the fact that I chose to look gets filtered. The fact that this is possible is why not finding evidence can matter at all. Otherwise it'd just be a charade.
Relatedly: I have a coin in my pocket. I don't feel like checking it for bias. Does that make it more likely that the coin is biased? Maybe. But if I could magically show you that I'm not looking because I honestly do not care one way or the other and don't want to waste the effort, and it doesn't affect me whether it's biased or not… then you can't use my disinterest in checking the coin for bias as evidence of some kind of subconscious deception about the coin's bias. I'm just refusing to do things that would inform you of the coin's possible bias.
If this kind of reasoning weren't possible, then it seems to me that glomerization wouldn't be possible.
It's not very hard to detect when someone's deceiving them self…
A few notes:
- Sometimes this is obviously true. I agree.
- It's a curious question why many folk turn their attention away from someone else's self-deception when it's obvious. Often they don't, but sometimes they do. Why they (we) do that is an interesting question worthy of some sincere curiosity.
- Confirmation bias. You don't notice the cases where you don't pick up on someone else's self-deception.
…people should notice more and disincentivise that
Boy oh boy do I disagree.
If someone's only option for dealing with a hostile telepath is self-deception, and then you come in and punish them for using it, thou art a dick.
Like, do you think it helps the abused mothers I named if you punish them somehow for not acknowledging their partners' abuse? Does it even help the social circle around them?
Even if the "hostile telepath" model is wrong or doesn't apply in some cases, people self-deceive for some reason. If you don't dialogue with that reason at all and just create pain and misery for people who use it, you're making some situation you don't understand worse.
I agree that getting self-deception out of a culture is a great idea. I want less of it in general.
But we don't get there by disincentivizing it.
…I went in the other direction: trying to self-deceive little, and instead be self-honest about my real motivations, even if they are "bad PR".
Yep. I'm not sure why you think this is a "very different" conclusion. I'd say the same thing about myself. The key question is how to handle the cases where becoming conscious of a "bad PR" motivation means it might get exposed.
And you answer that! In part at least. You divide people into three categories based on (a) whether you need occlumency with them at all and (b) whether you need to use occlumency on the fact that you're using occlumency.
I don't think of it in terms this explicit, but it's pretty close to what I do now. People get to see me to the extent that I trust them with what I show them. And that's conscious.
Am I misunderstanding you somehow?
Moreover, having an extremely difficult high-stakes problem is not just a strong reason to self-deceive less, it's also strong reason to become more truth-oriented as a community. This means that people with such a common cause should strive to put each other at least in category 2 above, tentatively moving towards 3 (with the caveat of watching out for bad actors trying to exploit that).
I both agree and partly disagree. I tagged your comment with where.
Totally, yes, having a real and meaningful shared problem means we want a truth-seeking community. Strong agreement.
But I think how we "strive" to be truth-seeking might be extremely important. If it's a virtue instead of an engineering consideration, and if people are shamed or punished for having non-truth-seeking behaviors, then the collective "striving" being talked about will encourage individual self-deception and collective untalkaboutability. It's an example of inducing adaptive entropy.
Relatedly: mathematicians don't have truth-seeking collaboration because they're trying hard to be truth-seeking. They're trying to solve problems, and they can verify whether their proposed solutions actually solve the problems they're working on. That means truth-seeking is more useful for what they're doing than any alternatives are. There's no need for focusing on the Virtue of Seeking Truth as a culture.
Likewise, there's no Virtue of Using a Hammer in carpentry.
What puts someone in category 2 or 3 for me isn't something I can strive for. It's more like, I can be open to the possibility and be willing to look for how they and I interact. Then I discover how my trust of them shifts. If I try to trust people more than I do, I end up in more adaptive entropic confusion. I'm pretty sure this is lawful on par with thermodynamics.
This might be what you meant. If so, sorry to set up and take a swing at a strawman of what you were saying.
I think I disagree. I'll add some precision to point out how. Happy to hear if I'm missing something.
E is Bayesian evidence of X if E is more likely to happen when X is true than when it's not.
If Bob says "As a policy, I'm not going to check whether I'm running an Omega-C deception", that's equally likely whether Bob is running a deception or not. (Hence the "as a policy" part.) It just fully happens in both cases. So from Omega-C's point of view, it's not Bayesian evidence that distinguishes between the two versions of Bob.
It would be evidence if the choice were made from a stance of "Oh shoot, that might be self-deception! Well, I'm now going to adopt the no-looking policy so that I don't have to check it!" Then yeah, sure, that's clearly evidence — which is precisely why that method of deciding not to look isn't what can work.
The policy of always deeply investigating oneself can produce evidence for Omega-C, but the act of choosing that policy might not. Choosing the policy not to look just doesn't produce evidence.
Or at least that's how it seems to me.
I can secondhand lend some affirmation to the newcomb case.
Oh yeah, that's a cool example.
Another solution is illegible-ization/orthogonalization of preferences to the hostile telepath so that you don't overlap in anything they might care about or overpower you with.
You mean something like, look boring to them? Like, I don't care how good Putin is at reading people, I just don't have anything he wants, so I'm safe as long as I keep (apparently) not having anything he wants?
Cool. I knew there at least used to be "antisocial personality disorder", which I thought was under cluster B along with narcissism and borderline. And I thought "psychopathy" was a different term for APD. Thanks for the correction.
The main thing I wanted to gesture at there is that I wasn't using "psychopath" as something derogatory. I didn't mean "bad guys". I meant something more like "people who are naturally unconstrained by social pressures and have no qualms breaking even profound taboos if they think it'll benefit them". (I just now made that up.) It seems to me that it's a pretty specifically different mental/emotional architecture.
Some cultures used to, and maybe still do, have a solution to the hostile telepaths problem you didn't list: perform rituals even if you don't mean them.
Ah, yep! True that!
Your point relates more directly to my main interest, memetics. I bet there are memes that encourage both (a) these rituals and (b) the telepathic attacks that make those rituals necessary.
Oh huh. Yeah. It's not a solution by itself since there are lots of other cues hostile telepaths can use. But rigidity might dampen what they can read for sure!
This is testable. It predicts that improved skill with occlumency and/or gaining power should sometimes cause a release of chronic tension.
I think breastfeeding is different because… public health people decided it should be, and we’ve internalized their messaging.
I haven't gone around and checked much, but my gut impression isn't that this is about public health people. I think it's more like a Chesterton's Fence backlash against previous generations' experts claiming that formula was obviously better. IIRC, mothers were warned against using breastmilk and told to go to formula instead, because it's Scientific™. So it took some cultural pushback to reclaim evolution's solution to feeding newborns.
I haven't read OP yet, just a quick translation note:
The Sanskrit word "tanha" shares an etymology with English words like "tenacious", "tendency", and "tenet". The PIE root means "grip" or "hold".
I think most folk in my social circles who use "tanha" these days are referencing Romeo's "(mis)Translating the Buddha":
Tanha is usually translated as desire or craving but this is wrong and misleading. Tanha is more literally translated as 'fused to' or 'welded to'. It immediately follows the mental moment that you zoom in with the attentional aperture on something. It could be that a flower or an item on the shelf at the supermarket captures your attention, or you turn your head to catch more detail as you pass by an accident on the road. Many hundreds of thousands of such events take place in the course of a single day. With most of them attention then relaxes and makes space for the next thing. But with some small proportion you find the mind doesn't quite 'unclench' from the object or some aspect of the object. This tension aspect is why it is sometimes translated as ‘grasping’ which is closer. Imagine something you aren’t finished with being pulled out of your hand and you tensing your fingers to resist.
That seems maybe true. What's the problem you see with that?
I consider ultra-BS a primarily 'central route' argument, as the practitioner uses explicit reasoning to support explicit narrative arguments. […]
Putting someone off balance, on the other hand, is more 'peripheral route' persuasion. There's far more emphasis on the implicit messaging.
Ah! This distinction helped clarify a fair bit for me. Thank you!
…I think I might conclude that your implicit primers and vibes are very good at detecting implicit persuasion, which typically but not always has a correlation with dark artsy techniques.
I agree on all accounts here. I think I dumped most of my DADA skill points into implicit detection. And yes, the vibes thing isn't a perfect correlation to Dark stuff, I totally agree.
Is this example satisfying?
It's definitely helpful! The category still isn't crisp in my mind, but it's a lot clearer. Thank you!
Thanks for the response in any case, I really enjoy these discussions! Would you like to do a dialogue sometime?
I've really enjoyed this exchange too. Thank you!
And sure, I'd be up for a dialogue sometime. I don't have a good intuition for what kind of thing goes well in dialogues yet, so maybe take the lead if & when you feel inspired to invite me into one?
Can you spell this out a little more? Did Brent and LaSota employ baloney-disclaimers and uncertainty-signaling in order to bypass people's defenses?
I think Brent did something different from what I'm describing — a bit more like judo plus DOS attacks.
I'm not as familiar with LaSota's methods. I talked with them several times, but mostly before I learned to detect the level of psychological impact I'm talking about with any detail. Thinking back to those interactions, I remember it feeling like LaSota was confidently asserting moral and existential things that threatened to make me feel inadequate and immoral if I didn't go along with what they were saying and seek out the brain hemisphere hacking stuff they were talking about. And maybe even then I'd turn out to be innately "non-good".
(Implied here is a type of Dark hack I find most folk don't have good defenses against other than refusing to reason and blankly shutting down. It works absurdly well on people who believe they should do what they intellectually conclude makes sense to do.)
The thing I was referring to is something I personally stumbled across. IME rationalists on the whole are generally more likely to take in something said in a low-status way. It's like the usual analyze-and-scrutinize machinery kind of turns off.
One of the weirder examples is, just ending sentences as though they're questions? I'm guessing it's because ending each thing with confidence as a statement is a kind of powerful assertion. But, I mean, if the person talking is less confident then maybe what they're saying is pretty safe to consider?
(I'm demoing back & forth in that paragraph, in case that wasn't clear.)
I think LaSota might have been doing something like this too, but I'm not sure.
(As a maybe weird example: Notice how that last sentence is in fact caveated, but it's still confident. I'm quite sure this is my supposition. I'm sure I'm not sure of the implied conclusion. I feel solid in all of this. My impression is, this kind of solidity is a little (sometimes a lot) disturbing to many rationalists (with some exceptions I don't understand very well — like how Zvi and Eliezer can mostly get away with brazen confidence without much pushback). By my models, the content of the above sentence would have been easier to receive if rewritten along the lines of, "I'm really not sure, but based on my really shaky memories, I kinda wonder if LaSota might have been doing something like this too — but don't believe me too much!")
Does that answer what you'd hoped?
Yep, I think you're basically right on all accounts. Maybe a little off with the atheist fellow, but because of context I didn't think to share until reading your analysis, and what you said is close enough!
It's funny, I'm pretty familiar with this level of analysis, but I still notice myself thinking a little differently about the bookstore guy in light of what you've said here. I know people do the unbalancing thing you're talking about. (Heck, I used to quite a lot! And probably still do in ways I haven't learned to notice. Charisma is a hell of a drug when you're chronically nervous!) But I didn't think to think of it in these terms. Now I'm reflecting on the incident and noticing "Oh, yeah, okay, I can pinpoint a bunch of tiny details when I think of it this way."
The fact that I couldn't tell whether any of these were "ultra-BS" is more the central point to me.
If I could trouble you to name it: Is there a more everyday kind of example of ultra-BS? Not in debate or politics?
I'm gonna err on the side of noting disagreements and giving brief descriptions of my perspective rather than writing something I think has a good chance of successfully persuading you of my perspective, primarily so as to actually write a reply in a timely fashion.
Acknowledged.
I don't see this as showing that in all domains one must maintain high offensive capabilities in order to have good defenses.
Oh, uh, I didn't mean to imply that. I meant to say that rejecting attention to military power is a bad strategy for defense. A much, much better defensive strategy is to study offense. But that doesn't need to mean getting good at offense!
(Although I do think it means interacting with offense. Most martial arts fail spectacularly on this point for instance. Pragmatically speaking, you have to have practice actually defending yourself in order to get skillful at defense. And in cases like MMA, that does translate to getting skilled at attack! But that's incidental. I think you could design good self-defense training systems that have most people never practicing offense.)
I think these problems aren't that hard once you have community spaces that are willing to enforce boundaries. Over the last few years I've run many events and spaces, and often gotten references for people who want to enter the spaces, and definitely chosen to not invite people due to concerns about ethics and responsible behavior. I don't believe I would've accepted these two people into the spaces more than once or twice at most.
Nice. And I agree, boundaries like this can be great for a large range of things.
I don't think this helps the Art much though.
And it's hard to know how much your approach doesn't work.
I also wonder how much this lesson about boundaries arose because of the earlier Dark exploits. In which case it's actually, ironically, an example of exactly the kind of thing I'm talking about! Only with lessons learned much more painfully than I think was necessary due to their not being sought out.
But also, maybe this is good enough for what you care about. Again, I don't mean to pressure that you should do anything differently.
I'm mostly pushing back against the implication I read that "Nah, our patches are fine, we've got the Dark Arts distanced enough that they're not an issue." You literally can't know that.
My position is that most thinking isn't really about reality and isn't truth-tracking, but that if you are doing that thinking then a lot of important questions are surprisingly easy to answer.
Totally agree. And this is a major defense against a lot of the stuff that bamboozles most folk.
I think there's a ton of adversarial stuff going on as well, but the primary reason that people haven't noticed that AI is an x-risk isn't because people are specifically trying to trick them about the domain, but because the people are not really asking themselves the question and checking.
I agree — and I'm not sure why you felt this was relevant to say? I think maybe you thought I was saying something I wasn't trying to.
(I think there's some argument to be made here that the primary reason people don't think for themselves is because civilization is trying to make them go crazy, which is interesting, though I still think the solution is primarily "just make a space where you can actually think about the object level".)
This might be a crux between us. I'm not sure. But I think you might be seriously underestimating what's involved in that "just" part ("just make a space…"). Attention on the object-level is key, I 100% agree there. But what defines the space? What protects its boundaries? If culture wants to grab you by the epistemic throat, but you don't know how it tries to do so, and you just try to "make a space"… you're going to end up way more confident of the clarity of your thinking than is true.
I acknowledge that there are people who are very manipulative and adversarial in illegible ways that are hard to pin down. […] …I think probably there are good ways to help that info rise up and get shared…. I don't think it requires you yourself being very skilled at engaging with manipulative people.
I think there's maybe something of a communication impasse happening here. I agree with what you're saying here. I think it's probably good enough for most cases you're likely to care about, for some reasonable definition of "most". It also strikes me as obvious that (a) it's unlikely to cover all the cases you're likely to care about, and (b) the Art would be deeply enriched by learning how one would skillfully engage with manipulative people. I don't think everyone who wants to benefit from that enrichment needs to do that engagement, just like not everyone who wants to train in martial arts needs to get good at realistic self-defense.
I've said this several times, and you seem to keep objecting to my implied claim of not-that. I'm not sure what's going on there. Maybe I'm missing your point?
I do sometimes look at people who think they're at war a lot more than me, and they seem very paranoid and to spend so many cognitive cycles modeling ghosts and attacks that aren't there. It seems so tiring!
I agree. I think it's dumb.
I suspect you and I disagree about the extent to which we are at war with people epistemically.
Another potentially relevant point here is that I tend to see large groups and institutions as the primary forces deceiving me and tricking me, and much less so individuals.
Oh! I'm really glad you said this. I didn't realize we were miscommunicating about this point.
I totally agree. This is what I mean when I'm talking about agents. I'm using adversarial individuals mostly as case studies & training data. The thing I actually care about is the multipolar war going on with already-present unaligned superintelligences. Those are the Dark forces I want to know how to be immune to.
I'm awfully suspicious of someone's ability to navigate hostile psychofauna if literally their only defense against (say) a frame controller is "Sus, let's exclude them." You can't exclude Google or wokism or collective anxiety the same way.
Having experienced frame control clawing at my face, and feeling myself become immune without having to brace… and noticing how that skill generalized to some of the tactics that the psychofauna use…
…it just seems super obvious to me that this is really core DADA. Non-cognitive, very deep, very key.
- Personally I would like to know two or three people who have successfully navigated being manipulated, and hopefully have them write up their accounts of that.
Ditto!
- I think aspiring rationalists should maneuver themselves into an environment where they can think clearly and be productive and live well, and maintain that, and not try to learn to survive being manipulated without a clear and present threat that they think they have active reason to move toward rather than away from.
Totally agree with the first part. I think the whole thing is a fine choice. I notice my stance of "Epistemic warriors would still be super useful" is totally unmoved thus far though. (And I'm reminded of your caveat at the very beginning!)
I'm reminded of the John Adams quote: "I must study Politicks and War that my sons may have liberty to study Mathematicks and Philosophy. My sons ought to study mathematics and Philosophy, Geography, natural History, naval Architecture, navigation, Commerce and Agriculature, in order to give their Children a right to study Painting, Poetry, Musick, Architecture, Statuary, Tapestry and Porcelaine."
I note that when I read your comment I'm not sure whether you're saying "this is an important area of improvement" or "this should be central to the art", which are very different epistemic states.
Oh, I don't know what should or shouldn't be central to the Art.
It just strikes me that rationality currently is in a similar state as aikido.
Aikido claims to be an effective form of self-defense. (Or at least it used to! Maybe it's been embarrassed out of saying that anymore?) It's a fine practice, it has immense value… it's just not what it says on the tin.
If it wanted to be what it claims, it needs to do things like add pressure testing. Realistic combat. Going into MMA tournaments and coming back with refinements to what it's doing.
And that could be done in a way that honors its spirit! It can add the constraints that are key to its philosophy, like "Protect everyone involved, including the attacker."
But maybe it doesn't care about that. Maybe it just wants to be a sport and discipline.
That's totally fine!
It does seem weird for it to continue claiming to be effective self-defense though. Like it needs its fake meaning to be something its practitioners believe in.
I think rationality is in a similar state. It has some really good stuff in it. Really good. It's a great domain.
But I just don't see it mattering for the power plays. I think rationalists don't understand power, the same way aikido practitioners don't understand fighting. And they seem to be in a similar epistemic state about it: they think they basically do, but they don't pressure-test their understanding to check, best as I can tell.
So of your two options, it's more like "important area for improvement"… roughly like pressure-testing could be an important area of improvement for aikido. It'd probably become a kind of central if it were integrated! But I don't know.
And, I think the current state of rationality is fine.
Just weak in one axis it sometimes claims to care about.
Well, that particular comment had a lot of other stuff going on…
That's really not a central example of what I meant. I meant more like this one. Or this one.
But also, yeah, I do kinda feel like "downvoting people when they admit they did something bad" is a thing we sometimes do here and that's not great incentives. If someone wants to avoid that kind of downvote, "stop admitting to the bad thing" seems like an obvious strategy. Oops! And like, I remember times when I asked someone a question and they got downvoted for their answer, and I did think it was a bad answer that in a vacuum deserved downvotes, but I still upvoted as thanks for answering.
Yep. This is messy and unfortunate, I agree.
Someone might not have realized the thing they did was bad-according-to-LW, and the downvotes help signal that.
It's not possible to take the downvotes as a signal of this if downvotes get used for a wide range of things. If the same signal gets used for
"This was written in bad form, but if you'd written it differently it would have been welcome"
and
"Your attitude doesn't belong on this website, and you should change it or leave"
and
"I don't like your vibe, so I'm just gonna downvote"
then the feedback isn't precise enough to be helpful in shaping behavior.
If someone did a bad thing and doesn't care, maybe we just don't want them here.
True.
Although if the person disagrees with whether it was bad, and the answer to that disagreement is to try to silence them… then that seems to me like a pretty anti-epistemic norm. At least locally.
I'd also really like to see a return of the old LW cultural thing of, if you downvote then you explain why. There are some downvotes on my comments that I'm left scratching my head about and going "Okay, whatever." It's hard for downvotes to improve culture if the feedback amounts to "Bad."
I think there's currently too many things that deserve downvotes for that to be realistic.
I have a hard time believing this claim. It's not what I see when I look around.
The dynamic would be pretty simple:
- After I downvote, I skim the replies to see if someone else already explained what had me do the downvote. If so, I upvote that explanation and agree-vote it too.
- If there's no such explanation, I write one.
Easy peasy. I seriously doubt the number of things needing downvotes on this site is so utterly overwhelming that this approach is untenable. The feedback would be very rich, the culture well-defined and transparent.
I don't know why LW stopped doing this. Once upon a time it used to cost karma to downvote, so people took downvotes more seriously. I assume there was some careful thought put into changing that system to the current one. I haven't put more than a sum total of maybe ten minutes of thinking into this. So I'm probably missing something.
But without knowing what that something is, and without a lot of reason for me to invest a ton more time into figuring it out… my tentative but clear impression is that what I'm describing would be way better for culture here by a long shot.
…I think another pretty good option is "a master rationalist would definitely avoid surrounding themselves with con artists and frauds and other adversarial actors".
I think that's a great option. I'd question a "master rationalist's" skills if they couldn't avoid such adversarial actors, or notice them if they slip through the cracks.
I do think there are real skills you are pointing to, but to some extent I prefer the world where I don't have those skills and in place of that my allies and I coordinate to identify and exclude people who are using the dark arts.
I like your preference. I'll say some things, but I want to start by emphasizing that I don't think you're making a wrong or bad choice.
I want to talk about what I think the Art could be, kind of for aesthetic reasons. This isn't to assert anything about what you or any given individual should or shouldn't be doing in any kind of moral sense.
So with that said, here are three points:
(1) I think there's a strong analogy here to studying combat and war. Yes, if you can be in a pacifist cluster and just exclude folk who are really into applied competitive strategy, then you have something kind of like a cooperate/cooperate equilibrium. But if that's the whole basis of your culture, it's extremely vulnerable, the way cooperate-bot is vulnerable in prisoners' dilemmas. You need military strength, the way a walled garden needs walls. Otherwise folk who have military strength can just come take your resources, even if you try to exclude them at first.
At the risk of using maybe an unfair example, I think what happened with FTX last year maybe illustrates the point.
Clearer examples in my mind are Ziz and Brent. The point not being "These people are bad!" But rather, these people were psychologically extremely potent and lots of folk in the community could neither (a) adequately navigate their impact (myself included!) nor (b) rally ejection/exclusion power until well after they'd already had their impact.
Maybe, you might hope, you can make the ejection/exclusion sensitivity refined enough to work earlier. But if you don't do that by studying the Dark Arts, and becoming intimately familiar with them, then what you get is a kind of naïve allergic response that Dark Artists can weaponize.
Again, I don't mean that you in particular or even rationalists in general need to address this. There's nothing wrong with a hobby. I'm saying that as an Art, it seems like rationality is seriously vulnerable if it doesn't include masterful familiarity with the Dark Arts. Kind of like, there's nothing wrong with practicing aikido as a sport, but you're not gonna get the results you hope for if you train in aikido for self-defense. That art is inadequate for that purpose and needs exposure to realistic combat to matter that way.
(2) …and I think that if the Art of Rationality were to include intimate familiarity with the Dark Arts, it would work way way better.
Things like the planning fallacy or confirmation bias are valuable to track. I could stand to improve my repertoire here for sure.
But the most potent forms of distorted thinking aren't about sorting out the logic. I think they look more like reaching deep down and finding ways to become immune to things like frame control.
Frame control is an amazing example in my mind precisely because of the hydra-like nature of the beast. How do you defend against frame control without breaking basic things about culture and communication and trust? How do you make it so your cultural and individual defenses don't themselves become the manual that frame controllers use to get their desired effects?
And this barely begins to touch on the kind of impact that I'd want to call "spiritual". By which I don't mean anything supernatural; I'm talking about the deep psychological stuff that (say) conversing with someone deep in a psilocybin trip can do to the tripper. That's not just frame control. That's something way deeper, like editing someone's basic personality operating system code. And sometimes it reaches deeper even than that. And it turns out, you don't need psychedelics to reach that deep; those chemical tools just open a door that you can open other ways, voluntarily or otherwise, sometimes just by having a conversation.
The standard rationalist defense I've noticed against this amounts to mental cramping. Demand everything go through cognition, and anything that seems to try to route around cognition gets a freakout/shutdown/"shame it into oblivion" kind of response. The stuff that disables this immune response is really epistemically strange — things like prefacing with "Here's a fake framework, it's all baloney, don't believe anything I'm saying." Or doing a bunch of embodied stuff to act low-status and unsure. A Dark Artist who wanted to deeply mess with this community wouldn't have to work very hard to do some serious damage before getting detected, best as I can tell (and as community history maybe illustrates).
If this community wanted to develop the Art to actually be skillful in these areas… well, it's hard to predict exactly what that'd create, but I'm pretty sure it'd be glorious. If I think of the Sequences as retooling skeptical materialism, I think we'd maybe see something like a retooling of the best of Buddhist psychotechnology. I think folk here might tend to underestimate how potent that could really be.
(…and I also think that it's maybe utterly critical for sorting out AI alignment. But while I think that's a very important point, it's not needed for my main message for this exchange.)
(3) It also seems relevant to me that "Dark Arts" is maybe something of a fake category. I'm not sure it even forms a coherent cluster.
Like, is being charismatic a Dark Art? It certainly can be! It can act as a temptation. It seems to be possible to cultivate charisma. But the issue isn't that charisma is a Dark Art. It's that charisma is mostly symmetric. So if someone has a few slightly anti-epistemic social strategies in them, and they're charismatic, this can have a net Dark effect that's even strategic. But this is a totally normal level of epistemic noise!
Or how about something simpler, like someone using confirmation bias in a way that benefits their beliefs? Astrology is mostly this. Is astrology a Dark Art? Is talking about astrology a Dark Art? It seems mostly just epistemically hazardous… but where's the line between that and Dark Arts?
How about more innocent things, like when someone is trying to understand systemic racism? Is confirmation bias a helpful pattern-recognizer, or a Dark Art? Maybe it's potentially in service to Dark Arts, but is a necessary risk to learn the patterns?
I think Vervaeke makes this point really well. The very things that allow us to notice relevance are precisely the things that allow us to be fooled. Rationality (and he explicitly cites this — even the Keith Stanovich stuff) is a literally incomputable practice of navigating both Type I and Type II errors in this balancing act between relevance realization and being fooled.
When I think of central examples of Dark Arts, I think mostly of agents who exploit this ambiguity in order to extract value from others.
…which brings me back to point (1), about this being more a matter of skill in war. The relevant issue isn't that there are "Dark Arts". It's that there are unaligned agents who are trying to strategically fool you. The skill isn't to detect a Dark toolset; it's to detect intelligent intent to deceive and extract value.
All of which is to say:
- I think a mature Art of Rationality would most definitely include something like skillful navigation of manipulation.
- I don't think every practitioner needs to master every aspect of a mature Art. Much like not all cooks need to know how to make a roux.
- But an Art that has detection, exclusion, & avoidance as its only defense against Dark Artists is a much poorer & more vulnerable Art. IMO.
The unspoken but implicit argument is that Russia doesn't need a reason to nuke us. If we give them the Arctic there's no question, we will get nuked.
Ah, interesting, I didn't read that assumption into it. I read it as "The power balance will have changed, which will make Russia's international bargaining position way stronger because now it has a credible threat against mainland USA."
I see the thing you're pointing out as implicit though. Like an appeal to raw animal fear.
For a successful nuclear first strike to be performed Russia must locate all of our military assets (plus likely that of our NATO allies as well), take them all out at once, all while the CIA somehow never gets wind of a plan.
That makes a lot of sense. I didn't know about the distributed and secret nature of our nuclear capabilities… but it's kind of obvious that that's how it'd be set up, now that you say so. Thank you for spelling this out.
Reactions like yours are thus part of what I was counting on when making the argument. It works because in general I can count on people not having prior knowledge. (don't worry, you're not alone)
Makes sense!
And I wasn't worried. I'm actually not concerned about sounding like (or being!) an idiot. I'm just me, and I have the questions I do! But thank you for the kindness in your note here.
It also seems rather incongruous with most people's model of the world […]. Suppose Russia was prepared to nuke the US, and had a credible first strike capability. Why isn't Uncle Sam rushing to defend his security interests? Why haven't pundits and politicians sounded the alarm? Why has there been no diplomatic incidents? A second Cuban missile crisis? A Russian nuclear attack somewhere else?
I gotta admit, my faith in the whole system is pretty low on axes like this. The collective response to Covid was idiotic. I could imagine the system doing some stupid things simply because it's too gummed up and geriatric to do better.
That's not my main guess about what's happening here. I honestly just didn't think through this level of thing when I first read your arctic argument from your debate. But collective ineptitude is plausible enough to me that the things you're pointing out here just don't land as damning.
But they definitely are points against. Thank you for pointing them out!
I hope that answers your question! Is everything clear now?
For this instance, yes!
There's some kind of generalization that hasn't happened for me yet. I'm not sure what to ask exactly. I think this whole topic (RE what you're saying about Dark Arts) is bumping into a weak spot in my mind that I wasn't aware was weak. I'll need to watch it & observe other examples & let it settle in.
But for this case: yes, much clearer!
Thank you for taking the time to spell all this out!
Do you mind providing examples of what categories and indicators you use?
I can try to provide examples. The indicators might be too vague for the examples to help much with though!
A few weeks ago I met a fellow who seems to hail from old-guard atheism. Turn-of-the-century "Down with religion!" type of stuff. He was leading a philosophy discussion group I was checking out. At some point he said something (I don't remember what) that made me think he didn't understand what Vervaeke calls "the meaning crisis". So I brought it up. He started going into a kind of pressured debate mode that I intuitively recognized from back when I swam in activist atheism circles. I had a hard time pinning down the moves he was doing, but I could tell I felt a kind of pressure, like I was being socially & logically pulled into a boxing ring. I realized after a few beats that he must have interpreted what I was saying as an assertion that God (as he thought others thought of God) is real. I still don't know what rhetorical tricks he was doing, and I doubt any of them were conscious on his part, but I could tell that something screwy was going on because of the way interacting with him became tense and how others around us got uneasy and shifted how they were conversing. (Some wanted to engage & help the logic, some wanted to change the subject.)
Another example: Around a week ago I bumped into a strange character who runs a strange bookstore. A type of strange that I see as being common between Vassar and Ziz and Crowley, if that gives you a flavor. He was clearly on his way out the door, but as he headed out he directed some of his… attention-stuff… at me. I'm still not sure what exactly he was doing. On the surface it looked normal: he handed me a pamphlet with some of the info about their new brick-and-mortar store, along with their online store's details. But there was something he was doing that was obviously about… keeping me off-balance. I think it was a general social thing he does: I watched him do it with the young man who was clearly a friend to him and who was tending the store. A part of me was fascinated. But another part of me was throwing up alarm bells. It felt like some kind of unknown frame manipulation. I couldn't point at exactly how I was being affected, but I knew that I was, because my inner feet felt less firmly on inner ground in a way that was some kind of strategic.
More blatantly, the way that streetside preachers used to find a corner on college campuses and use a loudspeaker to spout off fundamentalist literalist Christianity memes. It's obvious to me now that the memetic strategy here isn't "You hear my ideas and then agree." It's somehow related to the way that it spurs debate. Back in my grad school days, I'd see clusters of undergrads surrounding these preachers and trying to argue with them, both sides engaging in predetermined patter. It was quite strange. I could feel the pull to argue with the preacher myself! But why? It has a snare trap feeling to it. I don't understand the exact mechanism. I might be able to come up with a just-so story. But looking back it's obvious that there's a being-sucked-in feeling that's somehow part of the memetic strategy. It's built into the rhetoric. So a first-line immune response is "Nope." Even though I have little idea what it is that I'm noping out of. Just its vibe.
I don't think all (any?) of these fall under what you're calling "ultra-BS". That's kind of my point: I think my rhetoric detector is tracking vibes more than techniques, and you're naming a technique category. Something like that.
I think this part stands alone, so I'll reply to the rest separately.
Thank you. I found this exchange very enriching.
In particular, it highlights a gap in my way of reasoning. I notice that even after you give examples, the category of "ultra-BS" doesn't really gel for me. I think I use a more vague indicator for this, like emotional tone plus general caution when someone is trying to persuade me of something.
In the spirit of crisping up my understanding, I have a question:
Now, I understand I sound obviously crazy already, but hear me out. Russia's Kinzhal hypersonic missiles, which have a range of roughly 1,000 miles, cannot hit the US from the Russian mainland. But they can hit us from the Arctic. I add that hypersonic missles are very, very fast. [This essentially acts as a preemptive rebuttal to my opponent's counterargument (but what about MAD?).] If we're destroyed by a first strike, there is no MAD, and giving Russia the Arctic would immediately be an existential threat.
Of course, this is ridiculous…
I think I'm missing something obvious, or I'm missing some information. Why is this clearly ridiculous?
Isn’t this an ironic choice of metaphor? The situation rather more resembles you insisting that it’s your daughter’s arm, being certain of this despite many other people thinking that you’re not quite in touch with reality, being impervious to demonstrations or proofs that it’s your arm, etc.
Of course it's not ironic. What do you think the patient must think about the doctor's certainty?
…the current site culture, moderation policies, etc., actively discourage such explanations.
How so? What's the discouragement? I could see people feeling like they don't want to bother, but you make it sound like there's some kind of punishment for doing so…?
I'd also really like to see a return of the old LW cultural thing of, if you downvote then you explain why. There are some downvotes on my comments that I'm left scratching my head about and going "Okay, whatever." It's hard for downvotes to improve culture if the feedback amounts to "Bad."
For instance, my review has been pretty heavily downvoted. Why? I can think of several reasons. But the net effect is to convey that LW would rather not have seen such a review.
Now why would that be?
I notice that there's also a -16 on the agree/disagree voting, with just three votes. So I'm guessing that what I said seriously irked a few people who probably heavy-downvoted the karma too.
But if it's really a distributed will, it's curious. Do you really want me not to have shared more context? Not to have reflected on where I'm at with the post? Or is it that you want me to feel differently about the post than I do?
I guess I don't get to know!
It's worth remembering that karma downvoting has a technical function. Serious negative karma makes a comment invisible by default. A user who gets a lot of negative karma in a short period of time can't post comments for a while (I think?). A user who has low karma overall can't post articles (unless that's changed?).
So a karma downvote amounts to saying "Shut up."
And a strong-downvote amounts to saying "Shut the fuck up."
If that's really the only communication the whole culture encourages for downvotes… that doesn't really foster clarity.
It seems dead obvious to me that this aspect of conversation culture here is quite bad.
But this isn't a hill I intend to die on.
…it looks like Valentine is never going to write the promised post…
It was Mythic Mode. I guess that went over everyone's heads.
I had a sequence in mind, on "ontology cracking". I gave up on that sequence when it became obvious that Less Wrong really wasn't interested in that direction at all. So I ended up never describing how I thought mythic mode worked on me, and how it might generalize.
But honestly, Mythic Mode has all the ingredients you need if you want to work it out.
It also seems worth noting, I've gotten way more PCK on the whole thing since then, and now I have approaches that are a fair bit more straightforward. More zen-like. Kinder. So the approach I advocate these days feels different and is more grounded & stable.
I might try to share some of that at some point.
This might be related to his statement in a followup discussion that he is unable to provide any cake. (It is an odd discussion, I think, and reading Valentine's attempts to comment there remind me of Kennaway's comment.)
You seem to have quite missed the point of that exchange.
But honestly, I'm tired of arguing with logic machines about this. No, I cannot prove to you that it's not your daughter's arm. No, that fact does not cause me to question my certainty that it's not your daughter's arm. Yes, I understand you think I'm crazy or deluded. I am sorry I don't know how to help you; it is beyond my skill, and my human heart hurts for being so misunderstood so much here.