A Sense That More Is Possible

Eliezer, I have recommended to you before that you read The Darkness That Comes Before and the associated trilogy. I repeat that recommendation now. The monastery of Ishual is your rationalist dojo, and Anasurimbor Kellhus is your beisutsukai surrounded by a visible aura of formidability. The book might even give you an idea or two.

My only worry with the idea of these dojos is that I doubt the difference between us and Anasurimbor Kellhus is primarily a difference in rationality levels. I think it is more likely to be akrasia. Even an irrational, downright stupid person can probably think of fifty ways to improve his life, most of which will work very well if he only does them (quit smoking, quit drinking, study harder in school, go on a diet). And a lot of people with pretty well developed senses of rationality whom I know, don't use them for anything more interesting than winning debates about abortion or something. Maybe the reason rationalists rarely do that much better than anyone else is that they're not actually using all that extra brainpower they develop. The solution to that isn't more brainpower.

Kellhus was able to sit down, enter the probability trance, decide on the best course of action for the immediate future, and just go do it. When I tried this, I never found the problem was in the deciding - it doesn't take a formal probability trance to chart a path through everyday life - it was in following the results. Among the few Kellhus-worthy stories I've ever heard from reality was you deciding the Singularity was the most important project, choosing to devote your life to it, and not having lost that resolve fifteen years later. If you could bottle that virtue, it would be worth more than the entire Bayesian corpus combined. I don't doubt that it's positively correlated with rationality, but I do doubt it's a 1 or even .5 correlation.

Why aren't "rationalists" surrounded by a visible aura of formidability? Why aren't they found at the top level of every elite selected on any basis that has anything to do with thought? Why do most "rationalists" just seem like ordinary people, perhaps of moderately above-average intelligence, with one more hobbyhorse to ride?

Because they don't win? Because they don't reliably steer reality into narrow regions other people consider desirable?

I've met and worked with several irrationalists whose models of reality were, to put it mildly, not correlated to said reailty, with one explicit, outspoken anti-rationalist with a totally weird, alien epistemology among them. All these people had a couple of interesting things in common.

On one hand, they were often dismal at planning – they were unable to see obvious things, and they couldn't be convinced otherwise by any arguments appealing to 'facts' and 'reality' (they universally hated these words).

On the other hand, they were surprisingly good at execution. All of them were very energetic people who didn't fear any work or situation at all, and I almost never saw any of them procrastinating. Could this be because their minds, due to their poor predictive ability, were unable to see the real difficulty of their tasks and thus avoided auto-switching into procrastination mode?

(And a third observation – all these people excelled in political environments. They tended to interpret their surroundings primarily in terms of who is kin to whom, who is a friend of who, who is sexually attracted to whom, what others think of me, who is the most influential dude around here etc etc. What they lost due to their desynchronization with factual reality, they gained back thanks to their political aptness. Do rationalists excel in political environments?)

Unfortunately, this hasn't aged very impressively.

Despite the attempts to build the promised dojo (CFAR, Leverage/Paradigm, the EA Hotel, Dragon Army, probably several more that I'm missing), rationalists aren't winning in this way. The most impressive result so far is that a lot of mid-tier powerful people read Slate Star Codex, but I think most of that isn't about carrying on the values Eliezer is trying to construct in this sequence - Scott is a good writer on many topics, most of which are at best rationality-adjacent. The second most impressive result is the power of the effective altruism movement, but that's also not the same thing Eliezer was pointing at here. 

The remaining positive results of the 2009 rationality community are a batch of happy group houses, and MIRI chugging along its climb (thanks to hard-to-replicate personalities like Eliezer and Nate).

I think the "all you need is to try harder" stance is inferior to the "try to make a general postmortem of 'rationalist dojo' projects in general" stance, and I'd like to see a systematic attempt at the latter, assembling public information and interviewing people in all of these groups, and integrating all the data on why they failed to live up to their promises.

Why aren't "rationalists" surrounded by a visible aura of formidability? Why aren't they found at the top level of every elite selected on any basis that has anything to do with thought? Why do most "rationalists" just seem like ordinary people, perhaps of moderately above-average intelligence, with one more hobbyhorse to ride?

I'm relatively new to rationality, but I've been a nihilist for nearly a decade. Since I've started taking developing my own morality seriously, I've put about 3500 hours of work into developing and strengthening my ethical framework. Looking back at myself when nihilism was just a hobbyhorse, I wasn't noticeably moral, and I certainly wasn't happy. I was a guy who knew things, but the things I knew never got put into practice. 5 years later, I'm a completely different person than I was when I started. I've made a few discoveries, but not nearly enough to account for the radical shifts in my behavior. My behavior is different because I practice.

I know a few other nihilists. They post pictures of Nietzsche on Facebook, come up with clever arguments against religion, and have read "the Anti-Christ." They aren't more moral just because they subscribe to an ethos that requires them to develop their own morality, and from that evidence I can assume that rationalists won't be more rational just because they subscribe to an ethos that demands the think more rationally. Changing your mind requires more than just reading smart things and agreeing with them. It requires practice.

In the spirit put up or shut up, I'm going to make a prediction. My prediction is that if we keep track of how often we use a rationalist technique in the real world, we will find that frequency of use correlates to the frequency at which we visualize and act out using that technique. Once we start quantifying frequency of use, we'll be able to better understand how rationalism impacts our abilities to reach our goals. Until we differentiate between enthusiasts and practitioners, we might as well be tracking whether liking a clever article on Facebook correlates to success.

While developing a rationality metric is obviously crucial, I have this nagging suspicion that what it may take is simply a bunch of committed wanna-be rationalists to just get together and, well, experiment and teach and argue, etc with each other in person regularly, try to foster explicit social rules that support rather than inhibit rationality, and so on.

From there, at least use a fuzzy this "seems" to work/not work type metric, even if it's rather subjective and imprecise, as a STARTING POINT, until one can more precisely do that, until one gets a better sense of exactly what to look for, explicitly.

But, my main point is my suspicion that "do it, even if you're not entirely sure yet what you're doing, just do it anyways and try to figure it out on the fly" may actually be what it takes to get started. If nothing else, it'll produce some nice case study in failure that at least one can look at and say "okay, let's actually try to work out what we did wrong here"

EDIT: hrm... maybe I ought reconsider my position. Will leave this up, at least for now, but with the added note that now I'm starting to suspect myself of basically just trying to "solve the problem without having to, well, actually solve the problem"

Every dojo has its sensei. There is a need for curriculum, but also skilled teachers to guide the earnest student. LessWrong and Overcoming Bias have, to some extent, been the dojo in which the students train. I think that you may find a lot of value in just jumping into a project like this: starting a small school that meets two times a week to practice a particular skill of rationality. A key goal to the budding school is to train the future's teachers.

One of my barriers to improving my rationality is little awareness of what the good reading and study material is. A curriculum of reading material -- rationalist homework -- would help me greatly. Furthermore, I have no friends that are similarly interested in the subject to bounce ideas off of or "train" with.

I train Jiu-Jitsu with several friends. We learn the same lessons, but learn at different rates. We discover different insights and share them. We practice techniques on each other and on opponents more and less skilled than ourselves. This dynamic is something rationalist dojos could benefit from.

Edit, Additional Comments:

The sense that a particular skill should be systematized and trained comes, in part, from the realization that the training conveys a measurable formidability. Problem #1 and Problem #2, then, are entangled: without a way to validate training, one can not say "I have defeated my opponent because of my training" and it is the ability to demonstrate mastery that motivates others to become students.

Many readers of this site and Overcoming Bias are here because of the demonstration of budding formidability found in the insights in OB posts. We read insights and learn techniques -- sloppily, like learning to wrestle from a mail order program -- and we want to be able to produce similar insights and be similarly formidable.

And conversely they don't look at the lack of visibly greater formidability, and say, "We must be doing something wrong."

Is the "lack of visibly greater formidability" actually visible? The wandering master sees the local students' deficiencies that the students are blind to. It is only when those students' established leader is defeated by the wandering master that the greater formidability becomes apparent. Where is rationality's flying guillotine?

For a nice literary description of what it means to have an "aura of awesome" try "The String Theory" by David Foster Wallace. Wallace writes of a mid-level pro tennis player: "The restrictions on his life have been, in my opinion, grotesque... But the radical compression of his attention and sense of himself have allowed him to become a transcendent practitioner of an art."

Perhaps in the future humans will achieve the same level of excellence at the Art of Rationality as some currently do at the Art of Tennis.

http://www.esquire.com/features/sports/the-string-theory-0796

On a side note, we have religious schools where a religion, such as Christianism, is part of the cursus. This indoctrinates young minds very early in their life, and leaves them scared, biased in most cases for the rest of their existence.

If we had, on the other hand, schools where even just basics of rationality and related topics, such as game theory, economics, scientific method, probabilities, biases, etc. were taught, what a difference it would make.

The sooner you kickstart rationality in a person, the longer they have to learn and practice it, obviously. But if those teachings are part of their formative experiences, from childhood to early adulthood, where their personalities, dreams and goals are being put together, how differently would they organize their lives ...

deleted

I see what you're saying about rationality being trained in a pure fashion (where engineering, the sciences in general, etc. is - hopefully - "applied rationality"). One thing I don't see you mention here but it was a theme in your 3 worlds story, and which is also a factor in martial arts training, is emotional management. That's crucial for rationality, since it will most likely be our feelings that lead us astray. Look at how the feeling of "trust" did in Madoff's investors. Muay thai and Aikido deal with emotions differently, but each train people to overcome their basic fear reactions with something else. An awesome rationalist, to me, would be someone who can maintain rationality when the situation is one of high emotion.

Just an observation: Few modern American karate schools ever let you hit someone, except when a lot of padding is involved. Fighting is not usually an element in exams below the blackbelt level. Competition is usually optional and not directly linked to advancement. I've seen students attain advanced belts without having any real-life fighting ability.

(The term "dojo" is Japanese, and I think most Japanese martial artists study Judo or Aikido, which are not subject to these criticisms.)

The thing that comes to mind, when I think of "formidable master of rationality", is a highly experienced engineer trying to debug problems, especially high-urgency problems that the normal customer support teams haven't been able to handle.  You have a fresh phenomenon, which the creators of the existing product apparently didn't anticipate (or if they did, they didn't think it worth adding functionality to handle it), which casts doubt on existing diagnostic systems.  You have priors on which tools are likely to still work, priors on which underlying problems are likely to cause which symptoms; tests you can try, each of which has its own cost and range of likely outcomes, and some of which you might invent on the spot; all of these lead to updating your probability distribution over what the underlying problem might be.

Medical diagnostics, as illustrated by Dr. House, can be similar, although I suspect the frequency of "inventing new tests to diagnose a never-before-seen problem" is lower there.

Isn't this a description of what a liberal arts education is supposed to provide? The skills of 'how to think' not 'what to think'? I'm not too familiar with the curriculum since I did not attend a liberal arts college, instead I was conned into an overpriced private university, but if anyone has more info please chime in.

Has Eliezer made explicit updates about this? Maybe @Rob Bensinger [LW · GW] knows. If he has, I'd like to see it posted prominently and clearly somewhere. Either way, I wonder why he doesn't mention it more often. Maybe he does, but only in fiction.

[...] I think that recognizing successful training and distinguishing it from failure is the essential, blocking obstacle.

Does this come up in the Dath Ilan stories?

There are experiments done now and again on debiasing interventions for particular biases, but it tends to be something like, "Make the students practice this for an hour, then test them two weeks later."  Not, "Run half the signups through version A of the three-month summer training program, and half through version B, and survey them five years later."

Surely there is more to say about this now than in 2009. Eliezer had some idea of the replication crisis back then, but I think he has become much more pessimistic about academia in the time since. 

But first, because people lack the sense that rationality is something that should be systematized and trained and tested like a martial art, that should have as much knowledge behind it as nuclear engineering, whose superstars should practice as hard as chess grandmasters, whose successful practitioners should be surrounded by an evident aura of awesome.

I think there's gotta be more to say about this too. Since then we have seen Tetlock's Superforecasting, Inadequate Equilibria^[1], the confusing story of CFAR^[2][3][4], and the rise to prominence of EA. I can now read retrospectives by accomplished rationalists arguing [LW · GW] over [LW · GW] whether rationality increases accomplishment, but I always come away feeling highly uncertain. (Not epistemically helpless, but frustratingly uncertain.) What do we make of all this?

Eliezer asks:

Why are there schools of martial arts, but not rationality dojos?  (This was the first question I asked in my first blog post.)  Is it more important to hit people than to think?

My answer, which gets progressively less charitable, and is aimed at no one in particular: thinking rationally appears to be a lower priority than learning particular mathematical methods, obtaining funding and recruits, mingling at parties, following the news, scrolling social media, and playing videogames.

1. ^{^}

   Consensus is that Eliezer verifiably outperformed the medical establishment with the lumenator, right [LW · GW]?

2. ^{^}

   https://www.lesswrong.com/posts/B9kP6x5rpmuCzpfWb/comment-reply-my-low-quality-thoughts-on-why-cfar-didn-t-get [LW · GW] 

3. ^{^}

   https://rationality.org/studies/2015-longitudinal-study

4. ^{^}

   https://www.lesswrong.com/posts/5K7CMa6dEL7TN7sae/3-levels-of-rationality-verification#AZb895EcZBdueP6QC [LW(p) · GW(p)] 

5. ^{^}

   https://www.lesswrong.com/posts/MajyZJrsf8fAywWgY/a-lesswrong-crypto-autopsy [LW · GW] 

6. ^{^}

   https://www.astralcodexten.com/p/why-im-less-than-infinitely-hostile

What kinds of tests or contests might we have? One that I can think of would be to have students try to create some visible, small scale effect in a society, with points for efficiency.

Elizer raises the issue of testing a rationality school. I can think of a simple way to at least approach this: test the students for well-understood cognitive biases. We have tests for plenty of biases; some of the tests don't work if you know about them, which surely these students will, but some do, and we can devise new tests.

For example, you can do the classic test of confirmation bias where you give someone solid evidence both for and against a political position and see if they become more or less certain. Even people who know about this experiment should often still fall prey to it—if they don't, they have demonstrated their ability to escape confirmation bias.

As a thought, could it be that one of the major obstacles standing in the way of the creation of a "rationality dojo" is the public perception (however inaccurate) that such already exists in not just one but multiple forms? Take the average high school debate club as one example: participants are expected to learn to give a reasoned argument, and to avoid fallacious reasoning while recognizing it in their opponents. Another example would be maths classes, wherein people are expected to learn how to construct a sound mathematical proof. I very much doubt that most people would understand the distinction between these and the proposed "rationality dojo", which would make it very hard to establish one.

It's easy to define success in martial arts. Defining 'rationality' is harder. Have you done so yet, Eliezer?

Even in martial arts, many of the schools of thoughts are essentially religions or cults, completely unconcerned with fighting proficiency and deeply concerned with mastering the arcane details of a sacred style passed on from teacher to student.

Such styles often come with an unrealistic conviction that the style is devastatingly effective, but there is little concern with testing that.

See also: http://www.toxicjunction.com/get.asp?i=V2741

I've read a great many comments and articles by people talking about how karate black belts are being seriously beaten by people with real-world fighting experience - pimps, muggers, etc. Becoming skilled in an esoteric discipline is useful only if that discipline is useful.

Do not seek to establish yourself as a sensei. Do not seek to become a "master of the art". Instead, try to get better at fighting - or, in this case, thinking correctly - even you don't get to wear a hood and chant about 'mysteries'.

Why aren't rationalists more formidable? Because it takes more than rationality to be formidable. There's also intelligence, dedication, charisma, and other factors, which rationality can do little to improve. Also, formidability is subjective, and I suspect that more intelligent people are less likely to find others formidable. As for why there isn't an art of rationality, I think it's because people can be divided into two groups: those who don't think rationality is particularly important and don't see the benefits of becoming more rational, and those who see rationality as important but are already rational for the most part, and for them, additional rationality training isn't going to result in a significant improvement.

[Third, because current "rationalists" have trouble working in groups: of this I shall speak more.]

Yes, this is the most important part. Rationalists apply rationality along a hierarchical level. And, they must optimize their goal structure according to a socially-irrational world. (The entire world, where the thinking of large numbers of other people determines the outcome. In perhaps narrow engineering worlds that are solely dependent on the outcomes of the innovator, and there is ALREADY an existing rational and competitive market, the rationalist slips in, and occupies a dominant position. But what happens when he has to beg for a business permit? What happens when he has to get licensed, or prevent his competition from stealing his intellectual property? What happens when he has to defend against the irrational world? Unless he's recognized the threat, and identified a complete goal structure for dealing with the irrational, he must accomplish this task IN ADDITION TO the massive work and time he's spent on his rational goal structure, and its implementation.)

Also: life hands you bad genes. It hands you cancer. There is a lot of "noise." In a community of rationalists, there are enough rationalists who have prioritized their top goal to be the same as your top goal, and you naturally start working with them. ...But if you're poor, you have to spend most of your time at your day job, so your first prioritization has, once again, been to deal with the irrational masses, and part them from their Federal Reserve Notes.

And then there's the problem of personalities amongst rationalists. Many of them aren't going to get along with each other.

This is why libertarianism is so important: if you don't first recognize that it's irrational to attack your neighbor and cofiscate his wealth, he probably isn't going to like you very much. ...And lesswrong has 50 people that identify themselves as communists, with a majority identifying themselves as "liberals." (My guess is that they're not the "Hayek" kind of liberal when they say that. LOL)

What I'm saying amounts to: "To study rationality, you must first master the irrational." Oooh, deep thought du jour.

A round table discussion amongst Libertarian Party ballot access activists/petitioners STARTS with this knowledge, but generally lacks the formalized training in rationality that is available here at less wrong. It's very interesting to see how a group of people who has rationally prioritized their lives can optimize their existences based on the introduction of new information.

As far as being "motivated enough to pick up your phone and call your broker," ...have you joined Casey Research yet? You should allocate your stock-picking decisions to him, because he takes into account the irrationality of the masses --which is your most difficult, time-wasting computation. His mining stock picks have returned something like a 250% return over the last few years. Compounding your initial investment returns with that kind of return would have made you very wealthy, had you simply begun allocating your excesses to his picks, assuming a salary with an excess of $15,000 to invest per year. However, given unlimited investments, you're forced to invest in a noise and irrationality-filled field, especially if you didn't have time to applying your rationality to finding (fellow libertarian, fellow rationalist, natural ally) Doug Casey.

If you haven't even reached out to the most highly-successful other rationalists, and decided who is best to delegate to, it sounds to me like you're irrationally estimating your bandwidth (cranium space) to be greater than what it is. This isn't a slur, you're clearly a smart guy. ...But you're also human, and more limited than what you might believe yourself to be after some successes.

I think this post of yours asks the right questions. I will address it point-by-point later on (since some of my points here seem trivial, out of context to the original statement. I've always hated "general replies" that miss the heart of the most important questions and address the ones the author thought most trivial). Until then, the gist of what I argue for is what lesswrong is already trying to do, but with MUCH more feedback, (or better interactivity programming) to eliminate conflicting personalities: typing is a waste of time, it's hard to quickly see/know who your allies are here. There isn't a +10 or -10 ranking system that is as good as one single phone conversation. You exclude many of Kevin Kelly, Freidrich Hayek's, and Monica Anderson's primary points, regarding the logic of decentralization, and emergent allocation of resources, here with this limited blog format.

(And you ban newbies, who may carry disproportionate value, by "lack of Karma." If you want adoption, you may need to eliminate the barrier to entry to your market.)

For a phone conversation, people are welcome to call me. Jake Witmer: 312-730-4037. That way, you'll be able to figure out more quickly if I'm an idiot, if I'm a useful idiot, or if I'm a potential strong ally in a certain area, or all areas. And, I'll be able to quickly find our the same about you. Naturally, if you're calling from here, you're out of the idiot category, (unless you're an infiltrator whose purposes are contrary to the site's). The last "paranoia part" is not something I think is likely, I just mentioned it because you had 50 communists on here, and over 300 "liberals." One doesn't have to be a knowing infiltrator, to be an infiltrator.

Also: Let's say you've decided to steal for a living, so that you could be in a socially dominant position, with a long term goal of making society more rational. So you joined the NWO, and became a multi-billionaire by trading "carbon credits." This is essentially theft, but it's put you in a position to be a strong rational actor. Well, it would be rational for all the rationalists here, without your additional knowledge, to attack you. They didn't know your long-term goals, or that you profiled yourself as "unable to compete, mathematically," and "not worth competing for less than billions of FRN influence." ...So, natural allies can seem like antagonists, and natural antagonists can seem like allies.

It's a lot to sort out, before you set up a laboratory work environment with someone. Also, since you're not continually filtering people here via feedback, you're not isolating out who is going to wind up working with you on a daily basis. Maybe that's what you need. A hotline full of people trained to profile high-level, highly-motivated rationalists. Perhaps you could even have Peter Voss's AGI do it, based on Watson-like heuristics. (Ie: If they stay on the line more than 20 minutes and direct the conversation to these repeated keywords, we might want to call them back.)