Mandatory Secret Identities

What does this post even mean? I don't have access to my own respect function, and I don't know if I'd mess with it this way even if I did.

If you were to say tomorrow "I've been lying about the whole AI programmer thing; I actually live in my parents' basement and have never done anything worthwhile in any non-rationality field in my entire life," then would I have to revise my opinion that you're a very good rationality teacher? Would I have to deny having learned really valuable things from you?

Or would I have to say, "Well, this guy named Eliezer taught me everything I know, he's completely opened my mind to new domains of knowledge, and you should totally read everything he's written - but he's not all that great and I don't have any respect for him and you shouldn't either" when referring people to your writing?

Or to put it another way...let's say there are two rationality instructors in my city. One, John, is a world famous physicist, businessman, and writer. The other, Mary, has no particular accomplishments outside her rationality instruction work. However, Mary's students have been observed to do much better at their careers than John's, and every time the two dojos go up against each other in Rationalist Debating or calibration tests or any other kind of measurement, Mary's students do better. Wouldn't it be, well, irrational for me to go to John's dojo instead of Mary's? Would the Bayesian Police have to surround Mary's dojo and make sure her students don't say nice things about her or pay her more money than John is making?

Is the point about respect for instructors supposed to generalize to instructors of disciplines other than rationality?

If so, what do you make of Nadia Boulanger? Her accomplishments as a musician (or otherwise) are unimpressive relative to those of her students and peers, and yet she is regarded as one of the greatest music teachers ever, and is accorded correspondingly deep respect by music historians, composers, etc. Are they all wrong to respect her so much, or does it not apply to music or this case?

It seems to me that a better formula for determining respect would somehow reflect the respect given to her students which they say is significantly due to her influence as a teacher. For example, if Aaron Copland singles her out as an amazing teacher who profoundly affected his musical life & education, then she deserves some of the respect given to him. And likewise for her many other students who went on to do great things.

There seems to be an implicit underlying belief in this post that teaching is not (or should not be) an end in and of itself, or at least not a worthy one. I think Boulanger and teachers of her caliber show that that's just not the case.

Has this requirement been successfully implemented for CFAR instructors?

1. In the boost phase of a newly launched idea, it's actually a really good idea to train teachers. That gives you exponential growth.

2. It's a fail if the discipline gets into a death spiral about teaching teachers to teach teachers, iff the recursion lacks a termination condition. (Suitable conditions left as an exercise.)

3. Even in the cruise phase, an idea needs a teacher replacement rate of >= 1.

4. In cruise phase, it's a fail if every student wants to teach. But I don't see how it's a fail if some students want to teach and proceed to do so. Nor do I see how it's a fail if they end up being most of the teachers.

5. The intersection of two very rare categories of people is nobody.

6. Aren't you the same guy who, just a few days ago, pointed out how much better a trained professional is at his job than some volunteer? Teaching is a nontrivial skill.

No.

You're wasting huge amounts of optimization power, here, in two different ways. Firstly, you're saying that no one should focus his efforts on becoming a good rationality instructor, that any work he does on that is entirely meaningless unless he is at least as good at something else. Secondly, you're saying that no one should focus his efforts on instructing people in rationality, that they should spend most of their time on whatever other thing it is that makes them impressive. If you have someone who is naturally better at instructing people in rationality than in anything else, you are wasting most of the surplus you could have gained from him in these two ways.

I'm sympathetic to your concern, but surely there must be a way we can avoid throwing out the baby with the bathwater?

Well if we develop rationality tests, then you should rely on the teachers who help their students do better on tests. And if you can't develop tests, then I don't see why you'd think you had evidence that any particular person was good at teaching rationality. Relying on their ability to do something useful as a predictor of their ability to teach rationality seems nearly as bad as relying on their publication record, or their IQ, or wealth, etc. I say focus on developing tests.

(Blinks.)

I wonder if this idea comes as a shock because everyone was planning on becoming rationality instructors, i.e., I should have warned everyone about this much earlier?

Is it offputting on some other level?

But I must also consider that it might really be that stupid. Damn, now I wish I knew the actual number of upvotes and downvotes!

Some thoughts from my experience in a martial arts dojo:

1. We avoid lots of failure modes by making sure (as far as reasonably possible) that people are there to train first and everything else second. One consequence of this is that we don't attach a whole lot of our progress to any particular instructor; we're blessed with a number of people who are really good at aikido, and we learn from all of them, and from each other.

2. On setting the bar too high for instructors: Most martial arts rely on a hierarchy of instructors, where the average dojo head is a reasonably normal person who is expert but not necessarily elite at the discipline. The "famous" people in the art travel around and deliver seminars to everybody else. Dojo head type people will also travel to attend more seminars than the average junior student, for obvious reasons.

All sorts of human enterprises work this way (although the formality of the hierarchy varies widely); everything from yoga to religions to Linux Users Groups. It's a good system.

How much of what you're trying to do could be accomplished by largely tabooing the term "rationality" in rationality dojos, and having the community be really really attached to that tabooing? So that the dojos are for "finding ways of thinking that actually bring accurate beliefs" and "finding ways of thinking that actually help people reach their goals", with mostly no mention of a term like "rationality" that's easy to reify? If we talked like that, actual and prospective students and teachers might naturally look outward, to the evidence that various thinking processes were or weren't helping. Such evidence would be found partly in terms of the actual "day job" accomplishments (or lack of accomplishments) of the teacher, and also in terms of "day job" accomplishments of the students after vs. before joining the group, and also in terms of any measures that a group of active, experimentally minded rationality students could think up of whether they were actually becoming better at forming accurate beliefs.

What work is the word "secret" doing in this post? It seems to me that you're talking about public identities, ones visible to outsiders, ones that potential students (not yet enrolled in the Conspiracy) can look at to evaluate would-be instructors. Are you using the phrase "secret identities" merely because it sounds cool?

How much of Objectivism's failure was due to its teachers not having developed sufficient awesomeness elsewhere, and how much was due to the fact that it, say, tried to claim that it had the One True Method of Thought, instead of fostering an environment where all teachings were conjectural, teachers were facilitators of investigation instead of handers-down of The Answer, and everyone together tried to figure out what worked?

I mean, to what extent can we avoid similar failure modes by fostering a culture that doesn't reify anyone's teachings, but that instead tries to foster a culture of experimenting, thinking up new strategies, pooling data, and asking how we can tell what does and doesn't work?

Among the failure modes of martial arts dojos, I suspect, is that a sufficiently dedicated martial arts student, will dream of...

...becoming a teacher and having their own martial arts dojo someday.

I do not think this analogy fits. Martial arts is a self-contained bubble. What else is there to do but teach? To use a variation on the analogy, if someone being trained in the United States Marine Corps were given the question of what a truly dedicated student of the USMC were to become, they would probably answer along the lines of someone who kills things and doesn't die while doing it.

(Minor point) Martial arts do have tournaments and the like, so I suppose that is an alternate path. There is an inherent time-limit on those activities because of the human aging process, however, and after you retire from fighting, what else is there to do but teach what you learned?

To see what's wrong with this, imagine going to a class on literary criticism, falling in love with it, and dreaming of someday becoming a famous literary critic just like your professor, but never actually writing anything.

This second analogy is in a world where there is something more to do than teach. Choosing not-writing over writing is a failure mode in literary arts but not for literary criticism. Literary criticism is too specific. If you study literary criticism it naturally follows that you will become a literary critic. (Minor point) I could be completely misunderstanding what you meant but "writing anything".

Also, in a sense you digress from the pattern in the first analogy. Being a student of martial arts is to teaching martial arts as being a student literary criticism is to teach literary criticism.

In my opinion, the lesson to be learned here is to study something that can be applied to the real world in a form other than simply teaching other people what you learned. (Someone else mentioned something similar to this in the comments, but I cannot find it again via skimming.)

All that being said, this statement does loosely follow:

Similarly, I propose, no student of rationality should study with the purpose of becoming a rationality instructor in turn. You do that on Sundays, or full-time after you retire.

But I imagine that a lot of people who desire to eventually teach will do so after a full life of other activities. I also imagine this is a decent way to keep generational bias out of a system of education, since the next generation of students will likely spawn new variations and having those variations inserted back into the system by a "newer" instructor can encourage growth. Having the same ol' instructors around limits that somewhat.

The computer science analogy of this would be classes taught by old fogies who still think of programming as playing with decks of punch cards. Even if they move beyond that in their knowledge, their mind is potentially tainted by what-once-was.

I've been expecting a deliberately daft post from Eliezer Yudkowsky and/or Robin Hanson to see whether we vote them up just based upon status.

I think this is it.

Among the failure modes of martial arts dojos, I suspect, is that a sufficiently dedicated martial arts student, will dream of.. becoming a teacher and having their own martial arts dojo someday.

I think that accademia is also subject to this mode of failure. As an exercise, try to think of great literary figures who were also professors of literature at major universities. Off the top of my head, I can think of exactly one: Vladimir Nabokov, and he was notably contemptuous of his colleagues. Can anyone else think up anymore?

Unsurprisingly, Paul Graham has some interesting thoughts on the subject (incidentally, I seem to be developing a reputation on another forum that I post on as the "obligatory Paul Graham link guy"):

Tests are least hackable when there are consistent standards for quality, and the people running the test really care about its integrity. Admissions to PhD programs in the hard sciences are fairly honest, for example. The professors will get whoever they admit as their own grad students, so they try hard to choose well, and they have a fair amount of data to go on. Whereas undergraduate admissions seem to be much more hackable. One way to tell whether a field has consistent standards is the overlap between the leading practitioners and the people who teach the subject in universities. At one end of the scale you have fields like math and physics, where nearly all the teachers are among the best practitioners. In the middle are medicine, law, history, architecture, and computer science, where many are. At the bottom are business, literature, and the visual arts, where there's almost no overlap between the teachers and the leading practitioners. It's this end that gives rise to phrases like "those who can't do, teach."

In this case, what we should really worry about is developing good tests to distinguish good rationalists from the phonies beyond just "runs a rationality dojo". Obviously, being able to apply the principles of instrumental rationality inorder to succeed at a field unrelated to rationality is one such test, but is it the only one? I think the issue warrants further study.

Obviously success in other realms is bayesian evidence that someone would make a better rationality instructor. But as many others have argued, in this post Eliezer exaggerates the importance of this type of evidence.

I have a question: why are you panicking about this now? It's not like we have a huge problem yet with too many teachers, or too many freshly founded schools.

Makes sense, though I will quibble with your opening line. What you say about martial arts dojos was probably true up until about twenty years ago, but today I suspect a sufficiently dedicated martial arts student is in fact dreaming of becoming a champion MMA fighter.

And you know, now that I think about it, even twenty years ago, I'm not sure anyone was dreaming of becoming a dojo owner. That was just what they could practically achieve. But they were dreaming of becoming a Dark Lord:

"Surely you've wanted to hurt people," said Professor Quirrell. "You wanted to hurt those bullies today. Being a Dark Lord means that people you want to hurt get hurt.

I guess the failure mode that you're concerned with is a slow dilution because errors creep in with each successive generation and there's no external correction.

I think that the way we currently prevent this in our scientific efforts is to have both a research and a teaching community. The research community is structured to maximise the chances of weeding out incorrect ideas. This community then trains the teachers.

The benefits of this are that you get the people who are best at communicating doing the teaching and the people who are the best at research doing research.

Is it possible that having taught yourself you haven't so directly experienced that there's not necessarily a correlation between a persons understanding of a subject and their ability to teach it?

The mini-intro to this post on the craft and community sequence page says that it was not well received. But the requirements that this write up recommends really act as beautiful safeguard against becoming pedantic. If I hadnt read this page quite early (before I got past the 25% mark on the sequences), I doubt I would have stopped myself from falling into a happy death spiral (I honestly still really struggle with that one all the time).

It's really hard for me even now to "not speak over much of the way" (though, I mostly think it to myself, not too many friends are into this kind of thing). But knowing how important that is, certainly helps.

If you have "something to protect", if your desire to be rational is driven by something outside of itself, what is the point of having a secret identity? If each student has that something, each student has a reason to learn to be rational -- outside of having their own rationality dojo someday -- and we manage to dodge that particular failure mode. Is having a secret identity a particular way we could guarantee that each rationality instructor has "something to protect"?