Course selection based on instructor

post by VipulNaik · 2013-09-08T19:13:35.161Z · LW · GW · Legacy · 28 comments


  Why I think instructor selection matters:
  Things to keep in mind from a human capital + signaling (grades) perspective:
  Getting the most out of attending sample classes
  Other criteria
  Looking for thoughts

I've been trying to compile advice on how to select good instructors in college, assuming that the major and course level are given. Of course, many people don't have a lot of flexibility with instructor choice, so this advice has limited applicability. The reason I'm posting to LessWrong is to get feedback from interested LessWrong readers regarding some of the factual assertions I make. I've marked the parts where I'm most eager for feedback. My experience with instruction (direct and indirect) has been focused largely on mathematics. Thus, it's possible that some of the points I raise have limited applicability in more discussion-based subjects.

Note also that although my experiences have been partly shaped by the undergraduate teaching I've done in four years as a graduate student at the University of Chicago, I'm neither representing the University nor am I critiquing or describing any features specific to the University of Chicago.

I'm working within the conceptual framework that there are three broad types of value that students derive from college courses:



Why I think instructor selection matters:


For the discussion below, I largely assume that the student is selecting between different sections of the same course, where each section follows a similar curriculum but the instructor has flexibility in terms of the final examinations and grades. What do LessWrong readers think about how common this is relative to a setup where all sections have the same final examinations and grades? However, much of the advice is general.





Things to keep in mind from a human capital + signaling (grades) perspective:


  1. The degree of gap between the score requirements for grades. An easy test where a 90 is an A, an 85 is an A-, etc., means that one careless error can affect your grade adversely. A hard test where an 80 is an A, a 60 is an A-, etc., means that the measurement is considerably less likely to be influenced by small measurement error issues.

  2. The number of questions on the test, and the number of tests and other assessment methods used. Generally, the more the number of items used, and the more distinct questions on the test, the more reliable the measurement (this is something to do with the law of large numbers).

  3. Whether or not instructors award partial credit in tests. Partial credit makes measurement more accurate, by indirectly increasing the number of items being measured.


  1. Providing sample tests to eliminate uncertainty about the test format.

  2. Providing review materials or conducting review sessions that are closely optimized to maximal test performance.

In general, if you are a good student with a reasonable shot at getting a fairly good grade, and you are interested in both human capital and grades, give preference to a criterion-referenced instructor who boasts low measurement error (lots of questions, lots of tests, partial credit, and hard tests) and teaches to the test. Subject to these constraints, favor the instructor whose overall grades are easiest/best.

If you are not interested in the human capital component of the course much, and are looking purely for a good grade, the choice between criterion-referenced and norm-referenced grading is harder. If choosing a norm-referenced grader, keep in mind the student population you are with. Depending on other courses, the student body during some quarters or semesters can be considerably better than during others, making it relatively harder to get a good grade.

If you are not very good with the material and not keen on deep learning, it might actually pay off to choose an instructor with higher measurement error. This is similar to the idea that people with the odds against them would have the most chance of winning if they made a small number of big bets, whereas people with the odds in their favor would have the most chance of winning by placing a large number of small bets.

Getting the most out of attending sample classes

It's quite rare for students to attend sample classes with instructors that they plan to study with (there could be many reasons -- do readers have any thoughts?). Update: When talking of "sample classes" I was referring to classes with the same instructor in an earlier term, because it's often very difficult to change classes once the term has begun, due to complications with scheduling, or classes getting full. The same instructor may be teaching a somewhat different course in the preceding term, so such sampling is useful only in so far as it captures generic aspects of instructors that transfer across courses. There may also be an element of difference between universities with a semester system and a quarter system. Those on the quarter system have less time to shop around between classes because the schedule is more compressed overall.

For those who do choose to attend sample classes, the following tips may be useful.



Other criteria

The following may be harder to gauge from a single lecture, but can usually be better gathered by talking to students who have studied with the instructor and in some cases by reviewing student evaluations that include long form responses.


Looking for thoughts

What do readers think of the lists I've offered above? Are there items you disagree with? Things you think I missed? Arguments that the entire question is ill-considered? All feedback would be very much appreciated.


PS: I've deliberately omitted other factors, such as scheduling and space constraints and peer choices, from the discussion. In many cases, the fact that a close friend or study buddy is taking a particular section is good reason to take that section. Similarly, scheduling and space constraints can be binding at times. I don't think that these meaningfully alter the shape of the preceding advice, but they do constrain the scope within which it can be applied.


Comments sorted by top scores.

comment by benkuhn · 2013-09-09T03:18:13.573Z · LW(p) · GW(p)

You missed one type of value, which is peer group. Most of the more difficult courses I've taken have given me at least some value by granting better access to more smart, competent people. This has a number of benefits:

  • Better human capital. By watching how more competent people work and think, you can often pick up useful study habits and better techniques for the subject you're studying. I've found this especially true in CS classes, where I've had this experience from both sides, e.g. teaching classmates how to use Git and picking up C coding style and tricks from better programmers.

  • Advice/planning help: both more advanced students and instructors can be very useful for the academic advice they provide later. Knowing talented students has given me info about several excellent courses, as well as summer opportunities, I wouldn't otherwise have known about. A professor who can become a good mentor is also invaluable (although if you don't plan to go into academia, you may have trouble finding a professor who can sympathize with your goals enough to give you good advice).

  • Better networking later: this is hopefully obvious, but knowing smart, competent people is useful for pretty much everything ever.

  • Friends/fun: people who think fast and have good background are less predictable/more interesting to talk to.

This suggests the following advice:

  • Seek out classes that signal difficulty or are known to be challenging. Many colleges have an accelerated introductory math sequence that will be good for this; in computer science, courses on operating systems and compilers are often known for their difficulty and attract the best/most interested students.

  • Avoid courses that are hard requirements for anything. These courses are the worst at filtering for good students because they're required. Additionally, they'll probably be taught at a low enough level that the good students will be disengaged and hard to spot.

  • Among sets of required alternatives, look for smaller courses. The largest course is probably the lowest common denominator. This is particularly important for general-education courses, in my experience.

Replies from: VipulNaik, John_Maxwell_IV
comment by VipulNaik · 2013-09-09T18:52:37.676Z · LW(p) · GW(p)

Thanks, this is a great comment. A lot of these are points that I hadn't considered in depth, though some are points that I deliberately decided against including. That's because the focus of this post was about selecting between instructors who are offering different sections of what, on paper, is the same course. The questions of whether to take honors classes or what course level to do are questions I plan to explore in depth in future posts.

I did mention peer groups, but relegated that mention mostly to the PS of my post. That's because, in the particular narrow context of my post, I don't think there is a huge peer selection effect. What I mean is that different sections of what is nominally the same course will generally see fairly similar student populations, despite differences in the actual quality of the instructor or the instruction. There's two broad reasons: (i) most students aren't optimizing too much for instructor within sections at the same course level, but rather, are affected by scheduling considerations, and (ii) departments often impose space constraints on class sizes in such a manner as to make sure that each section has a similar size. Within these space limits, treatment tends to be first-come-first-serve. Thus, even if better peers want to take a class with a specific instructor, in the highly space-constrained and first-come-first-serve world, they're unlikely to be able to reliably succeed at doing so.

The case where peer selection effects would be strongest in such a first-come-first-serve enrollment process would be where an instructor develops a good reputation with students at a certain quality level and a bad reputation with students at lower quality levels, so that those students are actually trying to avoid the instructor.

Replies from: benkuhn
comment by benkuhn · 2013-09-09T21:38:57.869Z · LW(p) · GW(p)

Oh, sorry. I think I interpreted "assuming that the major and course level are given" differently than you intended. My university is too small to offer different sections of the same course by different instructors.

comment by John_Maxwell (John_Maxwell_IV) · 2013-09-09T06:29:50.880Z · LW(p) · GW(p)

Better networking later: this is hopefully obvious, but knowing smart, competent people is useful for pretty much everything ever.

I'd actually like to see this better explored/supported... I believe it's true, and I've experienced it myself, but maybe if we could break down the specific ways in which networking can be useful, we could do it in a more optimized way (although if we optimize too hard, I assume we run the risk of coming across as phony?)

Avoid courses that are hard requirements for anything.

On the other hand, if you're not sure what you want to major in, you might try taking lots of these sort of classes to learn what you liked, start on long chains of prerequisites early, and keep your options open.

Replies from: benkuhn
comment by benkuhn · 2013-09-09T21:54:06.920Z · LW(p) · GW(p)

I'd actually like to see this better explored/supported... I believe it's true, and I've experienced it myself, but maybe if we could break down the specific ways in which networking can be useful, we could do it in a more optimized way (although if we optimize too hard, I assume we run the risk of coming across as phony?)

  • Collaboration: e.g., if I start a software company I'll be very happy that I know so many good programmers personally.

  • Signalling: for instance, people assume I'm more of a hardcore math person than I am because many of my roommates did well in math competitions.

  • Halo: I'm reasonably well-networked in some communities and people there seem to give my opinions a surprising amount of weight. Not sure if this is causation but it seems probable.

  • Favors: This has been the most important for me so far. Knowing people (who want to do things for you) is an almost fully general substitute for having skills. I'm not very good at biology, but my roommate knows enough to explain anything I don't understand. I'm bad at promoting my organization's events, but I can ask my friends on the student newspaper to help.

With regard to coming off as phony, I think in some circles there may be an implicit understanding that "networking" relationships are based on mutual benefit. YMMV though.

comment by John_Maxwell (John_Maxwell_IV) · 2013-09-08T23:44:23.675Z · LW(p) · GW(p)

Instructors differ significantly from one another in terms of their quality of teaching (human capital + consumption value) and ease of grading (signaling). I'm curious here about the extent to which LessWrong readers think variation between instructors for similar courses compares to variation between institutions. For instance, how much does variation between instructors at Harvard compare with variation between UCLA and Harvard?

I had a few great professors (e.g. this guy), and mostly good ones, at De Anza College, a community college in Cupertino, California, to the point where I was a bit disappointed after transferring to UC Berkeley.

Replies from: aelephant
comment by aelephant · 2013-09-08T23:57:00.000Z · LW(p) · GW(p)

I had an Organic Chemistry teacher at the small community college in my home town that blew away my big university's Medicinal Chemistry prof. The big university prof was tenured & could treat his students as abusively as he desired without any fear of significant consequences.

comment by DanDascalescu · 2013-09-09T03:16:58.277Z · LW(p) · GW(p)

I think there's a 4th type of value that students derive, implicitly, from courses that involve physical participation, especially: personal connections. Often these last well beyond the class, and many prominent businesses have been built by people who met in college.

Instructor aspects that would predict a student body likely to be a good pool of connections include assigning group tasks, encouraging peer-to-peer communication (e.g. debates, playing devil's advocate), organizing field trips, and borrowing from the corporate world: team-building exercises and outings in general.

comment by Douglas_Knight · 2013-09-08T20:56:58.071Z · LW(p) · GW(p)

It's quite rare for students to attend sample classes with instructors that they plan to study with

Every school I've been at had a strong tradition of a "shopping period" at the start of term where students tried out different classes, including different sections of the same class. Is this what you don't see happening? If this is something that varies a lot from school to school, it's probably just a matter of inertia, not of students having better methods. You suggest that lack of flexibility may discourage students from doing this, but my impression is that large inflexible state schools force a different kind of shopping through wait lists.

I've never seen it, but it might be better to sample a class in the middle of the previous term, to get a more representative sample; the drawback is that it might be incomprehensible.

Replies from: VipulNaik
comment by VipulNaik · 2013-09-08T21:48:07.348Z · LW(p) · GW(p)

@Douglas: " but it might be better to sample a class in the middle of the previous term" -- that's actually what I meant. I'll update the post to clarify. At UChicago, classes generally get full before the quarter begins, so it's hard to make changes even if one wants to, after the quarter begins. So people who don't sample classes the previous quarter are often stuck.

Also, thanks for sharing your thoughts! I really appreciate it.

comment by falenas108 · 2013-09-08T20:52:05.443Z · LW(p) · GW(p)

Huh, this is a very different perspective than I've had at the U of C (Going into my third year undergraduate)

I've found that people chose a lot based on instructors. Even in my majors, chem and physics, where there's almost no electives, people often defer taking courses for a year when there's a better instructor.

As for evaluations, I can't tell you what most other students do. But when talking about the evaluations what my friends to decide whether or not to take a class, most of us don't pay much attention to the specific numbers they got, and focus on whether in the comments section people say the instructor knows what they're talking about, is interesting, and has fair tests.

Replies from: 9eB1, VipulNaik
comment by 9eB1 · 2013-09-08T23:41:04.435Z · LW(p) · GW(p)

As a sidenote, saying U of X for every X isn't a very specific determiner, and it appears you were attempting to be specific. For example, University of Calgary is the top-rated result for "U of C" on google, but you could easily be referring to University of Chicago or University of Connecticut, among other U of C institutions.

Replies from: Kindly, falenas108
comment by Kindly · 2013-09-08T23:45:57.760Z · LW(p) · GW(p)

In context, it's almost certainly University of Chicago, given that this was the U of C mentioned in the post.

Replies from: 9eB1
comment by 9eB1 · 2013-09-09T03:31:23.979Z · LW(p) · GW(p)

Yes, that's a good point. I had forgotten about that mention by the time I got to the comments.

comment by falenas108 · 2013-09-08T23:58:00.081Z · LW(p) · GW(p)

I assumed it was obvious here, because the OP said he was talking about the University of Chicago.

comment by VipulNaik · 2013-09-08T22:55:30.545Z · LW(p) · GW(p)

@falenas108, thanks for sharing your thoughts. I genuinely appreciate it.

I've found that people chose a lot based on instructors.

Yes, I agree. I think some of my post carried the connotation that they don't, but that was unintended. I wrote:

Students routinely neglect instructor selection or use suboptimal criteria, relative to its importance. [...] Often, students decide their courses for the next semester or quarter after a meeting of a few hours with an advisor, based on time scheduling constraints.

So, to clarify, although I think it "often" happens that people select instructors randomly, it doesn't always happen. My impression is that there's heavy availability bias: if students can easily access evaluations, or know friends who have taken classes with a particular instructor, they'll use that data. If not, they'll select randomly. In general, my impression is that students are risk-averse and satisfice on this count: as long as an instructor seems "good enough" they won't generally try to look for a better one (this is particularly true for people in multi-course sequences with an instructor, we could call this a "status quo bias" or an "endowment effect" depending on your perspective). More proactive approaches, such as sitting in on classes that the instructors are teaching in the previous term, seem to be relatively rare.

My other point is that even when students are making proactive choices, the criteria they use may be suboptimal. I am not aware of high quality advice that would help students select instructors effectively. This is, I believe, in contrast with the vast (though possibly not very high quality) literature available on how to select a college or a major. If intra-institutional variation in instructor quality is comparable to inter-institutional variation, then there are probably unrealized gains in selecting instructors according to better criteria.

As for evaluations, I can't tell you what most other students do. But when talking about the evaluations what my friends to decide whether or not to take a class, most of us don't pay much attention to the specific numbers they got, and focus on whether in the comments section people say the instructor knows what they're talking about, is interesting, and has fair tests.

I've added an update to the post in response to this.

Replies from: linkhyrule5
comment by linkhyrule5 · 2013-09-08T23:20:01.135Z · LW(p) · GW(p)

One thing I would note is that good teachers are rare and not very "good," but bad teachers are common and very very bad. This ties into Trevor's post: once you've avoided the bad teachers, it's probably not worth the much larger effort required to find an extra-bonus-good teacher that may or may not exist.

Replies from: VipulNaik
comment by VipulNaik · 2013-09-09T00:09:33.007Z · LW(p) · GW(p)

@linkhyrule5: Thanks for your thoughts, I really appreciate it.

I'd be grateful if you could provide some context regarding the reference class of educational environments for which you've made this statement (that the main problem is avoiding really bad teachers, rather than looking for really good ones).

Replies from: linkhyrule5
comment by linkhyrule5 · 2013-09-09T01:28:13.484Z · LW(p) · GW(p)

I should probably have qualified my earlier post, as the only data I have to draw on is my own anecdotal evidence.

Nevertheless, with the qualifications that I went to a private high school and am currently at an Ivy college:

Sample size: since freshman high school, 26 "teachers" and nine "professors."
Definitions: a teacher/professor is good-I if I was particularly interested in his/her course (and not just the subject), and good-R if I retained particularly more from that course. A teacher/professor is bad-I if I was particularly uninterested in his/her course (and not just the subject), and bad-R if I retained particularly little from that course.

Of my teachers, 2 good-I teachers and 0 good-R teachers, with two "maybes" - not particularly good but above-average. 3 teachers who were both bad-I and bad-R - all of these are extreme cases, in which I learned almost nothing and loathed the class.

Of my professors, 0 good-I and no data on good-R (I'm a sophomore), but already 2 bad-I and I highly suspect bad-R.

Replies from: VipulNaik
comment by VipulNaik · 2013-09-09T01:40:43.133Z · LW(p) · GW(p)

Thanks, I appreciate the clarification.

comment by [deleted] · 2013-09-08T22:11:35.437Z · LW(p) · GW(p)

A good instructor will often offer a good class that good students can take advantage of. A bad instructor will never offer a good class and no one can take advantage of it. Avoiding bad teachers is therefore always a move in a good direction, while selecting good teachers is only sometimes a move in a good direction.

I've been working in K-12 and college since 2005 as a sign language interpreter. The qualities of good instructors varies, but the qualities of bad instructors are shared. Late, unprepared, digressive without profit, argumentative, close minded, uses the class as political platform, uses the class as therapy session, uses the class as amateur comedy, unable / unwilling to earn respect (grudging or otherwise) of class - those are some of the things bad instructors share. Avoid those and you're more likely to succeed in any class.

Replies from: Decius
comment by Decius · 2013-09-09T02:36:22.522Z · LW(p) · GW(p)

A list of what to avoid is not anywhere near as useful as advice on how to identify those characteristics before the cost of not taking that course is too high (e.g. add/drop deadlines, full classes)

comment by pslunch · 2013-09-09T04:11:37.381Z · LW(p) · GW(p)

Another major factor for grads or advanced undergrads is the research that the professor is doing. This primarily comes into play during office hours (which are generally empty except before exams). Especially for established figures, this can be the only chance you'll have to get to know them (most are far too busy to take casual callers).

Even the worst lecturers are sometimes extraordinary one-on-one and, even when not, people doing interesting work tend to have far more than average contacts with other people doing interesting work. Show them that you're engaged and curious and invisible doors will open to you (many of the most interesting positions are never posted, but instead filled by qualified recommendations).

comment by benkuhn · 2013-09-09T03:28:51.927Z · LW(p) · GW(p)

Data from my experience at Harvard: variability between instructors is very high (probably because many of them got professorships on the basis of their research rather than teaching; the two correlate but with a lot of noise). I'd guess this variance is much greater than institutional variance, though obviously I can't say for sure.

Also, your phrasing suggested that you thought Harvard had very high-quality teaching. It's probably good, but I wouldn't be surprised if most non-research colleges had higher average teaching quality, at least for college-level courses that don't require deep knowledge of highly advanced material. I'd guess that the advantage of taking classes at Harvard is more in the peer group (see other comment), and maybe the advanced undergrad or grad-level courses where command of the subject matter might vary more. (However, I've only ever taken classes at Harvard, so someone who's transferred in or out would have better info.)

Also, students here weigh instructor quite heavily in their course decisions, at least in the math department where professors switch courses almost every year.

comment by TsviBT · 2013-09-09T03:06:51.982Z · LW(p) · GW(p)

Going into 3rd year undergrad at UChicago, I've selected all but one of my classes based on having been in at least a few class sessions with the same instructor. (The exception has extremely positive course evals.) I think being in a class with an instructor is the only valuable, reliable, consistently available source of information here - since students vary so much in learning style, learning ability, and background, it's hard to get good info from them. I think students don't shop as much as they could because that requires doing stuff.

The only times I pay attention to course evals is when they are mostly unanimous, or when they give specific indications of qualities that are definitely good or definitely bad. E.g. good: "gives lots of examples quickly, without being repetitive", "the homeworks are enlightening/challenging/not tedious" and bad: "doesn't speak clearly", "gives obscure/tedious homework problems".

By the way, Vipul, I think I have your old Linear Programming book from the Eckhart table. :)

comment by Dre · 2013-09-09T04:52:37.161Z · LW(p) · GW(p)

Going off of what others have said, I'll add another reason people might satisfice with teachers.

In my experience, people agree much more about which teachers are bad than about which are good. Many of my favorite (in the sense that I learned a lot easily) teachers were disliked by other people, but almost all of those I thought were bad were widely thought of as bad. If you're not as interested in serious learning this might be less important.

So avoiding bad teachers requires a relatively small amount of information, but finding a teacher that is not just good, but good for you requires a much larger amount. So people reasonably only do the first part.

comment by jsteinhardt · 2013-09-09T02:27:48.711Z · LW(p) · GW(p)

Strongly agree that instructor choice is very important.

comment by MarkL · 2013-09-09T01:27:36.396Z · LW(p) · GW(p)

For math courses, I always took the "honors" version. This was not because I was an over-achiever but because the honors versions were taught by professors whereas the normal versions were taught be TAs. The instruction was soooooooooooo much better, which made up for the extra work and/or extra difficulty. My annoyance level was kept low, which was worth it.