Musings on the LSAT: "Reasoning Training" and Neuroplasticity

I can share my personal experience, but I'm an outlier on many axes.

I identify strongly with a lot of the things you wrote about liking tests.

I studied for and took the LSAT in the mid-2000s. I scored either a 166 or a 168 (can't remember) on my first practice exam, which I took cold. Nearly all of the points I lost were on the games section.

Incidentally, this was a pretty emotionally difficult experience for me, because I had never done this poorly on a 200-800 (120-180) scored standardized test before. This one felt especially like an IQ test, so not being able to crack the 700 (170) barrier was very threatening to my self-esteem.

Your guess that most LSAT prep focuses on developing written systems for the games section is correct. Powerscore was the most highly regarded producer of LSAT prep materials for high scorers when I took the LSAT, and their reputation was larlgey due to their succeess in teaching people how to create accurate and efficient diagrams for the games. I think they also produced fairly well-regarded materials for the other sections too, but I didn't investigate those because my deficiency was the games section.

My preparation consisted of working through the Powerscore games materials and taking lots of practice tests. LSAC published old LSATs in books of ten tests each, and three such books had been published at the time I was studying, so I bought them all and just took those 30 LSATs with a stopwatch. I ran out of old LSATs prior to the test, so I went online and found a torrent of dubiously legal low-quality scans of more recent LSATs, though I only had time to take one of those before the exam because the school year started.

On the logical reasoning and reading comprehension sections, I would guess that my mean missed questions per section prior to studying was about 3, with a standard deviation of about 1. After studying, it was probably about a mean of 2 with a standard deviation of maybe 0.7. I typically missed questions because I would read too quickly and miss something subtle and important. I did not improve significantly at this, even though I did a lot of practice. I did not use symbolic logic or any other systematized written system to answer any logical reasoning or reading comprehension questions; it's possible that this would have helped.

On the games sections, my pre-studying mean missed questions per section was probably about 10, with a standard deviation of 3. After studying, the mean was probably 1 with a standard deviation of 1. Games went from being my weakest to my strongest section once I learned how to make and use good diagrams, and it became a fairly regular occurrence for me to finish the entire section with time to spare and complete certainty in the correctness of all of my answers.

Doing some of the older tests before I mastered the technique for the games, I scored as low as, IIRC, 158. This resulted in much hyperventilating and cruel self-talk. As I improved at the games and worked my way up through more recent tests, my scores improved to the point where I was consistently in the 170s, and achieved a few 180s. That one torrented practice test that I took the week before the actual test was the last practice test I took, and I got a 169 on it somehow, which was, of course, terrible for my confidence going into the exam.

I scored a 175 on the actual test.

The entire reason the LSAT is hard is the time pressure. I think that, given enough time, any 150s-scorer could arrive at the right answer to any LSAT question. I agree with ThisSpaceAvailable that the games section is very much a "can-you-follow-instructions-under-time-pressure" test, and my experience was like ioshva's in that my missed logical reasoning questions were due to lapses in focus while reading under time pressure rather than inability to grasp the logic.

I do not believe the process of studying for the LSAT inculcated any rationalist habits of mind in me. For me, because of my strengths and weaknesses, the process of studying for the LSAT was largely the process of learning how to complete contrived little puzzles that are isomorphic to sudoku. I suppose there might be some Lumosity-type minor positive effect on working memory, but I doubt the LSAT is much better for that than any other mentally engaging task. The research you cited in the OP just makes me want a control group of people playing Lumosity games. I can imagine that, for someone whose weakness was the logical reasoning section and who was missing lots of logical reasoning questions because of a deficit in understanding of the basic mechanics of logic, studying for the LSAT would be one way to correct that deficiency. But I suspect that ioshva is correct in that just learning logic directly would be a more effective way of achieving the same result.

I do not believe the process of studying for the LSAT helped me in law school. I think the correlation between LSAT score and first-year law school GPA exists largely because high-IQ and high-conscientiousness people tend to do well on both. I performed abysmally in law school despite my relatively high LSAT score because of pathologically low conscientiousness. I was only able to overcome low conscientiousness during the LSAT preparation process because I was living with my parents at the time and they put considerable pressure on me to study hard.

LSAT score is certainly a huge factor in law school admissions, and I was admitted to a moderately prestigious law school solely on the strength of my LSAT score and in spite of a truly awful undergraduate GPA. In general, LSAT score, undergraduate GPA, and underrepresented racial minority status are really the only meaningful factors in the law school admission process. This holds except in weird edge cases where people have distinguished themselves in exceptional ways, like former pro athletes or people who have founded successful businesses or nonprofits. Things like recommendation letters and essays can potentially keep you out if they're bad, but will not get you in except, again, in weird edge cases like having a Supreme Court justice write you a recommendation insisting that you're the brightest mind of your generation.

Incidentally, on the "think like a lawyer" point raised by MaximumLiberty - in my experience, this usually refers to a couple of specific analytic skills that lawyers are trained to develop: (1) reasoning by analogy and distinction, as with case law analysis, and (2) thinking strategically about the way systems of rules constrain or fail to constrain the behavior of agents within them, as with advising a client on how to avoid punishment. I'm not sure that this sort of software specialization would be reflected in the hardware imaged by an fMRI.

I'm a lawyer, over 20 years out from law school. I took the LSAT cold, so I'm not a good candidate for your questions. I've always liked taking tests and always did well on standardized ones. I did well on the LSAT.

The reason I am responding is to add a bit of information. Lawyers talk, among ourselves and to law students, about what it means to "think like a lawyer." It is a topic of fairly serious debate in jurisprudence for a number of reasons. One is that lawyers have a lot of power in American society. There are issues of justification and effects there. Another is the underlying sense that we really do think differently from most people. We see it in our everyday lives and it sparks our curiosity. There are many other reasons.

So, it makes me wonder what the MRI images would show when comparing lawyers' brains to comparable non-lawyers brains.

It seems to me that if you remove the time constraint, the g-level is really quite low. It says "Jamaica will not be its destination in week 4.", so just go through the options and cross out all the options where Jamaica is its destination in week 4. That gets rid of D. "Trinidad will be its destination in week 7." gets rid of E. "Guadeloupe will be its destination in the week preceding any voyage it makes to Jamaica." gets rid of C. "at least one voyage to Guadeloupe will occur in some week between those two voyages." gets rid of B. That leaves A. I don't see how intelligence comes in, except insofar as it allows one to perform the task more quickly. It's not a "Can you do logical reasoning"? test, it's a "Can you follow simple instructions under time pressure?" test.

According to the 2013 survey, only 2.2% of you are in law-related professions, but I was wondering (1) if anyone has personal experience studying for this exam, (2) if they felt like it improved their logical reasoning skills, and (3) if they felt that these effects were long-lasting. Studying for this test seems to have the potential to inculcate rationalist habits-of-mind; I know it's just self-report, but for those who went on to law school, did you feel like you benefited from the experience studying for the LSAT?

(1) Yes, but I'm an outlier. I started in the 99th percentile and "improved" 6 points through self-study.

(2) Honestly? Not really. For me, most of the difference in performance from test to test was not due to logical misapprehension, but because I skimmed a question or misjudged the time limit. If you have taken an undergrad logic course and have a grasp on conjunction/disjunction, sufficient/necessary, etc., then your experience will likely be similar. For instance, you said this about the games:

I'd guess that most LSAT test prep is about strategies for dumping this burden into some kind of written scheme that makes it all more manageable.

This is true, but it's also true of the args, too. I've taught for two of the major test prep companies and the courses are mostly just an undergrad logic course bolted onto their own proprietary shorthand and systematic categorization so students can recognize types of questions and diagram them accordingly. When I tutored independently, I just used regular old symbolic logic.

P1: AW↑ ==> Right
P2: AW↓ <=> Wrong
P3: ____________
C: ~AW↑ & ~AW↓ ==> Right

The only tricky part about teaching that question (which I can't recall teaching specifically) would be that most novices will diagram AW– or something similarly distinct from the premise elements for neutral welfare. So you have to teach them to diagram conclusions in the form of premises whenever possible.

(3) Well, I already had most of these skills, but I would say I definitely got a lot out of teaching. It's a fun test compared to, say, the MCAT. I don't think it would be all that great as a self-improvement tool, though. Without a tutor you won't always understand where you're screwing up, and some of the questions are sometimes tricky for the sake of being tricky. For instance, there may be a version of that question where the answer turns not on logic, but on one of the premises subtly leaving out the word "reasonably." Additionally, there really aren't that many different kinds of questions. Once you start looking at more than a few tests you will start to recognize lots of questions that are logical repeats with different subject matter, or maybe a reversed answer condition (which of the following is NOT implied versus which of the following IS). If all you want is to improve your logical reasoning personally, I'd just take the undergraduate logic courses.

1) I took it, but I didn't do much studying for it. (Basically, I signed up for it at nearly the very last moment after I saw someone mention that all it took to get into law school was a good LSAT--I had been pursuing a different career and had not previously thought of going to law school, but I had started doing legal-related work in a volunteer gig.) Maybe a week before the exam I went to the library and checked out a prep book. And the logic games section was already something I basically knew, so what I did spend time on was careful reading of the critical reading sections; I tend not to read carefully and miss instructions, and I wanted to learn the kinds of tricks they were likely to use to get me to do just that.

2 and 3) No; I used the logical reasoning skills I had already from studying math. (Also, from having taken every vaguely logic-related course at my undergrad.) Those were long-lasting. But I enjoyed math because many of those skills were already natural to me. I learned refinements and additional techniques and became better at it, but I was already inclined to thinking that way and enjoyed it.

As a lawyer now, one of my major strengths lies in analytical reasoning--I like to consider situations and take apart the possible situations that may arise, what happens if they're taken to their logical conclusions, where contradictions might arise from sets of terms, what logical inconsistencies exist in a proposal. (The biggest and most enjoyable project I've worked on has been license drafting.)

Would you consider schedule C,

(C) Jamaica, Martinique, Guadeloupe, Martinique, Guadeloupe, Jamaica, Trinidad

to fail due to the following requirement?

Guadeloupe will be its destination in the week preceding any voyage it makes to Jamaica.

The language seems slightly ambiguous in the case where the trip to Jamaica is in the first week, but my guess is that they intended to rule that out.

I don't understand what the neuro stuff in the second half of the post is meant to contribute here.

If the LSAT (or studying for it) has measurable cognitive benefits, or is predictive of future success, awesome! So who cares whether you can detect some cerebral blood flow difference with fMRI?

If the LSAT has no predictive (or other) value, sad! So who cares whether you can detect some cerebral blood flow difference with fMRI?

Obviously this is in no way specific to your post, don't mean to single you out. But the "it happens IN THE BRAIN so it's super important" way of thinking is to my mind a corrosive trope in contemporary discourse which I like to call attention to whenever I see it. In my opinion these kind of New Yorker-friendly brain imaging studies tend to obscure more than they illuminate for most issues.

But to answer your questions in the final paragraph 1) Yes I studied for it, quite a bit actually (was studying for the GRE in parallel, which was actually more important, but the LSAT studying was more fun). 2) I'd say yes it definitely improved my logical reasoning skills but no, (3) there was likely little lasting benefit.
Two reasons: (a) because as a general rule interventions that boost cognitive performance in the short term tend to wash out over longer timescales, so my default position would be that the same is true for the LSAT, and (b) LSAT-type reasoning wasn't terribly far removed from the kind of thinking I was used to, so the marginal benefit was probably small.