Reading Papers in Undergrad

post by ryan_b · 2017-11-09T19:24:13.044Z · LW · GW · 7 comments

Contents

7 comments

I went to school for electrical engineering, and in my capstone electromagnetic theory course I encountered a problem that I needed to read papers to learn about. I asked my professor for his advice on how to read them, owing to my lack of expertise, and he recommended the following:

Don't read the abstract. This is written by experts for other experts, so they can quickly see if it is relevant to their work. It will provide no useful information, and may intimidate or confuse you.

Do read the introduction. This will contain an overview of experiment, and most importantly describe some of the background and motivation for the experiment. As a non-expert, this context is at a premium. If the introduction does not suggest there will be useful information, you can stop here.

Skip the body of the paper. The methods and construction of the work are usually not the goal of students, and this part is the most dense and requires the greatest expertise.

Do read the conclusion. This is usually the meat of what a student is looking for - what do experts think, in their own words, about their work. With this information in hand, you can decide whether you need to go deeper into the paper or have what you need. If you decide to go deeper:

Return to the body of the paper. This is especially useful if you are greatly surprised by the conclusion, or if you need to be able to reference images or graphs in an informed way. Even as a non-expert you can profitably think about how the conclusion follows from the steps they have laid out. It is naturally required if you want to attempt to duplicate the experiment.

I have had pretty good success with this method, even in subjects for which I have less than undergrad expertise. I will add two additional steps I found necessary, both obvious:

Reread the paper. As you follow up on the problem with additional reading, you will gain important new perspectives and context that led to missing things the first pass.

Follow-up the references. The assumptions about how diligent readers are with following through vary; this can make something aggravatingly opaque suddenly clear because it was covered well in the reference, or reveal a gaping hole the authors failed to address.

Anecdotally, the problem I was thinking about was the electrical signature of cancer. I had found a few modern references to work done on the subject in animals, but couldn't lay hold of work on humans for the life of me. Finally on the third pass over a survey paper covering related work I saw an obscure reference, and following through with that led me to all of the work which had been done on humans previously.

7 comments

Comments sorted by top scores.

comment by Unnamed · 2017-11-10T08:30:38.031Z · LW(p) · GW(p)

In psychology I'd give very different advice from this.

You do want to start with the abstract of a psychology paper. It tells you what the paper is about, often relatively clearly. I often decide which papers are worth reading based on the abstract, and which parts of the paper look most interesting.

If the paper mainly looks interesting as a summary of previous research, then I'll start with the introduction. If the paper mainly looks interesting for its new studies, then I'll typically start by looking at the pictures (figures and tables). I recall saying more about this in a previous LW comment... here.

Replies from: ryan_b
↑ comment by ryan_b · 2017-11-11T15:35:27.100Z · LW(p) · GW(p)

Have you always read psychology papers the same way? Have your results changed much as you gained a broader base of psychological knowledge?

Replies from: Unnamed
↑ comment by Unnamed · 2017-11-11T21:40:39.300Z · LW(p) · GW(p)

Experience has helped me get better at quickly finding the most relevant parts of the paper and at understanding what they mean without needing the rest of the paper for context.

I started out in undergrad reading papers straight through start to finish, since that was the obvious default thing to do (plus it felt like what I was supposed to do with a paper that was assigned in class). I gradually developed this approach over time. So it's somewhat hard to know how this approach would've worked when I was first starting out; it does at least seem worth trying (and thus worth sharing here).

Also, my description is just an approximation of what I do, which varies in complicated-to-describe ways from paper to paper (and depending on what I want out of the paper). And much of the work is in picking which papers to look at, and in getting background information from other sources (like Wikipedia).

comment by waveman · 2017-11-09T22:19:20.809Z · LW(p) · GW(p)

Unfortunately a lot of academic papers report false or much exaggerated conclusions (https://hardsci.wordpress.com/2016/08/11/everything-is-fucked-the-syllabus/). I have found from 30 years of reading them that there is no substitute for a) going through the methods closely and with a suspicious mind b) looking at the declared affiliations and sponsorhip and even looking outside the paper for affiliations and sponsorship. because sponsorship hugely skews results c) Closely looking at the statistics, which in turn requires a good understanding of statistics. If you do not or cannot do these things you are just reading headlines in my bitter experience.

Replies from: None
↑ comment by [deleted] · 2017-11-11T18:05:37.255Z · LW(p) · GW(p)

For someone like who feels a little inadequate analyzing things like experimental design / statistical analysis, I still largely agree with this.

My shallow experience with psychology papers is that abstracts still often try to be grander than what the paper's actually about, and looking into exactly what they did is important. Some obvious things like looking at methodology (e.g. "How did they attempt to measure the thing they claim to have measured?") and sample sizes ("Was this done with 16 people or 1600?") can still give you a better idea.

EX: I recall reading a study about budding altruism/charitability in young children that felt very contrived, involving a story around magic boxes, marshallows, and a teddy bear. In this case, I might be skeptical that the results they tried to generalize from were actually there.

comment by Vaughn Papenhausen (Ikaxas) · 2017-11-10T03:30:18.469Z · LW(p) · GW(p)

This seems like really good advice for papers in data-driven and/or technical fields (modulo the caveat pointed out by waveman, which also seems worth taking into consideration), that is, papers in fields like e.g. physics, the aforementioned electrical engineering, psychology, economics, perhaps math, etc. My experience as a final-year undergraduate reading philosophy papers, though, is that I do get a lot out of reading the abstract, and out of not skipping the middle of the paper (since there isn't really an analogue to the "methods" section in philosophy papers). I'm not sure if this is unique to philosophy or if there are other fields that are also exceptions; would be interested to hear people's thoughts from other disciplines.

Replies from: ryan_b
↑ comment by ryan_b · 2017-11-11T15:32:34.718Z · LW(p) · GW(p)

This is a good point - for example, I find history papers to be easily readable in order, and the abstracts are useful. My expectation is that this will work for any of the humanities, are more generally anything lacking a minimum threshold for methodological background.

I do frequently encounter difficulties with highly specialized terms: sociology is a common subject where that problem appears.