Scholarship: How to Do It Efficiently

post by lukeprog · 2011-05-09T22:05:42.156Z · LW · GW · Legacy · 147 comments


  Review articles and textbooks are king
  Going granular

Scholarship is an important virtue of rationality, but it can be costly. Its major costs are time and effort. Thus, if you can reduce the time and effort required for scholarship - if you can learn to do scholarship more efficiently - then scholarship will be worth your effort more often than it previously was.

As an autodidact who now consumes whole fields of knowledge in mere weeks, I've developed efficient habits that allow me to research topics quickly. I'll share my research habits with you now.


Review articles and textbooks are king

My first task is to find scholarly review (or 'survey') articles on my chosen topic from the past five years (the more recent, the better). A good review article provides:

  1. An overview of the subject matter of the field and the terms being used (for scholarly googling later).
  2. An overview of the open and solved problems in the field, and which researchers are working on them.
  3. Pointers to the key studies that give researchers their current understanding of the topic.

If you can find a recent scholarly edited volume of review articles on the topic, then you've hit the jackpot. (Edited volumes are better than single-author volumes, because when starting out you want to avoid reading only one particular researcher's perspective.) Examples from my own research of just this year include:

If the field is large enough, there may exist an edited 'Handbook' on the subject, which is basically just a very large scholarly edited volume of review articles. Examples: Oxford Handbook of Evolutionary Psychology (2007), Oxford Handbook of Positive Psychology (2009), Oxford Handbook of Philosophy and Neuroscience (2009), Handbook of Developmental Cognitive Neuroscience (2008), Oxford Handbook of Neuroethics (2011), Handbook of Relationship Intitiation (2008), and Handbook of Implicit Social Cognition (2010). For the humanities, see the Blackwell Companions and Cambridge Companions.

If your questions are basic enough, a recent entry-level textbook on the subject may be just as good. Textbooks are basically book-length review articles written for undergrads. Textbooks I purchased this year include:

Use Google Books and Amazon's 'Look Inside' feature to see if the books appear to be of high quality, and likely to answer the questions you have. Also check the textbook recommendations here. You can save money by checking Library Genesis and for a PDF copy first, or by buying used books, or by buying ebook versions from Amazon, B&N, or Google.

Keep in mind that if you take the virtue of scholarship seriously, you may need to change how you think about the cost of obtaining knowledge. Purchasing the right book can save you dozens of hours of research. Because a huge part of my life these days is devoted to scholarship, a significant portion of my monthly budget is set aside for purchasing knowledge. So far this year I've averaged over $150/mo spent on textbooks and scholarly edited volumes.

Recent scholarly review articles can also be found on Google scholar. Search for key terms, and review articles will often be listed near the top of the results because review articles are cited widely. For example, result #9 on Google scholar for procrastination is "The nature of procrastination" (2007) by Piers Steel, the first half of which is a review article, while the second half is a meta-analysis. Bingo.

You can also search Amazon for key terms. I recently searched Amazon for 'attention neuroscience.' Result #2 was a 2004 scholarly edited volume on the subject. A bit old, but not bad for my first search! I found the PDF on

In order to find good review articles, textbooks, and scholarly edited volumes you may first need to figure out what the terminology is. When I wanted to understand the neuroscience of pleasure and desire, it took me a while to figure out that the neuroscience of emotions is called affective neuroscience. After consuming that field, I had learned a lot about pleasure but not much about desire. I then realized that I didn't care about desire as an emotion but instead as a driver of action under uncertainty. That aspect of desire, it turns out, is studied not under the field of affective neuroscience but instead neuroeconomics.

Similarly, when I was originally looking for 'scientific self-help', I had trouble finding review articles or textbooks on the subject. It took me months to discover that professionals call this the psychology of adjustment. Who would have guessed that? But once I knew the term, I quickly found two textbooks on the subject, which were good starting points for understanding the field.

Note that not every scholarly edited volume is a volume of review articles. New Waves in Philosophy of Action is a collection of new research articles, not a collection of review articles. It is a poor entry point into the field. Some edited volumes are okay entry points into the field because they are a mix of review articles and original research, for example Machine Ethics (2011). But remember that a 'good' edited volume on a subject does not protect you from the entire field being mostly misguided, like machine ethics or mainstream philosophy.

Also note that if you can't find an edited volume on your subject, one may be just around the corner. In 2007 there was no decent edited volume on neuroeconomics, but there were three review articles. Then in 2008, Decision Making and the Brain was released.


Going granular

Once textbooks and review articles have given you a good overview of the key concepts and terms, open and closed problems, studies and researchers on your chosen topic, it's time to go granular.

Textbooks and review articles will point you to the articles most directly relevant for answering the questions you have, and the researchers working on the problems you care about. Visit researchers' home pages and check their 'recent publications' lists. Find the papers on Google Scholar and read the abstracts. Make a list of the ones you need to read more closely. You'll be able to download many of them directly from links found on Google Scholar. For others, you'll need to visit a university library's computer lab to download the papers. The university will have subscribed to many of the databases that carry the papers, and university computers will let you past the paywall (but on-campus wifi will not). To get access to a paper you can't get at a nearby university, you can:

  1. Contact the author via email and request a copy (or a preprint), explaining that you can't get it elsewhere.
  2. Ask your friends at other universities to check if their university has access to it.
  3. Look to see if the article has been published in a book that is available at your library or online.

I've never purchased an article from an online database because the prices are outrageous: $15-$40 for a 20-page article, usually. If I absolutely can't get access to an article, I make a judgement as to how much weight to give the study's conclusions, inferring this from the researcher's history and the abstract and responses to the article I can read and other factors.

Skim through promising research articles for the information you want, watching for obvious problems in experimental design or quality of argument. This is where your time investment in scholarship can explode, so be conscious of the tradeoffs involved when reading 100 abstracts vs. reading 100 papers.

You can also try contacting individual researchers. This works best when the subject line of your email is very descriptive, and is obviously about a detail in their recent work. The content of your email should ask a very specific question or two, quoting directly from their paper(s). Researchers are often excited to hear that somebody is actually reading their work closely, though philosophers get more excited than neuroscientists (for example). Neuroscientists are called for comment by the media somewhat regularly. This doesn't happen to philosophers.

Finally, if you've done all this work already and you're feeling generous, perhaps you could take a little time to write up the results of your research for the rest of us! Or, help make Wikipedia better.



Comments sorted by top scores.

comment by Brihaspati · 2011-09-16T05:42:22.341Z · LW(p) · GW(p)

If a book exists in PDF on the web somewhere, I estimate a 90% of chance of finding it by at least one of the following four methods, which altogether take only a few minutes if you're practiced at it:

  1. Search (You'll have to sign up for a free account the first time.)
  2. Search Library Genesis.
  3. Search Scrapetorrent.
  4. Use a custom Google search and keeping clicking through links until you find a filesharing site link for the book - one that isn't broken yet.
Replies from: None, hg00, None, gwern, None
comment by [deleted] · 2012-03-24T21:41:08.558Z · LW(p) · GW(p) looks pretty nice. Found a book I had a hard time finding elsewhere, in multiple editions.

comment by hg00 · 2013-04-21T20:32:55.394Z · LW(p) · GW(p)

OK, I recently learned of a new way to pirate stuff: Usenet. The first rule of Usenet is supposedly "don't talk about Usenet", to avoid having the RIAA/MPAA learn about its usefulness, but I'll let you rationalist types in on the secret.

This appears to be the procedure for downloading pirated stuff off of Usenet (edit: googling around seems to yield improvements to this procedure, e.g. providers, indices, also this guide is more detailed):

  • Search on or a similar Usenet search site to see if what you want exists on Usenet.

  • If it does, Giganews appears to be the most beginner-friendly Usenet provider (I assume once you get comfortable with Usenet, you can find a cheaper provider, but Giganews seems like a solid choice anyway). Sign up for Giganews (they give you a 2-week free trial, so you might want to plan out your downloads in advance to take advantage of the trial).

  • Use Binsearch to generate a .nzb file for the files you want to download (similar to a torrent file). Get an NZB downloader like SabNZBD. Configure it with the Giganews server ( and your Giganews login credentials. Then feed it your .nzb file, which it should start to download.

Other Usenet client applications like Thunderbird and Pan have more capabilities, like allowing you to browse the Usenet newsgroup hierarchy, read messages, and post. As far as I can tell, Usenet is full of cranks, old people, spammers, tons of porn (much weird and some illegal), and tons of pirated stuff. I did find some relatively sane discussion here and there.

EDIT: A friend recommends for an improved piracy experience. It's not obvious from their website how it works, but you can google " piracy" for more info.

comment by [deleted] · 2012-09-04T10:46:39.163Z · LW(p) · GW(p)

Thank you for making this list! It was still useful one year later on for me.

Please consider updating though. is down due to busybodies.

Replies from: hg00
comment by hg00 · 2013-03-13T06:30:05.493Z · LW(p) · GW(p)

Scribd, Oyster, and Kindle Unlimited all give you a "netflix for books" type experience where you pay a monthly fee of about $10 and read as many books as you want (not newer books, unfortunately). (Kindle Unlimited might be better if you have a non-Kindle-Fire kindle device it will work well with, but since publishers don't like Amazon it will never have as good quality of a selection as the other two.) Your local library may also have ebook lending options.

BTW, if you want papers rather than books, this browser extension or this thread (actually, use this more recent one) may be of interest, esp. this site or this site or this site or this site (some of these might be searching the same database) or or the #icanhaspdf twitter hashtag or this Facebook group

Note when using libgen search engines, gwern writes: "I've noticed the Libgen search engines seem to have problems with long titles and/or colons" so you may wish to strip those.

Someone else recommends searching on the Pirate Bay, especially when combined with "pdf"/other typical book file extensions.

Another cool site. reddit discussion of book piracy. List of LibGen mirrors. EBook search engine? Another list of sites. And another even longer one. Quora thread compiling sites. Another list. Another list. Haven't tried this site yet. Or this.

Don't forget about libraries either!

Reddit claims duckduckgo can be good for finding pirated stuff.

Replies from: hg00, hg00, hg00, hg00, hg00
comment by hg00 · 2014-12-31T07:04:36.712Z · LW(p) · GW(p)

Some info on private trackers for pirated ebooks.

Replies from: hg00, Jiro
comment by hg00 · 2019-06-23T23:44:13.859Z · LW(p) · GW(p)

Just saw this Facebook group for getting papers. There's also this. And

comment by Jiro · 2014-12-31T22:15:28.251Z · LW(p) · GW(p)

You can get invites on the IRC channel, but make sure you already have a few ebooks on hand you can offer to upload

Is that like needing experience to get a job and needing a job to get experience?

People who don't have access to private trackers already would probably not have any material that anyone needs unless they personally pirated their own book direct from the source. (I remember back when people had to do anime trading by postal mail, how that would lead to a similar catch-22 where you couldn't get anything unless you had something to trade.)

It also fails in the scenario where someone is not really a pirate, but they just want a single item which is impossible to reasonably get through normal channels and so have resorted to pirating in this one instance.

Replies from: hg00
comment by hg00 · 2015-01-01T06:48:49.322Z · LW(p) · GW(p)

Is that like needing experience to get a job and needing a job to get experience?

Well, you might try this thread full of other methods for pirating books ;) Or buy ebooks and pirate them, but yes this is a bigger investment.

comment by hg00 · 2013-09-29T03:58:24.909Z · LW(p) · GW(p)

Just saw this recommended on /r/nootropics for getting papers; can't vouch for quality/usefulness:

comment by hg00 · 2013-07-13T21:53:21.438Z · LW(p) · GW(p)

More ways to get scientific papers:


comment by gwern · 2012-02-03T18:22:18.284Z · LW(p) · GW(p)

There's also #ebooks on Hard to use, but on occasion they have something that hasn't made it to any of the others.

Replies from: siodine
comment by siodine · 2012-02-03T18:45:47.804Z · LW(p) · GW(p) is useful for finding books on sites like rapidshare and megaupload. is another good one.

comment by [deleted] · 2012-02-03T19:14:02.124Z · LW(p) · GW(p)


comment by autodidaterous · 2011-05-14T11:18:23.437Z · LW(p) · GW(p)

I take strong issue with the entire field in two weeks claim. As a practicing "autodidact" for more than ten years post-university, I have assimilated many fields from the standard sciences (biochemistry, medicine, neuroscience, pharmacology, chemistry, physics, electronics, math, statistics, programming) to postgraduate level, to some arts (design, writing & drawing comics) and business topics.

Retrospectively fields like neuroscience were quite simple after learning biochemistry, and I would believe your two weeks claim for a specific aspect (i.e. LTP in hippocampus), but not for the entire field. For math it took me nearly two years of rigorous study to be able to talk to Math Phd's on a slightly lower level, and that was considered quite fast by my mentors (mathematics prof's).

Often I will want to know more about a specific part of a subject, e.g. antenna design. This took me over six months to get to grips with, even after all of my combined experience.

So, either I'm just a slow learner, you are some kind of super genius, the topics you are tackling aren't actually outside your domain knowledge or you are only learning superficially. Even reading enough to cover a field would take me two weeks, full time, let alone do all the exercises required to truly master it.

My suggestion to you (which I use as a reference point): download a universities postgrad exams for your topic. If you can't answer at least 50% of them correctly, you haven't mastered your subject area.

Replies from: lukeprog
comment by lukeprog · 2011-05-15T03:12:47.177Z · LW(p) · GW(p)

Oh goodness, no. I wasn't claiming mastery of these fields. I was claiming to have understood them well enough to get the information I wanted from them - well enough to have written the (relatively well-researched) posts I linked to in that sentence. Mastery of nearly all fields is not worth the investment of my time. That's what division of labor is for. But I have a much, much better understanding of these fields than reading a few Scientific American and pop-sci books will give someone.

Replies from: autodidaterous
comment by autodidaterous · 2011-05-15T07:00:14.576Z · LW(p) · GW(p)

You may want to clarify that when you say things like:

"As an autodidact who now consumes whole fields of knowledge in mere weeks"

People might take it out of context :)

Division of labor is all well and good, but if you've spent much time around others in a business you soon realise that it isn't all that it's cracked up to be. There's a reason why so many of histories prolific inventors had an enormous array of skills in many different areas: because the only person you can really count on to be there is yourself. Employees and colleagues come and go, the only constant is you.

Replies from: None
comment by [deleted] · 2012-03-03T11:17:47.405Z · LW(p) · GW(p)

"As an autodidact who now consumes whole fields of knowledge in mere weeks"

I also read this in a way that for a whole minute made me despair at my knowledge-acquisition abilities.

comment by InquilineKea · 2011-05-14T23:21:48.145Z · LW(p) · GW(p)

What I do is to try to obtain as much information as possible in written form (this is also why I generally avoid lectures and colloquia unless I feel like they're the only source of information on something). That way, I can refer to it months after I initially read it, and can also search for it (in my hard drive) as well. Furthermore, reading is faster than hearing someone say something, and you can always skip ahead if you want to.

This is also why I generally prefer email to real-life conversation, although I recognize that most professors prefer the latter right now (or they're more responsive if you're talking to them one-on-one).

I also try to post as much online as possible with my unique internet name (with my archiving utilities ready to archive them all) since it makes me easy to google what I've written. It also makes it easier for other people to find me, and sometimes they bring up things that I might have forgotten years ago. I'm not a scholar yet, but I can definitely imagine the potential for crowdsourcing if I become one.


I do appreciate tools like CliffNotes and SparkNotes. Other intelligent people may see those as signals of "low intelligence", but my time is extremely important to me, and if I can learn something faster with those (or with a dummies book), then so be it. In fact, I do believe that a lot of things are deliberately inefficient just to sort out the "highly intelligent" from the "less intelligent", but those things often impose an efficiency cost on the highly intelligent.

What's often important is humility (and a consideration that some people may think you less intelligent). Oftentimes, for the sake of being efficient, you will have to ask stupid questions that are asked (mostly) be less intelligent people.


Also, forum debates are oftentimes useless, but sometimes, they actually give you the drive to explain+defend your position more clearly than you would on your own. Maybe your explanation will be lost on the other side (and on everyone else for that matter), but what ultimately matters is that you made it, and can use it somewhere else.


If you want more time, you can take it out of your sleep with modafinil. But this is risky, and I wouldn't do it more than a few times a year until we know more about the connection between amyloid beta and sleep deprivation and alzheimers (there was a nature article about it last year that I can show later)


I'm not especially organized though, and that's another thing to take into consideration: I rarely delete things, so I have to find creative ways to organize them (certain types of software really help though). But this is something I'm trying to find a solution to. Sometimes I'll just create a folder (or section) that's labeled "DUMP". Sometimes I even put them at the end of my papers or powerpoint presentations. On occasion (probabilities of 1% build up over time), you might actually find a researcher who's actually INTERESTED in what you've put in your DUMP.


Minimizing commute time is also EXTREMELY important (not to mention that commute times intensely increase stress). If you can read on the bus, then take the bus rather than drive (although I've taken the bus before, and I've found transfers to be IMMENSELY stressful since they disrupt my reading because I always have to look at what the bus is doing). Try to live as close to work as possible. There was a period of time when my parents forced me to commute to school rather than to live in the dorms, which wasted a significant amount of time due to the length of my commute (so I ended up only registering for "skippable" classes and often slept overnight in the undergraduate library)


Also, putting less emotional weight on things that don't matter/things that can get others unnecessarily angry. So I put low emotional weight on most political issues of the day (and religion), even though I do have my views of what is "better"/"more effective"


Anyways, if anyone else wants to discuss strategies, feel free to PM me (or email me, or visit my page at This is something I actively think about (as a polymath who is also studying multiple fields, although I'm still younger and have less experience than most of you), and will ultimately write something about.

What i ultimately hope to do is to introduce forums as the primary mechanism of scientific discourse, rather than real-life. There are so many interesting and productive conversations that are now permanently inaccessible because they were communicated verbally, rather than online.

Just note that people sometimes don't like my unorthodox routes (they even offend some people, so trying to avoid offense by ANYTHING is another important part of rationality). For example, I frequently skip class because it's inefficient and prevents me from working on some of my other side projects. I've written more at

Replies from: malcolmocean, lukeprog
comment by MalcolmOcean (malcolmocean) · 2013-07-13T23:52:32.214Z · LW(p) · GW(p)

In fact, I do believe that a lot of things are deliberately inefficient just to sort out the "highly intelligent" from the "less intelligent", but those things often impose an efficiency cost on the highly intelligent.

Huh. I feel like it would be very valuable to create a list of other things that have an isomorphic property to this one, either for intelligence or some other ability. I'm not sure if I can think of any though, and unfortunately I just realized this is a 2yo post that I just saw now due to a recent comment. Oh well.

comment by lukeprog · 2011-10-02T07:05:40.932Z · LW(p) · GW(p)

This is great stuff.

comment by Desrtopa · 2011-05-10T00:26:09.238Z · LW(p) · GW(p)

One thing that I've found quite frustrating in my own experiences with academia is that rather than encouraging efficient scholarship, political expediency often requires one to be inefficient. One of my professors who has a pretty good record for getting students published taught us that it's usually best to try to name drop everyone working in the same narrow field you're trying to publish an article in, both to demonstrate comprehensive familiarity with the literature, and to flatter the egos of the people reviewing your paper, who're likely to be among the specialists in that field.

This generally requires scholarship well beyond the point of diminishing returns for learning useful, relevant information about the topic at hand, and I suspect the effort barrier contributes strongly to the insularity of many sub-fields.

Replies from: byrnema
comment by byrnema · 2011-05-10T01:53:50.649Z · LW(p) · GW(p)

I agree with your professor, it is good politics and also good scholarship to name drop everyone working in the same narrow field as your published article. It shows that you do have a comprehensive familiarity with the literature (and you should) and, valuably, it provides a resource for the next person working on your topic. Finding a 'frame' to introduce each paper can be time consuming, but it is a useful task for understanding how your paper fits in the scheme of things. For this reason, reading a paper without the links is really annoying for someone new to a field. But a paper with well-developed links -- especially a recent review article -- can be the best place to start learning a new topic or to build a citation list from.

Hmm, in such agreement ... do you suppose I might be your professor? (Just kidding.)

Replies from: fiddlemath, Desrtopa
comment by fiddlemath · 2011-05-10T02:08:07.819Z · LW(p) · GW(p)

But a paper with well-developed links -- especially a recent review article -- can be the best place to start learning a new topic or to build a citation list from.

This is actually a pretty frustrating place to start from. Often, the so-built "frame" is setting out to flatter the authors mentioned therein, instead of pointing out what's useful or informative. Moreover, since these sections are more about giving credit and inflating egos than about informing the reader, you're much more likely to see the paper in which an idea was introduced, rather than a more-informative survey paper, written 10 years later, after the important aspects of the concept are really understood.

Replies from: Jordan
comment by Jordan · 2011-05-10T05:05:01.678Z · LW(p) · GW(p)

I lament this state of affairs with the subdued passion of a 1000 brown dwarf suns.

It's ridiculous that wikipedia is more structured and useful that most of the academic literature. I would like to start some kind of academic movement, whereby we reject closed journals, embrace the open source mentality, and collaborate on up-to-date and awesome wikis on every modern research area.

Replies from: wedrifid, David_Gerard, timtyler
comment by wedrifid · 2011-05-10T05:15:52.890Z · LW(p) · GW(p)

I would like to start some kind of academic movement, whereby we reject closed journals, embrace the open source mentality, and collaborate on up-to-date and awesome wikis on every modern research area.

Ok, your next task is to figure out a way to make academics gain status by participation in that plan. :)

Replies from: Jordan
comment by Jordan · 2011-05-10T06:48:36.977Z · LW(p) · GW(p)

That is the heart of the social engineering problem at hand.

Programmers gain status by creating and contributing to open source projects, and by answering questions on StackOverflow, etc. I think that is a stable equilibrium, both for programmers and for academics. The question is how to get to that equilibrium in the first place.

First, I think it needs to become generally accepted that the current equilibrium is broken and that there are alternatives. To that end I encourage all academics to discuss it as openly as possible. Once that happens I think (hope) it will just be a matter of high status individuals throwing their weight around properly.

Replies from: Cayenne, DSimon
comment by Cayenne · 2011-05-10T09:49:12.060Z · LW(p) · GW(p)

An 'open-source science' original-research version of Wikipedia, perhaps? With everything explicitly licensed under an attribution-required copyright?

Edit - please disregard this post

comment by DSimon · 2011-05-12T14:55:23.015Z · LW(p) · GW(p)

Seconding the recommendation of following the Open Source model, particularly Stack Overflow. I'm also a big fan of the many OSS-focused IRC channels, where you'll typically be able to find grouchy-but-helpful people to advise you on the fine points of nearly any nearly piece of software.

comment by David_Gerard · 2011-05-10T09:45:58.028Z · LW(p) · GW(p)

It's ridiculous that wikipedia is more structured and useful that most of the academic literature. I would like to start some kind of academic movement, whereby we reject closed journals, embrace the open source mentality, and collaborate on up-to-date and awesome wikis on every modern research area.

I understand that this is sort of what happens in physics - arXiv preprints (where anything good is expected to be developed into a peer-review-worthy journal article) and a specialist blogosphere. The exchange of prestige and hence the academic credit economy seems to still happen. I suspect the key factor here is arXiv being open-access. So a possible first step is to set up a preprint archive for that field and get the researchers blogging.

Replies from: rhollerith_dot_com
comment by rhollerith_dot_com · 2011-05-10T16:01:53.298Z · LW(p) · GW(p)

So a possible first step is to set up a preprint archive for that field

Arxiv accepts papers in any field. Researchers in medicine, chemistry, etc, just do not use it.

ADDED. Oops, I was wrong. From now on, I'll think more before I hit that "Comment" button. (I still think that setting up a preprint server has already been tried in all academic fields except for those where it was obvious that it would not work. Also, I am pretty sure that tried to extend into computer science but never got a sizeable fraction of the papers in that field.)

Replies from: timtyler, David_Gerard
comment by timtyler · 2011-05-11T10:06:19.541Z · LW(p) · GW(p)

Arxiv accepts papers in any field.

Where are you getting that from? The front page says:

arXiv is an e-print service in the fields of physics, mathematics, non-linear science, computer science, quantitative biology, quantitative finance and statistics.

Also there is this:

While arXiv serves a variety of scientific communities, not all subjects are currently covered.

Replies from: rhollerith_dot_com
comment by rhollerith_dot_com · 2011-05-13T06:10:22.035Z · LW(p) · GW(p)

Point taken.

comment by David_Gerard · 2011-05-11T09:53:58.126Z · LW(p) · GW(p)

OK, the first step is to get them to use it :-) Why does physics do this but not chemistry?

comment by timtyler · 2011-05-11T09:59:42.573Z · LW(p) · GW(p)

It's ridiculous that wikipedia is more structured and useful that most of the academic literature. I would like to start some kind of academic movement, whereby we reject closed journals, embrace the open source mentality, and collaborate on up-to-date and awesome wikis on every modern research area.

That sounds rather like Scholarpedia's plan:

Replies from: Perplexed
comment by Perplexed · 2011-05-11T12:40:23.985Z · LW(p) · GW(p)

... academic movement, whereby we reject closed journals, embrace the open source mentality, and collaborate on up-to-date and awesome wikis on every modern research area.

That sounds rather like Scholarpedia's plan

Not completely. And working through the fine print of my disagreement here helps to show just how rich the field of possibilities is for an alternative to the current system.

In some ways, Scholarpedia is more closed than the current print journal system. After all, anyone can start a journal - there are journals of intelligent design studies, for example. But it probably would not be possible to get Dr. Izhikevich's approval for an encyclopedia of ID under the scholarpedia umbrella, nor to get the curator to allow an ID-promoting article into Scholarpedia's evolution encyclopedia. Scholarpedia promotes open access for readers, but not for authors.

There is also some question of whether Scholarpedia embraces the open source mentality - there is the whole complicated question of derivative works.

Replies from: None, timtyler
comment by [deleted] · 2011-05-11T14:45:49.394Z · LW(p) · GW(p)

One difficulty with having "awesome wikis on every modern research area" (e.g., waffling) is that there just aren't enough people in the intersection of people who are on the frontier of waffling and people who want to contribute to the waffling wiki.

For a more concrete example, the DispersiveWiki basically runs on the fame of Tao alone. In the past thirty days, his was the only non-userpage edit. The Tricki is another example, this time running off of Gowers' fame.

comment by timtyler · 2011-05-11T14:07:00.541Z · LW(p) · GW(p)

Intelligent design seems to have found an online home here.

comment by Desrtopa · 2011-05-10T02:09:17.207Z · LW(p) · GW(p)

You and he may agree less than you think. When pressed, he admitted that this will frequently lead to citing quite a lot of information that isn't particularly relevant to your paper. When you're working in a particular sub-field, it's good to be apprised of the current state of the research, but anyone who simply needs to know the content of your research will end up being bombarded with more information than is actually pertinent. If you want to create a resource for someone who's not already a specialist to become apprised of all the research in your field, you can do a literature review article.

Replies from: billswift
comment by billswift · 2011-05-10T07:44:26.996Z · LW(p) · GW(p)

Go read Robin's posts on academia on OB. Academic publishing isn't about helping others to learn, it's primarily about signalling.

comment by XiXiDu · 2011-05-12T10:08:28.741Z · LW(p) · GW(p)

One might be underestimating the value that video lectures offer to certain people like myself. Reading a textbook demands to be proactive. If you are easily distracted, or don't really enjoy the subject, you have to force yourself to keep reading. In the case of video lectures you only have to bring yourself to start the video. Once the video is playing, your attention is naturally drawn to the ongoing action, whereas text is just inactive and has to be animated actively by the reader. Videos exhibit a tractive force, videos drag you along as they play.

Replies from: knb, Swimmer963, timtyler
comment by knb · 2011-05-14T07:50:03.057Z · LW(p) · GW(p)

Khan Academy really is amazing. I might write a post about how crazy effective Khan academy has been for my pupil. Mostly just by working through the practice exercises and watching the videos my (home-schooled) tutee has gone from struggling with long division to college level calculus in less than 6 months.

Just letting people see their progress (for math exercises) is really very motivating.

Replies from: sketerpot
comment by sketerpot · 2011-05-14T20:40:03.248Z · LW(p) · GW(p)

Just letting people see their progress (for math exercises) is really very motivating.

I'm excited about a related aspect of it, that usually gets overlooked: the Khan Academy guys can get fine-grained statistics over a large population to see what's working, what isn't, and to what degree. And they can vary things, doing A/B tests to figure out what problems or explanations or reward systems are the most useful for which clusters of students.

Looking at the sort of people they're hiring, I'd say this is pretty much inevitable, and I look forward to it.

comment by Swimmer963 · 2011-05-12T10:42:41.161Z · LW(p) · GW(p)

That's interesting. I've never been able to watch video lectures for anything. I think speed is an issue. Reading, I can go at my own (fast) pace. I have a lot of randomly scattered general knowledge from past reading, so in textbooks there are bits and pieces I can skim over if I know I already understand those areas. With videos, the slower pace of speech feels dragging, and I can't tell without actually listening to the whole video whether or not I'm missing something. Videos might be an effective learning method, but for me they're less efficient. (Then again, I don't think distraction or disliking the subject has ever been my biggest problem when reading.)

Replies from: jimrandomh, malcolmocean
comment by jimrandomh · 2011-05-12T14:24:20.199Z · LW(p) · GW(p)

You might want to try watching videos in a player that lets you adjust the speed, like VLC player. You still can't skim video as fast as text, but being able to apply a 2x speedup makes a lot of things much more watchable.

comment by MalcolmOcean (malcolmocean) · 2013-07-13T23:54:35.698Z · LW(p) · GW(p)

I ended up on this old thread, and just wanted to point out that most MOOCs now have players with adjustable speed. Not sure what was available 2 years ago.

comment by timtyler · 2011-05-14T13:48:31.434Z · LW(p) · GW(p)

The benefits of video learning - explained by Mr Khan at TED.

comment by Kaj_Sotala · 2011-05-12T07:34:04.044Z · LW(p) · GW(p)

My main worry about autodidactism is that it seems dangerously easy to mistake a specific, technical term for its everyday meaning and get a twisted understanding of something. Take for instance the subtleties involved in the concept of "heredity", which have at times confused even co-authors of books on the subject:

These issues are pathetically misunderstood by Charles Murray. In a CNN interview reported in The New Republic (January 2, 1995), Murray declared "When I - when we - say 60 percent heritability, it's not 60 percent of the variation. It is 60 percent of the IQ in any given person." Later, he repeated that for the average person, "60 percent of the intelligence comes from heredity" and added that this was true of the "human species," missing the point that heritability makes no sense for an individual and that heritability statistics are population-relative. In a letter to the editor in which Murray complains about being quoted out of context (January 30, 1995), Murray quotes more of what he had said: ". . . your IQ may have been determined overwhelmingly by genes or it may have been - yours personally - or overwhelmingly by environment. That can vary a lot from individual to individual. In the human species as a whole, you have a large genetic component." The Bell Curve itself does not make these embarrassing mistakes. Herrnstein, the late co-author, was a professional on these topics. But the upshot of part of this essay is that the book's main argument depends for some of its persuasive force on a more subtle conflation of heritability and genetic determination. And Murray's confusion serves to underscore just how difficult these concepts can be, even for someone so numerate as Murray.

There are also plenty of other subtle concepts which are easily and frequently misunderstood, even by people with degrees and publications in the field. (Just look at some of the statistics-related criticisms here...) Are there any good ways for an autodidact to avoid making such mistakes?

Replies from: Risto_Saarelma
comment by Risto_Saarelma · 2011-05-12T07:48:24.891Z · LW(p) · GW(p)

There are also plenty of other subtle concepts which are easily and frequently misunderstood, even by people with degrees and publications in the field. (Just look at some of the statistics-related criticisms here...) Are there any good ways for an autodidact to avoid making such mistakes?

Look for informal forums where actual professionals from the field hang out (like computer science forums for example, or blogs of scientists), and try to catch them complaining about people constantly misusing some term?

Replies from: Emile
comment by Emile · 2011-05-12T08:35:40.898Z · LW(p) · GW(p)

Seconded - also, discussing the topic in those forums is a good way to get your wrong notions bashed out of you by genuine philantropists.

If you don't know how to pronounce a word, say it loud!

Replies from: lukeprog, rhollerith_dot_com
comment by lukeprog · 2011-05-14T07:42:34.995Z · LW(p) · GW(p)

Kaj raises a significant problem, and I agree with the advice given by Risto and Emile. You can also contact experts directly, though they are less likely to respond than if you stick your neck out on a professional forum or blog and get corrected. These methods are far more efficient that actually getting a PhD in a subject merely to prevent a few such mistakes.

comment by rhollerith_dot_com · 2011-05-12T08:56:08.394Z · LW(p) · GW(p)

genuine philantropists.

Cute. Playing the midi file there while reading the lyrics is recommended.

Replies from: Emile
comment by Emile · 2011-05-12T15:33:37.796Z · LW(p) · GW(p)

To compliments inflated I've a withering reply;
And vanity I always do my best to mortify;
A charitable action I can skillfully dissect;
And interested motives I'm delighted to detect;

Apparently Gilbert and Sullivan knew Robin Hanson :)

comment by SilasBarta · 2011-05-10T15:02:46.456Z · LW(p) · GW(p)

One thing you should avoid doing is cite an article when you've only read the abstract.

Replies from: nerzhin
comment by nerzhin · 2011-05-10T20:01:53.569Z · LW(p) · GW(p)

Always? Why?

Replies from: SilasBarta
comment by SilasBarta · 2011-05-10T20:09:39.998Z · LW(p) · GW(p)

Because you don't want to pad your bibliography and give the impression you know more about the topic than you really do. Also, the body may not match the abstract, the body may poorly substantiate its claims in the abstract, you should understand any document you're relying on to make your point, etc.

Why is this controversial?

Edit: In fairness, some people do intend "efficient scholarship" to mean "cite any paper with an abstract that looks like it agrees with you and hope no one asks questions", but I don't think that's what lukeprog means.

Replies from: nerzhin
comment by nerzhin · 2011-05-10T20:18:41.403Z · LW(p) · GW(p)

I'm not sure it's controversial, but I disagree very slightly on the margin. All your points are good. However, if

  1. I have already read some papers from an author, and

  2. I trust that their abstracts are honest representations of their work, and

  3. I am not relying on their work as a basis for my own, but just pointing it out to my readers

I will cite it after just reading the abstract.

comment by ChrisHallquist · 2011-11-28T04:17:00.495Z · LW(p) · GW(p)

Long after first seeing this post, I decided to go back and upvote this and related lukeprog articles. The reason is that I've started reading Luke's draft paper The Singularity and Machine Ethics, and I'm sufficiently impressed that I now think Luke may have figured out how to do philosophy correctly. I now encourage everyone to take what he says about philosophy, and scholarship in general, extra-seriously.

comment by BrandonReinhart · 2011-05-10T02:25:08.633Z · LW(p) · GW(p)

Here's a question: does learning to read faster provide a net marginal benefit to the pursuit of scholarship? Are there narrow, focused, and confirmed methods of learning to read faster that yield positive results? This would be beneficial to all, but perhaps moreso to those of us that have full time jobs that are not scholarship.

Replies from: lukeprog, David_Gerard
comment by lukeprog · 2011-05-10T02:36:21.466Z · LW(p) · GW(p)

I've never had success with 'speed reading' in a way that allows me to consume more words per minute and have the same degree of retention and comprehension, especially for dense scholarly material.

Efficient scholarship benefits much more, I think, from learning to be strategic and have good intuitions about what to read - on the level of fields of knowledge, on the level of books and articles, and on the level of paragraphs within books and articles. I've been doing something like what I described in this post for at least two years and I have the impression that this is where I've gained the most utility.

The difference between somebody who is just getting into continuous scholarship and myself is, I suspect, almost entirely to be found in the fact that I can be extremely strategic about which fields of knowledge to consume, which books and articles to consume within those fields, and which paragraphs within those books and articles to consume. That's only what it seems like to me, though.

Genuine 'speed reading' can be achieved with a different brain architecture than I have, of course.

Replies from: BrandonReinhart
comment by BrandonReinhart · 2011-05-10T02:48:27.017Z · LW(p) · GW(p)

Here is another question, regarding the basic methdology of study. When you are reading a scholastic work and you encounter an unfamiliar concept, do you stop to identify the concept or continue but add the concept to a list to be pursued later? In other words, do you queue the concept for later inspection or do you 'step into' the concept for immediate inspection?

I expect the answer to be conditional, but knowing what conditions is useful. I find myself sometimes falling down the rabbit hole of chasing chained concepts. Wikipedia makes this mistake easy.

Replies from: Gray, lukeprog
comment by Gray · 2011-05-11T01:26:59.396Z · LW(p) · GW(p)

Adding to the tangent, in my opinion, the concepts of scholastic philosophy are actually incredibly useful for rationality in general. They usually end up being logic terms, and they are employed well outside of their concept even in modern works. A lot of times, for example, when you read an argument and understand there is something wrong with the argument, but have a hard time putting your finger on what is wrong with the argument, there's typically some scholastic term that will nail it for you. The scholastics were incredibly subtle, and are typically the ones ridiculed when the expression "splitting hairs" comes to fore. But usually that ridicule is made by people who aren't subtle, and don't realize that the distinctions are incredibly important.

comment by lukeprog · 2011-05-10T05:56:58.818Z · LW(p) · GW(p)

It depends on whether the concept appears to be necessary to my understanding of what I care about or not. Sorry I can't give an example right now.

comment by David_Gerard · 2011-05-10T09:36:42.087Z · LW(p) · GW(p)

It helps with skimming material that isn't very dense, which would be approximately none of what this post is about. If it's comprehension you're after, do a skim followed by slow reading. This is work, but is more likely to gain you understanding.

comment by snarles · 2011-05-14T17:34:26.104Z · LW(p) · GW(p)

Anyone who is studying technical material on their own should be aware of the network of sites and These sites are incredibly useful for getting answers for questions which aren't answered in the textbooks. And if you aren't automatically coming up with those kinds of questions regularly while reading the material, you're doing it wrong.

EDIT: Corrected typo.

Replies from: lukeprog
comment by lukeprog · 2011-05-15T03:04:37.073Z · LW(p) · GW(p)

How do you use effectively? It's a for-sale domain, I suspect you may have intended a different URL.

Replies from: JGWeissman
comment by JGWeissman · 2011-05-15T04:32:36.227Z · LW(p) · GW(p)

It looks like Snarles dropped the "c" in "exchange" in the URL, so that is probably why you are getting a for-sale domain. Interestingly, you have typed the URL correctly in your own comment.

Replies from: lukeprog
comment by lukeprog · 2011-05-15T04:38:13.879Z · LW(p) · GW(p)

Lol! You are correct. Thanks.

comment by utilitymonster · 2011-05-10T02:07:57.867Z · LW(p) · GW(p)

How do you record your findings for future use, and how do you make sure you don't forget the important parts?

Replies from: lukeprog, Pablo_Stafforini
comment by lukeprog · 2011-05-10T02:20:28.814Z · LW(p) · GW(p)

My own brain understands what I've learned best when I write up something that looks similar to a post intended for Less Wrong. Other people's brains may work differently. At the very least, I write down sources where all the most important experiments and concepts are explained.

Replies from: David_Gerard
comment by David_Gerard · 2011-05-10T09:37:37.321Z · LW(p) · GW(p)

My own brain understands what I've learned best when I write up something that looks similar to a post intended for Less Wrong.

You mean, you can understand it when you try to explain it to someone else?

Replies from: lukeprog
comment by lukeprog · 2011-05-10T13:46:52.410Z · LW(p) · GW(p)

Certainly. That seems to be common. But also I understand new material better when I rewrite it in my own words even without the intent to share those words with others.

Replies from: Jonathan_Graehl, Gray
comment by Jonathan_Graehl · 2011-05-10T17:04:40.158Z · LW(p) · GW(p)

This is true for me also, and the primary reason I blog. If I weren't doing that, I'd be emailing a friend who'd be interested, or failing that, writing to my diary.

I haven't thought about this before, but it seems like in (pretending to) communicate your assessment of the evidence, that you risk amplifying consistency bias unless you cultivate "I update on new evidence and publicly admit (and diagnose post-mortem) when I was wrong" as a cherished part of your persona.

comment by Gray · 2011-05-11T02:34:28.679Z · LW(p) · GW(p)

I don't know if this makes sense to anyone else, but one thing that I've started that seems to be useful to me is to write down a bunch of notes about the topic before, and while, researching. I think part of this is because I've become used to criticism, and I find I can criticize my own thoughts better after I have written them down, or while I'm writing them down. I just use a blank text editor (using org-mode) for this. It is also helpful dumping whatever preconceptions I might have about the subject matter, before I know what to search for. It also helps me clarify when and where the research is more, or less, superficial than my own understanding. Or maybe the research is only tangential to what I was actually looking for.

comment by Pablo (Pablo_Stafforini) · 2011-05-10T18:58:55.207Z · LW(p) · GW(p)

Summarizing the main ideas of the material you read in your own words, as Luke advises here and elsewhere, does appear to increase retention. After experimenting with several different note-taking software programs, I've reached the conclusion that WikidPad is the best option available. I also try to write one-sentence summaries of every article and book I read, for future reference. Here a citation-management application might help. Of the ones I tried, citeulike strikes me as the most useful.

Replies from: Pablo_Stafforini
comment by Pablo (Pablo_Stafforini) · 2012-02-17T18:11:02.680Z · LW(p) · GW(p)

Update: In case this is of use to anyone, I'm now using Springpad instead of WikidPad, and Diigo instead of citeulike.

comment by recumbent · 2011-10-21T00:50:12.725Z · LW(p) · GW(p)

There are three things I do that save hours a week each, giving me more time for scholarship: 1) voice-recognition software: most people can talk a lot faster than they can type, even including corrections 2) reading while riding a stationary recumbent bike: can transcribe highlighted sections later, or even read from a computer and copy and paste 3) phone headset: do housework or exercises on the phone

I also have a list of tips that save minutes a week each, if people are interested.

Replies from: Regex
comment by Regex · 2015-12-14T22:18:10.615Z · LW(p) · GW(p)

A few years late, but I'm interested!

comment by FloweringThought · 2011-05-15T22:05:48.584Z · LW(p) · GW(p)

I'm a slow reader, so one of the most exciting things about the future of scholarship for me is the huge increase in video lectures on youtube. This is a really exciting time to be an autodidact.

comment by HughRistik · 2011-05-11T00:48:05.362Z · LW(p) · GW(p)

How do you keep track of PDFs of studies?

Replies from: wilkox, AdeleneDawner, lukeprog, lunchbox, TimMc
comment by wilkox · 2011-05-11T01:55:15.198Z · LW(p) · GW(p)

Mendeley is good for this, and specifically designed for managing a library of academic papers. It supports tagging and full text searches, as well as some half-baked "social" features which can be safely ignored. The most useful feature for me is that it can watch a directory for new papers, and add them to its library as well as my directory tree (author/year/paper). It can also maintain a bibtex file for the entire library which is handy for citations.

Replies from: lukeprog
comment by lukeprog · 2011-05-11T02:26:42.645Z · LW(p) · GW(p)

Alas, Mendeley always crashes when I tell it to watch the directory on my NAS for new papers.

comment by AdeleneDawner · 2011-05-11T01:46:20.661Z · LW(p) · GW(p)

Depending on how many you save per month, a premium Evernote might be useful for this - it allows you to organize and search inside PDFs, among other things. A premium account has a limit of 1 Gb of saved stuff a month, but that seems pretty reasonable to me.

(Edit to clarify: There is no limit to the amount of stuff you can save in an evernote account, just a limit to how much you can add to it per month. There does not, unfortunately, seem to be a way to add a backlog of things all at once, though, so it might not be the best option for people with extensive collections already.)

comment by lukeprog · 2011-05-11T01:12:07.995Z · LW(p) · GW(p)

Poorly. I have a NAS that holds tens of thousands of PDFs. They are organized in folders, one for each letter of the alphabet, by last name of the first author. It begins:

AAAI - Interim report August 2009.pdf
Abbey - Charles Taylor.pdf
Abbott - A note on the nature of water.pdf
Abbott - Fodor and Lepore on Meaning Similarity and Compositionality.pdf
Abbott - Realism, model theory, and linguistic semantics.pdf
Abbott - Water = H2O.pdf
Abbruzzese - On using the multiverse to avoid the paradoxes of time travel.pdf
Abdel-Khalek - Happiness, health, and religiosity, significant relations.pdf

comment by lunchbox · 2011-05-14T16:40:59.405Z · LW(p) · GW(p)

CiteULike is quite nice for this.

Connotea is a similar "personal research library" service but it doesn't let you store PDFs, just links to articles.

comment by TimMc · 2011-05-14T13:37:57.465Z · LW(p) · GW(p)

Zotero (a Firefox extension) is fantastic for this.

comment by atucker · 2011-05-09T22:33:40.190Z · LW(p) · GW(p)

Do you have any particular recommendations on how to do this while you still have a day job?

Replies from: lukeprog, CharlesR
comment by lukeprog · 2011-05-09T22:59:47.953Z · LW(p) · GW(p)

I had a full-time regular day job up to March 4th of this year. To have time to do scholarship, it helps to:

  • have a very non-needy and independent significant other
  • have no children
  • no TV, no video games, no movies, no internet wandering
  • eat well and sleep well so that you have energy without needing much sleep
  • be synthesizing your research discoveries in your mind while in the shower, while going to work, while eating
  • be absolutely obsessed with figuring out the world
Replies from: Jordan, None, thomblake
comment by Jordan · 2011-05-10T05:47:03.563Z · LW(p) · GW(p)

Can you comment on what the end goal is for all your scholarship, aside from satisfaction?

Replies from: lukeprog
comment by lukeprog · 2011-05-10T05:59:47.801Z · LW(p) · GW(p)

Right now, solving the 'first stage' of metaethics and then making novel progress on CEV. That's where almost every heavily-cited post I've written on LW has been heading.

Replies from: wallowinmaya
comment by David Althaus (wallowinmaya) · 2011-05-10T15:31:05.687Z · LW(p) · GW(p)

Do you plan to get involved with academia, or to be more general, is it advisable to aim for an academic career? Or do you consider academia to be too costly, ineffective and mainly focused in narrow, obscure or otherwise irrelevant topics?

Replies from: lukeprog
comment by lukeprog · 2011-05-10T21:36:09.495Z · LW(p) · GW(p)

Carl Shulman is a better person to ask this of.

comment by [deleted] · 2011-05-09T23:11:07.534Z · LW(p) · GW(p)

As yet another data point, I have a half-time academic job, and even so I barely have enough time to do scholarship. I differ from the above list in that I do internet-wander and I do sleep poorly.

Hey, I'm working on it.

comment by thomblake · 2011-05-09T23:05:55.776Z · LW(p) · GW(p)

As another data point, I differ from that list in that I have a somewhat-needy significant other, some tv / video games / movies / internet wandering and not enough sleep, and I've had a terrible time doing scholarship over the past several years.

comment by CharlesR · 2011-05-09T22:49:37.471Z · LW(p) · GW(p)

Don't watch television.

Replies from: David_Gerard
comment by David_Gerard · 2011-05-10T09:49:35.058Z · LW(p) · GW(p)

And don't use the Internet as effectively your television.

(I am terrible for doing just this, so don't do as I do.)

Replies from: Laoch
comment by Laoch · 2011-05-10T09:54:45.053Z · LW(p) · GW(p)

A truly addictive medium TV! I must admit also that I too use the internet as such however I think it's a step in the right direction if I use the internet over the TV.

Replies from: David_Gerard
comment by David_Gerard · 2011-05-10T09:59:26.819Z · LW(p) · GW(p)

Yes, you just have to take the other 999 steps as well ;-)

comment by beriukay · 2012-02-22T15:56:14.276Z · LW(p) · GW(p)

You're freaking right about reviews being the holy grail! Every source they cite (at least in the ones I've looked at) has had a downloadable pdf at the top of the google search that mentions the name. Sometimes even with just the year and last names of the people involved.

comment by lukeprog · 2011-11-20T06:36:58.272Z · LW(p) · GW(p)

Those who love independent study may wish to support the kickstarter project Don't Go Back to School.

comment by sriku · 2011-05-11T07:34:07.188Z · LW(p) · GW(p)

Question to lukeprog: Do you have any efficiency recommendations for more technical subjects? Stuff on the lines of Eliezer's quantum physics sequence (aiming more than that, but at least that much). The thing that weighs on my mind most when dealing with such subjects is testing my own competence ... and so it takes me a considerable about of time.

Replies from: lukeprog
comment by lukeprog · 2011-05-11T13:57:19.958Z · LW(p) · GW(p)

To test your own competence, you could make an Anki deck for a subject you just read through, and used spaced repetition to build up a mastery of the material.

Replies from: rhollerith_dot_com
comment by rhollerith_dot_com · 2011-05-11T14:24:35.461Z · LW(p) · GW(p)

Oh, what fun! I get to correct the new guy that everyone admires :)

Do not take the suggestion in parent because it would take time away from the time-tested way of testing one's competence with technical material: doing calculations and proving theorems. About half of the work that goes into the production of a technical textbook should go into the creation of exercises that ask the reader to prove theorems and do calculations. There is in fact a well-regarded series of textbook supplements called Schaum's Outlines that are nothing but exercises.

Although the most effective learners will tend to spend a lot of their learning time proving theorems and doing calculations of their own choosing, it is important for the student of a technical subject to own textbooks with lots of exercises created by masters of the craft because (especially in the beginning) the student will sometimes lack the knowledge and (vitally) the motivation (specifically, the curiosity) required to choose the theorems and calculations from the space of all possible theorems and calculations in the subject.

Replies from: lukeprog, sriku, None
comment by lukeprog · 2011-05-11T14:50:23.103Z · LW(p) · GW(p)

I accept the correction. Anki decks are useful, but for technical material the exercises are probably even better.

But I'm afraid if you want to correct the new guy that everyone 'admires', that line goes around the block. See: the most-upvoted comments on almost all my posts.

Replies from: rhollerith_dot_com, wedrifid
comment by rhollerith_dot_com · 2011-05-13T07:36:18.991Z · LW(p) · GW(p)

I accept the correction. Anki decks are useful, but for technical material the exercises are probably even better.

People with responsibilities may not be able to afford to learn technical material by doing proofs and calculations. When I spend a lot of my energy on proofs or calculations (or complicated computer programs), my ability to stay on top of the rest of my life goes down quite a bit -- and when I stop with the proofs or calculations or technical problem-solving, it seems to take months for my ability to handle the non-technical aspects of life to get back up to where it would have been all along if I hadn't gone into "technical problem-solving mode". (The reverse is also true: if I haven't been in technical problem-solving mode for a while, it takes a few months of adjustment to reach the level of skill I would have been if I did it all the time.)

One of the things I really appreciate about Eliezer's sequences is how much technical knowledge they were able to teach me without my having to disrupt my life by putting myself in "proofs, calculations and computer programs" mode.

If I had not spent a lot of my youth in "proofs and calculations" mode, however, I might not have acquired the technical competence to have be able to identify Eliezer's writings as particularly worth reading relative to all the other technical exposition out there.

Every now and then someone comes on here and makes a big deal of the fact that Eliezer does not have any academic credentials, and it makes me feel sad for them because the reason they focus on credentials and other forms of social proof is probably that they have no better way to judge the quality of his writings. That must be a huge handicap!

So, to bring this back to Anki decks (why not?) the big limitation of Anki decks is that you have to have some way of judging the quality of the text that goes into the making of the decks. Proofs, calculations and computer programs do not have that limitation. (Although it is possible for people to delude themselves into believing that they've produced a valid proof or calculation when they have not, the level of rationality required to avoid such delusions is vastly lower than the level required to avoid learning false information from books.)

comment by wedrifid · 2011-05-11T19:45:54.269Z · LW(p) · GW(p)

that line goes around the block.

Can someone explain this figure of speech for me?

Replies from: AdeleneDawner
comment by AdeleneDawner · 2011-05-11T20:05:55.347Z · LW(p) · GW(p)

You aren't the only one; there are many people already waiting (or attempting) to do that.

Imagine a movie theater or other venue with a very, very long line to get in, such that the line extends all the way to, and around, the nearest street-corner.

comment by sriku · 2011-05-12T00:39:04.284Z · LW(p) · GW(p)

Thanks for validating what I do :)

What tipped off my original question was lukeprog's phrase "... now consumes whole fields of knowledge in mere weeks". I don't think I can manage that kind of speed with technical material! Months (without multiplexing) is more like it for me.

My question stands for anybody who has any tips for optimizing the "solve the exercises" method.

lukeprog: Your Anki tip is not in vain though. Still useful. Thanks.

comment by [deleted] · 2011-05-11T14:39:35.575Z · LW(p) · GW(p)

I completely agree with the parent. I've only been doing research for a couple months now, but I definitely fell into the trap of "well, if I know what's in paper X, then I know what's in paper X." Of course, once I had to use it, well, that was a different story...

comment by XFrequentist · 2011-05-10T14:34:25.553Z · LW(p) · GW(p)

As an autodidact who now consumes whole fields of knowledge in mere weeks, I've developed efficient habits that allow me to research topics quickly. I'll share my research habits with you now.

Mmmm... me want!

(Which is to say: Nice lead-in!)

comment by Feirefiz · 2014-10-08T19:42:47.174Z · LW(p) · GW(p)

While I'm all for spending time and effort on research, I'd like to add to the above post that finding sources that fit one's research topic doesn't add up to good scholarship. It's close reading and critical thinking that do the deal when it comes to evaluating sources. The contribution above lauds the article "The nature of procrastination" (2007) by Piers Steel as a time-saving, thorough review article on the issue of procrastination. That article, however, even though it apparently was written by a published author of a self-help book on procrastination, is a pretty good example for bad, if not spurious, scholarship (and writing).

Look at these examples:

  • "Continued research into procrastination should not be delayed" -- From the abstract -- is this a joke?
  • "Ferrari, Johnson, and McCown's (1995) book on the topic is extensive but focused primarily on measurement and theory, with less emphasis on empirical findings." -- Are measurements not empirical findings too? (p. 65).
  • Under the heading "History of Procrastination" the author mentions several hoax reviews or books on procrastination, including some that were never finished (p. 66). -- Why even include this information in a supposedly scientific article on procrastination?
  • The entire historical section aims to establish the extremely vague claim that procrastination "has long been a prevalent problem" without establishing the meaning or even actual prevalence of procrastination in its historical context. A vague, sweeping historical claim is made on the basis of random text snippets.
  • Steel discusses sources such as Roberts, Chernyshenko, Stark, and Goldberg (2005) to make claims about procrastination (p. 67) even though Roberts et al. does not even mention procrastination. The author's arguments are frequently extremely badly reasoned and not presented in a way that allows readers to actually evaluate the arguments presented.
  • All the effort spent on reviewing a large number of sources amounts to an insight that any freshwo_man could have formulated and that, in its generality and vagueness, is nearly meaningless: that procrastination has always been with us, is a major problem now and will most likely increase -- due to the temptations of modern life such as the internet.

I've picked only a few examples, can't see that the article presents any kind of useful review of the literature on procrastination.

comment by Gray · 2011-05-11T01:43:53.846Z · LW(p) · GW(p)

Great post. With respect to your final statement, I was wondering if using the results of your research to contribute to Wikipedia isn't the obvious thing to do? Not speaking to you specifically, lukeprog, but concerning the general topic.

Also, it would be interesting if your post here inspires other people here to also pursue similar research enterprises, which makes me think that this will involve a lot of duplicated effort. This is against efficiency, which makes me wonder if there's a good place for people to correspond when they are involved in similar research topics. But then I think the answer to this is also Wikipedia.

Replies from: lukeprog
comment by lukeprog · 2011-05-11T04:07:17.474Z · LW(p) · GW(p)

Good suggestion. Added.

comment by Barry_Cotter · 2011-05-10T01:47:24.869Z · LW(p) · GW(p)

One very interesting implication. If this applies to fields as diverse as philosophy, psychology and artificial intelligence it probably applies to any field that is not so diseased that there's no there there. Thus the barriers to going from knowing nothing about a field to being able to write a publishable paper are actually relatively low in quite a few fields, particularly those where you don't need lab equipment or great mathematical sophistication.

Academic paper reading slackers of Lesswrong, to the social science and law paper writing! It'll get you into a good grad school!

Replies from: Vladimir_M, ciphergoth
comment by Vladimir_M · 2011-05-10T07:02:57.374Z · LW(p) · GW(p)

Thus the barriers to going from knowing nothing about a field to being able to write a publishable paper are actually relatively low in quite a few fields, particularly those where you don't need lab equipment or great mathematical sophistication.

I think your impression is wrong. You are right that in many areas, if you're reasonably smart and have a strong amateur interest, it doesn't take very much time and effort to start asking questions and possibly even generating insight at the same level as accredited scholars. However, in such areas, and in many others as well, the most difficult obstacles are of different sorts.

First, a perfectly clear, logical, honest, and readable account of your work is often ipso facto unpublishable: what is required is writing according to unofficial, tacitly acknowledged rules that are extremely hard to figure out on your own. (If anything, academic publishing is so competitive that unless you have an earth-shattering breakthrough, it is difficult or even impossible to publish without intensely optimizing for passing the actual review and editorial process, rather than following some idealistic criteria of quality.)

Second, of course, there is the factor of brand-names, networking, and patronage. Each publication venue has some minimal status threshold for authors and their affiliation, below which your chances of publication are practically nil no matter what the content of your paper may be. (Again, with the possible hypothetical exception of evident stunning breakthroughs.)

Replies from: nerzhin, Barry_Cotter
comment by nerzhin · 2011-05-10T20:08:18.445Z · LW(p) · GW(p)

A perfectly clear, logical, honest, and readable account of your work is often ipso facto unpublishable: what is required is writing according to unofficial, tacitly acknowledged rules that are extremely hard to figure out on your own.

This has not been my experience. My experience with journal editors and reviewers has been that they want a clear and readable account, but it probably varies a great deal from field to field.

Your point about brand names and networks, however, is very well taken.

Replies from: Vladimir_M
comment by Vladimir_M · 2011-05-10T21:31:03.097Z · LW(p) · GW(p)

This has not been my experience. My experience with journal editors and reviewers has been that they want a clear and readable account, but it probably varies a great deal from field to field.

In many fields -- but not all, as you note, and it's hard to speculate on the exact proportion -- there's an evil arms race in fundamentally dishonest self-promotion. Basically, you must employ every imaginable spin short of outright lying to blow up your contribution out of all proportion and minimize the perceived shortcomings of your work. If instead you write up a complete, straightforward, and honest account that will leave the reader informed as accurately as possible, there's no way you're getting published unless it's a very extraordinary breakthrough.

Of course, even in such circumstances, you want your paper to be clear and readable in the sense that the reviewers will read it easily and end up convinced by your claims and impressed by its high-status qualities, without too may unpleasant questions occurring to them. But this is mostly about Dark Arts, not real clarity of exposition.

Replies from: IlyaShpitser, Error
comment by IlyaShpitser · 2011-05-15T05:56:10.311Z · LW(p) · GW(p)

It is true that people in academia, as everywhere else, have a combination of low and high motives for what they do. The question is, on what grounds should one assume an average less-wronger is better in this respect than an average academic?

In my experience the biggest barrier in "mathy" areas is getting up to speed. Getting ideas accepted is a significantly bigger hurdle than getting ideas published. Most stuff that gets published is either not great or not useful (obscure) and often both.

edit: to clarify: I don't see that the way one thinks about spreading lesswrong memes is different from the way one thinks about spreading "what I have published" memes. Both high and low motives are involved in both cases, and suboptimal equilibria are involved in both cases in the sense that one must think of ways of competing with other people's memes.

comment by Error · 2012-09-24T17:13:05.998Z · LW(p) · GW(p)

This sounds an awful lot like the situation in job hunting, at least in IT -- you have to lie as much as you can get away with because if you don't, the position will go to someone who did and you starve. The required level of bullshit increases over time in the manner of a dollar auction.

For professional academics I suppose it is job hunting, in a sense. I wonder if it generalizes to any situation where there's a surplus of competitors for a difficult-to-measure result and opting-out of the competition isn't an option.

comment by Barry_Cotter · 2011-05-10T12:20:48.246Z · LW(p) · GW(p)

I agree. I wasn't saying that anyone will pay attention to what you get published, (but IIRC from a Vassar comment somewhere, most tenured academics outside top departments don't get any attention either.)

I didn't say that you could published in top field journals after six months work, but I suspect that in at least some fields that would be possible, albeit maybe only in low quality journals. Without a doubt you need to conform to the unpublished as well as the public criteria for publication; otherwise you might as well do it as a blogpost. But if you can't figure out those criteria on your own there are still grad students and professors at low prestige universities who might co-author with you if you've got the start of something publishable and are persistent with enough of them.

Do you have any specific evidence on the prestige factor? Double blind peer review would seem to argue against this but then again papers are often refused before reaching this stage as "not suitable for us".

Getting papers published is probably not the most efficient way of spreading knowledge, except in highly technical fields where the criteria are likely to be relatively transparent and high anyway, but it would make getting into a graduate programme substantially easier, and might be worthwhile for other purposes.

Replies from: Vladimir_M
comment by Vladimir_M · 2011-05-10T18:37:15.054Z · LW(p) · GW(p)

Do you have any specific evidence on the prestige factor? Double blind peer review would seem to argue against this but then again papers are often refused before reaching this stage as "not suitable for us".

Well, clearly, I can't give any anecdotal evidence with too much detail in public. I'll just say that "prestige" is probably the most diplomatic term one might choose to use there.

Regarding double-blind review, it has always seemed to me as a farce. Any particular research community is a small world, so how can you possibly be competent to review a paper if you can't guess who the author might be based on the content and the work it builds on? Then, of course, there are the editors, who know everything, whose discretion is large, and who can often drop hints to the reviewers one way or another.

comment by Paul Crowley (ciphergoth) · 2011-05-10T07:21:21.683Z · LW(p) · GW(p)

I think it would take me a very long time to get good enough at category theory or string theory to publish a paper.

Replies from: bentarm
comment by bentarm · 2011-05-10T10:35:56.885Z · LW(p) · GW(p)

This actually seems unlikely - first year PhD students publish papers in string theory all the time. My guess is that it would suffice for you to have the right conversations with the right people.

Whatever the field, there is probably a lot of perfectly do-able research problems that no-one has yet gotten round to doing. The problem being, of course, that it's pretty hard to recognise which these problems are without having the sort of in-depth knowledge that comes from years studying inside the field.

comment by Mikhail Zybin · 2021-03-22T06:27:32.597Z · LW(p) · GW(p)

How do you determine the direction of your research?

comment by jazmt · 2012-11-09T06:57:57.368Z · LW(p) · GW(p)

What suggestions would you have for efficiently learning the modes of thinking from different fields? This seems important since different fields (and even different researchers) look at problems differently (often due to the nature of the subject e.g. physics vs. biology vs. computer science). Some elements of intuition seem universal but others seem to be emphasized in one field or another. While summary articles are good for the content they don't seem suited to learning how experts think about their classes of problems, similarly journal articles often obscure the intuitions and thought process which led to the conclusions by presenting only a polished conclusion. Developing intuition and thinking techniques seems to need seeing actual practitioners writing their thought, have you found good methods of doing so?

comment by Xom · 2011-11-06T23:35:48.542Z · LW(p) · GW(p)

I couldn't find a better place for this, but today I learned this tip:

A book's table of contents shows you its structure, but don't forget to skim the index too, to get a second look at how its content is distributed.

comment by Jabberslythe · 2011-05-16T22:46:23.129Z · LW(p) · GW(p)

What do people here think about using audiobooks for study? I have a job that let's me listen to them all day so I take advantage of that. It is possible to download video lectures as audio and listen to them as well. I also have an Ipod touch which lets me listen to them at x1.5 speed. The biggest problem with them is that it is hard to find good hard academic stuff.

Replies from: Karmakaiser
comment by Karmakaiser · 2011-10-05T18:33:56.712Z · LW(p) · GW(p)

I find that audiobooks are great for absorbing high level concept, or popular level books and it is the preferred method of exploring that level of material. However, I wouldn't study math through an audiobook, or any of the hard sciences in any in depth sort of way.

For instance, through the audiobook method I tried to listen to On Intelligence and while the core ideas of Hierarchical Memory and the Neo-Cortex as a pattern recognition system were easy to follow, at chapter 6, where he went into any high level of detail at all, it was hard to follow. My mind had trouble visualizing the figures and I ended up skipping that section and just resolved to find it in a library or bookstore and read the chapter there.

Personally, I use audiobooks solely for soft sciences like Psychology, fiction, entry level lectures, and popular science books. But I know I wouldn't be able to learn, say Calculus from an audiobook anymore than I could watch the Godfather through the radio. I could get some bits and pieces, but details and rules would be hard to construct and any work I was doing would be sub par. Of course you may have a higher level of retention of crystallization and if you can grok spoken equations then by all means make audiobooks your primary means of study, for me however, they remain a secondary source of information.

comment by Matvey_Ezhov · 2011-05-15T05:35:38.630Z · LW(p) · GW(p)

Q&A services are of a great use for figuring out the terminology. I saw many answered questions of this type on Quora.

comment by Fhyve · 2011-05-15T04:16:06.613Z · LW(p) · GW(p)

reddit/r/scholar looks like it could be a useful resource for scholarly articles. I have not used it (yet) though so I don't know how useful it is.

comment by NancyLebovitz · 2011-05-14T14:24:36.322Z · LW(p) · GW(p)

Heuristics for searching, specifically for history. I'm not sure how much of this generalizes, but it's a look at such questions as whether you've collected enough information, and some emphatic material (mostly in the comments) against using advanced search too early.

Searching for primary literature for use in laboratory write-ups

comment by jschulter · 2011-05-10T22:48:32.514Z · LW(p) · GW(p)

I may be experiencing a fluke, but it appears that my university's library's website allows any computer to use it as proxy for viewing and downloading articles from many paywalled sites (in fact, every site it gives me access to with my student login, which is a very large selection). I only discovered this by accident, and I'm hoping it isn't unintentional on their part. If anybody is interested, the address is here. If you try it and it doesn't work, please tell me.

ETA: It appears that my browser simply cached my login, and that this service is unfortunately not actually available to the general public. Sorry for any confusion.

Replies from: rhollerith_dot_com, badger, sriku, Swimmer963
comment by rhollerith_dot_com · 2011-05-10T23:27:05.105Z · LW(p) · GW(p)

I get, "Enter your NetID and Password." Have you considered the possibility that you have a cookie on your computer that eliminates the need for you to log in?

Replies from: jschulter
comment by jschulter · 2011-05-11T02:35:19.543Z · LW(p) · GW(p)

Okay, so there was some sort of fluke. I checked again and I did have to relogin; I'm guessing it was cached data rather than the cookie which stores my login, because other sites still failed to recognize me. Sorry for getting peoples hopes up,

comment by badger · 2011-05-10T22:54:28.490Z · LW(p) · GW(p)

There must be something else going on. I couldn't replicate at least.

comment by sriku · 2011-05-11T07:38:07.272Z · LW(p) · GW(p)

Universities subscribe to these databases. There is a kind of redirection via a proxy that happens when I'm logged in via my univ's network which allows me to download articles as you mention. I do have to agree to a "I declare I won't violate copyrights" button before proceeding.

Its cool to be in school :)

Replies from: jschulter
comment by jschulter · 2011-05-12T07:54:57.190Z · LW(p) · GW(p)

I'm aware that the proxying service is available while I'm logged in, I was just misled by a cached login to believe that it was working without logging in, which would have been very odd, though beneficial. I've decided to update the comment to reflect that.

comment by Swimmer963 · 2011-05-10T22:54:41.566Z · LW(p) · GW(p)

Based on the policy at my university, don't you need a student number and login to use those services?

comment by Jayson_Virissimo · 2011-05-11T11:03:04.067Z · LW(p) · GW(p)

You can save money by checking Library Genesis and for a PDF copy first...

This article is worth an upvote merely for containing this sentence. How could I not have known about these earlier?

comment by Cayenne · 2011-05-10T09:45:18.598Z · LW(p) · GW(p)

This seems to be focused mostly on scientific research in specific fields. Do you have strategies for learning more general, lower-level skills such as essay-writing, report-writing, math, or programming?

Edit - please disregard this post

Replies from: RobinZ
comment by RobinZ · 2011-05-10T13:21:45.090Z · LW(p) · GW(p)

The remark on textbooks seems relevant - essay-writing, report-writing, math, and programming are all skills which undergraduates are expected to develop.

comment by [deleted] · 2014-06-13T09:57:33.163Z · LW(p) · GW(p)

Studying in any way other than autodidactically is like rehearsing apophenia. Unless you are actively learning in a constructivist manner, which is only sustainable when self directed, you're probably wasting time.

"The fallacy is characterized by a lack of specific hypothesis prior to the gathering of data, or the formulation of a hypothesis only after data has already been gathered and examined.[4] Thus, it typically does not apply if one had an ex ante, or prior, expectation of the particular relationship in question before examining the data. For example one might, prior to examining the information, have in mind a specific physical mechanism implying the particular relationship. One could then use the information to give support or cast doubt on the presence of that mechanism. Alternatively, if additional information can be generated using the same process as the original information, one can use the original information to construct a hypothesis, and then test the hypothesis on the new data. See hypothesis testing. What one cannot do is use the same information to construct and test the same hypothesis (see hypotheses suggested by the data) — to do so would be to commit the Texas sharpshooter fallacy.""