Software tools for efficient scholarship
post by lukeprog · 2011-09-15T00:17:02.255Z · LW · GW · Legacy · 16 commentsI've figured out a few things about efficient scholarship, but my process isn't optimal. In particular, I'm probably not using the best software tools. I and probably many Less Wrongers could benefit from using the best scholarship software around.
Please post your scholarship software recommendations below!
In particular, the thing I'm looking for is a good citation manager (for Mac): one that will scan all the PDFs on my NAS (network attached storage), query web databases to fill out their metadata in a searchable database, and allow me to export bibliographic records in a variety of formats. Here are the failure modes of the ones I've tried so far:
- Zotero: When I try to import any of the 80,000 pdfs on my NAS so that it can check internet databases for their metadata, it makes a local copy of each PDF even though I have "automatically attach PDFs..." unchecked in the preferences window. Also, the program crashes when I try to import more than about 10 PDFs at a time, even though my new Macbook Air is lightning fast doing everything else.
- Mendeley: As far as I can tell, Mendeley cannot use a NAS drive as a 'watched folder'. Maybe there's a hack for this?
- Sente: Cannot batch-import without prompting for interaction for almost every PDF imported.
- BibDesk: Does not import PDFs.
- Bookends: Cannot batch-import dozens of PDFs quickly.
- RefDB: No Mac binary, not sure if it can import PDFs.
Garr.
16 comments
Comments sorted by top scores.
comment by gwern · 2011-09-15T14:13:11.793Z · LW(p) · GW(p)
it makes a local copy of each PDF even though I have "automatically attach PDFs..." unchecked in the preferences window. Also, the program crashes when I try to import more than about 10 PDFs at a time, even though my new Macbook Air is lightning fast doing everything else.
This is probably pointless to ask, but you did file bugs yes?
comment by jsalvatier · 2011-09-15T00:36:26.243Z · LW(p) · GW(p)
For those of us that have < .5 Tb of papers, which tool have you liked the best?
Replies from: lukeprogcomment by Richard_Kennaway · 2011-09-15T13:13:54.746Z · LW(p) · GW(p)
What are these tools for? I haven't been able to get much of a sense of that from their web sites.
My collection of papers amounts to about 1600 in 2.4GB, and I keep track of them by putting them roughly into folders for different topics, and maintaining a plain text file of minimal bibliographic details -- filename, title, URL, authors, and whatever notes I've made about them, sorted in order of addition to the collection, most recent first. Text search takes care of finding stuff. I use git to synchronise the library among a few different computers. I don't seem to have any need that any of these tools would be an answer to.
But perhaps there is a need that I do not know I have. If you were an honest salesman for one of these tools, how would you sell it to me?
comment by jsalvatier · 2011-09-15T00:29:08.716Z · LW(p) · GW(p)
I'm computer literate but I had to look up NAS (network attached storage).
Replies from: lukeprogcomment by Metus · 2011-09-15T18:39:18.755Z · LW(p) · GW(p)
gPapers ( http://gpapers.org/ ) is currently being developed. Maybe it can serve your purposes? Sadly, it is in an alpha state.
comment by Morendil · 2011-09-15T07:06:40.547Z · LW(p) · GW(p)
I'm using BibDesk. I tried Mendeley but came away unconvinced.
I've found BibDesk's handling of PDFs and other "attached documents" quite adequate, other than importing.
I solved the "import" issue in a different way than you seem to be approaching it. I don't know that any existing tool does this "import metadata" really well, so what I did was Google the title of my papers, then steal the .bib entry for that paper from ACM or IEEE or whatever repository had the paper on file. This is manual and time-consuming but leads to better data quality.
Replies from: lukeprog↑ comment by lukeprog · 2011-09-15T07:50:59.149Z · LW(p) · GW(p)
Yeah, right now I'm using Zotero basically streamline that manual process.
Replies from: Morendil, Morendil, Morendil↑ comment by Morendil · 2012-02-11T14:45:42.440Z · LW(p) · GW(p)
I've tried a recent version of Sente, and it does a really nice job of streamlining the process of importing PDFs into bib entries, but it's still a manual, one-by-one process. I was very happy to get through about 50 unsorted papers in, I'd say, a quarter hour. For my needs that may be sufficient.
Have you found anything able to digest your 80K papers?
Replies from: lukeprog↑ comment by Morendil · 2011-09-15T07:59:35.462Z · LW(p) · GW(p)
Interesting, I'll give it a try.
ETA: now that I see it, I remember trying it back when I was still using Firefox. Have you been using the new standalone version? It's possible that this will be an improvement over the FF version - I switched to Chrome a while back finding FF just too damn slow and resource-hogging.
Replies from: lukeprogcomment by XFrequentist · 2011-09-15T04:41:51.687Z · LW(p) · GW(p)
Peaya paper is pretty good, comparable to Mendeley but without the social network business. No idea about Mac or server issues, sorry.
Replies from: lukeprog