SRS advice
post by Dr_Manhattan · 2011-05-10T20:49:15.839Z · LW · GW · Legacy · 15 commentsI have made some significant progress in organizing myself with org-mode (basically a really well thought out emacs outliner) - consider this a plug :).
Now I think I am ready to bite the bullet and automate another part of my mental apparatus, memorization. I'd like to hear other people's experiences with SRS - spaced repetition - (negative, too), what software they use, what do they use it for, how much time they spend. I expect these to vary, so stating your reasons is worth an extra upvote (and thanks ahead)
ETA: When do you decide something is worth memorizing vs. putting it into a searchable database?
15 comments
Comments sorted by top scores.
comment by [deleted] · 2011-05-10T22:24:01.944Z · LW(p) · GW(p)
(I have a todo somewhere in my org files about writing up all my Anki experience in a nice comprehensive article. One day I'm gonna get to it.)
tl;dr: Use Anki, read its FAQ, keep it simple, always do your reps.
I love my SRS. Without it, I would have failed classes, would never have gotten anywhere with Japanese, could never handle my high book throughput without forgetting them all again. Anki and emacs are easily my two most important tools. I don't consider a topic understood until I have written a short outline in org-mode and put my notes into Anki.
A few general points.
I highly recommend using one big deck for everything (full argument). Keep cards simple and keep all the context you need to answer them on there. A card should essentially ask you for exactly one bit of information - as little as you can get away with. Some redundancy among cards is useful, too. In general, Supermemo's 20 rules of formulating knowledge are key.
Keep your sessions short. I'd recommend timeboxes of maybe 10-15 minutes. If you can spread your reps through your day (by using the Iphone app or whatever), do that. I like doing them on the train.
When in doubt, delete. If a card feels wrong, boring or sucks up your energy, throw it out. If the point is important, suspend it and look it over in a few days. Maybe redo the card. But mostly just throw it out and if it matters, you'll add it back in time anyway. Long-term success is more about managing your energy and motivation than individual cards. If you start dreading your reps, you lost.
Use your SRS for everything you wanna remember. I use it for languages, math problems, important points from books, personal insights, emacs hotkeys and so on. There's really no topic that can't benefit from memorization. (Of course, only remember things you care about. This may change over time, so delete away.)
Also, don't cheat. What I mean is, don't add cards about things you don't really understand or answer them too optimistically (as in "correct" after you took 30 seconds). You'll only screw yourself over. I'm not sure if this is a common problem, but I made that mistake early on, so yeah. Just because you can add hundreds of pre-made cards before properly having studied the topic doesn't mean you should. Your reps should be fast and easy. A few seconds, an effortless answer, next. Anything else gets a "fail" or "delete".
I check my success rate and consistency with daily reps as a sign of health. If I drop below 80% retention rate for normal cards (90% for old ones), then my cards are either boring or I'm adding them too early (i.e. before I really understood them). If I stop doing daily reps, then something is sucking away my energy. I typically fix this by limiting the most draining topic and adding more "fun" cards (mostly questions about books I'm reading or trivia).
Personally, I currently have a deck of about 1500 cards with something like 50-100 reps a day. I delete often and have gone through somewhere between 5-10k cards over the last 2 years or so. Something like <100 reps/day and at most 20 new cards/day is my sweet spot. I can theoretically go up to 1000 reps a day (takes about 2 hours and a few cups of coffee), but then I'm useless for the rest of the day (and enter mania due to the coffee). Intentionally doing less than your maximum so you don't exhaust yourself is crucial.
Should you be interested in languages or list-like material (poems, lyrics, digits of π, ...), then I can add a few specific tips. Generally speaking, Anki's mailing list and plugins address virtually any problem you might encounter.
Replies from: Kaj_Sotala, Goobahman, Antisuji↑ comment by Kaj_Sotala · 2011-05-11T17:32:39.387Z · LW(p) · GW(p)
What I mean is, don't add cards about things you don't really understand or answer them too optimistically (as in "correct" after you took 30 seconds).
When studying for exams, I've had moderate success with adding e.g. algorithms or formulas before I understood them. On the first time when they come up, I don't even try to recall them, but instead look at the answer, try to figure out how it works, and then mark it as failed.
Say it was an algorithm. When the card comes up the next time, during the same day, I let myself mark it as correct if I remember the rough outline of how it worked. The next day, implementation details. I want at least one day where I can pretty much write out the whole thing from memory, even if this took, say, ten minutes. That's to ensure that I've actually understood it and I'm not passing over some fine detail that's trickier than it seems. After that, I mark the card as correct for as long as I can recall the rough outline of its function.
Admittedly, this is rather mentally tiring, and I've ended up deleting several of those cards afterwards. Working this way let me remember them pretty well on exams, however. To some extent, it teaches me to re-derive the algorithms / formulas as needed.
↑ comment by Antisuji · 2011-05-11T06:36:01.736Z · LW(p) · GW(p)
I've been using Anki for about two weeks now for solidifying my knowledge of English vocabulary (yes, it's my first language). I've already noticed a difference in my everyday speech.
I'd be interested in hearing your tips on learning list-like material.
Replies from: None↑ comment by [deleted] · 2011-05-12T18:58:18.076Z · LW(p) · GW(p)
I'd be interested in hearing your tips on learning list-like material.
There are three components that I found crucial to learning lists.
First, add plenty of context before and after the current item. For lyrics, about 2 lines before and after work well. A question looks like this:
They tell us "Autumn's a comin'
and soon everything around us will die
[...]
he is different from the birds in the sky
All those birds go chasin'
(I wrote a simple python script to do this automatically.)
For algorithms or math proofs, I typically put the whole algorithm on the card minus the current step, so something like this:
ElGamal: Alice chooses:
sk = x
pk = g^xBob encrypts:
C = (c1, c2) = (g^y, m * g^(xy))Alice decrypts:
s = [...]
m = c2 / s = m * g^(xy) / g^(xy) = m
Anki's cloze deletion makes this really easy. Just copy the full list, select one item, press F9, card done. Paste the list, select another item, next card done.
When you later try to reproduce the list (say, in an exam), you only need to remember one item. This item will work as your context from which you can retrieve the rest in either direction. This keeps the whole list connected in your memory.
Second, make those items really small. Again, it should contain one bit of information. How large this bit is depends on your proficiency. ;) For new algorithms, I might put each variable or operation in one step, but if it's based on something I already know, I might just put "do a Dijkstra" in one step. You'll know whether your steps are too large if you can't put all of it into your working memory. Don't worry about creating tons of cards. Reviews will be really fast and reinforce each other.
I also break down lists hierarchically if possible. Say you want to learn the Noble Eightfold Path, but all those items sound so similar. So you first group them into "Wisdom", "Conduct" and "Concentration" and make a card for each. Then you break "Wisdom" into "Right Understanding" and "Right Intention", learn that as a 2-item list. That way, each individual list is small and has a maximal contrast between items.
Third, and most importantly, chain backwards. I took this from Don't Shoot the Dog. It only applies if the list has a fixed order.
I'll just quote Karen Pryor on that:
Behavior chains break down and the behavior goes to pieces if there are unlearned behaviors in the chain, or behaviors that have not been brought under stimulus control [i.e. are directly associated with an automatic trigger]. You can't reinforce the subject with a cue if it doesn't recognize the cue, or if it cannot accomplish what the cue indicates. This means that behavior chains should be trained backward. Start with the last behavior in the chain; make sure it has been learned and that the signal to begin it is recognized; then train the next-to-last one, and so on. For example, in memorizing a poem, a piece of music, a speech, or lines in a play, if you divide the task up into, say, five sections, and memorize the sections in reverse order, starting with the last, you will always be going from weakness to strength, from the stuff you're not quite sure of yet into the great, reinforcing, well-memorized stuff you know cold. Memorizing material in the order in which it is written and will be presented necessitates plowing continuously from familiar ground into the more difficult and unknown, a most unreinforcing experience. Treating memorization tasks as behavior chains not only shortens the needed memorizing time considerably, it also makes the whole experience more pleasant.
This really works. If you chain forwards, you always have to walk through steps you already know and when it's time for the new one, you forgot it already. You are constantly unsure what to do next. But going backwards, learning the last step first, means you can directly practice the new step and add it to an established one. You will always know how to go on and can focus on the one new component.
Pryor gives the example of training a dog to fetch a Frisbee:
What people complain of is that their dog, when encouraged, will leap for the Frisbee and try to grab it as it is waved around, but when they throw it, the dog just stands there and watches it go. Or the dog chases and grabs it, but never brings it back. There are two training problems in this game: [...] The second is that the game is a behavior chain: First the dog chases the Frisbee, then the dog catches the Frisbee, then the dog brings the Frisbee back for another throw. So each behavior must be trained separately, and the last behavior in the chain, retrieving, must be trained first. You can teach retrieving over very short distances - indoors, even - with something easy to hold: an old sock, maybe. [...] When the dog will carry things to you on cue and give them up, you shape catching the Frisbee. First you get the dog all excited about the Frisbee, waving it around his face. You let him take it and have him give it back a few times, praising him madly for returning it, of course. Then you hold it in the air, let him have it when he leaps for it, and make him give it back. Then you toss it momentarily into the air and make a big fuss when he catches it. When he has the idea, you can start shaping the first behavior of the chain, the chase, by tossing the Frisbee up and out from you a few feet so the dog has to move off after it, to catch it. And now you are on your way to having a great Frisbee dog.
This is a general rule: learn the behavior first, then add the cue. Otherwise you won't associate cue and behavior (because you don't know what to do) and won't get anywhere.
Of course, your Anki cards will eventually become randomized in your reviews, so this applies only to adding new ones. It helps to occasionally just recite the whole list from scratch, too.
(Oh, and should you want to learn π, group digits into n-tuples (3<=n<=5 is typical) and assign a vivid mental image to each tuple. Then chain them as a simple story. And if someone asks you for π, stop reciting after 4 digits or everyone will know what a giant nerd you are.)
Replies from: arundelo↑ comment by arundelo · 2011-12-30T15:03:08.229Z · LW(p) · GW(p)
- John Conway on memorizing pi.
- A Toothpaste for Dinner comic on how many digits of pi people have memorized.
[From great-grandparent]: I have a todo somewhere in my org files about writing up all my Anki experience in a nice comprehensive article. One day I'm gonna get to it.
That would be nice. I liked these comments a lot.
Replies from: None↑ comment by [deleted] · 2011-12-30T15:37:34.221Z · LW(p) · GW(p)
Gwern wrote the article I wanted to write. :) Everything worthwhile he doesn't talk about I either already mentioned here or it's very specific to a given purpose, like the use of morphological analysis to sort language cards.
Replies from: arundelo↑ comment by arundelo · 2011-12-30T17:12:09.263Z · LW(p) · GW(p)
Thanks. Gwern's piece was a huge influence on me finally getting around to using spaced repetition. Also influential: various people on LW, including you, but a particular shout-out to a couple lukeprog posts that, while not directly relevant to my hardware, made me realize how quickly I could take the first step.
Now I am better at doing arithmetic in my head and I know my immediate family's birthdays! (And more.)
comment by Kaj_Sotala · 2011-05-11T17:40:50.044Z · LW(p) · GW(p)
Anki is great for learning conceptually complicated material, because such material tends to be composed of a large number of elements that build on each other. For the more complicated parts to make sense, you need to remember and comprehend all of the less complicated parts that it builds on. Spaced repetition helps ensure you really do remember all of them.
I also find that memorizing various formulas is actually very useful for one's comprehension. In order to remember a large number of them, you have to think about their contents: "I think the formula went... no, wait, that doesn't make sense, so what would?" In effect, you learn to quickly rederive the formulas each time you're prompted about them, until finally you know them so well that you don't need to think about them.
Doing it this way, you also start to notice when you don't really understand how a formula works. The formula looks like this, but why? You start to think about it on and off over a span of several days, or you might go looking for the answer. Some of the formulas used in the statistics course I did involved using concepts which were never really explained within the course itself, which started bothering me. So I looked them up on Wikipedia, and after having looked up those concepts, made new cards about them. This gave me a broader understanding about the topic than I'd have gotten from the course itself.
(Some of those new cards I made involved getting to the old to-be-memorized formula from the new knowledge I'd picked up. For instance, I edited a "the formula for judging a hypothesis about the variance of a normally distributed variable" card to a "from the knowledge that the chi squared distribution is a distribution of the sum of squares of n normally distributed variables, derive the formula for judging a hypothesis about the variance of a normally distributed variable" card.)
I also find this to be more enjoyable than the traditional process. Previously, if I was reading some conceptually advanced text, I had to spend a lot of effort into both understanding it and remembering it. Now it's enough that I understand it well enough to make Anki cards out of it. On this "knowledge extraction pass", I don't need to worry too much about whether or not I understand some particular formula or algorithm. That will come later. I just enter it into a card, though frequently I do need to think about it somewhat in order to make sure that the answers and questions I'm typing make sense. Then when I'm actually memorizing the cards, it doesn't feel like it took a lot effor either, because the knowledge comes in small-sized chunks (even if they actually do connect to a lot of other information). Somehow this tricks my brain into learning vast amounts of information without it ever seeming like work.
Also, learning more and more cards triggers in me the same kinds of reward mechanisms that are activated by gathering experience and leveling up in video games, which helps make it feel more enjoyable. On some occasions, I've been known to go through a day's cards, be disappointed that I ran out of them, and then work ahead in a book just so I could make myself new ones to memorize. Partially because of this, I've picked up a habit of entering the content of any non-fiction book I read into Anki cards.
comment by gwern · 2011-05-10T22:58:01.327Z · LW(p) · GW(p)
Pretty much everything I have to say is in EDIT http://www.gwern.net/Spaced%20repetition
Replies from: MartinB↑ comment by MartinB · 2012-09-27T05:36:04.240Z · LW(p) · GW(p)
Link unavailable, new one seems to be: http://www.gwern.net/Spaced%20repetition
comment by hairyfigment · 2011-05-11T01:30:09.508Z · LW(p) · GW(p)
Replies from: Perplexed↑ comment by Perplexed · 2011-05-11T12:58:52.885Z · LW(p) · GW(p)
Thank you for that. But would you consider the experience positive or negative? :)
It always annoys me when people use acronyms or abbreviations with which I am not familiar. In this case, though, using the LW search gizmo on "SRS" led me quickly to the apparently intended meaning. That doesn't always happen - "CEV" is a particularly bad example.
comment by timtyler · 2011-05-10T20:59:19.207Z · LW(p) · GW(p)
Spaced Repetition Software...?
Doesn't "automating memory" mean using the memory of a computer? ;-)
comment by jsalvatier · 2011-05-10T21:07:05.094Z · LW(p) · GW(p)
Luke Stebbing convinced me to try Anki and it seems pretty useful and required little effort to learn (videos were a good intro).