Learning as closing feedback loops

post by ambigram · 2022-04-17T09:50:57.804Z · LW · GW · 0 comments

Contents

  Framework
    Make an attempt
    Observe the feedback
    Reflect & hypothesize
  Ideas on how to learn better
    Increase the number of attempts
    Convert existing experiences to closed feedback loops
    Get better at interpreting feedback
    Find ways to get better feedback
    Reflect on how you would do things differently
  Other thoughts
None
No comments

Epistemic status: Just a pattern I've observed from my own learning experiences and advice I've encountered about learning. I am not qualified and the essay is not properly justified - it's meant to be more like a source of ideas on things to try^[1]. I would be happy to work on this further if there is interest.

When I think about my own learning experiences and the various studying techniques, advice on how to learn better, or methods to improve faster at skills, I observe some commonalities - it seems like most advice can be interpreted as ways to improve our feedback loops in learning^[2].

In this essay, I will be describing the key steps in a learning feedback loop and the ideal conditions and common challenges for each step. I will then list the different ways of improving learning that are suggested by this framework, and provide examples of how to apply them. Finally, I will end with some rambly questions and thoughts.

Framework

I like to think of learning^[3] as closing feedback loops, because it helps contextualize the different advice and techniques, and also helps me identify bottlenecks in my learning.

Here's the learning feedback loop: 

 

 

1. Make an attempt: try something based on a goal (e.g. practice a song, with the goal of playing it well)
2. Observe the feedback: observe how your experience and the outcomes differ from what you desired, to see if you are getting better or worse (e.g. compare your playing with an expert's rendition of the song, or listen to your teacher's response to your playing to see how well you did)
3. Reflect & hypothesize: analyze your experience and generate hypotheses to explain why your attempt was better or worse, and what you can do differently next time (e.g. you observe that the expert's rendition uses a wider range of dynamics, and conclude that varying the loudness/softness would make your version more interesting)
4. Go back to Step 1, making a new attempt that tests your hypotheses
    

Examples of not-learning

• Mindlessly playing through a song 100 times, because your teacher told you to practice
• Reading through practice questions, answering them in your head briefly in vague terms (e.g. to answer this question, I will apply concept X), then reading the answers immediately, concluding that you have understood the topic because the provided answer (which is much more detailed) matches your answer
• Observing that you've allocated too little time for your essay when you're up at 3am the day before the deadline (like always), but still using the same method of time estimation the next time you have an essay due

Make an attempt

An experience counts as an attempt when we see that our experience was just one possibility, and that we could have chosen to act differently.

Ideally, we want to be able to make as many attempts as possible, so we have more opportunities to iterate and improve. However this is not always possible:

• Attempts may be expensive e.g. you have limited funds for starting a new company, or it may be very difficult to switch career tracks.
• You only have one chance e.g. you cannot change your mind about having kids and then decide to return your child.
• Each attempt can take a long time, so you have limited tries within your lifetime. 

Observe the feedback

After making an attempt, we observe to see if we are on the right track. We want feedback so we can tell if we are getting better or worse.

Ideal feedback is

• clear, easy to interpret: it is easy to tell if you are moving in the right direction (e.g. being given a score of 7/10 by judges is easier to interpret than hearing applause from an audience),
• immediate, so you can easily relate the feedback to the attempt, and so that you can complete more feedback loops (since each loop takes less time),
• reliable (signal, not noise): consistent and accurate (e.g. a student's grades for each essay should be strongly correlated with the quality of the submitted essay),
• informative: it is easy to tell what you should work on next (e.g. "Good diction, but a little too fast." is clearer than "Not bad").

For simple tasks, the outcome is the feedback. You can tell if you are getting better or worse (and maybe even figure out why) just by looking at the outcome. For example, if you are working on your spelling, you can run your answers through a spellchecker or look up a dictionary to see if your spelling is correct. You can also easily tell which words you frequently misspell and what types of mistakes are more common.

However, most of the time, the outcome doesn't directly show if you are on the right track. In such cases, being able to interpret the feedback becomes very important.

 Here are some reasons why feedback may be hard to interpret:

• High dimensional reality: There are many possible factors affecting the outcome, so it can be difficult to figure out which factors are relevant and how they affect the outcome. Without an expert's intuition, it can be hard to tell what information to pay attention to, e.g.
  • A teacher tries giving more frequent feedback/comments when teaching a class. Many students score better on the next quiz. Is this because giving frequent feedback helps them learn better?  Or because the topic is easier? Or because some students attended a workshop which taught them better study techniques? Or is it just variance in the data?
• Unknown precision^[4]: data is noisy, so you need to use the right level of precision (e.g. if you can jump 0.1 cm farther after changing your technique, it doesn't mean the technique helped improve your jump). However, we may not know the right precision to use, e.g.
  • Your student performs worse after being praised. Does this mean praise makes them perform worse? Maybe we should try looking at their performance for the next 10 attempts? Or maybe their performance at the next quiz?
• Ambiguous timelines: Actions can have long-term consequences, so you may not know the true costs or benefits, e.g.
  • You feel more lethargic after taking a new pill regularly. Is this just a transition period, or does it actually make you worse? How long should you wait?

Reflect & hypothesize

For feedback to be useful, we need to convert it into actionable insights, so we can apply and test it. Ideally,

• our conclusions are meaningful and lead us towards exploring more relevant areas of the solution space,
• our insights change how we actually do things (i.e. we apply our lessons learned),
• insights are (suitably) generalized such that we can benefit from the lessons learned even in other areas.

Potential difficulties:

• It can be difficult to generate ideas on things to try next when we are beginners or when we hit a plateau.
• Different people can come to different conclusions given the same experience, and some of them will have more accurate conclusions than others, but I don't really know why.

Note that this step isn't always necessary or relevant. For example, after doing Feldenkrais exercises, my posture tends to be better simply because I am more aware of unnecessary tension. I don't think about sitting straight or correcting my posture - my body automatically sits up properly when I slump because the wrong posture feels unpleasant.  Drawing my attention to the contrast between my expectations and reality also seems sufficient for changing my emotional beliefs.

Ideas on how to learn better

The suggestions are mostly taken from elsewhere (e.g. books, articles, advice). They are generally things that I think do work, either because I have tried it or something similar before, or because others have suggested it and it seems to make sense. However, I'm not much good at implementing them consistently. 

Increase the number of attempts

• Practice more (up to a limit): someone who practices daily will improve faster than someone who practices once a week, ceteris paribus. (see also: deliberate practice [? · GW])
• Quantity over quality^[5]: e.g. take more photographs rather than spending a long time making sure each shot is set up perfectly.
• Make attempts cheaper, so you can do more of them: e.g. paper prototyping instead of coding an application, learning via simulations instead of just real-world practice, rehearsing a dance mentally rather than dancing it out.
• Establishing safety, so you are more willing or able to try, e.g.
  • Learning in a class with encouraging teachers and classmates
  • Learning in an environment that tolerates mistakes appropriate for your current level (e.g. letting a toddler explore a playpen, whereas you might let a child explore a safe neighborhood)
• Attempts don't have to be your own: read about how other people have tried approaching the same problem, or get your friends to also try the technique.
• Work at multiple levels at the same time, e.g. pay attention to both the contents of a lecture and how the professor is teaching, so that you can learn about the topic and how to give clear explanations, structure a lecture, manage a class etc.
• When you few opportunities to try, focus on the skill instead, e.g. instead of trying to get better at choosing a life partner, focus on getting better at making good decisions, reading people, understanding yourself, improving your model of relationships etc. 
• Pausing to wonder if there other ways of doing things, so it turns a habit into an experiment?

Convert existing experiences to closed feedback loops

• Reflect on your experiences and act on the lessons learned, such as by keeping a learning journal and reviewing it periodically.
• Record your attempts and outcomes, so you can turn them into feedback by evaluating and making comparisons. This also means you will be able to revisit your past experiences to see if you can learn from them e.g. after you encounter a new concept that changes the way you interpret your past experiences.
  • Examples:
    • Record yourself speaking, so you can observe how well you are enunciating your words, varying your pitch etc.
    • Keep track of your predictions^[6] to measure how well-calibrated you are.
    • Decision log: record the context, your thought processes, assumptions, intent etc. when making a decision, so that you can look back to evaluate your decision-making process after the decision plays out. (e.g. pre-mortems)
• Notice [? · GW] your attempts or available feedback:
  • An experience becomes an attempt when you recognize that your action was a choice. Think about how you could have done things differently, or how others have done things differently, as well as the consequences.
  • Paying attention to your mood, energy levels etc. can help you spot new patterns. For example, you may realize that talking to people or going on walks energize you.
• Make predictions [LW · GW] beforehand^[7]. (Predictions can also help you notice when things are going wrong e.g. there is a misunderstanding, or there's something different about this situation compared with what you typically deal with.)
  • Examples:
    • Test yourself when studying for exams, rather than just reading through your notes.
    • Guess the output of your code before running the program, instead of relying on the compiler or tests to catch errors. 
    • Estimate amount of time required for completing your work (or even how much time you think others will take to complete their work), even when it's not required by your boss/client etc.
    • Predict how the different stakeholders will respond to your proposal (e.g. anticipate criticisms of your idea), so you get better at modeling and managing stakeholders.
    • When someone senior is working on a problem, think about how you would approach the problem, so you can later compare your approach with how your senior solved it. It does not have to be the solution itself -- it can be things like how you would approach the problem (e.g. I would look at X and Y to see if Z).
  • If you can't predict beforehand, you can do it post hoc, e.g. think about how you would do things differently when you're listening to a presentation, reading an article, watching someone make conversation etc. The problem with this approach is that you might be influenced by the "answer".
  • Caveat: Make predictions with the goal of proving yourself wrong, otherwise you run the risk of confirmation bias [? · GW]. Remember that we learn when we are surprised, and the goal here is to learn.

Get better at interpreting feedback

• Focus on identifying factors to pay attention to in a domain (i.e. finding the gears-level model [? · GW]?):
  • Consult existing research, people with extensive experience etc. and see how they model the domain e.g. concept of emotional bids in relationships
  • Listen to critiques by experts, and pay attention to how they think: What factors do they talk about? Are there any patterns in their comments? What do they not talk about? Why do they choose to focus on certain aspects but not others? e.g. 
    • Listen when your teacher is giving feedback to your classmates
    • Watch coding livestreams^[8]
  • Find key factors by looking for examples that are as similar as possible and trying to spot the differences and look for patterns: e.g. 
    • Look at how different writers retell a specific fairy tale and find potential reasons why some approaches work better than others, rather than looking at generic short stories by different authors. It probably works even better if you can get samples from writers who are at the same standard, rather than say, comparing works by an amateur vs experienced writer.
  • Vary your attempts as much as possible, or look at diverse examples, so that real patterns stand out more clearly, e.g. if you experience working with many different types of people, you can better identify behaviors or traits that strongly correlate with better teamwork.
• Find ways to get a broader view: sometimes we miss a pattern because it requires a factor that we didn't keep track of.
  • Examples:
    • Learn from different domains so you have more ideas on things to try
    • Practice creativity, like trying to generate 10 ideas a day?
    • Observe what experts pay attention to - are they looking at something that you don't? 
• Cater for variability in outcomes:
  • Wait for more data before testing a new hypothesis: e.g. observe the consistency of your performance, instead of just the accuracy of your next throw.
  • Consult people who have relevant experience on suitable time scales to use, since they have more knowledge on how much outcomes usually vary.
  • Look for existing data to see how outcomes usually vary.
  • Let multiple people test out a hypotheses, so you can see how results vary.
• Keep your conclusions tentative, and update them periodically over time, e.g.
  • Periodically review past outcomes, such as by re-reading your journals

Find ways to get better feedback

• Look for clearer feedback:
  • Quantify e.g. mood tracking, recording number of pages read, number of hours of deep work (but beware Goodhart's Law [? · GW])
  • Make your attempt as similar as possible to the reference (or look for a reference as similar as possible to what you are attempting?), e.g.
    • Try to recreate a specific piece by a good writer [LW(p) · GW(p)], so you can compare your attempt with the author's attempt, rather than writing a piece then having to figure out what worked and what didn't.
    • Instead of trying to compare your drawing of a face with the actual face to figure out how to fix your drawing, just overlay your drawing on a reference picture so that differences are obvious.
  • Be specific when making your attempts, so it is easy to notice if you're wrong/surprised, e.g.
    • "I predict that X will increase slightly because of Y and Z may happen but it won't affect X." instead of "I predict X will increase."
    • Record what you were observing, feeling, thinking so you can better spot potential reasons why things went wrong or went right.
  • Set specific goals for each attempt, so it is easier to tell if you are improving or getting worse, e.g.
    • "My goal for this writing exercise is to practice being more concise." instead of "My goal for this writing exercise is to practice writing better." (Note that your ability to identify specific goals depends on your expertise -- you develop a clearer idea of things to work on as you gain more experience. You should be refactoring your goals as you learn.)
• Find quicker sources of feedback (i.e. tighten feedback loops):
  • Ask people for feedback, rather than waiting for performance reviews.
  • Use tools to help [LW · GW] e.g. apps to evaluate if you are singing on pitch.
  • Learn to evaluate your own attempts rather than just relying on the teacher or on outcomes, e.g.
    • Learn cues you can use to check your own attempts (i.e. how does the teacher notice a mistake), so you can correct yourself when you're practicing at home instead of waiting for the teacher's feedback at weekly classes.
    • Develop good taste so you can tell if your work is getting better or worse^[9], e.g. read a lot (of both good and bad^[10] examples of writing) to get better at identifying good writing.
    • Hone your proprioception: pay attention to how an action feels within your body when it's done correctly vs wrongly (e.g. how your weight is distributed across your feet, which muscles are engaged), so you get instantaneous feedback instead of having to rely on the mirror, video recording, or your someone else's feedback.^[11]

Reflect on how you would do things differently

• Get feedback from people who have domain expertise (so their models are more accurate) and are introspective: Being good at a skill is different from knowing how to teach it. For example, if a skill comes very naturally to them and they have not needed to teach it to other people, then they may not really know why they're good at it (compared to someone who had to try different methods to become good at the skill), so their feedback tends to be less concrete and less useful.
• Relate to your other experiences to look for patterns: is your experience similar to past experiences or things you have read? How is it similar? What was unexpected about your experience? e.g.
  • When hearing a piece of advice, think about related advice and experiences. Does this advice conflict with what you know? If so, is it because they're intended for different contexts? Or maybe they have different goals?
  • Looking at your current and past attempts at conflict resolution, you notice that things seem to work out better when people are calm, so you hypothesize that "people tend to be more receptive when they are given time to calm down".
• Make insights concrete by thinking about how you would do things differently next time, e.g.
  • Plan to do a timeout next time there's an argument, so that both parties have time to calm down.
  • Instead of just stopping at the conclusion that you tend to underestimate time needed for your work, plan to double your time estimates the next time you have a project.

---

Other thoughts

• I don't have much experience with the humanities (my experience is mostly in more clear-cut/objective environments like programming). Does this framework still apply in subjects like History, where you can only observe, not act? Or how do people gain expertise in areas like public policy, where you don't really get to experiment as an individual, the way a programmer can just write programs?
• I usually try to define key terms in essays, but I don't how to define "learning". 
  • I was clearly learning things even as a child. Yet, the quality of learning felt very different before and after I realized how to do reflections. What's the difference?
  • It feels different from practicing, maybe because there's an element of discovery to it?
• I think these are some of things I had in the back of my mind while writing the essay: kind vs wicked learning environments, tacit knowledge (I am not really familiar with these though). 
• Is this framework actually useful or is it just an interesting pattern? Are there other theories/models are more useful?
• Is there better (existing?) terminology for the different concepts discussed?
• Possible directions to explore:
  • Refining the essay: adding pictures, refining examples, improving clarity etc.
  • Looking for relevant research (e.g. pedagogy, learning theories) and writing a summary
  • Elaborating on specific points as a separate essay
  • Writing essays based on use cases, e.g. how to learn better as an autodidact, how to learn an unexplored domain, how to provide better feedback
  • Related topics like evaluating your own level of mastery, practicing effectively

1. ^{^}

   This essay is an attempt at How To Write Quickly While Maintaining Epistemic Rigor [LW · GW].

2. ^{^}

   I'm not sure if there is a point in writing this essay, because the concept of learning as closing feedback loops sounds ...obvious, e.g. reinforcement learning, or advice like "tighten feedback loops". It wasn't obvious to me though.

3. ^{^}

   Tentatively defined as the process where data/experience/information changes the way you interact with reality?

4. ^{^}

   Inspired by Beware of Tight Feedback Loops by Brian Lui (which I disagree with in some aspects) and the corresponding Hacker News discussion.

5. ^{^}

   Parable about quantity vs quality for students learning ceramics, from the book "Art & Fear" by David Bayles and Ted Orland:

   [A] ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. 

   His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: fifty pound of pots rated an “A”, forty pounds a “B”, and so on. Those being graded on “quality”, however, needed to produce only one pot — albeit a perfect one — to get an “A”. 

   Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work – and learning from their mistakes — the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.

   https://austinkleon.com/2020/12/10/quantity-leads-to-quality-the-origin-of-a-parable/ 

6. ^{^}

   For example, https://predictionbook.com/ lets you record and track your predictions

7. ^{^}

   Inspired by a story about how experienced neonatal nurses and firefighters notice when something's wrong because what they observe doesn't match what they expect. I think it's from the book "The Power of Intuition" by Gary Klein, but I don't really remember.

8. ^{^}

   Suggestion taken from How to Use YouTube to Learn Tacit Knowledge by Cedric Chin. 

9. ^{^}

   This is a very important focus for me when I'm learning, but Ira Glass' quote on the taste gap suggests that some people start out with good taste so your experience may vary. Personally, I develop taste only as I learn, so I am often blissfully ignorant about how terrible I am in the beginning. 

   Nobody tells people who are beginners — and I really wish somebody had told this to me — is that all of us who do creative work … we get into it because we have good taste. But it’s like there’s a gap, that for the first couple years that you’re making stuff, what you’re making isn’t so good, OK? It’s not that great. It’s really not that great. It’s trying to be good, it has ambition to be good, but it’s not quite that good. But your taste — the thing that got you into the game — your taste is still killer, and your taste is good enough that you can tell that what you’re making is kind of a disappointment to you, you know what I mean?

   A lot of people never get past that phase. A lot of people at that point, they quit. And the thing I would just like say to you with all my heart is that most everybody I know who does interesting creative work, they went through a phase of years where they had really good taste and they could tell what they were making wasn’t as good as they wanted it to be — they knew it fell short, it didn’t have the special thing that we wanted it to have.

   And the thing I would say to you is everybody goes through that. And for you to go through it, if you’re going through it right now, if you’re just getting out of that phase — you gotta know it’s totally normal.

   And the most important possible thing you can do is do a lot of work — do a huge volume of work. Put yourself on a deadline so that every week, or every month, you know you’re going to finish one story. Because it’s only by actually going through a volume of work that you are actually going to catch up and close that gap. And the work you’re making will be as good as your ambitions. It takes a while, it’s gonna take you a while — it’s normal to take a while. And you just have to fight your way through that, okay?

   -- Ira Glass from a video recording "Ira Glass on Storytelling Part 3"

10. ^{^}

    It's common advice to read good writing to improve at writing, but I feel like reading fanfiction made it much easier for me to learn what was good vs bad writing because there's such a huge range in quality. As a beginner, it's easier to see the differences between "bad" and "good", than "good" and "very good".

11. ^{^}

    This is important in dance, but I imagine it also applies to any domain involving physical skill.

0 comments

Comments sorted by top scores.