Posts
Comments
How large do you think the marginal benefits of doing the full workout you recommend in Updates and Reflections on Optimal Exercise after Nearly a Decade versus the quicker version in this post?
Useful clarification and thanks for writing this up!
Inspired by and building on this, I decided to clean up some thoughts of my own in a similar direction. Here they are on my short forum: What are the actual use cases of memory systems like Anki?
What are the actual use cases of memory systems like Anki?
Epistemic status: spent 30min cleaning up some notes from my Obsidian I jotted down yesterday. This ontology is rough and a bit illegible but potentially useful for narrowing down the actual use cases of memory systems.
Inspired by @Saul Munn's recent short form: Active Recall and Spaced Repetition are Different Things. The concepts of active recall and spaced repetition apply pretty well here, but I saw Saul's post after writing most of the text below.
Roughly, there are types of knowledge in domains (recalling from Scott Young's Ultralearning, I might be slightly off):
- Proceduralized knowledge
- Conceptual knowledge
- Factual knowledge
I think spaced repetition systems are useful for three types of domains based on the nature of the cues in the domains:
- Domains where you are already cued on the learnings often (e.g., key business procedures in your full-time job). You naturally get spaced repetition in these domains, so memory systems like Anki are less useful.
- Domains where you are not cued on the learnings often or at properly spaced intervals. This may include domains where you were an expert at one point and then stopped being an expert.
- Domains where you need to understand cues before being effective. (e.g. understanding a new language or technical domain like quantum mechanics; see Quantum Country.)
Combining the two frameworks above, Anki is useful for:
- Proceduralized knowledge where you can appropriately cue yourself and you're not cued often on.
- Cueing procuduralized knowledge is hard, though. For example, if I wanted to review my knowledge of 'how-to-play-beach-volleyball' on a regular basis, I have to spend a time coordinating or attending an event.
- Though, some proceduralized knowledge is possible to cue: for example, I have been using Anki to review LeetCode problems.
- Conceptual and factual knowledge that you're (2 above) not cued often on or (3 above) need to meet a criteria of knowing.
- Conceptual and factual knowledge is much easier than procedural knowledge because these types of knowledge neatly fit in flashcards. I'm still not sure about the strict boundary between conceptual and factual knowledge, though.
- And I think the general heuristic that conceptual knowledge is harder to cue than factual knowledge is true: medical students are obsessed with Anki while students in other vocational schools (e.g. law) don't seem to be
Accordingly, memory systems may not be useful for:
- Domains where you are cued on information often enough to get natural spaced repetition.
- Are highly procedural in a way that can't be Ankified and therefore active recall is hard.
Another caveat:
- I am believable and have expertise in very few major life skills, and possibly don't have expertise in the thing you're asking advice for.
Related note: I think developing the skill of identifying believability and expertise is very powerful (though I have only been applying said skill for a couple of years explicitly; caveat emptor, lol.)
Here's Cedric Chin outlining believability defined by Ray Dalio:
Technique summary:
Believable people are people who have 1) a record of at least three relevant successes and 2) have great explanations of their approach when probed.You may evaluate a person's believability on the subject matter at hand by applying this heuristic. When interacting with them:
- If you’re talking to a more believable person, suppress your instinct to debate and instead ask questions to understand their approach. This is far more effective in getting to the truth than wasting time debating.
- You’re only allowed to debate someone who has roughly equal believability compared to you.
- If you’re dealing with someone with lower believability, spend the minimum amount of time to see if they have objections that you’d not considered before. Otherwise, don’t spend that much time on them.
Here's Gary Klein, founder of Naturalistic Decision Making, outlining seven dimensions of expertise:
We want pragmatic guidelines for deciding which if any purported experts to listen to when making a difficult and important decision. How can we know who is really credible?
Bottom line: We cannot know for sure. There are no iron-clad criteria.
However, there are soft criteria, indicators we can pay attention to. I have identified seven so far, drawing on papers such as Crispen & Hoffman, 2016, and Shanteau, 2015, and on suggestions by Danny Kahneman and Robert Hoffman. Even though none of these criteria are fool-proof, all of them seem useful and relevant:
(a) Successful performance—measurable track record of making good decisions in the past. (But with a large sample, some we do very well just by luck, such as stock-pickers who have called the market direction accurately in the past 10 years.)
(b) Peer respect. (But peer ratings can be contaminated by a person’s confident bearing or fluent articulation of reasons for choices.)
(c) Career—number of years performing the task. (But some 10-year veterans have one year of experience repeated 10 times and, even worse, some vocations do not provide any opportunity for meaningful feedback.)
(d) Quality of tacit knowledge such as mental models. (But some experts may be less articulate because tacit knowledge is by definition hard to articulate.)
(e) Reliability. (Reliability is necessary but not sufficient. A watch that is consistently one hour slow will be highly reliable but completely inaccurate).
(f) Credentials—licensing or certification of achieving professional standards. (But credentials just signify a minimal level of competence, not the achievement of expertise.)
(g) Reflection. When I ask "What was the last mistake you made?" most credible experts immediately describe a recent blunder that has been eating at them. In contrast, journeymen posing as experts typically say they can't think of any; they seem sincere but, of course, they may be faking. And some actual experts, upon being asked about recent mistakes, may for all kinds of reasons choose not to share any of these, even ones they have been ruminating about. So this criterion of reflection and candor is not any more foolproof than the others.
[I only skimmed the post so you might have addressed this, but…]
I once met a sometime who made super intense eye contact all the time, and it gave me weird vibes. Sort of uncanny valley vibes, if I try to put words to it? It was like they were staring at me in normal conversation. One hypothesis I had as to why is that maybe he did one of these type of activities that you outline here (iirc, I was introduced to the idea being a thing through an old LW post on scientology doing it?).
Epistemic status: I don’t know if an event like this was the cause of this person’s intense eye contact. I don’t know if anyone at your events has this result. But just thought I would mention it in the off chance that “getting in the habit of giving super intense eye contact” is a possible failure mode of doing activities like this.
Tacit Knowledge Videos added to the list from May–August, 2024. Enjoy!
- Kenneth Folk, pranayama breathing.
- “Kenneth Folk is an instructor of meditation who has received worldwide acknowledgement for his innovative approach to secular Buddhist meditation. After twenty years of training in the Burmese Theravada Buddhist tradition of Mahasi Sayadaw, including three years of intensive silent retreat in monasteries in Asia and the U.S., he began to spread his own findings, successfully stripping away religious dogma to render meditation accessible to modern practitioners" (Website).
- Keith Johnstone, teaching improv.
- Author of Impro.
- "A pioneer of improvisational theatre, he was best known for inventing the Impro System, part of which are the Theatresports" (Wikipedia).
- Gwern, "Internet Search Case Studies"
I’ve had a busy past few months (if a three-month meditation retreat counts as busy). There have been more videos submitted than videos added to the post in this batch. I will add these in the coming months.
Thanks!
If there were a bunch of videos of people Google-fu-ing like this, I wouldn't add this article to the post.
However, since this is one of the few good resources on Google-fu that I know of, I'm adding it to the post. Despite it not being a video.
I've added this as a note. Thanks!
Thanks for the kind words! I'd be curious to hear more about what makes you think that!
Edited - thanks!
How would you go about answering this question post-said insight? What would the mental moves look like?
I'm never good at giving an answer to my favorite book/movie/etc.
I’m considering live-streaming (or just making a video) of myself doing this on a totally new topic to show how I do this in real-time, since it works so well for me. Let me know if you think this could be helpful.
I'd be curious to see this video of you creating flashcards on a new topic! And would very likely add it to the post.
Any chance you could pin the 'Updates Thread' too?
Below are the new tacit knowledge videos added to the post since mid-April, 2024.
- Paul Meehl, Philosophical Psychology 1989 course lectures, "deep introduction to 20c philosophy of science, using psychology rather than physics as the model science -- because it's harder!" (via @Jonathan Stray)
- "Meehl was a philosopher of science, a statistician, and a lifelong clinical psychologist. He wrote a book showing that statistical prediction usually beats clinical judgement in 1954, and a paper on the replication crisis in psychology in 1978. He personally knew people like Popper, Kuhn, Lakatos, Feyerabend, etc. and brings their insights to life in these course lectures."
- Me: I was hesitant to add a lecture series to this list at first. I changed my mind after listening to the first video, where Meehl provides interesting details (gossip, almost) about the life of an academic and the various personalities of his successful academic peers.
- Kenneth Folk, Guided Tour to 13 Jhanas.
- “Kenneth Folk is an instructor of meditation who has received worldwide acknowledgement for his innovative approach to secular Buddhist meditation. After twenty years of training in the Burmese Theravada Buddhist tradition of Mahasi Sayadaw, including three years of intensive silent retreat in monasteries in Asia and the U.S., he began to spread his own findings, successfully stripping away religious dogma to render meditation accessible to modern practitioners” (Website).
- Dave Whipple, building a "[s]imple off grid Cabin that anyone can build & afford (and many other builds on his channel). (via @Vitor)
- "Construction contractor, DIY living off-grid in Alaska and Michigan."
- "He and his wife bootstrapped themselves building their own cabin, then house, sell at a profit, rinse and repeat a few times. There are many, many videos of people building their own cabins, etc. Dave's are simple, clear, lucid, from a guy who's done it many times and has skin in the game."
- Seymour Bernstein, teaching piano. (via @lfrymire)
- "Pianist and composer, performed with the Chicago Symphony Orchestra, Adjunct Associate Professor of Music and Music Education at New York University."
- "Tonebase (a paid music learning service) recorded a number of free to watch conversations with Bernstein while he plays through or teaches a piece. Bernstein is about 90 years old at the time of recording and shares an incredible amount of tacit knowledge, especially about body mechanics when playing piano."
- RegLocal, advanced driving. (via @masasin)
- "former police driving instructor; he has a book, but the videos themselves are so helpful"
- Ryan Farran (Missionary Bush Pilot), flying small aircraft in Papua New Guinea. (via @masasin)
- "My job as a bush pilot is to fly missionaries, medical flights, and cargo into mountain and jungle airstrips throughout all of [Papua New Guinea]" (YouTube).
- Misha Glouberman, Recorded Coaching Session. (via @Misha Glouberman)
- "Consultant, Business Coach, and Co-Author of The Chairs Are Where The People Go."
- Testimonials: Mark Surman, President of Mozilla; Shenda Tanchak, Registrar & CEO of Ontario College of Pharmacists; Michael Bungay Stanier, Author of The Coaching Habit; others (Website).
- Max Egorov, "[b]ushcraft and off-grid craftsmanship". (Russian narration) (via @TANSTAAFL)
- "Advoko has a site in the woods near Lake Ladoga in Russia where he films himself building various improvements by hand with local materials. Very competent craftsman, professional touch with no hype."
- Various skilled CAD users and instructors, CAD vs. CAD Speedrunning Tournament. (via @zookini)
- "Watch some of the best SOLIDWORKS, OnShape, Fusion 360 and Inventor users Speedrun some challenging models while going head to head and sharing their screens" (YouTube).
- Scott Chacon, "So You Think You Know Git" (Part 2). (via @Max Entropy)
- Co-founder of GitHub and author of Pro Git.
- Inigo Quilez, computer graphics programming. (via @Robert Diersing)
- Has worked in roles dealing with computer graphics at Pixar Animation Studios, Oculus Story Studio, Oculus+Facebook, Adobe, and other places since 2003 (Website).
- Dan Gelbart, Building Prototypes (18 Part Series). (via @Adrian Kelly)
- "Dan Gelbart has been Founder and CTO of hardware companies for over 40 years, and shares his deep knowledge of tips and tricks for fast, efficient, and accurate mechanical fabrication. He covers a variety of tools, materials, and techniques that are extremely valuable to have in your toolbox."
- Hrishi Olickel, creating a proof-of-concept Web App using LLMs.
- CTO at Greywing (YC W21) (GitHub).
- Carl Rogers, Frederick Perls, Albert Ellis, Everett Shostrom, Arnold Lazurus, Aaron Beck: Three Approaches to Psychotherapy—recorded therapy sessions.
- Carl Rogers. Founder of person-centered psychotherapy; one of the founders of humanistic psychology (Wikipedia).
- Frederick Perls. Developed Gestalt therapy with his wife, Laura Perls (Wikipedia).
- Albert Ellis. Founder of rational emotive behavior therapy (REBT) (Wikipedia).
- Everett Shostrom. Put together the film. "He also produced well known tests and inventories including the Personal Orientation Inventory, Personal Orientation Dimensions, the Pair Attraction Inventory, and the Caring Relationship Inventory " (Wikipedia).
- Arnold Lazurus. "Authored the first text on cognitive behavioral therapy (CBT) called Behaviour Therapy and Beyond" and won various awards including two from the American Psychological Association and the American Board of Professional Psychology (Wikipedia).
- Aaron Beck. "He is regarded as the father of cognitive therapy and cognitive behavioral therapy (CBT)" (Wikipedia).
- Aswath Damodaran, "Reading a 10K".
- "Professor of Finance at the Stern School of Business at New York University, where he teaches corporate finance and equity valuation. [...] Damodaran is best known as the author of several widely used academic and practitioner texts on Valuation, Corporate Finance and Investment Management as well as provider of comprehensive data for valuation purposes" (Wikipedia).
- Anecdote from an experienced finance friend: "Damodaran is an NYU prof who's super credible and well regarded for his practical tutorials on valuations and corporate finance, I used to refer to his blog often."
- Zane Carney; composing, recording, and producing music live.
- Guitarist who has contributed to albums like Thundercat's "Drunk" and John Mayer's "Paradise Valley." Has toured with Jonny Land and John Mayer (Website).
- Shannon (House Improvements), "How to build a deck" (6 Part Series).
- Has been in the construction industry for decades. Runs his own renovation business. 925K YouTube subscribers (Channel Trailer).
- Me: A friend of mine successfully built a deck using this playlist as a guide.
- René Rebe, live streaming Linux, open source, and low-level programming hardware and software projects.
- CEO of ExactCODE GmbH since 2005 (LinkedIn).
- BNYX, Olswelm, and other indie (?) music producers; music production livestream VODs.
- William Lin, competitive programming.
- "[S]ophomore at MIT [...], IOI 2020 Winner, Codeforces Max Rating 2931 (International Grandmaster), CodeChef Max Rating 2916 (7 stars)" (YouTube About).
- Steven Ramsey, 200 days of woodworking projects during the COVID-19 lockdowns.
- Hobbyist woodworker turned woodworking content creator (1.9M YouTube subscribers); formerly a professional graphic designer (Website).
- Ben Eater, "Build a 65c02-based computer from scratch".
Updates Thread. Below are ~monthly updates with lists of new tacit knowledge videos so you don't have to scroll through the list again to find new videos.
You can subscribe to the Tacit Knowledge Video Updates Substack to have these emailed to you or sent to an RSS feed (https://tacitknowledgevideos.substack.com/feed).
Thanks! Added.
Thanks! Added.
Thanks for the Git recommendation; added!
Update: added the disclaimer.
Thanks for the feedback! I too am skeptical of the finance videos, agreeing that the video probably came across my radar due to the figures being popular rather than displaying believable tacit knowledge.
I've gone back and forth on whether to remove the videos from the list or just add your expert anecdata as a disclaimer on the videos. In the spirit of quantity vs. quality, I'm leaning toward keeping the videos on the list.
Thanks for the feedback! I'd be curious to hear more about (1) what subjects you're referring to and (2) how learning tacit knowledge with video has changed your learning habits (if your view here is based on your own experience).
Thanks! Added.
Thanks! Added.
Added. Thanks!
Thanks, added! I look forward to seeing what else you have.
I would find forecasting videos would be interesting to watch.
Thanks! Added.
Thanks! Added.
Thanks! Added.
Relevant note from the entry:
Me: I was hesitant to add a lecture series to this list at first. I changed my mind after listening to the first video, where Meehl provides interesting details (gossip, almost) about the life of an academic and the various personalities of his successful academic peers.
Fundraising videos?
Sales tacit knowledge videos?
@habryka / @mods - would it be possible to pin (1) the 'Review Thread' and (2) this thread?
I think these will be the two most valuable comments on this post. The comment video submissions are a bit cluttered due to embeds and submissions are more accessible/navigatable through the OP.
Adding to this: an interesting frame is to think about how subcultures develop illegible shadow structures beyond their legible structure and communities. Similar to how banks/bureaucracies do: https://www.bitsaboutmoney.com/archive/seeing-like-a-bank/
Review of: Elie Hassenfeld, Holden Karnofsky, Timothy Ogden, Rob Reich, Tom Rutledge, Brigid Slipka, Cari Tuna, Julia Wise: GiveWell's Public Board Meetings (2007–2020 have audio).
I'm a college student with only pretty low-stakes work experience. I listened to the first 5–10 meetings as I would a podcast last week. Some takeaways, emphasizing that I only just watched them last week:
- It was interesting to follow the narrative of Holden and Elie getting started on the project. Like, anecdotes about people's experiences starting a startup are everywhere, but it was interesting hearing them actually talking about the struggles and business decisions they were making.
- Holden worked 100h/wk in the first year; that's a lot of time spent on a project! (Then 60h/wk in the second, afaict.)
- Interesting generally how assertive the business meetings were compared to everyday conversation.
- I am familiar with GiveWell as a popular charity in the Rat/EA space, but I never really spent the time to understand the research methodology. It was interesting hearing the practical and strategic discussions between the founders and the board on the methodology. It also seemed to change every year in the first three years (I haven't watched beyond the first three years).
- Interesting from a marketing and fundraising perspective to watch as GiveWell, which seems to have found its market now, tested and went about finding one.
- Interesting to instantiate generally 'what are board meetings? who are the people in board meetings? what are their skills?'
- Discussions around productivity were interesting. I've learn 'productivity' skills to improve my time spent studying and working on projects. I was surprised to hear that this is something that was talked about in board meetings, let alone for 10s of minutes.
I would be curious to hear a review from someone with more business experience. If you are to go about watching them, I recommend starting from the beginning. I've tried watching a few more recent recordings before this past week and found them less engaging, maybe due to me having less context about the organization.
[pasting a comment of mine on Zvi's recent monthly roundup]
If anyone has anecdotes as to why they think the videos have been useful to them I'd be curious to hear. I'm still unsure of their benefit; the interest could just be novelty/insight-porn (Andy Matuschak speculates something in this direction, though he too seems ambivalent). I wrote the post partly as a test to see if there is much use.
Do people really learn anything from these streams? People certainly claim to learn things from my note-writing stream. I can believe it, maybe, but I wonder to what extent people are deluding themselves. Certainly it’s extremely inefficient: what’s the insight-per-minute?
— Andy Matuschak, Could streaming help convey tacit knowledge? (Working Notes)
Do people really learn anything from these streams? People certainly claim to learn things from my note-writing stream. I can believe it, maybe, but I wonder to what extent people are deluding themselves. Certainly it’s extremely inefficient: what’s the insight-per-minute?
— Andy Matuschak, Could streaming help convey tacit knowledge? (Working Notes)
Post attempts to compile The Best Tacit Knowledge Videos on Every Subject. I notice I lack motivation to use this modality, and think it would be a poor fit for how I learn, and that it is relatively less tempting now than it would have been two years ago before LLMs got good. The problem is that you don’t direct where it goes and can’t interact, so they’re not so likely to be teaching you the thing you don’t know and are ready to learn. But many people benefit?
If anyone has anecdotes as to why they think the videos have been useful to them I'd be curious to hear. I'm still unsure of their benefit; the interest could just be novelty/insight-porn (Andy Matuschak speculates something in this direction, though he too seems ambivalent). I wrote the post partly as a test to see if there is much use.
Glad to see that people find the post useful! I hope it will see many future contributions as well. In case interesting to anyone, I've just put together a google form Substack (edited 5.6.24) to create an email list for those who would like to be sent lists of newly added videos every month or so.
Situating the Contributions of LessWrong to the Philosophy of Language by @Suspended Reason seems to address this topic in a pretty thorough fashion.
Meal prepping Tacit Knowledge Videos?
Thanks! Added.
Thanks for sharing! Added.
I did not know 'conlangers' were a thing. Thanks for sharing and added.
Thanks for sharing! Added. I'd be curious if anyone has this but for meal prepping instead of cooking a single meal.
Tacit Knowledge Videos of eurogames.
Agreed and added.
Thanks sharing sharing this! I've added one and intend to add more of them when I have more time.
Thanks! Added.
Thanks! Added.
Thanks! Added.