Hammertime Postmortem
post by alkjash · 2018-03-22T18:10:01.627Z · LW · GW · 13 commentsContents
1. Hammertime Report Card Writing Practice: 90/100 Personal CFAR Review: 95/100 Entertainment: 65/100 Teach Instrumental Rationality: 50/100 Overall: 75/100 2. Hammers by Power Level S/A Tier Focusing: 100/100 Yoda Timers: 95/100 Design: 90/100 B/C Tier Bug Hunt: 80/100 CoZE: 80/100 Silence: 80/100 TDT for Humans: 75/100 Friendship: 75/100 D/F Tier Murphyjitsu: 65/100 TAPs: 60/100 Internal Double Crux: 50/100 Aversion/Goal Factoring: 30/100 None 13 comments
A bit less than two months ago, I set out to write about instrumental rationality every day for thirty days. In this post, I will quickly evaluate how well I felt I did along each of my four stated objectives. I will simultaneously evaluate all the Hammertime techniques and ideas by their effectiveness to my life.
This period was my deadline to 80/20 instrumental rationality. Thus, I do not plan to blog any more about it for a while. However, I do want to express my strong intent to write a fourth cycle of Hammertime in the early months of 2019, if only to check my long-term progress.
1. Hammertime Report Card
I will grade myself on the four goals I stated in the first Intermission thread:
My reasons for writing this sequence were, in clear order of importance: (a) to practice writing, (b) to review CFAR techniques for my own benefit, (c) to entertain, and (d) to teach instrumental rationality.
On reflection, these were equally important goals and I only listed them in that relative order because I believed the later ones would be harder to achieve. I will grade everything out of 100, counting up from zero. Only the relative sizes of the numbers mean anything.
Writing Practice: 90/100
This worked out quite well. I produce content about three times faster than I did at the beginning of Hammertime, with perhaps the slightest decrease in quality. Speed I value as much as strength, so this was an amazing improvement. There are things like organization and style I should have played around with more, and a Yoda Timer of copy-editing after each post would have benefited the writing quality greatly.
Personal CFAR Review: 95/100
Through this process I was forced to reflect on, try out, and push the boundaries of almost every single technique in the manual. Other than a handful of techniques that don’t click with me at all, this two-month period has been the perfect amount of time to throw at dedicated instrumental rationality practice. The long-term value of the learning I did at CFAR at least tripled because I did this.
Entertainment: 65/100
Hit or miss. Handful of posts that were really fun to write, and still look fun to read. I noticed a number of clear limitations in my writing toolkit that don’t seem to be fixable in a day or two (but might be if I actually tried). Despite my best efforts, I’m still not Eliezer or Scott.
What am I missing? I plan to experiment more with dialogue, which I’m awful at writing but seems to make some of Eliezer’s and Scott’s funniest stuff. Also, detailed and entertaining expositions of science are sorely missing in my writing – this seems like a gold mine as well.
Teach Instrumental Rationality: 50/100
Not sure this sequence is any better as pedagogical material than just the CFAR Handbook, which is a moderately dry reference manual. Perhaps that’s good enough. A handful of people seemed to benefit quite a bit, but my sense is that even among the people who read every post, few did any of the exercises or got any mileage out of this sequence over learning what the concept handles are. In the end, I always made decisions in favor of “write what’s interesting for me” rather than “write what I think would be most useful to the reader.”
Perhaps an interested reader would like to take a couple hours and reassemble the most useful parts of Hammertime into a cleaner subsequence. As a resource on instrumental rationality instruction at most half of the posts in Hammertime are of high value.
Overall: 75/100
Very impressed with myself that I followed through with this project with only minor delays. Everything went approximately as well as could be Outside-View expected.
My main takeaway is to continue throwing myself headlong into medium-term projects without thinking too much about them, and trust my instincts. It’s not obvious that more planning or structure would have helped in net – it may even have soured the whole Hammertime project and caused me not to finish at all.
2. Hammers by Power Level
I will go through the core techniques I covered in Hammertime, and grade them each based on effectiveness in my own life.
I’ll sort them into three tiers of awesome. Note that the techniques in Hammertime were already pre-selected from a larger pool of techniques based on how good they seemed to me just after CFAR.
S/A Tier
Focusing: 100/100
Doesn’t always work, but when it does … life-changing insights. Probably had three or four over the course of Hammertime. Would recommend.
Yoda Timers: 95/100
Timers and deadlines really up my game. I think I’ve always shied away from using them because “contest math,” “speed,” and “competitiveness” became low-status after high school, but man am I built for this. Sometimes I think that if grad school was structured as a serious of olympiads except with open problems, I would get a lot more work done.
Design: 90/100
Amazingly underrated technique. Amortizing everything, allowing myself to remove trivial inconveniences, spending time making my physical space better. Substantially improved my baseline quality of life: sleep quality, overall comfort, aesthetics. If I gave up actively using instrumental rationality right now, the effects of the Design choices I made in the last two months would still last for years.
B/C Tier
Bug Hunt: 80/100
Very useful to practice every so often. Ups your noticing game quite a bit for a long time.
CoZE: 80/100
Another solid technique. Gave me the tools to push through many minor unendorsed aversions and try things instinctively. Doesn’t work as well by itself on the bigger aversions – in my experience, these require the aid of Focusing and Focusing is the one doing the work.
Silence: 80/100
I feel as if combating the tendrils of nihilism in everyday life is one of the biggest problems to solve. Silence was my first attempt at framing the problem and offering a partial solution. As always, people need to allow themselves to babble more.
TDT for Humans: 75/100
Important principle that finally allowed me to understand the appeal and utility of virtue ethics/deontology. Requires more iteration and work to make it actionable.
Friendship: 75/100
Noticing the value of and setting up long-term iterated conversations with friends was extremely valuable. Experimenting with this also led me into a handful of awkward social situations and unproductive conversations. I’ve updated towards there existing even fewer people than I thought with whom I can have interesting conversations on a regular basis.
D/F Tier
Murphyjitsu: 65/100
It feels as painful and difficult to practice as reading ability in Go – life is too chaotic. For now, it’s only useful on the five-second level: what are the obvious things that will go wrong? Perhaps after I collect more data about common failure modes Murphyjitsu will be more useful. As of now, I feel woefully uncalibrated.
On the plus side, did inspire my longest work of fiction to date.
TAPs: 60/100
Weird and unnatural to practice. Handful of useful things I thought I installed rapidly faded with time. TAPs seem to last about a week for me without some other regular reinforcement mechanism.
Internal Double Crux: 50/100
Too many steps. The only real value seems to be as a method for generating Focusing targets. This is pretty valuable, but still.
Aversion/Goal Factoring: 30/100
Tried a few times, didn’t stick. Much weaker than Focusing. Usually, what I need to do is “find out my true main motive and aversion towards the thing,” and once that is done the path forward becomes clear.
13 comments
Comments sorted by top scores.
comment by [deleted] · 2018-03-22T18:38:34.169Z · LW(p) · GW(p)
Thanks for writing this up! I think postmortems are great, and now I'm imagining what having a specific section on the site dedicated to postmortems would look like... You'd be able to browse through and see how projects like yours, MetaMed, Arbital, etc. all ended and see if there were lessons learned across them, etc.
Anyway, I think this one part:
Perhaps an interested reader would like to take a couple hours and reassemble the most useful parts of Hammertime into a cleaner subsequence. As a resource on instrumental rationality instruction at most half of the posts in Hammertime are of high value.
is quite interesting to note. Especially the bit about cleaner subsequences. I'm wondering if there's ample post-processing happening to blog posts written, as efforts to crystallize and clarify / distill feel like they could be quite high value for pedagogical purposes.
Also some stuff from Kuhn about field-building, but those thoughts are still percolating right now.
Replies from: ingres, Raemon↑ comment by namespace (ingres) · 2018-03-22T19:02:39.081Z · LW(p) · GW(p)
Might I suggest that rather than have 'a special section' dedicated to many things which the dev team would have to create interfaces for, we bring back tagging but do it in the style of delicious.
http://idlewords.com/talks/fan_is_a_tool_using_animal.htm
Crowdsourced tags, that are saved in a users profile so they have an incentive to do it (and perhaps some karma reward), which are aggregated/ranked by popularity to get the right tags for an article. Then you make a page showing articles by tag and you could get a 'section' for postmortems, another for surveys, etc.
Though now that you mention it, I should probably add a "postmortem" section to my Diaspora Project Map.
↑ comment by Raemon · 2018-03-22T22:58:16.596Z · LW(p) · GW(p)
I think postmortems are great, and now I'm imagining what having a specific section on the site dedicated to postmortems would look like
The simplest way to do this using the current set of tools on the site would be to create a sequence. (Note: anyone can create a sequence and add anyone's posts to it).
I think I roughly agree with Hypothesis that longterm, some kind of tagging solution is preferable so that anyone can easily add to it, although I can imagine a good execution of a Postmortem sequence that actually adds a fair amount of context and infrastructure beyond just listing the articles.
comment by alkjash · 2018-03-22T23:27:39.027Z · LW(p) · GW(p)
Reply to this comment with your personal grades of any of the Hammertime techniques you tried. What worked and didn't work for you? How could the exposition have been improved?
Replies from: geoishard↑ comment by geoishard · 2018-03-23T00:34:11.380Z · LW(p) · GW(p)
S Tier:
Timeless decision theory for humans.
Understanding Newcomb's problem (http://acritch.com/deserving-trust/) especially made me see a bunch of ways I had been 2-boxing. I often tried to approximate the best input instead of maximizing the expectation of my algorithm, or kept trying to recompute the best option (with varying outputs depending on my mood). For example:
1. Juggling lists of rules for socializing better in various contexts. 1-boxing is trying to maximize my own fun.
2. Feeling guilty/unhappy staying with things because there are/will be other options, and I probably didn't pick the best one. Stuff like what math to learn, what self-improvement to do, what girl to date. Now that making a choice feels like 1-boxing, it's easier to pick a smaller number of things and trust the decision.
3. Stressing over how often to do/practice something new (e.g. a new rationality technique) when I can just do as much as I feel like and schedule a self-check-in later.
Also, having regularly scheduled conversations with friends feels like good TDT when it works out.
IDC/typing to self:
For me it goes: I notice confusion, then I IDC/type to myself for a while, then it reliably makes way more sense. IDCing started slow but now feels pretty similar to single-streaming which is why I'm conflating them, also I often single stream for a bit until I find something to IDC. Paper is usually worse than typing, but can be good for finding new topics.
Meh Tier:
TAPs
For me these are just convenience tricks, like "if I need to remember do to something, send myself an email now." They mostly pattern-match to hacky scaffolding in my head, like if I'm starting rationality and have trouble going to bed before 4am, maybe a TAP will help. The manual installation process feels bizarre to me, the ones I have are more organic, so maybe they don't really count. Still I like having the background awareness that TAP-like internal algorithms are good for making simple stuff work consistently.
Aversion/Goal Factoring: I do this occasionally when it feels I'm conflating things in my head and getting confused. I agree that it doesn't feel like the heavy duty part.
Design
Design seems like something I mostly did already. But your writing about it led to a couple of minor life improvements, like putting pretty math pictures on my walls. I'm glad to have it as a named concept.
Want to try in the future:
Focusing (I have decent introspective access but don't really use my body. At least I've never noticed that a specific body part felt a certain way.)
Yoda Timers
Bug hunt
Silence
comment by silentbob · 2019-09-12T07:20:24.468Z · LW(p) · GW(p)
As mentioned in the final exam, here's my personal summary of how I experienced hammertime.
I feel like following the sequence was a very good use of my time, even though it turned out a bit different from what I had initially expected. I thought it would focus much more on "hammering in" the techniques (even after reading Hammers & Nails and realizing the metaphor worked in a different way), but it was more about trying everything out rather briefly, as well as some degree of obtaining new perspectives on things. This was fine, too, but I still feel like I haven't got a real idea about whether things such as goal/aversion factoring, mantras, internal double crux, focusing or timeless decision making actually do anything for me. I applied some of the techniques once, but it didn't really lead to any tangible results. I may have done them incorrectly, or I may need to practice more, or maybe they just don't work for me, and it's now up to me to figure this out in detail.
I derived a lot of value from creating and frequently updating the bug list. TAPs are a neat concept, and those that work are really helpful, but many fail for me. Maybe I'll get a better feeling for which triggers work for me so I can tell beforehand instead of going through days and weeks of consistent failure with a trigger. Design surely works, but I've got some aversions to applying it which I'll have to unravel. CoZE is something I never really doubted, and I like the new framing of basically just becoming the kind of person who's open to new things, as opposed to forcing oneself to do scary things. I've been following the "all else being equal expanding my comfort zone is good" heuristic for a few years already, and will continue to do so, as my natural instinct otherwise is usually to exploit rather than to explore.
Yoda timers/resolve cycles and murphyjitsu probably had the greatest effect on me. On day 10 I murphyjitsued three of my major quarter goals and increased my expected value of how many of them I'd achieve from 1.24 to 2.08 in the process. At the time these were merely predictions and would be worthless if they were not correlated with reality - now however I can say that I'm on track to reach 2.5/3, and I'm highly confident that murphyjitsu made a huge counterfactual difference and I hadn't simply been underconfident before.
I followed the sequence with a group of other people, sharing our progress in a slack channel, which kept up my motivation and probably made it a lot more interesting than "merely" following a year old sequence on my own, so that's certainly something I recommend to others who are interested in giving it a try.
To provide some numbers, I've identified 175 bugs by now, 35 of which I consider solved, and around half of which I expect to solve within the next year, which isn't overly ambitious but still in the order of "life-changing" if things work out, which sounds good enough for me.
So, overall: Thanks a lot alkjash!
Replies from: silentbob↑ comment by silentbob · 2020-12-11T13:32:57.714Z · LW(p) · GW(p)
Update a year later, in case anybody else is similarly into numbers: that prediction of achieving 2.5 out of the 3 major quarter goals ended up being correct (one goal wasn't technically achieved due to outside factors I hadn't initially anticipated, but I had done my part, thus the .5), and I've been using a murphyjitsu-like approach for my quarterly goals ever since which I find super helpful. In the three quarters before Hammertime, I achieved 59%, 38% and 47% respectively of such goals. In the quarters since the numbers were (in chronological order, starting with the Hammertime quarter) 59%, 82%, 61%, 65%, 65%, ~82%. While total number and difficulty of goals vary, I believe the average difficulty hasn't changed much whereas the total number has increased somewhat over time. That being said, I also visited a CFAR workshop shortly after going through Hammertime, so that too surely had some notable effect on the positive development.
My bug list has grown to 316 as of today, ~159 of which are solved, following a roughly linear pattern over time so far.
comment by Qiaochu_Yuan · 2018-03-27T23:26:29.052Z · LW(p) · GW(p)
Congratulations! I really enjoyed watching you write Hammertime. Seems like a great way to get the most out of having attended a CFAR workshop, and I'm definitely updating on the importance of writing / blogging as a tool.
comment by 9eB1 · 2018-03-25T22:11:58.362Z · LW(p) · GW(p)
It's too bad the link for the referenced *"Focusing," for skeptics* article in your post on the tactic only leads to a 404 now. I wonder if it was taken down intentionally?
Replies from: alkjash↑ comment by alkjash · 2018-03-25T22:54:10.396Z · LW(p) · GW(p)
It seems like the author took down all his posts, my best guess is he's porting them to a new account.
Replies from: Qiaochu_Yuan↑ comment by Qiaochu_Yuan · 2018-03-27T14:52:36.587Z · LW(p) · GW(p)
Nope, he just took them down.
comment by jamii · 2018-03-23T10:33:46.815Z · LW(p) · GW(p)
TAPs seem to last about a week for me without some other regular reinforcement mechanism.
For a few weeks I've been writing them down in a text file. I read and rehearse them every morning over coffee, and just before I go to bed I look through them and reflect on whether I missed any triggers. It fits into journal habits that I already had so the inconvenience is quite low. So far I've been noticing triggers at a higher rate, but it's still in the novelty phase.
Replies from: efim