The Cognitive Bootcamp Agreement

post by Raemon · 2024-10-16T23:24:05.509Z · LW · GW · 0 comments

Contents

  The Format
  The Goal: Solve confusing, intractable problems
  The Curriculum
  Skills
  Multiple feedback-loops
    The Immediate Metric: General Strategies
  The Disclaimers (Important!)
    Prerequisites
    This will be very meta
    What's the evidence that this works?
      First Workshop 6-month followup
    What’s up with the cost? Can I get a discount?
None
No comments

For the next Cognitive Bootcamp, I wanted to experiment with a format that is 

The next workshop is Friday, Oct 25 at 4pm thru Sunday at 7:30pm. It will cost $600 ($400 if you are staying for more than two weeks at the Lighthaven Eternal September season). I will raise prices in the future as I become more confident in the material. 

I might do one the weekend of November 15th (same times), which will most likely be $800 if it happens. It will depend on how the Oct 25th one goes and what else Lightcone is doing that month, but if you want to start tentatively planning for it let me know.

This post is the terms I want people to agree to beforehand. If you are interested in attending a workshop, please read and then fill out this form.

The target audience is people who:

Note this is still an experimental beta-test (see the Disclaimers section for more info). 

The Format

The format will evolve, but the frame I’m trying next is:

  1. By coming to the workshop, you agree that you will spend the whole time either:
    1. aiming to work basically as hard as you can.[1]
    2. napping / taking a walk / etc (preferably without devices[2])
    3. talking to me if the workshop feels off in some way.
       
  2. Your goal is to learn at least one new skill, starting from "it’s too cumbersome to use productively", and aiming to reach "Juuuust fluent enough that you can start applying it to your day job, practicing so it becomes easy." 
     
  3. Beforehand, you send me your default plan for the next ~week, month or quarter – whatever the longest timescale you plan on. You’ll work on improving this plan (though the main workshop goal is still “gain one new skill”)

I’ll present ~4 skills I think are valuable for solving confusing problems, and exercises that are helpful for grinding on them. 

You’re welcome to pick other skills, or other exercises, that you think will help you better. But, for each session, you need to clearly explain a) why I think the original exercise was important, b) why the thing you’ll do instead is better. (We’ll chat until we both understand each other and feel good about it)

Previously, I’ve found that giving people too much leeway in changing the curriculum results in the workshop losing focus, but too little leeway results in people not quite getting what they need. I’m interested in trying out this compromise.

The Goal: 
Solve confusing, intractable problems

Many of the most important problems in the world are confusing, intractable, and seem impossible. They are really important to solve anyway. 

But many people who try, end up in failure modes such as…

  1. Rabbit-holing, fixating on an approach which doesn’t work, or takes too long.
  2. Goodharting, substituting an easier problem, maybe without even noticing.
  3. Despair. It’s impossible. You give up.
  4. Zombified Agency, mechanically execute virtuous-seeming plans, but something inside you is dead and hollow and the plan probably won’t work and maybe you’ll hurt yourself.

I want you to leave the workshop with at least one new skill for solving impossible problems. 

The Curriculum

The default curriculum focuses on “making better plans” via…

  1. Generating multiple plans, so you don’t over-anchor on your first plan.
  2. Tracking multiple goals, so you don’t over-anchor on your first goal.
  3. Identifying your cruxes, and confidently deciding when to pivot or persevere.
  4. Making cruxy predictions, so over time you can credibly, calibrated trust your intuition (and, know the limits of where you cannot)

An optional, higher level frame is Fractal Strategy – thinking of plans and goals at multiple levels, tracking how smaller plans fit into bigger plans, and when it’s time to move up, down or sideways in plan/goal-space.

If you feel like you roughly have all these skills, you might still want to come to the workshop to basically “do the cognitive practice you believe in with good form, attending carefully to each step of the process.” (I’ll want to talk to you beforehand about your plan in detail)

Skills

The one skill I will require everyone to attempt at least twice is generating metastrategies (see below). 

Each of the bullets in the Curriculum section has a corresponding skill. Some particular skills that will be presented that will come up if they seem helpful to a particular student are:

Multiple feedback-loops

I don’t have perfect feedback-loops to tell you if this workshop is working for you. So, there are four different feedback-loop types, with different tradeoffs:

  1. Predictions
    1. Guess whether a given strategy will work, then see if you were right.
  2. Toy Exercises.
    1. They only vaguely resemble your real problems, but you’ll know for sure whether you got the right answer in two hours.
  3. Big picture planning.
    1. You’ll generate at least one new plan. You won’t really know if it’s good, but a) you’ll have intuitions about it, which are at least some information. And, b) you’ll make predictions about whether it’ll seem worth having thought about in a year.
  4. Object-level work, in 1-hour blocks
    1. Spend a few timeblocks doing object level work on your second likeliest plan. Each hour, you’ll make conscious choices about how to spend your time and attention. And then, reflect on whether that seemed useful.
      (in addition to crosstraining your skills on the practical object-level, this will help make your second-likeliest plan feel more real)

The Immediate Metric: General Strategies

The fundamental skill of the workshop is “brainstorming metastrategies.” That is, general strategies that work on a wide variety of problems (usually by helping you generate more “tactical” strategies appropriate to the situation)

Each session, you’ll be doing a puzzle, or intellectual work. The point isn’t whether you succeed or not on the puzzle. The point is whether you learn generalized strategies that either definitively helped you on the problem, or seem likely to help you in the future.

After each session, you’ll:

Each time you successfully use a strategy already listed on the whiteboard (including the first time you write it down), put your initials next to it.

The (slightly Goodharty but hopefully productive) goal of the workshop is to fill the board with strategies that get used lots of times.

There will be prizes for people who generate the most-used strategies. (Exact prize TDB, probably more like a fun token than, like, money)

The way you’re supposed to tell if the workshop is helping is “are you generating strategies that help you?”.

If by the end of Day 1, if you haven’t generated any metastrategies, we’ll chat about whether we should adjust anything on Day 2 for you. If by the end of Day 2 if you haven’t generated any strategies, we’ll check in on whether the workshop is working for you or you are otherwise getting value out of it, and possibly call it quits, or radically change what you focus on for Day 3.

The Disclaimers (Important!)

Prerequisites

This is not an entry level rationality workshop. It has several prerequisites:

  1. Executive function. You can sit down and think about a confusing problem and not immediately bounce off. Your problem should be more like “when I sit down to plan, I don’t know what to do” than “I have trouble sitting down to plan,” or “if I make a decision, I can’t follow it through.”[3]
     
  2. Project Ownership. You have control over an (at least somewhat) open-ended decisionmaking processi.e. you get at least some leeway to set priorities at your day job, or you have a lot of slack for ambitious hobbies.. A primary skill at this workshop is learning when to pivot. You need to be capable of deciding when you’re pivoting. When you leave the workshop, you have a project you expect to be applying the techniques to on a daily basis.
     
  3. Self awareness. You can notice when pushing yourself hard is bad for you, instead of good for you. You have at least some awareness of when things are going subtly wrong and you need to slow down. I'll do my best to help you notice this sort of thing, but there’s a limit to how much I can help.

If you are at risk of burnout (if you’ve recently worked much harder than usual, or a nagging voice inside is worried about spending a weekend doing very intense thinking) I recommend you do not attend right now. You can register your interest for future workshops though.

This will be very meta

We are going to think about thinking. 

We are going to apply feedback-loops to our feedback-loops. 

While we’re doing that, I will be thinking about thinking about you thinking about those things (You don’t have to do that, just me).

Meta-level optimization is the mechanism by which, maybe, I expect people to get compounding returns on thinking, and there is a real possibility that this can be leveraged into dramatically better plans.

Well designed feedback loops are the grounding mechanism to check whether we’re doing masturbatory meta, vs useful meta.[4]

I think there are basically two flavors of "I feel like thi meta sucks". One is that you've lost track of your goals, you've spent more time meta-optimizing than makes sense, and your subconscious is (correctly) flagging that it's time to get back to work. Another is "it feels overwhelming, like you can't track what's going on." 

I think it can be correct (nearterm) to stop if you're overwhelmed, but one of the skills the workshop is trying to impart is the ability to navigate complex problems without getting overwhelmed (complex metacognition included).

If you are feeling disoriented or annoyed or have a nagging feeling the current amount of meta won’t help:

  1. First, pause (if I’m the problem, say “hey Ray stop”)
  2. Probably, go back to doing a more object level thing
  3. But, also: consider getting out a sheet of paper / google-doc and writing things down so you can actually track what’s going on. Or, chatting with me about it and seeing if we can find a way to make it more manageable.[5]

Part of the point of becoming fluent in “working memory extension” is that your meta processes are much easier to understand, and you can leverage them more strategically.

What's the evidence that this works?

I am talking a pretty big game. I think it’s an active ingredient for me to present content confidently, and for people to lean into a mindset where they trust it’ll work, or at least is worth trying.

But an important truth is that this workshop series does not (yet) have a strong empirical track record. If the curriculum does not intuitively make sense to you, I don’t think you should particularly believe in it.

I am trying to stick my neck out such that if the workshop is not working, it should be obvious (i.e. people will not be generating strategies that help), and over time if the workshop does not help, it should be clear that the people who take it don't end up standing on giant heaps of utility. And if I haven't gotten that to obviously work in ~a year, I'll give up.

Meanwhile: here are some facts about past participant experience:

First Workshop 6-month followup

When I ran the first workshop (which I charged $200 for), I asked the 9 participants what was the most they’d have paid for the workshop. Numbers ranged from $300 to ~$2000, on average $800.

When I asked again six months later, two “$400” people and one "$1500" changed their number to “$2000”. 

Meanwhile two people reported back they'd probably lowered their estimate:

When I ran the second workshop, average rating was $540, or $650 if you throw out one data point from someone (who rated it $0) who came for nonstandard reasons, and I wouldn’t have really expected to get much value out of it, if we’d talked a bit more. There was another person who also reported they were burned out[6] and now wasn't a good time for the workshop, although I don't think they were actively hurt.

What’s up with the cost? Can I get a discount?

I charge a fair amount of money (and plan to charge more as I become more confident in the curriculum), because:

In general, if you didn’t feel like you got your money’s worth, I prefer to solve this by giving you some free followup coaching until you feel like you got your money’s worth.

If you earnestly tried to get a thousand dollars worth of value from the workshop, including explicitly strategizing with me on how to adapt it to you, and it didn’t really pay off and it seems like my general frame or skillset isn’t useful to you, chat with me and I’ll consider a refund.

After reading through this, if you're interested in either the October 25th workshop or a future one, please fill out this form.

The schedule for the weekend is:

You are welcome to stay at Lighthaven during the workshop (I think immersion is good, but also getting enough sleep is also good and encourage you to choose whatever will make sure you're best rested).

I request you do not have other plans during the weekend that might distract you.

 

  1. ^

    You don't necessarily need to succeed at thinking the hardest you ever thought, but, you should be aiming for a spirit of pushing your boundaries.

    The workshop is not meant to be sustainable longterm, but is meant to renew your sense of what you're capable of, and what you can be comparing day-to-day thought against.

    You should not be avoiding the next-obvious-thought because it felt too effortful to think.

  2. ^

    This is negotiable since, like, it might really be a loadbearing part of your resting structure. But I will talk to you skeptically about it, and I will probably make you make a fatebook prediction about whether you ultimately endorse having taken your devices.

  3. ^

    I do think planning can be a useful skill for fixing executive function issues (i.e. "make a plan to fix it"), but, you wouldn't really capitalize on most of the workshop, and I'd probably recommend a workshop much more focused on executive function stuff if you have a workshop-shaped-hole in your life.

  4. ^

    Feedback-loops are kind of meta, but I argue that having many different types of feedback-loops should make you less skeptical of "cognitive meta", rather than more. Some feedbackloops should be things like "your gut feels like you are still on the right track, and isn't getting carried away Goodharting on the easier-to-measure feedback loops."

  5. ^

    If you're overwhelmed by meta and pen-and-paper isn't helping, unfortunately trying to diagnose what's going wrong... sort of intrinsically involves more meta, which won't work. I am admittedly still figuring out my process for helping in this situation, but I default to "just stop."

  6. ^

    They didn't exactly say "burned out", the longer description was "they'd just spent awhile being very strategic and they realized they needed a break now."

  7. ^

    i.e. you get at least some leeway to set priorities at your day job, or you have a lot of slack for ambitious hobbies.

0 comments

Comments sorted by top scores.