The Trump Administration is on the verge of firing all ‘probationary’ employees in NIST, as they have done in many other places and departments, seemingly purely because they want to find people they can fire. But if you fire all the new employees and recently promoted employees (which is that ‘probationary’ means here) you end up firing quite a lot of the people who know about AI or give the government state capacity in AI.

This would gut not only America’s AISI, its primary source of a wide variety of forms of state capacity and the only way we can have insight into what is happening or test for safety on matters involving classified information. It would also gut our ability to do a wide variety of other things, such as reinvigorating American semiconductor manufacturing. It would be a massive own goal for the United States, on every level.

Please, it might already be too late, but do whatever you can to stop this from happening. Especially if you are not a typical AI safety advocate, helping raise the salience of this on Twitter could be useful here.

Do you (or anyone) have any gears as to who is the best person to contact here?

I'm slightly worried about making it salient on twitter because I think the pushback from people who do want them all fired might outweigh whatever good it does.

The Trump Administration is on the verge of firing all ‘probationary’ employees in NIST, as they have done in many other places and departments, seemingly purely because they want to find people they can fire. But if you fire all the new employees and recently promoted employees (which is that ‘probationary’ means here) you end up firing quite a lot of the people who know about AI or give the government state capacity in AI.

This would gut not only America’s AISI, its primary source of a wide variety of forms of state capacity and the only way we can have insight into what is happening or test for safety on matters involving classified information. It would also gut our ability to do a wide variety of other things, such as reinvigorating American semiconductor manufacturing. It would be a massive own goal for the United States, on every level.

Please, it might already be too late, but do whatever you can to stop this from happening. Especially if you are not a typical AI safety advocate, helping raise the salience of this on Twitter could be useful here.

Do you (or anyone) have any gears as to who is the best person to contact here?

I'm slightly worried about making it salient on twitter because I think the pushback from people who do want them all fired might outweigh whatever good it does.

I've now worked with 3 Thinking Assistants, and there are a couple more I haven't gotten to try out yet. So far I've been doing it with remote ones, who I share my screen with. If you would like to try them out I can DM you information and my sense of their various strengths.

The baseline benefit is just them asking "hey, are you working on what you mean to work on?" every 5 minutes. I think I a thing I should do but haven't yet is have them be a bit more proactive in asking if I've switched tasks (because sometimes it's hard to tell looking at my screen), and nagging me a bit harder about "is this the right thing?" if I'm either switching a lot, or doing one that seems at-odds with my stated goals for the day.

Sometimes I have them do various tasks that are easy to outsource, depending on their skills and what I need that day.

I have a google doc I have them read in advance that lays out my overall approach, and which includes a journal for myself I'm often taking notes in, and a journal for each assistant I work with for them to take notes. I think something-like-this is a good practice. 

For reference, here's my intro:

Intro

There’s a lot of stuff I want done. I’m experimenting with hiring a lot of assistants to help me do it. 

My plans are very in-flux, so I prefer not to make major commitments, just hire people piecemeal to either do particular tasks for me, or sit with me and help me think when I’m having trouble focusing.

My working style is “We just dive right into it, usually with a couple hours where I’m testing to see if we work well together.” I explain things as we go. This can be a bit disorienting, but I’ve tried to write important things in this doc which you can read first. Over time I may give you more openended, autonomous tasks, if that ends up making sense.

Default norms

• Say “checking in?” and if it’s a good time to check in I’ll say “ok” or “no.” If I don’t respond at all, wait 30-60 seconds and then ask again more forcefully (but still respect a “no”)
• Paste in metastrategies from the metastrategy tab into whatever area I’m currently working in when it seems appropriate.

For Metacognitive Assistants

Metacognitive Assistants sit with me and help me focus. Basic suggested workflow:

• By default, just watch me work (coding/planning/writing/operations), and occasionally give signs you’re still attentive, without interrupting.
• Make a tab in the Assistant Notes section. Write moment to moment observations which feel useful to you, as well as general thoughts. This helps you feel more proactively involved and makes you focused on noticing patterns and ways in which you could be more useful as an assistant.
• The Journal tab is for his plans and thoughts about what to generally do. Read it as an overview.
• This Context tab is for generally useful information about what you should do and about relevant strategies and knowledge Ray has in mind. Reading this helps you get a more comprehensive view on what his ideal workflow looks like, and what your ideal contributions look like.

Updating quickly

There’s a learning process for figuring out “when it is good to check if Ray’s stuck?” vs “when is it bad to interrupt his thought process?”. It’s okay if you don’t get it perfectly right at first, by try… “updating a lot, in both directions?” like, if it seemed like something was an unhelpful interruption, try speaking up half-as-often, or half-as-loudly, but then later if I seem stuck, try checking in on me twice-as-often or twice-as loudly, until we settle into a good rhythm.

The "10x" here was meant more to refer to how long it took him to figure it out, than how much better it was. I'm less sure how to quantify how much better.

I'm busy atm but will see if I can get a screeshot from an earlier draft

Thanks! I'll keep this in mind both for potential rewrites here, and for future posts.

Curious how this takes you typically?

Well, this is the saddest I've been since April 1st 2022.

It really sucks that SB 1047 didn't pass. I don't know if Anthropic could have gotten it passed if they had said "dudes this this fucking important, pass it now" instead of "for some reason we should wait until things are already 

It is nice that at least Anthropic did still get to show up to the table, and that they said anything at all. I sure wish their implied worldview didn't seem so crazy. (I really don't get how you can think it's workable to race here, even if you think Phase I alignment is easy, as well as it seeming really wrong to think Phase I alignment is that likely to be easy)

It feels like winning pathways right now mostly route through:

• Some kind of miracle of Vibe Shift (ideally mediated through a miracle of Sanity). I think this needs masterwork-level communication / clarity / narrative setting.
• Just... idk, somehow figure out how to just Solve The Hard Part Real Fast.
• Somehow muddle through with scary demos that get a few key people to change their mind before it's too late.

Oh lol right.

You wouldn't guess it, but I have an idea...

...what.... what was your idea?

I don't know if I'd go as strong as the OP, but, I think you're being the most pro-social if you have a sense of the scale of other things-worth-doing that aren't in the news, and consciously checking how the current News Thing fits into that scale of importance.

(There can be a few different ways to think about importance, which this frame can be agnostic on. i.e. doesn't have to be "global utilitarian in the classical sense")

FYI I do currently think "learn when/how to use your subconcious to process things" is an important tool in the toolbox (I got advice about that from a mentor I went to talk to). Some of the classes of moves here are:

• build up intuitions about when it is useful to background process things vs deliberate-process them
• if your brain is sort of subconsciously wandering in a rut, use a small amount of agency to direct your thoughts in a new direction, but then let them wander once you get them rolling down the hill in that new direction

I feel less optimistic about the "forgetting something on the tip of your tongue", and pretty optimistic about the code debugging.

The feedbackloops in escalating "realness" here for me are:

• Do I identify principles/skills/habits/etc that seem like they should successfully cut down on time spent on things I regularly do?
• Do I successfully identify moments where it seems like I should "think something faster the first time?", by applying a technique?
• Do I do that? Does it seem to save time?

("does it seem to save time?" isn't an ironclad feedbackloop obviously. But, I think it + common sense is at least pretty good)

I've been doing some-kind-of-variant on this since 2023 with the Thinking Physics exercise "reflection portion". Everything in Skills from a year of Purposeful Rationality Practice I think at least somewhat counts as habits that I've gained that allow me to think either think-things-faster, or, think-things-at-all.

I workshopped and ad-hoc "review your thinking for 10 minutes" after various exercises into the ~hour-long exercise you see here, a few months ago. In that time, some new things I try at least sometimes

• Look at my checklist for debugging code, and do the things on it. These so far include:
  • "actually adopt a stance of 'form hypotheses and try to disprove them'"
  • "patiently follow the code all the way up the stack" (instead of bouncing off after the second step)
  • "binary search for where the problem is by commenting out ~half the code in the relevant section."
    
    (these may seem obvious but I'm just not that strong a developer, and exercises like this are the main mechanism by which I've gotten better at basic debugging skills)
• Try the simple dumb thing first. (I still fail to do this an embarrassing amount of time, but am working on it)
• When I notice myself flailing around myopically,
  • a) these days, try getting a Thinking Assistant for the day.
  • b) back in December, when I first was noticing I was struggling to focus, I decided to write the Thinking Assistants post and spin up a Thinking Assistant community. The general form of that is "consider spinning up whole-ass subcommunities to deal with problems." (I knew from previous experience that finding a single thinking assistant was a brittle solution)
• Also when I'm myopically flailing, try forming a more complete model of my constraints (as described in this blogpost), and then solve for them.

The first three things feel like they're straightforwardly working, although it's hard to tell how much they actually speed me up. (Often the thing I would previously do when failing to debug code was "ask someone for help", so it's less that there's a speedup exactly and more that I interrupt my colleagues less)

The fourth one, I feel like I'm still workshopping into a form that reliably works for me, because "make a map of the constraints" is made of a lot of subskills, which vary depending on the situation. I anticipate it turning into something pretty important over the next year but it's too early to tell.

I think this is sort of right, but, if you think big labs have wrong worldmodels about what should be useful, it's not that helpful to produce work "they think is useful", but isn't actually that helpful. (i.e. if you have a project that is 90% likely to be used by a lab, but ~0% likely to reduce x-risk, this isn't obviously better than a project that is 30% likely to be used by a lab if you hustle/convince them, but would actually reduce x-risk if you succeeded.)

I do think it's correct to have some model of how your research will actually get used (which I expect to involve some hustling/persuasion if it involves new paradigms)

I agree that it requires upfront investment, but, a few comments on this post are reminding me "oh right the default thing is that everyone falls into The Meta Trap", wherein people invest in meta things that end up not paying off.

My solution to this is to set standards for myself that involve keeping up a "ship quickly" momentum, and generally aim to spend ~10% of your time on meta.

Mm, that does make sense, thanks for the warning.

Part of my overall flow is "during working hours (where I'm trying to apply this sort of thing), any 'meta' I do is something that needs to pay off within a week." (I let myself do meta in weekends/evenings with less restriction, coming more out of my "hobby/after-hours-self-improvement" budget).

"Have a handle on when you're going overboard and need to focus back on just getting momentum on object-level progress" makes sense as an important foundational skill here. 

On my day-to-day, I do the 5-10 minute version of this in the morning.

Do you feel right now like you found a happy balance, or are you kinda in "recovering alcoholic, not even one drink of think-it-faster-meta is safe?"

I think you totally can use rationality (that is: "intentionally choosing cognitive algorithms that perform better" for practical things, it's just that for most practical things, "practice being better at rationality" is less useful than "practice being better at the-thing-itself."

If you find rationality practice intrinsically rewarding (as I, and probably many people on this site do), then yeah you should do that. But, purposeful practice is particularly exhausting and effortful. I think most people aren't doing purposeful practice because they anticipate it being exhausting and effortful and also not super paying off compared to other things they could do, and they are probably correct. 

If you have chosen to invest a bunch in rationality, yes you totally should see benefits in practical things.

Also: My actual background / college degree was in filmmaking so I have at least some context on that.

What is my immediate goal? To get good at general problem solving in real life, which means better aligning instrumental activities towards my terminal goals. My personal terminal goal would be to make films and music videos that are pretty and tell good stories. I could list maybe 30 metacognitive deficiencies I think I have, but that would be of no interest to anyone.

What is my 1-3 year goal? Make very high production value music videos that tell interesting stories.

Cool, that's helpful.

I'm probably answering this question in the wrong way but this particular question is not helpful to me, because I can only describe the results - the end result is I make videos with higher production values that communicate better stories.

This was a fine answer. "The end result is that I make videos with higher production values that communicate better stories." (To fit my question frame, I'd say "people would observe me making music videos somehow-or-other, and then, those music videos being higher quality than they otherwise would.")

So, it might totally be that General Problem Solving is the skill it makes sense for you to get better at, but I wouldn't assume that from the get-go. You might instead just directly study filmmaking.

I realize this is a bit annoying given that you did make an honest attempt at the exercise I laid out (which I think is super cool and I appreciate, barely anyone does that). Before it makes sense to figure out how to develop general problem solving or metacognition, it's important to doublecheck whether those are the appropriate tool for your goal.

So, (I mean this as an earnest question, not like a gotcha) why are you currently interested in general problem solving (as opposed to filmmaking?) Is it because general problem solving is intrinsically interesting/rewarding to you (if you could find a path to doing so?). Or because it just seemed pretty likely to be the a good step on your journey as a filmmaker? Or just because I gave a prompt to see if you could figure out a way to apply general problemsolving to your life, and there was at least some appeal to that?

I do think basically none of this makes sense if you don't have some particular flavor of ambitious goals. If you don't have ambitious goals that (probably) depend on leveling up a bunch, then yeah, don't do that. (unless you intrinsically value the leveling up)

If you sort-of-kind-of have ambitious goals, but you're not totally bought into them, or have mixed feelings about them, then you probably want to somehow resolve that (which I don't think is a "deliberate practice" shaped problem, more like a therapy/emotional processing problem, or a "just find a fun project or better work environment that makes things more naturally motivating" problem)

The first thing that comes up when I look at this is I'm not that sure what your goals are, and I'm not sure whether the sort of thing I'm getting at in this post is an appropriate tool. 

You say:

I suspect that if I set myself a weekly challenge of editing a sequence from found footage that pertained to a pseudo-random topic of theme that this might possibly pay dividends in terms that generalize to metacognition.

This sounds like you're seeing the metacognition as more like a terminal goal, than an instrumental goal (which I think doesn't necessarily make sense).

I do think metacognition is generally useful, but in an established domain like video-editing or self-promotion in a fairly understood field, there are probably object-level skills you can learn that pay off faster than metacognition. (Most of the point of metacognition there is to sift out the "good" advice from the bad).

I want to separate out...

• purposefully practicing metacognition
• purposefully practicing particular object level skills, such as videoediting or self-promotion (which involves figuring out what the subskills are that you can get quicker feedback on)
• purposefully practice "purposeful practice", such that you get better at identifying subskills in various (not-necessarily-metacognition-y) domains.

...as three different things that might (or might not) be the right thing for you.

Right now I can't really tell what your goal is, so I would first just ask "what is it you are trying to achieve?" 1-3 years from now, how would you know if [whatever kind of practice you did] turned out to work? (I think it's helpful to imagine "what would an outside observe watching a video recording see happening differently")

That... is not supposed to be visible at all. Is it still there when you refresh?

Nod. 

The thing I'm currently hearing you saying (either in contrast to this post, or flagging that this post doesn't really acknowledge) is:

• there's a bunch of technical knowledge (which is a different type of thing than "metacognitive skill training", and which also requires a ton of work to master)
• the amount of work going into all of this is just, like, a ton, and the phrasing in the post (and maybe other conversations with me) doesn't really come close to grappling with the enormity of it?

Are there other things you meant? 

Probably will have a bunch more to say, but immediate question is "what's your story about the gears for Soares/Turner?"

Okay, yeah this should have been dealt with in the OP. I have thoughts about this but I did write the essay in a bit of a rush. I agree this is one of the strongest objections.

I had someone do some review of the transfer learning literature. There was nonzero stuff there seemed to demonstrably work. But mostly it just seemed like we just don't really have good experiments on the stuff I'd have expected to work. (And, the sorts of experiments that I'd expect to work are quite expensive)

But I don't think "universal transfer learning" is quite the phrase here. 

If you learn arithmetic, you (probably) don't get better at arbitrary other skills. But, you get to use arithmetic wherever it's relevant. You do have to separately practice "notice places where arithmetic is relevant." (like it may not occur to you that many problems you face are actually math problem. Or, you might need an additional skill like Fermi Estimation to turn them into math problems)

The claim here is more like: "Noticing confusion", "having more than 1 hypothesis", "noticing yourself flinching away from a thought that'd be inconvenient" are skills that show up in multiple domains.

My "c'mon guys" here is not "c'mon the empirical evidence here is overwhelming." It's more like "look, which world do you actually expect to result in you making better decisions faster: the one where you spend >0 days on testing and reflecting on your thinking in areas where there is real feedback, or the one where you just spend all your time on 'object level work' that doesn't really have the ability to tell you you were wrong?".

(and, a host of similar questions, with the meta question is "do you really expect the optimal thing here to be zero effort on metacognition practice of some kind?")

Obviously there is a question of how much time to spend on this is optimal, and it's definitely possible (and perhaps common) to go overboard. But I also think it's not too hard to figure out how to navigate that.

"Isn't it weird an arbitrary to make a front-and-center choice 'hey wanna turn off the AI content'?" yes, which is why we didn't do it the last couple times we talked about it.

Yes but it's an out-of-the-way button mixed in with a bunch of buttons, as opposed to like, kinda the primary choice you're expected to make on the site.

I do think we should probably just build the "AI" "Some AI" "No AI" buttons. It felt kinda overkill the last time I considered it but seems probably worth it.

Okay I think I have more of an idea where you're coming from. (although I get some sense of something being at stake for you here that I still don't understand).

I maybe want to clarify, when I suggested "taboo bad faith" as a title, I was imagining a pretty different purpose for the title (and I don't super strongly defend that title as a good one here). I was looking for a succinct way of describing the suggestion "when you want to accuse someone of bad faith, you should probably say something more specific instead." (i.e. "Taboo Bad Faith" is a recommendation for what to do in the wild, rather than "a thing the post itself was doing.")

(This isn't a flagrant violation of the norms but this comment gives me a vibe of trying to score points more than problem solve)

I maybe also want to note: The most interesting argument against "deliberate practice" as a frame I've read was from Common Cog, in his post Problems with Deliberate Practice.

This was the post that introduced me to the term "purposeful practice", which is "deliberate practice when you don't really know what you're doing yet or how to train effectively." I do think most of what I'm advocating for is in fact purposeful practice (but, I'm holding myself to the standard of pushing towards a deliberate practice curriculum)

He later has a post reviewing the book Accelerate Expertise, in which he advocates throwing out the "deliberate practice" paradigm, because it's dependent on brittle skill trees of subskills that are hard to navigate if there isn't an established literature, or if (in the case of the military in Accelerated Expertise), you find that circumstances change often enough that rebuilding the skill tree over and over isn't practical.

But, the solution they end up with there is "throw people into simulations that are kind of intense and overwhelming, such that they are forced to figure out how to achieve a goal in a way that organically works for them." This is actually not that different from my approach (i.e. finding confusing-challenges that are difficult enough you will need to learn to navigate confusion and creative strategy to solve, and then do reflections /  "Think It Faster" exercises. 

I see these two approaches as rounding out each other. While doing Toy exercises (interleaved with your day job), you can learn to notice subskills that are bottlenecking you, and focus directly on those. This is more of a guess than a claim, but I expect that trying to combine the two approaches will yield better results.

Ah thanks. I think I might have seen that a long time ago, but when I was in a different headspace.

I am sort of confused at what you got from this post. You say "But also, it doesn't do the core intellectual work of replacing a pointer with its substance.", but, I think he explicitly does that here?

What does "bad faith" mean, though? It doesn't mean "with ill intent." Following Wikipedia, bad faith is "a sustained form of deception which consists of entertaining or pretending to entertain one set of feelings while acting as if influenced by another." The great encyclopedia goes on to provide examples: the solider who waves a flag of surrender but then fires when the enemy comes out of their trenches, the attorney who prosecutes a case she knows to be false, the representative of a company facing a labor dispute who comes to the negotiating table with no intent of compromising.

That is, bad faith is when someone's apparent reasons for doing something aren't the same as the real reasons. This is distinct from malign intent. The uniformed solider who shoots you without pretending to surrender is acting in good faith, because what you see is what you get: the man whose clothes indicate that his job is to try to kill you is, in fact, trying to kill you.

It feels like you wanted some entirely different post, about how to navigate when someone accuses someone of bad faith, for a variety of possible definitions of bad faith? (As opposed to this post, which is mostly saying "Avoid accusing people of bad faith. Instead, do some kind of more specific and useful thing." Which honestly seems like good advice to me even for most people who are using the phrase to mean something else. "I disapprove of what you're doing here and am leaving now" seems totally fine)

I edited the OP but wanted to add for people who missed it (bold part is new)

(Please generally be cautious on LessWrong talking about politics. I am interested in people commenting here who have read the LessWrong Political Prerequisites sequence. I'll be deleting or at least unhesitatingly strong downvoting comments that seem to be doing unreflective partisan dunking)

((But, that's not meant to mean "don't talk about political actions." If this is as big a deal as it sounds, I want to be able to talk about "what to do do?". But I want that talking-about-it to feel more like practically thinking through an action space, than blindly getting sucked into a political egregore))

(someone else tagged this as a longfom review. I did originally plan to include a second half of this post that would have felt like it qualified as such, but I think the current form is ranty enough and light on "self review" enough that I don't feel good about it taking up a slot in the Frontpage Review Widget)

To be clear, I think it's okay to be more political. What I don't want is "unreflectively partisanly political." (Maybe try DMing what you had in mind to me and I'll see if it feels productive)

Presumably the "someone dies" means like, within a few years, and not because of x-risk or a major pandemic.

Someone just noted that the Review Voting widget might imply that the "Jan 5" end time is meant to be inclusive of a full 24 hours from now, which wasn't the intent, but given that people may have been expecting that, and that the consequences for Not Extending aren't particularly bad, I'm going to give people another 24 hours.

Meanwhile, I said it elsewhere but will say again here: if that there were any posts you were blocked from commenting on that you want to write a review of... I forgot to fix that in the code and it'll take a little while to fix, but meanwhile you can write a shortform comment and DM me (in which I will manually set it to be a review of that post), and/or write a top level post and tag it with 2023 Longform Reviews (which will cause it to appear in the Frontpage Review widget)

(Note: I currently plan to make it so, during the Review, anyone write Reviews on a post even if normally blocked on commenting. Ideally I'd make it so they can also comment on Review comments. I haven't shipped this feature yet but hopefully will soon)

Blarg, I totally failed to followup on this. It's pretty last minute now, but I'll work on a fix for it soon if for no reason other than to make sure it gets done for future years.

Meanwhile, if you're a blocked person who wants to write a review of this post, you can write a shortform comment, DM me, and I'll manually set it as a review of this post. You can also write a top level post tagged with 2023 Longform Reviews.

I'm working on some under-the-hood changes to the Best of LessWrong vote tallying. I haven't actually done much user-interviewing of the new Best of LessWrong page. We changed it for last year's Review Winner announcement, and then I made some additional changes a over the past few months.

I'm curious to hear about:

• "Do you use the Best of LessWrong page? Does it feel useful or interesting to you?"
• "What would/did you expect when you clicked on a link to a 'Best of LessWrong' page?"
• "What would/do you want out of a Best of LessWrong page?"
• "How do you feel about the current implementation of the page? Anything you wish was differnet?"

I followed up on this with a year exploring various rationality exercises and workshops. My plans and details have evolved a bunch since then, but I still think the opening 7 bullets (i.e. "Deliberate Practice, Metacognition" etc, with "actually do the goddamn practice" and "the feedbackloop is the primary product") are quite important guiding lights.

I've written up most of my updates as they happened over the year, in:

• Rationality Research Report: Towards 10x OODA Looping?
• "Fractal Strategy" workshop report
• The Cognitive Bootcamp Agreement
• Skills from a year of Purposeful Rationality Practice 

The biggest overall updates since this post:

Fluent enough for your day job. 

The primary aim of my workshops and practice is to get new skills fluent enough that you can appy them to your day job, because that's where it's practical to deliberate practice in a way that pays for itself rather than being an exhausting extra thing you do.

"Fluency at new skill that seem demonstrably useful" is also a large enough effect size that there's at least something you can measure near term, to get a sense of whether the workshop is working.

Five minute versions of skills. 

Relatedly: many skills have elaborate, comprehensive versions that take ~an hour to get the full value of, but you're realisitically not going to do those most of the time. So it's important to boil them down into something you can do in 5 minutes (or, 30 seconds).

Morning Orient Prompts.

A thing I've find useful for myself, and now think of as one of the primary goal of the workshop to get people to try out, is a "morning orient prompt list" that you do every day.

It's important that it be every day, even when you don't need it too much, so that you still have a habit of metacognition for the times you need it (but, when you don't need it too much, it's fine/good to do a very quick version of it)

It's useful to have a list of explicit prompts, because that gives you an artifact that's easier to iterate on.

What about RCT-style science?

The original post noted an expensive plan that would be "actual science shaped", which would give a clearer sense of whether rationality training worked. I do still think that's the right goal to be aspiring to. 

It is quite an expensive goal – not just in my effort but in a lot of smart people's time. 

Oddly, the two major objections I frequently get are both "why haven't you done an RCT?" and "Doing [the real RCT version of this] is too expensive so the plan is bad."

My actual belief right now is that the most important feedbackloops to be developing are more focused on "how to track if this is helping with your day-job", because I think that's what most people's cruxes actually are (and, should be). RCTs would at best tell you what works on average for People of Some Reference Class, but still don't actually tell you personally if it's worthwhile for you.

A lot of my focus right now is helping people who have attended a workshop to integrate the practice into their daily life, and seeing what sort of problems come up there and how to deal with them. (Where my current partial-answers to that are the previous three sections)

This story is moved me a lot, and I am giving it a substantial vote,

But... I do still really wish this line...

And they turn away and go back to their work—all except for one, who brushes past the grasshopper and whispers “Meet me outside at dusk and I’ll bring you food. We can preserve the law and still forgive the deviation.”

Came where it originally was located, significantly later in the post, after these sections:

The ants start to receive dozens of requests for food, then hundreds—and while many are fraudulent, enough are real that they are moved to act. In order to set incentives correctly, the ants decide to only give food to those who can prove that they lost their own food supplies through no fault of their own, and set up a system for vetting claims.

This works well for a time—but as fraudsters grow more sophisticated, the ants’ bureaucratic requirements grow more onerous. In order to meet them, other creatures start to deposit their food in large group storehouses which can handle the administrative overhead. But now the food supply is exposed to systemic risk if the leaders of those storehouses make poor decisions, whether from carelessness or greed.

One year several storehouses fail; in trying to fill the shortfall, the ants almost run out of food for themselves. To avoid that ever happening again, they set up stringent regulations and oversight of permissible storehouses, funded by taxes levied throughout the year. At first this takes only a small proportion of their labor—but as their regulatory apparatus inevitably grows, they need to oversee more and more aspects of the ecosystem, and are called upon to right more and more injustices. Eventually the ants—originally the most productive of all creatures—stop producing any food of their own, so busy are they in tending to the system they’ve created.

...

“And therefore, to reduce risks from centralization, and to limit our own power, we can’t give you any food”, the ants conclude. And they turn away and go back to their work, with a quiet sense of satisfaction that they’ve given such legible and defensible reasons for focusing on their own problems and keeping all the food for themselves.

In it's original positioning, the "We can preserve the law and still forgive the deviation" line felt like the climax of a mini-arc. Previously Richard had said he moved it for two reasons:

Firstly, if the grasshopper was just unlucky, then there's no "deviation" to forgive—it makes sense only if the grasshopper was culpable. Secondly, the earlier parts are about individuals, and the latter parts are about systems—it felt more compelling to go straight from "centralized government" to "locust war" than going via an individual act of kindness.

But, what I find particularly meaningful about it is that even when giant systems have evolved that are turning grotesque with their burueacratic weight... and when maybe those systems are necessary or inevitable or something... individuals can still choose to just... do a nice thing, when they have local knowledge suggesting it's the right thing to do in this case. (And meanwhile, I had no problem backfilling and appropriate history for this particular snippet. I can imagine a grasshopper who slacked off a bit, and also got a bit unlucky, and also seems to have earnestly learned and will make better choices next time)

I'll add one more gear here: I think you can improve on how much you're satisfying all three tradeoffs at once – writing succinctly and clearly and distilling complex things into simpler ones are skills. But, those things take time (both to get better at the skill, and to apply the skill).

LessWrong certainly could do better than we currently have, but we've spent 100s of hours on things like 

• make the new user guide, and link new users to it
• make the rate limits link people to pages that explain more about why you're rate limited
• explain why the moderation norms are what they are

etc. We could put more even more effort into it, but, well, there's also a lot of other stuff to do.

+9. This argues that some key puzzle pieces of genius include "solitude," and "sitting with confusion until open-curiosity allows you to find the right questions." This feels like an important key that I'm annoyed at myself for not following up on more. 

The post is sort of focused on "what should an individual do, if they want to cultivate the possibility of genius?".

One of the goals I have, in my work at Lightcone, is to ask "okay but can we do anything to foster genius at scale, for the purpose of averting x-risk?". This might just be an impossible paradox, where trying to make it on purpose intrinsically kills it before it can flower. I think it might be particularly impossible to do at massive scale – if you try to build a system for 1000s of geniuses, that system can't help but develop the pathologies that draw people into fashions that stifle the kind of thinking you need.

But, it doesn't seem impossible to foster-genius-on-the-margin at smallish scales.

Challenges that come immediately to mind:

1. how do you actually encourage, or teach people (or arrange for them to discover for themselves), how to sit with confusion, and tease out interesting questions?
2. by default, if you suggest a bunch of people spend time alone in thought, I think you mostly end up wasting a lot of people's time (and possibly destroying their lives?). Many geniuses don't seem that happy, and people who don't actually have the right taste/generators for that thinking to be productive I bet end up even more unhappy. If you try to filter for "has enough taste and/or IQ to have a decent shot", you probably immediately reintroduce all the problems this post argues against.
3. somehow the people need enough money to live, but any system for allocating money for this would either be hella exploitable, or very prone to centralization/fashion/goodhart collapse.
4. eventually, when an idea seems genuinely promising, your creativity needs to survive contact with others forming expectations of you.

I've been working on "how to encourage sitting with confusion." 

I think I've been less focused on "how to sit with open curiosity." (Partly because I am bad at it, and it feels harder to thread a needle between "enough open curiosity to identify novel important questions and reframings" without just failing to direct your attention towards the (rather quite dire and urgent) problems the x-risk community needs to figure out)

(But, Logan Strohl seems to be doing a pretty good job at teaching the cultivation of curiosity, which at least gives me hope that it is possible)

I think the unfortunate answer to the third bullet is "well, probably people basically need to self-finance during a longish period where they don't have something legibly worth funding." (But, this maybe suggests an ecosystem where it's sort of normal to have a day job that doesn't consume all your time)

...

What sort of in-person community is good for intellectual progress?

Notably: Lightcone had been working on the sort of coworking space this post argues against. We did stop work on the coworking space before this post even came out. I did have some concrete thoughts when reading this "maybe, with Lighthaven, we'll be able to correct this – Lighthaven more naturally fits into an ecosystem where people are off working mostly alone, but periodically they gather to share what they're working on, in a space that is optimized for exploring curiosity in a spacious way."

But, we haven't really put much attention into "foster a thing where more people make some attempt to go off and think alone on purpose" part.

+4. I most like the dichotomoy of "stick to object level" vs "full contact psychoanalysis." And I think the paragraphs towards the end are important:

The reason I don't think it's useful to talk about "bad faith" is because the ontology of good vs. bad faith isn't a great fit to either discourse strategy.

If I'm sticking to the object level, it's irrelevant: I reply to what's in the text; my suspicions about the process generating the text are out of scope.

If I'm doing full-contact psychoanalysis, the problem with "I don't think you're here in good faith" is that it's insufficiently specific. Rather than accusing someone of generic "bad faith", the way to move the discussion forward is by positing that one's interlocutor has some specific motive that hasn't yet been made explicit—and the way to defend oneself against such an accusation is by making the case that one's real agenda isn't the one being proposed, rather than protesting one's "good faith" and implausibly claiming not to have an agenda.

I think maybe the title of this post is misleading, and sort of clickbaity/embroiled in a particular conflict that is, ironically, a distraction. 

The whole post (admittedly from the very first sentence) is actually about avoiding the frame of bad faith (even if the post argues that basically everyone is in bad faith most of the time). I think it's useful if post titles convey more of a pointer to the core idea of the post, which gives it a shorthand that's more likely to be remembered/used when the time is right. ("Taboo bad faith"? "The object level, vs full-contact psychanalysis"? Dunno. Both of those feel like they lose some nuance I think Zack cares about. But, I think there's some kind of improvement here)

...

That said: I think the post is missing something that lc's comment sort of hints at (although I don't think lc meant to imply my takeaway)

Some people in the comments reply to it that other people self-deceive, yes, but you should assume good faith. I say - why not assume the truth, and then do what's prosocial anyways?

I think the post does give a couple concrete strategies for how to navigate the, er, conflict between conflict and truthseeking, which are one flavor of "prosocial." I think it's missing something about what "assume good faith" is for, which isn't really covered in the post. 

The problem "assume good faith" is trying to solve is "there are a bunch of human tendencies to get into escalation spirals of distrust, and make a bigger deal about the mistakes of people you're in conflict with." You don't have to fix this with false beliefs, you can fix this by shifting your relational stance towards the person and the discussion, and holding the possibility more alive that you've misinterpreted them or are rounding them off to and incorrect guess as to what their agenda is. 

I think the suggestions this post makes on how to deal with that are reasonable, but incomplete, and I think people benefit from also having some tools that more directly engage with "your impulse to naively assume bad faith is part of a spirally-pattern that you may want to step out of somehow." 

That's not really what I had in mind, but I had in mind something less clear than I thought. The spirit is about "can the AI come up with novel concepts",

I think one reason I think the current paradigm is "general enough, in principle", is that I don't think "novel concepts" is really The Thing. I think creativity / intelligence mostly is about is combining concepts, it's just that really smart people are 

a) faster in raw horsepower and can handle more complexity at a time

b) have a better set of building blocks to combine or apply to make new concepts (which includes building blocks for building better building blocks)

c) have a more efficient search for useful/relevant building blocks (both metacognitive and object-level).

Maybe you believe this, and think that "well yeah, it's the efficient search that's the important part, which we still don't actually have a real working version of?"?

It seems like the current models have basically all the tools a moderately smart human have, with regards to generating novel ideas, and the thing that they're missing is something like "having a good metacognitive loop such that they notice when they're doing a fake/dumb version of things, and course correcting" and "persistently pursue plans over long time horizons." And it doesn't seem to have zero of either of those, just not enough to get over some hump.

I don't see what's missing that a ton of training on a ton of diverse, multimodal tasks + scaffoldin + data flywheel isn't going to figure out.

One major counterargument here is "is control a necessary piece of the 'solve alignment in time plan'"? Like, it may be "5-10x less important" than dealing with slop, but, still, in if you didn't eventually solve both you don't get useful carefully-implemented-slightly-superhuman work done, and it (might be) that our surviving worlds look like either that, or "get a serious longterm pause."

I think in my case issue was I just totally didn't know how to pronounce lsuser. 

Oh, I maybe flipped the sign on what you meant to be saying.

Not that I have particularly clear evidence about the inner workings of the sphere here, but, what's your reasoning for thinking:

It's a loose group of people who read each others' blogs and argued with each other on the same Discord servers. Ziz isn't in charge in any meaningful sense.