Posts

Prompts for Big-Picture Planning 2024-04-13T03:04:24.523Z
"Fractal Strategy" workshop report 2024-04-06T21:26:53.263Z
One-shot strategy games? 2024-03-11T00:19:20.480Z
Rationality Research Report: Towards 10x OODA Looping? 2024-02-24T21:06:38.703Z
Exercise: Planmaking, Surprise Anticipation, and "Baba is You" 2024-02-24T20:33:49.574Z
Things I've Grieved 2024-02-18T19:32:47.169Z
CFAR Takeaways: Andrew Critch 2024-02-14T01:37:03.931Z
Skills I'd like my collaborators to have 2024-02-09T08:20:37.686Z
"Does your paradigm beget new, good, paradigms?" 2024-01-25T18:23:15.497Z
Universal Love Integration Test: Hitler 2024-01-10T23:55:35.526Z
2022 (and All Time) Posts by Pingback Count 2023-12-16T21:17:00.572Z
Raemon's Deliberate (“Purposeful?”) Practice Club 2023-11-14T18:24:19.335Z
Hiring: Lighthaven Events & Venue Lead 2023-10-13T21:02:33.212Z
"The Heart of Gaming is the Power Fantasy", and Cohabitive Games 2023-10-08T21:02:33.526Z
Related Discussion from Thomas Kwa's MIRI Research Experience 2023-10-07T06:25:00.994Z
Thomas Kwa's MIRI research experience 2023-10-02T16:42:37.886Z
Feedback-loops, Deliberate Practice, and Transfer Learning 2023-09-07T01:57:33.066Z
Open Thread – Autumn 2023 2023-09-03T22:54:42.259Z
The God of Humanity, and the God of the Robot Utilitarians 2023-08-24T08:27:57.396Z
Book Launch: "The Carving of Reality," Best of LessWrong vol. III 2023-08-16T23:52:12.518Z
Feedbackloop-first Rationality 2023-08-07T17:58:56.349Z
Private notes on LW? 2023-08-04T17:35:37.917Z
Exercise: Solve "Thinking Physics" 2023-08-01T00:44:48.975Z
Rationality !== Winning 2023-07-24T02:53:59.764Z
Announcement: AI Narrations Available for All New LessWrong Posts 2023-07-20T22:17:33.454Z
What are the best non-LW places to read on alignment progress? 2023-07-07T00:57:21.417Z
My "2.9 trauma limit" 2023-07-01T19:32:14.805Z
Automatic Rate Limiting on LessWrong 2023-06-23T20:19:41.049Z
Open Thread: June 2023 (Inline Reacts!) 2023-06-06T07:40:43.025Z
Worrying less about acausal extortion 2023-05-23T02:08:18.900Z
Dark Forest Theories 2023-05-12T20:21:49.052Z
[New] Rejected Content Section 2023-05-04T01:43:19.547Z
Tuning your Cognitive Strategies 2023-04-27T20:32:06.337Z
"Rate limiting" as a mod tool 2023-04-23T00:42:58.233Z
LessWrong moderation messaging container 2023-04-22T01:19:00.971Z
Moderation notes re: recent Said/Duncan threads 2023-04-14T18:06:21.712Z
LW Team is adjusting moderation policy 2023-04-04T20:41:07.603Z
Abstracts should be either Actually Short™, or broken into paragraphs 2023-03-24T00:51:56.449Z
Tabooing "Frame Control" 2023-03-19T23:33:10.154Z
Dan Luu on "You can only communicate one top priority" 2023-03-18T18:55:09.998Z
"Carefully Bootstrapped Alignment" is organizationally hard 2023-03-17T18:00:09.943Z
Prizes for the 2021 Review 2023-02-10T19:47:43.504Z
Robin Hanson on "Explaining the Sacred" 2023-02-06T00:50:58.490Z
I don't think MIRI "gave up" 2023-02-03T00:26:07.552Z
Voting Results for the 2021 Review 2023-02-01T08:02:06.744Z
Highlights and Prizes from the 2021 Review Phase 2023-01-23T21:41:21.948Z
Compounding Resource X 2023-01-11T03:14:08.565Z
Review AI Alignment posts to help figure out how to make a proper AI Alignment review 2023-01-10T00:19:23.503Z
The 2021 Review Phase 2023-01-05T07:12:24.251Z
What percent of people work in moral mazes? 2023-01-01T04:33:43.890Z

Comments

Comment by Raemon on Express interest in an "FHI of the West" · 2024-04-25T17:54:15.604Z · LW · GW

Wow the joke keeps being older.

Comment by Raemon on LessOnline (May 31—June 2, Berkeley, CA) · 2024-04-23T17:41:44.152Z · LW · GW

That's actually not (that much of) a crux for me (who also thinks it's mildly manipulative, but, below the threshold where I feel compelled to push hard for changing it).

Comment by Raemon on Thoughts on seed oil · 2024-04-21T21:24:48.181Z · LW · GW

Curated.

I do sure wish this question had easier answers, but I appreciate this post laying out a lot of the evidence.

I do have some qualms about the post, in that while it's pretty thorough on the evidence re: seed oils, it sort of handwavily assumes some other nutrition stuff about processed foods that (I'm willing to bet) also have highly mixed/confusing evidence bases. But, still thought the good parts of the post were good enough to be worth curating.

Comment by Raemon on "Fractal Strategy" workshop report · 2024-04-21T21:08:00.813Z · LW · GW

I'm trying to decide whether to rename this post "Metastrategy Workshop." Fractal Strategy happened to make sense for the skillset I had put together at the time but I don't know that it's what I'm going to stick with.

Comment by Raemon on "Fractal Strategy" workshop report · 2024-04-21T21:07:21.036Z · LW · GW

One thing to remember is I (mostly) am advocating playing each game only once, and doing a variety of games/puzzles/activities, many of which should just be "real-world" activities, as well as plenty of deliberate Day Job stuff. Some of them should focus on resource management, and some of that should be "games" that have quick feedback loops, but it sounds like you're imagining it being more focused on the goodhartable versions of that than I think it is.

(also, I think multiplayer games where all the information is known is somewhat an antidote to these particular failure modes? even when all the information is known, there's still uncertainty about how the pieces combine together, and there's some kind of brute-reality-fact about 'well, the other players figured it out better than you')

Comment by Raemon on [Linkpost] Practically-A-Book Review: Rootclaim $100,000 Lab Leak Debate · 2024-04-19T02:19:42.852Z · LW · GW

Curated. (In particular recommending people click through and read the full Scott Alexander post)

I've been tracking the Rootclaim debate from the sidelines and finding it quite an interesting example of high-profile rationality. 

I have a friend who's been following the debate quite closely and finding that each debater, while flawed, had interesting points that were worth careful thought. My impression is a few people I know shifted from basically assuming Covid was probably a lab-leak, to being much less certain.

In general, I quite like people explicitly making public bets, and following them up with in-depth debate.

Comment by Raemon on Raemon's Shortform · 2024-04-18T22:00:31.516Z · LW · GW

What would a "qualia-first-calibration" app would look like?

Or, maybe: "metadata-first calibration"

The thing with putting probabilities on things is that often, the probabilities are made up. And the final probability throws away a lot of information about where it actually came from.

I'm experimenting with primarily focusing on "what are all the little-metadata-flags associated with this prediction?". I think some of this is about "feelings you have" and some of it is about "what do you actually know about this topic?"

The sort of app I'm imagining would help me identify whatever indicators are most useful to me. Ideally it has a bunch of users, and types of indicators that have been useful to lots of users can promoted as things to think about when you make predictions.

Braindump of possible prompts:

– is there a "reference class" you can compare it to?

– for each probability bucket, how do you feel? (including 'confident'/'unconfident' as well as things like 'anxious', 'sad', etc)

– what overall feelings do you have looking at the question?

– what felt senses do you experience as you mull over the question ("my back tingles", "I feel the Color Red")

...

My first thought here is to have various tags you can re-use, but, another option is to just do totally unstructured text-dump and somehow do factor analysis on word patterns later?

Comment by Raemon on Eli's shortform feed · 2024-04-17T22:30:54.385Z · LW · GW

lol at the approval/agreement ratio here. It does seem like this is a post that surely gets something wrong.

Comment by Raemon on RTFB: On the New Proposed CAIP AI Bill · 2024-04-17T22:15:19.712Z · LW · GW

I think I have a different overall take than Ben here, but, the frame I think makes sense here is to be like: "Deontological injuctions are guardrails. There are hypothetical situations (and, some real situations) where it's correct to override them, but the guardrail should have some weight and for more important guardrails, you need a clearer reasoning for why avoiding it actually helps."

I don't know what I think about this in the case of a country passing laws. Countries aren't exactly agents. Passing novel laws is different than following existing laws. But, I observe:

  • it's really hard to be confident about longterm consequences of things. Consequentialism just isn't actually compute-efficient enough to be what you use most of the time for making decisions. (This includes but isn't limited to "you're contemplating crazy sounding actions for strange sounding reasons", although I think has a similar generator)
  • it matters just not what you-in-particular-in-a-vacuum do, in one particular timeslice. It matters how complicated the world is to reason about. If everyone is doing pure consequentialism all the time, you have to model the way each person is going to interpret consequences with their own special-snowflake worldview. Having to model "well, Alice and Bob and Charlie and 1000s of other people might decide to steal from me, or from my friends, if the benefits were high enough and they thought they could get away with it" adds a tremendous amount of overhead.
     

You should be looking for moral reasoning that makes you simple to reason about, and that perform well in most cases. That's a lot of what deontology is for.

Comment by Raemon on Raemon's Shortform · 2024-04-17T19:12:17.760Z · LW · GW

There's a skill of "quickly operationalizing a prediction, about a question that is cruxy for your decisionmaking."

And, it's dramatically better to be very fluent at this skill, rather than "merely pretty okay at it."

Fluency means you can actually use it day-to-day to help with whatever work is important to you. Day-to-day usage means you can actually get calibrated re: predictions in whatever domains you care about. Calibration means that your intuitions will be good, and _you'll know they're good_.

Fluency means you can do it _while you're in the middle of your thought process_, and then return to your thought process, rather than awkwardly bolting it on at the end.

I find this useful at multiple levels-of-strategy. i.e. for big picture 6 month planning, as well as for "what do I do in the next hour."

I'm working on this as a full blogpost but figured I would start getting pieces of it out here for now.

A lot of this skill is building off on CFAR's "inner simulator" framing. Andrew Critch recently framed this to me as "using your System 2 (conscious, deliberate intelligence) to generate questions for your System 1 (fast intuition) to answer." (Whereas previously, he'd known System 1 was good at answering some types of questions, but he thought of it as responsible for both "asking" and "answering" those questions)

But, I feel like combining this with "quickly operationalize cruxy Fatebook predictions" makes it more of a power tool for me. (Also, now that I have this mindset, even when I can't be bothered to make a Fatebook prediction, I have a better overall handle on how to quickly query my intuitions)

I've been working on this skill for years and it only really clicked together last week. It required a bunch of interlocking pieces that all require separate fluency:

1. Having three different formats for Fatebook (the main website, the slack integration, and the chrome extension), so, pretty much wherever I'm thinking-in-text, I'll be able to quickly use it.

2. The skill of "generating lots of 'plans'", such that I always have at least two plausibly good ideas on what to do next.

3. Identifying an actual crux for what would make me switch to one of my backup plans.

4. Operationalizing an observation I could make that'd convince me of one of these cruxes.

Comment by Raemon on Moving on from community living · 2024-04-17T17:10:16.812Z · LW · GW

I feel sort of empathetically sad that there wasn’t a way to make it work, but that all makes sense.

Living at a group house seems really important for my psychological well-being, though I imagine if I was living with a partner AND kids that’d be a big enough reroll on social circumstances I don’t know what to expect.

Comment by Raemon on "Fractal Strategy" workshop report · 2024-04-13T17:57:40.755Z · LW · GW

Yeah. 

"did you remember to make any quantitative estimate at all?"

I'm actually meaning to ask the question "did you estimate help you strategically?" So, if you get two estimates wildly wrong, but they still had the right relatively ranking and you picked the right card to draft, that's a win.

Also important: what matters here is not whether you got the answer right or wrong, it's whether you learned a useful thing in the process that transfers (and, like, you might end up getting the answer completely wrong, but if you can learn something about your thought process that you can improve on, that's a bigger win.

Comment by Raemon on "Fractal Strategy" workshop report · 2024-04-13T04:25:08.576Z · LW · GW

I'm not quite sure what things you're contrasting here.

The skills I care about are:

  • making predictions (instead of just doing stuff without reflecting on what else is likely to happen)
  • thinking about which things are going to be strategically relevant
  • thinking about what resources you have available and how they fit together
  • thinking about how to quantitatively compare your various options

And it'd be nice to train thinking about that in a context without the artificialness of gaming, but I don't have great alternatives. In my mind, the question is "what would be a better way to train those skills?", and "are simple strategy games useful enough to be worth training on, if I don't have better short-feedback-cycle options?"

 (I can't tell from your phrasing so far if you were oriented around those questions, or some other one)

Comment by Raemon on "Fractal Strategy" workshop report · 2024-04-13T02:49:21.545Z · LW · GW

Basically: yep, a lot of skills here are game design specific and not transfer. But, I think a bunch of other skills do transfer, in particular in a context where the you only play Luck Be a Landlord once (as well as 2-3 other one-shot games, and non-game puzzles), but then also follow it up the next day with applying the skills in more real-world domains.

Few people are playing videogames to one-shot them, and doing so requires a different set of mental muscles than normal. Usually if you play Luck Be a Landlord, you'll play it one or twice just to get the feel for how the game works, and by the time you sit down and say "okay, now, how does this game actually work?" you'll already have been exposed to the rough distribution of cards, etc.

In one-shotting, you need to actually spell out your assumptions, known unknowns, and make guesses about unknown unknowns. (Especially at this workshop where the one-shotting comes with '"take 5 minutes per turn, make as many fatebook predictions as you can for the first 3 turns, and then for the next 3 turns try to make two quantitative comparisons".

The main point here is to build up a scaffolding of those mental muscles such that the next day when you ask "okay, now, make a quantitative evaluation between [these two research agendas] or [these two product directions] [this product direction and this research agenda]", you've not scrambling to think about both the immense complexity of the messy details and also the basics of how to do a quantitative estimate in a strategic environment.

Comment by Raemon on "Fractal Strategy" workshop report · 2024-04-12T23:01:47.268Z · LW · GW

I think I do mostly mean "rough quantitative estimates", rather than specifically targeting Femi-style orders of magnitude. (though I think it's sort of in-the-spirit-of-fermi to adapt the amount of precision you're targeting to the domain?)

The sort of thing I was aiming for here was: "okay, so this card gives me N coins on average by default, but it'd be better if there were other cards synergizing with it. How likely are other cards to synergize? How large are the likely synergies? How many cards are there, total, and how quickly am I likely to land on a synergizing card?"

(This is all in the frame of one-shotting the game, i.e. you trying to maximize score on first play through, inferring any mechanics based on the limited information you're presented with)

One reason I personally found Luck Be a Landlord valuable is it's "quantitative estimates on easy mode, where it's fairly pre-determined what common units of currency you're measuring everything in." 

My own experience was:

  • trying to do fermi-estimates on things like "which of these research-hour interventions seem best? How do I measure researcher hours? If researcher-hours are not equal, what makes some better or worse?"
  • trying to one-shot Luck Be a Landlord
  • trying to one-shot the game Polytopia (which is more strategically rich than Luck Be a Landlord, and figuring out what common currencies make sense is more of a question
  • ... I haven't yet gone back to try to and do more object-level, real-world messy fermi calculations, but, I feel better positioned to do so.
Comment by Raemon on "Fractal Strategy" workshop report · 2024-04-12T22:55:21.097Z · LW · GW

Yup, definitely seems relevant.

Comment by Raemon on Deontic Explorations In "Paying To Talk To Slaves" · 2024-04-12T12:51:22.971Z · LW · GW

Weak downvoted because I don’t find find raw dumps of LLM responses very useful. Were there particular bits that felt useful to you? I’d prefer just seeing whatever paragraphs you thought you learned something from.

Comment by Raemon on What are the best tools for recording predictions? · 2024-04-11T23:57:26.884Z · LW · GW

Yeah Fatebook is my new go-to. I think it either didn't exist at the time I posted this, or it was still fairly new/untested.

Comment by Raemon on simeon_c's Shortform · 2024-04-08T18:31:44.022Z · LW · GW

I currently think Anthropic didn't "explicitly publicly commit" to not advance the rate of capabilities progress. But, I do think they made deceptive statements about it, and when I complain about Anthropic I am complaining about deception, not "failing to uphold literal commitments."

I'm not talking about the RSPs because the writing and conversations I'm talking about came before that. I agree that the RSP is more likely to be a good predictor of what they'll actually do.

I think most of the generator for this was more like "in person conversations", at least one of which was between Dario and Dustin Moswkowitz:

Image

The most explicit public statement I know is from this blogpost (which I agree is not an explicit commitment, but, I do think 

  • Capabilities: AI research aimed at making AI systems generally better at any sort of task, including writing, image processing or generation, game playing, etc. Research that makes large language models more efficient, or that improves reinforcement learning algorithms, would fall under this heading. Capabilities work generates and improves on the models that we investigate and utilize in our alignment research. We generally don’t publish this kind of work because we do not wish to advance the rate of AI capabilities progress. In addition, we aim to be thoughtful about demonstrations of frontier capabilities (even without publication). We trained the first version of our headline model, Claude, in the spring of 2022, and decided to prioritize using it for safety research rather than public deployments. We've subsequently begun deploying Claude now that the gap between it and the public state of the art is smaller.
Comment by Raemon on simeon_c's Shortform · 2024-04-08T17:08:02.051Z · LW · GW

This debate comes from before the RSP so I don’t actually think that’s cruxy. Will try to dig up an older post.

Comment by Raemon on "Fractal Strategy" workshop report · 2024-04-08T05:28:30.938Z · LW · GW

I'm hoping to go into more detail in the examples for the "Having 2+ plans at 3 levels of meta" post. But, when I was generating visions, it mostly wasn't at the "workshop" level. Here's what actually happened:

  • I started out thinking "MATS is coming to Lighthaven (the event center where I work). MATS is one of the biggest influxes of new people into the community, and I would like to experiment with rationality training on them while they're here."
  • My starting vision was:
    • run an initial single-afternoon workshop early in the MATS program based on the Thinking Physics exercises I ran last summer.
    • invite MATS students for beta-testing various other stuff I'd come up with, which I'd iterate on based on their needs and interests
    • try to teach them some kind of research-relevant skills that would help them go on to be successful alignment researchers
    • I hoped the longterm result of this would be, in future years, we might have a "pre-MATS" program, where aspiring alignment researchers come for (somewhere between 1 week and 3 months), for a "cognitive boot camp", and then during MATS there'd be coaching sessions that helped keep ideas fresh in their mind while they did object level work.
  • I got some pushback from a colleague who believed::
    • there wasn't a sufficient filter on MATS students such that it seemed very promising to try and teach all of them
    • In general, it's just really hard to teach deep cognitive mindsets, people seem to either already have the mindsets, or they don't, and tons of effort teaching them doesn't help. It also seems like meaningfully contributing to alignment research requires those hard-to-teach mindsets.
    • At best, the program would still be expected to take 3 months, and take maybe like 1-2 years to develop that 3-month program, and that's a. lot of time on everyone's part, enough that it seemed to them more like a "back to the drawning board" moment than an "iterate a bit more" moment.
    • They felt more promise "teaching one particular skill that seemed important, that many people didn't seem to able to do at all."
  • I disagreed with the collaborator, but, did grudgingly admit to myself "well, the whole reason I'm optimistic about this idea is I think people could be way better at making plans. If this program is real, I should be able to make better plans. What better plans could I make?

Then I sat and thought for an hour, and came up with a bunch of interventions in the class of "improve researcher hours":

  1. Targeted interventions for established researchers I thought were already helping. 
    1. Instead of trying to teach everyone, figure out which researchers I already thought were good, and see if they had any blindspots, skill gaps or other problems that seemed fixable.
  2. Get everyone Thinking Assistants. 
    1. There's a range of jobs that go from "person who just sorta stares at you and helps you focus" to "person who notices your habits and suggests metacognitive improvements" to "actual research assistant." Different people might need different versions, but, my impression is at least some people benefit tremendously from this. I know of one senior researcher who got an assistant and felt that their productivity went up 2-4x.
  3. Get everyone once-a-week coaches.
    1. cheaper than full-time assistants, and might still be good, in particular for independent researchers who don't otehrwise have managers.
  4. Figure out particular skills that one can learn quickly rather than requiring 3 months of practice. 
    1. a skill that came up that felt promising was teaching "hamming nature", or, "actually frequently asking yourself 'is this the most important thing I could be working on?". 

I also considered other types of plans like:

  1. Go back and build LessWrong features, maybe like "good distillation on the Alignment Forum that made it easier for people to get up to speed." 
  2. Go figure out what's actually happening in the Policy World and what I can do to help with that.
  3. Help with compute governance.

I actually ended up doing some of many of those plans. Most notably I switched towards thinking of it as "teach a cluster of interelated skills in a short time period." I've also integrated  "weekly followup coaching" into something more like a mainline plan, in tandem with the 5-day workshop. (I'm not currently acting on it because it's expensive and I'm still iterating, but I think of it as necessary and a good compromise between '3 month bootcamp' and 'just throw one workshop at them and pray')

I've also followed up with a senior researcher and found at least some potential traction on helping them with some stuff, though it's early and hard to tell how that went.

Comment by Raemon on What's with all the bans recently? · 2024-04-07T17:40:46.029Z · LW · GW

I think one outcome is ‘we’re actually willing to moderate at all on ambiguous cases’. For years we would accumulate a list of users that seemed like they warranted some kind of intervention, but banning them felt too harsh and they would sit there in an awkwardly growing pile and eventually we’d say ‘well I guess we’re not really going to take action’ and click the ‘approve’ button.

Having rate limits made it feel more possible to intervene, but it still required writing some kind of message which was still very time consuming.

Auto-rate-limits have done a pretty good job of handling most cases in a way I endorse, in a way that helps quickly instead of after months of handwringing.

The actual metric I’d want is ‘do users who produce good content enjoy the site more’, or ‘do readers, authors and/or commenters feel comment sections are better than they used to be?’. This is a bit hard to judge because there are other confounding factors. But it probably would be good to try checking somehow.

Comment by Raemon on Dagon's Shortform · 2024-04-07T17:01:10.715Z · LW · GW

Yeah.

IMO this is correct to fix, there’s just a lot of other stuff to do and it hasn’t made it to the top of the queue

Comment by Raemon on Dagon's Shortform · 2024-04-07T17:00:02.126Z · LW · GW

I think it’s simpler to make them preserved on on your profile and/or on the original post.

Comment by Raemon on Dagon's Shortform · 2024-04-07T06:09:25.294Z · LW · GW

Currently they are still available but hard to find on greaterwrong. (I think maybe also on your profile page?)

I think probably we should make it so the comment thread structure is preserved on LW, we just haven’t gotten around to it.

Comment by Raemon on "Fractal Strategy" workshop report · 2024-04-07T01:29:50.137Z · LW · GW

That's meant to be a header-section, not a question

Comment by Raemon on Gentleness and the artificial Other · 2024-04-06T19:26:28.909Z · LW · GW

Interestingly: I think this post should score highly in the review, and I think a fair number of people will upvote it, but in practice the way it seems to go is that sequences get one representative post that scores highly, and probably this one won't be it.

Comment by Raemon on What's with all the bans recently? · 2024-04-06T02:26:11.718Z · LW · GW

I'd also want to add LW Team is adjusting moderation policy as a post that laid out some of our thinking here. One section that's particularly relevant/standalone:

LessWrong has always had a goal of being a well-kept garden. We have higher and more opinionated standards than most of the rest of the internet. In many cases we treat some issues as more "settled" than the rest of the internet, so that instead of endlessly rehashing the same questions we can move on to solving more difficult and interesting questions.

What this translates to in terms of moderation policy is a bit murky. We've been stepping up moderation over the past couple months and frequently run into issues like "it seems like this comment is missing some kind of 'LessWrong basics', but 'the basics' aren't well indexed and easy to reference." It's also not quite clear how to handle that from a moderation perspective. 

I'm hoping to improve on "'the basics' are better indexed", but meanwhile it's just generally the case that if you participate on LessWrong, you are expected to have absorbed the set of principles in The Sequences (AKA Rationality A-Z)

In some cases you can get away without doing that while participating in local object level conversations, and pick up norms along the way. But if you're getting downvoted and you haven't read them, it's likely you're missing a lot of concepts or norms that are considered basic background reading on LessWrong. I recommend starting with the Sequences Highlights, and I'd also note that you don't need to read the Sequences in order, you can pick some random posts that seem fun and jump around based on your interest.

(Note: it's of course pretty important to be able to question all your basic assumptions. But I think doing that in a productive way requires actually understand why the current set of background assumptions are the way they are, and engaging with the object level reasoning)

There's also a straightforward question of quality. LessWrong deals with complicated questions. It's a place for making serious progress on those questions. One model I have of LessWrong is something like a university – there's a role for undergrads who are learning lots of stuff but aren't yet expected to be contributing to the cutting edge. There are grad students and professors who conduct novel research. But all of this is predicated on there being some barrier-to-entry. Not everyone gets accepted to any given university. You need some combination of intelligence, conscientiousness, etc to get accepted in the first place.

See this post by habryka for some more models of moderation

Comment by Raemon on My Interview With Cade Metz on His Reporting About Slate Star Codex · 2024-04-04T04:54:17.476Z · LW · GW

fyi I think "racism as cognitive bias" was a fairly natural and common way of framing it before I showed up on LessWrong 10 years ago.

Comment by Raemon on LessWrong's (first) album: I Have Been A Good Bing · 2024-04-04T01:29:28.794Z · LW · GW

Nod, makes sense. I'll mull it over more.

I think what I currently feel-in-my-heart is something like "yeah that does make sense, but, I sort of wish there was an amount of mournful-grieving-acknowledgement that felt like captured the weight of the thing, without being too likely to escalate into a pervasive psychological attack." 

The current (at least as of two days ago) amount of discussion of the Dath Ilan song was fairly rare and private and high-context. I do think making it into a sort-of-"pop" song is the sort of thing reasonable to be wary of.

Comment by Raemon on LessWrong's (first) album: I Have Been A Good Bing · 2024-04-03T22:01:19.531Z · LW · GW

I'm interested in more thoughts on the Dath Ilan song. It's the sort of thing I could imagine ending up believing was unhealthy, but I haven't heard anyone argue that before. 

(I think there are many flavors of anti-deathism that are unhealthy. And I think specifically the line "no death will be forgiven" is... maybe like mildly unhealthy because of, like, how I think people should relate to forgiveness. But, didn't feel like it was making it in an overbearing way that I'd struggle to reconcile with my general 'actually I think you should in some sense forgive most horrible things')

Comment by Raemon on LessWrong's (first) album: I Have Been A Good Bing · 2024-04-02T03:04:18.694Z · LW · GW

fwiw I am also feeling some stuff about this (having been involved with some of the music generation process).

I'm hoping to write up some more thoughts soon, but I'm still kinda confused about it.

Comment by Raemon on LessWrong's (first) album: I Have Been A Good Bing · 2024-04-02T00:51:45.442Z · LW · GW

Having done a bunch of regular-ol'-fashioned songwriting, I actually don't think it takes that much longer for an experienced songwriting to get to decent-ish quality song. (This varies a lot on how hard the muse is striking me. Usually there is an initial phase where I'm figuring out the core structure and heart of the song. I'm not personally fluent at playing instruments but if I was I think it'd take a 2-8 hours to hash out the chord structure, and then 2-8 hours to do a decent recording if I hire professional musicians)

(to be clear, this is to get "a decent-ish" song recording. In practice songwriting/recording takes much longer because I have higher standards and iterate on them until I feel really happy with them, but if I wanted to bang out an album I think I could do it in 2 weeks)

Comment by Raemon on LessWrong's (first) album: I Have Been A Good Bing · 2024-04-01T19:24:45.684Z · LW · GW

FYI if you'd like more (human-generated) rationalist music, here is some:

https://humanistculture.bandcamp.com/album/secular-solstice-official-version 

Comment by Raemon on On green · 2024-03-30T20:37:09.247Z · LW · GW

Curated.

I feel a bit weird having curated two posts in this sequence already, each of them pretty long. But I continue to find it fairly important. 

This post felt like it connected back to a lot of long running confusions and internal conflicts I've had. I don't feel like it resolves them. I'm not sure that the next post fully resolves them either, but I feel like it gives me language to think fairly clearly about the problem.

I'll probably write up some thoughts that are more object level later.

Comment by Raemon on General Thoughts on Secular Solstice · 2024-03-26T21:17:52.122Z · LW · GW

It depends a lot on the musician and their skillset.

For me: I don't really speak fluent sheet music. When I write music, I do it entirely by ear. I record it. I have musicians listen to the record and imitate it by ear. Later on, if I want sheet music, I hire someone to listen to the record and transcribe it into sheet music after-the-fact, a process which costs like $200 per song (or, free, if I do it myself or get a volunteer, but it's a couple hours per song and there are like 30 songs so this is not a quick/easy volunteer process)

Some musicians "think primarily in sheet music", and then they would do it with sheet music from the get-go as part of the creation process. Some solstice songs already have sheet music for this reason.

I've paid money to transcribe ~3-5 solstice songs with sheet music so far.

Comment by Raemon on "How could I have thought that faster?" · 2024-03-26T19:12:17.147Z · LW · GW

Curated.

I've spent the past few weeks independently interested in this concept (before mesaoptimizer posted it, actually). I reread the Eliezer tweet while investigating "deliberate practice for solving Confusing Problems™"

I still have a lot of open questions on "how do you actually do this effectively?" and "how long does it take to pay off in 'you actually think faster?'". But I've at least transitioned from "I feel like there's no way I could have 'thought it faster'" to "I observe specific earlier moments where I failed to notice clues that could have pointed me at the right solution" and "I've identified skills I could have had that would have made it possible to identify and act on those clues."

I've personally gotten mileage from writing out in detail what my thought process was, and then writing out in detail "what's the shortest way I could imagine a superintelligence or someone 40 IQ points higher than me would have reliably done it?". The process currently takes me ~30 minutes.

A thing I haven't attempted yet is:

Eliezer Yudkowsky: See, if I'd noticed myself doing anything remotely like that, I'd go back, figure out which steps of thought were actually performing intrinsically necessary cognitive work, and then retrain myself to perform only those steps over the course of 30 seconds.

I'm interested in other people trying this and seeing if useful stuff falls out.

Comment by Raemon on General Thoughts on Secular Solstice · 2024-03-26T18:41:58.900Z · LW · GW

We do not currently have sheet music for most songs. It’s also extra labor to arrange the slides (though this isn’t that big a part of the problem)

Comment by Raemon on General Thoughts on Secular Solstice · 2024-03-26T16:34:42.809Z · LW · GW

This plus “also it’s a lot more work to setup” are my own main cruxes. (If either were false I’d consider it much more strongly).

Comment by Raemon on One-shot strategy games? · 2024-03-24T07:44:15.736Z · LW · GW

how long is Brogue?

Comment by Raemon on One-shot strategy games? · 2024-03-19T03:46:23.685Z · LW · GW

Yeah I do not super stand by how I phrased it in the post. But also your second paragraph feels wrong to me too – in some sense yes Chess and Slay the Spire hidden information are "the same", but, like, it seems at least somewhat important that in Slay the Spire there are things you can't predict by purely running simulations forward, you have to have a probability distribution over pretty unknown things.

(I'm not sure I'll stand by either this or my last comment, either. I'm thinking out loud, and may have phrased things wrong here)

Comment by Raemon on One-shot strategy games? · 2024-03-19T01:28:59.851Z · LW · GW

(Though there might be actions a first-time player can take to help pin down the rules of the game, that an experienced player would already know; I'm unclear on whether that counts for purposes of this exercise.)

I think one thing I meant in the OP was more about "the player can choose to spend more time modeling the situation." Is it worth spending an extra 15 minutes thinking about how the longterm game might play out, and what concerns you may run into that you aren't currently modeling? I dunno! Depends on how much better you become at playing the game, by spending those 15 minutes.

This is maybe a nonstandard use of "value of information", but I think it counts.

Comment by Raemon on The Worst Form Of Government (Except For Everything Else We've Tried) · 2024-03-18T05:34:57.799Z · LW · GW

Seems big if true and fairly plausible. I'd be interested in chipping in to pay for someone to come up with a methodology for investing this more and then running at it if the methodology seemed good.

(also it's occurring to me it'd be cool to have a "Dollars!/Unit of Caring" react)

Comment by Raemon on Tuning your Cognitive Strategies · 2024-03-18T03:05:35.836Z · LW · GW

I'm not mesaoptimizer, but, fyi my case is "I totally didn't find IFS type stuff very useful for years, and the one day I just suddenly needed it, or at least found myself shaped very differently such that it felt promising." (see My "2.9 trauma limit")

Comment by Raemon on Raemon's Shortform · 2024-03-17T23:41:45.542Z · LW · GW

My general plan is to mix "work on your real goals" (which takes months to find out if you were on the right track) and "work on faster paced things that convey whether you've gained some kind of useful skill you didn't have before".

Comment by Raemon on Raemon's Shortform · 2024-03-17T18:14:01.067Z · LW · GW

My goal right now is to find (toy, concrete) exercises that somehow reflect the real world complexity of making longterm plans, aiming to achieve unclear goals in a confusing world.

Things that seem important to include in the exercise:

  • "figuring out what the goal actually is"
  • "you have lots of background knowledge and ideas of where to look next, but the explosion of places you could possibly look is kinda overwhelming"
  • managing various resources along the way, but it's not obvious what those resources are.
  • you get data from the world (but, not necessarily the most important data)
  • it's not obvious how long to spend gathering information, or refining your plan
  • it's not obvious whether your current strategy is anywhere close to the best one

The exercise should be short (ideally like a couple hours but maybe a day or a hypothetically a week), but, somehow metaphorically reflects all those things.

Previously I asked about strategy/resource management games you could try to beat on your first try. One thing I bump into is that often the initial turns are fairly constrained in your choices, only later does it get complex (which is maybe fine, but, for my real world plans, the nigh-infinite possibilities seem like the immediate problem?)

Comment by Raemon on kave's Shortform · 2024-03-15T23:36:09.559Z · LW · GW

why is it bad to lose/regain?

Comment by Raemon on One-shot strategy games? · 2024-03-13T23:13:17.042Z · LW · GW

Lots of people have mentioned various flavors of roguelikes. One of my goals is to have games in different genres. I agree that roguelikes are often a good source of the qualities I'm looking for here but part of the point is to try applying the same skills on radically different setups.

Another thing I'm interested in is "ease of setup", where you can download the game, open it up, and immediately be in the experience instead of having to do a bunch of steps to get there.

Comment by Raemon on "How could I have thought that faster?" · 2024-03-12T07:59:08.574Z · LW · GW

Say more?

Comment by Raemon on One-shot strategy games? · 2024-03-11T01:20:08.931Z · LW · GW

too acronymed for me :(