[Meta?] Using the LessWrong codebase for a blog

I believe the only other [EDIT: fully independent] deployment of the LW codebase today is the EA forum: https://forum.effectivealtruism.org/

My guess is that it is massively overkill for an individual blog, and that both the level of complexity and the level of churn is not what you would want to deal with.

Have you considered either (a) blogging directly on LW or (b) modifying a simpler blogging engine to look like LW?

I am really glad to hear that you like the site!

Overall, my guess is for a blog where you don't expect many other contributors at the top-level, the current LW codebase is pretty overkill. I am already sad about how bloated our client-bundle is for our use-case, and my feeling is that things like our javascript execution time and bundle size and server overhead would be really overkill for a personal blog. 

There will also be bugs and things will break. We got everything running pretty well for our use-case, but sometimes the EA Forum runs into different problems than we do even though their use-case is only very slightly different, and that sometimes surfaces previously undiscovered bugs. For the EA Forum we have a close collaboration that usually allows us to address these reasonably quickly, but my guess is that wouldn't be an option for you.

As a concrete example: Spammers use all kinds of different tactics to post things. We covered most of the tactics that spammers use on LW in our defenses, but sometimes spammers use different tactics on the EA Forum (like editing wiki articles instead of creating comments or posts), and then we have to quickly build something that prevents that attack vector. Similar things might happen with a personal blog.

That said, because it seems good to have this written down and because I don't know the alternatives and the tradeoffs, here are answers to all your questions:

Are the makers of LW explicitly fine and open to it being used by other people or is it open source mainly for the sake of community debugging?

Yes, we are totally fine and even happy about other people using the code!

What kind of machine (very roughly speaking) would you need to handle top volumes ~20,000 visitors/hr (say, max 3,000/min) without the core functionality breaking and ~500 visitors/hr (say, max 100/min) with top-notch user experience (assuming optimal db, distro and reverse proxy choices)

This is maybe the part where it starts seeming like a bad idea to do this for a blog. Since we have a dev team of 4-5 people, all of which could make a decent salary as full-time engineers in the Bay Area, we really didn't optimize much to bring the server costs down (though this might change with some upcoming big refactors that drastically change how we deploy the site and how it runs). 

Our current AWS balance runs into about $600 a month for the main site, or something like $6k-$8k annually. This is of course fine for us to pay, given that our staff costs are much higher, but I would of course be a bit hesitant to pay this much for hosting a blog. My guess is to hit your targets, you could maybe pay half of that, so about $3-$5k annually. 

How hard is it to setup the sys admin side of things ? Deploying a prod server behind an nginx with a non-sqlite db and pointing it to your own cdn

I would be happy to help you get set up with stuff. We are currently deploying with AWS beanstalk, so there is basically just a file where you plop your DB credentials, your AWS credentials and your server settings, and then press the deploy button, and then the site should go up. The codebase doesn't currently touch on any CDN stuff. Images we deliver are currently uploaded to the CKEditor CDN (the people who built our editor framework), and I would be happy to give you a subaccount on that and bill you the costs from time to time (or if the usage is under a threshold of something like $30 a month just ignore it and let you use it for free). If you wanted to have your own image CDN for images in posts and comments, you could just deactivate the CKEditor upload plugin and then use externally hosted images in posts. 

How hard is it to modify the theme (e.g. fonts, color-scheme, icons) ?

The very basics of the theme (like the primary color and font) are in a single theme file and very easy to change. This will affect all the things that are different between LW and the EA Forum. 

Most other things would have to be changed in React and in the JSS + SCSS itself. 

How hard is it to integrate your own 3rd party service or get rid of them ? Specifically, add your own app cred for the signups and remove google analytics, intercom and all other 3rd party integrations that would make Richard stallman cry which aren't critical to the commenting experience.

This depends a lot on the specific service. Getting rid of our search service (Algolia) would probably be a big pain, since we use it it in a ton of different places around the codebase and all kinds of stuff break without it. Getting rid of Google Analytics is just a single line in a config file somewhere. Getting rid of Intercom is also just a single line, I am pretty sure. My guess is you can remove most of them pretty easily, but it might be that something turns out to be more of a pain.

What are particularly difficult/annoying/deal-breaking parts of the setup that were unexpected?

Things that you would probably currently find most annoying: 

• In terms of using the site on a phone, it really isn't remotely as performant as it could be, and you will get complaints about this. We are improving this, but my guess is it will still be an issue.
• A bunch of stuff you won't need will be a bit hard to deactivate. Like we have a PM system and an events system and a "put yourself on this map" system, and a "notify me of nearby events" system and a whole set of moderation tools that we use to keep tabs on new posters, and a bunch of code written for Petrov Day and a Question and Answer system and a wiki and tagging system, and a sequences system, and the whole annual review system and so many more small things. My guess is you won't need all of those, but getting rid of them might require some amount of work (I am not sure how much, but would be surprised if it turns out to be less than 40h of time to remove all of them cleanly).
• We make lots of changes to the codebase, all over the place. Merging with our upstream codebase is a good amount of work if you have your own changes on top if it. Not sure how much, but my guess is it takes the EA Forum team something like 4-5h a week. If you make fewer changes it might be a lot faster, but you do have to keep up with the database migrations we are running, and sometimes we sunset features that nobody uses, and if you needed them, you might be stuck with maintaining them yourself. 

Here are some things that are currently annoying, but likely won't be issues in 1-2 months because we are in the middle of a big refactor that fixes them: 

• Meteor really isn't a good choice these days. We used it because we built off of an existing forum system, but it will confuse and befuddle you in hundreds of ways.
• Deployment takes a really long time (50 minutes) because of a dumb bug in a script we are using to deploy to AWS Beanstalk. 
• Restarting a server after making a change also takes really far too long (~30 seconds on most machines). 

How hard is it to setup the sys admin side of things ? Deploying a prod server behind an nginx with a non-sqlite db and pointing it to your own cdn

It's pretty poor, and also in flux. We use Meteor and mup-aws-beanstalk, but are dissatisfied with them (due to slow client-side Javascript initialization, slow development-cycle build time, and extremely slow server deploy time), so I'm converting it to esbuild and a yet-to-be-chosen deploy/CI setup. So if you deploy an instance of the LW codebase today, you will probably find that the recommended deployment mechanism has changed to something else entirely different a month from now. A month from now it'll hopefully be pretty good, though!

What kind of machine (very roughly speaking) would you need to handle top volumes ~20,000 visitors/hr (say, max 3,000/min) without the core functionality breaking and ~500 visitors/hr (say, max 100/min) with top-notch user experience (assuming optimal db, distro and reverse proxy choices)

Logged-out users visiting the front page and a few post pages (ie, getting Slashdotted) will all be served from the page cache, so pretty much only limited by bandwidth. Lesswrong itself runs on a dynamically scaling pool of t2.small instances (though one would probably be enough) and a MongoDB Atlas M30 cluster.

How hard is it to modify the theme (e.g. fonts, color-scheme, icons) ?

If you're familiar with CSS/JSS, this should be pretty straightforward.

How hard is it to integrate your own 3rd party service or get rid of them ? Specifically, add your own app cred for the signups and remove google analytics, intercom and all other 3rd party integrations that would make Richard stallman cry which aren't critical to the commenting experience.

OAuth, Google Analytics, and Intercom are all already used, so integrating them is just be a matter of getting an API key and putting it into the right config setting.

Are the makers of LW explicitly fine and open to it being used by other people or is it open source mainly for the sake of community debugging?

We are explicitly fine with this. We just haven't gotten around to optimizing it much for this use case.

What are particularly difficult/annoying/deal-breaking parts of the setup that were unexpected?

Site search is powered by Algolia, which is kind of expensive and not especially good.