Designing Low Upkeep Software

post by jefftk (jkaufman) · 2021-09-25T18:50:05.900Z · LW · GW · 10 comments

It's common for me to get excited about a personal project, put a bunch of time into it, and then lose interest. As much as possible, I want these projects to continue working; upkeep isn't fun. What does designing for minimum maintenance look like?

The biggest piece is minimizing your dependencies, and limiting them to ones that value backwards compatibility. Most of my projects are on the web, which means I have an unavoidable dependency on browsers, but this is fine because browsers strongly prioritize keeping existing websites working.

On the server I run Ubuntu LTS until it's near the end of its supported lifetime. Every two years I run do-release-upgrade and move to the next one. This cost is shared over many jefftk.com projects, so it isn't too bad.

Many projects can just be hand-written HTML/JS/CSS. No build system, no libraries, just speaking the browser's native languages. These don't need any technical maintenance; they just keep working.

One step from there are projects where I need a little bit of server-side support. For example, I write my blog posts as plain HTML, but I need headers, footers, indexes, and RSS. Each time I change my site I run a script which creates/updates all these additional files. If I want to change something about the presentation of my site then I need to dig into code, and very occasionally something will break when updating the site, but just displaying my existing stuff doesn't require any maintenance.

Then we get to the tricky stuff: projects which need real time custom code. Things like Bucket Brigade and blog comment inclusion. How I approach this:

For example, I taught my comment requester to fetch comments from the EA Forum and LessWrong two years ago, and I don't expect to have to think about that code again until there are breaking changes to the upstream API.

This is not how I approach design at work, or necessarily how I would recommend doing so. Having a budget where initial creation is essentially free (fun!) while maintenance is extremely expensive (drugery!) is a dramatic exaggeration for most software development. But this process works very well for my hobby code.

Comment via: facebook

10 comments

Comments sorted by top scores.

comment by clone of saturn · 2021-09-26T07:01:06.294Z · LW(p) · GW(p)

I would suggest Common Lisp for this purpose--it hasn't had a significant compatibility breaking change for over 30 years.

comment by DanB · 2021-09-26T01:12:22.997Z · LW(p) · GW(p)

Having a budget where initial creation is essentially free (fun!) while maintenance is extremely expensive (drugery!) is a dramatic exaggeration for most software development.

My feeling is that most software development has exactly the same cost parameters; the difference is just that BigTech companies have so much money they are capable of paying thousands of engineers handsome salaries, to do the endless drudgery required to keep the tech stacks working.

The SQLite devs pledge to support the product until 2050.

Replies from: CraigMichael, jkaufman
↑ comment by CraigMichael · 2021-09-26T05:24:43.282Z · LW(p) · GW(p)

I do tend to use SQLite when a flat file would work.

↑ comment by jefftk (jkaufman) · 2021-09-26T11:35:07.758Z · LW(p) · GW(p)

most software development has exactly the same cost parameters; the difference is just that BigTech companies have so much money they are capable of paying thousands of engineers handsome salaries, to do the endless drudgery required to keep the tech stacks working

The payback is also very different at tech companies, or in any professional environment. I make something because I'm excited about it, and the payback is some combination of getting to use it and being glad other people can use it. When a company makes something, the payback is typically that people pay for it, or perhaps use it while looking at ads. This dramatically changes the incentives around adding features and generally changing it: you need to keep improving the product to compete with others that people might use instead. Once you need to keep the product live, in the sense that there are always multiple engineers spun up on it, the cost of switching to updated versions of dependencies is a small portion of the overall cost. And designing for minimal upkeep makes it harder to add new features.

comment by matto · 2021-09-27T13:47:23.160Z · LW(p) · GW(p)

When I first go into software, it was a lot of fun to try out as many things as possible during each project. This meant that my tiny blog was based on Django (with its dozens of dependencies), Gunicorn, and a full-fledged database server (postgres). But the cost to maintain (update, service, re-tweak) all that was too damn high! So over a series of moves, my blog is a bunch of static HTML files served by nginx; these files are compiled by a python script that probably gets 0.5 commits per year on average.

That site and a few others I host on the same Ubuntu server get backed up using a simple cronjob that fires off a bash script. In case the server disappears, which has happened to me once, I have another cronjob, this one running on a raspberry Pi at home, that uses SSH to download the backups. Recently, anything I've added to the Pi, I've coded up in Ansible first. It's a little extra cost up front, but then I can reproduce the setup by running one command.

Apart from actual dependencies, I think it's important to have the least number of "mental" dependencies. For example, working with files instead of a database not only cuts out the need for maintaining a database, but also for thinking in database terms -- ie. no need to re-learn/google for obscure admin commands that you run once a year etc.

comment by DPiepgrass · 2021-09-26T13:29:47.105Z · LW(p) · GW(p)

Huh, if I thought LessWrong were a water cooler for software developers, I might have posted my Top 11 principles & practices of programming here.

Low upkeep software is valuable, but so is rapid application development. I aim for both at once. In web development, I have found React + MobX + holders + TypeScript to be an effective combination, and probably Vue 3 and Svelte are too. The latter libraries will probably evolve faster (and TypeScript hasn't stopped doing breaking changes, but it's a price worth paying)... but we can expect they will use semantic versioning, and there is no obligation to upgrade. So, does the choice of Vue 3 increase upkeep if Vue 4 exists and you don't upgrade? Hmm.

If I were doing blog-like software, I might well stick to plain HTML+CSS and minimal JS, though. React is just what I use for "apps".

Replies from: jkaufman
↑ comment by jefftk (jkaufman) · 2021-09-26T20:46:17.529Z · LW(p) · GW(p)

This is a good description of the approach that I don't like for this kind of development ;)

If you don't keep your dependencies up to date, you are likely to have unpatched well-known security vulnerabilities. So you need to stay on a supported version. Vue 3 will work until it goes end of life, at which point you are stuck choosing between moving to a later version or taking down your project.

Replies from: DPiepgrass
↑ comment by DPiepgrass · 2021-09-28T17:48:52.979Z · LW(p) · GW(p)

Okay, but it's supposed to be mainly the backend that is responsible for security.

Replies from: jkaufman
↑ comment by jefftk (jkaufman) · 2021-09-28T19:28:34.132Z · LW(p) · GW(p)

Vulnerabilities in your front end code give XSS

comment by Jarred Filmer (4thWayWastrel) · 2021-09-25T20:57:03.366Z · LW(p) · GW(p)

Thanks for sharing, I'm about to move into a season of more time for hobby code and this seems like good advice to keep in mind