Death by Red Tape

In the Zones of Thought universe, there is a cycle of civilization: civilizations rise from stone-age technology, gradually accumulating more technology, until they reach the peak of technological possibility. At that point, the only way they can improve society is by over-optimizing it for typical cases, removing slack [? · GW]. Once society has removed its slack, it's just a matter of time until unforeseen events force the system slightly outside of its safe parameters. This sets off a chain reaction: like dominoes falling, the failure of one subsystem causes the failure of another and another. This catastrophe either kills everyone on the planet, or sets things so far back that society has to start from scratch.

Reminds me of this:

In 1988, Joseph Tainter wrote a chilling book called The Collapse of Complex Societies. Tainter looked at several societies that gradually arrived at a level of remarkable sophistication then suddenly collapsed: the Romans, the Lowlands Maya, the inhabitants of Chaco canyon. Every one of those groups had rich traditions, complex social structures, advanced technology, but despite their sophistication, they collapsed, impoverishing and scattering their citizens and leaving little but future archeological sites as evidence of previous greatness. Tainter asked himself whether there was some explanation common to these sudden dissolutions.

The answer he arrived at was that they hadn’t collapsed despite their cultural sophistication, they’d collapsed because of it. Subject to violent compression, Tainter’s story goes like this: a group of people, through a combination of social organization and environmental luck, finds itself with a surplus of resources. Managing this surplus makes society more complex—agriculture rewards mathematical skill, granaries require new forms of construction, and so on.

Early on, the marginal value of this complexity is positive—each additional bit of complexity more than pays for itself in improved output—but over time, the law of diminishing returns reduces the marginal value, until it disappears completely. At this point, any additional complexity is pure cost.

Tainter’s thesis is that when society’s elite members add one layer of bureaucracy or demand one tribute too many, they end up extracting all the value from their environment it is possible to extract and then some.

The ‘and them some’ is what causes the trouble. Complex societies collapse because, when some stress comes, those societies have become too inflexible to respond. In retrospect, this can seem mystifying. Why didn’t these societies just re-tool in less complex ways? The answer Tainter gives is the simplest one: When societies fail to respond to reduced circumstances through orderly downsizing, it isn’t because they don’t want to, it’s because they can’t.

In such systems, there is no way to make things a little bit simpler – the whole edifice becomes a huge, interlocking system not readily amenable to change. Tainter doesn’t regard the sudden decoherence of these societies as either a tragedy or a mistake—”[U]nder a situation of declining marginal returns collapse may be the most appropriate response”, to use his pitiless phrase. Furthermore, even when moderate adjustments could be made, they tend to be resisted, because any simplification discomfits elites.

When the value of complexity turns negative, a society plagued by an inability to react remains as complex as ever, right up to the moment where it becomes suddenly and dramatically simpler, which is to say right up to the moment of collapse. Collapse is simply the last remaining method of simplification.

If you are a smart person I suggest working in domains where the regulators have not yet shut down progress. In many domains, if you want to make progress most of your obstacles are going to be other humans. It is refreshingly easy to make progress if you only have to face the ice.

I think you're far too charitable toward the FDA, TSA et al. I submit that they simultaneously reduce slack, increase fragility ... and reduce efficiency. They are best modeled as parasites, sapping resources from the host (the general public) for their own benefit.

[edit] Something I've been toying with is thinking of societal collapse as a scale-free distribution [1], from micro-collapse (a divorce, perhaps?) to Roman Empire events. In this model, a civilization-wide collapse doesn't have a "cause" per se, it's just a property of the system.

[1] https://en.wikipedia.org/wiki/Self-organized_criticality

While I agree with the general point, I think this part:

(The TSA spends 5.3 billion on airline security annually. It's difficult to put a price on 9/11, but quick googling says that total insurance payouts were $40 billion. So very roughly, the utilitarian question is whether the TSA stops a 9/11-scale attack every 8 years.)

is rather poorly phrased in a way that detracts from the overall message.  I don't think this accurately captures either the costs of the TSA (many of which come in the form of lost time waiting in security or of poorly-defined costs to privacy) or the costs of 9/11 (even if we accept that the insurance payouts adequately capture the cost of lives lost, injuries, disruption, etc. there are lots of extra...let's call them 'geopolitical costs').

At least in the TSA case, you have to consider that your adversity may be optimizing against you. Maybe you don't get very many attempted attacks precisely because you're putting effort into security. This isn't to say the current level of effort is optimal, just that you need more than simple cost-benefit calculation if your intervention decisions feed back into the event distribution.