Postmodern Warfare

At the moment, centralized human command-and-control norms make large militaries similarly fragile. They've gotten around this in recent decades by letting smaller, more autonomous parts take over when there are enough dead bodies, but this does not last long. The same problems in policy elsewhere continue within militaries. While elements of the military have seen this problem (it's an old problem best explained by the writing of David Hackworth, Erwin Rommel, and B.H Liddell-Hart), they can't really change it from the inside, so there's been a shift toward retirees who've started consulting firms with an implicit aim of changing culture from the outside.

Progress in technology will yield the same problems that it did in the 60s (when militaries had access to better radio communication, they used it to try to enforce tighter control on their units while avoiding the ground- commanders literally flew in helicopters instead of having their boots on the ground): militaries will attempt to use that technology for more central control, which will continue making its systems more rigid and vulnerable to a sudden catastrophic failure.

One of the most important decisions in war is when to stop.  Humans evolved fear to solve this problem; there's a point at which soldiers will de-escalate the conflict (i.e. flee the battlefield rather than stay and die).  However, signalling fear makes you a target so people don't discuss it candidly.  I am concerned that military leaders may, in the calm of the office, design AI that has no provisions for de-escalating conflict; this seems very likely to lead to nuclear war.

We seem to be in the midst of a trend away from direct confrontation and toward sowing discord. We want to be able to spread chaos in the ranks of our opponents, and have that intervention be perceived as a failure in our opponents' ability to enforce discipline or coordinate effectively. As a concrete example, Russia may interfere with American social media websites in order to promote violent mysogyny or racism, provoking attacks by Americans on Americans. Americans may then blame the perpetrators, our own citizens, and our own culture for these outcomes, even if Russian interference was a necessary condition for many of these attacks.

I could imagine that AI will enhance the ability of militaries and intelligence organizations to execute such attacks in other ways. For example, we could imagine a whole field of "bug design," in which AI systems are created and sold with deliberately built-in bugs that nonetheless appear accidental, and cause the AI to act up at some pre-determined point in the future. AI might not just help programmers write code - it might help them write buggy code that nevertheless appears bug-free. The outcome can be framed as an accident, the developers of the weaponized bug may be impossible to identify, and it becomes very difficult to know what a proportional response looks like.

Another example would be a country imposing healthy eating incentives on its own population, while subsidizing the export of tasty and unhealthy foods to its adversaries. Then the population in the other country blames itself, or its own economic structure, for the spread of obesity.

I feel OK about posting this here, because I think that on net, the risk of spreading new conceptual ideas to an attacker is lower than the value of a population considering how to defend itself against such attacks on its sense of responsibility. The attack can be done by a small number of specialists. Defending against such an attack will require a heightened level of awareness at the population level, and that requires free and open discussion.

This sounds to me a lot more like "next-generation modern warfare"; postmodern warfare evokes Wizeman's Lethal Theory (pdf), edited for brevity:

This essay belongs to a larger investigation of the ways in which contemporary military theorists are conceptualizing the urban domain. What are the terms they are using to think about cities? What does the language employed by the military to describe the city tell us about the relationship between organized violence and the production of space? What does this language tell us about the military as an institution? Not least important is the question of the role of theory in all these operations. ...

There is a considerable overlap among the theoretical texts considered “essential” by military academies and architectural schools. Indeed, the reading lists of contemporary military institutions include Deleuze, Guattari, and Debord, as well as more contemporary writings on urbanism, psychology, cybernetics, and postcolonial and poststructuralist theory ... the discourses which shaped thinking in various academic fields toward the end of the 20th century have been employed for the reinvigoration of warfare.

See also the blog post Nakatomi Space; 'firehose of falsehood' propoganda tactics, etc.

I haven't explicitly modeled out odds of war with China in the coming years, in any particular timeframe. Some rationalist-adjacent spheres on Twitter are talking about it, though. In terms of certainty, it definitely isn't in the "China has shut down transportation out of Wuhan" levels of alarm; but it might be "mysterious disease in Wuhan, WHO claims not airborne" levels of alarm.

I'd expect our government to be approximately as competent in preparing for and succeeding at this task as they were at preparing for and eliminating COVID. (A look at our government's actions[albeit from a China-sympathetic American] suggests general incoherence.)

If someone with greater domain expertise than me has looked at this, I'd be interested in an in-depth dive.

Computer command systems can coordinate perfectly and instantly.

It is important to note that this is not strictly speaking true. Computer command systems can coordinate far more accurately and far faster than humans in many cases; this is neither perfect nor instant. (And fundamentally cannot be in a battlefield environment - the Byzentine Generals problem rules out the former and a finite speed of light the latter.)

It's easier to visualize if you try to work out the hierarchy of software agents you might use for this.  
 

First, most of the bigger drones will probably some kind of land vehicle, whether a legged infantry or a robot on tracks.  This is for obvious range and power reasons - a walking or rolling robot can carry far more weapons and armor than anything in the air.  And in a battlespace where everyone on the enemy side has computer controlled aim, flying drones without armor will likely only survive for mere seconds of exposure.  

So at the bottom level the drones need to be able to plan and locomote to a given location on the battlefield, or report that they are unable to reach a particular location.  (Due to inaccessibility or damage to that particular unit - robotic units obviously won't stop fighting when damaged)

At a slightly higher level you have an agent that coordinates small "units" of drones to accomplish a mission from a finite set of trained "missions".

Missions might be things like "clear this structure of enemy fighters".

The agent at these 2 layers have been trained with collectively millions of years, with the red team agents controlling simulated enemy drones or simulated human bodies.  So the red team will likely be more combat effective at being a human than typical actual human soldiers.  So the trained policy of these agents will assume their opponent is doing the best that is possible.  

We don't know what the trained policy would look like but I suspect it involves a lot of careful control of exposure angles, and various unfair strategies.  

The layer above the bottom agent doesn't know how to perceive an environment, or how to locomote, or ballistics - it queries the lower level agent whenever it needs to know if a proposed action is feasible.  

Then the layer above this agent handles the battle itself, creating units of action of appropriate size and assigning missions.  This is where human generals coordinate.  They might choose a section of city and drag a box over it, ordering that the layer 3 agent subdivide that city section and clear every building.  

Each agent must query the layer below it to function, exporting these subtasks to an agent specialized in performing them.

Even the "level 1" agent doesn't actually directly locomote, it's similarly tiered internally.

The actual compute hardware is hosted such that many redundant vehicles run a VM hosted copy of the agent they need and the agent a level up.  Agents are perfectly deterministic - given the same input and an RNG seed they will always issue the same 'orders'.  This makes redundancy possible - multiple vehicles in parallel can run a perfect model of their 'commander' agent a level up, such that enemy fire destroying the vehicle that hosts a 'commander' will not degrade capabilities for even 1 frame.  

(each 'frame', several 'commanders' broadcast their orders, taking into account information from prior frames.  Each command is identical to all the others so the 'orders' must match or the majority will be used.  So at any given time there are 3-5 sets of 'orders' being broadcast to all agents in this subswarm.  So an incoming shot that blows up a commander, or jams it's communication leaves plenty of redundant copies of 'orders')

This distinction reminds me of the battles in Ender's Game.

As I recall, Ender was the overall commander, but he delegated control of different parts of the fleet to various other people, as most modern militaries do.

The bugs fought as a hive mind, and responded almost instantly across the entire battlefield, which made it challenging for the humans to keep up in large-scale battles.

Yeah, I read that bit in the article when Bret Deveraux linked to it, and I winced hard at this confidence that China's approach of wanting to use lots of AI was obviously was a bad idea. 

Modern warfare is expensive

The Taliban who didn't spend much money and still beat the US army because the US army is mismanaged enough that it can't win a battle against a much worse funded enemy.

FYI re: your footnote- mechanized infantry would have tracked vehicles organically attached to their formation. The term for an equivalent unit with wheeled vehicles would be motorized infantry.