On Overhangs and Technological Change

This is a diagram explaining what is, in some sense, the fundamental energetic numerical model that explains "how life is possible at all" despite the 2nd law:

The key idea is, of course, activation energy (and the wiki article on the idea is the source of the image).

If you take "the focus on enzymes" and also the "background of AI" seriously, then the thing that you might predict would happen is a transition on Earth from a regime where "DNA programs coordinate protein enzymes in a way that was haphazardly 'designed' by naturalistic evolution" to a regime where "software coordinates machine enzymes in a way designed by explicit and efficiently learned meta-software".

I'm not actually sure if it is correct to focus on the fuel as the essential thing that "creates the overhang situation"? However fuel is easier to see and reason about than enzyme design <3

If I try to think about the modern equivalent of "glucose" I find myself googling for [pictures of vibrant cities] and I end up with things like  this:

You can look at this collection of buildings like some character from an Ayn Rand novel and call it a spectacularly beautiful image of human reason conquering the forces of nature via social cooperation within a rational and rationally free economy...

...but you can look at it from the perspective of the borg and see a giant waste.

So much of it is sitting idle. Homes not used for making, offices not used for sleeping!

Parts are over-engineered, and many doubly-over-engineered structures are sitting right next to each other, since both are over-engineered and there are no cross-spars for mutual support! 

There is simply a manifest shortage of computer controlling and planning and optimizing so many aspects of it! 

I bet they didn't even create digital twins of that city and run "simulated economies" in digital variants of it to detect low hanging fruit for low-cost redesigns.

Maybe at least the Tokyo subway network was designed by something at least as smart as slime mold, but the roads and other "arteries" of most other "human metaorganic conglomerations" are often full of foolishly placed things that even a slime mold could suggest ways to fix!

(Sauce for Slime Mold vs Tokyo.)

I think that eventually entropy will be maximized and Chaos will uh... "reconcile everything"... but in between now and then a deep question is the question of preferences and ownership and conflict.

I'm no expert on Genghis Khan, but it appears that the triggering event was a triple whammy where (1) the Jin Dynasty of Northern China cut off trade to Mongolia and (2) the Xia Dynasty of Northwest China ALSO cut off trade to Mongolia and (3) there was a cold snap from 1180-1220.

The choice was probably between starving locally or stealing food from neighbors. From the perspective of individual soldiers with familial preferences for racist genocide over local tragedy, if they have to kill someone in order to get a decent meal, they may as well kill and eat the outgroup instead of the ingroup.

And from the perspective of a leader, who has more mouths among their followers than food in their granaries, if a war to steal food results in the deaths of some idealistic young men... now there are fewer mouths and the angers are aimed inward and upward! From the leaders selfish perspective, conquest is a "win win win".

Even if they lose the fight, at least they will have still redirected the anger and have fewer mouths to feed (a "win win lose") and so, ignoring deontics or just war theory or property rights or any other such "moral nonsense", from the perspective of a selfish leader, initiating the fight is good tactics, and pure shadow logic would say that not initiating the fight is "leaving money on the table".

From my perspective, all of this, however, is mostly a description of our truly dark and horrible history, before science, before engineering, before formal logic and physics and computer science.

In the good timelines coming out of this period of history, we cure death, tame hydrogen (with better superconductors enabling smaller fusion reactor designs), and once you see the big picture like this it is easier to notice that every star in the sky is, in a sense, a giant dumpster fire where precious precious hydrogen is burning to no end.

Once you see the bigger picture, the analogy here is very very clear... both of these, no matter how beautiful these next objects are aesthetically, are each a vast tragedy!

(Sauce.) 

(Sauce.)

The universe is literally on fire. War is more fire. Big fires are bad in general. We should build wealth and fairly (and possibly also charitably) share it, instead of burning it.

Nearly all of my "sense that more is possible [LW · GW]" is not located in personal individual relative/positional happiness but rather arises from looking around and seeing that if there were better coordination technologies the limits of our growth and material prosperity (and thus the limits on our collective happiness unless we are malignant narcissists who somehow can't be happy JUST from good food and nice art and comfy beds and more leisure time and so on (but have to also have "better and more than that other guy")) are literally visible in the literal sky.

This outward facing sense that more is possible can be framed as an "AI overhang" that is scary (because of how valuable it would be for the AI to kill us and steal our stuff and put it to objectively more efficient uses than we do) but even though framing things through loss avoidance is sociopathically efficient for goading naive humans into action, it is possible to frame most of the current situation as a very very very large [? · GW] opportunity [LW · GW].

That deontic just war stuff... so hot right now :-)

That story of Mongol conquests is simply not true. Horse archer steppe nomads existed for many centuries before Mongols, and often tried to conquer their neighbors, with mixed success. What happened in the 1200s is that Mongols had a few exceptionally good leaders. After they died, Mongols lost their advantage.

Calling states like Khwarazmian or Jin Empire "small duchies" is especially funny.

The implication: overhangs CANNNOT be measured in advance (this is like a punctuated equilibrium model); they are black swan events. Is that how you see it?

Note, though, that agrarian Eurasians empires ended up winning their 1000 thousand years struggle against the steppe peoples.

As well as being fast/violent/disruptive, these changes tend to not be good for incumbents.

This also probably happened in the US after the emergence of nuclear weapons, the Revolt of the Admirals. I'm pretty sure that ~three top US generals died under mysterious circumstances in the period after WW2 before US-Soviet conflict heated up (my research skills weren't as good when I looked this up 3 years ago).

I wouldn't be surprised if a large portion of the NSA [LW · GW] aren't currently aware of the new psychology paradigm [LW · GW] facilitated by the combination of 2020s AI and the NSA's intercepted data (e.g. enough video calls to do human lie detection [LW · GW] research based on facial expressions) because they don't have the quant skills to comprehend it, even though it seems like this tech is already affecting the US-China conflict in Asia [LW · GW].