Why Yudkowsky is wrong about "covalently bonded equivalents of biology"

Why is flesh weaker than diamond?  Diamond is made of carbon-carbon bonds.  Proteins also have some carbon-carbon bonds!  So why should a diamond blade be able to cut skin?

I reply:  Because the strength of the material is determined by its weakest link, not its strongest link.  A structure of steel beams held together at the vertices by Scotch tape (and lacking other clever arrangements of mechanical advantage) has the strength of Scotch tape rather than the strength of steel.

Or:  Even when the load-bearing forces holding large molecular systems together are locally covalent bonds, as in lignin (what makes wood strong), if you've got larger molecules only held together by covalent bonds at interspersed points along their edges, that's like having 10cm-diameter steel beams held together by 1cm welds.  Again, barring other clever arrangements of mechanical advantage, that structure has the strength of 1cm of steel rather than 10cm of steel.

Bone is stronger than wood; it runs on a relatively stronger structure of ionic bonds, which are no locally weaker than carbon bonds in terms of attojoules of potential energy per bond.  Bone is weaker than diamond, then, because... why?

Well, partially, IIUC, because calcium atoms are heavier than carbon atoms.  So even if per-bond the ionic forces are strong, some of that is lost in the price you pay for including heavier atoms whose nuclei have more protons that are able to exert the stronger electrical forces making up that stronger bond.

But mainly, bone is so much weaker than diamond (on my understanding) because the carbon bonds in diamond have a regular crystal structure that locks the carbon atoms into relative angles, and in a solid diamond this crystal structure is tesselated globally.  Hydroxyapatite (the crystal part of bone) also tesselates in an energetically favorable configuration; but (I could be wrong about this) it doesn't have the same local resistance to local deformation; and also, the actual hydroxyapatite crystal is assembled by other tissues that layer the ionic components into place, which means that a larger structure of bone is full of fault lines.  Bone cleaves along the weaker fault line, not at its strongest point.

But then, why don't diamond bones exist already?  Not just for the added strength; why make the organism look for calcium and phosphorus instead of just carbon?

The search process of evolutionary biology is not the search of engineering; natural selection can only access designs via pathways of incremental mutations that are locally advantageous, not intelligently designed simultaneous changes that compensate for each other.  There were, last time I checked, only three known cases where evolutionary biology invented the freely rotating wheel.  Two of those known cases are ATP synthase and the bacterial flagellum, which demonstrates that freely rotating wheels are in fact incredibly useful in biology, and are conserved when biology stumbles across them after a few hundred million years of search.  But there's no use for a freely rotating wheel without a bearing and there's no use for a bearing without a freely rotating wheel, and a simultaneous dependency like that is a huge obstacle to biology, even though it's a hardly noticeable obstacle to intelligent engineering.

The entire human body, faced with a strong impact like being gored by a rhinocerous horn, will fail at its weakest point, not its strongest point.  How much evolutionary advantage is there to stronger bone, if what fails first is torn muscle?  How much advantage is there to an impact-resistant kidney, if most fights that destroy a kidney will kill you anyways?  Evolution is not the sort of optimizer that says, "Okay, let's design an entire stronger body."  (Analogously, the collection of faults that add up to "old age" is large enough that a little more age resistance in one place is not much of an advantage if other aging systems or outward accidents will soon kill you anyways.)

I don't even think we have much of a reason to believe that it'd be physically (rather than informationally) difficult to have a set of enzymes that synthesize diamond.  It could just require 3 things to go right simultaneously, and so be much much harder to stumble across than tossing more hydroxyapatite to lock into place in a bone crystal.  And then even if somehow evolution hit on the right set of 3 simultaneous mutations, sometime over the history of Earth, the resulting little isolated chunk of diamond probably would not be somewhere in the phenotype that had previously constituted the weakest point in a mechanical system that frequently failed.  If evolution has huge difficulty inventing wheels, why expect that it could build diamond chainmail, even assuming that diamond chainmail is physically possible and could be useful to an organism that had it?

Talking to the general public is hard.  The first concept I'm trying to convey to them is that there's an underlying physical, mechanical reason that flesh is weaker than diamond; and that this reason isn't that things animated by vitalic spirit, elan vital, can self-heal and self-reproduce at the cost of being weaker than the cold steel making up lifeless machines, as is the price of magic imposed by the universe to maintain game balance.  This is a very natural way for humans to think; and the thing I am trying to come in and do is say, "Actually, no, it's not a mystical balance, it's that diamond is held together by bonds that are hundreds of kJ/mol; and the mechanical strength of proteins is determined by forces a hundred times as weak as that, the part where proteins fold up like spaghetti held together by static cling."

There is then a deeper story that's even harder to explain, about why evolution doesn't build freely rotating wheels or diamond chainmail; why evolutionary design doesn't find the physically possible stronger systems.  But first you need to give people a mechanical intuition for why, in a very rough intuitive sense, it is physically possible to have stuff that moves and lives and self-repairs but is strong like diamond instead of flesh, without this violating a mystical balance where the price of vitalic animation is lower material strength.

And that mechanical intuition is:  Deep down is a bunch of stuff that, if you could see videos of it, would look more like tiny machines than like magic, though they would not look like familiar machines (very few freely rotating wheels).  Then why aren't these machines strong like human machines of steel are strong?  Because iron atoms are stronger than carbon atoms?  Actually no, diamond is made of carbon and that's still quite strong.  The reason is that these tiny systems of machinery are held together (at the weakest joints, not the strongest joints!) by static cling.

And then the deeper question:  Why does evolution build that way?  And the deeper answer:  Because everything evolution builds is arrived at as an error, a mutation, from something else that it builds.  Very tight bonds fold up along very deterministic pathways.  So (in the average case, not every case) the neighborhood of functionally similar designs is densely connected along shallow energy gradients and sparsely connected along deep energy gradients.  Intelligence can leap long distances through that design space using coordinated changes, but evolutionary exploration usually cannot.

And I do try to explain that too.  But it is legitimately more abstract and harder to understand.  So I lead with the idea that proteins are held together by static cling.  This is, I think, validly the first fact you lead with if the audience does not already know it, and just has no clue why anyone could possibly possibly think that there might even be machinery that does what bacterial machinery does but better.  The typical audience is not starting out with the intuition that one would naively think that of course you could put together stronger molecular machinery, given the physics of stronger bonds, and then we debate whether (as I believe) the naive intuition is actually just valid and correct; they don't understand what the naive intuition is about, and that's the first thing to convey.

If somebody then says, "How can you be so ignorant of chemistry?  Some atoms in protein are held together by covalent bonds, not by static cling!  There's even eg sulfur bonds whereby some parts of the folded-spaghetti systems end up glued together with real glue!" then this does not validly address the original point because: the underlying point about why flesh is more easily cleaved than diamond, is about the weakest points of flesh rather than the strongest points in flesh, because that's what determines the mechanical strength of the larger system.

I think there is an important way of looking at questions like these where, at the final end, you ask yourself, "Okay, but does my argument prove that flesh is in fact as strong as diamond?  Why isn't flesh as strong as diamond, then, if I've refuted the original argument for why it isn't?" and this is the question that leads you to realize that some local strong covalent bonds don't matter to the argument if those bonds aren't the parts that break under load.

My main moral qualm about using the Argument From Folded Spaghetti Held Together By Static Cling as an intuition pump is that the local ionic bonds in bone are legitimately as strong per-bond as the C-C bonds in diamond, and the reason that bone is weaker than diamond is (iiuc) actually more about irregularity, fault lines, and resistance to local deformation than about kJ/mol of the underlying bonds.  If somebody says "Okay, fine, you've validly explained why flesh is weaker than diamond, but why is bone weaker than diamond?" I have to reply "Valid, iiuc that's legit more about irregularity and fault lines and interlaced weaker superstructure and local deformation resistance of the bonds, rather than the raw potential energy deltas of the load-bearing welds."

The stuff Yudkowsky is reacting to is in ‘Nanosystems’ by Drexler. Looking in ‘Engines of Creation’ gets you the popularization, not the solid physics and chemistry. That’s all in ‘Nanosystems’, which shows how machines built out of covalently-bonded materials can be much more capable than biology. You may disagree with the arguments presented there, in which case I’d be very interested in your arguments. Unfortunately, by reacting to popularizations and tweets, you’ve inadvertently fought some straw men instead of your real opponent.

As another comment points out, when Yudkowsky says ‘proteins are held together’ he means how they are held to each other, not how they are held internally.

It’s somewhat of an exaggeration to say that proteins are held to each other by static cling. There are also hydrogen bonds. So it is more correct to say that they are held by static cling and surface tension.

The proteins themselves are primarily covalent, but a quick google search says that the forces in the lipid layer surrounding cells are primarily non-covalent, and the forces between cells seem also non-covalent. Aren't those forces the ones we should be worrying about? 

It seems like Eliezer is saying "the human body is a sand-castle, what if we made it a pure crystal block?", and you're responding with "but individual grains of sand are very strong!"

I personally would have liked more focus on "would protein actually be better with more covalent bonds," and less basics. But that's because I already know the basics, and maybe the average audience doesn't.

IMO, the more important strike against atomic-manufacturing bots is why it seems like nanobots (including cells) work better with diffusive transport and catalysis of reactions, rather than point to point transport and large energy gradients.

I think you're modeling the audience as knowing a lot less than we do. Someone who didn't know high school chemistry and biology would be at risk of being misled, sure. But I think that stuff should be treated as a common-knowledge background. At which point, obviously, you unpack the claim to: the weakest links in a structure determine its strength, biological structures have weak links in them which are noncovalent bonds, not all of those noncovalent bonds are weak for functional reasons, some are just hard to reinforce while constrained to things made by ribosomes. The fact that most links are not the weakest links, does not refute the claim. The fact that some weak links have a functional purpose, like enabling mobility, does not refute the claim.

Good points are made in other comments about the significance of weakest bonds, but mostly I want to say that I like this post because it's making a clear point with clear reasoning, and was very readable.

When discussing the stability of proteins, I mostly think of their folding, not whether their primary or secondary structure breaks.

The free energy difference between folded and unfolded states of a typical protein is allegedly (not an expert!) in the range 21-63 kJ/mol. So way less than a single covalent bond.

I have a friend who does his physics PhD on protein folding, and from what I remember he mostly simulates the surface charge of proteins, i.e. cares about dipole-dipole interactions (the weaker version of ionic bonds) and interaction effects with the surrounding water (again dipole-dipole afaict).

This suggests that vdW forces aren't all that important, but the energy scale you get from imagining vdW forces is still way better than when imagining covalent bonds.

Regarding how to do enzyme-like catalysts with covalent nanotech: my first guess is that we'd want to build a structure that has several "folded"/usable states close in energy, e.g. due to rotational degrees of freedoms in the covalent bonds. This way "unfolding"/breaking the machine requires a lot of energy, while it can still mechanically move to catalyze a chemical reaction at low activation energies.

Another effect that is very important is determining how proteins fold is the fact that they're dissolved in liquid water, and so hydrophilic parts of the protein want to be on the surface, while hydrophobic parts want to be on the inside, near other hydrophobic parts. This is largely an entropic force/effect.

Some other things that are true:

• 100% of the bonds in hydrogen gas are covalent.
• Most of the fundamental particles in a water molecule are held together by the strong nuclear force, which is a much stronger binding than covalent bonds.
• If you pull on a protein and stretch it apart, it looks like a long chain (maybe with a few crosslinks).
  • This is a non-zero amount of structure, but it looks nothing like the fully folded protein.
  • If we ask how the chain with the crosslinks is held together, the answer is covalent bonds that were either in the amino acids originally, or were formed when the ribosome assembled them into a chain, (or in the case of the disulfide crosslinks, were formed during the folding process).
  • But if we ask where the rest of the protein's structure came from, then the answer is hydrogen bonds and hydrophobic/hydrophilic forces.

Titotal, do you agree with Eliezer’s larger point that a superintellience engineering physical actuators from the ground up can probably do much better than what our evolutionary search process produced? If so, how would you steel man the argument?

This depends a bit on what you mean with "held-together" the covalent bonds are what holds the string together but the 3D structure needs the hydrogen bonds. If you exert force on the protein the hydrogen bonds will break first and the 3D structure breaks. 

The magic of protein folding is that you have a machine that can create a 2D string of amino acids gives you a 3D protein with a stable shape and reliably the same shape. For that to happen you likely inherently require that the 2D string is hold together is a way that's strong while it wiggles around and finds the desired 3D shape. If there would be a lot of ways to create stronger bonds it would likely misfold a lot easier. And when biology does need stronger bonds Cysteine with it's sulfide bond is available. If

Proteins are a good solution given the design constraints under which they operate, so putting them in the same category of the human retina is wrong, but I don't think your post does a good explanation as to why.

* I studied bioinformatics but it has been a while since I have been in my molecular biology lectures. 

I also read nanosystems and Drexlers blog. The entire concept of nanotechnology is a series of mechanical catalysts.

A mechanical catalyst is a carved out niche in the catalyst. You use nano robotics to shove in the inputs - you probably need different bonding strategies - and you press together the multiple parts of the catalyst.

Internally the shape of the catalyst allows only one possible product, and by controlling the inputs you control what gets made. (vs solution in glassware in current chemistry)

Proteins as catalysts are in fact shapes held together by many forms of bonds. This is why they are so fragile and they have failure nodes where a misfolded proteins catalyzes peers to fail. Their floppiness also allows side products and limits the complexity of the molecules that nature is able to build.

You obviously need to use a strong material for a mechanical catalyst. Diamond or platinum, etc, and its covalently bonded.

This makes the side reactions happen less often, and it theoretically could allow for higher temperature operation.

I also disagree with Yudnowsky on the feasibility of advanced AI quickly developing nanotechnology. I suspect it would require a large scale effort and a large amount of intermediate steps that a fugitive or low resource ASI would be unable to accomplish. But I think he's correct on this claim.

I found this to be an interesting discussion, though I find it hard to understand what Yudkowsky is trying to say. It's obvious that diamond is tougher than flesh, right? There's no need to talk about bonds. But the ability to cut flesh is also present in biology (e.g. claws). So it's not the case that biology was unable to solve that particular problem.

Maybe it's true that there's no biologically-created material that diamond cannot cut (I have no idea). But that seems to have zero relevance to humans anyway, since clearly we're not trying to compete on the robustness of our bodies (unlike, say, turtles).

The most general possible point, that there materials that can be constructed artificially with properties not seen in biology, is obviously true, and again doesn't seem to require the discussion of bonds.

Yeah, Yudkowsky doesn't know what he's talking about here.

Still, one of the ways protein engineers can make proteins more heat-stable is by adding more covalent bonds (in particular, disulfide crosslinks to prevent unfolding). See the many results for https://scholar.google.com/scholar?q=thermostability+disulfide+bond&hl=en