Humans, chimpanzees and other animals

I would argue that a significant factor in the divide between humans and other animals is that we (now) accumulate knowledge over many generations/people. Humans a 100 000 years ago might already have been somewhat notable compared to chimpanzees, but we only hit the threshold of really accumulating knowledge/strategies/technology of millions of individuals very recently.

If I would have been raised by chimpanzees (assuming this works), my abilities to have influence over the world would have been a lot lower even though I would be human.

Given that LLMs already are massively beyond human ability in absorbing knowledge (I could not read even a fraction of all the texts that they now use in training), we have good reasons to think that future AI will be beyond human abilities, too.

Further than this, we have good reasons to think that AIs will be able to evade bottlenecks which humans cannot (compare life 3.0 by Max Tegmark). AIs will not have to suffer from ageing and an intrinsic expiration date to all of their accumulated expertise, will be in principle able to read and change their own source code, and have the scalability/flexibility advantages of software (one can just provide more GPUs to run the AI faster or with more copies).

I think the divide could be "is potentially immortal and can make copies of themselves".

A weaker version of this ("potentially immortal and can make loyal servants") is a stereotypical vampire, if you excuse the fictional evidence [? · GW], and they seem pretty powerful, and their only weakness is that you can destroy their bodies -- which would be much more difficult if they could make copies that share the same goals and can update each other's memory.

*

The human superpower over animals is "can remember lots of things and cooperate with its own kind on a large scale, including learning/teaching across generations". Most other things are downstream of this.

Ants are also quite good at cooperation, and in absence of humans would be very impressive. If they had bigger brains, I could imagine them win.

The difference between humans and chimpanzees is purely one of imitation. Humans have evolved to sustain cultural evolution, by imitating existing culture and expanding upon it during a lifetime. Chimpanzees don’t imitate reliably enough to sustain this process and so their individual knowledge gains are mostly lost at death, but the individual intelligence of a human and a chimpanzee is virtually the same. A feral human child, that is, a human without human culture, does not behave more intelligently than a chimpanzee.

The slow accumulation of culture is what separated humans from chimpanzees. For AI to be to humans what humans are to chimps is not really possible because you either accumulate culture (knowledge) or you don’t. The only remaining distinction is one of speed. AI could accumulate a culture of its own, faster than humans. But how fast?

This is still n=1 for intelligence (and specifically individual intelligence aggregated by culture into "power to change the environment") over the super-dunbar organization threshold.  Other examples of intelligence distance are all low absolute values which leave other aspects of evolution and impact more important.

We just don't know where the next "threshold" is, where it's different enough that all other differences are irrelevant.  

When presenting claims that the cognitively superior agent wins, often the AI safety community makes an analogies with 2-player zero-sum games such as Chess and Go where the smartest and most ruthless players prevail.  However, most real-world interactions are best modeled by repeated non-zero-sum games.

In an ecological context, Maynard Smith and Price introduced the Hawk-Dove game to try to explain the fact that in nature, many animals facing contests over scarce resources such as mates or foods engage in only limited conflict rather than wiping out rivals (Maynard Smith and Price, 1973); for example, a male stag will often yield to a rival without a fight.  



When we model how hard-coded strategies for this game propagate via genetic reproduction in a large well-mixed population, it turns out that, for certain payoff structures, peaceful strategies ("Doves") co-exist with aggressive strategies ("Hawks") in a stable equilibrium (see https://sphelps.net/teaching/egt.html for illustrative numerical simulations). 

The Hawk-Dove game is also sometimes called "the Chicken Game" because we can imagine it also models a scenario in which two opposing drivers drive on a collision course and simultaneous choose whether to swerve or drive straight.  Neither player wants to "look like a chicken" by serving, but if both players drive straight they crash and die.  

In the Chicken Game, cognitive superiority does not always equate to winning.  For example, by pre-commiting to driving straight and making this common-knowledge we can beat a rational opponent whose best response is then to swerve.  Similar arguments were put forward during the cold-war to argue for removing rational deliberation from the decision to retaliate against a first strike by the enemy by removing humans from the loop, and putting strategic weapons systems on hair-trigger automated alert, because in the absence of pre-commit a threat to retaliate is not credible since there is no aposterior advantage to retaliation once the opponent actually strikes.  Notice that this dynamic is the exact opposite of the power-seeking behaviour posited by reinforcement-learning agents which seek to maximise expected utility by expanding their possible choices. With non-zero-sum games, in contrast, it can make sense to reduce one's choices.  Moreover, the cognitive capacity required to enact policies over reduced choices is lower -  stupid can beat smart.      

Although Maynard Smith and Price originally formulated the logic of animal conflict in terms of evolutionary adaptation, the same dynamic model can be used to model social learning by boundedly-rational agents (see Phelps and Wooldridge, 2013 for a review).  As regards the cognitive capacity of non-human animals to deliberate in Hawk-Dove interactions, see Morikawa et al. (2002).

References

Morikawa, T., Hanley, J.E. and Orbell, J., 2002. Cognitive requirements for hawk-dove games: A functional analysis for evolutionary design. Politics and the Life Sciences, 21(1), pp.3-12.

Phelps, S. and Wooldridge, M., 2013. Game theory and evolution. IEEE intelligent systems, 28(04), pp.76-81.

Maynard Smith, J. and Price, G.R., 1973. The logic of animal conflict. Nature, 246(5427), pp.15-18.

There is a threshold where intelligence becomes much more useful, and this threshold is an ability to make better weapons than other animals have. In a world where this is not possible at all (for example, animals have zerg-level natural weapons, and there is no metal to make guns), having human-level intelligence is just not worth downsides from a big brain.

From David Chapman's "Better without AI", section "Fear AI Power":

"The AI risks literature generally takes for granted that superintelligence will produce superpowers, but which powers and how this would work is rarely examined, and never in detail. One explanation given is that we are more intelligent than chimpanzees, and that is why we are more powerful, in ways chimpanzees cannot begin to imagine. Then, the reasoning goes, something more intelligent than us would be unimaginably more powerful again. But for hundreds of thousands of years humans were not more powerful than chimpanzees. Significantly empowering technologies only began to accumulate a few thousand years ago, apparently due to cultural evolution rather than increases in innate intelligence. The dramatic increases in human power beginning with the industrial revolution were almost certainly not due to increases in innate intelligence. What role intelligence plays in science and technology development is mainly unknown;"

Good point. As I understood it, humans have an OOM more parameters than chimps. But chimps also have an OOM over a dog. So not all OOM's are created equal (so I agree with your point).

I am very curious about the qualatative differences between humans and and superintelligence. Are there qualatative emergent capabilities aboven human leven intelligence that we cannot imagine or predict at the moment.