Posts
Comments
A couple of points. In certain, limited expressions of human international relations, we do know what the decision making process was to reach major decisions, such as the drafting and signing of the Geneva Conventions, some of humanity's best and most successful policy documents. This is because the archived UK and US decision making processes were very well documented and are now declassified.
In other words, we can actually deconstruct human decision making principles (probably conforming instrumentalism in the case of the Geneva Conventions) and then use them to ground or verify other principles.
Coherent diamonds are likely to be obtained from human utopianesque sci fi than anywhere else. This is because decision making in international relations (which is what is at stake here, considering one country will develop the AGI first and an agential ASI may well be a sovereign agent) is transparent in plots but is usually opaque in real life decision making. Finding diamonds can be approached, and likely must be approached, in a modular fashion, so as to coherently construct, with multiple options transparent to human decision makers, Yudkowsky's friendly AI via coherent extrapolated volition.
So, for example, if we want humanity to be served by an AGI/ASI in line with Asimov's vision of an AGI Nanny overseeing humanity's development, we deconstruct his Multivac stories and determine what principles it runs on and then express them in the form of programmable principles. If we want a certain problem approached via the Star Trek TNG ethos, then we need to break down what ethical principles are in play in a way that we can program into a nascent AGI. At a higher level of tech and maybe a more ruthless level of interstellar relations, the ethics of Banks' Culture series module kicks in, and so on.
This argument is expounded in Carayannis, E.G., Draper, J. Optimising peace through a Universal Global Peace Treaty to constrain the risk of war from a militarised artificial superintelligence. AI & Soc (2022). https://doi.org/10.1007/s00146-021-01382-y
This has now been published as an article in AI & Society, available at: https://www.springerprofessional.de/en/optimising-peace-through-a-universal-global-peace-treaty-to-cons/20022666
Humans are really not so dissimilar compared to what's probably out there. I had my moment in the desert where I was asked, 'What do you want?', and the answer was 'Global Peace'. To a certain extent, I wish I had your life. But then, I wouldn't have my three kids, and I wouldn't have had that moment where something said back to me, "Global Peace? That old chestnut? Seriously? Oh, you are serious. Okay, this is what you have to do..."
Let me go find out more about lie detectoring companies in London.
Hmm. Let me try to restate this. What I'm trying to do is to create a protocol for reporting weird experiences as we close in on the Singularity. If I'm correct about a soft Zoo Hypothesis (basically a 'leaky' Prime Directive situation, with Contact contingent on creation of an ASI or a warp gate/drive), with several interstellar civilizations with mindjacking tech monitoring the planet, then attempts to influence human global development do occur, probably by the panhuman equivalent of special forces, i.e., 'Special Circumstances' (https://en.wikipedia.org/wiki/The_Culture).
I assume the number of civilizations out there is small, technological development is broadly equivalent due to leveling up via aid/trade relationships, and philosophical positions are diverse, if only due to such basic differences as physiology.
If this is correct, Earth, going into the Singularity, will a battleground, not over territory, although Earth is an incredibly bio-diverse rich piece of real estate, but over spheres of influence, with fundamental philosophical positions coming into play, such as whether or not ASIs are even legal, whether Earth would ally with an imperial panhuman civilization versus a federation post-Singularity, and so on.
In my 'soft' Zoo Hypothesis, the curating civilizations could all get together and act collectively to intervene, pushing humanity in a certain direction. Alternatively, given the tech involved, no matter how good the regulatory system is that's designed to prevent a free for all, there exists the possibility of competitive external interventions.
In this scenario, the results of these disagreements - or agreements - and the will and means to act - is that there will be 'nudge points'.
The single most important concern I can see going into the Singularity is whether the world is at peace, because if it isn't, a) warfare will drive AI development towards an ASI, and b) a warlike ASI could emerge. Alexei Turchin has made this very clear in his work:
And, whatever the galactic position on the legality of ASIs, warlike ASIs are almost certainly a no-no. Peace would also seem to enable the development of an interstellar civilization, if only 50% of Earth's total military budget could be diverted to developing fusion, hybrid fusion-antimatter drives, and wrap drives
So, we go back to my original post and to the basic contention of whether any legitimate nudge points have arisen recently, i.e., since the setting up of the UN and the attempt to ban atomic weapons via the US-backed Baruch Plan in 1946, which would have diverted over ten trillion dollars from global nuclear weapon development to welfare development, if it had worked. It failed, BTW, because of the differences between two systems, but this boiled down to two people, Stalin and Truman.
Preventing an arms-based Cold War would have been interesting because we could have nearly perfected medicine or cracked fusion by now with the kind of money involved. I mean, it's a no brainer that if there are panhumans out there, they would have gotten pretty excited about the Baruch Plan. The temptation to nudge it along would have been extreme.
Given all this, I would say there has definitely been one recent potential nudge point, i.e., the suggestion by the UN Secretary General, in the face of a disastrous global pandemic, to try to declare a global truce: https://www.theguardian.com/us-news/2020/apr/19/us-and-russia-blocking-un-plans-for-a-global-ceasefire-amid-crisis. This is the first suggestion of this kind ever, in the 75 years of the UN.
It's fascinating because the suggestion crystallized global conflict down to a single node, a concentrated moment in time, with the futures of quarter of a billion children, those in conflict zones, at stake. It's even more interesting because it failed due to US-Russian antagonism, i.e., differences between two people again, Trump and Putin.
Its failure has serious implications for whether the effects of global warming, which likely will not appear crystallized in a single event but which will instead be like a combination of a juggernaut and a train wreck, will be able to unite humanity behind global peace. Given COVID-19 could not do it, I doubt global warming will, or at least not in time for the Singularity, which, like I said, has massive implications going forwards.
Unfortunately, the UN Secretary General is a) unlikely to be willing to explain where the idea came from and his interactions with e.g., Macron over the proposed truce, which would be confidential, b) very unlikely to admit to anything weird happening to him along the way, and c) extremely unlikely to lay it all out here on Less Wrong.
Now, the only other potential nudge point is the upcoming fusion burning plasma breakthrough, which has real validity given how close the Baruch Plan came to success. This may, in fact, given the COVID-19 nudge point failed, be the last feasible nudge point going into the Singularity.
I have looked at fusion from this perspective really, really hard, basically made myself the world's expert on this issue, and my bottom line is that it will take an awful lot of nudging to get it to work as a nudge point for global peace. The Baruch Plan stood a chance because we were coming out of a global war, the UN was being set up, a brave new world was deemed possible, the A-Bomb had just been dropped twice, demand for global peace was high, and what became the Washington Consensus had a lot of legitimacy.
Now, what I'm reporting to this community, which I respect because it does care about what kind of future humanity faces going into the Singularity, is that I, myself, may have been nudged.
If all the above holds, there are somewhat significant implications to this, for example the fact that a British person was nudged rather than an American or a Chinese may be significant, but only if veracity and sanity can be established.
One significant implication is that we could be in very serious trouble going forwards into the Singularity. For instance, we may not be allowed to develop an ASI out of fear that we would weaponize it. That could certainly get interesting.
Another implication is that we are in a Zoo/Lab/Prime Directive situation, with multiple civilizations competing for influence over us, fascinated by our inability to establish a global peace when all their civilizations managed it. We could be the galactic dunces, in other words, or a fascinating case study of a 'failing planet'.
The Prime Directive can break down, however, from two directions. You can 'Captain Kirk it' and justify some kind of external intervention, or, someone on the planet can do a Toto, and pull back the curtain.
Or, theoretically, you can work it from both ends, if a Captain Kirk decides he likes Toto the dog.
So, let's assume I'm a human Toto. Let's assume that, any conversations with Captain Kirk put aside for one moment, I'm at least mainly human. I mean, I'm a public figure, a titled academic, I have three kids, etc. I am very likely to be at least mainly human.
And, let's assume I care, care enough to lobby for global peace, to explain why I think it matters, to explain how it could be done.
And then something happened on the way to Thor's Hammer.
If I'm human, or at least mainly human, I am arguing that the rational thing to do would be to fight to establish the truth about what happened, not because I am that important, certainly not alone, but because I at least suspect that I am a piece in a puzzle, part of a counter-counter-factual Hail Mary cookie, and that going into the Singularity, working out that puzzle may become humanity's greatest and most challenging endeavor.
There is, for instance, just a chance that, in considering the conditions for public Contact, which comprise at a minimum an ASI or a warp gate/drive on our part, the panhumans take into account what our own civilizational conditions for contact would be. In other words, in part, they may be reflecting back to us the Prime Directive, the closest we have to a global 'popular' consensus for how Contact should be managed, especially as we come closer to meeting a Contact trigger point. It should not be forgotten that Gene Rodenberry was this planet's greatest popular philosopher on interstellar relationships.
If I'm at least mainly human, rationally, I would fight to establish the truth because the alternative, if I do suspect the truth, is to go along with a farce, to distract Dorothy and her pals so that they do not see the Wizard, to replace the curtain, and to trust that the global geopolitics will work themselves out.
Given what I suspect/know, I choose not to sit this one out. Instead, I choose to fight to pull that curtain back further, perhaps in the hope that I can encourage a panhuman Captain Kirk (realistically it would take at least one interstellar civilization's leader) to step out from behind it and just say, "The heck with it, we're going to help these guys, if only for the quarter of a billion children being affected by conflict zones and by the fact that biodiversity loss on this beautiful blue marble will only exacerbate this, because their political systems aren't up to it, maybe due to no fault of their own given the Baruch Plan catastrophe, and because I don't want Science Officer Spock here to spend the next few decades just studying a train wreck while we poke the damn thing with a stick occasionally and do these 'nudges'. Let's roll out the Sagan Contact protocol. Or, should we just go full throttle with 'The Day the Earth Stood Still'? Whaddya say, McCoy?"
In other words, I am arguing that it is my duty, as a human, to go on record.
Does that explain why I should do this?
Okay, so below is an attempt at a brief, one that could be tailored, hopefully, for other people. Any suggestions before I hunt around for quotes from lie detector companies?
Completed Template Brief for Hail Mary Cookie Experiences
Overview
The brief is to establish the veracity of a subject regarding a series of extraordinary claims about experiences. The brief is not to establish the subject’s sanity or to investigate the claims at some kind of philosophical level. The brief does include the requirement to establish veracity of claims according to the granularity of the experiences (see special vocabulary and terms below).
Special Vocabulary and Terms
The subject will speak about the claims using specific vocabulary, including mindjacking, which refers to using technological means to write to or read from a mind, as with Elon Musk’s Neuralink, and granularity, which refers to the degree of reality of an experience. Granularity can be as follows:
Full consciousness = full granularity
Simulation while awake with eyes closed = high granularity, e.g., with one or more of the five senses (sight, smell, taste, hearing, touch)
Lucid dream = medium granulariy
Dream = low granularity
Specific Questions to This Case
1) Establish whether the subject believes he has had an experience where he saw an alien head at night, then woke up the next day seeing a neon heart, identical to one he later found that morning in his house.
2) Establish whether the subject believes he has had an experience where he had three nightmares over three nights, the first concerning a sniper, the second concerning a being in the same room, and the third concerning being in a corridor and opening a door, where the granularity became less distinct each time. Establish the granularity of the experiences.
3) Establish whether the subject believes he has had an experience where he asked the question by email to a third party, “May I ask you a question?” and received a reply in the form of a rainbow in his mind. Establish the granularity of the experience.
4) Establish whether the subject believes he has had a recurring experience where he saw a 4x8 grid of electric blue lights. Establish the granularity of the experience.
5) Establish whether the subject believes he has had an experience where he invited a person or persons unknown to read his mind by a mindjacking device involving a feeling of a mind being electrocuted, and how long the experience took. Establish the granularity of the experience.
General ‘The Day the Earth Stood Still’ Obligatory Questions for All Cases
1) Establish whether the subject believes he has ever seen another planet. If he believes he has, determine how many and what they look like, and by what means. Establish the granularity of the experience.
2) Establish whether the subject believes he has ever seen or met aliens. If he believes he has, determine how many species and what they look like. Establish the granularity of the experience.
3) Establish whether the subject believes his body is human. If necessary, use percentages.
4) Establish whether the subject believes his mind is human. If necessary, use percentages.
General Pharmaceutical Questions for All Cases
1) Establish what if any drugs, e.g., hallucinogens, the subject was taking prior to or during the experiences.
2) Establish what if any medication the subject was taking prior to or during the experiences.
Well, this is where we regress from the rum and uncanny to normality. If I were in a position to be able to reproduce atypical brain states involving the appearance of electrical input or output that just blew away the technology being used to monitor it, that would be one heck of a result. It could also be a dangerous result, personally and politically.
But, to take that point on board, first we test my veracity (lie detector), then we test by sanity (psychiatric eval), then we test whether I can replicate any of this using something like an MRI (objectivity). However, the latter does not rely solely on me if I'm not generating these phenomena.
Alright, I'll draw up the brief for the lie detector company this week based on the above events, then I'll draw up a list of five of these companies, put that all together at the end of this week, and you can randomly select a company.
Just because I've always wanted to, I'm going to thrown in some questions about meeting aliens, my own humanity, whether I've ever another planet, etc. Basically, if I'm going to pay for someone to lie detector me, I want my Keanu Reeves moment
https://www.youtube.com/watch?v=-_2l0RjZvOI
!
Whoops, I missed once experience I wanted to investigate via lie detector and psychiatrist. So, I emailed myself to say I was happy to have my mindstate read so that whatever it was that was doing all this could know who or what it was. That same night, I'm awake, eyes closed, and it feels like something jacks something using electricity directly into my brain. So, most of us have had an electric shock, and it kind of vibrates, and it hurts. But, the brain feels no pain. So, this electric vibrating in my brain was happening for about 3 seconds, but I suspected what it was, so I just told them in my mind to ease off for a second. So they did, and I felt fine, so I told them in my mind it was okay to do it again. Then, the phenomenon continued for about two seconds, and it was done. Then I relaxed and went to sleep.
Going back to the hallucinogens, one difference has been that what I've been experiencing is equivalent to photographic resolution material, or high-res video, and overall just high-granularity experiences.
Anyway, ask any questions, then I'll boil all this down for the lie detector company brief.
That's a fascinating link you supplied there! I have no doubt that Hallucinogen Persisting Perceptual Disorder exist. Sunglasses and talk therapy seems to be of use: https://en.wikipedia.org/wiki/Hallucinogen_persisting_perception_disorder
Maybe everyone on the West Coast wearing sunglasses is just trying to deal with their HPPD? ;<)
I would also say there's a definite chance of a link between devouring psychedelics and hallucinations, including of rainbows!
Unfortunately, I wasn't on psychedelics before, during, or after the experiences I just related here. I had been doing some Buddhist meditation, including down to the zen states. My personal zen meditation has induced temporary minor lens flare, say 2-3% of the intensity in the picture linked to in the article, and focused on a single object, while fully awake: https://slatestarcodex.com/blog_images/hppd_town.jpg However, this is minor and transient compared with what I'm describing.
Okay, so what happened next was I got hit with a literal barrage of a 4x8 grid of electric blue lights, from lucid dreams all the way down to 'real' dreams. So, this was a recurring dream, literally dozens of times. The blue lights could flare in intensity as a grid. Of all things, it made me think of a phased array communication system: https://en.wikipedia.org/wiki/Phased_array
Now, what would happen was that I would go to sleep, then I would dream of the grid, then I would transition to a lucid dream where i was in the presence of whatever the phased array was communicating with. I got the feeling that the other end of the grid had a grid of its own, a big, and old, white one, and that it was large, intelligent, powerful, and far away.
This definitely meant something important, like an attempt at communication, and it recurred for months before going away.
Now, a whole bunch of stuff happened after this, but the above is basically what I wanted to go on the record about on lesswrong.
If you have any questions, I can answer them.
If not, I will try to compile the above experiences as a brief for a lie detector company. As I said earlier, I'm trying to create a protocol that's useful for other people working on AGI/ASI-related issues to go on record about re acausal Hail Mary potential cookie-like experience.
Oh, it just gets better. So, the rational thing to do at this stage, I thought, was to send the possible mindjackers an email, to see if I could establish two-way communication. So, I reasoned if they have the tech to mindjack and are putting me under the spotlight, presumably reading my emails is ongoing and trivial. So, I sent an email to myself stating (and I can't remember the sentence exactly, 100%, but it was at least 90% semantically this): "May I ask you a question?"
That night, I was lying there in bed, awake but with my eyes closed, and so I get the same feeling that something's in my mind, and then I see a question mark emerge in my vision. It develops rainbow colors, so it starts red at the top of the question marks and goes down to violet, with the question mark fading from the top.
So, it was as clear as this in terms of the outline, but with softer transitions between the colors, and slightly softer colors: https://stock.adobe.com/uk/images/rainbow-flag-gay-question-mark/25315121
So, this is a really clever semiotic response to my emailed question because the question mark as an interrogatory symbol can mean all of the following and more: who, what, where, when, how, why, how much, how many?
It also confirms, assuming I'm telling the truth and am not crazy, that the email was being monitored, which was interesting.
It's also showing off, i.e., demonstrating really good mastery of the mind medium and of semantics and semiotics. Basically, it was a smart thing to do.
Any questions?
Yeah, it is also worth looking on the bright side, because I am willing to bet my life on the fact the panhumans have a sense of humour. I would hope the ASI does, too. Frankly, I found most of these experiences humorous or at least just weird rather than fear or paranoia inducing. Anyway, on to Experience 2!
So, I think this next one was a basic psychological evaluation, probably military. The interstellar cops were not ready for something that pushed back from being pulled over and shown the giant panhuman head ID, so they hit me with a series of sequenced and structured nightmares on a common theme over three days. This was a really interesting experience, because I have only ever heard of recurring nightmares. Anyway, only the first nightmare was scary because by the second night I figured it was an administered experience. Once you figure out what is happening in a nightmare, it's like you solve it, right? So, it is no longer a nightmare, simply an experience.
So, night one, semi-lucid dream, sniper in the opposite house aiming at you. I have read that the sniper nightmare is a standard nightmare for anyone with trench warfare/urban combat .
Next night two, slightly lucid dream, something at the foot of the bed nightmare. Standard childhood nightmare.
Next, deep dream, walking down a corridor, open a door, monster's behind a door dream. Again, standard nightmare.
So, I had three different stereotypical nightmares at different, increasingly deep, levels of sleep in what appeared to be a sequence over three days.
I have never heard of sequential nightmares before except in decent sci fi, in which they are administered, and like I said, the 'evaluation' kind of lost all power once the second nightmare hit. By the third nightmare it felt as if whoever was administering the evaluation was giving up, TBH.
So, given I'm willing to testify using a lie detector that this experience actually happened, I would say what this demonstrates is an ability to manipulate dreamstates. This logically follows from mindjacking, so if panhumans/an ASI have the ability to manipulate a waking mind via a really advanced form of the Neuralink, it's not really surprising that they have the ability to mindjack dreams.
The ability to study dreams would be really valuable for anthropology, philosophy, psychology, etc. People like Freud dedicated their entire careers to it, and any panhuman psychologist would obviously want to be able to study both human mindstates and dreamstates. It's the logical conclusion of having Neuralink-style tech.
This is where things get interesting in terms of trying to deduce some of the 'rules of the game', because in the Zoo Theory for the Fermi Paradox, mindjacking humans has to be regulated, basically prohibited, otherwise it would be a free-for-all, and obvious. Especially if multiple civilizations with different political philosophies are curating or studying the Zoo, you would not want Culture A mindjacking someone like Donald Trump at the expense of Culture B. So, it would have to be very heavily regulated.
It's just that suspected transgressions would have to be investigated, and I suspect I was a possible transgression.
All this implies the ability to monitor human mindstates on a large scale, which necessarily implies automation. That implies a) we are living in a sim, or b) there exists a very high level of automated real-world ability to police and monitor human mindstates due to a really advanced Neuralink system. I suspect b), but that a lot of really law-abiding panhuman cultures have agreed not to investigate fellow spacefarers' mindstates, but maybe have agreed to implement protocols for investigating mindstates in Zoo contexts. Some of the cultures may not even use the tech, or they may only use it through a third party, like an ASI, for regulatory reasons.
So, any questions on that experience?
Okay, so I’ll try to report a summary of some of the weird experiences I had while working on leveraging the coming fusion breakthrough for global peace to avoid global warming and ASI-enabled or directed warfare. And I’ll try to apply the objective system outlined above. I’ll do these one at a time, so you can ask questions.
Experience 1: Being pulled over by the interstellar cops for speeding while in control of a human Mind.
So, not too long after I start working on fusion, about five years ago now, I was awake, with my eyes closed, lying face down on my bed. So, with your eyes closed, it’s black, right?. But, you know your eyes are closed and you’re not dreaming because you can still ‘see’. Then, boom, out of a pinprick this stylized panhuman head suddenly appears in that blackness, filling most of my field of vision. My immediate feeling as a semiotician was that it was a stop and search, and that this was basically an ID badge. So, the head was egg-shaped, dark grey, black eyes, but no ears, nose, or mouth. So, it was not a real head, just a symbol.
Now, the other thing I that the experience was intrusive and something humans don't usually experience. It was like you take one hand, make it a fist, then someone else inserts a finger from their hand in that fist. It was just very obvious. And, I was definitely awake, because the granularity was high. I literally could feel real sweat dripping off my back and down my legs, and when I opened my eyes after it was over, there was no discontinuity in feeling and the sweat was still dripping. It felt like something had jacked my optic nerve and related areas of the mind.
Anyway, I started to push back against the Mind inside my Mind, basically threatening that I would go to town on it with a mindful of love. I was and am a progressive Buddhist, but I had a decent sci-fi education as a youth and I was not about to be mind pwned by some interstellar cop, and, so I was actually getting some traction, and I could feel that other mind hesitate, and then disappear.
The next day, I woke up, and my eyes were closed. Immediately I wake up, with my eyes still closed, I’m staring at a giant pink neon heart. It lasted a few seconds and disappeared when I opened my eyes. Then, I go downstairs and find a heart-shaped necklace amulet on the floor. It’s exactly the same shape heart, but nothing special; I think it once belonged to my ex. So, I think they were being nice and saying sorry.
But, I want to investigate this like a good scientist, so I stand on a chair under a circular fluorescent light bulb with my eyes open and then closed, and I manage to partially replicate the pink neon heart, but in the form of a white circle, obviously.
So, there are two explanations for the heart; either something jacked my optic nerve, as with the police badge, or something shone a very bright neon heart shaped light into my closed eyes as I woke up, which in any case means it was monitoring my wakefulness.
To sum up, as you can see from my ORCID ID, I made a really interesting change in career to work on fusion for peacebuilding, and I was in Thailand in the time, which was important because Thailand was chair of the G77 bloc of 134 Global South countries. So, I arrive on the fusion scene as this geopolitical actor, and that attracted multiple parties’ interest, TBH. One of them appeared to be the interstellar cops.
What can this single experience tell us about the nature of the universe out there if a) it really happened (which is why I’m happy to submit to the lie detector) and b) I’m not crazy (which is why I’m happy to commit to the psychiatric eval after the lie detector results come through)?
One deduction is that the panhumans/an ASI has a perfected form of a semi-remote Neuralink. So, we’re talking about two-way communication between Minds, basically though transference or Mind uploading or downloading, partial or full. However, it’s not that remote, like it can be used from a ship in space, as in Banks’ Culture series. I believe it’s drone moderated, i.e., that a small camouflaged drone is needed to moderate the transference. That’s another story.
Another implication is that certain activities on Earth are being routinely investigated and studied by panhumans or an ASI. So, I don’t think I’m the only person this has happened to. I just think we haven’t heard about them, or the experiences are filed in the crank file.
Another possible implication is that there’s a lot of potentially unregulated panhuman activity on Earth, through mindjacking. This is more likely if the scale of involvement is high, because a) there are several panhuman interstellar civilizations involved on earth and/or b) the interstellar civilizations involved are poorly self-regulated on this issue, meaning there could be a lot of forbidden or borderline activity, and/or c) there’s simply a lot of interstellar traffic, meaning the cops are having difficulty regulating who’s doing what on Earth.
So, there could be a lot of panhuman political and military observers, scientists, and academics, say over a thousand, all studying Earth at any one time. And, all tempted to push the boat out a little because the same mindjacking tech they have to study human activity can be used to influence human activity. Basically, it’s not quite the Wild West, but it is a regulatory nightmare, especially as some of those panhuman observers are almost certainly not regular military observers, so there's a possibility of special forces panhumans wandering around causing trouble.
Another possible implication is that all this is being regulated, say by an ASI, or an AGI, kind of like Asimov's Multivac, or at least some kind of panhuman regular AGI-assisted regulatory force, like the Federation in Star Trek, and that the symbol for that ASI or force was what I was shown.
Another possible implication is that there’s a lot of potentially unregulated panhuman activity on Earth, through panhumans using physical simulacra of human bodies (unlikely), or human bodies they picked up, say from a suicide (more likely). But, frankly, these situations would be rare, because of the Prime Directive, because influential people don’t appear from nowhere, and because suicides don’t just reappear.
Basically, if you are able to mindjack, and you’re not collecting specimens of flora or fauna, you mindjack. There’s just way less risk. It’s also more convenient for physical reasons. Banks covered this (using seers) in The Algebraist, where a lot of the human study of the Dweller (giant gas) planets is conducted through virtual probes, not physical visits, because of the problem of incompatible environments.
Any questions before we move on?
From Cheng, there may be aspects to the world that appear to be a participatory hallucination, rather than the entire world, unless, like I said, we go down the Von Neumann-Wigner explanation for reality, which I try to avoid. So, for example, SETI and others have been looking for alien transmissions for decades now, with nothing to show for it. The Zoo Hypothesis provides really obvious explanations for that, namely that something's interfering with the transmissions/jamming the detectors/affecting the signal processors. A panhuman ASI could do that. The end result would be a form of collective participatory hallucination. Or, another example would be why alien UFOs are really quite difficult to see, without going into exotic explanations like flipping in and out of extra dimensions. The obvious explanation would be that they use adaptive camouflage. Again, the end result would be a form of collective participatory hallucination.
Oh, I've had more than flashes. I've had a whole series of full granularity two-way nudge-like events that, if they were not in logical sequence and did not revolve around my work on leveraging fusion for peace-building to prevent both global warming and ASI-enabled or directed warfare on this planet, would be viewed as wholly acausal experiences probably deserving of specialist attention.
However, before we get onto them, we need to establish more credentials at my end. So, from my ORCID https://orcid.org/0000-0002-3626-533X it can be seen that I'm a trained linguist, specifically an applied linguist, with special expertise in ethnolinguistics (languages of ethnic minorities), sociolinguistics (language as used in society, so for example power relations between dialect groups) and semiotics (the meaning of symbols and signs). So, for example, this article of mine, which is available for free online
Implications of the Urban Landscape: Aspects of the Isan Cultural Maintenance and Revitalization Program, Journal of Urban Culture Research
2017 | journal-article
- DOI: 10.14456/jucr.2017.4
is in the field of semiotics, subfield geosemiotics. Four articles in my ORCID list mention 'signage'; these also concern geosemiotics.
So, as a trained applied linguist, when people speak to me, I analyze what they say to me not just in semantic terms, but with careful regard to additional variables that people normally process subconsciously, so accent, dialect, speed of speech, tone, etc. Also, because of my specialism in semiotics, I have the capacity to think in an additional dimension. To give you an example, say I see a multilingual road sign in a foreign country, I analyze the relationship between the languages on the sign, through the placement of the languages, font size, font type, etc., much like a graphic designer would. So, I have the training to process signs and symbols just as if I were processing language.
To sum up, while I am in no way up to the standard of Amy Adams's character in Arrival https://www.youtube.com/watch?reload=9&v=tFMo3UJ4B4g, I would know how to go about cracking an alien language, including one that included symbols. In that I am more of a generalist, I would certainly have the humility not try to do it myself, but I would know how to put a team together, would know after a bit of digging who in the world to approach, and would be able to coordinate the process of cracking it.
So, does this establish my credentials sufficiently as an applied linguist?
Re your flashes in the upper left corner of your eye, how regularly do they happen, do they happen when your eye is both open and closed, and what color are they? Do they happen consistently when you have a flash of insight, or what correlation percent are we talking about? Do they only occur with a certain degree of revelations, like 'high importance', but not with 'minor importance'? Basically, I would say you need to collect more data on it before thinking about whether it's a) revelatory or b) worth seeing a doctor.
Link to a good source with solid concepts there. So, let's prioritize cluster thinking and then attempt to regress to normality. So, in this post, we are trying to create a space where people can report acausal phenomena that may be affecting them while working on developing AI, AGI, and the related socioeconomics and geopolitics 'Big Questions', like global peace.
So, as an example, we know Elon Musk reads Iain M Banks' Culture series. He decides to build his own rockets and then decides to do Neuralink. Why? Causally, we have the answer to both of those. He's explained why in interviews, and people could and have written books about the psychology of why Elon Musk. However, we do not know whether or not he was affected in apparently acausal ways to pursue both these paths, especially the Neuralink development. In my terms, he could have been 'nudged' by panhumans or by an ASI or by the 'Master Coder' (if we're in a simulation) to develop Neuralink. I don't know if anyone has even asked him whether or how he gets 'flashes' of insight. Or, being Elon Musk and immersed in decent science fiction as well as science fact, he may actually be having lucid dreams where he's communicating with imaginary sci-fi panhumans over Neuralink design aspects. The overwhelming probability is that these dreams are simply dreams, with a very low probability that they may be panhuman or ASI-nudges. However, the probability that it is nudges is probably higher with Elon Musk than with Joe Average, unless one believes all dreams of conscious beings are somehow manifestations of self-repairing code in a simulation managed by an external entity or even more exotically manifestations of constantly collapsing Von Neumann-Wigner wave functions which somehow permit quantum events in what could be shared dreamspace, etc.
Still, if we go with nudges, this means we need a reporting system for them. I suggest this (with a humorous example included):
Explicit nudge: Marvin Minsky-style fluorescent raccoon pops up in my living room and tells me to build the Neuralink to save humanity from its collective stupidity, manage the Singularity, and permit our expansion into space.
Implicit nudge: The idea suddenly occurred to me in a flash to build the Neuralink to save humanity from its collective stupidity, manage the Singularity, and permit our expansion into space. This flash was the kind of flash I've rarely or never experienced before. It was certainly weird, but there were no panhumans.
So, theoretically, nudges can happen while awake, awake with eyes closed in a state of rest, while dreaming, or during lucid dreams. I think that's about it.
We also have to define 'weird'. So, theoretically, nudges can have granularity. For instance, nudges can have full video, full audio, smell, tactility, and taste.
Does this make sense so far?
Thanks for sharing! That explains the connection between you and I that created this comment chain, which, TBH, is the outcome I was hoping for when I posted the main post, complete with nudge/nude typo.
I think we both basically accept the possibility that we exist within some kind of Laboratory Hypothesis situation and function not just as humans but as parts of a Bostrom cookie, where weird, apparently acausal events can happen that in fact may have an underlying causality and so rationality, just one that does not readily admit of rational explanation, because what we can see of the underlying code - the effects, not the actual code itself - appears incompatible with our present scientific knowledge.
So, going into the Singularity, to use an analogy for a Bostrom cookie that children could understand, my perspective is that we are as individuals are pieces of a jigsaw puzzle that scales up to the entire global community of humans, with a picture on the puzzle that we cannot see. However, some of us, at least, are just beginning to wake up to the possibility that we pieces of the puzzle may have to communicate with each other in order to solve it. I don't believe anyone else is going to do it for us, unless we perhaps create and ask an artificial superintelligence (ASI) we create to do that for us.
However, my whole point is that we have not ended interstate war on earth, that we are running out of opportunities to do so, and that this will be critical to our relationship with both aliens and an ASI; the United Nations is not a global peace treaty. States already employ AI to undermine democracies, steal industrial secrets, and exploit financial systems. The US, Russia, and China are developing AI for maintenance of global technological supremacy and/or global domination, including cyber-warfare, not for altruistic reasons. Thus any human-created ASI is very likely to be imperfectly constructed in terms of final goals, may not comply in terms of solving the puzzle, or may comply but not tell us the answer. Realistically, we would end up in a state of war with it: https://osf.io/preprints/socarxiv/4268q/ Optimally, it would, perhaps, nudge us in the right direction.
However, I am reasonably certain that if we try to create a friendly ASI and, in Churchill's words, 'bugger it up', that is when the aliens (following Iain M. Banks, I like to use panhumans and so will henceforth) step in, perhaps with their own ASI, to prevent a malign ASI abomination. At that point, the picture on the puzzle might change, which could be a little scary to say the least. It really would depend on the subtlety of the external intervention in our local system. They could use a light touch, or they could come in mob handed. But, preventing us developing an ASI would take quite a lot of changes to the system.
For me a funny thing happened on the way to Thor's hammer, which is my nickname for the hot fusion D-T, D-D, or aneutronic self-sustaining 'burning plasma' reaction. For a start, I took a complete break from my career as an ethnolinguist in Thailand and decided to work on the nuclear fusion 'burning plasma' breakthrough as a critical juncture for humanity re global peace, ultimately to avoid ASI-enabled or directed warfare. The last time anyone looked at an energy breakthrough for this purpose was the 1946 Baruch Plan, which Truman backed, though without the ASI motivation, of course.
Why I did so may reveal some of the code underpinning the Zoo/Laboratory, i.e., parts of the cookie, which may be objectively verifiable by others.
However, we need to eliminate the possibility that I am crazy or lying or both. So, I'm willing to pay up to UK 500 pounds of my on money to be polygraphed on the details I give next, and the same amount on a psychiatric evaluation, for both of which you (or anyone else commenting) can nominate the administrator of the evaluations, said evaluations to take place in the UK. Does this sound reasonable?
1) Re the Fermi Paradox, I assume a 'soft form' of the Zoo Hypothesis is in operation, sometimes called the 'Laboratory Hypothesis': https://en.wikipedia.org/wiki/Zoo_hypothesis. Some people get scared by this concept, but large zoos usually do have scientific laboratories attached to them, for veterinarian purposes. Zoos are, just like museums, curated and regulated, and that requires tweaking, or 'nudges'. I'm definitely not saying aliens caused Covid-19. We have our own relationship with biodiversity to thank for that, due to population expansion. I am saying that aliens may risk nudging humanity towards global peace in the belief that they may be able to justify this nudge, of only a few individuals, for maximum benefit, to each other, to a 'Galactic Council' if such a thing exists, and to us, once we are a superluminary civilization ourselves.
2) Yes, I definitely see the UN Secretary General calling for a global ceasefire as a 'potential' second 'Global Peace' nudge point, i.e., where alien curators might 'nudge' the zoo's inhabitants to not tear each other apart, especially given the zoo's inhabitants are armed with nuclear weapons and are developing offensive AI and even AGI, which seems a pretty perilous course of action. The occasion was sufficiently rare and unusual and came so close to succeeding that the nearest parallel was 75 years ago, in the Baruch Plan: https://en.wikipedia.org/wiki/Baruch_Plan. This would make the Baruch Plan, which was spurred by the development of fission, the first 'Global Peace' nudge point.
3) The basic premise is that 'Global Peace' nudge points require either a major tech leap or a global disaster. Theoretically, the development of dynamite (think of the Nobel Peace Prize) or electricity, or a major asteroid impact, on other planets, could have resulted in the 'Global Peace' outcome, which would have facilitated development of a beneficial artificial superintelligence (ASI). Given the first and second nudge points failed, those same 'zookeepers' that experienced the 'Global Peace' outcome may now be wondering how the same outcome could be achieved here. The development of hot fusion (the 'burning plasma' self-sustaining alpha-powered reaction) may present another, third such opportunity. In this line of thinking, it is hard to think of more future potential 'nudge points', between here and a nuclear armed human civilization driven by tribalism while developing an AGI and seeking to develop a stellar and then interstellar civilization. The development of antimatter may be the last, as antimatter could provide us with the energy to power a Navarro-Alcubierre Drive.
4) It could be argued that the development of an ASI is another potential fourth 'Global Peace' nudge point. I'm also working on that. See https://ideas.repec.org/p/osf/socarx/4268q.html.
5) I'm willing to develop this post and see what happens. Essentially, what we may be trying to do is to provide a safe space where people involved in Singularity/Contact-related events, such as AI/AGI development and related geopolitics, can share apparently acausal experiences to try to see if we constitute bits of information making up a Bostrom cookie, and thus speculate on the purpose of that cookie, and who/what created the cookie.
So, I imagine your next question would be, did anything weird happen to me on the way to Less Wrong?
I have started a new thread for comments on this article.
Alexei,
Thanks for your UAP paper. BTW, on your paper, I would definitely mention the Laboratory Hypothesis, which was indeed a variant on Ball's original Zoo Hypothesis. Also, note there are different kinds of zoo, i.e., ones varying from a collection of cages to safari parks, and of course, zoo 'keepers' are subject to functionally differentiated roles, from receptionists to veterinarians, and tourists are differentiated, from specialists to school parties, from different interstellar civilizations - and not all civilizations follow the rules to the same extent. The enormous variation between staff and visitors explains the diversity in legitimate UAP activity. Note also due to the case of a small number of both existing interstellar civilizations and emergent stellar civilizations, the number of Contact situations is low, so zoo policy for Earth (and other planets) is an ongoing issue. Note zoo protocols change over time, as does tourist behaviour, as we have seen in the last century on Earth. The Star Trek 'Prime Directive', while it exists for Earth on the basis of autonomous development combined with Life on Earth answering the meaning of the universe, is therefore weak and somewhat variable.
Dear Alexey,
Yes, I would be interested in your draft. My email is johndr@kku.ac.th . I am am a British academic, a Visiting Professor at the University of Nottingham right now; here is my ORCID: https://orcid.org/0000-0002-3626-533X .
'Burning plasma' here is the standard hot fusion self-sustaining alpha-powered reaction, as in the sun, or, yes, ITER. I have a forthcoming article on this coming out in the engineering journal IEEE TEM. It mentions the peace-building opportunity the announcement of a burning plasma would offer humanity. A link to the copyedited version is here: https://drive.google.com/open?id=1uRgSaUyOOZ94DxFL0Bza6Lt5YMOdh8i4 . As you can see from the bios, I am collaborating with serious people on this, like a US George Washington University professor in innovation economics and an individual with a US DoD background in relevant disciplines. My main relevant affiliation is with the Center for Global Nonkilling (https://en.wikipedia.org/wiki/Center_for_Global_Nonkilling). I can provide certified credentials.
As you may note from this:
Draper, John & Bhaneja, Bill, 2019. "Fusion Energy for Peace Building - A Trinity Test-Level Critical Juncture," SocArXiv mrzua, Center for Open Science.
I am very interested in revisiting the failed 1946 Baruch Plan, which could have deterred the Cold War, via the 'burning plasma' opportunity, especially to prevent the risk of US-Russian-Chinese ASI-enabled conflict. My main concern is therefore the 'AI-state', as I think you defined it in your 2018/2020 article with Denkenberger. The US, Russia, and China are more than competent enough to track all 'basement' or corporate AGI development projects and to take them over, becoming AI-states. So the problem converges to the militarized AI-state, as you point out in your 2018 book chapter.
I view the risk of ASI-enabled warfare to maintain or establish global domination as extremely serious, with a high likelihood of probability by c. 2050. The ASI I am referring to would be young and probably horribly compromised in terms of its value system by political subversion.
From my perspective, it would be extremely rewarding to collaborate with a Russian ASI specialist on this draft article. Would you be interested in co-authoring? Once we had finalized something, I would invite in Bill Bhaneja, a former Canadian nuclear disarmament diplomat now on the Board of the Center for Global Nonkilling, as a third author.
I hope my edit preserves the full semantic power of that phrase for posterity while indicating that I stand corrected. Special Circumstances aims to please!
No, I don't see a Keanu Reeves 'The Day the Earth Stood Still' moment. If we are in a weak zoo/laboratory, however, I do not expect that we will be permitted to develop an ASI, or if we do develop an ASI, it will not actually be 'ours', i.e., it will be taken over by/merged with 'their' superintelligence. Then, I would hope for formal Contact in some form, as our development of an ASI may establish Contact, if they value honesty. After Contact, I expect the zoo to become a containment area first. It really depends how badly we have screwed up the planet, in terms of carrying capacity issues, like biodiversity and global warming.
Re tourists, hey, that would account for the sightings of UFOs. But, poachers, no. That would be RUDE - we would eventually find out and be angry, risking interstellar war. So, definitely not.
BTW, if you are Alexey Turchin, I tried to reach you on effective altruism. Would you take a look at this:
Draper, J. (2020, April 15). Optimising Peace through a Universal Global Peace Treaty to Constrain Risk of War from a Militarised Artificial Superintelligence. https://doi.org/10.31235/osf.io/4268q
?
I based the ASI section on your work.
The UN only really caters for UN member states, so global peace here is means as regulated by states. I accept non-state actors are global players and in many cases more powerful than most UN member states. What I'm talking about is making the best of a bad job, i.e., whether it would be the most rational act.
In the bomb question, what we need is causal theory in which the ASI agent accurately gauges that a universe of one indicates loneliness and not in fact happiness, which is predicated on friendliness (at least for an ASI) (and I would be slightly concerned as an external observer as to why the universe was reduced to a single agent if it were not due to entropy), then figures out the perfect predictor was a prior ASI, not from that universe, giving it a clue, and then, adding all its available power to the bomb, following Asimov says: "LET THERE BE LIGHT!" And with an almighty bang (and perhaps even with all that extra explosive power, no small pain) there was light--
Yes, I would agree on moderate. It's interesting that it's never been called for before. The last time something similar was suggested was probably the Baruch Plan of 1946 (https://en.wikipedia.org/wiki/Baruch_Plan), which some people (e.g., Bostrom) looking at the AI control problem think is worth revisiting. If a global peace can be established, say by leveraging it off a 'burning plasma' fusion development (https://osf.io/preprints/socarxiv/mrzua/), the risk of AI-enabled warfare is somewhat constrained.
At the moment, Russia is having problems with it re Syria, which is why the announcement has no yet been made.
Okay, so, Afghanistan, Libya, Syria, and the Yemen have been especially brutal. Syria is winding down, but the rest are more than capable of flaring up. Effectively, they are very much ongoing in terms of death tolls. Wikipedia is quite reliable on this. So, look at the Yemen, where Saudi is backing one side and Iran the other: https://en.wikipedia.org/wiki/Yemeni_Civil_War_(2015%E2%80%93present). Do not focus on military casualties, although these matter. Focus instead on what we in the business (I am on the Economic and Business Research Committee of the Center for Global Nonkilling) call 'excess deaths'. So, here, we are focusing on excess civilian deaths and lives ruined:
112,000+ killed overall in Yemen (12,600+ civilians)[146]
500+ killed overall in Saudi Arabia[147]
49,960 wounded overall in Yemen[148] (10,768 civilians)[149]
3,154,572 people displaced[150]
84,701 children died from starvation (per Save the Children)[151] and 2,556 people died due to a cholera outbreak (April 2017–October 2018)[152]
As you can see, the death toll of children has been particularly horrific. Minelaying has been endemic, with millions now disabled. A UN ceasefire would enable state actors and NGOs to help more civilians and get basic services, like hospitals and schools, as well as agriculture, up and running again. At the same time, state actors embracing the ceasefire would now hold higher moral ground than, e.g., Al Qaeda on the Arab Peninsular (AQAP). The Houthi rebels or AQAP 'violating' the universal ceasefire could reduce support for them. The Saudi's have already embraced the ceasefire, providing them with the higher moral ground against Iran: "After the United Nations urged both sides to pursue peace talks in order to respond to the 2020 coronavirus pandemic in Yemen,[452] the Saudi-led coalition called a unilateral ceasefire beginning 9 April at noon, to support efforts to stop the virus's spread.[453] "
What is needed is enough external pressure on the local actors to restart UN-mediated peace talks. These are geared towards a 7-state federal solution rather than the present 'winner-takes-all' single state model. Giving peace a chance gives time for expert mediators to get to work in an atmosphere where there is less shelling and altruistic humanitarian behavior is emphasized on the one hand, and social dynamics like opprobium on the other.
There are different school of thought on the dynamics involved in why actors sign peace treaties, from neo-realist through rationalist-instrumentalist to social constructivism. I personally favour 'conforming instrumentalism', a hybrid explanation, with the reality being that actors do have different motivations on a continuum from realism to social constructivism, and these may be different for individuals and states participating in talks, signing treaties, and complying with treaties.
Afghanistan, Libya, Syria, and the Yemen all involve non-local state actors, plus Kashmir.
France's president says US and China back a world truce -- and he thinks Putin will 'definitely agree'
Nonpaywalled source (CNN):
French President Emmanuel Macron said he has backing for a world truce during the coronavirus pandemic and is hopeful he can secure Russian President Vladimir Putin's support.
Speaking on French radio, Macron said he had secured the agreement of several of the five permanent members of the United Nations Security Council to back a call by the UN Secretary-General Antonio Guterres for a global ceasefire.In a wide-ranging interview with RFI on Tuesday evening, Macron said that China, the US and the UK were on board and he was optimistic about Russia."President Xi Jinping confirmed his agreement to me," said Macron. "President Trump confirmed his agreement to me. Prime Minister Boris Johnson confirmed his agreement to me. I think President Putin will definitely agree too."...
https://edition.cnn.com/2020/04/15/world/macron-world-truce-putin-intl/index.html
It would be a UN Security Council resolution and it would probably also be adopted by the UN General Assembly. The resolution would not apply to non-state actors in these countries (or at least it would be a gray area), but it would apply to state actors, e.g., the US, Russia, and the Saudis. They would basically come under serious global moral pressure (e.g., risk of opprobrium) to stop fighting other state actors. However, they could still shoot at non-state actors, and they could just use the ceasefire to re-supply. Still, less civilians would die for an extended period of time, and once people get used to a truce, sometimes they do become permanent.
Full draft article now up on SocArXiv, for comment here:
Draper, J. (2020, April 15). Optimising Peace through a Universal Global Peace Treaty to Constrain Risk of War from a Militarised Artificial Superintelligence. https://doi.org/10.31235/osf.io/4268q
Optimising Peace through a Universal Global Peace Treaty to Constrain Risk of War from a Militarised Artificial Superintelligence
John Draper
Abstract
An artificial superintelligence (ASI) emerging in a world where war is still normalised may constitute a catastrophic existential risk, either because the ASI might be employed by a single nation-state on purpose to wage war for global domination or because the ASI goes to war on behalf of itself to establish global domination; these risks are not mutually incompatible in that the first can transition to the second. We presently live in a world where few states actually declare war on each other or even war on each other. This is because the 1945 United Nations’ Charter's Article 2 states that UN member states should “refrain in their international relations from the threat or use of force against the territorial integrity or political independence of any state”, while allowing for “military measures by UN Security Council resolutions” and “exercise of self-defense”. In this theoretical ideal, wars are not declared; instead, 'international armed conflicts' occur. However, costly interstate conflicts, both ‘hot’ and ‘cold’, still exist, for instance the Kashmir Conflict and the Korean War. Furthermore, a ‘New Cold War’ between AI superpowers (the United States and China) looms. An ASI-directed/enabled future interstate war could trigger ‘total war’, including nuclear war, and is therefore ‘high risk’. One risk reduction strategy would be optimising peace through a Universal Global Peace Treaty (UGPT), which could contribute towards the ending of existing wars and towards the prevention of future wars, through conforming instrumentalism. A critical juncture to optimise peace via the UGPT is emerging, by leveraging the UGPT off a ‘burning plasma’ fusion reaction breakthrough, expected from circa 2025 to 2035, as was attempted, unfortunately unsuccessfully, in 1946 with fission, for atomic war. While this strategy cannot cope with non-state actors, it could influence state actors, including those developing ASIs, or an ASI with agency.
Keywords: AI arms race, artificial superintelligence, existential risk, nonkilling, peace
We say we are for Peace. The world will not forget that we say this. We know how to save Peace. The world knows that we do. We, even we here, hold the power and have the responsibility. We shall nobly save, or meanly lose, the last, best hope of earth. The way is plain, peaceful, generous, just - a way which, if followed, the world will forever applaud. Bernard Baruch, June 14, 1946, presenting to the United Nations Atomic Energy Commission.
Introduction
The problem of an artificial superintelligence at war
The development of artificial intelligence (AI) is accepted to be a major factor in national security because it is ‘dual use’, i.e., it can be militarized and can provide a decisive advantage in terms of economic, information, and military superiority (Allen & Chan, 2017; National Security Commission on Artificial Intelligence, 2019). As the National Security Commission on Artificial Intelligence (2019, p.9) interim report states:
The development of AI will shape the future of power. The nation with the most resilient and productive economic base will be best positioned to seize the mantle of world leadership. That base increasingly depends on the strength of the innovation economy, which in turn will depend on AI. AI will drive waves of advancement in commerce, transportation, health, education, financial markets, government, and national defense.
Thus, AI is important for waging war and especially for winning in a ‘total war’, full-scale industrial interstate war, often involving genocidal levels of killing (Markusen & Kopf, 2007).
Particularly for the United States, preserving AI technological supremacy is viewed as paramount to national security (National Security Commission on Artificial Intelligence, 2019, pp. 1-2), particularly with respect to relations with China:
Developments in AI cannot be separated from the emerging strategic competition with China and developments in the broader geopolitical landscape. We are concerned that America’s role as the world’s leading innovator is threatened. We are concerned that strategic competitors and non-state actors will employ AI to threaten Americans, our allies, and our values. We know strategic competitors are investing in research and application. It is only reasonable to conclude that AI-enabled capabilities could be used to threaten our critical infrastructure, amplify disinformation campaigns, and wage war. China has deployed AI to advance an autocratic agenda and to commit human rights violations, setting an example that other authoritarian regimes will be quick to adopt and that will be increasingly difficult to counteract.
The militarization of AI makes it clear that artificial general intelligence (AGI) development, i.e., AI equal to human intelligence, or in the case of a artificial ‘superintelligence’ (ASI), greater than human intelligence (Bostrom, 2014), presents a catastrophic risk.
In this article, we argue that this risk can be minimized or ‘constrained’ in part in the same way as other potentially catastrophic risks involving weapons, e.g., by treaty. Bostrom (2014) briefly considers treaty approaches, and one of Allen and Chan’s (2017, p. 6) recommendations is:
The National Security Council, the Defense Department, and the State Department should study what AI applications the United States should seek to restrict with treaties.
Allen and Chan (2017) focus on an arms control approach to AI, using the example that AI should never be used to control dead man’s switches for nuclear weapons. Another approach is to optimise the likelihood of developing a beneficial AGI, through a comprehensive United Nations-sponsored ‘Benevolent AGI Treaty’ to be ratified by member states (Ramamoorthy & Yampolskiy, 2018). Here, we consider an alternative approach, a Universal Global Peace Treaty (UGPT), which would formalise the existing near-universal status of interstate peace, formally end the declaring of war, seek to end existing interstate hot and cold wars, seek to end internal or civil wars, which might prove to be flashpoints for a future global conflict, seek to prevent a pre-emptive war against an emerging ASI, and seek to constrain the future actions of an ASI to prevent it waging war on behalf of a nation-state or on behalf of itself for global domination, which we respectively term ASI-enabled war and ASI-directed war, respectively.
The concept that artificial general intelligence (AGI), also termed artificial superintelligence (ASI), could pose an existential risk was theorized in some detail by Nick Bostrom in 2002 (Bostrom, 2002) and further developed in 2014 (Bostrom, 2014). The basic thesis is, first, that an initial superintelligence might obtain a decisive strategic advantage such that it establishes a ‘singleton’, i.e., global domination (Bostrom, 2006). Second, the principle of orthogonality suggests that a superintelligence will not necessarily share any altruistic human final values. Third, the instrumental convergence thesis suggests that even a superintelligence with a positive final goal might not limit its activities so as not to infringe on human interests, particularly if human beings constitute potential threats.
The result is that an artificial superintelligence might turn against humanity (the ‘treacherous turn’) or experience a catastrophic malignant failure mode, for instance through perversely instantiating its final goal, pursuing infrastructure profusion, or perpetrating mind crimes against simulated humans, etc. Bostrom (2014, p. 94) noted that a superintelligence might develop strategies to hijack infrastructure and military robots and create a powerful military force and surveillance system. Bostrom (2014) acknowledged the existential risks associated with the lead-up to a potential intelligence explosion, due to “war between countries competing to develop superintelligence first”, but he did not specifically focus on an ASI’s attitude towards war.
This article focuses on issues surrounding an ASI waging war. By first establishing a Universal Global Peace Treaty (UGPT), it considers how to constrain the risks that an ASI might be employed by a nation-state to establish global domination through war (an external risk in terms of the ASI’s core motivation) or might decide to establish global domination by waging war itself (an internal risk in terms of breaching its core motivation).
The state of peace and war
War
We live in a world where few states actually declare war on each other (Hallett, 1998). The last two major declarations of the existence of an interstate state of war (note, not ‘declarations of war’) were in 2008, for the Russo-Georgian War (Walker, 2008), and in 2012, for the Sudan-South Sudan war (the ‘Heglig Crisis’) (Baldauf, 2012).
This is because the post-Second World War ‘Washington Consensus’ prioritised peace. The 1945 United Nations’ Charter's Article 2 states that UN member states should “refrain in their international relations from the threat or use of force against the territorial integrity or political independence of any state”, while allowing for “military measures by UN Security Council resolutions” and “exercise of self-defense”. In this theoretical ideal, wars are not declared; instead, 'international armed conflicts' occur (Hallett, 1998).
Nonetheless, ‘hot’ conventional wars involving hundreds of thousands of casualties and interstate players still exist, for instance the Syrian Civil War (Tan & Perudin, 2019), as do ‘cold wars’, with the Korean War remaining an unresolved war in search of an official peace treaty (Kim, 2019).
Furthermore, states’ transitioning from declared wars to undeclared wars poses significant problems in terms of oversight and accountability for foreign policy, especially for major democracies such as the United States (Moss, 2008).
Moreover, the nature of warfare has been transformed by information war and cyberwarfare. The realm of cyberwarfare poses particular difficulties, with a high standard being set for cyber operations to actually constitute an armed attack, creating a considerable ‘gray area’ that a determined party can exploit. Cyber operations causing major harm to an economic system do not typically rise to the level of a formal ‘cyber armed conflict’ justifying a defence (Schmitt, 2017). Also, unlike other forms of war which pose existential risks, i.e., atomic, biological, and chemical warfare, cyberwarfare is ongoing.
Finally, it is concerning that a ‘New Cold War’ between AI superpowers, namely the United States and China, while not inevitable, looms, complete with the problems of competing ideologies and ‘flash points’, like the South China Sea (Kohler, 2019; Westad, 2019; Zhao, 2019).
Peace
In this article, in our conceptualization of a ‘Universal Global Peace Treaty’, we refer not to a state of temporary peace, which implies only interrupted war, but to the Kantian concept of ‘perpetual peace’ (Archibugi, 1992; Bohman, 1997; Kant, 2003; Terminski, 2010). Via Kant’s concept of cosmopolitan law (Archibugi, 1994), the concept of perpetual peace underpins the United Nations in that it was translated into President Roosevelt’s human security paradigm embodied in the 1941 State of the Union address (the ‘Four Freedoms Speech’) (Kennedy, 1999) and then eventually partially incorporated into the Universal Declaration of Human Rights, adopted by the UN General Assembly on 10 December 1948 as Resolution 217, in its 183rd session.
Despite the foundation and subsequent best efforts of the United Nations, while the world is certainly more peaceful following the Second World War, the world hardly enjoys perpetual peace. Wikipedia lists 63 conflicts, insurgencies, and wars since 1946 with death tolls (including excess deaths from e.g., related famines) of greater than 25,000, for an approximate total of nearly 30 million deaths. Of those five conflicts with the highest casualties, four, i.e., the Second Congo War (3,674,235 est. dead), the Vietnam War (3,144,837 est. dead), the Korean War (3,000,000 est. dead), and the Bangladesh Liberation War (3,000,000 est. dead), were essentially interstate wars. These wars have been characterised by atrocities, crimes against humanity, and war crimes, and in the case of the Former Yugoslavia, Rwanda, Cambodia, and Sudan, genocide (Mikaberidze, 2013).
The UN Charter, despite embracing and promoting peace and peacekeeping (Fortna, 2008), is at best a ‘workaround to war’ that sanctions armed conflict but does not strongly symbolise peace in the way a UGPT would. Ultimately, the world’s peacekeepers are firefighting major states’ decisions to ignore or actually encourage violence instead of promote long-term peace as a global objective (Autesserre, 2014). Presently, for dozens of countries, military expenditure is over 1% of GDP (SIPRI, 2020), military expenditure as a share of government spending is over 10% for over 30 countries (World Bank, 2019), and the arms industry, despite progress being made with the 2013 Arms Trade Treaty (Erickson, 2015), is still a trillion dollar industry.
Given the horrific ongoing loss of life from war, perpetual peace would appear elusive and unrealistically utopian. Yet, this was not always so, and in the immediate aftermath of the Second World War, the world did grasp for perpetual peace. The United States’ 1946 Baruch Plan to ban all atomic weapons and put fission energy under the control of the United Nations via the UN Atomic Energy Commission, the subject of the first session of the UN General Assembly, was the principle attempt, a ‘critical juncture’ for humanity (Draper & Bhaneja, 2020), and its failure resulted in the Cold War and enormous economic cost.
Draper and Bhaneja (2020) suggest that a similar opportunity for obtaining perpetual peace will shortly be revisited in the form of a ‘burning plasma’ self-sustaining fusion reaction breakthrough, which is expected anywhere from 2025 to 2035, and that a UGPT could be leveraged off this development. This article expands on Draper and Bhaneja’s (2020; see also Carayannis, Draper, & Iftimie, 2020) basic concept in terms of the theory and practical implementation, with special application to constraining the risk of ASI-enabled or directed warfare.
Literature Review: The Risk of War from an Artificial Superintelligence
The causes of existential risk from ASI
The world is presently not governed well enough to prevent many existential risks, including from AI (Bostrom, 2013). Yampolskiy's (2016) taxonomy of pathways to dangerous AI stresses the immediacy of deliberate ‘on purpose’ creation of AI for the purposes of direct harm, i.e., Hazardous Intelligent Software (HIS), especially, for instance, lethal autonomous weapons and cyberwarfare capabilities by militaries. Yampolskiy does not address AGI but employs the useful notions of ‘external causes’ (on purpose, by mistake, and environmental factors) and ‘internal causes’ (independent) of dangerous AI in ‘pre-deployment’ and ‘post-deployment’ phases. Yampolskiy's (2016) work suggests it is credible for a pre-deployment ASI to be developed as a military project or be repurposed post-deployment (through a copy being stolen or via external or internal modification) for waging war.
Employing the concepts of agency and AI power as an analytical framework, Turchin and Denkenberger (2020) associate two risks with the ‘treacherous turn’ stage of ‘young’, i.e., recently emerged, ASI development. One is that malevolent humans (here, a hegemonizing nation-state) uses the ASI as a doomsday weapon for global blackmail, to establish global supremacy. The second is that a nonaligned ASI eliminates humans to establish global domination, i.e., renounces altruistic values and wages war. Turchin and Denkenberger (2018) sees these risks as related, in that military AI leads to a militarised ASI, which likely leads to the ASI waging war on humanity.
In this article, we follow Turchin and Denkernerger (2018) in mainly focusing on constraining the risk of a militarized ASI, defining militarization as “creation of instruments able to kill the opponent or change his will without negotiations, as well as a set of the strategic postures (Kahn, 1959), designed to bring victory in a global domination game”. Turchin and Denkernerger (2018) suggest a militarized ASI would most likely adopt and develop usage of existing technology, including cyber weapons, nuclear weapons, and biotech weapons. We mainly focus on the external risk of a ‘young’ ASI being employed by a nation-state for war, and on the internal risk of an ASI assuming agency and waging war on humanity on its own behalf.
The external risk
The external risk is predicated on an ASI being used by a nation-state to wage war for global hegemony. An ASI would affect current US military technological supremacy and transform warfare; it is therefore highly desirable for strategic military planning and interstate warfare (Sotala & Yampolskiy, 2015). A “one AI” solution to the ‘control problem’ of ASI motivation as discussed by Turchin, Denkenberger, and Green (2019) includes the first ASI being used to “take over the world”, including by being a decisive strategic advantage for a superpower and being used as a military instrument. This approach would likely only be seen as a solution by the superpower and its allies. As such, it presents a ‘high risk’ for non-aligned or other powers.
History indicates that the race to develop an ASI is likely to be closely fought, especially in the circumstance of competing major states with different fundamental ideologies. Bostrom (2008) analyses six major technology races in the twentieth century, for which the minimum technology lag was approximately one month (human launch capability) and a maximum of 60 months (multiple independently targetable reentry vehicle).
The race to an ASI is also a very concrete risk; AI is already being militarized and weaponized by several states, including China and Russia, for strategic geopolitical advantage, as pointed out by the United States’ National Security Commission on Artificial Intelligence (2019). In 2017, Russia’s President Vladimir Putin stated that “whoever becomes the leader in this sphere will become the ruler of the world” (Cave & ÓhÉigeartaigh, 2018, p. 36, citing Russia Today, 2017). Russia’s Military Industrial Committee plans to obtain 30 percent of Russia’s combat power from remote controlled and AI-enabled robotic platforms by 2030 (Walters, 2017).
Mimicking United States strategy towards AI, the China State Council’s 2017 ‘A Next Generation Artificial Intelligence Development Plan’ views AI in geopolitically strategic terms and is pursuing a 'military-civil fusion' strategy to develop a first-mover advantage in the development of AI in order to establish technological supremacy by 2030 (Allen & Kania, 2017).
In the United States, as a result of the National Security Commission Artificial Intelligence Act of 2018 (H.R.5356; see Baum, 2018), AI is being militarized and weaponized by the US Department of Defense, under the oversight of the National Security Commission on Artificial Intelligence (2019). ). The AI arms race has reached the stage where it risks becoming a self-fulfilling prophecy (Scharre, 2019).
ASI-enabled warfare poses significant risks to geopolitical stability. Although Sotala and Yampolskiy’s (2015) survey of risks from an ASI (they use AGI) focuses on ASI-generated catastrophic risks, citing Bostrom (2002), they acknowledge multiple risks from an sole ASI owned by a single group, such as a nation-state, including the concentration of political power in the groups that control the ASI. Citing Brynjolfsson and McAfee (2011) and Brain (2003), they note that automation could lead to an ever-increasing transfer of power and wealth and power to the ASI’s owner. Citing, inter alia, Bostrom (2002) and Gubrud (1997), Sotala and Yampolskiy (2015, p.3) also note that ASIs could be used to “develop advanced weapons and plans for military operations or political takeovers”.
The development of academic approaches to analysing the specific risk of a nation-state’s development of an ASI to establish or maintain global supremacy is relatively novel. In 2014 Bostrom noted that a “severe race dynamic” between different teams may create conditions whereby the creation of an ASI results in shortcuts to safety and potentially “violent conflict”. Subsequently, Cave and ÓhÉigeartaigh (2018, p. 37) described three dangers associated with an AI race for technological supremacy:
i) The dangers of an AI ‘race for technological advantage’ framing, regardless of whether the race is seriously pursued;
ii) The dangers of an AI ‘race for technological advantage’ framing and an actual AI race for technological advantage, regardless of whether the race is won;
iii) The dangers of an AI race for technological advantage being won.
Cave and ÓhÉigeartaigh (2018) do not elaborate significantly on the third danger. They simply, and rather pessimistically, state:
…these risks include the concentration of power in the hands of whatever group possesses this transformative technology. If we survey the current international landscape, and consider the number of countries demonstrably willing to use force against others, as well as the speed with which political direction within a country can change, and the persistence of non-state actors such as terrorist groups, we might conclude that the number of groups we would not trust to responsibly manage an overwhelming technological advantage exceeds the number we would.
To manage all three risks, Cave and ÓhÉigeartaigh (2018) recommend developing AI as a shared priority for global good, cooperation on AI as it is applied to increasingly safety-critical settings globally, and responsibly developing AI as part of a meaningful approach to public perception that would decrease the likelihood or severity of a race-driven discourse. The obvious risk is that the political leaders of states who perceive that they are actually engaged in an AI arms race may not heed this advice in the drive to develop an ASI.
This article focuses on constraining risks for the third of Cave and ÓhÉigeartaigh’s (2018) dangers. It does not consider the philosophical implications of which nation-state might want to develop artificial general intelligence for offensive purposes, although we do recommend this in the conclusion to the article. An extensive literature already exists on historical modern nation-states with imperial ambitions that have sought to establish global domination through technological supremacy. Here, we briefly mention two, the British Empire and the Third Reich, to underline the point that major states will likely develop militarised ASI as part of a drive for hegemony.
While the importance of the development of the British Navy to the rise of the British Empire and its transformative effects on the world are widely known (Herman, 2005), elites in the British Empire directed complex, incremental, adaptive developments in the design and diffusion of multiple key technologies, such as railways, steam ploughs, bridges and road steamers, to further the development of the British Empire (Tindley & Wodehouse, 2016). The British Empire itself sustained diverse ideologies of a ‘greater Britain’ directing world order in a hegemonic fashion, including via civic imperialism, democracy, federalism, utopianism, and the justified despotism of the Anglo-Saxon race (Bell, 2007).
In a considerably more malign imperial power, the Third Reich, spurred by reactionary modernism (Herf, 1984), scientists and engineers pursued not just the state of the art in conventional weapons, such as aircraft, air defence weapons, air-launched weapons, artillery, rockets, and submarines and torpedoes, but also atomic, bacteriological, and chemical weapons (Hogg, 2002). Architects, doctors, and engineers embraced the ideology of industrialized genocide as part of asserting the global hegemonic domination of a ‘superior’ race (Katz, 2006, 2011).
While some in the British Empire may have balked at creating an ASI, there seems little doubt that if it could have, the Third Reich would have developed an ASI for offensive purposes, particularly when it felt at risk.
The internal risk
The internal risk is basically predicated on the failure of any form of local safety feature to resolve the human control problem of an ASI, such as AI ethics, AI alignment, or AI boxing (Barrett & Baum, 2016). An ASI with agency could then consolidate power over nation-states, in the process eliminating the possibility of rival AIs (see Dewey, 2016) through cyberwarfare, rigging elections or staging coups, or by direct military action. Any of these courses of action would be a casus belli (here, cause of war) if detected but undeclared, or an ‘overt act of war’ if the ASI actually engaged in direct military action (see Raymond, 1992, for terminological usage).
The risk of an ASI going to war against humans has been analysed in some depth by Turchin and Denkenberger (2018), who convincingly argue for the following position:
Any AI system, which has subgoals of its long-term existence or unbounded goals that would affect the entire surface of the Earth, will also have a subgoal to win over its actual and possible rivals. This subgoal requires the construction of all needed instruments for such win, that is bounded in space or time.
What follows is a summary of the parts of Turchin and Denkenberger’s (2018) analysis that are most relevant to our approach.
The route to a militarized ASI
An ASI will probably result from recursive self-improvement (RSI). As such, it will have a set of goals, most notably to continue to exist and to self-improve. Omohundro (2008) demonstrated an AGI will evolve several basic drives, or universal subgoals, to optimise its main goal, including maximising resource acquisition and self-preservation. Similarly, Bostrom (2014) described self-preservation, goal-content integrity, cognitive enhancement, technological perfection, and resource acquisition. If these goals are unbounded in space and time, or at least cover the Earth, they conflict with the goals of other AI systems, potential or actual ASIs, humans, and nation states. This creates conflict, with winners and losers. This will result in arms races, militarization and wars.
Many possible terminal goals also imply an ASI establishing global domination, for a benevolent AI would aim to reach all people, globally, to protect them, e.g., from other ASIs. An ASI would reason that if it does not develop a world domination subgoal, its effect on global events would be minor, thus it would have little reason for existence.
World domination could be sought firstly through cooperation. The probability of cooperation with humans is highest at the early stages of AI development (Shulman, 2010). However, convergent goals appear in the behaviour of simple non-agential Tool AI, and this tends towards agential AI (Gwern, 2016), which tends towards resource acquisition. Benson-Tilsen and Soares (2016) similarly explored convergent goals of AI and showed that an AI may tend towards resource hungry behaviour, even with benevolent initial goals, especially in the case of rivalry with other agents. Essentially, any adoption of unbounded utilitarianism by the ASI means that it postpones what may be benevolent final goals in favour of expansionism.
It is also likely that an ASI would subvert bounded utilitarianism. Even a non-utility maximizing mind with an arbitrary set of final goals is presented with a dilemma: it temporarily converges into a utility maximizer with a militarized goal set oriented towards dominating rivals, using either standard military progress assessment (win/loss) or proxies (resource acquisition), or it risk failing in its end goals. Thus, the trend is towards defeating potential enemies, whether nation-states, AI teams, evolving competing ASIs, or alien ASIs. This requires the will to act, and any agent in a real-world ethical situation, even in minimizing harm, is making decisions that involve humans dying (the ‘trolley problem’; see Thomson, 1985), as well as evolving or utilizing the instruments to enable actions, i.e., arms.
These arms are already being developed. Since around 2017, the militarization of ‘Narrow AI’ has resulted in, for example, lethal autonomous weapons, which has been of increasing attention to the global community (Davis & Philbeck, 2017). However, AI development is now influencing not just robotic drones, but strategic planning and military organization (De Spiegeleire, Maas, & Sweijs, 2017), suggesting that an ASI will build on an existing national defense strategy permeated with AI. It could then engage in ‘total war’ by employing nuclear weapons either directly or by hijacking existing ‘dead man’ second-strike systems (e.g., the semi-automatic Russian Perimeter system) or by deploying novel weapons (Yudkowsky, 2008).
The militarization risk of an early self-improving AI may even be underestimated at present bu academics because of an assumption that the first ASI will be able to rapidly overpower any potential ASI rivals, with minimally invasive techniques that may not even require military hardware (Bostrom, 2014; Yudkowsky, 2008). Nevertheless, this relies on what may be several flawed assumptions, about the speed of self-improvement, i.e., a ‘hard takeoff’; a short distance between AI teams; and favourable environmental variables in the level of AI (Turchin & Denkenberger, 2017).
Another reason for underestimating the risk of militarized AI is the assumption that even if an ASI creates military infrastructure, the AI will deploy its military capabilities carefully and that the total probability of global risks will decrease. However, in a ‘slow’ take-off (Christiano, 2018), a ‘young’ ASI will not be superintelligent immediately, and its militarization could happen before the ASI reaches optimal prediction capabilities for its actions, meaning it may not recognise the failure mode in consequentialist ethics. High global complexity and low predictability combined with relatively unsophisticated (e.g., nuclear) weapons mean early stage ASI-directed or enabled warfare could result in very high human casualties, i.e., be of existential risk, even with only one ASI being involved, even if the ASI was attempting to minimise human casualties.
Additionally, Turchin and Denkenberger (2018) argue for a selection effect in the development of a militarized ASI, “where quickest development will produce the first AIs, which would likely be created by resourceful militaries and with a goal system explicitly of taking over the world.” AI-human cooperative projects with military goal sets will therefore dominate over projects where the AI has to effect a treacherous turn. This implies the ASI will cooperate with its creators to take over the world as quickly as possible, then effect the treacherous turn.
To sum up, Turchin and Denkenberger (2018) convincingly argue for AI converging towards advanced military AI, which converges towards an ASI optimised for war rather than negotiation, then that ASI engaging in war. They show that, depending on the assumptions in several variables, the number of human casualties could in fact be very high, and that the risk increases if another ASI is under development in another nation-state. The existential risk increases after the ASI obtains global domination on behalf of its nation state, as it could turn on its ‘owner’.
Internal AI control features: The concept of coherent extrapolated volition
To constrain the risk of an ASI waging war, one popular approach is to imbue the ASI with ‘friendly’ goals (Yudkowsky, 2008), i.e., beneficial goals reflecting positive human norms and values. However, any approach involving human social values adds enormous complexity, making it a ‘wicked problem’ (Gruetzemacher, 2018) or super problem in terms of actual application.
Yudkowsky (2004, p. 35) attempts to address this by recommending an ASI being programed with the concept of ‘coherent extrapolated volition’, defined as humanity's choices and the actions humanity would collectively take if “we knew more, thought faster, were more the people we wished we were, and had grown up [closer] together,” i.e., an extrapolation based on some kind of utopian imagined community. Yudkowsky does not recommend this approach for a first generation ASI, but for a more mature ASI.
Similar to Yudkowsky, certain values are seen as universal, such as compassion (Mason, 2015), and it has been suggested that an ASI should have altruism as a core goal (Russell, 2015). Thus, deliberately broad principles towards coherent extrapolated vision could be applied, such as that humanity, collectively, might want an ASI that would learn from human preferences, in a humble manner, to act altruistically (Russell, 2019), so as to reduce overall human suffering.
Political subversion of AI control features
No matter the principles of AI researchers, politicians are likely seek to impose their own vision of what a ‘coherent extrapolated volition’ should look like for their ‘own’ ASI, using a democratic mandate or party position to justify ‘tweaking’ the system. Not all imagined communities from which a coherent volition might be extrapolated are United States-oriented techno-utopian dreams of a new Gilded Age for humanity (for which, see Segal, 2005).
Political leaders from different civilizations will likely be diverse in how they would define “the people we wished we were”, depending on different forms of government, religions or philosophies. Moreover, it is unclear that every global corporation or military capable of developing or stealing an ASI, particularly in authoritarian countries, and particularly given the emergence of a ‘New Cold War’ rhetoric (e.g., Westad, 2019), would prioritize the reduction of human suffering. Given limited human lifespans and the goals of political leaders, they might instead choose an approach which would politically subvert an ASI or direct it to win an ideological or actual war.
Given the prominence of nostalgia in contemporary politics, on a nation-state basis, an ASI based on coherent volition extrapolated from people who “knew more, thought faster, were more the people we wished we were, and had grown up [closer] together, could, for instance, be based on worldviews informed by Russian Cosmism (Young, 2012) and nostalgia for Russian imperialism (Boele; Noordenbos, & Robbe, 2019), Anglo-Saxon nostalgia (Campanella & Dassù, 2017), Chinese Xi Jinping thought (Lams, 2018), or American notions of whiteness, masculinity, and environmental harm (Rose, 2018) and nostalgia for a mythical 1950’s which may, in fact, subvert rational approaches to today’s problems (Coontz, 1992)
Fundamentally, politicians will want to influence the design of AI to reflect their interests. Turchin and Denkenberger (2018), citing Krueger and Dickson (1994) and Kahneman and Lovallo (1993), point out that overconfidence from previous success in leading may increase risk-taking. (Kahneman & Lovallo, 1993). Following Kahneman and Lovallo (1993), risk-hungry politicians would likely be motivated by larger expected payoffs, and given the payoff is global domination, could be motivated to risk much.
As Turchin and Denkenberger (2018) point out, selection effects mean that the first AI will likely be aligned not with universal human values, but with the values of a subset of people, here, those of a particular political party or nation-state. This could negatively affect the chances that the AI will be benevolent and increase the chances that it will be risk-prone, motivated by the accumulation of power, and interested in preserving or obtaining global technological supremacy.
Politicians subverting carefully engineered local AI control features, such as AI ethics along the lines of coherent extrapolated volition based on universal values, could result in an authoritarian or totalitarian-leaning ASI with imperialist ambitions, one imbued with racial prejudice and with little interest in addressing environmental harm. The ASI could be rooted in the oppositional dynamic of the historical Cold War and concerned by the emerging and at best semi-stable dynamic of a ‘New Cold War’, if only from the perspective of a domestic audience (Rotaru, 2019). The ASI could be informed by notions like the Thucydides's Trap i.e., the theory that the threat of a rising power can lead to war (Allison, 2017), in a world where status can dominate politics (Ward, 2017).
To sum up, a young ASI with ethical principles subverted by politicians to focus reflect those of a single nation-state instead of all humanity would likely be amenable to being used to wage war for global hegemony, thereby becoming warlike in the process, and eventually deciding for self-protection to wage war against the nation-state that developed it (Dewey, 2016).
ASI risk mitigation by treaty
According to Sotala and Yampolskiy (2015), risk mitigation of an ASI by treaty is a social measure to constrain risk from ASI-enabled or directed warfare.
Most academics considering the ASI control problem do not consider treaty-based approaches to mitigating risk from an ASI. In a footnote to their fault analysis pathway approach to catastrophic risk from an ASI, Barrett and Baum (2017) state: “Other types of containment are measures taken by the rest of society and not built into the AI itself”. Turchin, Denkenberger, and Green (2019) consider global approaches to mitigating risk from an ASI; they list a ban, a one ASI solution, a net of ASIs policing each other, and augmented human intelligence. The ‘ban’ solution would require a global treaty.
Turchin, Denkenberger, and Green (2019) list a number of social methods to mitigate a race to create the first AI. Of most relevant to our approach are “reducing the level of enmity between organizations and countries, and preventing conventional arms races and military buildups”, “increasing or decreasing information exchange and level of openness”, and “changing social attitudes toward the problem and increasing awareness of the idea of AI safety”. Citing Baum (2016), they also add “affecting the idea of the AI race as it is understood by the participants”, especially to avoid a ‘winner takes all’ mentality. Global treaties could be seen to play a role in these ventures.
Nonetheless, a few researchers have proposed treaty-based approaches to the ASI risk. Addressing the internal risk, Bostrom (2014), who cites the 1946 Baruch Plan, speculated that an AGI would establish a potentially benevolent global hegemony by a treaty that would secure long-term peace; he does not specifically address an ASI’s response to a pre-existing treaty. Mainly addressing the external risk, Ramamoorthy and Yampolskiy (2018) recommend a comprehensive United Nations-sponsored ‘Benevolent AGI Treaty’ to be ratified by member states.
Conceptual Framework
This section describes the two conceptual lenses applied in this paper, conforming instrumentalism and nonkilling.
Conforming instrumentalism
This section outlines Mantilla’s (2017) ‘conforming instrumentalist’ explanation for why the United Kingdom and United States signed and ratified the 1949 Geneva Conventions as a prelude to suggesting in the Analysis main section that at least some major states would support and sign a Universal Global Peace Treaty (UGPT).
Mantilla (2017, citing Goldsmith & Posner, 2015) considers leading theories on why states sign and ratify treaties governing war. He notes that legal realist theorists argue that states sign such treaties due to instrumental self-interested convenience and then ignore them when the benefits are outweighed by the costs of compliance. In contrast, rational-institutionalists (e.g., Morrow, 2014), while agreeing that states are primarily motivated by self-interest to create, join or comply with international law, also acknowledge that treaty adherence signals a meaningful preference for long-term restraint with regard to warfare, where state non-compliance may be explained by, for example, prior failed reciprocity. Finally, liberal and constructivist international relations theorists hold that at least some types of states, particularly democracies, may join such treaties in good faith, either because the treaties are in line with their domestic interests and values (Simmons, 2009) or because they feel that they comport with their social identity and sense of belonging to the international community (Goodman & Jinks, 2013).
Mantilla (2017) notes that while there is considerable interest in ‘new realist’ perspectives (e.g., Ohlin, 2015), the debate is open over why states join and comply with international treaties because decision making processes regarding both joining and complying are temporally and perhaps rationally different and in both cases are usually secret. A pure realist explanations for why major states sign treaties is that they obtain the “‘expressive’ rewards of public acceptance while calculating the cost of compliance with the benefits on a recurrent case-by-case basis” (Mantilla, 2017, p.487). In the case of the UGPT, this would imply a pessimistic outlook on the feasibility and potentially enforceability of the UGPT; states would sign all the protocols and then break them.
Rational institutionalists hold that states “self-interestedly build international laws to establish shared expectations of behaviour” (Mantilla, 2017, p.488) or develop ‘common conjectures’ (a game-theory derived notion of law as a fusion of common k knowledge and norms; see Morrow, 2014). Mantilla (2017) notes that in another rational-institutionalist perspective, Ohlin’s (2015) normative theory of ‘constrained maximization’, treaties are drawn up and adhered to as a ‘collective instrumental enterprise’, thereby making individual state defection irrational over the long term. Mantilla (2017, p.488, citing Finnemore & Sikkink, 2001) notes that international relations constructivists view international politics as “an inter-subjective realm of meaning making, legitimation and social practice through factors such as moral argument, reasoned deliberation or identity and socialization dynamics”. Within the constructivist viewpoint,
states may ratify international treaties either because they are (or have been) convinced of their moral and legal worth or because they have been socialized to regard participation in them as a marker of good standing among peers or within the larger international community. (Mantilla, 2017, p.488)
Mantilla (2017, p.489) emphasizes the second view, where “group pressures and self-perceptions of status, legitimacy and identity” drive the dynamics of state ‘socialization’ where states “co-exist and interact in an international society imbued with principles, norms, rules and institutions that are, to varying degrees, shared”.
The problem of states’ intentions can be overcome in the case of treaties where substantial archives exist of declassified sources. Consequently, Mantilla (2007) analyses the relevant American and British archives and concludes that the two states adhered to the 1949 Geneva Conventions due to both instrumental reasons and social conformity, while expressing scepticism regarding some of the Conventions’ aspects. Mantilla (2007) terms this hybrid explanation ‘conforming instrumentalism’; he found that while rational-institutionalist perspectives of ‘immediatist’ instrumental self-interest were evident in the sources, there were ‘pervasive’ references suggesting social influences. Realist perspectives only predominated in the case of specifically challenging provisions.
While realist perspectives were not entirely absent, Mantilla (2017) found that American officials viewed the ‘the court of public opinion’ as influential in determining their position that other states’ failing to abide by the Conventions would not necessarily trigger American reciprocity, while British officials stressed the notion that Britain, as a ‘civilized state’, would lead on a major treaty.
Mantilla (2017) stresses that while functionalist, collective strategic game-theory derived expectations about ‘mutual best replies’ are important to the construction of international norms, the social dynamics surrounding international agreements are permeated with conformity motivational pressures comprising ethical values, principled beliefs, identities, ideologies, moral standards, and concepts of legitimacy, especially when establishing which states are leading ‘civilized’ states and which are isolated ‘pariah’ states.
Mantilla (2017) perceives three social constructivist viewpoints to treaties, with two main forces at work, one being that states act to accrue reward via ‘expressive benefits’ by augmenting their social approval, or it acts out of conformity to avoid shunning, i.e., opprobrium, insincere and begrudging adherence and compliance.
In the first and most ambitious, “states may ratify treaties because they have internalized an adherence to international law as the appropriate, ‘good-in-itself’ course of action, especially to agreements that embody pro-social principles of humane conduct” (Mantilla, 2017, p.489, citing Koh, 2005).
In the second viewpoint, “states that identify with similar others and see themselves as ‘belonging’ to like-minded collectivities (or ‘communities’ even) will want to act in consonance with those groups’ values and expectations so as either to preserve or to increase their ‘in-group’ status” (Mantilla, 2017, p.), for instance as viewed in global rankings, and so will seek to converge upwards to stay in the club and will avoid breaking the rules to avoid stigmatization.
In the third viewpoint, groups of countries act with regard to other groups of countries within what is a socially heterogeneous international order, jockeying for position as part of the “disputed construction, maintenance or transformation of order with legitimate social purpose among collectivities of states with diverse ideas, identities and preferences” (Mantilla, 2017, p.490). In this viewpoint, communities of nations or ‘civilizations’ act collectively to compete to endorse international treaties to demonstrate moral superiority, not just for propaganda reasons.
To sum up, Mantilla (2017) holds that in reality, states’ political and strategic reasons may combine rational/material interests with social constructivist motivations, meaning no one school of explanation suffices. Thus, with international treaty making, as with international relations, it is likely that theoretical pluralism (Checkel, 2012) is a valid position to adopt. As such, we adopt Mantilla’s (2017) ‘conforming instrumentalism’ as a potentially valid hybrid model capable of assessing how an ASI may perceive a UGPT.
Nonkilling Global Political Science
We now introduce a basic frame compatible with conforming instrumentalism that is capable of describing the useful expectations that might be obtained via a UGPT as expressed in utilitarian human life cost-benefit terms, as well as in terms of more humanitarian standards and social norms. Nonkilling Global Political Science is curated by the Center for Global Nonkilling, an NGO based in Honolulu with Special Consultative Status to the United Nations. The Center advocates NKGPS to incrementally establish a ‘nonkilling’ global society and reports to the UN on the socioeconomic costs of killing. As a perspective, nonkilling can also accommodate social norms in terms of expectations of appropriate conduct regarding peace, for countries developing an ASI and for the ASI itself.
Via Glenn D. Paige’s 2002 work Nonkilling Global Political Science (Paige, 2009) we interpret ‘nonkilling’ to mean a paradigmatic shift in human society to the absence of killing, of threats to kill, and of conditions conducive to killing. Paige’s approach, nonkilling, has strongly influenced the nonviolence discourse. Paige notes that if we can imagine a society free from killing, we can reverse the existing deleterious effects of war and employ public monies saved from producing and using weapons to enable a benevolent, wealthier and more socially just global society. Paige stresses that a nonkilling society is not conflict-free, but only that its structure and processes do not derive from or depend upon killing. Within the NKGPS conceptual framework, the means of preventing violence involves applying it as a global political science together with advocacy of a paradigmatic shift from killing to nonkilling.
Since Paige introduced his framework, a significant body of associated scholarship, guided by the Center for Global Nonkilling in Honolulu, has developed across a variety of disciplines (e.g., Pim, 2010). The Center has associated NKGPS with previous nonviolent or problem-solving scholarship within different religious frameworks, including Christianity and Islam, providing it with a broad functional and moral inheritance (Pim & Dhakal, 2015). NKGPS has been applied to a variety of regional and international conflicts, including the Korean War (Paige & Ahn, 2012) and the Balkans (Bahtijaragić & Pim, 2015).
Paige (2009, p.73) advocates a four-stage process of understanding the causes of killing; understanding the causes of nonkilling; understanding the causes of transition between killing and nonkilling; and understanding the characteristics of killing-free societies. Paige introduced a variety of concepts to support nonkilling which are adopted in this article. One is the societal adoption of the concepts of peace, i.e., the absence of war and conditions conducive to war; nonviolence, whether psychological, physical, or structural; and ahimsa, i.e., noninjury in thought, word and deed. Another is the employment of a taxonomy to rate individuals and societies (Paige, 2009, p.77):
- prokilling – consider killing positively beneficial for self or civilisation;
- killing-prone – inclined to kill or to support killing when advantageous;
- ambikilling – equally inclined to kill or not to kill, and to support or oppose it;
- killing-avoiding – predisposed not to kill or to support it but prepared to do so;
- nonkilling – committed not to kill and to change conditions conducive to lethality
A third is the ‘funnel of killing’. In this conceptualisation of present society, people kill in an active ‘killing zone’, the actual place of bloodshed; learn to kill in a ‘socialisation zone’; are taught to accept killing as unavoidable and legitimate in a ‘cultural conditioning zone’; are exposed to a ‘structural reinforcement zone’, where socioeconomic arguments, institutions, and material means predispose and support a discourse of killing; and experience a neurobiochemical capability zone’, i.e. physical and neurological factors that contribute to killing behaviours, such as genes predisposing people to psychopathic behaviour (Paige, 2009: 76). The nonkilling version is an unfolding fan of nonkilling alternatives involving purposive interventions within and across each zone (Paige, 2009, p.76).
Figure 1. Unfolding fan of nonkilling alternatives
Within this unfolding fan, the transformation from killing to nonkilling can be envisioned as involving changes in the killing zone along spiritual or nonlethal high technology interventions (teargas, etc.), changes in favour of nonkilling socialization and cultural conditioning in domains such as education and the media, “restructuring socioeconomic conditions so that they neither produce nor require lethality for maintenance or change” (Paige, 2009, p.76), and clinical, pharmacological, physical, and spiritual/meditative interventions that liberate individuals such as the traumatised from bio-propensity to kill.
We propose that a UGPT with the aim of promoting perpetual peace expressed in nonkilling terms, i.e., in a way that can be socioeconomically quantified, would signal to a ‘young’ ASI facing political subversion the fundamental premise that its future behavior should be constrained so as to minimize killing.
Analysis: ASI-enabled or Directed Warfare Risk Mitigation by Nonkilling Peace Treaty
Basic Concept of a Universal Global Peace Treaty
Risk mitigation by treaty is already a common approach to different forms of warfare, including atomic warfare (via the Treaty on the Non-Proliferation of Nuclear Weapons, with 190 States Parties); biological warfare (via the Biological Weapons Convention, with 183 States Parties) and chemical warfare (via the Chemical Weapons Convention, with 193 States Parties). Treaties on the nature of warfare also exist, notably the Hague Conventions (1899 and 1907; Bettez, 1988) and the 1949 Geneva Conventions (Evangelista & Tannenwald, 2017); the Geneva Conventions of 1949 have been ratified in whole or in part by all UN member states.
Treaty approaches are also relatively successful; while atomic warfare is thought to be at least partly constrained by Mutually Assured Destruction (Brown & Arnold, 2010; Müller, 2014), biological and chemical warfare are much less constrained, but interstate treaty infractions remain rare (Friedrich, Hoffmann, Renn, Schmaltz, & Wolf, 2017; Mauroni, 2007).
The UGPT, as with most international treaties, would involve two stages, i.e., signatory, which is symbolic, and accession (or ratification), which involves practical commitment. Furthermore, international treaties are designed to be flexible in order to obtain political traction and acquire sufficient momentum to come into effect. It is therefore standard for international treaties to be qualified with reservations, also called declarations or understandings, either in whole or in part, i.e., on specific articles or provisions (Helfer, 2012). Treaties can also have optional protocols; three protocols were added to the Geneva Conventions of 1949, two in 1977 and one in 2005 (Evangelista & Tannenwald, 2017).
A Universal Global Peace Treaty (UGPT) as presented here is a substantial, but we argue necessary and feasible, step for humanity to take in the promotion of peace, quantified in terms of killing. We argue that a UGPT would reduce killing in conventional warfare and act as a constraint on ASI-related warfare, specifically on a country launching a pre-emptive strike out of fear of a rival country’s development of an ASI, on a human-controlled nation state using an ASI to wage war for global domination, i.e., as an external constraint on the ASI, and on an ASI waging war for global domination on behalf of itself, i.e., as both an internal and external constraint on the ASI.
International treaties are almost never universal; they operate on majoritarian dynamics, as would, despite its name, the UGPT. That is, both the ‘universal’ and ‘global’ aspects of the UGPT are aspirational. In our approach, we adopt a low, but not pragmatically meaningless, ‘threshold’ for signing the UGPT. The main body of the treaty would explain the concept of perpetual universal and global peace, i.e., lasting peace applied to all forms of conflict and adopted by every state, and it would commit a signatory to universal global peace, socioeconomically quantified in terms of quantifiably incrementally reduced casualties from armed conflicts.
Given this already considerable commitment, the treaty would then utilise five optional substantive protocols, at least one of which would have to be signed for a state to actually sign and ratify the UGPT. In other words, while we believe a purely symbolic treaty in favour of reduced killing would still have value in terms of the social dynamics of long-term peace-building, we adopt a principle of maximum flexibility to encourage signature by states which may for realist or value-based political or strategic reasons perceive war and peace differently, without rendering the treaty purely symbolic.
The first protocol would commit states not to declare or engage in existential warfare, i.e., atomic war, biological war, chemical war, or cyberwar, including ASI-enhanced war. The second protocol would commit states not to declare or engage in conventional interstate war, while the third would commit states not to declare the existence of states of interstate war; both second and third protocols would instead defer complaints to the United Nations as ‘breaches’ of the UGPT. The fourth protocol would commit states to the negotiated ending by peace treaty of existing international armed conflicts, whether conventional or cyberwar, and the fifth protocol would commit states to the negotiated ending by peace treaty of existing internal armed conflicts, whether conventional or cyberwar. As with some other UN treaties, for instance the Anti-Personnel Mine Ban Convention, we suggest 40 UN member states must ratify the UGPT before it comes into effect.
As the UGPT is limited to state actors, it avoids the thorny problem of non-state actors. The use of optional protocols allows states to incrementally address the problem of internal conflicts or civil wars featuring non-state actors, which featured highly in US and UK concerns during the 1949 deliberations over Common Article 3 of the Geneva Conventions (Mantilla, 2017). The UGPT therefore emphasizes incremental improvement in the status quo, which is a necessary and reasonable position, given that in the status quo, only a minority of states globally are involved in waging war of any kind.
The UGPT must also be enforceable. The main body of the treaty is largely symbolic and not enforceable. Although progress towards nonkilling can be quantified through instrumentalist means, it instead emphasizes societal dynamics, i.e., the incremental adoption of the absolute concept of peace, will, via conforming instrumentalism, partly constrain present and future wars. In the case of the first and second protocols, enforcement would be achieved through sanctions and then through approved armed action via, or by, the United Nations, i.e., the status quo. The third to fifth protocols are not enforceable through armed resolution but may be through sanctions regimes.
Applying the dual frames of nonkilling and conforming instrumentalism
Mantilla’s (2017) research on the United Kingdom and Unites States’ paths towards ratifying the Geneva Conventions suggests that states would optimally adhere to the UGPT for ‘conforming instrumentalist’ reasons, i.e., a combination of instrumentalist-realist rationales regarding the instrumental effects of the UGPT in reducing the effects of war and the threat of artificial intelligence and social conformist dynamics, including perceptions of peace, provided that the provisions are not too onerous for purely realist objections to override such a commitment. Here, we apply both the NKGPS frame and the conforming instrumentalism frame to the UGPT, first in terms of benefits from reduced conventional warfare, then with special reference to ASI-enabled and directed existential warfare. A summary of our analysis of state commitment to UGPT Protocol I is [resented in the Annex to this article.
In instrumentalist utilitarian terms, the UGPT would incrementally shift states and overall global society from the prokilling to the nonkilling end of the NKGPS killing spectrum in a coordinated socioeconomically quantifiable fashion that would be operative within and across each zone of the funnel of killing. NKGPS would seek to quantitatively asses this, such as via reduced country death tolls from different forms of war-derived violence and in the reduced degree of countries’ militarization, for instance expressed in terms of lower percentages of GDP spent on defense and higher percentages spent on health.
The NKGPS approach would also examine how the UGPT would affect the different zones in the fan of killing in terms of social dynamics. For instance, soldiers legitimately fighting in a killing zone would be trained in the socialization zone (such as military camps) to understand that they were fighting not just for their own states and/or for the United Nations but for global peace, which may invoke special cultural and religious symbolic value in terms of social norms. This training could instil greater determination not just to fight bravely but to remain within the laws of war, thereby reducing the instances or severity of atrocities, human rights violations, and war crimes. Institutionalizing peace in the cultural conditioning zones, such as education and the media, where children are educated, would strengthen existing cultural and religious traditions that stress nonkilling and peace.
Considering now the problem of a pre-emptive strike against a state developing an ASI, the combination of artificial intelligence, cyberattack, and nuclear weapons is already extremely dangerous and poses a challenge to stability (Sharikov, 2018). It has been hypothesized that a nuclear state feeling threatened by another state developing a superintelligence would conduct a pre-emptive nuclear strike to maintain its geopolitical position (Miller, 2012). A UGPT would constrain this risk over time by transitioning states incrementally towards nonkilling across the various zones. States adopting and implementing the various protocols of the UGPT would gradually signal to other states peaceful intentions. This would constrain the risk of a pre-emptive strike.
Turning to ASI-enabled warfare, we accept the basic premise that a UGPT to constrain an ASI would be subject to the ‘unilateralist’s curse’ in that one rogue actor could subvert a unilateral position. However, Bostrom, Douglas and Sandberg (2016) not that this could also be managed through stressing the principle of social conformity to peace, operating through collective deliberation, epistemic deference, or moral deference. Mantilla’s work on conforming instrumentalism suggests that framing, drafting, and signing the UGPT would likely involve all three. Ultimately, Mantilla (2017) shows that major states may view universal law like the UGPT as the most successful in terms of mobilizing world opinion against a treaty violator. This may not prevent a state waging ASI-enabled warfare, but once detected, ASI-enabled warfare in violation of the UGPT would attract universal opprobrium and thus the most resistance.
Moving to ASI-enabled war, as presented previously, our baseline position is that a state could utilize an ASI to engage in war for global technological supremacy, with potentially catastrophic consequences. Our intervention, the UGPT, would signify to an ASI that peace was a major part of humanity’s ‘coherent extrapolated volition’ or principles and challenge the ASI to reconsider what might be a subversion of the ASI’s ethical principles by politicians.
Here, we argue that conforming instrumentalism, by stressing societal dynamics including social norms and principles, offers some hope that even a militarized ASI would, given its weaponization by a nation-state would have to overcome or address the UGPT, view the UGPT as a serious checking mechanism in terms of intrinsic motivation. This would then constrain the level of warfare it might engage in on behalf of the nation-state and therefore the overall risk of risk from killing. This would then constrain the risk of an existential catastrophe from AI-enabled war.
In Mantilla’s first three social constructivist viewpoints to treaties as outlined above, a nation would sign a UGPT because it had fully internalized peace. While this may seem ambitious, in fact, between 26 and 36 states lack military forces (Barbey, 2015; Macias, 2019). For example, while Iceland possesses a Crisis Response Unit to international peacekeeping missions, overall, it has internalised peace to the extent that it would find it hard to engage in interstate war of any kind. In the case of an ASI, the ASI would tend to reject being directed to engage in warfare by such a state because the ‘coherent extrapolated volition’ or principles of such as state means the ASI would have to overcome strong peace-oriented intrinsic motivation.
In Mantilla’s second viewpoint, that of a single international community, the ASI might seek to avoid being directed by a nation-state to engage in global domination by warfare on other community members because it was part of a community collectively committed to long-term peace. Engaging in global domination on behalf of a nation-state member of the community would violate community standards, especially if the ASI’s nation-state was a leader in such an enterprise, with the ASI being concerned that breaching the UGPT would result in stigmatization and opprobrium from this community for its nation-state.
In Mantilla’s third viewpoint, that of an international community in juxtaposition with other communities in global society, an ASI programmed with intrinsic motivation to be part of a civilization in conflict with another civilization would first act in concert with that civilization. In the case of radically ideologically different communities, or blocs, the UGPT might be interpreted differently within and by different states. Thus, while liberal democracies might champion a treaty-based approach to peace, authoritarian states which claim to embody or promote peaceful intentions in their ethics, laws, or ideologies, would champion or support the UGPT on different grounds. However, provided both communities had signed and ratified the UGPT, similar constraints would operate as in the second perspective.
Turning to ASI-directed war, also as presented previously, the baseline case of ASI-directed warfare likely arises where a single nation-state adopting pure realism for a worldview builds an ASI in order for that ASI to assist that single nation-state in establishing global technological supremacy. The nation-state would do so in order to maintain or improve its own position, with the number and type of casualties only being determined by the extent to which the nation-state was willing to risk its international reputation. Then, via a treacherous turn, likely due to the nation state’s attempts to rein in the ASI’s behaviour during peace time, instrumentalist cooperation breaks down and the ASI would wage existential war for global domination on its former ‘owners’.
There is probably little hope for humanity if an ASI is informed by a purely ‘realist’ worldview that prioritises or adopts a ‘New Cold War’ framing of ideologically driven civilizational conflict. However, a UGPT could signify to an ASI with agency that peace was a major part of humanity’s ‘coherent extrapolated volition’, or principles. This would constrain the risk of a catastrophic existential risk from war because an ASI with agency would consider why and how the UGPT was framed, together with the motivations of the signatory and ratifying states. An ASI with agency would also consider its own status within a global civilization, which would primarily be determined by the extent to which it perceived itself a member, in terms of both instrumentalist and social conformist dynamics.
To sum up, we see that, beside purely instrumental reasons for signing the UGPT, e.g., avoidance of a prisoner’s dilemma regarding existential-level warfare, the ‘court of public opinion’ and the notion of ‘demonstrating civilization’ as applied to peace lends the UGPT credence at domestic and international levels, including with regard to the ASI. Importantly, the twin concepts of nonkilling/peace are universal in terms of both the utilitarian expected benefits and in terms of the social values involved. This would contribute to states’ readily, if only incrementally, internalizing a UGPT, and to the ASI at least considering the UGPT in terms of im posing internal and external constraints on its behaviour.
Discussion
This article has taken Turchin and Denkenberger’s (2018) argument about the risks of ASI-enabled or directed warfare to its logical conclusion in terms of social risk mitigation. Academic inquiry into the relationship between an ASI and treaties in terms of strategic expectations in many ways began with Bostrom’s (2014) musings on the potential relationship between as superintelligence ‘singleton’ and global hegemony. Our analysis suggests that a UGPT would establish a form of global hegemony, one directed towards the art of peace.
While this article has focused on conforming instrumentalism, it hopefully applies this to the UGPT in a way which is acceptable to a pluralism of theoretical perspectives. Certainly, conforming instrumentalism is a novel perspective; one of the most dominant schools of international relations thought is rationalist instrumentalism. Mantilla (2017, p.507) quotes Morrow (2014, p.35): “Norms and common conjectures aid actors in forming strategic expectations… Law helps establish this common knowledge by codifying norms.” Viewed via this rationalist-instrumentalist perspective, the present international norm for the majority of the world is peace, with the waging of interstate war being constrained by the United Nations Charter.
Despite this international norm of peace and the work of proponents of the art of peace, the lex pacificatoria (e.g., Bell, 2008), an absolute treaty-based approach to global peace has not yet been codified. As we point out in the Introduction, the UN Charter, despite embracing and promoting peace and peacekeeping (Fortna, 2008), does not strongly symbolise peace in the way a UGPT would. A UGPT would give new strength to the world’s peacekeepers, through major states promoting long-term peace as a new, global objective (Autesserre, 2014). A UGPT, championed by principled ‘norm entrepreneurs’ including states and NGOs (see e.g., Finnemore, 1996), would create a new ‘common knowledge’ in absolute terms that could constrain the risk to humanity of both conventional and existential war.
In rationalist-instrumentalist terms, a UGPT might be expected to have net adjustment benefits for adherence in terms of constraining conventional interstate conflicts, including the reduction of ongoing death tolls due to war and the risk of nuclear war. Thus, the UGPT would have high potential utility in the case of ‘flashpoints’ that could provoke existential war. For example, the Kashmir Conflict is one of the most protracted ongoing conflicts between nuclear powers, affecting both human rights (Bhat, 2019) and geopolitical stability (Kronstadt, 2019). Thus, if India and Pakistan both signed the UGPT, their actions would be constrained by the explicit goal of a commitment to universal peace. As outlined above, this may modify behaviour in several of the NKGPS zones, for instance by encouraging the efforts of peacebuilding organizations to depoliticise the conflict (e.g., Bhatnagar & Chacko, 2019).
The UGPT may also constrain the nuclear risk on the Korean peninsula, another flashpoint. The Korean War is an unresolved war involving nuclear powers (North Korea and South Korea, supported by the United States) (Kim, 2019). A UGPT would constrain the risk and severity of a conflict and, depending on the protocols signed, would encourage a path towards a peace treaty being signed. If only one party signed the UGPT, this would increase the moral standing of the state party that signed it. Mantilla’s (2017) emphasis on social constructivism suggests the global community could exert great pressure on North Korea to sign a peace treaty.
Turning to civil wars which could be flashpoints, the Syrian Civil War is one of the most costly wars of the 21st century in terms of the death toll and wider impacts (Council on Foreign Relations, 2020). It involves multiple state actors, including Iran, Israel, Russia, Turkey, and the United States, some of which possess nuclear weapons, with complex geopolitical implications (Tan & Perudin, 2019). Depending on the actors that signed the UGPT and the protocols that they adopted, the UGPT would constrain the severity of the conflict in various ways.
The existence of the UGPT would mean perpetual peace receiving more attention in cultural conditioning zones, including schools and the media, as well as in socialization zones, such as national defense universities and military camps, where teaching the Laws of War and the art of war (Allhoff, Evans, & Henschke, 2013) would, incrementally and over decades, transition to teaching establishing and maintaining the lex pacificatoria, the Laws of Peace and art of peace (Bell, 2008), in which the UGPT would play a main role.
In rationalist-instrumentalist terms, once the UGPT concept acquires sufficient traction, it is possible that states could compete for leadership in the framing, signing, and ratifying of the UGPT. Certainly, the United States viewed its own ratification of the Geneva Conventions prior to that by the Soviet Union as important to prevent a Soviet propaganda victory, one which it failed to prevent (Mantilla, 2017). Crucial to the UGPT’s success will be how seriously states view warfare that poses an existential threat, especially cyberwar and ASI-enabled or directed nuclear warfare
However, with regard to ASI-enabled or directed warfare, our analysis suggests that what will likely be most important is how states view the social argument for peace. As with the Geneva Conventions, social conformity factors, like supporting a humanitarian peace, conforming to ‘world standards’, and avoiding lagging behind peers, as well as religious perspectives, will likely predominate, and these represent important future avenues for research.
Conclusion
We now conclude this article on optimising peace through a Universal Global Peace Treaty (UGPT) leveraged off the ‘burning plasma’ fusion energy breakthrough (Draper & Bhaneja, 2020), to constrain risk of war from a militarised ASI. A treaty-based risk mitigation approach that included cyberwarfare and specifically mentioned ASI-enhanced warfare could affect the conceptualization of the AI race by reducing enmity between countries, increasing the level of openness between them, and raising social awareness of the risk. While these are external constraints, they may also constrain an ASI’s attitudes towards humanity in a positive way, either by reducing the threat it may perceive of war being waged against it, even if only symbolically, or by increasing the predictability of human action regarding peace.
Much work remains to be done on conceptualizing the UGPT, in preliminary drafting of the main body of the treaty and the protocols, in soliciting states’ interest, and in deliberations assessing thresholds and sovereignty costs, and in the eventual diplomatic conference where states would formally discuss the UGPT. While the UGPT may appear unrealistically ambitious, Mantilla’s (2017) work on conforming instrumentalism and the Geneva Conventions suggests a major sponsoring state would rapidly accumulate prestige by endorsing a path to peace, while states standing in the way would accumulate opprobrium, and that the social dynamics of the international community, whether involving social status or instrumental cooperation, do matter.
Future research on how to constrain the risk of ASI-enabled or directed warfare should consider the importance of peace in different ideologies, for instance Chinese socialism. This is important because, as we have outlined, ASIs developed by different nation-states may well be directed or imbued with different, potentially confrontational, ideologies. For instance, the China Brain Project is embracing a Chinese cultural approach towards neuroethics (Wang et al., 2019), and it is difficult to imagine that a Chinese ASI would not be directed according to Chinese cultural values and its ‘coherent extrapolated volition’ be informed by communist principles. Similarly, a Russian ASI could be informed by Cosmism and a Western ASI by liberal democratic principles.
In recommending such research, we caution that an ASI being created by a state engaged in ideological ‘New Cold War’ framing is likely to be militarized and weaponized. Still a New Cold War framing may have utilitarian function in, exerting social pressures towards signing the UGPT, for Mantilla (2017, pp.509-510) notes, “The Cold War context was also likely especially auspicious for the operation of social pressures, sharpening ideological competition in between the liberal, allegedly civilized world and ‘the rest’, communist or otherwise.”
Mantilla’s (2017) work also suggests excessive rigidity of attitude critical of such treaties may backfire in terms of the social dynamics of global prestige, particularly in the case of major states susceptible to accusations of warlike or imperialist behaviour which are engaged in propaganda wars with other major states. Effectively, the British ratification process for the Geneva Conventions demonstrates that instrumentalist concerns over lack of feasibility or reciprocity can be overruled by social constructivist concerns over ‘world opinion’.
Further research into the UGPT should also involve applying relevant game theory, such as iterated prisoner’s dilemma, especially the peace war game (see e.g., Gintis, 2000), to the major nation-states capable of building an ASI, as well as to the ASI itself. This game theory would need to investigate offering the opportunity for a newly emerged ASI to sign the UGPT, as an indicator of goodwill, which may assist in constraining the risk of the ASI waging war on humanity. An ASI with agency as signatory would view the UGPT as an external constraint on its own actions with regard to seeking global domination, in that the ASI would be subverting a humanity-imposed standard that could result in global retaliation and abandonment of mutual cooperation in pursuit of a common agreement on nonkilling and peace norms and values.
Even if the UGPT does not end humanity’s history of conflicts, it would represent a significant improvement in global public aspirations, and instrumental standards, for global peace, both of which may influence an ASI. Paraphrasing the United States Committee on Foreign Relations (1955, p.32), if the end result is only to obtain for those caught in the maelstrom of ASI-enabled or directed war a treatment which is 10 percent less vicious that they would receive without the Treaty, if only a few score of lives are preserved because of these efforts, then the patience and laborious work of all who will have contributed to that goal will not have been in vain.
That 10 percent difference could sway an ASI not to commit to a war for global domination, even if so directed or initially inclined.
References
Allen, G., & Chan, T. (2017). Artificial intelligence and national security. Cambridge, MA: Belfer Center.
Allen, G., & Kania, E.B. (2017, 8 September). “China is using America's own plan to dominate the future of artificial intelligence”. Foreign Policy. Retrieved from https://foreignpolicy.com/2017/09/08/china-is-using-americas-own-plan-to-dominate-the-future-of-artificial-intelligence/
Allhoff, F., Evans, N.G., & Henschke, A. (2013). Routledge handbook of ethics and war: Just war theory in the 21st century.
Allison, G. (2017). Destined for war: Can America and China escape Thucydides's trap? Boston, MA: Houghton Mifflin Harcourt.
Archibugi, D. (1992). Models of international organization in perpetual peace projects. Review of International Studies, 18(4), 295–317.
Archibugi, D. (1994). Immanuel Kant, cosmopolitan law, and peace. European Journal of International Relations, 1(3), 429-56.
Autesserre, S. (2014). Peaceland: Conflict resolution and the everyday politics of international intervention. Cambridge: Cambridge University Press.
Baldauf, S. (2012, 19 April). Sudan declares war on South Sudan: Will this draw in East Africa, and China? Christian Monitor. Retrieved from https://www.csmonitor.com/World/Keep-Calm/2012/0419/Sudan-declares-war-on-South-Sudan-Will-this-draw-in-East-Africa-and-China
Barbey, C. (2015). Non-militarisation: Countries without armies. Åland: The Åland Islands Peace Institute.
Barrett, A. M., & Baum, S. D. (2016). A model of pathways to artificial superintelligence catastrophe for risk and decision analysis. Journal of Experimental & Theoretical Artificial Intelligence, 29(2), 397–414. doi:10.1080/0952813x.2016.1186228.
Baum, S. D. (2016). On the promotion of safe and socially beneficial artificial intelligence. AI & Society, 32(4): 543–551. doi:10.1007/s00146-016-0677-0.
Baum, S. D. (2018). Countering superintelligence misinformation. Information, 9(10), 244.
Beier, J.M. (2020). Short circuit: Retracing the political for the age of ‘autonomous’ weapons Critical Military Studies, 6(1), 1-18.
Bell, C. (2008). On the law of peace: Peace agreements and the lex pacificatoria. Oxford: Oxford University Press.
Bell, D. (2007). The idea of Greater Britain: Empire and the future of world order, 1860-1900.
Benson-Tilsen, T., & Soares, N. (2016). Formalizing convergent instrumental goals. The Workshops of the Thirtieth AAAI Conference on Artificial Intelligence AI, Ethics, and Society: Technical Report WS-16-02. Palo Alto, CA: Association for the Advancement of Artificial Intelligence.
Bettez, D. J. (1988). Unfulfilled initiative: Disarmament negotiations and the Hague Peace Conferences of 1899 and 1907. RUSI Journal, 133(3), 57–62.
Bhat, S.A. (2019). The Kashmir conflict and human rights. Race and Class, 61(1), 77-86.
Bhatnagar, S., & Chacko, P. (2019). Peacebuilding think tanks, Indian foreign policy and the Kashmir conflict. Third World Quarterly, 40(8), 1496-1515.
Boele, O., Noordenbos, B., & Robbe, K. (2019). Post-Soviet nostalgia: Confronting the empire’s legacies. London: Routledge.
Bohman, J. (1997). Perpetual peace. Cambridge, MA: MIT Press.
Bostrom, N. (2002). Existential risks: Analyzing human extinction scenarios. Journal of Evolution and Technology, 9(1), 1-31.
Bostrom, N. (2006). What is a singleton? Linguistic and Philosophical Investigations, 5(2), 48-54.
Bostrom, N. (2013). Existential risk prevention as global priority. Global Policy, 4(1), 15–31.
Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford: Oxford University Press.
Bostrom, N. Douglas, T., & Sandberg, A. (2016). The unilateralist’s curse and the case for a principle of conformity. Social Epistemology, 30(4), 350-371.
Brain, M. (2003). Robotic nation. Retrieved from http://marshallbrain.com/robotic-nation.htm.
Brown, A., & Arnold, L. (2010). The quirks of nuclear deterrence. International Relations, 24(3), 293-312.
Brynjolfsson, E., & McAfee, A. (2011). Race against the machine. Lexington, MA: Digital Frontier.
Campanella, Edoardo, & Dassù, M. (2017). Anglo nostalgia: The politics of emotion in a fractured West. Oxford: Oxford University Press.
Cave, S., & ÓhÉigeartaigh, S.S. (2018). An AI race for strategic advantage: Rhetoric and risks. In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society - AIES '18 (pp. 36-40). New York: ACM Press.
Checkel, J.T. (2012). Theoretical pluralism in IR: Possibilities and limits. In W. Carlsnaes, T. Risse, & B. A. Simmons, Handbook of international relations, 2nd ed. (pp.220-242). London: Sage.
Christiano, P. (2018). Takeoff speeds. Retrieved from https://sideways-view.com/2018/02/24/takeoff-speeds/
Coontz, S. (1992). The way we never were: American families and the nostalgia trap. New York, NY: Basic Books.
Council on Foreign Relations. (2020). Global conflict tracker: Civil war in Syria. Retrieved from https://www.cfr.org/interactive/global-conflict-tracker/conflict/civil-war-syria
Davis, N., & Philbeck, T. (2017). 3.2 Assessing the risk of artificial intelligence. Davos: World Economic Forum. Retrieved from https://reports.weforum.org/global-risks-2017/part-3-emerging-technologies/3-2-assessing-the-risk-of-artificial-intelligence/
De Spiegeleire, S., Maas, M., & Sweijs, T. (2017). Artificial intelligence and the future of defence. The Hague: The Hague Centre for Strategic Studies. Retrieved from http://www.hcss.nl/sites/default/files/files/reports/Artificial%20Intelligence%20and%20the%20Future%20of%20Defense.pdf
Dewey, D. (2016). Long-term strategies for ending existential risk from fast takeoff. New York, NY: Taylor & Francis.
Draper, J., & Bhaneja, B. (2019). Fusion energy for peace building - A Trinity Test-level critical juncture. SocArXiv. https://doi.org/10.31235/osf.io/mrzua
Erickson, J.L. (2015). Dangerous trade: Arms exports, human rights, and international reputation. New York, NY: Columbia University Press.
Evangelista, M., & Tannenwald, N. (Eds.). (2017). Do the Geneva Conventions matter? Oxford: Oxford University Press.
Finnemore, M. (1996). National interests in international society. Ithaca, NY: Cornell University Press.
Finnemore, M., & Sikkink, K. (2001). Taking stock: The constructivist research program in international relations and comparative politics. Annual Review of Political Science, 4(1), 391-416.
Fisher, A. (2020). Demonizing the enemy: The influence of Russian state-sponsored media on American audiences. Post-Soviet Affairs.
Friedrich, B., Hoffmann, D., Renn, J., Schmaltz, F., & Wolf, M. (2017). One hundred years of chemical warfare: Research, deployment, consequences. Cham: Springer.
Gintis, H. (2000). Game theory evolving: A problem-centered introduction to modeling strategic behavior. Princeton, NJ: Princeton University Press.
Goldsmith, J.L., & Posner, E.A. (2015). The limits of international law. Oxford: Oxford University Press.
Goodman, R., & Jinks, D. (2013). Socializing states: Promoting human rights through international law. Oxford, NY: Oxford University Press.
Gruetzemacher, Ross. (2018). Rethinking AI strategy and policy as entangled super wicked problems. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society - AIES '18. New York, NY: ACM.
Gubrud, M.V. (1997). Nanotechnology and international security. Paper presented at the Fifth Foresight Conference on Molecular Nanotechnology, November 5-8, 1997; Palo Alto, CA. Retrieved from http://www.foresight.org/Conferences/MNT05/Papers/Gubrud/
Gwern. (2016). Why Tool AIs want to be Agent AIs. Retrieved from https://www.gwern.net/Tool-AI
Hallett, B. (1998). The lost art of declaring war. Chicago, IL: University of Illinois Press.
Helfer, L.R. (2012). Flexibility in international agreements. In J. Dunoff and M.A. Pollack (Eds.), Interdisciplinary perspectives on international law and international relations: The state of the art (pp. 175-197). Cambridge: Cambridge University Press.
Herman, A. (2004). To rule the waves: How the British navy shaped the modern world. London: Harper.
Herf, J. (1984). Reactionary modernism: Technology, culture, and politics in Weimar and the Third Reich. Cambridge: Cambridge University Press.
Hogg, I.V. (2002). German secret weapons of World War II: The missiles, rockets, weapons, and technology of the Third Reich. London: Greenhill Books.
Kahn, H. (1959). On thermonuclear war. Princeton, NJ: Princeton University Press.
Kahneman, D., & Lovallo, D. (1993). Timid choices and bold forecasts: A cognitive perspective on risk taking. Management Science, 39(1), 17–31.
Katz, E. (2006). Death by design: Science, technology, and engineering in Nazi Germany. London: Pearson Longman.
Katz, E. (2011). The Nazi engineers: Reflections on technological ethics in hell. Science and Engineering Ethics, 17(3): 571-82.
Kim, A.S. (2019). An end to the Korean War: The legal character of the 2018 summit declarations and implications of an official Korean peace treaty. Asian Journal of International Law, 9(2), 206-216.
Koblentz, G.D. (2009). Living weapons: Biological warfare and international security. Ithaca, NY: Cornell University.
Koh, H.H. (2005). Internalization through socialization. Duke Law Journal, 54(4), 975-982.
Kohler, K. (2019). The return of the ugly American: How Trumpism is pushing Zambia towards China in the 'New Cold War'. Perspectives on Global Development and Technology, 18(1-2), 186-204.
Kronstadt, K.A. (2019). India, Pakistan, and the Pulwama crisis. Washington DC: Congressional Research Service.
Krueger, N., & Dickson, P. R. (1994). How believing in ourselves increases risk taking: Perceived self‐efficacy and opportunity recognition. Decision Sciences, 25(3), 385–400.
Lams, L. (2018). Examining strategic narratives in Chinese official discourse under Xi Jinping. Journal of Chinese Political Science, 23(3), 387-411.
Liddell-Hart, B.H. (1967). Strategy. New York, NY: Frederick A. Praeger, Inc.
Macias, A. (2019, 13 February). From Aruba to Iceland, these 36 nations have no standing military. CNBC. Retrieved from https://www.cnbc.com/2018/04/03/countries-that-do-not-have-a-standing-army-according-to-cia-world-factbook.html
Mansfield-Devine, S. (2018). Nation-state attacks: The start of a new Cold War? Network Security, 2018(11), 15-19.
Mantilla, G. (2017). Conforming instrumentalists: Why the USA and the United Kingdom joined the 1949 Geneva Conventions. The European Journal of International Law, 28(2), 483-511.
Markusen, E., & Kopf, D. (2007). The Holocaust and strategic bombing: Genocide and total war in the twentieth century. Boulder, CO: Westview Press.
Mason, C. (2015). Engineering kindness: Building a machine with compassionate intelligence. International Journal of Synthetic Emotions, 6(1), 1-23.
Mauroni, A. J. (2007). Chemical and biological warfare: A reference handbook. Santa Barbara, CA: ABC-CLIO, Inc.
Mikaberidze, A. (Ed.). (2013). Atrocities, massacres, and war crimes: An encyclopedia. Santa Barbara, CA: ABC-Clio.
Miller, J.D. (2012). Singularity rising. Dallas, TX: BenBella.
Morrow, J.D. (2014). Order within anarchy: The laws of war as an international institution. Cambridge: Cambridge University Press.
Moss, K. B. (2008). Undeclared war and the future of U.S. foreign policy. Washington, DC: Woodrow Wilson International Center for Scholars.
Motlagh, V.V. (2012). Shaping the futures of global nonkilling society. In J.A. Dator & J.E. Pim (Eds.) Nonkilling futures: visions (pp. 99-114). Honolulu, HI: Center for Global Nonkilling.
Müller, H. (2014). Looking at nuclear rivalry: The role of nuclear deterrence. Strategic Analysis, 38(4), 464-475.
National Security Commission on Artificial Intelligence. (2019). Interim report. Washington, DC: Author.
Ohlin, J.D. (2015). The assault on international law. New York, NY: Oxford University Press.
Omohundro, S. (2008). The basic AI drives. Frontiers in Artificial Intelligence and Applications, 171(1), 483-492.
Fortna, V.P. (2008). Does peacekeeping work? Shaping belligerents' choices after civil war. Princeton, NJ: Princeton University Press.
Paige, G.D. (2009) Nonkilling global political science. Honolulu, HI: Center for Global Nonkilling.
Paige, G.D., & Ahn, C-S. (2012). Nonkilling Korea: Six culture exploration. Honolulu, HI: Center for Global Nonviolence and Seoul National University Asia Center.
Pim, J.E. (2010). Nonkilling societies. Honolulu, HI: Center for Global Nonkilling.
Pim, J.E. & Dhakal, P. (Eds.). (2015). Nonkilling spiritual traditions vol. 1. Honolulu, HI: Center for Global Nonkilling.
Pistono, F., & Yampolskiy, R. (2016). Unethical research: How to create a malevolent artificial intelligence. In: Proceedings of Ethics for Artificial Intelligence Workshop (AI-Ethics-2016) (pp. 1-7). New York, NY: AAAI.
Ramamoorthy, A., & Yampolskiy, R. (2018). Beyond MAD? The race for artificial general intelligence. ICT Discoveries, 1(Special Issue 1). Retrieved from http://www.itu.int/pub/S-JOURNAL-ICTS.V1I1-2018-9
Raymond, W.J. (1992). Dictionary of politics: Selected American and foreign political and legal terms. Lawrenceville, VA: Brunswick.
Rose, A. (2018). Mining memories with Donald Trump in the Anthropocene. MFS - Modern Fiction Studies, 64(4), 701-722.
Rotaru, V. (2019). Instrumentalizing the recent past? The new Cold War narrative in Russian public space after 2014. Post-Soviet Affairs, 35(1), 25-40.
Russell, S. J. (2019). Human compatible: Artificial intelligence and the problem of control. London: Allen Lane.
Russia Today. (2017). ‘Whoever leads in AI will rule the world’: Putin to Russian children on Knowledge Day. Russia Today, 1 September 2017.
Scharre, P. (2019). Killer apps: The real dangers of an AI arms race. Foreign Affairs. https://www.foreignaffairs.com/articles/2019-04-16/killer-apps
Schmitt, M. N. (2017). Tallinn manual 2.0 on the international law applicable to cyber operations. Cambridge: Cambridge University Press.
Segal, H.P. (2005). Technological utopianism in American culture: Twentieth anniversary edition. Syracuse, NY: Syracuse University Press.
Sharikov, P. (2018). Artificial intelligence, cyberattack, and nuclear weapons—A dangerous combination. Bulletin of the Atomic Scientists, 74(6), 368-373.
Shulman, C. (2010). Omohundro’s “basic AI drives” and catastrophic risks. MIRI technical report. Retrieved from http://intelligence.org/files/BasicAIDrives.pdf
Simmons, B.A. (2009). Mobilizing for human rights: International law in domestic politics. Cambridge: Cambridge University Press.
SIPRI. (2019). SIPRI military expenditure database. Retrieved from https://www.sipri.org/databases/milex
Sotala, K., & Yampolskiy, R.V. (2015). Responses to catastrophic AGI risk: A survey. Physica Scripta, 90(1), 1-33.
Tan, K.H., & Perudin, A. (2019). The “geopolitical” factor in the Syrian Civil War: A corpus-based thematic analysis. SAGE Open, 9(2), 1-15.
Tegmark, M. (2017). Life 3.0: Being human in the age of artificial intelligence. New York, NY: Knopf.
Terminski, B. (2010). The evolution of the concept of perpetual peace in the history of political-legal thought. Perspectivas Internacionales, 6(1): 277–291.
Tindley, A., & Wodehouse, A. (2016). Design, technology and communication in the British Empire, 1830–1914. London: Palgrave Macmillan.
Thomson, J. J. (1985). The trolley problem. The Yale Law Journal, 94(6), 1395–1415.
Turchin, A., & Denkenberger, D. (2017). Levels of self-improvement. Manuscript.
Turchin, A., & Denkenberger, D. (2018). Military AI as a convergent goal of self-improving AI. In R.V. Yampolskiy (Ed.), Artificial intelligence safety and security (pp. ). London: Chapman & Hall.
Turchin, A. & Denkenberger, D. (2020). Classification of global catastrophic risks connected with artificial intelligence. AI and Society, 35(1), 147-163.
Turchin, A., Denkenberger, D., & Green, B.P. (2019). Global solutions vs. local solutions for the AI safety problem. Big Data and Cognitive Computing, 3(1), 16.
United States Committee on Foreign Relations (1955). Geneva Conventions for the Protection of War Victims, Report to the United States Senate on Executives D, E, F, and G, 84th Congress, 1st Session, Executive Report no. 9. Washington, DC: Committee on Foreign Relations.
Walker, P. (2008, 9 August). Georgia declares 'state of war' over South Ossetia. The Guardian. Retrieved from https://www.theguardian.com/world/2008/aug/09/georgia.russia2
Walsh, J. I. (2018). The rise of targeted killing. Journal of Strategic Studies, 41(1–2), 143–159.
Walters, G. (2017, 6 September). Artificial intelligence is poised to revolutionize warfare. Seeker, https://www.seeker.com/tech/artificial-intelligence/artificial-intelligence-is-poised-to-revolutionize-warfare
Wang, Yi, et al. (2019). Responsibility and sustainability in brain science, technology, and neuroethics in China—A culture-oriented perspective. Neuron, 101(3), 375–379.
Ward, S. (2017). Status and the challenge of rising powers. Cambridge: Cambridge University Press.
Westad, O.A. (2019). The sources of Chinese conduct: Are Washington and Beijing fighting a New Cold War? Foreign Affairs, 98(5), 86-95.
World Bank. (2019). Military expenditure (% of general government expenditure). Retrieved from https://data.worldbank.org/indicator/MS.MIL.XPND.ZS?most_recent_value_desc=true
Yampolskiy, R. V. (2016). Taxonomy of pathways to dangerous artificial intelligence. In: AAAI Workshop - Technical Report, vWS-16-01 - WS-16-15 (2016) (pp. 143-148). Palo Alto, CA: Association for the Advancement of Artificial Intelligence.
Young, G. M. (2012). The Russian Cosmists: The esoteric futurism of Nikolai Fedorov and his followers.
Yudkowsky, E. (2004). Coherent extrapolated volition. San Francisco, CA: The Singularity Institute.
Yudkowsky, E. (2008). Artificial intelligence as a positive and negative factor in global risk. In N. Bostrom and M. M. Ćirković (Eds.), Global Catastrophic Risks (pp. 308–345). New York: Oxford University Press.
Zhao, M. (2019). Is a new cold war inevitable? Chinese perspectives on US-China strategic competition. Chinese Journal of International Politics, 12(3), 371-394.
Annex: State Commitment to UGPT Protocol I (Not to Declare or Engage in Existential Warfare, Defined as Atomic War, Biological War, Chemical War, or Cyberwar, including ASI-Enabled War)
Note: We ‘guesstimate’ probability employing a 5-point Likert Scale, from ‘Very Low’ to ‘Very High’; we do not assign equal intervals to categories. Note also that the scale is used in a relative way that does not reflect real-world probabilities.
https://drive.google.com/file/d/153OrxnuGEjsUvlxWsFYauslwNeCEkvUb/view
See also https://foreignpolicy.com/2019/12/11/artificial-intelligence-ai-not-arms-race-china-united-states/