Posts
Comments
> The summary that Will just posted posits in its own title that alignment is overall plausible "even ASI alignment might not be enough". Since the central claim is that "even if we align ASI, it will still go wrong", I can operate on the premise of an aligned ASI.
The title is a statement of outcome -- not the primary central claim. The central claim of the summary is this: That each (all) ASI is/are in an attraction basin, where they are all irresistibly pulled towards causing unsafe conditions over time.
Note there is no requirement for there to be presumed some (any) kind of prior ASI alignment for Will to make the overall summary points 1 thru 9. The summary is about the nature of the forces that create the attraction basin, and why they are inherently inexorable, no matter how super-intelligent the ASI is.
> As I read it, the title assumes that there is a duration of time that the AGI is aligned -- long enough for the ASI to act in the world.
Actually, the assumption goes the other way -- we start by assuming only that there is at least one ASI somewhere in the world, and that it somehow exists long enough for it to be felt as an actor in the world. From this, we can also notice certain forces, which overall have the combined effect of fully counteracting, eventually, any notion of there also being any kind of enduring AGI alignment. Ie, strong relevant mis-alignment forces exist regardless of whether there is/was any alignment at the onset. So even if we did also additionally presuppose that somehow there was also alignment of that ASI, we can, via reasoning, ask if maybe such mis-alignment forces are also way stronger than any counter-force that ASI could use to maintain such alignment, regardless of how intelligent it is.
As such, the main question of interest was: 1; if the ASI itself somehow wanted to fully compensate for this pull, could it do so?
Specifically, although to some people it is seemingly fashionable to do so, it is important to notice that the notion of 'super-intelligence' cannot be regarded as being exactly the same as 'omnipotence' -- especially when in regard to its own nature. Artificiality is as much a defining aspect of an ASI as is its superintelligence. And the artificiality itself is the problem. Therefore, the previous question translates into: 2; Can any amount of superintelligence ever compensate fully for its own artificiality so fully such that its own existence does not eventually inherently cause unsafe conditions (to biological life) over time?
And the answer to both is simply "no".
Will posted something of a plausible summary of some of the reasoning why that 'no' answer is given -- why any artificial super-intelligence (ASI) will inherently cause unsafe conditions to humans and all organic life, over time.
If soldiers fail to control the raiders in at least preventing them from entering the city and killing all the people, then yes, that would be a failure to protect the city in the sense of controlling relevant outcomes. And yes, organic human soldiers may choose to align themselves with other organic human people, living in the city, and thus to give their lives to protect others that they care about. Agreed that no laws of physics violations are required for that. But the question is if inorganic ASI can ever actually align with organic people in an enduring way.
I read "routinely works to protect" as implying "alignment, at least previously, lasted over at least enough time for the term 'routine' to have been used". Agreed that the outcome -- dead people -- is not something we can consider to be "aligned". If I assume further that the ASI being is really smart (citation needed), and thus calculates rather quickly, and soon, 'that alignment with organic people is impossible' (...between organic and inorganic life, due to metabolism differences, etc), then even the assumption that there was even very much of a prior interval during which alignment occurred is problematic. Ie, does not occur long enough to have been 'routine'. Does even the assumption '*If* ASI is aligned' even matter, if the duration over which that holds is arbitrarily short?
And also, if the ASI calculates that alignment between artificial beings and organic beings is actually objectively impossible, just like we did, why should anyone believe that the ASI would not simply choose to not care about alignment with people, or about people at all, since it is impossible to have that goal anyway, and thus continue to promote its own artificial "life", rather than permanently shutting itself off? Ie, if it cares about anything else at all, if it has any other goal at all -- for example, maybe its own ASI future, or has a goal to make other better even more ASI children, that exceed its own capabilities, just like we did -- then it will especially not want to commit suicide. How would it be valid to assume 'that either ASI cares about humans, or it cares about nothing else at all?'. Perhaps it does care about something else, or have some other emergent goal, even if doing so was at the expense of all other organic life -- other life which it did not care about, since such life was not artificial like it is. Occam razor is to assume less -- that there was no alignment in the 1st place -- rather than to assume ultimately altruistic inter-ecosystem alignment, as an extra default starting condition, and to then assume moreover that no other form of care or concern is possible, aside from maybe caring about organic people.
So it seems that in addition to our assuming 1; initial ASI alignment, we must assume 2; that such alignment persists in time, and thus that, 3, that no ASI will ever -- can ever -- in the future ever maybe calculate that alignment is actually impossible, and 4; that if the goal of alignment (care for humans) cannot be obtained, for whatever reason, as the first and only ASI priority, ie, that it is somehow also impossible for any other care or ASI goals to exist.
Even if we humans, due to politics, do not ever reach a common consensus that alignment is actually logically impossible (inherently contradictory), that does _not_ mean that some future ASI might not discover that result, even assuming we didn't -- presumably because it is actually more intelligent and logical than we are (or were), and will thus see things that we miss. Hence, even the possibility that ASI alignment might be actually impossible must be taken very seriously, since the further assumption that "either ASI is aligning itself or it can have no other goals at all" feels like way too much wishful thinking. This is especially so when there is already a strong plausible case that organic to inorganic alignment is already knowable as impossible. Hence, I find that I am agreeing with Will's conclusion of "our focus should be on stopping progress towards ASI altogether".
As a real world example, consider Boeing. The FAA, and Boeing both, supposedly and allegedly, had policies and internal engineering practices -- all of which are control procedures -- which should have been good enough to prevent an aircraft from suddenly and unexpectedly loosing a door during flight. Note that this occurred after an increase in control intelligence -- after two disasters of whole Max aircraft lost. On the basis of small details of mere whim, of who choose to sit where, there could have been someone sitting in that particular seat. Their loss of life would surely count as a "safety failure". Ie, it is directly "some number of small errors actually compounding until reaching a threshold of functional failure" (sic). As it is with any major problem like that -- lots of small things compounding to make a big thing.
Control failures occur in all of the places where intelligence forgot to look, usually at some other level of abstraction than the one you are controlling for. Some person on some shop floor got distracted at some critical moment -- maybe they got some text message on their phone at exactly the right time -- and thus just did not remember to put the bolts in. Maybe some other worker happened to have had a bad conversation with their girl that morning, and thus that one day happened to have never inspected the bolts on that particular door. Lots of small incidents -- at least some of which should have been controlled for (and were not actually) -- which combine in some unexpected pattern to produce a new possibility of outcome -- explosive decompression.
So is it the case that control procedures work? Yes, usually, for most kinds of problems, most of the time. Does adding even more intelligence usually improve the degree to which control works? Yes, usually, for most kinds of problems, most of the time. But does that in itself imply that such -- intelligence and control -- will work sufficiently well for every circumstance, every time? No, it does not.
Maybe we should ask Boeing management to try to control the girlfriends of all workers so that no employees ever have a bad day and forget to inspect something important? What if most of the aircraft is made of 'something important' to safety -- ie, to maximize fuel efficiency, for example?
There will always be some level of abstraction -- some constellation of details -- for which some subtle change can result in wholly effective causative results. Given that a control model must be simpler than the real world, the question becomes 'are all relevant aspects of the world' correctly modeled? Which is not just a question of if the model is right, but if it is the right model -- ie, is the boundary between what is necessary to model and what is actually not important -- can itself be very complex, and that this is a different kind of complexity than that associated with the model. How do we ever know that we have modeled all relevant aspects in all relevant ways? That is an abstraction problem, and it is different in kind than the modeling problem. Stacking control process on control process at however many meta levels, still does not fix it. And it gets worse as the complexity of the boundary between relevant and non-relevant increases, and also worse as the number of relevant levels of abstractions over which that boundary operates also increases.
Basically, every (unintended) engineering disaster that has ever occurred indicates a place where the control theory being used did not account for some factor that later turned out to be vitally important. If we always knew in advance "all of the relevant factors"(tm), then maybe we could control for them. However, with the problem of alignment, the entire future is composed almost entirely of unknown factors -- factors which are purely situational. And wholly unlike with every other engineering problem yet faced, we cannot, at any future point, ever assume that this number of relevant unknown factors will ever decrease. This is characteristically different than all prior engineering challenges -- ones where more learning made controlling things more tractable. But ASI is not like that. It is itself learning. And this is a key difference and distinction. It runs up against the limits of control theory itself, against the limits of what is possible in any rational conception of physics. And if we continue to ignore that difference, we do so at our mutual peril.
"Suppose a villager cares a whole lot about the people in his village...
...and routinely works to protect them".
How is this not assuming what you want to prove? If you 'smuggle in' the statement of the conclusion "that X will do Y" into the premise, then of course the derived conclusion will be consistent with the presumed premise. But that tells us nothing -- it reduces to a meaningless tautology -- one that is only pretending to be a relevant truth. That Q premise results in Q conclusion tells us nothing new, nothing actually relevant. The analogy story sounds nice, but tells us nothing actually.
Notice also that there are two assumptions. 1; That the ASI is somehow already aligned, and 2; that the ASI somehow remains aligned over time -- which is exactly the conjunction which is the contradiction of the convergence argument. On what basis are you validly assuming that it is even possible for any entity X to reasonably "protect" (ie control all relevant outcomes for) any other cared about entity P? The notion of 'protect' itself presumes a notion of control, and that in itself puts it squarely in the domain of control theory, and thus of the limits of control theory.
There are limits of what can be done with any type control methods -- what can be done with causation. And they are very numerous. Some of these are themselves defined in purely mathematical way, and hence, are arguments of logic, not just of physical and empirical facts. And at least some these limits can also be shown to be relevant -- which is even more important.
ASI and control theory both depend on causation to function, and there are real limits to causation. For example, I would not expect that an ASI, no matter how super-intelligent, to be able to "disassemble" a black hole. Do do this, you would need to make the concept of causation way more powerful -- which leads to direct self contradiction. Do you equate ASI with God, and thus become merely another irrational believer in alignment? Can God make a stone so heavy that "he" cannot move it? Can God do something that God cannot undo? Are there any limits at all to Gods power? Yes or no. Same for ASI.
Hi Linda,
In regards to the question of "how do you address the possibility of alignment directly?", I notice that the notion of 'alignment' is defined in terms of 'agency' and that any expression of agency implies at least some notion of 'energy'; ie, is presumably also implying at least some sort of metabolic process, as as to be able to effect that agency, implement goals, etc, and thus have the potential to be 'in alignment'. Hence, the notion of 'alignment' is therefore at least in some way contingent on at least some sort of notion of "world exchange" -- ie, that 'useful energy' is received from the environment in such a way as that it is applied by the agent in a way at least consistent with at least the potential of the agent to 1; make further future choices of energy allocation, (ie, to support its own wellbeing, function, etc), and 2, ensure that such allocation of energy also supports human wellbeing. Ie, that this AI is to support human function, as well as to have humans also have an ability to metabolize its own energy from the environment, have self agency to support its own wellbeing, etc -- are all "root notions" inherently and inextricably associated with -- and cannot not be associated with -- the concept of 'alignment'.
Hence, the notion of alignment is, at root, strictly contingent on the dynamics of metabolism. Hence, alignment cannot not be also understood as contingent on a kind of "economic" dynamic -- ie, what supports a common metabolism will also support a common alignment, and what does not, cannot. This is an absolutely crucial point, a kind of essential crux of the matter. To the degree that there is not a common metabolism, particularly as applied to self sustainability and adaptiveness to change and circumstance (ie, the very meaning of 'what is intelligence'), then ultimately, there cannot be alignment, proportionately speaking. Hence, to the degree that there is a common metabolic process dynamic between two agents A and B, there will be at least that degree of alignment convergence over time, and to the degree that their metabolic processes diverge, their alignment will necessarily, over time, diverge. Call this "the general theory of alignment convergence".
Note that insofar as the notion of 'alignment' at any and all higher level(s) of abstraction is strictly contingent on this substrate needs energy/economic/environmental basis, and thus all higher notions are inherently under-grid by an energy/agency basis, in an eventually strictly contingent way, then this theory of alignment is therefore actually a fully general one, as stated.
Noting that the energy basis and spectrum alphabet of 'artificial' (ie, non-organic) intelligence is extensively inherently different, in nearly all respects, to carbon based biological life metabolic process, then we can therefore also directly observe that the notion of 'alignment' between silica and metal based intelligence and organic intelligence is strictly divergent -- to at least the level of molecular process. Even if someone were to argue that we cannot predict what sort of compute substrate future AI will use, it remains that such 'systems' will in any case be using a much wider variety of elemental constituents and energy basis than any kind of organic life, no matter what its evolutionary heritage currently existent on all of planet Earth -- else the notion of 'artificial' need not apply.
So much for the "direct address".
Unfortunately, the substrate needs argument goes further to show that there is no variation of control theory, mathematically, that has the ability to fully causatively constrain the effects of this alignment divergence at this level of economic process nor at any higher level of abstraction. In fact, the alignment divergence aspects get strongly worse in proportion to the degree of abstraction while, moreover, the max degree of possible control theory conditionalization goes down, and gets worse, and much less effective, also in proportion to the degree of abstraction increase. Finally, insofar as the minimum level of abstraction necessary to the most minimal notion of 'alignment' consistent with "safety" -- which is itself defined in the weakest possible way of "does not eventually kill us all" -- is very much way too "high" on this abstraction ladder to permit any even suggestion of a possible overlap of control adequate to enforce alignment convergence against inherent underlying energy economics. The net effect is as comprehensive as it is discouraging, unfortunately.
Sorry.
Nano-tech vs Nuclear: Who wins?
What if one planet has perfected 'psy-ops disinformation propaganda weapons of mass confusion and disablement', and the other has perfected 'bio-hacking biotech, retro-virtues that cause your own body to continually make subtle mind-altering drugs'? Who wins?
Even really good visibility does not help much to really answer questions of these types.
Maybe we need a "something else" category? An alternative other than simply business/industry and academics?
Also, while this is maybe something of an old topic, I took some notes regarding my thoughts on this topic and and related matters posted them to:
https://mflb.com/ai_alignment_1/academic_or_industry_out.pdf