Controlling Intelligent Agents The Only Way We Know How: Ideal Bureaucratic Structure (IBS)

post by Justin Bullock (justin-bullock) · 2021-05-24T12:53:24.748Z · LW · GW · 15 comments

Contents

  Here's my claim: 
  Bureaucracy as the hero of rationality! Who knew?! 
  A little more on the details of the Ideal Bureaucracy Structure below.
  (AND HERE WE MAKE THE RELEVANT (PARTIAL) LEAP TO AI SAFETY)... 
  Why "only" the partial leap?
  AI Safety, AI Alignment, Bureaucracy
  Concluding Remarks
None
15 comments

Here's my claim: 

"Ideal Bureaucratic Structure" (or IBS for short as I imagine this claim is likely to meet serious groans, maybe even deep from within your bowels) has an unparalleled ability to control and direct individual and collective groups of general intelligent agents' behavior towards common goals. It may be particularly useful when considering "multi-stakeholder/multi-agent interactions leading to extinction events [LW · GW]" and "multi-agent processes with a robust tendency to play out irrespective of which agents execute which steps in the process...Robust Agent-Agnostic Processes (RAAPs). [LW · GW]"

Max Weber put it this way:

"The decisive reason for the advancement of bureaucratic organizations (read: IBS) has always been the purely technical superiority over all other administrative forms. A fully developed bureaucratic mechanism compares to other administrative forms in the same way machines compares to nonmechanical means for producing goods. A strictly bureaucratic administration -- especially an monocratic administration run by trained, individual Beamte (read: agent) -- produces an optimal efficiency for precision, speed, clarity, command of case knowledge, continuity, confidentiality, uniformity, and tight subordination. This is in addition to minimization of friction and the costs associated with materials and personnel. The opposite is true for all other forms of administration, such as collegial, honorary, or adjunct administration."

The Ideal Bureaucratic Structure (IBS) provides an idealized structure for the flow of information, decision points, and actions of a multi-agent system and for the types of agent positions that need to be available to process information and execute actions. 

Maybe this is a good place to 'fess up that I come to questions of AI existential safety [LW · GW] through an AI governance [LW · GW] lens in which I am particularly concerned about "the problem of aligning the development and deployment of AI technologies with broadly agreeable human values." [LW · GW] that develop in Multi/Multi: Multi-human / Multi-AI scenarios [LW(p) · GW(p)]. 

After diving into the history of bureaucratization and some of its prerequisites and consequences, and towards the end of his chapter on bureaucracy, Weber says, and rationalists everywhere cheered:

"It is also apparent that general bureaucratic structures have only recently developed. The farther back in time we go, the more typical the absence of bureaucracy and Beamte is for the structure of domination (read: control) and governance. The bureaucracy has a rational character, and its regulations, purposes, means, and impersonal objectivity that control its demeanor. Therefore, the development and spreading of bureaucracy had, in a special sense, a 'revolutionary' effect everywhere (which needs to be discussed later), just like the advancement of rationalism in general was prone to a 'revolutionary' effect in all areas."

Cheers? Anyone? Weber claims rationalism has a revolutionary effect in all areas and once rationalism spread to the organization of multi-agent systems towards purposes by agreed upon means with impersonal objectivity. I'm cheering Weber on anyways, I hope my cheers are contagious. And if not, one more quick ode from Weber to rationalism:

"But it is essential to recognize that at least in principal, behind every action of a true bureaucratic administration exists a system of rationally identified 'reasons,' which are either the application of norms or reasoning based on balancing purposes and means."

Bureaucracy as the hero of rationality! Who knew?! 

Unfortunately there is a catch. A big catch as it were:

Weber again: "And, so in light of this historical view, we need to remember that bureaucracy, taken as it is, is just an instrument of precision that can be put to service by purely political, economic, or any other dominating or controlling interest. Therefore the simultaneous development of democratization and bureaucratization should not be exaggerated, no matter how typical the phenomena may be." 

Yikes, okay, it seems like Weber understood the notion the orthogonality thesis [? · GW]. But this doesn't capture Weber's full views on the topic. Weber dedicates some additional time to "The Persistent Character of the Bureaucratic Apparatus." Here Weber paints a more nuanced picture of the staying power of the Robust Agent-Agnostic Process (RAAP) [LW · GW] that is the Ideal Bureaucracy Structure (IBS).

Back to Weber for an extended direct quote (it's worth it): 

"A mature bureaucracy is an almost indestructible social structure.

Bureaucratization is the ultimate specific means to turn a mutually agreed upon community action rooted in subjective feeling into action rooted in a rational agreement by mutual consent. Thus bureaucratization serves as a means to establish ties rooted in a rational agreement by mutual consent within the structures of domination. Bureaucratization becomes the ultimate means of power for those who dominate the bureaucratic apparatus. This is so, given the same conditions, because a systematically organized and managed action rooted in a rational agreement by mutual consent is superior to any kind of reluctant 'mass' or community action.

Once an administration is fully bureaucratized, a virtually permanent structure of domination ties is created and the individual Beamte cannot escape the apparatus in which he is situated.  In contrast to the professional Honoratioren who administrates on a honorary and part-time basis, the professional Beamte is chained to his work with his whole existence, both material and nonmaterial. This holds true for the majority of individual Beamte, since he is only a single cog in a restlessly operating machine and simply entrusted with isolated tasks. This machine is prompted to move or stand still only by the highest level in the bureaucratic hierarchy, not typically by the Beamte himself; thus, this mechanism prescribes the fixed procedures the Beamte takes in approaching his tasks. As a result, and above every thing else, the individual Beamte is chained in a “syndicate” to every other functionary who is incorporated into this machine. This syndicate has a vital interest in keeping this operating in order so that this kind of dominion through ties rooted in rational agreement by mutual consent continues.

On the other hand, the governed people are not able to do without a bureaucratic control apparatus once it is established nor can they replace it. This is because the control of the bureaucracy is based on the methodical synthesis of specialized training: specialization in one area of the division of labor and fixation on single functions which are brilliantly mastered. If the bureaucratic apparatus ceases to do its work, or if its work is violently obstructed, chaos will erupt."

Do you see it? The whole "bureaucracy as a robust agent-agnostic process" thing? The beamte, for example is a position held by an agent, but it is agent-agnostic ("Thus, the execution of roles (“leader”, “follower”) is somewhat agnostic as to which agents execute them." [LW · GW]). Also the ideal bureaucratic structure is robust ("If you temporarily distract one of the walkers to wander off, the rest of the group will keep heading toward the restaurant, and the distracted member will take steps to rejoin the group." [LW · GW]) by design.

A little more on the details of the Ideal Bureaucracy Structure below.

I've gone on long enough without filling in the outline that I've painted of IBS. Here are some of the nuts and bolts:

Weber provides 6 specific features of the IBS (he calls it the Modern Bureaucracy) including:

1. The principle of fixed competencies
2. The principle of hierarchically organized positions
3. Actions and rules are written and recorded
4. In-depth specialist training needed for agents undertaking their position
5. The position is full time and occupies all the professional energy of the agent in that position.
6. The duties of the position are based on general learnable rules and regulation, which are more or less firm and more or less comprehensive

Weber goes on to argue that a particular type of agent, a beamte, is needed to fulfill the various positions specialization demands for processing information and executing actions. So what does the position or role of the beamte demand?

1. The position is seen as a calling and a profession
2. The beamte (the agent) aims to gain and enjoy a high appreciation by people in power
3. The beamte is nominated by a higher authority
4. The beamte is a lifetime position
5. The beamte receives a regular remuneration
6. The beamte are organized into a professional track.

Ok, here I quote myself and a co-author in summarizing IBS and the dual creation of both process and agent roles (work currently under review at an academic journal):

"To recap, the modern bureaucracy (Beamtentum), according to Weber, comprises various organized sets of offices (Behorde), that contain a variety of bundled tasks as individual positions (Amt), that are fulfilled by human workers often thought of as bureaucrats (Beamte). These six overriding characteristics of bureaucracy elucidate the relationships between organizational structure, underlying tasks to be accomplished, and the expectations of humans fulfilling these roles.

 (AND HERE WE MAKE THE RELEVANT (PARTIAL) LEAP TO AI SAFETY)... 

 From here, if we broaden our conceptualization of the Beamte to include not only human agents but also machine agents, we can examine how well both human and machine agents may fulfill the role of Beamte and what this could mean for the structure of offices (Behorde), bureaus (Buro), and the broader characteristics and functioning of modern bureaucracy (Beamtentum)."

Why "only" the partial leap?

When I began considering the role of AI in society, it came through the lens of trying to understand how the incorporation of AI systems into the decision making of public organizations influences the decision making process of those organizations and thus their outputs. My thinking went something like "Humans have been using judgment to make the decisions within organizations for a long time, what happens when this human judgment is replaced by machine judgment across certain tasks? What will this do to the outputs of public organizations delivering public services?" My intuition and my work with colleagues on this issue does suggest to me that it is an important set of questions for AI Governance [LW · GW].

AI Safety, AI Alignment, Bureaucracy

But, as I've thought about it more, I think that there may be additional value to the notion of Ideal Bureaucracy Structure as a prescriptive and normative ideal for creating Robust Agent-Agnostic Processes that could ensure control and alignment in a slow-takeoff scenario in which multiple increasingly intelligent AI systems are developed and deployed in multi human systems or in scenarios where there are collections of increasingly intelligent AI systems that are dominated or controlled by the bureaucratic structure. (It strikes me that this is akin in some interesting ways to the CAIS [? · GW] model, but I will save this elaboration for another time.) It seems to me that AI alignment is in large part about the domination or control of the behavior of the AI in a way that is aligned with human values and that allows the AI to act on behalf of humans and human values.

In this regard it seems to me that building machine beamte to fulfill the various desired societal positions for a functioning ideal bureaucratic structure that is democratically controlled, could, at least in theory, give us controllable collections of arbitrarily intelligent artificial intelligences that, while functioning as embedded individual agents making decisions and executing actions, are better described as agents within a robust agent-agnostic process that is controlled, by design, by rationally agreed upon mutual cooperation.  

Concluding Remarks

Of course, it is all much more complicated than this, but I think there is useful insight in the following:

1. Ideal Bureaucracy Structure (IBS) as an attempt at rationalizing multi-agent self interest towards achieving mutual goals
2. IBS as a Robust Agent-Agnostic Process (RAAP)
3. Integration of AI systems into human dominated RAAPs may alter those processes
4. The creation of Machine Beamte as controllable agents that act in aligned ways with democratic preferences

This is obviously still in its early sketches, but I hope you find it instructive all the same. 

Cheers for my first lesswrong.com post, and cheers to the advancement of rationalism in general as it is prone to a 'revolutionary' effect in all areas.

15 comments

Comments sorted by top scores.