The Other Earth (Chapter 2) 2021-12-31T16:20:11.641Z
Machine Agents, Hybrid Superintelligences, and The Loss of Human Control (Chapter 1) 2021-11-30T17:35:32.607Z
How Humanity Lost Control and Humans Lost Liberty: From Our Brave New World to Analogia (Sequence Introduction) 2021-11-15T14:22:53.130Z
Controlling Intelligent Agents The Only Way We Know How: Ideal Bureaucratic Structure (IBS) 2021-05-24T12:53:24.748Z


Comment by Justin Bullock (justin-bullock) on How Humanity Lost Control and Humans Lost Liberty: From Our Brave New World to Analogia (Sequence Introduction) · 2021-11-16T14:53:09.742Z · LW · GW

Thank you! I’m looking forward to the process of writing it, synthesizing my own thoughts, and sharing them here. I’ll also be hoping to receive your insightful feedback, comments, and discussion along the way!

Comment by Justin Bullock (justin-bullock) on Reflection of Hierarchical Relationship via Nuanced Conditioning of Game Theory Approach for AI Development and Utilization · 2021-06-05T16:52:23.987Z · LW · GW

Thank you for this post, Kyoung-Cheol. I like how you have used Deep Mind's recent work to motivate the discussion of the consideration of "authority as a consequence of hierarchy" and that "processing information to handle complexity requires speciality which implies hierarchy." 

I think there is some interesting work on this forum that captures these same types of ideas, sometimes with similar language, and sometimes with slightly different language.

In particular, you may find the recent post from Andrew Critch on "Power dynamics as a blind spot or blurry spot in our collective world-modeling, especially around AI" to sympathetic to core pieces of your argument here. 

It also looks like Kaj Sotala is having some similar thoughts on adjustments to game theory approaches that I think you would find interesting.

 I wanted to share with you an idea that remains incomplete, but I think there is an interesting connection between Kaj Sotala's discussion of non-agent and multi-agent models of the mind and Andrew Critch's robust agent-agnostic processes that connects with your ideas here and the general points I make in the IBS post.

Okay, finally, I had been looking for the most succinct quote from Herbert Simon's description of complexity and I found it. At some point, I plan to elaborate more on how this connects to control challenges more generally as well, but I'd say that we would both likely agree with Simon's central claim in the final chapter of The Sciences of the Artificial:

"Thus my central theme is that complexity frequently takes the form of hierarchy and that hierarchic systems have some common properties independent of their specific content. Hierarchy, I shall argue, is one of the central structural schemes that the architect of complexity uses." 

Glad you decided to join the conversation here. There are lots of fascinating conversation that are directly related to a lot of the topics we discuss together.

Comment by Justin Bullock (justin-bullock) on Controlling Intelligent Agents The Only Way We Know How: Ideal Bureaucratic Structure (IBS) · 2021-05-26T18:10:13.992Z · LW · GW

Thanks for this. I tabbed the Immoral Mazes sequences. On cursory view it seems very relevant. I'll be working my way through it. Thanks again.

Comment by Justin Bullock (justin-bullock) on Controlling Intelligent Agents The Only Way We Know How: Ideal Bureaucratic Structure (IBS) · 2021-05-26T18:04:50.139Z · LW · GW

Thanks. I think your insight is correct that governance requires answers to the "how" and "what" questions, and that the bureaucratic structure is one answer, but it leave the "how" unanswered. I don't have a good technical answer, but I do have an interesting proposal by Hannes Alfven in the book "The End of Man?" that he published under the pseudonym of Olof Johnneson called Complete Freedom Democracy that I like. The short book is worth the read, but hard to find. The basic idea is a parliamentary system in which all humans, through something akin to a smart phone, to rank vote proposals. I'll write up the details some time! 

Comment by Justin Bullock (justin-bullock) on Controlling Intelligent Agents The Only Way We Know How: Ideal Bureaucratic Structure (IBS) · 2021-05-26T17:50:10.813Z · LW · GW

Thank you for the comment. There are several interesting points I want to comment on. Here are my thoughts in no particular order of importance:

  • I think what I see as your insight on rigidity versus flexibility (rigid predictable rules vs. innovation) more generally is helpful and something that is not addressed well in my post. My own sense is that an ideal bureaucracy structure could be rationally constructed that balances tradeoffs across rigidity and innovation. Here I would also take Weber's rule 6 that you highlight as an example. As represented in the post it states "The duties of the position are based on general learnable rules and regulation, which are more or less firm and more or less comprehensive." I take this as rules and regulation need to be "learnable" not stable. A machine beamte (generally intelligent AI) should be able to quickly update on new rules and regulations. The condition of "more or less firm and more or less comprehensive" seems akin to more of a coherence condition rather than one that is static
  • This builds towards what I see as your concern of an ideal bureaucracy structure being consisted of fixed rules, ossification, and general inability to adapt successfully to changes in the type and character of complexity in the environment in which the bureaucracy is embedded. My sense is that these are not fundamental components of a rationally applied bureaucratic structure, but rather of the limited information and communication capabilities of the agents that hold the positions within the bureaucratic structure. My sense is that AIs could overcome these challenges given some flexibility in structure based on some weighted voting mechanism by the AIs.
  • One note here is that for me an ideal bureaucracy structure doesn't need to perfectly replicate Weber's description. Instead it would appropriately take into account what I see as the underlying fact that complexity demands specialization and coordination which implies hierarchy. An ideal bureaucracy structure would be one that requires multiple agents to specialize and coordinate to solve problems of any arbitrary level of complexity, which requires specifying both horizontal and vertical coordination. Weber's conceptualization as described in the post, I think, deserves more attention for the alignment problem, given that I think bureaucracies limitations can mostly be understood in terms of human limitation for information processing and communication.
  • I think I share your concern with a single bureaucracy of AI's being suboptimal, unless the path to superintelligence is through iterated amplification of more narrow AI's that eventually lead to joint emergent superintelligence that is constrained in an underlying way by the bureaucratic structure, training, and task specialization. This is a case where (I think) the emergence of a superintelligent AI that in reality functions like a bureaucracy would not necessarily be suboptimal. It's not clear to me that if the bureaucratic norms and training could be updated for better rules and regulation to be imposed upon it why it would need to be overthrown.
  • I would suggest that market competition and bureaucratic structure are along a continuum of structures for effectively and efficiently processing information. One takes a more decentralized processing based largely on prices to convey relevant value and information the other takes a more centralized approach implied by loosely organized hierarchical structures that allow for reliable specialization. It seems to me that market mechanisms also have their own tradeoffs across innovation and controllability. In other words, I do not see that the market structure dominates the bureaucratic or centralized approach across these tradeoffs in particular.
  • There are other governance models that I think are helpful for the discussion as well. Weber is one of the oldest in the club. One is Herbert Simon's Administrative Behavior (which is generalized to other types of contexts in his "The Sciences of the Artificial"). Another is Elinor Ostrom's Institutional Analysis and Development Framework. My hope is build out posts in the near future taking these adjustments in structure into consideration and discussing the tradeoffs.


Thanks again for the comment. I hope my responses have been helpful. Additional feedback and discussion are certainly welcomed! 

Comment by Justin Bullock (justin-bullock) on Controlling Intelligent Agents The Only Way We Know How: Ideal Bureaucratic Structure (IBS) · 2021-05-26T16:27:56.884Z · LW · GW

I think this approach may have something to add to Christiano's method, but I need to give it more thought. 

I don't think it is yet clear how this structure could help with the big problem of superintelligent AI. The only contributions I see clearly enough at this point are redundant to arguments made elsewhere. For example, the notion of a "machine beamte" as one that can be controlled through (1) the appropriate training and certification, (2) various motivations and incentives for aligning behavior with the knowledge from training, and (3) nominated by a higher authority for more influence. These are not novel considerations of course, but I think they do very much point to the same types of concerns of how to control agent behavior in an aligned way when the individual intelligent agents may have some components that are not completely aligned with the goal function of the principal (organization in this context, keeping superintelligent AI controlled by humanity as another potential context).

Thanks for the follow up.

Comment by Justin Bullock (justin-bullock) on Open and Welcome Thread - May 2021 · 2021-05-26T16:13:43.025Z · LW · GW

Thank you for this. I pulled up the thread. I think you're right that there are a lot of open questions to look into at the level of group dynamics. I'm still familiarizing myself with the technical conversation around the iterated prisoner's dilemma and other ways to look at these challenges from a game theory lens. My understanding so far is that some basic concepts of coordination and group dynamics like authority and specialization are not yet well formulated, but again, I don't consider myself up to date in this conversation yet.

From the thread you shared, I came across this organizing post I found helpful:

Thanks for the comment.

Comment by Justin Bullock (justin-bullock) on Controlling Intelligent Agents The Only Way We Know How: Ideal Bureaucratic Structure (IBS) · 2021-05-26T00:03:30.350Z · LW · GW

Thank you for the insights. I agree with your insight that "bureaucracies are notorious homes to Goodhart effects and they have as yet found no way to totally control them." I also agree with you intuition that "to be fair bureaucracies do manage to achieve a limited level of alignment, and they can use various mechanisms that generate more vs. less alignment." 

I do however believe that an ideal type of bureaucratic structure helps with at least some forms of the alignment problem. If for example, Drexler is right, and my conceptualization of the theory is right (CAIS) expects a slow takeoff of increasing intelligent narrow AIs that work together on different components of intelligence or completing intelligent tasks. In this case, I think Weber's suggestions both of how to create generally controllable intelligent agents (Beamte) and his ideas on constraining individual agents authority to certain tasks who are then nominated to higher tasks by those with more authority (weight, success, tenure, etc) has something helpful to say in the design of narrow agents that might work together towards a common goal. 

My thoughts here are still in progress and I'm planning to spend time with these two recent posts in particular to help my understanding:


One final thing I would add is that I think many of problems with bureaucracies can often be characterized around limits of information and communication (and how agents are trained and how they are motivated and what are the most practical or useful levels of hierarchy or discretion). I think the growth of increasingly intelligent narrow AIs could (under the right circumstance) drastically limit information and communication problems. 

Thanks again for your comment. The feedback is helpful. I hope to make additional posts in the near future to try and further develop these ideas.

Comment by Justin Bullock (justin-bullock) on Open and Welcome Thread - May 2021 · 2021-05-21T18:28:46.598Z · LW · GW

My name is Justin Bullock. I live in the Seattle area after 27 years in Georgia and 7 years in Texas. I have a PhD and Public Administration and Policy Analysis where I focused on decision making within complex, hierarchical, public programs. For example, in my dissertation I attempted to model how errors (measured as improper payments) are built into the US Unemployment Insurance Program. I spent time looking at how agents are motivated within these complex systems trying to develop general insights into how errors occur in these systems. Until about 2016, I was very much ignorant of the discussions around AI. I was introduced to the arguments around AGI and alignment through the work PR works of Sam Harris and Max Tegmark leading me eventually to the work of Nick Bostrom and Eliezer Yudkowsky. It's been a wild and exciting ride.

I currently have a tenured Associate Professor position at Texas A&M University that I'm resigning on July 1 to focus more on writing, creating, and learning without all of the weird pressures and incentives that come from working within a major public research university in the social sciences. In preparation for changing my employment status, I've been considering the communities I want to be in discussion with and the LessWrong and AlignmentForum communities are among the most interesting on that list. 

My writing is on decision making, agents, communication, governance and control of complex systems, and how AI and future AGI influences these things. I've been thinking about the issue of control of multi-agent systems a lot lately and what types of systems of control can be used to guide or build robust agent-agnostic processes of AI and human constitution. In agreement with George Dyson's recent arguments, I also worry that we have already lost meaningful human control over the internet. Finally, I've recently been significantly influenced by the works of Olaf Stapledon (Star Maker, Last and First Men, Sirius) and Aldous Huxley (The Perennial Philosophy) in thinking more carefully about the mind/body problem, the endowment of the cosmos, and the nature of reality.  

My hope is that I can learn from you all and bring to this conversation thoughts on alignment, control, governance (in particular of multi-agent systems that contain only humans, humans and AI, and only AI), and form a map together that better reflects the territory . I look forward to engaging with the community!