Can Kauffman's NK Boolean networks make humans swarm?

post by Yori-92 (yori-ong) · 2024-05-08T02:09:58.708Z · LW · GW · 2 comments

Contents

  Introduction
  A communication protocol on NK Boolean networks
    What does the protocol do?
    When is the protocol useful?
    Why would it be used?
  Strategies for service providers and organizers 
    Adapting toward the edge of chaos
  Ethical versus strategic decisions
None
2 comments

With this article, I intend to initiate a discussion with the community on a remarkable (thought) experiment and its implications. The experiment is to conceptualize Stuart Kauffman's NK Boolean networks as a digital social communication network, which introduces a thus far unrealized method for strategic information transmission. From this premise, I deduce that such a technology would enable people to 'swarm', i.e.: engage in self-organized collective behavior without central control. Its realization could result in a powerful tool for bringing about large-scale behavior change. The concept provides a tangible connection between network topology, common knowledge and cooperation, which can improve our understanding of the logic behind prosocial behavior and morality. It also presents us with the question of how the development of such a technology should be pursued and how the underlying ideas can be applied to the alignment of AI with human values. The intention behind sharing these ideas is to test whether they are correct, create common knowledge of unexplored possibilities, and to seek concrete opportunities to move forward. This article is a more freely written form of a paper I recently submitted to the arXiv, which can be found here.

Introduction

Random NK Boolean networks were first introduced by Stuart Kauffman in 1969 to model gene regulatory systems.[1] The model consists of N automata which are either switched ON (1) or OFF (0). The next state of each automaton is determined by a random boolean function that takes the current state of K other automata as input, resulting in a dynamic network underpinned by a semi-regular and directed graph. It can be applied to model gene regulation, in which the activation of some leads to the activation or suppression of others, but also to physical systems, in which a configuration of spins acting on another will determine whether it flips up or down.

NK Boolean networks evolve deterministically: each following state can be computed based on its preceding state. Since the total number of possible states of the network is finite (although potentially very large), the network must eventually return to a previously visited state, resulting in cyclic behavior. The possible instances of Boolean networks can be subdivided between an ordered and a chaotic regime, which is mainly determined by the number of inputs for each node, K. In the ordered regime, the behavior of the network eventually gets trapped in cycles (attractors) that are relatively short and few in number. When a network in the ordered phase is perturbed by an externally induced 'bit-flip', the network eventually returns to the same or slightly altered ordered behavior. If the connectivity K is increased beyond a certain critical threshold, the network's behavior transitions from ordered to chaotic. States of the network become part of many and long cycles and minute external perturbations can easily change the course of the network state's evolution to a different track. This is popularly called the 'butterfly effect'.

It has been extensively demonstrated that human behavior is not just determined by our 'own' decisions. Both offline and online social networks determine the input we receive, and causally influence the choices we make and opinions we adopt autonomously.[2] However, social networks are not regular, social ties are often reciprocal instead of directed and people are no automata. NK Boolean networks are therefore not very suitable for modeling an existing reality. What is nevertheless possible in the digital age, is to conceptualize and realize online communication networks based on its logic: just give N people a 'lightbulb application' on their smartphone that they can turn ON or OFF depending on the state of K lightbulbs of other people. What would the use of such a technology be? What meaning could we assign to the lightbulb? Can we expect the behavior of such a network to also exhibit a transition between order and chaos, and what would that even imply? 

On the one hand, ordered behavior is predictable. However, when predictable behavior leads to predictable problems, it indicates that the system is not sufficiently adaptive. On the other hand, chaos means that disproportionate responses to small triggers render the system unpredictable and unstable. The boundary between the two phases, or the 'edge of chaos', is generally considered the optimum at which complex adaptive systems (including life itself) sustain themselves by balancing both stability and evolvability.[3] The main hypothesis outlined by this article is that connecting human decisions via NK Boolean networks can remediate a lack of adaptability in social systems and drive them to the 'edge of chaos'. I will discuss in the following sections what a technology with this function would do and what it suggests about the logic behind motivation and cooperation. Additionally, I raise the question whether it is a good idea to create such a technology, how it should be done and who would be 'up for it'.

A communication protocol on NK Boolean networks

A necessary assumption to make for describing a functional communication protocol is that the protocol and availability of its infrastructure are common knowledge. to all potential users. The concept of common knowledge is pivotal in this discussion and refers to the state in which a proposition is known by all members in a group, all of them know that all know, all know that all know that all know, and so on ad infinitum. When the concept is rigorously approached, this state is near impossible to achieve in distributed systems. However, instead of an infinite composition of nested knowledge, common knowledge among people is most likely a distinct cognitive state at which we arrive through inference, which makes us strategize as if common knowledge were rigorous. This epistemic state would more accurately be called a sufficiently certain common p-belief (everyone believes that everyone believes, etc.), where p represents the Bayesian probability of the belief being true. In this article, the term common knowledge is understood to include this meaning as well, corresponding to Steven Pinker's approach to the concept.[4]

I refer to the protocol and infrastructure with Gridt, for which I will provide the etymology at the end of this section. Users are given the choice to connect to and participate in various networks, which are underpinned by a directed and semi-regular graph (NK network topology). The communication protocol is described as follows:

Figure 1. Example user interface (simplified from design by Nerds&Company) and Bob's extended network neighborhood in a network with K = 4. Funny people avatars were designed by Freepik.

What does the protocol do?

Clearly, the communication network does not support conversation among users, since directed links prohibit mutual exchange of information. However, it does support conditions which I claim to be crucial for self-organization and coordination.

First of all, the NK topology allows for the underpinning graph to evolve over time while keeping the observed neighborhoods of each node constant. A new participant can join at any time, adding a node to the network, without affecting the inputs of others. Similarly, active participants who choose to rewire their links can do so without having to 'break up' with someone. These decisions, which result in the self-organization of the network, are therefore made by fully autonomous individuals. Regardless of the size of the network, all participants receive K incoming links, while knowledge of their outgoing set is completely distributed over all participants. Since leaving the network could impact another user's input, removing a node is effectuated only upon resetting all inputs.

Due to the directedness of links, participants in the network have no common knowledge of any information that is transmitted between users. When Alice transmits information to Bob, Bob knows that Alice doesn't know that he knows, and he cannot communicate back to her what he knows. However, participants can infer common knowledge of how the protocol works and what the signal means, if this knowledge is verifiable and obtained by anyone joining the network. This is therefore the only basis of common knowledge that shapes their strategic environment, and provides the basis for N-player games on the network. Since communication among participants does not modify this basis, the structure of the game can also not be changed or disrupted by participants themselves. 'Free speech' can only accompany an affirming signal and would therefore become self-inconsistent if it contradicts the network's basis of common knowledge. The anonymous 'nudge' or 'pat-on-the-back' signals can only inform a participant that there is at least one active user receiving their signal, which is nice, but deliberately insufficient for establishing common knowledge between participants.

When is the protocol useful?

We can consider communication to be useful when transmitted information is influential. Influential information affects a receiver's belief of the state of the world and potentially changes their preferences for a decision. For example, when Alice asks Bob a question, she puts him in a position in which he has the choice to express anything ranging from nothing to a complete truth or lie. Subsequently, Bob's response (or lack thereof) updates Alice's belief of the state of the world. However, the utility of Alice's question to Bob hinges on them having common knowledge of the fact that Alice asked Bob the question.

The Gridt protocol prohibits any transmitted information from converting to common knowledge between sender and receiver. In exchange, it preserves common knowledge of the signal's meaning among the entire network. Under which conditions can we then consider this form of communication to be useful? The signal will update a receiver's beliefs, if it is expected to have some correlation to the actual state of the world. In this case, this state corresponds to another participant having chosen to indeed do the prescribed action or not. People who do not benefit from coordinating their actions with others, or prefer to anticoordinate, will not benefit from using the protocol, since disclosing their choices to others brings no strategic advantage. The utility of the protocol is therefore strongly biased toward supporting coordination in collective action games, i.e.: strategic games in which each player obtains the highest payoff if all cooperate, but cooperation is costly if only few choose to do so. By defining the action that is indicated by the signal, the strategic benefit of a Gridt network is defined for a set of potential users, who can form the network through self-organization. In a collective action game, the signal has credibility because its truthfulness would increase the payoff that both sender and receiver can expect from cooperating.

Costless and unverifiable information transmission that leaves the payoff structure of players unchanged is the definition of cheap talk. Cheap talking has been demonstrated theoretically and computationally to benefit pro-social, but not self-interested agents.[5] However, it is only meaningful to speak of cheap talk after the strategic environment has been sufficiently defined. Defining a game and assuming common knowledge thereof is the basis of a game theoretical model. However, this assumption becomes increasingly unrealistic in real life with large numbers of players who constantly and mutually exchange information. Knowing that any pair or subgroup of people has the opportunity to continuously modify an established basis of common knowledge among themselves means that the assumption can never hold on a large scale. The ubiquity of online and offline conversation networks breaks down the basis for this assumption even further. The Gridt protocol can fulfill a threefold function in creating a solid basis for common knowledge and coordination in a real-world setting. It defines the set of players through self-organization, establishes and preserves their basis of common knowledge and facilitates influential signal transmission, all of which are made possible by the NK topology.

Figure 2. Left: on networks for conversations and public discussion, information cannot be exchanged without the risk of also affecting the strategic context. This is the result of all communicated information converting to common knowledge between sender and receiver. Right: directed signal transmission via the Gridt protocol allows for unambiguity in the transmission of information, because the strategic context remains fixed. This is achieved preventing all transmitted information from converting to common knowledge, by keeping senders of signals uncertain about who their receivers are. People icons were designed by Freepik.

Why would it be used?

Purposefully transmitting information without converting it to common knowledge is a near impossible strategy to realize without digital technology. Normally, a sender chooses whom to target with their communication and knows when their message has been transmitted; the receiver typically knows who the sender is and can infer that they know that the message has come accross, leading to the inference of common knowledge. However, communicating while avoiding common knowledge of certain information is particularly useful for influencing the decisions of others. In daily life, this is done through ambiguous or 'indirect' speech.[6] For example, "would you like a mint?" is typically preferable over "I think you should do something about your breath", although they both intend to elicit a similar decision from the receiver. Common knowledge of a speaker's intention puts the receiver in the position to either ratify or reject their influence, potentially damaging their relation. Without a clear strategic context, Alice telling Bob "I've brushed my teeth this morning" is also not just an elementary proposition, since Bob also learns that Alice considers it necessary to tell him in particular. Such implicit information only becomes unimportant when the strategic context is sufficiently clear and remains unaffected by it. Conversations are therefore poorly suited for achieving large-scale coordination, since they can both modify the strategic context and carry unintended and counterproductive information. Signaling over directed links transmits unambiguous information and conserves the strategic context, but comes at the cost of certain knowledge that the signal has been observed. A realization of the Gridt protocol would allow humans to choose this strategy consciously in general situations.

Which part of connecting to a conversation-free network and signaling to unknown receivers is appealing to an individual? In other words, is it also fun or rewarding to do? To address this question, let's break down the experience of example user Bob (see figure 1). Bob has joined the 'Clean Teeth Movement', a network of people who coordinate with each other so that they remember to floss daily, in addition to brushing at least twice daily. 

Breaking down Bob's observations and inferences illustrates that the way he experiences his relation to others is largely determined by what he knows that others know and don't know, or in other words: where common knowledge can be inferred and where it is prevented. This factors into Bob's experience of autonomy in his decisions, being in a similar state of mind as others, and being able to trigger a certain experience in someone else. These experiences can be recognized as autonomy, relatedness and competence or empowerment in interactions with others. Self Determination Theory identifies these as a human's three basic psychological needs[7]. By reasoning inductively, I suggest we can formulate general conditions for Bob to satisfy these needs when interacting with another person, based on his representation of their mental state (theory of mind[8]). 

Psychologists have theorized and validated that securing these basic human needs is the basis for nurturing intrinsically motivated behavior. The Gridt protocol gives us a path toward operationalizing these psychological concepts with logical and computational definitions.[9] Computer scientists have already demonstrated substantial achievements in applying intrinsically motivated learning to artificial agents. In particular, introducing an intrinsic reward for agents proportional to their potential causal influence or 'transfer-empowerment' over other agents has been shown to result in improved cooperation. The Gridt protocol could provide a unique opportunity to validate the applicability of such computational definitions  to improving human cooperation.

Formulating the conditions to fulfill psychological needs in terms of strategic environments, mental representations and common knowledge reveals their direct relation to the topologial properties of the network through which people interact. Additionally, it illustrates the trade-off between them. For example, maximizing one's influence would come at the cost of their sense of relatedness and the autonomy of others. The semi-regular and directed topology of Kauffman's NK networks and the derived Gridt protocol does something quite interesting here: participation in the network contributes a little bit to all three basic needs, without violating any. I therefore hypothesize that, given the availability of a well-designed digital infrastructure and appropriate collective action networks, participation is also individually preferred over non-participation, with human intrinsic motivation as its driver. The technology could therefore offer a relatively cheap and highly scalable strategy for bringing about collective behavior change, driven by people's need for autonomy, relatedness, competence and minimizing cognitive dissonance (by acting consistently with the signals they send).

For Bob, who is learning to keep up with improved dental hygiene habits, the Gridt protocol fosters his intrinsic motivation for the task, since the long-term payoff of healthy teeth or improved public oral health cannot be directly experienced in relation to a particular action. This explains the etymology of 'Gridt': it is a blend of grid, a regular network or a network for distributing a resource, and grit, the personality trait to persevere in the pursuit of long-term goals.[10]

Strategies for service providers and organizers 

Other than the participants in a collective action network, there are at least two other 'player roles' that we cannot omit when conceptualizing a real-world implementation of the Gridt protocol. These are the entities who develop and provide the digital infrastructure and organize each network. For many people and organizations with the resources, creating the tools for self-organizing collective action is actually quite feasible. By doing so, these players can also consolidate their autonomy and power, and reap the payoffs of the collective actions they facilitate. Given the scalability and general applicability of the protocol, providing this option to even a fraction of 7 billion internet users could have far-reaching consequences. On an organizational and societal level, converting this thought experiment into reality presents us with a high-stakes game. Let us therefore contemplate its effective strategies.

So far, we focused on the participants in a network, in which their decision spaces are deliberately limited. On a higher level, the strategic environment looks like a coalition formation game[11] with a large decision space for all players. It is relevant to acknowledge some aspects of the Gridt protocol they have to work with:

Preserving and preventing common knowledge does not only shape the strategic environment within each network, but also the higher-level coalition formation game. Coalitions of service providers, organizers and participants are not held together by binding contracts: especially participants could easily jump ship as soon as better options present themselves. Just like each network, coalitions are primarily bound together by common knowledge of aligned interests between members, which must therefore be established and preserved. To optimize their own payoffs in this game, players must see to align their interests with the most valuable coalition. Generally speaking, this would be the coallition that mobilizes the greatest number of participants. Without tangible and verifiable evidence for broadly aligned interests, a coalition can easily become unstable. We can deduce that a transparent operation and an open codebase are the first requirements for operating this technology successfully. Common commercial practices, such as purposeless profit-seeking and hoarding personal user data, become weak strategies in this game. Alignment of interests in these areas can be made tangible with appropriate business ownership structures (steward-ownership[12]) and the use of data-autonomy affirming standards (e.g.: Solid Pods[13]).

Preventing an individual's activity from becoming common knowledge between players also applies to organizers and service providers. Hoarding these data by default would infringe on the autonomy that the network is meant to foster. These data also would not have the same utility as they have on other online platforms, where they inform modifications to choice architectures to influence individuals. On a Gridt network, users choose to put themselves in a prespecified choice environment that gives them influence. Strategic information transmission from organizers to users would acknowledges their autonomy and stimulates them to join networks whose collective interest they share. This will of course be easiest if an organizer's interest is factually aligned with a large collective interest.

While individual activity is kept private or shared (but not common) knowledge among participants, precise knowledge of collective activity and network topology can only be observed by monitoring the network infrastructure. Service developers and providers have the tactical options to hide or reveal this knowledge, and even to keep it distributed. Participants obtain too little information from their network neighborhoods, and cannot extrapolate them meaningfully to learn what happens on the scale of the network. They and others can only go by the information that is disclosed, the observable impact of collective action and their beliefs of the actions of others to determine their strategy. Given the large numbers of potential players, decisions and high stakes, it is reasonable to suspect that the dynamics of coalition formation could be turbulent.

Adapting toward the edge of chaos

Service providers and network organizers cannot directly control the decisions of individual participants, but do control parameters that shape their strategic environments and affect the networks' collective behavior. Their choices will influence how a network's size and structure evolves over time, when activity spikes or whether participants have an incentive to signal deceptively. One of the most influential parameters to control is the connectivity parameter K of networks. 

The connectivity parameter K divides Kauffman's original NK boolean network model in an ordered (or 'frozen') and chaotic regime. The value at which this transition occurs depends on the bias of each node's Boolean function toward either 1 or 0. Although there are several ways in which the Gridt protocol differs from Kauffman's model, abrupt changes in collective behavior depending on connectivity can still be expected. What evidence is there to back this up? And what is the reason for choosing K = 4 in the illustrated example? To obtain the greatest utility of the protocol, we would like to know the network properties that give each individual signal the greatest influence on its receiver, potentially triggering a cascade of activity.

We can expect an individual signal to be the most influential if it causes the greatest shift in an observer's perception of the 'norm' in the network. The connectivity parameter K must be sufficiently large for the observation to be a meaningful sample of the entire network. Increasing K also decreases the probability that signals are sent with nobody to receive them. However, if K is too large, changing the state of a single signal will only marginally impact on an observer's perception of the norm. With a computational approach, it can be argued that trading off the potential causal influence of a signal against the probability of there being no observer results in a ballpark estimate of 3 < K < 6.[14] Experimental evidence has indicated that in order to shift behavioral norms in a group, the critical fraction of individuals committing to a new norm lies around 25%.[15] Once this tipping point has been reached, the new norm can be expected to take over in the network. Another piece of evidence is provided by theoretical and experimental work on small groups, which have shown that success rates in coordination drop sharply once a group's size is increased beyond six members. Such abrupt changes in a group's behavior are indicative of phase transitions. The transition between an ordered and chaotic phase is common to Boolean networks, and also does not require that the network has a directed or semi-regular topology. We can thus expect these two regimes to also exist in the dynamics of Gridt networks and I estimate the critical value of K to be around K = 4, so that each individual signal corresponds to 25% of an oberver's input.

We are neither interested in the ordered or frozen phase, in which disturbances die out and collective behavior remains stuck in repeating patterns, nor are we interested in fully unpredictable chaos. The edge of chaos is where we would like the system to be, where it behaves as a coherent, but adaptive whole. Tuning toward the edge involves more than making the best guess for the connectivity parameter K. The distribution of outgoing links, conditions for resetting activated signals and announcements to participants are other factors that influence how the system behaves. Keeping coalitions alive and active would likely be a continuous process of feedback, adaptation and evolution, with service providers and organizers having the crucial power to regulate their behavior.

Ethical versus strategic decisions

Unless the presented concept is a complete dud, which I believe it not to be, it is now time to ask the question: is it a good idea to do this? In the current context, this is somehwat of a trick question, since we rationalized human decision making down to its suspected fundamental human intrinsic motivators and which did not involve questions about good or bad. Nevertheless, the discussion is relevant, since new strategies for information transmission can make a great impact in a world where 7 billion people use online mobile devices. In this final section, I share some of my thoughts on these and other questions as seeds for what will hopefully be a rich discussion.

Can the Gridt protocol be used for good things only or also for bad things? 

Without downplaying the importance of dental hygiene[16], the example of collectively brushing and flossing clearly serves as a placeholder. The protocol has utility for any conceivable action that produces an increased payoff, or externality, when taken collectively. In the current state of the world, applications for climate action, public health and education easily come to mind as cases for which the protocol could help initiate and sustain collective behavior. However, other forms of collective behavior with more short-term horizons could also be supported with the same infrastructure. Self-organized acts of protest, influencing economic decisions and diffusion of knowledge and opinions are examples of use cases that would benefit some but surely not all. Commonly knowing that these processes are taking place below the radar of anyone's common knowledge could have an interesting impact on our collective conscience. However, it is relevant to remember that for players to benefit from this 'cheap talking' protocol, signals must be sufficiently trustworthy, which is only the case if players' interests align. Incentives to anticoordinate with the collective or extrinsic motivators to signal promote deceptive signaling and nullify the effect of the protocol. Behaviors that are intended to provoke, or which are incentivized by common knowledge of an external reward or threat are therefore not effectively supported by the network.

It is tempting to claim that promoting actions driven by autonomy, empowerment and connectedness are 'good' and that obstructing them is 'bad'. The utility of the protocol for self-organizing cooperation is strongly biased toward collective actions we can usually defend as ethical. However, it would be fitting to the current context to recognize that notions of morality seem to emerge from more fundamental physical and logical concepts, including the topology of information transmission networks, inferred common knowledge and intrinsic reward mechanisms. A more rational way of judging the Gridt's potential impact would be to say that new coalition formation games and collective action games bring new ways for some players to win and for others to lose.

Who is responsible for the impact (good or bad) of collective action? 

The concept of responsibility is highly relevant in ethics, politics and law. However, its application to collective action problems and collective behavior results in a category error. Responsibility can be defined for individual actors or entities with a clear organizational structure (e.g. businesses), but the term is simply not suitable for evolving networks, even though they are comprised of the same elements. In a similar way, sand can be described as being fine or coarse, but a boulder cannot; ice can be hard, but liquid water cannot. Such category errors also occur when blaming a company or president for collective action problems: they are emotional reflexes that remain without consequences, because a property of 'being responsible' is fundamentally misattributed.

Online collective action has previously resulted in significant societal impact, for which attributing responsibility to a person or group is simply not feasible. Does a dedicated communication infrastructure for self-organization make this any different? I claim that this is debatable and paradoxical, because the notion of responsibility is closely related to having common knowledge. We can hold someone responsible for consequences of their action if we know that they know what they caused, and if they know that we know that they know, etc. If Alice's signal is transmitted to Bob and this motivates him to floss, Alice is not responsible for making Bob floss, because she cannot know her signal ended up with him. However, the function of the Gridt protocol is to ensure that it is common knowledge that Alice cannot know she reached Bob, which is what conserves Bob's sense of autonomy. Alice knows that common knowledge of her ignorance is exactly why her signal has its potential influence, which contributes to her feeling empowered. Since Bob's observable neighborhood in the network is translationally equivalent to Alice's, any responsibility she has, Bob has as well. Distributing knowledge, also among organizers and service providers, is therefore a way to both secure the power of the protocol, but also to uproot the concept of responsibility. Depending on the scale, purpose and consequences of self-organized collective actions, this could lead to foreseeable challenges in, for example, applying legislation.

What is the relevance for AI alignment?

Artificial intelligence is built by human intelligence. Aligning artificial intelligence with human values will remain a tough job if we do not know how to align human intelligence with human values: one problem cannot be solved without solving the other. The Gridt protocol gives us an important clue as to how this might be achieved, as well as something to try out. By conceptualizing an 'atomic change' to the structure of communication networks (from undirected social networks to NK boolean networks), we deduced that cooperation games would emerge in which strategically preferred decisions coincide with ethical decisions. The theoretical frame in which we have reasoned about common knowledge, models of other agents (theory of mind) and intrinsic motivation is applicable to both humans and machines, which helps the transferability to AI alignment.

During the years that I have worked to get this ball rolling as an aspiring entrepreneur, I learned that merely shared knowledge of a potentially powerful idea is not sufficient to impact other people's decisions. Luckily, I now recognize this as being fully consistent with the logic of the concept itself. For ideas to change other people's strategies, they must be commonly known, or 'out there'. However, the reverse should also logically hold: common knowledge of 'payoff relevant' information must cause changing strategies. This first publication of the Gridt concept to the LessWrong community is therefore part of a validation experiment. What does the community think? Let's discuss!

  1. ^

    Some literature on the behavior of Boolean networks and their transition from order to chaos:

    Kauffman, S. A. (1969). Metabolic stability and epigenesis in randomly constructed genetic nets. Journal of Theoretical Biology, 22(3), 437–467. https://doi.org/10.1016/0022-5193(69)90015-0 

    Kauffman, S. A. (1984). Emergent properties in random complex automata. Physica D: Nonlinear Phenomena, 10(1–2), 145–156. https://doi.org/10.1016/0167-2789(84)90257-4 

    Derrida, B., & Pomeau, Y. (1986). Random Networks of Automata : A Simple Annealed Approximation. EPL-Europhysics Letters, 1(2), 45–49. https://doi.org/10.1209/0295-5075/1/2/001ï 

    Fronczak, P., Fronczak, A., & Hołyst, J. A. (2008). Kauffman Boolean model in undirected scale-free networks. Physical Review E - Statistical, Nonlinear, and Soft Matter Physics, 77(3). https://doi.org/10.1103/PhysRevE.77.036119 

  2. ^

    Examples of evidence for the spread of behavior as a 'contagion' through online and offline networks:

    Centola, D. (2010). The spread of behavior in an online social network experiment. Science, 329(5996), 1194–1197. https://doi.org/10.1126/science.1185231

    Fowler, J. H., & Christakis, N. A. (2008). Dynamic spread of happiness in a large social network: longitudinal analysis over 20 years in the Framingham Heart Study. BMJ, 337(dec04 2), a2338–a2338. https://doi.org/10.1136/bmj.a2338 

  3. ^

    Discussion on adaptation to the 'edge of chaos':

    Teuscher, C. (2022). Revisiting the edge of chaos: Again? Biosystems, 218, 104693. https://doi.org/10.1016/j.biosystems.2022.104693 

  4. ^

    Psychological research into the role of common knowledge in coordination has been mostly contributed to by Steven Pinker and coworkers:

    de Freitas, J., Thomas, K., DeScioli, P., & Pinker, S. (2019). Common knowledge, coordination, and strategic mentalizing in human social life. Proceedings of the National Academy of Sciences of the United States of America, 116(28), 13751–13758. https://doi.org/10.1073/pnas.1905518116 

    Thomas, K. A., DeScioli, P., Haque, O. S., & Pinker, S. (2014). The psychology of coordination and common knowledge. Journal of Personality and Social Psychology, 107(4), 657–676. https://doi.org/10.1037/a0037037 

  5. ^

    Theory and computational demonstrations of the utility of cheap talking for coordination:

    Crawford, V. P., & Sobel, J. (1982). Strategic Information Transmission. Econometrica, 50(6), 1431. https://doi.org/10.2307/1913390 

    Farrell, J., & Rabin, M. (1996). Cheap Talk. Journal of Economic Perspectives, 10(3), 103–118. https://doi.org/10.1257/jep.10.3.103 

  6. ^

    Steven Pinker's work on indirect speech:

    Pinker, S., Nowak, M. A., & Lee, J. J. (2008). The logic of indirect speech. Proceedings of the National Academy of Sciences, 105(3), 833–838. https://doi.org/10.1073/pnas.0707192105 

  7. ^

    Built largely upon the work of Edward L. Deci and Richard Ryan, the breadth and depth of Self Determination Theory can be found here:

    The Theory – selfdeterminationtheory.org 

  8. ^

    Exploring the limits of human recursive mentalization:

    Kinderman, P., Dunbar, R., & Bentall, R. P. (1998). Theory-of-mind deficits and causal attributions. British Journal of Psychology, 89(2), 191–204. https://doi.org/10.1111/j.2044-8295.1998.tb02680.x 

  9. ^

    Examples of achievements in CS and AI research on the topics of intrinsic motivation and coordination:

    Oudeyer, P. Y., & Kaplan, F. (2009). What is intrinsic motivation? A typology of computational approaches. Frontiers in Neurorobotics, 3(NOV). https://doi.org/10.3389/neuro.12.006.2007 

    Schmidhuber, J. (2010). Formal theory of creativity, fun, and intrinsic motivation (1990-2010). IEEE Transactions on Autonomous Mental Development, 2(3), 230–247. https://doi.org/10.1109/TAMD.2010.2056368 

    Jaques, N., Lazaridou, A., Hughes, E., Gulcehre, C., Ortega, P. A., Strouse, D., Leibo, J. Z., & de Freitas, N. (2018). Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning. http://arxiv.org/abs/1810.08647 

  10. ^

    Research into grit, the character trait for individual 'longtermism', is spearheaded by Angela Duckworth:

    Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and Passion for Long-Term Goals. Journal of Personality and Social Psychology, 92(6), 1087–1101. https://doi.org/10.1037/0022-3514.92.6.1087 

  11. ^

    This is the realm of cooperative game theory. Lots of interesting references an information can be found on:

    http://www.coalitiontheory.net/ 

  12. ^

    The European epicentre of the movement toward responsible business ownership models is spearheaded by the Purpose Foundation:

    Steward-Ownership. For an economy fit for the 21st century. - Purpose (purpose-economy.org)

  13. ^

    "Solid is a specification that lets individuals and groups store their data securely in decentralized data stores called Pods". From their website at:

    Home · Solid (solidproject.org)

  14. ^

    For details, see the arXiv paper: https://arxiv.org/abs/2404.16240 

  15. ^

    Centola, D., Becker, J., Brackbill, D., & Baronchelli, A. (2018). Experimental evidence for tipping points in social convention. Science, 360(6393), 1116–1119. https://doi.org/10.1126/science.aas8827 

  16. ^

    Seriously, people, it is important. Educate yourselves and remember to brush and floss. Check for example:

    Oral health (who.int)

2 comments

Comments sorted by top scores.

comment by faul_sname · 2024-05-08T02:54:08.357Z · LW(p) · GW(p)

What does the community think? Let's discuss!

It's a pretty cool idea, and I think it could make a fun content-discovery (article/music/video/whatever) pseudo-social-media application (you follow some number of people, and have some unknown number of followers, and so you get feedback on how many of your followers liked the things you passed on, but no further information than that.

I don't know whether I'd say it's super alignment-relevant, but also this isn't the Alignment Forum and people are allowed to have interests that are not AI alignment, and even to share those interests.

Replies from: yori-ong
↑ comment by Yori-92 (yori-ong) · 2024-05-08T12:07:04.548Z · LW(p) · GW(p)

Nice! I actually had this as a loose idea in the back of my mind for a while, to have a network of people connected like this and have them signal to each other their track of the day, which could be actual fun. It is a feasible use case as well. The underlying reasoning is also that (at least for me) I would be more open to adopt an idea from a person with whom you feel a shared sense of collectivity, instead of an algorithm that thinks it knows me. Intrinsically, I want such an algorithm to be wrong, for the sake of my own autonomy :)

The way I see it, the relevance for alignment is to ask: what do we actually mean when saying that two intelligent agents are aligned? Are you and I aligned if we would make the same decision in a trolley problem? Or if we motivate our decisions in the same way? Or if we just don't kill each other? None of these are meaningful indicators of two people being aligned, let alone humans and AI. And with unreliable indicators, will we ever succeed in solving the issue? I'd say two agents are aligned when one agent's most rewarding decision results in a benefit of the other as well. Generalizing and scaling that alignment to many situations and many agents/people necessitates a 'theory of mind' mechanism, as well as a way to keep certain properties invariant under scaling and translation in complex networks. This is really a physicist's way of thinking about the problem and I am just slowly getting into the language that others in the AI/alignment fields use.