Why Large Bureaucratic Organizations?

post by johnswentworth · 2024-08-27T18:30:07.422Z · LW · GW · 52 comments

Contents

  Background: Dominance-Status
    Empirical Ontology Justification: Dominance-Status Is A Thing
    Some Key Pieces Of How Dominance-Status Works
  The Unconscious Economics of Managers
  Solve For The Equilibrium
    Budget Is The Limiting Factor
    Internal Alignment
    Increasing Managers’ Dominance-Status Is The Real Job
  Why Doesn’t Economic Selection Pressure Win?
  Summary
None
52 comments

Large bureaucratic organizations have long seemed… strange to me, on an intuitive level. Why do they exist? Like, in a world where the median person is John Wentworth (“Wentworld”), I’m pretty sure there just aren’t large organizations of the sort our world has. Nobody would ever build such an organization, because they’re so obviously wildly inefficient and misaligned. And even if somebody tried, everyone would demand prohibitively high prices to work either for the large organization or with it, since it’s just so deeply unpleasant to interface with. Nobody would buy anything sold by such an organization, or vote for such an organization to continue to exist, because the organization as an entity is so obviously both incompetent and untrustworthy. So how on Earth (as opposed to Wentworld) are large organizations stable?

The economists have some theorizing on the topic (google “theory of the firm”), but none of it makes me feel much less confused about the sort of large organizations I actually see in our world. The large organizations we see are clearly not even remotely economically efficient; for instance, they’re notoriously full of “bullshit jobs” which do not add to the bottom line, and it’s not like it’s particularly difficult to identify the bullshit jobs either. How is that a stable economic equilibrium?!?

In this post I’ll present a model which attempts to answer that ball of confusion. The summary is:

• “Status”, in the sense of a one-dimensional dominance hierarchy, is A Thing. We’ll call it dominance-status to make it clear that we’re not talking about some other kind of status.
  • The way dominance-status normally works in higher animals, newcomers to a group generally enter near the bottom of the hierarchy (even if they were previously high-status in some other group). Within a group, dominance-status is mostly reasonably stable.
  • So, one of the main ways group members can move “up” in dominance-status (i.e. get more members “below” them) without a risky fight, is simply to add more members to the group.
• Managers at large organizations are mostly motivated by dominance-status.
  • So, the main thing for which managers get de-facto social/cognitive positive reinforcement is increasing their dominance-status and/or avoiding decreases in their dominance-status.
• Then, the natural prediction is that those managers (at all levels) will tend to add as many people as possible to the hierarchy under them, and minimize firing people, since that’s what maximizes their dominance-status.
  • … buuuut the drive to expand the hierarchy is limited by the organization's budget. So in practice, organizations will tend to expand until all the profit is eaten up (in the case of for-profit organizations) or until all the allocated budget is eaten up. And then the hungry managers will fight for more budget.
  • Much of what looks like organizational “inefficiency” and “misalignment” from an standard economic efficiency perspective looks like well-aligned dominance-maximization.
• … so e.g. large companies or government agencies are basically runaway human monuments of dominance and submission, limited mainly by their budget.

There’s a lot of steps here, and I’m not super-confident in this model. But when I step into the model, large organizations no longer look strange and confusing; the model seems to generate a remarkably good description of most real large organizations, both private and public.

Now let’s walk through the model in more detail, starting with relevant background studies.

Background: Dominance-Status

Empirical Ontology Justification: Dominance-Status Is A Thing

“Status” typically connotes a mental model in which we could assign everyone a number/rank, and then some kind of behavior involving any two creatures is supposed to be well predicted by whose number/rank is greater. In particular, for dominance-status (beyond just humans), the number/rank is supposed to predict which of a pair tends to aggress and which tends to back down, between the two of them. That’s a substantive model which makes empirically testable predictions: there are possible patterns of dominance/submission which cannot be well predicted by assigning each creature a number/rank and then comparing numbers, no matter what ranking we use.

Suppose, for instance, that we have three chickens: Audrey Henburn, Beakoncé, and Chickira^[1]. We put two of them at a time in a cage with only one feed bowl. We find that Audrey chases Beakoncé away from the bowl, Beakoncé chases away Chickira, and Chickira chases away Audrey. We can represent this graphically, with an arrow B -> A indicating that A “wins against” B:

Notice that the graph contains a cycle! That means these dominance relationships cannot be represented by a ranking. Why? Well, the ranking would need to satisfy:

• rank(Audrey) > rank(Beakoncé)
• rank(Beakoncé) > rank(Chickira)
• rank(Chickira) > rank(Audrey)

… but put those all together and we get rank(Audrey) > rank(Audrey), which is not how numbers work. (And if we instead replace “>” with “≥”, we find that all three must have the same rank, which would make the model useless.)

On the other hand, if the graph looks like this:

… then we can represent the relationships with a ranking: rank(Audrey) > rank(Beakoncé) > rank(Chickira). All of the arrows in the graph go from the lower-ranking hen to the higher-ranking hen.

More generally: we can represent dominance relationships with a ranking exactly when the graph is acyclic.^[2] Of course in practice we expect the occasional violation of the dominance ranking, but that’s fine, approximation is a thing we can account for. So: we can empirically test whether the concept of a “status ranking” makes sense at all, as an ontological choice, by checking whether the dominance interaction graph contains statistically very few cycles.

… and it turns out the behavior researchers have done exactly that kind of test; they call it testing for “linearity”. Indeed, they’ve done it many many times over, with several different operationalizations of the statistics, in a whole slew of species. The paper I was reading which prompted this post - “Dynamics of Hierarchy Formation: The Sequential Development of Dominance Relationships” - looked at hens, basically as described above. They only used ten groups of chickens, but all groups developed a linear hierarchy after a day or two of interaction. The paper also cites a long list of similar work in other species: “wasps and bumble bees (WILSON, 1971), chaffinches (MARLER, 1955), red cross bills (TORDOFF, 1954), coyotes (BEKOFF, 1976), cows (SCHEIN & FOHRMAN, 1955), ponies (TYLER, 1972), pigs (RASMUSSEN et al, 1962), rhesus monkeys (SALE, 1967), baboons (HAUSFATER, 1975), vervets (STRUHSAKER, 1967) and human children and adolescents (MISSAKIAN, 1976; SAVIN-WILLIAMS, 1977, 1979, 1980)”. Note that last one: humans too.

My main tentative takeaway is that the justification for a “dominance status ranking” as an ontological choice is extraordinarily strong compared to the usual standards of social science. This looks like^[3] one of the best-justified ontological choices I’ve encountered in academic research, and it might be a good textbook example of the right way to “choose the ontology [LW · GW]”. Dominance-status as a ranking is real.

Some Key Pieces Of How Dominance-Status Works

Having established that we’re talking about a natural Thing at all, let’s talk about how it works. How does the dominance hierarchy form, and what determines the ordering?

Here’s the key quotes which got me thinking (from the same paper as the previous section):

… they formed two groups of male rhesus monkeys, let each group establish a dominance hierarchy, and then serially added the members of group 1 to group 2 at weekly intervals in reverse order of dominance in their home group (group 1). [...] First, there was no correlation (Spearman coefficient = -.05) between the original hierarchical ranks of group 1 males and their relative ranks within the merged hierarchy (BERNSTEIN & GORDON, 1980: 1036). Second, all the members of group 1 ranked below five out of six of the group 2 members, and only two out of 11 members of group 1 ranked above the previously lowest ranked member of group 2.

And, in a different experiment:

After the adult males formed a hierarchy in their respective groups, the alpha and beta animals of each group were introduced to the other's cage for short periods. [...] the relative ranks of the two pairs of alphas and betas depended upon the cage in which a visit took place. The alpha and beta of group 1 were at the bottom of the resident hierarchy when they visited in group 2's cage and vice versa.

So in rhesus monkeys, new members join the group at-or-near the bottom of the dominance-status ladder. Likewise when “visiting” other groups: the “visitor” enters at-or-near the bottom of the dominance-status ladder.

Once pointed out, that also sounds like how human status tends to work! The new hire at the company, the new kid at school, the new member to the social group, the visitor at another’s house… all these people typically have very low dominance-status, at least within their new context.

So hypothetically, if you’re looking to “status-hack” - e.g. secure a high ranking in a dominance-hierarchy without actually having the chin of gigachad - one strategy is to invite people “into your own territory” a few at a time, and slowly add people to whatever group(s) you’re in.

… ok, but how do you draw new people into your dominance hierarchy? Obvious answer: pay them money. We’ll come back to that shortly.

The Unconscious Economics of Managers

If you ask managers at large organizations what their main goals are in their work, presumably they will not say “grow the organization under me in order to establish dominance over as many people as possible”. Even if you could read off the thoughts of such managers, you’d probably see that they don’t think of building a dominance hierarchy under themselves as a major goal.

But I suspect that if you looked at the unconscious incentives involved, the things which make managers feel good or bad, dominance-status would play a much more central role. This is the domain of unconscious economics [LW · GW]: the “goals” which humans or organizations act like they’re pursuing tend to come less from explicit planning, and more from subconscious positive/negative reinforcement of behaviors and selection effects on people in various roles.

The idea that managers are mostly subconsciously motivated by dominance-status is an old one. For instance, from psychology, here’s Wikipedia on “Need Theory”^[4]:

Need theory, also known as Three needs theory [...] is a motivational model that attempts to explain how the needs for achievement, affiliation, and power affect the actions of people from a managerial context.

[...]

People who have a need for power prefer to work and place a high value on discipline. [...] A person motivated by this need enjoys status recognition, winning arguments, competition, and influencing others. With this motivational type comes a need for personal prestige, and a constant need for a better personal status.

[...] subsequent research, published in the 1977 Harvard Business Review article "Power is the Great Motivator", found that those in top management positions had a high need for power and a low need for affiliation.

From a different direction, here’s Investopedia on “Empire Building”:

Empire building is the act of attempting to increase the size and scope of an individual or organization's power and influence.

In the corporate world, this is seen at the intra-company level when managers or executives are more concerned with expanding their business units, their staffing levels, and the dollar value of assets under their control than they are with developing and implementing ways to benefit shareholders.

Yup, that all sounds like managers optimizing mostly for dominance-status.

Solve For The Equilibrium

We now have the basic recipe:

• Managers are mostly driven, motivationally, toward dominance-status.
• One of the simplest and most reliable ways to increase dominance-status is to add more people to your group, since new people generally enter at-or-near the bottom of the dominance hierarchy.
• … and the standard way to get people to do something (in this case, join a group) is to pay them.

… so the obvious prediction here is that managers will spend whatever budget they control to hire as many people as possible to work under them.

Now let’s go through various predictions this model makes, as well as some otherwise-confusing features of the world which make sense under this model.

Budget Is The Limiting Factor

In the case of for-profit companies, the model predicts that companies will expand headcount until the budget runs out. It’s not that companies hire to make more money, it’s that they make money in order to hire. Thus places like e.g. Google, which IIRC has like 30k technical employees, the majority of which don’t work on any of Google’s main cash cow products.

In the case of government organizations, the model predicts that e.g. departments will ~always spend all their budget, and in-practice the job of an agency head is mostly to secure more funding, not to make the agency cost-effective. I’ve heard that’s indeed how it typically works.

Within-org, the model predicts that middle managers’ battle for budget is the highest-stakes part of their job. We should expect much of managers’ job to center around making their department’s work look very difficult and important, so that lots of budget needs to be assigned to it. Note that actually solving problems, permanently, is highly detrimental to this goal. Solutions which require lots of ongoing maintenance are “better”, for purposes of chasing dominance-status.

Note that this model moderately-strongly predicts the existence of tiny hyperprofitable orgs - places founded by someone who wasn’t that driven by dominance-status and managed to make a scalable product without building a dominance-status-seeking management hierarchy. Think Instagram, which IIRC had 13 employees when Facebook acquired it for $1B. Or Berkshire Hathaway, which as a holding company owns subsidiaries with hundreds of thousands of employees, but IIUC only has 20-30 direct employees in the core business.

Internal Alignment

Under this model, how well are lower managers’ incentives aligned to upper management?

Recursion is good for status at the top: if e.g. your underling has underlings, then your underling has high status, and your dominance over them implies you have even higher status. Recurse over a very large organization, and CEOs of big orgs have completely ridiculous wirehead-level status. So lower managers' incentives are actually pretty well aligned with upper managers' incentives: lower management grows their status (by hiring), which grows their superiors’ status.

One notable implication: insofar as “the goal” is dominance-status for upper management, it makes sense for pay to scale with status rather than value-add. What the boss actually “wants” is dominance over the highest-status underlings they can get. (And for purposes of choosing underlings, the boss isn’t just interested in their underlings’ dominance-status, but also other forms of underling-status; e.g. high prestige also does the trick.) Paying more for status than for object-level competence makes sense, when dominance-status is what the org is actually optimizing for.

So in some ways, internal incentives are actually quite well aligned: lower-level managers are incentivized to increase their own status by hiring as many of the highest-status people as they can, and that’s exactly what increases higher-level management’s status.

On the other hand, below the level of upper management, middle managers also battle over shares of a limited budget. That part of their incentives is not so aligned with upper management; middle managers are incentivized to grab more budget even in ways which e.g. reduce the revenue of their company (in the private case), or decrease the budget available to other parts of the government where it might be “better” spent to more efficiently buy status (in the public case).

Increasing Managers’ Dominance-Status Is The Real Job

At the start of this post we mentioned “bullshit jobs” as a major piece of evidence that standard “theory of the firm” models of organization size don’t really seem to capture reality. What does the dominance-status model have to say about bullshit jobs?

Well, in some sense, increasing managers’ dominance status is the actual main job for most of the org.

You know how there’s a weird resistance to paying people more than their managers, even when the market value of the manager is clearly lower? That’s not a bug, that’s a feature of an organization whose managers optimize first for dominance-status, and only secondarily for making money.

You know how big orgs are notorious for policies or physical space layouts or norms which are kinda mildly demeaning toward low-level employees in ways that don’t actually make any money for the org? That’s not a bug, that’s a feature of an organization whose managers optimize first for dominance-status, and only secondarily for making money.

You know how actually pointing out the wild economic inefficiencies of a large org is itself often seen as an aggressive and impolite move, within the org’s culture? That’s not a bug, that’s a feature of an organization whose managers optimize first for dominance-status, and only secondarily for making money.

You know how the org structure is always hierarchical for some reason? And managers end up being massive communication-bottlenecks across the org, because horizontal coordination usually has to route through them? That’s not a bug, that’s a feature of an organization whose managers optimize first for dominance-status, and only secondarily for making money.

Why Doesn’t Economic Selection Pressure Win?

We started this post by asking how on Earth large organizations are stable. We’ve partially answered that question: managers are mostly motivated by dominance-status, and the bureaucratic hierarchy is shaped to satisfy that motivation. But to answer the question and wrap up the post, we still need to address one last piece: if large bureaucracies are so “wasteful”, if they’re pouring all their economic profits into building the biggest dominance hierarchy rather than e.g. making money for shareholders (in the private case) or satisfying voters (in the public case)... then why don’t these big bureaucracies die to competition?

When I look at real-world large organizations, at first glance it seems like there’s multiple answers. In some cases, economic selection pressure does win to a significant extent - e.g. industries like restaurants or game development or high-frequency trading have a relatively high proportion of relatively-small companies, and even the bigger companies seem less completely consumed by dominance optimization. In other cases, the big organization sits on a natural monopoly, like Facebook or telecom firms. Then there are government organizations, where voting systems basically fail to provide a strong enough counter-incentive to prevent the dominance-status instincts of politicians and managers from taking over. Then there are industries which are de-facto mostly about salespeople convincing customers to buy overpriced products, like car dealerships or most Wall Street firms, and for some such products the dominance-hierarchy itself seems to operate as a kind of sales tool - a thing to show off to prospective clients. Then there are industries where performance evaluation is very poorly coupled to purchase decisions, like education and healthcare.

So at first glance, there are many reasons why economic selection doesn’t kill Big Bureaucracy. But at second glance, I think there’s a unifying theme which mostly accounts for most of these examples.

Model: there’s a substantial chunk of the population which is motivated mainly by dominance-status. When such people find themselves with an economic surplus, they spend it on building the biggest dominance-hierarchy they can afford. And there’s lots of different ways someone can end up in control of an economic surplus - natural monopoly, government, information asymmetry (a.k.a. salespeople lying their asses off), etc. But big dominance hierarchies end up being a major convergent theme under all these different economic inefficiencies, because dominance-status is a thing which lots of people want to spend their economic surpluses on.

Furthermore, while different people are predominantly motivated by different things, dominance-status seems to be the most common motivator which drives people to optimize really hard. For instance, if we take Need Theory at face value (which, reminder, I don’t necessarily endorse), then the most common motivators cluster under “power”, “achievement”, and “association” - a.k.a. dominance-status, solving hard challenges, and socializing. Notably, unlike dominance-status, “achievement” and “association” do not particularly drive people to build or control big social systems, or to grab economic surpluses. Sure, securing an economic surplus is sometimes part of an interesting challenge, and it can presumably get one invited to lots of cool parties, but controlling surplus is typically not as central and necessary to “achievement” and “association” as to “power”. It’s not much surprise, then, that the sorts of people who actively seek out jobs in management, or run for high-ranking public offices, are the sort who are mostly motivated by dominance-status. And even if I don’t endorse Need Theory in general, the general idea that dominance-status is the main motivator which specifically drives people to grab lots of economic surpluses… seems pretty plausible.

Summary

Let's recap. Why large bureaucratic organizations?

• Dominance-status is A Thing
• One of the easiest ways to grow one’s dominance-status is to bring new members into one’s group, because new members tend to enter at the bottom of the hierarchy (even across species!)
• … and one of the easiest ways to bring new members into one’s group is to pay them.
• High dominance-status is the main motivator of most managers, and in general it’s the main motivator which specifically drives people to grab lots of economic surpluses.
• When dominance-status-driven people get their hands on an economic surplus, they tend to spend it on building the biggest dominance-hierarchy they can afford, by paying people to join the organization.
• … and that’s how large bureaucratic organizations happen. The apparent “inefficiencies” of large organizations are largely not inefficiencies at all, they’re just the organization being optimized for dominance-status, rather than profits (in the private case) or delivering value to voters (in the public case) or whatever else the organization’s nominal objective might be.

1. ^{^}

   Thankyou to Claude for these, um, excellent hen names.

2. ^{^}

   Proof: if acyclic, toposort the graph, ranking is the toposort order. If cyclic, find a cycle, and reuse the previous argument for a three-chicken cycle to prove that there is no ranking which represents the relationships.

3. ^{^}

   note that I have not done a deep lit review, so this is a surface level “looks like”

4. ^{^}

   Note that I don’t vouch for the empirical correctness or generalizability of Need Theory research. I’m citing it mainly to emphasize that this is an old idea, so the evidential bit-cost to at least privilege the hypothesis [LW · GW] has long since been paid multiple times over.

52 comments

Comments sorted by top scores.