The Succession Problem

post by Samo Burja · 2018-08-03T17:30:02.538Z · score: 27 (16 votes) · LW · GW · 5 comments

Contents

  Institutional longevity is a matter of skill and power
  Creative destruction is not a necessity for innovation
  Conclusion
None
5 comments
Tim Cook and Steve Jobs

[Crossposted from my Medium blog]

Only a few institutions fulfill their intended purpose. Such a functional institution stands out as remarkable. It is the exception rather than the rule, always tracing its beginnings to a founder. Such institutions at first always have a skilled pilot — he can alter and direct the institution in a way that preserves or improves its functionality. If he weren’t able to do so, he would not have been able to create a functional institution.

However, the founder cannot remain the pilot forever. Another pilot, a successor, has to step in and take the reins for the institution to remain piloted. Furthermore, in order for the institution to remain functional and a live player, this new pilot must also be skilled. Such a person extends the life of the institution, allowing it to achieve more than it otherwise would. Ensuring the institution acquires this new, skilled pilot is the succession problem.

Institutional longevity is a matter of skill and power

As we see, the succession problem has two components: power succession (handing off the reins of the institution, keeping it piloted) and skill succession (transferring the skill needed to pilot the institution well, keeping it a live player).

If the founder handles both parts of the succession problem, successfully handing off the institution to a person who can skillfully alter it as necessary, then the institution remains piloted and a live player. If neither part of the succession problem is handled, then the institution becomes unpiloted and a dead player.

If power succession is successful but skill succession is not, then the institution remains piloted, but not a live player. Someone is at the controls, but they don’t really know how to use them.

There are multiple possible outcomes to such a scenario. At worst, the pilot aggressively mismanages the institution. This situation can be catastrophic; the pilot might crash the plane. At best, the unskilled pilot remains at the controls but intervenes minimally, allowing the institution to function while also defending his ability to alter and direct it. Of course, if the pilot is also not skilled enough to maintain his power then the institution will eventually become unpiloted unless a skilled pilot steps in.

If skill succession is successful but power succession is not, then the institution becomes unpiloted and a dead player unless and until the skilled person gains the necessary institutional power to pilot it.

As an example, the founder of a company might retire, giving way to an outside CEO appointed by a rather conservative board. Even if there is a junior engineer or designer that has the passion and expertise needed for a whole new kind of product that the team could deliver, he will not be positioned to realize this vision.

To successfully change the company, the engineer would have to at best persuade, at worst bypass, the new management. Ideally the employee eventually maneuvers themselves to the position of CEO. If this happens at all, it can take years. Instead, he might do better to fundraise and recruit for a new start-up.

If you imagine a chaotic and disorganized parent company, with shards of responsibility and bureaucratic entrenchment, in the above example the problem of succession gets harder and not easier.

Over the lifespan of a bureaucracy, power lent out to various delegates becomes owned, allowing individuals to use organizational resources to pursue agendas at odds with the purpose of the organization. They form an entrenched opposition, that makes rendering the institution functional a notable challenge. Gaining control of institutions that have become unpiloted is often more difficult than founding one’s own institution, even for a skilled actor.

Even after becoming unpiloted, a functional institution can remain effective for a while, but it will decay and eventually cease to be functional or even cease to exist unless someone captures it and starts piloting it.

Creative destruction is not a necessity for innovation

Silicon Valley enthuses over disruption because we have become so used to the succession problem remaining unsolved. To disrupt an organization, industry, or culture can only be good if it isn’t possible to cooperatively transform it. Under such conditions each generation of innovators must start anew or waste their efforts with a sclerotic structure.

Successful skill succession coupled with failed power succession can lead to destructive strife. People of exceptional ability and ambition do not necessarily seek out conflict, but will generally be willing to endure it. Depending on their options it might be the best course of action to attempt to dismantle or destroy the old organization which they couldn’t work with.

Functioning firms are repositories of many kinds of capital that cannot be liquidated, and when they die, it is destroyed. Such capital includes position, team synchronization, good organization and tacit technical know-how. These are casualties of economic competition.

We have no problem identifying this phenomenon as problematic in politics. We view the destruction of an old political order by means such as civil war or political strife as regrettable necessity at best, not something to celebrate. This stands in stark contrast with our view of the phenomenon in the economy, likely because we overlook the destructive side.

Few mature technological companies today use their position to support effective innovation. Many companies spend significant resources on research, but few manage to aggressively implement and deploy. Amazon provides a contemporary proof of possibility, with its constant pursuit of technical innovation in service of ever larger economies of scale and logistical efficiency.

That Amazon is one of the exceptions reflects the poor health of the current batch of institutions, rather than the nature of mature companies or even underlying market incentives. Disruption should be the backup rather than the first choice for innovation. That it isn’t is the result of poor institutional health.

An overabundance of talent in the absence of sufficient opportunity and power succession can render society quite chaotic. If ambition is outlawed, only outlaws are ambitious. On the other side of the spectrum, buying stability through the absence of talent is futile in the long run. Institutions ultimately decay without renovation either from within or without.

Sclerotic institutions eventually break rather than bend, which is the source of catastrophic instability. When thinking of a company, this might result in a desolated company town, when thinking of a civilization, the result is societal collapse.

Organizations and societies that solve the succession problems will have a less harsh trade-off between stability and innovation. When institutions of the previous generation are actively handed off to the next, they retain needed flexibility to pursue restructuring.

While variation between individual organizations is notable, most rely on social technology that is widely distributed and implemented around their society. A society is best thought of as a dense ecosystem of institutions always borrowing from each other, outsourcing services and sometimes clashing over resources. It can be very difficult to implement a unique solution. If none of the institutions in a society solve a particular problem, the fragility of those institutions will be reflected in the fragility of society as a whole.

Conclusion

The foundation of a flourishing civilization is an abundance of functional institutions. These originate with founders who bring new social designs into being. In the natural course of events their institutional legacy decays, becoming less and less suited to achieving the desired positive effects.

The succession problem is the problem of ensuring founders can hand off institutions they have built to other founders. The key problems here are the creation and identification of sufficient skill, together with ensuring the next founder has inherited a position of sufficient power to remake the institution. Of course, even if the succession problem is handled once, it always returns.

If the succession problem is unsolved, the only process of institutional reform is the destruction of abandoned institutions by new ones, the process sometimes described as ‘creative destruction’.. That our society valorizes rather than bemoans such outcomes, unfortunately demonstrates that we have become accustomed to failed succession and notable dysfunction.

We should temper our enthusiasm for intense political and economic competition and instead develop a greater appreciation for the importance of successful succession. This change would go far in remedying contemporary institutional sclerosis and stagnation.

5 comments

Comments sorted by top scores.

comment by Uriel Fiori (uriel-fiori) · 2018-08-06T17:58:05.721Z · score: 8 (3 votes) · LW · GW

is there any example of successful succession? if there aren't, i think one should be tempted to think that most likely creative destruction (and thus disruptive adaptation rather than continuous improvement) is the norm for social systems (it definitely seems so in relation to other evolutionary environments).

comment by Dagon · 2018-08-03T20:00:20.533Z · score: 6 (4 votes) · LW · GW

I don't think this model helps with very many decisions or predictions. You need to model diffuse cooperation much more finely - there are lots of small organizations where the leader(s) are not particularly powerful, but there are shared beliefs among participants which make the group pretty effective. Most church congregations fall into this category - there's often people in leadership roles, but they're beholden to the members rather than controlling the organization.

For larger and more commercial/political organizations, the founder has moved on (or not), and power is shared among dozens or thousands of more committed or tenured members, and no one of them is critical to continuity of mission.

Even for the strong-leader model of startups and small-to-medium companies, succession is less of a problem than growth. How to increase in size, beyond the capacity of the single leader to know and influence everyone, is a problem that kills many organizations.

comment by Samo Burja · 2018-08-03T17:44:12.638Z · score: 6 (4 votes) · LW · GW

The government agencies and corporations that dominate our society are many decades, if not centuries, old. It is also clear they are in need of renewal.

Why did they reach such a state of misalignment? I believe that across society we had a notable failure of succession.

These things were created by people, and then they took on a life of their own, in an almost automated fashion, rather than continuing human oversight. As a result we are in a society that is more fragile, less cooperative and less coordinated than it could be.

comment by shminux · 2018-08-04T00:46:16.996Z · score: 3 (2 votes) · LW · GW

You are raising an interesting question, what might go wrong when a person who created, molded and guided a successful institution is no longer in charge, due to one reason or another. Succession is a very common and natural event in all kinds of organizations, political, economic, cultural, religious, family clans and so on.

You have highlighted a couple of likely scenarios leading to a change not envisioned by the original "pilot", and most likely not something they would have approved: when the new leader has the power of the predecessor, but not their skill, and when the opposite situation occurs, both, in your evaluation, likely leading to "chaos". These are interesting, but it is not clear why one would privilege those two over other potentially disruptive events. Additionally, your claim could use some real-world examples of hand-offs, and some analysis of each situation, all without cherry-picking a specific situation that necessarily supports your hypothesis. Only after that is done it will be worth discussing whether an apparently "unsuccessful succession" is a net negative for the environment the organization is in, and what, if anything, needs to change and how.

comment by Pattern · 2018-08-06T04:17:31.521Z · score: 2 (1 votes) · LW · GW

I noticed skill succession without power succession wasn't discussed - though the first thing that came to my mind was someone being trained etc. to takeover, and leaving instead.

Technical point: Cherry picking is choosing examples to suit one's story. Example-less evaluation is conjecture/theorizing because there are no cherries present, only characterizations.