When we encounter unsavory features of reality, it can be tempting to look away. Instead, we should ask, “What purpose does this serve?”
With this in mind, let’s look at bureaucracies. Some people fear bureaucracies; they fear “the Machine.” Others are bothered by the bureaucracies ’ apparent dysfunction. With a better understanding of bureaucracies — what they are, why they’re here, and how they work — both of these responses evaporate, because the reality is this: bureaucracies aren’t altogether bad. In fact, bureaucracies can be incredibly useful.
What is a bureaucracy?
A bureaucracy is an automated system of people created to accomplish a goal. It’s a mech suit composed of people. The owner of a bureaucracy, if an owner exists, is the person who can effectively shape the bureaucracy. Bureaucrats are people who are part of a bureaucracy (excluding the owner).
Not all organizations are bureaucracies. Most organizations are mixed — they have both bureaucratic and non-bureaucratic elements.
The Purpose of Bureaucracies
The purpose of a bureaucracy is to save the time of a competent person. Put another way: to save time, some competent people will create a system that is meant to do exactly what they want — nothing more and nothing less. In particular, it’s necessary to create a bureaucracy when you are both (a) trying to do something that you do not have the capacity to do on your own, and (b) unable to find a competent, aligned person to handle the project for you. Bureaucracies ameliorate the problem of talent and alignment scarcity.
Features of Bureaucracies
Bureaucrats are expected to act according to a script, or a set of procedures — and that’s it.
Owners don’t trust that bureaucrats will be competent or aligned enough to act in line with the owner’s wishes of their own accord. Given this lack of trust, owners should be trying to disempower bureaucrats. Bureaucracies are built to align people and make them sufficiently competent by chaining them with rules. When bureaucracies deliberately restrict innovation, they are doing it for good reason.
Bureaucrats are meant to have only borrowed power (power that can easily be taken away) given to them by the owner or operator of the bureaucracy.
What is an effective, owned bureaucracy? Why are effective bureaucracies owned? To begin, we must make two important distinctions: one between owned and abandoned bureaucracies, and one between effective and ineffective bureaucracies.
Owned bureaucracies are bureaucracies with an owner; they’re bureaucracies that someone can shape. Abandoned bureaucracies are bureaucracies without an owner.
If a bureaucracy is owned, the bureaucracy’s creator is likely the owner. The creator will have knowledge about the setup of the bureaucracy that is necessary for properly reforming it. Others, unless given this information, will not understand the bureaucracy well enough to properly reform it.
The person technically in charge of the bureaucracy (e.g. the C.E.O. of a company who is not its founder) might not be its owner simply because he or she doesn’t have sufficient information about the bureaucracy’s setup to guide it. As a result, the official head of a given bureaucracy may just be another bureaucrat.
While the owner is typically the creator, this needn’t be true, as long as the new owner has come to understand enough of the function of the bureaucracy to make effective adaptations to its procedures.
Effective bureaucracies are bureaucracies that are handling the project they were created to handle. Ineffective bureaucracies are bureaucracies that are not handling the project they were created to handle.
Bureaucracies that are properly set up will be effective at the start. Changes in reality require changes in procedures, however, so a bureaucracy’s procedures inevitably need to be altered appropriately for it to remain effective. Over time, abandoned bureaucracies, having no person who can functionally shape the bureaucracy to make these changes, quickly become ineffective bureaucracies.
Owned bureaucracies, on the other hand, have a shot at making these adaptations to prevent decay. If the owner is skilled, the bureaucracy’s procedures can be modified, and the bureaucracy will continue serving its original purpose. If the owner is unskilled, it is as if the bureaucracy is abandoned — the owner’s efforts to change the bureaucracy’s strategies won’t yield successful adaptation, and the bureaucracy will become ineffective. As a result, for a bureaucracy to remain effective over time, it must be an owned, not abandoned, bureaucracy with a sufficiently capable owner.
Losing and Dismantling Bureaucracies
Bureaucracies are best thought of as an extension of their creator and as a source of power for him or her. However, the owner can lose control of the bureaucracy over time, as bureaucrats convert borrowed power into owned power by exploiting information asymmetries. While owners will try to limit the owned power of their bureaucrats, the bureaucrats will have more than enough time to study the instruments of their control and will learn what is rewarded and what isn’t.
Imagine a bureaucrat that is supposed to be an assistant to the absentee owner of an institution. This senior assistant is supposed to research solutions to key problems, and then present several options to the owner, who then selects one. The assistant is then required to implement the one that was chosen. There is a very detailed document describing their job and requirements at every step of this process.
The key problem is that a very complex set of rules can be easily bent to acquire an arbitrary outcome. The outcome will be completely valid from the rule set. This is analogous to how in science a very complex model, that fits the data, is not very impressive. As Von Neumann put it: “With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.” Let’s walk through the described process the senior assistant is supposed to follow to demonstrate how bureaucrats wiggle their trunks.
You might require the assistant to not engage in original research, but rather work as a search engine through more objective academic literature or best practices in a particular industry. The assistant, however, can cherry pick seemingly objective academic papers to argue for their preferred policy outcome. It is actually much easier to start with a preconceived opinion and then find work confirming it, rather than review a literature as a whole. The plausibility of this shortcut should be intimately familiar to any university student who worked under the pressure a deadline for a class paper they didn’t much care about.
The chief assistant can craft several options. They can make option B, their favorite, the most appealing, and cripple options A and options C. Maybe even include point 14, their core agenda, into all three proposals that vary on points 1 to 13 they don’t much care about. Whatever the implementation of the selected solution is, the letter of the law can be bent and easily diverge from the spirit of the law.
In such a circumstance, an owner can lose control of the bureaucracy and the power that comes with it.
It is often beneficial for owners to dismantle bureaucracies after they have served their purpose to avoid losing ownership of them due to these information asymmetries. Bureaucracies of this type might grow to be independent powers that interfere with your plans.
Abandoned bureaucracies might also be viable targets for outside takeover. Such takeovers can be a serious problem if undertaken by your opposition. Bureaucracies nearly always carry a heavy legacy document footprint; when examined this footprint can not only produce, but also be used to carry out legal attacks. If the institution is vested with an authority or reputation, this can also be turned against you.
If it is too hard to regain ownership, dismantling the institution for resources may be the best option. These resources might be quite easily quantifiable, such as use of real estate or key employees. They might also be less tangible, such as the attention of your allies. Unless you formally retire a vehicle, these allies might mistakenly believe it active, causing communication issues or misunderstandings of your key priorities.
In short, when handling multiple organizations tying up loose ends becomes very important.
How to accomplish tasks in an institutional landscape
Building a bureaucracy is an effective way to accomplish your goals under the right circumstances, but it’s not the best option. In order of effectiveness, here are general options for getting things done:
If you can find a competent, aligned person who will do the project in question for you — let’s call them a delegate — then let them. This person can create a bureaucracy for you, if necessary, as projects of a certain scale will require bureaucratization. Unfortunately, because of the harsh talent and alignment scarcity mentioned earlier, finding delegates can be challenging. Furthermore, correctly assessing whether someone is a worthy delegate takes skill. Frequently people will accidentally delegate a project to someone who is insufficiently competent or aligned. Failed delegation is worse than building your own bureaucracy, because it will lead to project failure.
If you have access to a delegate, don’t treat them like a bureaucrat. This wastes a valuable resource: a delegate can perform tasks you didn’t know needed doing and build aligned systems beyond your design, a bureaucrat cannot.
Further such treatment invites disalignment with your delegate. It isn’t just a matter of interpersonal grace and respect, so it cannot be overcome with kindly management; rather if you are attempting to closely proceduralize the actions of a competent delegate, they might accurately conclude the best way to perform their job is to attempt to bypass your control. If you picked them well, they will be rather effective in doing so. They don’t need a script — if they’re competent enough for your purposes, they’ll be able to figure out how to do the project.
Give them owned power, otherwise you might run them off.
If you can’t find a delegate, then building your own bureaucracy (even if it’s small) is the best bet. Bureaucratizing some things and not others, on the basis of whether the task can be proceduralized, is typically more effective than bureaucratizing everything by default. Figure out when using an automated system is the best option.
Do it yourself
While doing it yourself may be most likely to result in a well run project, it is not always feasible — you have limited time and capacity. Without delegates or bureaucracies, the ambitiousness of the projects you can successfully execute will be bounded.
Don’t do it
Some things, though useful, aren’t worth doing…
Understanding the World Around You
An understanding of bureaucracies lets you analyze a given person’s power: is someone acting as a delegate or a bureaucrat? Is someone creating delegates or bureaucrats? If someone has created a bureaucracy, do they understand the function of bureaucracies? Do they own their bureaucracy, or is it abandoned? If they own their bureaucracy, is it effective or ineffective? Are they creating bureaucracies under the right conditions? What is the role of bureaucracies in their plan?
If a person is powerful, what does it mean if he’s created many bureaucracies? In some cases, the creation of many bureaucracies indicates the owner is extremely good at building automated systems. Alternately, he might have trouble delegating — perhaps because he can’t find competent, aligned people, or because he can’t assess people well. People who can work well with others and have access to sufficiently talented aligned people need fewer bureaucracies. Instead, they’ll delegate to others, who can either do the project themselves or create a bureaucracy of their own.
On the other hand, if a person is powerful, what does it mean if he’s created few or no bureaucracies? If he isn’t delegating, it means that he’s doing everything himself and possibly doesn’t know how to design automated systems. If he is delegating, he’s likely to be good enough at finding competent, aligned people that he doesn’t need a bureaucracy. Powerful people who don’t create bureaucracies can be just as powerful as people who do.
The framework can be applied to evaluating organizations. For a given organization, begin by asking if it’s a bureaucracy. If it is, expect it to behave in highly stereotyped ways, it will not be very adaptive to new challenges and will not accurately evaluate things outside the assumed ontology of its paperwork and internal division of labor.
If it’s a bureaucracy, we can ask: is it an owned or abandoned bureaucracy? If it is owned, expect that a large enough challenge will eventually cause it to reorganize. You’ll also be able to reach out to the owner to resolve problems or find a way to cooperate that the bureaucracy itself doesn’t understand.
Is it an effective or ineffective bureaucracy? If it is effective, you can rely on the interface it offers you to achieve the goal it claims to achieve. Ineffective ones will provide a sometimes bewildering service that might only tangentially be related to their efforts.
Remember that not all organizations are bureaucracies.
Some non-bureaucratic institutions will have to pretend they are bureaucracies on paper for legal compliance. This is an example of a more general principle: independent organizations interpret externally imposed regulation as damage, and route around it.
Organizations can be tightly coordinated groups the feature a lot of delegation and deference. In these, expect adaptive behavior; the ontology they are working in might rapidly change to respond to either your challenge or offer of cooperation. Most importantly there will be individuals beyond merely the leader who can exercise their own judgement.
Effectively Interacting with Existing Organizations
If an organization is not a bureaucracy but rather a tightly coordinated group, talk to the delegates if you want to get things done; they will have freedom to act competently within their own domain and will be easier to reach than leadership.
The key advantage of talking to people over engaging with automated systems is that you can bring considerations from outside their immediate institutional context into consideration. While the local balance of power might still be in the way of such considerations, it is surprisingly often viable to have them taken into account.
If it’s a bureaucracy, you can either (1) go along with it, (2) figure out how to bypass it, or (3) coordinate with its owner, if it is owned. You may prefer to bypass (or game) the bureaucracy if it is abandoned and thus dysfunctional, or if you aren’t aligned with its owner.
The origin of bureaucracies lies in them extending power and effects far beyond what a single individual can do. They can do so in the absence of expensive and difficult coordination, or difficult to train and evaluate individual talent.
Much like factories can produce cheap products at scale with unskilled labor, displacing craftsmen, so have bureaucracies displaced local social fabric as the generators of social outcomes.
We find ourselves embedded in a bureaucratized landscape. What can or cannot be done in it, is determined by the organizations composing it. The constant drive by talented individuals to both extend power and make due with unskilled white collar labor (a category that economists should recognize and talk more about) have littered the landscape with many large organizations. Some remain piloted, others are long abandoned. Some continue to perform vital social functions, others lumber about making life difficult.
Much as we might bemoan the very real human cost bureaucracies impose, they currently provide services at economies that are otherwise simply not possible. We must acknowledge our collective and individual dependence on them and plan to interact accordingly.
There's a large literature on bureaucracies, and it has a lot to say that is useful on the topic. Unfortunately, this post manages to ignore most of it. Even more unfortunately, I don't have time to write a response in the near future.
For those looking for a more complete picture - one that at least acknowledges the fact that most bureaucracies are neither designed by individuals, nor controlled by them - I will strongly recommend James Q. Wilson's work on the topic, much of which is captured in his book, "Bureaucracy." I'll also note that Niskanen's work is an important alternative view, as is Simon's earlier (admittedly harder to read, but very useful) work on Administrative Behavior.
Perrow's work, "Organizational Analysis: A Sociological View" is more dated, and I wouldn't otherwise recommend it, but it probably does the best job directly refuting the claims made here. In his first chapter, titled "Perspectives on Organizations," he explains why it is unhelpful to view organizations just as a function of the people who make them up, or as a function of who leads them. When I have more time, I will hope to summarize those points as a response to this post.
I appreciate the response. I poked around a bit looking for an online version of the two works and haven't found one yet, but will continue looking a bit and look forward to your eventual summary of those points.
(My own motivation for curating this stemmed from having observed a couple bureaucracies forming in realtime over the past year, which did roughly match the "owned by the creator" schema here, as well as encountering various other bureaucracies that seemed to match the "un-owned" schema, which were pathological in the ways you'd expect and that matched the model in the post. The assumption I took as given in this post, although not specified, is that most bureaucracies you encounter will be un-owned)
I think that most bureaucracies are the inevitable result of growth, and even when they were initially owned by the creator, they don't act that way once they require more than a few people. (See my Ribbonfarm Post, Go Corporate or Go Home)
Comparing the goals of a bureaucracy with the incentives and the organizational style, you should expect to find a large degree of overlap for small bureaucracies, trailing off, at best, around a dozen people, but almost none for larger ones. This isn't a function of time since formation, but rather a function of size - larger bureaucracies are fundamentally less responsive to owner's intent or control, and more about structure and organizational priorities. As an obvious case study, look at what happened at US DHS after 2002, which was created de novo with a clear goal, but it is clear in retrospect that the goal was immediately irrelevant to how the bureaucracy worked.
This all makes sense, just doesn't seem to me to be in conflict with the OP.
I don't know much about the US DHS, but a few obvious things that pop into mind:
In the schema of the OP, a large bureaucracy is harder to make an effective bureaucracy (for the same reason a large codebase is more likely to have bugs). Especially if that bureaucracy was created quickly. Even if it has a competent owner, it's just a harder task.
The DHS wasn't created in a vacuum, it was created a) as part of a weird political situation, b) I suspect it was also to some extent created by existing bureaucracies. I have little reason to believe that the stated goal of the DHS was ever the actual goal. I don't think it "got immediately compromised", my guess is it was compromised from conception. (But, I don't know a whole lot about it and wouldn't be that surprised if my guesses were off)
Something that the OP doesn't delve into much (and I do think makes it incomplete) is that bureaucracies might have multiple owners.
Just like a codebase is more likely to run into problems if it's being created by multiple teams with multiple goals, esp. if those people aren't aligned with each other, it'd make sense for bureaucracy goals to have degrees of coherence, depending on whether they were created by a single person or as part of political compromise.
I don't think people should feel obligated to read all that's been written on a topic before posting their thoughts on that topic to Less Wrong, especially if that writing is not supported by randomized controlled trials (unsure if this is true for the writings you cite).
Fair point about the bar for posting, but this doesn't read like "posting their [tentative] thoughts," it reads like conclusions based on extensive review. As a matter of good epistemics, the difference should be made clearer. Similarly, if you dismiss large parts of the literature, it would be good to at least let people know what you think should be ignored, so they don't waste their time, and even better, why, so they can decide if they agree.
As a side point, I think that considering RCTs as a source of evidence in this domain is a strange bar. There's lots of case study and other quantitative observational evidence that supports these other approaches, and specifying what evidence counts is, as the phrase goes, logically rude [LW · GW] - how would you even design an RCT to test these theories?
If it's hard to conduct RCTs in a domain, it's hard to have reliable knowledge about it period. Who's to say whether your anecdotal observations & conclusions beat mine or someone else's? One way is to check whether someone's job is high status enough that their writings on the topic can be considered part of "the literature". But this is a weak heuristic IMO.
I strongly disagree. There are many domains where we have knowledge with little or no ability to conduct RCTs - geology, evolutionary theory, astronomy, etc. The models work because we have strong Bayesian evidence for them - as I understood it, this was the point of a large section of the sequences, so I'm not going to try to re-litigate that debate here.
I do also think that Samo has engaged with a large part of the literature, and it just doesn't come through in this post. (My model is that he has dismissed a large part of the existing literature for pretty decent reasons, but am not confident)
Agreed - one is not expected to read all that's been written on a topic. However, one should acknowledge alternate models when they're pointed out, and your summaries and conclusions will carry a lot more weight if you can explain why you prefer those over the other ideas.
In general we try to curate things that are recent, so that curation serves in part to:
Help people keep up with what's new and exciting on LessWrong
Reward people for having produced good content.
But both of these have some tension with a different goal, which (on the margin) is good for curated: to be a list of posts that stand the tests of time and are worth referring to.
At the time Samo was writing his sequence, I had a hesitation about the entire thing summed up by some of Said's comments: it's fairly easy to armchair philosophize about society. This post would be better with clear examples, and I'd still encourage Samo to rewrite this post and others to feature examples and evidence.
Nonetheless, all of my own experiences with bureaucracy roughly matches the descriptions given here. I recently explicitly linked back to this post to explain a point, and more generally, this models this post play an important role in how I think about group coordination.
Robert Moses is the famous master builder of New York City, being responsible for many of the bridges, highways, and parks which currently define the landscape. He might be the single best example of the kind of thing Samo is talking about - he wanted to build things, and to that end he started four authorities (or public benefit corporations), and ran three commissions, and chaired the Council of Parks.
By the lights of the post this marks eight bureaucracies that were simultaneously owned and effective.
The most in-depth biography of him is Caro's The Power Broker, a recent review of which is here. I also found a contemporary article from the Atlantic.
I do wonder about the extent of the definition of 'competent person' in this case. Among the criticisms about Moses are that he was not an architect, engineer, or city planner by training; despite this the things he did were widely mimicked by those professions in other cities.
The sense I got from The Power Broker is that Moses's work was good when doing good work was aligned with his perceived interests, and not when not, and it wasn't that hard for him to find people competent at the relevant technical disciplines when that was needed (and his ability to accumulate power quickly initially gave him a lot of slack to hire based on merit, when delivering a conspicuously high-quality product seemed like it would be helpful for accumulating more power).
In general it doesn't really seem to require much technical expertise to lead a technical project, just a somewhat difficult to maintain mixture (in a political context) of the skills necessary to obtain and defend resources, and the mindset that still cares about getting the technical side right.
That's true but the bureaucracy isn't what builds parks. The person in charge bosses around a bunch of other people competent to design and build parks, and secures the land and other inputs needed to do so via political processes. The bureaucracy is what normalizes the arrangement so that it can interface with other things in control of resource flows, e.g. so that people can get paid for reporting to Moses.
Ritualizes might be more precise. Provides a stereotyped interface that plays nicely with other stereotyped interfaces. Military drill sort of serves a similar function, in the face of a different kind of entropy than the one this is a defense against.
Here's my vague overall impression from reading secondary sources not directly concerned with this question (probably more noisy but also more trustworthy than secondary sources making a direct argument about this.)
Overall the sense I get is that recordkeeping and action were kept separate in most ancient civilizations, even pretty big ones - no minutes of meetings or white paper equivalents or layers of approval and formalized decision delegation.
It seems to me like "clay tablet" cultures had extensive scribal institutions, but these were mostly used for rituals in temple cults (of unknown function), tax assessment, and central recording of contracts (the state served as a trusted third party for record storage and retrieval). You'd also need logistical records for many public works projects, but these were often very simple. Someone would be in charge and sometimes have to request resources from other people, who would keep track of what was sent, sometimes the king would want to know what was going on, so they had to know the broad outlines.
As I understand it the Persian empire's managerial and formal information-processing layer was extremely lean, the king would just personally send some guy to check on a whole province, there was a courier network but nothing on the scale of USPS or even Akkadian scribal records.
The TVA is arguably the most storied bureaucracy in the United States; it is the go-to example for arguing about bureaucracy itself at the federal level, for example in political science curricula. The agency was founded in 1933 to economically develop the greater Tennessee Valley region. This was to be done with water management and electricity generation, in particular through hydropower.
It doesn't match the "save the time of a competent person" description of purpose at all; economic development is not a routine competent-person task. It matches better the description of doing things one person cannot do at all. It is worth mentioning that in the 1920s Henry Ford attempted to build a private utility around the federal dam in Muscle Shoals, Alabama, which was claimed by its proponents to serve the same goals for which the TVA was created - this was blocked by Senator George Norris of Nebraska, who also drove the creation of the TVA.
By the standards Samo sets out, this would be an abandoned and ineffective bureaucracy; the creator is long dead, and the Great Depression long resolved. The area it serves has electricity, uses modern agricultural practices, etc. The authority is famous for having accumulated a bunch of tasks unrelated to the original function over the years; if I recall correctly at one point it was responsible for approving the specifications on what screws were to be used for all federal government projects.
Even from the outset it spent a lot of effort on things within the purview of its purpose but nominally outside of its mission. The dam manufactured fertilizer, the TVA spent a lot of time and effort modernizing local agricultural practices, and they had a role in providing vaccinations.
This example does not seem to match Samo's system very closely.
BUREAUCRAT: We shall find them all and determine their wealth at once, lord. The better to enrich you, of course.
Sometime later, a bureaucrat knocks on the door of a local Sogdian.
BUREAUCRAT: By order of the governor, we are conducting a census of the city. Only of Sogdians. We also require a list of your possessions, and a description of how easy they would be to transport to the governor's palace.
SOGDIAN: I....see. But where are my manners! You, an official of the court, and me with my home a shambles! I am shamed to be seen in such a sorry state. You must return tomorrow and my family and I will host you a dinner befitting your status!
BUREAUCRAT: But of course - it does you credit to recognize the dignity of an official of the governor. I will return tomorrow.
The Sogdians all exit, stage left.
BUREAUCRAT: [Returns] I see the home is empty, as are many others.
Sometime later, at the governor's palace.
BUREAUCRAT: My lord, we have conducted a census of the Sogdians and their wealth, and found there are none in the city.
A couple people had privately mentioned concerns to me about whether this post should have been curated, which seemed worth addressing.
Basically, the standard for curated is not, and cannot be, peer review.
I do personally think LessWrong should have something closer to actual peer review. I wrote about this in another recent curation notice. But if that was the standard for curation, curation would fail at two of it's main goals – providing people with a consistent set of posts worth reading, and providing a reward for LW authors that was a reasonable standard to aspire to (given that most LW authors are either hobbyists, or professionals whose primary focus isn't writing LW articles)
I don't think this post is more armchair-philosophy-esque than many other posts that we curate, I think it just happened to be of a genre that made that feel a bit more salient.
Don't forget that bureaucracy also exists as an effort to make gov't and other organizations more predictable, and often more egalitarian. Decisions such as whether you are allowed to build a huge house on your tiny lot are made through rules, rather than personal power and individual preference. So my neighbour, who has a lot of money and who is the grandchild of a former governor, has to apply to the same office, request the same permits, provide the same documentation, and pay the same fees, to (at least theoretically) get the same answer as I would, the upstart nouveau-not-that-riche. And the bureaucrat who 'decides' on my permit has to follow the same decision tree as the different bureaucrat who decides on my neighbour's.
This kind of predictability can only occur through a bureaucratic system. Anyone who has lived in a country where personal power, class, cronyism, caprice and corruption make the decisions instead, will appreciate bureaucracy, despite how annoying and frustrating it can be.
I like the insight that bureaucracies are composed of agents who are too incompetent or misaligned to be trusted with actual power. Starting with the stipulation that they're frustratingly obstructionist BY DESIGN is very helpful.
I think the next section vastly oversimplifies by assigning a few boolean dimensions (owned/abandoned, effective/ineffective), when in fact all organizations have elements in all 4 quadrants, and both dimensions have degrees that matter, not just on/off.
I think this would benefit by recognizing that this is mostly an specific case of the principal-agent problem, and it doesn't matter too much whether the agent is an individual, an automaton, or a collective bureaucracy.
I like the point about the principal-agent problem. I disagree only slightly, and think it would be worth distinguishing between individuals and automatons on one side, and a collective bureaucracy on the other: this is because the bureaucracy is in fact a chain of principal-agent links. It seems to me a big reason that any kind of organizational management is hard is because whatever anyone tries to do, it will be mediated by many principal-agent steps, with the predictably accumulated error.
I have just realized I never considered the possibility of an automaton (software, say) as an independent link in the chain. I don't know how correct it is, but it sure makes a lot of the problems I encounter on a daily basis more understandable.
I'm not sure there's a single post explaining the use of "aligned", but a common use of the phrase on LessWrong is to mean something like "having the same goals as you, or able to act as if they had the same goals as you such that you could grant them arbitrary power and wouldn't regret it."