A Golden Age of Building? Excerpts and lessons from Empire State, Pentagon, Skunk Works and SpaceX

post by jacobjacob · 2023-09-01T04:03:41.067Z · LW · GW · 26 comments

Contents

  Empire State Building
    Designs were made in groups that had representatives from all the key parts of the domain
    The design of the building was deliberately optimised for being fast to construct
    They made clever optimisations in storing and moving material
    The general contractor designed and optimized the work to disentangle different workers as much as they could
    Design and construction proceeded at the same time
    They used meticulous planning and supervision
  Pentagon
  Skunk Works
  SpaceX
    Team size at various points
    Decisions on the spot
    They were able to do much of the work themselves
    How they dealt with life side-constraints
    On ownership 
    For key roles, Elon hired experienced domain experts who were willing to have their thinking molded into his operational philosophy. For their teams, he hired younger generalists
  What changed? 
    1. Adherence to a Lengthy Bureaucratic Process
    2. Working off Onerous System Specifications
    3. Spending Little on Internal Research & Development
    4. Prioritizing Proposals Over Performance
    5. Tolerating Prolonged Failure
None
26 comments

Patrick Collison has a fantastic list of examples of people quickly accomplishing ambitious things together since the 19th Century. It does make you yearn for a time that feels... different, when the lethargic behemoths of government departments could move at the speed of a racing startup:  

[...] last century, [the Department of Defense] innovated at a speed that puts modern Silicon Valley startups to shame: the Pentagon was built in only 16 months (1941–1943), the Manhattan Project ran for just over 3 years (1942–1946), and the Apollo Program put a man on the moon in under a decade (1961–1969). In the 1950s alone, the United States built five generations of fighter jets, three generations of manned bombers, two classes of aircraft carriers, submarine-launched ballistic missiles, and nuclear-powered attack submarines.

[Note: that paragraph is from a different post.]

Inspired by partly by Patrick's list, I spent some of my vacation reading and learning about various projects from this Lost Age. I then wrote up a memo to share highlights and excerpts with my colleagues at Lightcone. 

After that, some people encouraged me to share the memo more widely -- and I do think it's of interest to anyone who harbors an ambition for greatness and a curiosity about operating effectively. 

How do you build the world's tallest building in only a year? The world's largest building in the same amount of time? Or America's first fighter jet in just 6 months?  

How?? 

Writing this post felt like it helped me gain at least some pieces of this puzzle. If anyone has additional pieces, I'd love to hear them in the comments. 

Empire State Building

The Empire State was the tallest building in the world upon completion in April 1931. Over my vacation I read a rediscovered 1930s notebook, written by the general contractors themselves. It details the construction process and the organisation of the project.

I will share some excerpts, but to contextualize them, consider first some other skyscrapers built more recently: 

	Design start	Construction end	Total time
Burj Khalifa	2004	2010	6 years
Shanghai Tower	2008	2015	7 years
Abraj Al-Balt	2002	2012	10 years
One World Trade Center	2005	2014	9 years
Nordstrom Tower	2010	2020	10 years
Taipei 101	1997	2004	7 years

(list from skyscrapercenter.com) 

Now, from the Empire State book’s foreword: 

The most astonishing statistics of the Empire State was the extraordinary speed with which it was planned and constructed. [...] There are different ways to describe this feat. Six months after the setting of the first structural columns on April 7, 1930, the steel frame topped off on the eighty-sixth floor. The fully enclosed building, including the mooring mast that raised its height to the equivalent of 102 stories, was finished in eleven months, in March 1931. Most amazing though, is the fact that within just twenty months -- from the first signed contractors with the architects in September 1929 to opening-day ceremonies on May 1, 1931 -- the Empire State was designed, engineered, erected, and ready for tenants. Within this time, the architectural drawings and plans were prepared, the Vicitorian pile of the Waldorf-Astoria hotel was demolished [demolition started only two days after the initial agreement was signed], the foundations and grillages were dug and set, the steel columns and beams, some 57,000 tons, were fabricated and milled to precise specifications, ten million common bricks were laid, more than 62,000 cubic yards of concrete were poured, 6,400 windows were set, and sixty-seven elevators were installed in seven miles of shafts. At peak activity, 3,500 workers were employed on site, and the frame rose more than a story a day, No comparable structure has since matched that rate of ascent.

How did they accomplish this? Here are some things that stood out to me about their process. 

Designs were made in groups that had representatives from all the key parts of the domain

The team faced a tricky high-dimensional problem requiring lots of different knowledge to pull off. Here’s a quote from the 1930 issue of Fortune magazine: 

These various elements fixed the perimeter of an oddly shaped geometric solid, bounded on one side by 83,360 square feet of land, on the other by $35,000,000, on the other by diminishing returns, on another by the laws of physics and the characteristics of structural steel, and on another by the conical exigencies of the zoning ordinances, and on still another by May 1, 1931. 

In response:

From the outset, the owners, architects, and builder worked in committee to develop the building’s [requirements]. This method avoided mistakes in design and costly delays in construction [...] The general contractor wrote: “I doubt there was ever a more harmonious combination than that which existed between owners, architects, and builder. We were in constant consultation with both of the others; all details of the building were gone over in advance, and decided upon before incorporation in the plans”. 

These initial planning sessions occupied four weeks in September 1929, and produced the complete technical, planning and economic requirements for the project. (p. 20)

For example, the mechanical engineers Meyer, Strong and Jones that noted building the elevators would involve larger installation, greater car sizes, heavier loads, higher speeds and longer travel than any previously known project. But still: “The proper simultaneous development of building, steel and elevator plans avoided the common error of attempting to fit an elevator plant into a previously fixed building arrangement and steel layout.”

The team’s effort to pull in the people with detailed object-level knowledge to surface constraints and potential blockers went really deep -- the designers even brought in metal fabricators to help consult on the drawings. 

“Because the architects, builders, and subcontractors did not feel competent to develop the specifications on their own, they called in representatives of the metal workers fabricating it, those who were to erect it, and the inspectors who were to test the sheets at several stages of preparation. Such a conference, [the architect] Shreve noted, ‘made possible decisions based on instant comparison of recommendations and the establishment of responsibilities of all those involved”. [my italics] (p. 24)

The design of the building was deliberately optimised for being fast to construct

[...] nearly every decision in the design and construction of the Empire State was affected by the need for speed. [...] As the general contractor Paul Starret claimed in reflecting on his career, “Never before in the history of building had there been, and probably never again will there be an architectural design so magnificently adapted to speed in construction.” (p. 18)

The last major section of the contractor’s notebook is subtitled “The Fascination of Speed” and concentrates on the four pacemakers of the construction: structural steel erection, concrete floor arch construction, exterior metal trim and aluminum spandrels, and exterior limestone. Starrett Brothers and Eken were extremely proud of the innovations they delivered in these four areas. (p. 24)

For the building’s windows, “the aim was to standardize the elements as much as possible, creating a sort of kit of parts that would speed both fabrication and erection, Among 5704 metal spandrels there were only eighteen variations”. For other parts of the facade pieces were left rough and unfinished if they could be covered with a trim, the erection sequence was redesigned to remove the traditional complex exterior scaffolding work and instead be erectable from inside the building, and pieces were designed to have fewer intersections with other pieces in order to make installation smoother. 

They made clever optimisations in storing and moving material

The team faced two key logistical problems: scheduling of deliveries and moving materials efficiently in both vertical and horizontal directions. 

Since they were building in the busy center of New York, they had only very little onsite storage of supplies, yet at peak operation they received materials from about 500 trucks a day -- or about one every minute throughout the 8-hour workday!

(Funnily enough, one of the oldest known construction laws is the prohibition of daytime passage of carts bearing building materials through the streets of imperial Rome.) 

They would always keep the ground floor free of temporary structures to allow trucks to drive in, then cut shafts from the top of the building to the first floor allowing them to dump have debris straight into down onto the back of trucks rather than into intermediate storage piles, and they even built a mini railway for moving material around inside the building. From the book: 

“For setting of our stonework, we cut out altogether the customary derrick. The stone trucks drove into the building with the stone in crates, which we call skips or slings. Marked for its proper section of the building, each crate was lifted off the truck by a small crane, operating from a monorail on the ceiling, and delivered to the flatcars of the industrial railway. Taken to the proper floor, it was unloaded at almost the exact location in which it was to be set. Two hoists handled all the stone for the building, not only eliminating a larger number of hoisting derricks and engines but, since the was inside the building, doing away with a grave source of danger to the public.”

The general contractor designed and optimized the work to disentangle different workers as much as they could

Trades move at different speeds, have special requirements, and may view the same detail in entirely different ways. By eliminating as much of the contract between trades as possible, the builders reduced the risk of cascading delays. [...] various portions of the work show the idea in action: the curtain wall support detailed removed the dependence of the steel fabricator and erector on masonry information; concrete batching on site freed the mixers from Manhattan’s traffic conditions, and the coordinated use of materials hoists, passenger elevators, and the industrial railroad system controlled the chaotic movements through the site. (p. 46)

Design and construction proceeded at the same time

Here’s a diagram they made for tracking the steel processing: it charts 5 processes happening in parallel, with associated target dates -- drawings from the architects, mill orders, shop drawings, steel delivery, and steel erection. 
 

You can see that the steel for the first floors is being erected before the drawings for the top floor have even been finished. 

The diagram clearly shows the amount of forethought required for fast-tracking to be successful [JL: fast-tracking is the term used for interleaving, rather than staggering, design and construction] . Each participant had several activities to perform. On any given day, steel for various tiers was in different stages of preparation, requiring attention from several of the consultants and construction companies. Fast-track speed required everyone to move together in step. 

They used meticulous planning and supervision

We saw an example of the need for detailed scheduling above, in the fast-track steel erection. 

This was also the case for receiving one truck delivery every minute, and general contractor Eken said “We ran trucks for that one the way they run train in and out of Grand Central. If a truck missed its place in line on Tuesday, it had to wait until Wednesday to get back in line”. 

The team also set various legible speed targets: they decided to attempt to erect the building at a rate of one story a day. (p. 29) 

In addition, they had a whole slew of watchmen and checkers. They would walk through the site, make notes that every man was actually on-site (they checked on each worker I think 4 times per day(!)), kept detailed track of inventory, as well as records of what got accomplished each day. 

Here’s their org chart, if you want to zoom in. 

Here’s an example page from a the daily job activity notes: 

Overall, the construction of the Empire State is a remarkable achievement. Yet: 

The final chapter of the general contractor’s autobiography ends not with bravado for his achievement, but with the startling statement: “After forty years of intense activity, the strain of erecting the Empire State Building in eleven months was too much for me and I suffered a rather severe nervous breakdown”. 

Pentagon

I didn't quite find the time to fully finish this section, but I care about the Pentagon example and it provides some interesting context. 

The same guy, General Groves ran two of the biggest and most efficient mega-projects of the 20th Century: the Pentagon construction and the Manhattan project. 

Construction began on Sept. 11 (the exact day the building would be struck by a plane 60 years later). Somervell pushed for an aggressive timetable — half a million square feet of office ready by March 1, 1942, and the remainder by Sept. 1. General Groves designers had already gotten a taste of the breakneck schedule. A few months earlier, as the initial plan for the Pentagon was being conceived, Somervell had famously given his designers, headed by American Institute of Architects President George Edwin Bergstrom, a single weekend in July in which to sketch out preliminary designs for what was planned to be a 4 million sq ft structure. It would be — and likely will remain — the largest office building in the world.

Architectural and structural design work for the Pentagon proceeded simultaneously with construction, with initial drawings provided in early October 1941, and most of the design work completed by 1 June 1942. At times the construction work got ahead of the design, with different materials used than those specified in the plans.

The building was built wedge by wedge;[38] each wedge was occupied as soon as it was completed, even as construction continued on the remaining wedges.[39][40]"

https://en.m.wikipedia.org/wiki/The_Pentagon

During construction, as many as 15,000 workers were on-site at a time. The design team included, Vogel wrote, “110 architects, 54 structural engineers, 43 mechanical engineers, 18 electrical engineers, 13 plumbing engineers, various specialists in roads, landscaping, and acoustics, as well as dozens of clerks and messengers.” Meanwhile, another group of more than 100 engineers, architects, and inspectors, located at the construction site, were authorized to make impromptu decisions — the building was truly being designed and built at the same time.

https://www.asce.org/publications-and-news/civil-engineering-source/civil-engineering-magazine/issues/magazine-issue/article/2021/09/building-for-the-american-age-the-pentagon

The work was stressful enough that Groves said he was “hoping to get to a war theater so I could find a little peace.”

Skunk Works

The Skunk Works was a sub-division of Lockheed, that built primarily spy aircraft between 1940s and 1970s. 

For example, in 1943 the Skunk Works both designed and built America’s first fighter jet, the P80 Shooting Star, in just 5 months. Chief engineer Kelly Johnson worked with a scrappy team of, at its peak, 23 designers and 105 fabricators. Nonetheless, a modified version of the resulting plane ended up being operationally used as a trainer by the air force for 40 years.

For comparison: Boeing used more than 10,000 engineers to design the 777. And that was with the aid of computers and 3D modelling software. Kelly made diagrams by hand and calculations by slide rule. 

The Skunk Works culture was all about keeping things small and fast:  

Kelly loved to tell how a general named Frank Carroll was so enthusiastic hearing Kelly describe the speed and maneuverability of the new P-80, America’s first jet, which he had been pushing for, that Carroll decided to bypass all the red tape delays and do all the purchase order paperwork himself. ‘We came back from a quick lunch at two in the afternoon. He had an official letter of intent for me to start work on the P-80 drafted, approved, signed, and sealed in time for me to catch the 3:30 flight back to California’ Kelly said, chuckling delightedly every time he told that story. The same thing happened with the F104 Starfighter. General Bruce Holloway, who was then head of Strategic Air Command, was a colonel in procurement back in the 1950s, and listened to Kelly’s pitch about building a supersonic jet. Holloway needed to obtain a list of Air Force requirements to match Kelly’s performance description as the first step toward forwarding a contract for a prototype. ‘By God, Kelly, I’ll write it myself’ he declared in a blaze of enthusiasm. Kelly helped him draft it, and the two of them carried it up the chain of command to a general named Don Yates, who signed off on it. Total elapsed time: two hours.”

(p. 290)

They also built this prototype ship that I'm including here because I really like how dope it seemed, even though it never became more than a prototype:

As described by Kelly’s protege, Ben Rich: 

“Our ship had a four-man crew — commander, helmsman, navigator and engineer. By contrast, a frigate doing a similar job had more than three hundred crewmen. 

Viewed from head-on the ship looked like Darth Vader’s helmet. Some Navy brass who saw her clenched their teeth in disgust at the sight of the most futuristic ship ever to ply the seas. A future commander resented having only a four-man crew to boss around on a ship that was so secret that the Navy could not even admit it existed. Our stealth ship might [hypothetically] be able to blast out of the sky a sizable soviet attack force, but in terms of an officer’s future status and promotion prospects, it was about as glamorous as commanding a tugboat. At the highest levels, the Navy brass was equally unenthusiastic about the small number of stealth ships they would need to defend carrier task forces. Too few to do anyone’s career much good in terms of power or prestige.”

But their crowning achievement was a plane so legendary that Elon Musk literally named his child after it... 

(The plane was called A-12 or SR-71, which are two different iterations on basically the same model.)

Why was it so great? 

Despite being a military plane that was used to overfly hostile territory, it didn’t have any guns, flares, or other defense mechanisms. Because it was so damn fast that if anyone shot at it the standard defensive maneuver was to accelerate and fly faster. 

Its cruising speed was literally faster than a speeding bullet. It flew a mile in 1.5 seconds. That’s Los Angeles to Washington in 1h. In spite of being shot at over 4000 times in its history, it was never shot down, and I believe it’s the only US military plane for which this is true. It flew high enough that the Soviet fighters couldn’t even reach its altitude. They just helplessly flew up to 20,000 feet below, put in full afterbuner and pushed their planes hard enough that they permanently damaged their engines, and hopelessly fired their missiles. 

Despite being built in the 1960s without the aid of computers, it remains to this day the fastest air-breathing manned aircraft ever constructed. 

Yet some of these advances also made it harder for Lockheed to sell it (though there were also additional strategic reasons it wasn't mass produced). As described by a CIA engineer inside the Skunk Works:

The airplane pushed all of us to our limits in dealing with it. A pilot had to have tremendous self-confidence just to set foot inside the cockpit knowing he was about to fly two and half times faster than he ever had before. I know that Kelly was determined to spread the blackbird technology onto the blue-suiters and make the whole damned Air Force sit up and pay attention to what he had produced. But I never gave him much chance to sell a lot of those airplanes because they were so far ahead of anything else flying that few commanders would feel comfortable leading a Blackbird wing or squadron. I mean this was a twenty.-first-century performer delivered in the early 1960s. No one in the Pentagon would know what to do with it. 

At its peak there were 75 engineers working on its design. (At the peak of production there were 8000 workers producing one plane a month.)

The construction story itself is insane. They couldn’t use normal materials because at the speeds the SR-71 was flying, the air around it got hot enough that it would just melt normal metals used for the fuselage. So they chose to use titanium, even though no one had ever built a plane out of titanium before. They didn’t even know how to work it: it was strong enough that their tools would break just trying to cut it. They had to invent novel manufacturing methods just to work it, and teach those to their fabricators. And they had to do some crazy detective work to deal with new failure modes that cropped up: for example, after noticing that some titanium panels would suddenly fail after only six or seven weeks. They eventually realized that the titanium was super vulnerable to chlorine, and that those panels had been welded during July and August, after which they had to be washed, yet this time of year the local water system was heavily chlorinated to prevent algae growth. 

The aircraft went from idea to service in 20 months. For comparison, the US most recent fighter generation of fighter jet, the F35, started development in 1995 and only started full-rate production in 2021.  

Okay. So how did they do it? 

One electrical engineer on the project recounts the following story: 

In late 1962, I walked into Kelly Johnson's office one morning, introduced myself, and told him I was having a bit of trouble understanding the Skunk Works organization. When someone asked me a "What if" question I really did not know who was asking and how I should respond. Kelly went to the blackboard and drew the following org chart.

                                                          |KJ|

_____________________________|____________________________

|    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |

He said KJ, that's me, and all the people who work here work for me.

Moreover, Kelly did us an amazing favor, in explicitly writing down his 14 rules for management. They were as follows: 

1. The Skunk Works manager must be delegated practically complete control of his program in all aspects. He should report to a division president or higher.
2. Strong but small project offices must be provided both by the military and industry.
3. The number of people having any connection with the project must be restricted in an almost vicious manner. Use a small number of good people (10% to 25% compared to the so-called normal systems).
4. A very simple drawing and drawing release system with great flexibility for making changes must be provided.
5. There must be a minimum number of reports required, but important work must be recorded thoroughly.
   
   [The small teams were a key component here. In order to avoid the need for lengthy documentation, Kelly would enforce a rule to “Never put an engineer more than fifty feet from the assembly area”. (p. 222)]
    
6. There must be a monthly cost review covering not only what has been spent and committed but also projected costs to the conclusion of the program.
7. The contractor must be delegated and must assume more than normal responsibility to get good vendor bids for subcontract on the project. Commercial bid procedures are very often better than military ones.
8. The inspection system as currently used by the Skunk Works, which has been approved by both the Air Force and Navy, meets the intent of existing military requirements and should be used on new projects. Push more basic inspection responsibility back to subcontractors and vendors. Don't duplicate so much inspection.
9. The contractor must be delegated the authority to test his final product in flight. He can and must test it in the initial stages. If he doesn't, he rapidly loses his competency to design other vehicles.
   
   [JL: this contrasts with some contracts, where the air force asks the contractor to build the plane, but insists on only testing it themselves. Kelly would personally do user interviews with fighter pilots to obtain the specs for his planes, and would also insist on joining the test pilots to fly them himself, in order to iterate on the design.]
    
10. The specifications applying to the hardware must be agreed to well in advance of contracting. The Skunk Works practice of having a specification section stating clearly [which important military specification items will not knowingly be complied with and reasons] therefore is highly recommended.
11. Funding a program must be timely so that the contractor doesn't have to keep running to the bank to support government projects.
12. There must be mutual trust between the military project organization and the contractor, the very close cooperation and liaison on a day-to-day basis. This cuts down misunderstanding and correspondence to an absolute minimum.
13. Access by outsiders to the project and its personnel must be strictly controlled by appropriate security measures.
14. Because only a few people will be used in engineering and most other areas, ways must be provided to reward good performance by pay not based on the number of personnel supervised.

In addition to these rules, here’s how Kelly’s successor described what made the Skunk Works tick: 

We became the most successful advanced projects company in the world by hiring talented people, paying them top dollar, and motivating them into believing that they could produce a Mach 3 airplane like the SR-71 a generation or two ahead of anyone else. Our design engineers had the keen experience to conceive the whole airplane in their mind’s eye, doing the trade-offs in their heads between aerodynamic needs and weapons requirements. We created a practical and open work environment for engineers and shop workers, forcing the guys behind the drawing boards on to the shop floor to see how their ideas were being translated into actual parts and to make any necessary changes on the spot. We made every shop worker who designed or handled a part responsible for quality control. Any worker -- not just a supervisor or manager -- could send back a part that didn’t meet his or her standards. That way we reduced rework and scrap waste. 

We encouraged people to work imaginatively, to improvise and try unconventional approaches to problem solving, and then got out of their way. By applying the most common-sense methods to develop new technologies, we saved tremendous amount of time and money, while operating in an atmosphere of trust and cooperation both with our government customers and between our white-collar and blue-collar employees. In the end, Lockheed’s Skunk Works demonstrated the awesome capabilities of American inventiveness when free to operate under near ideal working conditions. That may be our most enduring legacy as well as our source of lasting pride. 

SpaceX

I also read “Liftoff: the desperate early days that launched SpaceX”, which chronicles the company from like 2002 to 2008, when they were still young, scrappy and closer to a size that I feel able to reason about. Elon is well known as a magnate: but what were things like when he had a 10-person team? 

SpaceX are also relevant because they work in hardware, in an industry usually requiring extremely large upfront capital investments (it used to be only governments who launched rockets), and they were started in a time where the Molochian decay in America’s ability to build was already far gone. Gwynne Shotwell, SpaceX COO, said that, roughly, trying to explain to government customers how and why spaceX used an iterative design philosophy was “one of the hardest things I’ve had to work on for almost my entire career at SpaceX” (p. 104).

Judging from speed alone, doing rocketry from scratch seems harder than building skyscrapers or planes. SpaceX built their in-house engine in about 3.5 years, successfully made their first launch in 5 years, and made orbit in 6 years. For comparison, Jeff Bezos Blue Origin was founded earlier, in 2000, and has still not reached orbit after twenty years [at least when the Liftoff book was published]. 

Here I have collected some excerpts that stood out to me from the book, on areas relevant to current questions I’m mulling over about org structure at Lightcone. 

Team size at various points

SpaceX hired their 14th employee after about one year (Elon had a personal assistant from the beginning). They also spend about a year on design before building their first prototype. 

After three years, close to them finishing their first engine and getting ready for the first full launch test, they had a team of 160 people total, with about 12-30 engineers, technicians, and managers working on the little atoll in the Marshall Islands that they launched from, and about 20 people at their McGregor Texas test site. (p. 76, p. 114, p.156) 

Decisions on the spot

Musk could be difficult to work  for. But his early hires could see the benefits of working for someone who wanted to get things done and often made decisions on the spot. When Musk decided [in 2002] that [a new fuel tank supplier] could make good tanks for a fair price, that was it. No committees. No reports. Just, done. (p. 19)

And, relatedly: 

As Shotwell gained Musk’s confidence, her role continued to expand. She managed customer interactions at first, but eventually added human resources, legal, and day-to-day operations of SpaceX to her portfolio. Her presence allowed Musk to focus where he could be most effective. On the days he works at SpaceX -- which have varied widely over the years given his multitude of projects, but generally amount to half of a given week -- Musk said he spends 80 to 90 percent of his time on engineering questions. This includes making design decisions, and optimizing the process by which SpaceX acquires parts from suppliers and builds its engines, rockets nd spacecraft. During meetings, Musk will make snap decisions. This is one of the key things that enables SpaceX to move so quickly. 

“I make the spending decisions and the engineering decisions in one head”, he said. “Normally those are at least two people. There’s some engineering guy who’s trying to convince a finance guy that this money should be spent. But the finance guy doesn’t understand engineering, so he can’t tell if this is a good way to spend money or not. Whereas I’m making the engineering and spending decisions. So I know, already, that my brain trusts itself.”

They were able to do much of the work themselves

Some of their senior engineers were also good enough electricians to wire and iterate on avionics circuits themselves. Others weren’t, but had to learn: 

“In the early Falcon 1 day we did a little bit of everything”, structures engineer Li said. “I learned how to use a rivet gun, and how to weld things together.” (p. 131)

How they dealt with life side-constraints

[COO Gwynne Shotwell] and her ex-husband each kept their two children for a week at a time. On weeks with her kids, Shotwell would arrive early at SpaceX, and leave around 6 or 7pm, taking over for the nanny at home. During other weeks, she could go crazy, working late into the night and traveling as much as needed. (p. 103)

[Musk’s senior VPs Mueller and Buzza], who had young children, lived dual lives. For ten days, they would work twelve-to-fourteen-hour shifts in McGregor before flying back to California, where typically they wold have Thursday through Sunday afternoon Off. (p. 149)

Another early engineer couldn’t move to Los Angeles because his wife had gotten a job at Google in SF that she really liked. So Musk spoke to Larry Page, who agreed that the engineer’s wife could be transferred to the Google LA office instead. 

Thompson [SpaceX second employee], with a young family, shared the same hesitation about leaving a comfortable position in the Aerospace industry. During a phone call in late April, Musk sought to all those concerns, He recognised what Thompson was walking away from, so he put two years’ worth of salary into an escrow account. If Musk decided to prematurely pull the plug on the venture, he would still have a guaranteed income. (p. 14)

On ownership 

[In order to work with Musk in 2006, you had to walk a middle line. That is,] Musk listened to ideas. He encouraged debate. He empowered his senior employees funding and authority. But always, he had the final say. (p. 126)

And a related Skunk Works quote: 

All of us at the Skunk Works knew the two basic rules for getting along with Kelly Johnson: all the airplanes we built were Kelly’s airplanes. Whatever pride we secretly took, we kept to ourselves. And if a blue-suiter wore star on his shoulder, only Kelly Johnson was authorised to deal with him. (p. 289)

---

[In 2006] Musk would convene his different teams in a small conference room, be it his engineers working on propulsion, or structures, or avionics, and run down the major issues. If an engineer faced an intractable problem, Musk wanted a chance to solve it. He would suggest ideas and give his teams a day or two to troubleshoot, then report back to him. In the interim, if they needed guidance, they were told to email Musk directly, day or night. He typically responded within minutes. (p. 19)

---

Musk had waited very nearly a full year for a second attempt, and he wanted to launch. He pressed [launch director] Tim Buxxa for why the rocket needed to be fully detanked to investigate the problem. Not safe, he was told. Could they just ignore the abort signal, reboot the rocket, and go for another try that day? Not if the sensor had identified a real issue that could threaten the mission’s success.

Buzza was the launch director. It was his call. He ordered the rocket emptied of fuel. “Elon was super upset”, Buzza said. “I suspect if he was in the control room on Kwah he would have had his way, but having him five thousand miles away gave me a little wiggle room to take more time to figure out the issue.” (p. 134)

They frequently paid a lot for fast shipping [LW · GW] 

Musk would sometimes lend his private jet for shipping parts. At its peak, they spent $500k and some political favors to rent a massive military cargo plane to ship their prototype rocket to the Marshall Islands, instead of shipping it by ship for a few weeks. 

According to Kevin Brogan, early SpaceX employee: 

Musk would say that everything we did was a function of our burn rate and that we were burning through a hundred thousand dollars per day. [...]

Sometimes he wouldn’t let you buy a part for two thousand dollars because he expected you to find it cheaper or invent something cheaper. 

Other times, he wouldn’t flinch at renting a plane for ninety thousand dollars to get something to Kwaj because it saved an entire workday, so it was worth it. 

For key roles, Elon hired experienced domain experts who were willing to have their thinking molded into his operational philosophy. For their teams, he hired younger generalists

After Musk hired a few experienced hands to lead his propulsion, structures, and avionics departments -- Thompson, Mueller and Koenigsmann -- he mostly brought on recent college graduates. [Those graduates had fewer side-constraints and were willing to work extremely hard.] (p. 25)

Tim Buzza, launch director: 

We brought some heritage aerospace experience, but also were willing to be totally molded by Elon to change our thinking. (p. 246)

Moreover, in order to get started with in-house manufacturing, they didn’t learn everything themselves. They began by using a sub-contractors a firm to build their engines, but when that firm went bankrupt SpaceX acquired their machines and brought the whole team in-house to start their machining operation (p. 41). 

What changed? 

Why was the ability to build lost? Some relevant quotes: 

Kelly Johnson’s protege at Skunk Works, Ben Rich: “Kelly’s stubbornness angered several important blue-suiters and a few of our own senior corporate executives. […] All of them demanded the same thing: that I accommodate our blue-suit customers and that I recognise the management’s responsibilities to keep a close eye on me as Kelly’s successor until I had a chance to prove myself. The name of the game was ‘get along and go along’ — no more tyrannical Kelly Johnson types. I understood, as did Kelly, that he was unique in his power and independence, which was nontransferable to any successor. For better or worse, a new era was dawning for those left behind in a Kelly Johnson-less world. The Skunk Works was still expected to produce giant results, but the new guy sitting in the boss’s chair would be a lot smaller than the original Gulliver.” (p. 293)

“In my forty years at Lockheed I worked on twenty-seven different airplanes. Today’s young engineer will be lucky to build even one.” (p. 316)

“Kassouf [early SpaceX employee, senior avionics engineer]  had a friend at Lockheed Martin who worked on F-35 stealth aircraft, a lucrative program for the company. Eventually, the air force would buy more than two thousand units at a cost of $85 million each. It may sound like glamorous work, but it was not. Kassouf’s friend had just a single job, finding a supplier from a bolt on the aircraft’s landing gear and ensuring that it met all quality specifications. That single bolt was the totality of his employment.” (p.23 Liftoff)

Note that the F-35 was built by the *same* Lockheed that also ran the Skunk Works! :( 

Here’s from Peter Thiel-affiliated defense startup Anduril (my model is that they’re sort of like Palantir, but for hardware) on changes in defense spending: 

It can be difficult to imagine the Department of Defense, which today employs nearly 3 million Americans, moving quickly. But last century, it innovated at a speed that puts modern Silicon Valley startups to shame: the Pentagon was built in only 16 months (1941–1943), the Manhattan Project ran for just over 3 years (1942–1946), and the Apollo Program put a man on the moon in under a decade (1961–1969). In the 1950s alone, the United States built five generations of fighter jets, three generations of manned bombers, two classes of aircraft carriers, submarine-launched ballistic missiles, and nuclear-powered attack submarines.

But, starting in the 1960s and intensifying through the 70s and 80s, the pace of military innovation began to slow, while costs grew. In 1955, it had become clear that the pace of defense spending was unsustainable: the federal government was spending more on defense than everything else combined.⁹ Congress knew something had to change. Robert S. McNamara, a wunderkind World War II veteran with a sterling reputation for administrative efficiency at Ford Motor Company, was the man for the job.

In the early 1960s, under the direction of then Secretary of Defense McNamara, the Department of Defense instituted a labyrinth of new rules for acquiring military systems. McNamara was not merely reducing spending: his experience at Ford led him to believe that comprehensive reform was needed to reshape how the government bought technology. He revamped acquisitions to emphasize efficiency, the elimination of waste, and predictability. The Planning, Programming, and Budgeting System (PPBS, later changed to PPBE for “Execution”) process that he implemented is arguably the single most influential defense reform ever enacted, indelibly altering the incentives and business models of the major defense contractors.

[...]

1. Adherence to a Lengthy Bureaucratic Process

Under the PPBS system, before the government even considers purchasing a new military system, it embarks upon a years-long process of defining requirements, deciding where and how to allocate resources, and finally releasing an award for a new system. It is extremely difficult for the Department of Defense to rapidly acquire new technology, and as a result defense companies face no pressure to develop new systems quickly. Technology developed in the commercial sector hence takes years or decades to end up on the battlefield, if it does at all.

2. Working off Onerous System Specifications

Reflecting the prevailing belief that industry was re- quired to execute, not innovate, to beat the Soviet Union, the requirements for major military programs are spelled out in extensive detail. Unlike most industries, which are driven forward by the creativity of the most successful companies, defense contractors are rarely asked to find creative ways to solve problems, and are sometimes punished for doing so.

3. Spending Little on Internal Research & Development

Because McNamara’s reforms made it exceptionally difficult to buy new technology quickly or to buy technology for which there is no defined requirement, defense companies rarely develop products of their own accord. Typically, they sell existing systems (which was most of their business under McNamara) or they wait for the government to order specific R&D efforts, for which they are directly compensated. For a comparison, the largest technology companies today — whose revenues vastly exceed those of the largest defense companies — spend roughly 10–20% of their revenue on research and development. Newer or mid-sized technology startups might spend closer to 60% or 70%. The major defense companies spend 1–4%.¹¹

Research and Development across industries.¹²

4. Prioritizing Proposals Over Performance

McNamara believed duplicative programs to be at the root of DoD waste and would not tolerate government spending on multiple development efforts for similar new systems, whether between military services or among different vendors. This means that once a company is awarded a major contract, it is extremely difficult to take it from them. The incentive for large defense firms is hence to spend heavily on teams of lawyers and lobbyists to shape program requirements in line with the company’s existing technology. This political battle becomes just as important as building the product itself. These lobbyists are often former military officials: By 1969, over 2,000 military officers per year were leaving the DoD to work for a major defense contractor, three times as many as in 1959.¹³

5. Tolerating Prolonged Failure

In most industries, a company that fails to produce a functional product goes out of business. In the defense industry, when a company is three, five, ten years into a program and has failed to build what they promised, the government is stuck between a rock and a hard place — do they cancel the contract and throw away years of development, potentially bankrupting the vendor in the process? Or do they begrudgingly provide the vendor with even more money to salvage the program? Typically, it’s the latter. In the words of the infamous city planner Robert Moses, “Once you sink that first stake, they’ll never make you pull it up.”¹⁴

26 comments

Comments sorted by top scores.