Strategic Thinking: Paradigm Selection

My quite simplistic understanding is that (a) yes, there are already existing solutions, but (b) the people providing those solutions will charge you a lot of money, especially if you later become dependent on them; and you still need to check their work, and they may disagree with you on some details because at the end of the day they are optimizing for themselves, not for you.

Doing things yourself requires extra time and energy, but the money which would otherwise become someone else's profit now stays in your pockets. Essentially, as soon as you feel reasonably sure that the project will be successful, getting rid of each subcontractor means increasing your profit. You don't need to become an expert on everything, you can still hire the experts, but now they are your employees working for a salary, instead of a separate company optimizing for their own profit.

Not sure how realistic this is, but if you imagine that even a typical successful company somewhat resembles the Dilbert comic, then if you can build your own company better, you can just take over their people who do the actual work, and stop feeding the remaining ones.

EDIT: I don't have an experience running a company, but I am thinking about a friend who recently reconstructed his house. His original thoughts were "I am a software developer, this is my competitive advantage, so I will just pay the people who are experts on house construction", but it turned out that the real world doesn't work this way. Most of the so-called experts were quite incompetent, and he had to do a lot of research in their field of expertise just to be able to tell the difference. When the reconstruction was over, he already felt like he could start a new profession and do better than most of these experts. In this case, however, those experts were typically sole proprietors. If instead they would have been companies renting the experts, and if my friend would be in some kind of a business of repeatedly reconstructing houses, it would make sense for him to start optimizing the details seriously, and build replacements of what exists out there.

I agree with this response; using first principles is a heuristic, and heuristics always have pros and cons. Just in terms of performance, the benefit is that you can re-assess assumptions but the cost is that you ignore a great amount of information gathered by those before you. Depending on the value of this information, you should frequently seek it out, as least as a supplement to your derivation.

Unfortunately, for people who are not members of the inner circle this kind of optimization may be indistinguishable from mere incompetence, or malice. Do we produce sloppy code? Maybe delivering the code fast is more important than the code quality. Do we have an incompetent person on the team? Maybe he or she is a relative of someone important, and it is very important to gain a favor from that person. Did we actually deliver the sloppy code late? Maybe the delay way strategic somehow; maybe the company is paid by hour so delivering the product late was used as an excuse to extract more money from the customer; or maybe it made them more dependent for us; or maybe it was somehow strategically important to deliver it on Thursday. Is the company financially in loss? Maybe the key people are actually transferring company money to their private accounts, so everything goes according to the plan.

I don't know where is the balance between understanding that there may be some higher strategy that I am not aware of, and simply blindly trusting the authorities (it is easy to rationalize the latter as the former). I guess it is important to notice that the "higher strategy" is not necessarily optimizing in my favor, so in some sense from my point of view there sometimes needs not to be a difference between "it is all going to hell" and "it all goes according to the plan, but a part of the plan is sacrificing me". That means that unless I trust the secret wisdom and benevolence of the people behind the wheel, I should treat all apparent dysfunction as potentially bad news.

I am now writing an article where I explore this type of solutions.

One similar to ones you listed is to sell AI Safety as a service, so any other team could hire AI Safety engineers to help align their AI (basically it is a way to combine the tool and the way to deliver it.)

Another (I don't say it is the best, but possible) is to create as many AI teams in the world as possible, so hard-takeoff will always happen in several teams, and the world will be separated on several domains. Simple calculation shows that we need around 1000 AI teams running simultaneously, to get many fooms. In fact actual number of AI startups, research groups and powerful individual is around 1000 now and growing.

There are also some other ideas, hope to publish draft here on LW next month.