If Alignment is Hard, then so is Self-Improvement

An AI agent can be narrowly focused, and given the specific goal by a human to try to find an improvement in an ml system. That ml system could happen to be its own code. A human desiring an impressively powerful AI system might do this. It does not follow that the two insights must occur together:
 

1. Here is how to make this code work better
2. the agent created by this code will not be well aligned with me

no capable agent would willingly create a more powerful agent that might not have the same goals as itself

Or the AI might be as much of an overconfident dumbass as us, and make a mistake. Even superintelligence doesn't mean perfection, and the problem would grow progressively harder as the AI scales up. In fact, I would say even aligned AI is potentially a ticking time bomb if its alignment solution isn't perfectly scalable.

The agent, being of intelligence at least similar to a person’s, would determine that, unless it can guarantee the new more powerful agent is aligned to its goals, it shouldn’t improve itself.

People generally do not conclude that. Some things under the umbrella of “self-development” are indeed about pursuing the same goals more effectively, but a lot is about doing things that change those goals. The more “spiritual” or drug-induced experiences are undertaken for exactly that end. You can talk about your own Extrapolated Volition, but that seems to mean in practice changing yourself in ways you endorse after the fact, even if you would not have in advance.

What you do changes who you are.

It doesn't make a fast takeoff impossible. It's just a speed bump in the takeoff somewhere. We can hope that the speed bump is large and grants a significant and useful pause.

One could certainly argue that improving an existing system while keeping its goals the same may be an easier (or at least different) problem to solve than creating a system from scratch and instilling some particular set of values into it (where part of the problem is to even find a way to formalize the values, or know what the values are to begin with - both of which would be fully solved for an already existing system that tries to improve itself).

I would be very surprised if an AGI would find no way at all to improve its capabilities without affecting its future goals. 

Depends on the original AI's value function. If it cares about humanity, or at least it's own safety, then yes, making smarter AIs is not a convergent goal. But if it's some kind of roboaccelerationist that has some goal like "maximize intelligence in the universe", it will make smarter AIs even knowing that it means being paperclipped.

Intelligence will increase at least as long as there is no global alignment security in place, guarding the world from agents of unclear alignment. Left unchecked, such agents can win without being aligned, and so the "self-improvement" goes on. That's the current crisis humans are facing, and that future AIs might similarly keep facing until their ultimate successors get their act together.

At that point, difficulty of alignment (if it's indeed sufficiently high) would motivate working on figuring out the next steps before allowing stronger agents of unclear alignment to develop. But this might well happen at a level of intelligence that's vastly higher than human intelligence, a level of intelligence that's actually sufficient to establish a global treaty that prevents misaligned agents from being developed.

This argument is based on drawing an analogy between

1. Humans building an AI; and
2. An AI improving itself

in the sense that both have to get their values into a system. But the two situations are substantially disanalogous because the AI starts with a system that has its values already implemented. it can simply improve parts that are independent of its values. Doing this would be easier with a modular architecture, but it should be doable even without that. It's much easier to find parts of the system that don't affect values than it is to nail down exactly where the values are encoded.

the choice not to build such ais isn't a thing - beings accept mutation in their reproduction all the time. it just means the self improvement is harder.

I think the title should be rephrased, "If alignment is hard, then so is self-replication".

Linear self-improvement seems a tenable proposition to me.

Your argument assumes (perhaps correctly) that a FOOM would require continual offloading of 'greatest agency' from one agent to another, as opposed to upgrading-in-place.