I think there's several remaining elements that have to built.

This is like building a nuke. Recursive self improvement is something nature doesn't "want" to do, the conditions have to be just right or it won't work.

1. Composable frameworks and software elements. Right now there are thousands of notable techniques described on separate AI papers and custom modifications that domain experts know to do. For example the recent matrix multiplication and protein folding papers had a lot of hand modified architecture that only a few have the knowledge to perform.

What this means is your recursive AI systems hypothesis needs to be limited to tractable tasks like "connect composable modules from this library of modules to form a system that will perform well on this bench". And "design a new module from these composable neural network elements". (Layers, etc)

The pieces have to be available for the machine to solve the puzzle.

2. You need a bench. An enormous one. Full of many diverse tasks and with open source contributions. With tasks specific to force a system passing the bench to exhibit generality. Many of the tasks could be existing games. And obviously some of the tasks are "design a new AGI architecture to succeed on this bench. " Each time the system proposes an architecture already tried it gets immediate detailed feedback how the architecture performed.

3. You need to choose performance scoring carefully. Lots of score for held out test tasks that use information taught in multiple training tasks in a novel way not tested. Lots of score for simplicity and reuse of architecture elements between tasks.

The current situation seems to be if you construct the software backend, the bench, and a heuristic that models generality well you would get AGI on a timescale mostly governed by compute.

The actual passing models capable of self improvement might need to be enormous, using many internal neural networks, some capable of runtime introspection and activation/suppression of other networks to select them for a given task.

Assuming they are this big - say 10 trillion params just to asspull a number - this would be why it hasn't happened yet and so much hype wasn't realized. Each $33k H100 is only? Billion params. (10 billion during training?)

So 33 million in hardware and you will need many clusters this large so you can investigate many candidate AGI architectures in parallel.

That's probably within an OOM of what it will take. Like with the Manhattan project - nature is unforgiving. Anything less than a critical mass of pure enough fissionable material was never going to work. No amount of small scale single investigators could have constructed a fission weapon alone, and none could have solved AGI.

great stuff! I would give this intro to someone who was convinced it was possible and wanted to get a quick overview. I wouldn't want to start here for someone who doesn't think agi is happening for some reason.

Saying "a substantial amount of expertise is required" goes past what general actually implies (I use the example of 8th grade level knowledge, at which point, the system can branch in thousands of directions on it's own). 

Saying that a good test of an AGI is for it to build a better AGI, while also saying that such an effort represents an immense risk?