Max H's Shortform

Using shortform to register a public prediction about the trajectory of AI capabilities in the near future: the next big breakthroughs, and the most capable systems within the next few years, will look more like generalizations of MuZero and Dreamer, and less like larger / better-trained / more efficient large language models.

Specifically, SoTA AI systems (in terms of generality and problem-solving ability) will involve things like tree search and / or networks which are explicitly designed and trained to model the world, as opposed to predicting text or generating images.

These systems may contain LLMs or diffusion models as components, arranged in particular ways to work together. This arranging may be done by humans or AI systems, but it will not be performed "inside" a current-day / near-future GPT-based LLM, nor via direct execution of the text output of such LLMs (e.g. by executing code the LLM outputs, or having the instructions for arrangement otherwise directly encoded in a single LLM's text output). There will recognizably be something like search or world modeling that happens outside or on top of a language model.

--

The reason I'm making this prediction is, I was listening to Paul Christiano's appearance on the Bankless podcast from a few weeks ago.

Around the 28:00 mark the hosts ask Paul if we should be concerned about AI developments from vectors other than LLM-like systems, broadly construed.

Paul's own answer is good and worth listening to on its own (up to the 33 minute mark), but I think he does leave out (or at least doesn't talk about it in this part of the podcast) the actual answer to the question, which is that, yes, there are other avenues of AI development that don't involve larger networks, more training data, and more generalized prediction and generation abilities.

I have no special / non-public knowledge about what is likely to be promising here (and wouldn't necessarily speculate if I did); but I get the sense that the zeitgeist among some people (not necessarily Paul himself) in alignment and x-risk focused communities, is that model-based RL systems and relatively complicated architectures like MuZero have recently been left somewhat in the dust by advances in LLMs. I think capabilities researchers absolutely do not see things this way, and they will not overlook these methods as avenues for further advancing capabilities. Alignment and x-risk focused researchers should be aware of this avenue, if they want to have accurate models of what the near future plausibly looks like.