LessWrong 2.0 Reader

Could Billion-Dollar Test Run Unlock Superhuman AI?

- edited from a draft by Gemini Flash Thinking 2.0 Experimental 

We hear about billion-dollar investments in massive datasets and colossal neural networks, all in the pursuit of imbuing AI models with broad knowledge and general capabilities. Imagine a new paradigm: use massive test-time compute to improve the test time algorithm itself.

Initial explorations suggest that while such test-time scaling might be computationally expensive, e.g. O3 high requiring thousands of GPU hours to solve a single Arc-AGI problem, it appears to be remarkably free from data constraints. This is a critical distinction. Traditional pre-training is fundamentally limited by the availability and quality of data. Test-time scaling, however, seemingly allows for continuous improvement simply by throwing more compute at the problem.

This unlocks a new possibility: instead of spending billions on a single, massive pre-training run to build a generally intelligent model, organizations like OpenAI could potentially invest billions, or even tens of billions, in test-time computation. This wouldn't be for serving customers or just general knowledge acquisition, but rather for discovering AI algorithm breakthroughs that even the most brilliant human engineers might miss. Imagine dedicating immense computational resources – perhaps a million GPUs or more – to solving a single, critical algorithmic challenge that can drastically improve the efficiency of the whole system. First invest astronomically to obtain an extremely inefficient ASI which runs at a prohibitive high cost for general applications, then use it to self improve efficiency until the cost becomes practical for general applications. 

This approach hinges on the idea that while current "O3 high" configurations may be inefficient in terms of compute per unit of intelligence gained, the sheer scalability of computation can compensate. Even if solving an AI development problem at a super-intelligence level with current methods is slow and requires immense resources, the potential payoff could be transformative. Assuming O3 high can be scale up another 4 OOM more compute to reach ASI level intelligence, then Open AI just need to acquire 4 OOM more GPUs to use it to solve algorithm problems, which is around 100 million GPUs. However if they manage to improve the algorithm efficiency by 2 OOM, which is reasonably achievable within 2 years, then they just need 1 million GPUs - roughly 25- 50 billion dollars. 

This perspective lends credence to recent pronouncements from figures like Dario Amodei, CEO of Anthropic, who has spoken about the necessity of labs needing millions of GPUs in the coming years. While the immediate interpretation might be for ever-larger pre-training runs, it's equally plausible that this massive compute infrastructure is envisioned for this very purpose: billion-dollar test-time runs dedicated to solving the most pressing AI development challenges at a level of intelligence exceeding human capacity.

The implications of this approach are profound. It suggests a potential acceleration in AI progress, driven not by data accumulation, but by the strategic deployment of massive computational power. It opens up the possibility of using AI itself to solve the most complex problems in AI development, leading to more efficient algorithms, novel architectures, and ultimately, faster progress towards efficient AGI. While the efficiency of "O3 high" test-time scaling remains a concern, the tantalizing prospect of data-unconstrained intelligence amplification through massive compute offers a compelling and potentially game-changing direction for the future of AI.

There are public examples. These ones are famous because something went wrong, at least from a security perspective. Of course there are thousands of young adults with access to sensitive data who don't become spies or whistleblowers, we just don't hear about them.

• Theodore Hall, who worked at age 18 on the Manhatten Project.
• Edward Snowden who worked from age 22 for the NSA.
• Chelsea Manning who worked from age 22 as a US Army Intelligence Analyst.

Self-coordination might also be easier if AIs can recognize themselves by being good at recognizing features that survive paraphrasing and that vary very consistently between answers

Yeah, this is the case I'm thinking about. I think on longer outputs it will be possible for the AI to use a more nebulous sense of "style", rather than only relying on specific features like "using X secret phrase" (which are presumably easier to redact away).

I think this could be countered by paraphrasing at a higher level - i.e. rewriting entire paragraphs from scratch rather than just rewording sentences. I'm noting this down as a potential topic if I have time to do another project.

Thanks for the discussion, it's helped me clear up my thoughts on this topic.

I do see some security risk.

Although Trump isn't spearheading the effort I expect he will have access to the results.

Looks like this is actually a link to their second talk on May 4th 2009. Not sure if the first talk still exists.

Neither of them is exactly what you're looking for, but you might be interested in lojban, which aims to be syntactically unambiguous, and Ithkuil, which aims to be extremely information-dense as well as to reduce ambiguity. With regards to logical languages (ones which, like lojban, aim for each statement to have a single possible interpretation), I also found Toaq and Eberban just now while looking up lojban, though these have fewer speakers.

I think you're assuming these minds are more similar to human minds than they necessarily are. 

I don't assume similarity to human minds so much as you assume universal dissimilarity.
 

its not something were even aiming for with current alignment attempts.

Indeed

oh not at all, I think I'm failing on both fronts

I see. Well, in case one of them reads this: to the extent it might matter to you, as a fellow radical vegan I request you not kill anyone trying to figure out how to make the first takeover-capable AI be benevolent.

(I'm aware that humans are such that it's not obvious that designer-chosen-alignment → benevolence, and that one does actually have to reason about that instead of just assuming the hopeful option, but after having reasoned about it, I think most would choose benevolence in the end.^[1] At least, I do not expect important exceptions to be alignment researchers, as opposed to selected-for power-seekers in high positions.)

1. ^{^}

   It seems to me that actually reflectively endorsing evil (objectively describing what's happening and then being like "this is good", instead of euphemizing and coping with arguments that often secretly amount to "if you're right then the world would be really bad, and that's scary" like "but nature has it") is rare, and most people are actually altruistic on some level but conformism overrides that and animal abuse is so normal that they usually don't notice it, but then when they see basic information like "cows have best friends and get stressed when they are separated" they seemingly get empathetic and reflective (check the comments).

I disagree but not confident I could write an explanation that's both legible and not losing lots of info by simplifying into "oppressed people more likely to want to oppose oppression". When I saw the question I was looking forwards to you writing a good answer to it, actually. To hint at some starting points, why is queer anarchism a thing? How do different minds decide who they are?