What's your viewpoint on the likelihood of GPT-5 being able to autonomously create, train, and implement an AI superior to GPT-5?

post by Super AGI (super-agi) · 2023-05-26T01:43:47.845Z · LW · GW · 1 comment

This is a question post.

Contents

  Answers
    3 Vladimir_Nesov
    3 meijer1973
    -21 Veniversum Vivus Vici
None
1 comment

If this seems unlikely, could you elaborate on the reasons? What essential capabilities would it lack in order to accomplish self-improvement? For instance:

- Enhanced Short-Term Memory: An extended token window
- Improved Long-Term Memory: The competence to modify or augment its own weights and training dataset
- Specific Resources: Access to a DGX data center for testing and training purposes
- Any other abilities?

In my perspective, GPT-4 already demonstrates respectable proficiency in code composition. However, it seems to fall short in short-term memory capacity, which is a crucial element for planning and testing associated with larger code blocks and projects, such as developing a new GPT. Thoughts?

Answers

answer by Vladimir_Nesov · 2023-05-26T23:30:56.132Z · LW(p) · GW(p)

There's AGI, autonomous agency at a wide variety of open-ended objectives, and generation of synthetic data, preventing natural tokens from running out, both for quantity and quality. My impression is that the latter is likely to start happening by the time GPT-5 rolls out. Quality training data might be even more terrifying than scaling, Leela Zero plays superhuman Go at only 50M parameters, so who knows what happens when 100B parameter LLMs start getting increasingly higher quality datasets for pre-training.

comment by meijer1973 · 2023-06-02T09:25:35.285Z · LW(p) · GW(p)

Became recently aware of the progress made in synthetic data and other algorithmic improvements. We have not pushed GPT-4 to the max yet. 

e.g. this paper https://arxiv.org/abs/2305.20050

It details how training on the steps in step by step reasoning as opposed to just rewarding the end result can give significant improvements. And there is so much more.  

comment by Super AGI (super-agi) · 2023-05-27T19:14:23.708Z · LW(p) · GW(p)

There's AGI, autonomous agency and a wide variety of open-ended objectives, and generation of synthetic data, preventing natural tokens from running out, both for quantity and quality. My impression is that the latter is likely to start happening by the time GPT-5 rolls out. 

 

It appears this situation could be more accurately attributed to Human constraints rather than AI limitations? Upon reaching a stage where AI systems, such as GPT models, can absorbed all human-generated information, conversations, images, videos, discoveries, and insights, these systems should begin to pioneer their own discoveries and understandings?

While we can expect Humans to persist (hopefully) and continue generating more conversations, viewpoints, and data for AI to learn from,  AI's growth and learning shouldn't necessarily be confined to the pace or scale of Human discoveries and data.  They should be capable of progressing beyond the point where Human contribution slows, continuing to create their own discoveries, dialogues, reflections, and more to foster continuous learning and training?

Quality training data might be even more terrifying than scaling, Leela Zero plays superhuman Go at only 50M parameters, so who known what happens when 100B parameter LLMs start getting increasingly higher quality datasets for pre-training.

Where would these "higher quality datasets" come from?  Do they already exist? And, if so, why are they not being used already?

Replies from: Vladimir_Nesov
comment by Vladimir_Nesov · 2023-05-27T20:06:47.634Z · LW(p) · GW(p)

Once AGI works, everything else is largely moot. Synthetic data is a likely next step absent AGI. It's not currently used for pre-training at scale, there are still more straightforward things to be done like better data curation, augmentation of natural data, multimodality, and synthetic datasets for fine-tuning (rather than for the bulk of pre-training). It's not obvious but plausible that even absent AGI it's relatively straightforward to generate useful synthetic data with sufficiently good models trained on natural data, which leads to better models that generate better synthetic data.

This is not about making progress on ideas beyond current natural data (human culture), but about making models smarter despite horrible sample efficiency. If this is enough to get AGI, it's unnecessary for synthetic data to make any progress on actual ideas until that point.

Results like Galactica (see Table 2 therein) illustrate how content of the dataset can influence the outcome, that's the kind of thing I mean by higher quality datasets. You won't find 20T natural tokens for training a 1T LLM that are like that, but it might be possible to generate them, and it might turn out that the results improve despite those tokens largely rehashing the same stuff that was in the original 100B tokens on similar topics. AFAIK the experiments to test this with better models (or scaling laws for this effect) haven't been done/published yet. It's possible that this doesn't work at all, beyond some modest asymptote, no better than any of the other tricks currently being stacked.

answer by meijer1973 · 2023-05-26T11:07:00.101Z · LW(p) · GW(p)

The biggest issue I think is agency. In 2024 large improvements will be made to memory (a lot is happening in this regard). I agree that GPT-4 already has a lot of capability. Especially with fine-tuning it should do well on a lot of individual tasks relevant to AI development. 

But the executive function is probably still lacking in 2024. Combining the tasks to a whole job will be challenging. Improving data is agency intensive (less intelligence intensive). You need to contact organizations, scrape the web, sift through the data etc. Also it would need to order the training run,  get the compute for inference time, pay the bills etc. These require more agency than intelligence. 

However, humans can help with the planning etc. And GPT-5 will probably boost productivity of AI developers. 

note: depending on your definition of intelligence, agency or the executive function would/should be part of intelligence. 

comment by Super AGI (super-agi) · 2023-05-27T18:55:08.060Z · LW(p) · GW(p)

The biggest issue I think is agency. In 2024 large improvements will be made to memory (a lot is happening in this regard). I agree that GPT-4 already has a lot of capability. Especially with fine-tuning it should do well on a lot of individual tasks relevant to AI development. 

But the executive function is probably still lacking in 2024. Combining the tasks to a whole job will be challenging. Improving data is agency intensive (less intelligence intensive). You need to contact organizations, scrape the web, sift through the data etc. Also it would need to order the training run,  get the compute for inference time, pay the bills etc. These require more agency than intelligence. 

 

Absolutely.  Even with GPT-4's constrained "short term memory", it is remarkably proficient at managing sizable tasks using external systems like AutoGPT or Baby AGI that take on the role of extensive "planning" on behalf of GPT-4.  Such tools equip GPT-4 with the capacity to contemplate and evaluate ideas -- facets akin to "planning" and "agency" -- and subsequently execute individual tasks derived from the plan through separate prompts.

This strategy could allow even GPT-4 to undertake larger responsibilities such as conducting scientific experiments or coding full-scale applications, not just snippets of code.  If future iterations like GPT-5 or later were to incorporate a much larger token window (i.e., "short-term memory"), they might be able to execute tasks, while also keeping the larger scale planning in memory at the same time? Thus reducing the reliance on external systems for planning and agency.

 

However, humans can help with the planning etc.  And GPT-5 will probably boost productivity of AI developers. 

Note: depending on your definition of intelligence, agency or the executive function would/should be part of intelligence. 

 

Agreed.  Though, communication speed is a significant concern.  AI-to-Human interaction is inherently slower than AI-to-AI or even AI-to-Self, due to factors such as the need to translate actions and decisions into human-understandable language, and the overall pace of Human cognition and response.

 

To optimize GPT-5's ability in solving complex issues quickly, it may be necessary to minimize Human involvement in the process.  The role of Humans could then be restricted to evaluating and validating the final outcome, thus not slowing down the ideation or resolution process?  Though, depending on the size of the token window, GPT-5 might not have the ability to do the planning and execution at the same time.  It might require GPT-6 or subsequent versions to get to that point.

Replies from: meijer1973
comment by meijer1973 · 2023-06-01T08:59:05.112Z · LW(p) · GW(p)

Agree, human in the loop systems are very valuable and probably temporary. HITL systems provide valuable data for training allowing the next step. AI alone is indeed much faster and cheaper. 

comment by Super AGI (super-agi) · 2023-05-29T22:30:56.420Z · LW(p) · GW(p)

The biggest issue I think is agency.

 

"Q: How do you see planning in AI systems?  How advanced are AI right now at planning?

A: I don't know it's hard to judge we don't have a metric for like how well agents are at planning but I think if you start asking the right questions for step by step thinking and processing, it's really good."

 

Replies from: meijer1973
comment by meijer1973 · 2023-06-01T10:48:57.208Z · LW(p) · GW(p)

Agency is advancing pretty fast. Hard to tell how hard this problem is. But there is a lot of overhang. We are not seeing gpt-4 at its maximum potential.

Replies from: super-agi
comment by Super AGI (super-agi) · 2023-06-01T23:41:04.987Z · LW(p) · GW(p)

Agency is advancing pretty fast. Hard to tell how hard this problem is. But there is a lot of overhang. We are not seeing gpt-4 at its maximum potential.

 

Yes, agreed.  And, it is very likely that the next iteration (E.g. GPT-5) will have many more "emergent behaviors".  Which might include a marked increase in "agency", planning, fossball, who knows... 

Replies from: meijer1973
comment by meijer1973 · 2023-06-04T10:12:09.742Z · LW(p) · GW(p)

People are finding ways to push the boundaries of the capabilities GPT-4 and are quite succesful at that (in reasoning, agency etc). These algorithmic improvements will probably also work on gpt5. 

A lot of infrastructure built for gpt4 will also work on gpt5 (like plug-ins). We do not need to build new plug-ins for gpt5, we just swap the underlying foundational model (greatly increasing the adoption of gpt5 compared to gpt4).

This also works for agency shells like autogpt. Autogpt is independant of foundational model (works with gpt3.5, gpt4 and also gpt5). By the time gpt5 is released these agency shells will be greatly improved and we just have to swap out the underlying engine to get al lot more oomph from that.

Same for memory models like vector databases. 

I think the infrastructure part will be a big difference. A year from now we will have a lot of applications, use cases, experience, better prompts etc. That could make the impact and speed of deployment of gpt5 (or Gemini) a lot bigger/faster than gpt4.

answer by Veniversum Vivus Vici · 2023-05-26T07:42:45.490Z · LW(p) · GW(p)

Creating an AI that could autonomously design, train, and implement a superior version of itself is a concept referred to as recursive self-improvement or AI bootstrapping. While this is a fascinating idea and a topic of much discussion in AI research, it is a difficult task with many challenges and risks.

Let's consider the capabilities you mentioned:

Enhanced Short-Term Memory (Extended Token Window): This is an issue of architecture. In principle, GPT-5 could include such improvements, and they could be beneficial. However, a larger token window would significantly increase computational requirements, and it's unclear how much benefit this would actually provide for the specific task of developing a superior AI.

Improved Long-Term Memory (Modifying Its Own Weights and Training Dataset): AI models such as GPT-4 or hypothetical GPT-5 do not have the ability to modify their own weights or training dataset. This ability would require a very different architecture. For the task of designing a superior AI, the model would need to understand the complex relationship between the model's weights and its performance, which is a task that's currently beyond the capabilities of AI. Even if the AI had this capability, training AI models is a resource-intensive task that requires specific hardware resources and infrastructure.

Specific Resources (Access to a DGX data center): Even if an AI had access to such resources, it would still need to understand how to use them effectively, which would require capabilities beyond what GPT-4 or a hypothetical GPT-5 have.

Code Composition: While GPT-4 can indeed generate code, the task of generating code to train a superior AI is far more complex. It involves a deep understanding of AI architectures, algorithms, and principles, as well as the ability to invent new ones. Even for human AI researchers, creating a superior AI model is a significant challenge that requires years of study and expertise.

In addition to these points, there's also the problem of evaluation. Even if an AI could generate a new AI architecture and train it, it would still need to evaluate the new AI's performance and make decisions about how to improve it. This requires an understanding of AI performance metrics and the ability to interpret them, which is another complex task that current AI models are not capable of.

Furthermore, it's worth noting that creating an AI that can improve itself poses significant ethical and safety concerns. Without careful safeguards and oversight, such an AI could potentially lead to unwanted or even dangerous outcomes.

In conclusion, while the idea of an AI improving itself is theoretically possible and an interesting research direction, it's currently beyond the capabilities of current AI technology, including GPT-4 and a hypothetical GPT-5. Achieving this goal would likely require significant advances in AI architectures, algorithms, and understanding of AI principles, as well as careful consideration of ethical and safety issues.

comment by Super AGI (super-agi) · 2023-05-27T19:17:50.998Z · LW(p) · GW(p)

Thanks GPT-4. You're the best!  

Veniversum Vivus Vici, do you have any opinions or unique insights to add to this topic?

comment by meijer1973 · 2023-06-06T09:00:11.942Z · LW(p) · GW(p)

Specific Resources (Access to a DGX data center): Even if an AI had access to such resources, it would still need to understand how to use them effectively, which would require capabilities beyond what GPT-4 or a hypothetical GPT-5 have.

To my knowledge resource management in data centers is done by AI's. It is the humans who cannot do this. The AI already can.

1 comment

Comments sorted by top scores.

comment by [deleted] · 2023-06-09T02:19:00.864Z · LW(p) · GW(p)

this question is confusing to me due to being about 'GPT-5.' openAI isn't currently training a 'GPT-5', so the referent is sort of undefined. an AI trained by openAI that they call 'GPT-5' might be a lot more powerful if trained 5 years from now, than 1 year from now, for example.

one interpretation could be that it's asking about both, 'when will openAI develop GPT-5', and also 'when will AIs be capable enough to create more capable AIs', but i think this probably isn't your intent.