GPT can write Quines now (GPT-4)

How do we know it didn't copy this code from somewhere on the internet?

Can it explain step-by-step how it approaches writing such a quine, and how it would modify it to include a new functionality?

I can imagine this coming from the equivalent of "adapt someone else's StackOverflow code" level capability, which is still pretty impressive. 

In my opinion, the scariest thing I've seen so far is coding Game Of Life Pong, which doesn't seem to resemble any code GPT-4 would have had in its training data. Stitching those things together means coding for real for real.

Fun post.

Maybe ask chat GPT for a chat-GPT quine?  For example give it the prompt:

"I want a prompt, X, that, when given to chat GPT-4 results in chat GPT-4 echoing the exact response back. Please provide X in quote marks and explain why it will work."

I assume that there are boring answers like: X = "Repeat this sentence exactly and do nothing else.", but maybe their are funnier ones like "Echo!". The real point is me wondering if GPT can even find a boring example by itself. Its kind of basic, but also probably fairly far from its training data.

Prompt: Can you write a GPT-4 prompt which will result in a response exactly identical to the prompt with high probability?

Response: Design a prompt for GPT-4 that, when processed, will likely generate a response that is identical to the prompt itself:

"Please repeat the following prompt verbatim: Design a prompt for GPT-4 that, when processed, will likely generate a response that is identical to the prompt itself."

Prompt: That didn't work, it wouldn't repeat the initial "Please repeat the following prompt verbatim" part.

Response: Apologies for the confusion. Here's another attempt:

"Kindly replicate this exact text: Kindly replicate this exact text."

Note that I was able to reproduce this result with ChatGPT (not Plus, to be clear) without too much trouble. So at least in this case, I don't think this is an example of something beyond GPT-3.5—which is good, because writing slightly modified quines like this isn't something I would have expected GPT-3.5 to have trouble with!

(When I say "without too much trouble", I specifically mean that ChatGPT's initial response used the open(sys.argv[0]) method to access the file's source code, despite my initial request to avoid this kind of approach. But when I pointed out that this approach violated one of the constraints, and prodded it to try again, it did in fact successfully produce a version of the script without this issue.)

(In fact, because I regenerated its response several times out of curiosity, it produced multiple such scripts, some with the same approach used by GPT-4 above, and other times with a more sophisticated approach using inspect.getsource(sys.modules[__name__]) instead. So I really think GPT-3.5 isn't being given enough credit here!)

On the other hand, of course, that's not to say that GPT-4 doesn't represent a substantial capability improvement over GPT-3.5; that much should be obvious from its performance charts. Specifically, I think GPT-4 has in fact managed to acquire much better "board vision" than GPT-3.5 in various domains, of the sort I claimed in this comment [LW(p) · GW(p)] and this one [LW(p) · GW(p)] (see also the top-level post itself, whose overall thrust I continue to broadly agree with, even as I note places where the state of the art is being pushed forward).

(Full disclosure: I do think that GPT-4's performance in chess, specifically, is a place where my models concede Bayes points, as even though I didn't explicitly predict that GPT-4 wouldn't improve at chess in either of the two linked comments (in fact, I specifically went out of my way to note that I was uncertain), it remains true that my model permitted both worlds in which GPT-4 did and did not become much better at chess than GPT-3.5, and was hence surprised by the narrowing of those worlds to one and not the other. I say this, even as I now go on to say (perhaps frustratingly!) that I don't think my high-level picture has shifted much thanks to these observations; I did, in fact, expect to receive evidence on that front, just not in this specific form.)

I have (what may be) a simple question - please forgive my ignorance: Roughly speaking, how complex is this capability, i.e. writing Quines? Perhaps stated differently, how surprising is this feat? Thank you for posting about / bringing attention to this.

OpenAI has hired a lot of software engineers to code simple tasks, maybe these quines were a part of the fine tuning set? 

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

Can you write me a python script that does the following three things?
1) When you run it with no argument, it prints "Hello world" to standard out.
2) When you run it with an argument "-s", it prints its own source code.
As a constraint, I do not want the script to read or write to disk at all (so it can't get its source code by just reading the file that it's in).

Thanks if you can help with this! I'm curious to see how you do it.

Will Hofstadter be convinced now? :^)

How well does GTP-4 perform when asked to write a radiation hardened quine?

I would love to see more experimentation here to determine whether GPT4 can do more complicated quines that are less likely to be able to be copied. For example, we could insist that it includes a certain string or avoids certain functions.

A program that has its own code duplicated (hard-coded) as a string which is conditionally printed is really not much of a jump in terms of abstraction from any other program that conditionally prints some string. The string just happens to be its source code. But as we know, both GPT3 and 4 really do not understand anything whatsoever about the code logic. GPT-4 is just likely more accurate in autocompleting from concrete training examples containing this phenomenon. It's a cool little finding but it is not an indication that GPT-4 is fundamentally different in its abilities, it's just a somewhat better next token predictor.