Claude 3.5 Sonnet

IMO if any lab makes some kind of statement or commitment, you should treat this as "we think right now that we'll want to do this in the future unless it's hard or costly", unless you can actually see how you would sue them or cause a regulator to fine them if they violate the commitment. This doesn't mean weaker statements have no value.

I'm disappointed that there weren't any non-capability metrics reported. IMO it would be good if companies could at least partly race and market on reliability metrics like "not hallucinating" and "not being easy to jailbreak".

Edit: As pointed out in reply, addendum contains metrics on refusals which show progress, yay! Broader point still stands, I wish there were more measurements and they were more prominent.

It's quite possible that anthropic has some internal definition of "not meaningfully advancing the capabilities frontier" that is compatible with this release. But imo they shouldn't get any credit unless they explain it.

VN: Write a program that by default prints a number from the Fibonacci sequence, and on input --next prints a similar program that by default prints the next number in the sequence, and so on.

Claude 3.5 Sonnet: Here's a Python program that meets your requirements:

import sys

# Fibonacci sequence values
a, b = 0, 1

# The current Fibonacci number to print
current = a

# Program content template
program_template = '''import sys

# Fibonacci sequence values
a, b = {}, {}

# The current Fibonacci number to print
current = a

# Program content template
program_template = {!r}

def main():
    if len(sys.argv) > 1 and sys.argv[1] == "--next":
        # Generate the next program
        next_program = program_template.format(b, a + b, program_template)
        print(next_program)
    else:
        # Print the current Fibonacci number
        print(current)

if __name__ == "__main__":
    main()
'''

def main():
    if len(sys.argv) > 1 and sys.argv[1] == "--next":
        # Generate the next program
        next_program = program_template.format(b, a + b, program_template)
        print(next_program)
    else:
        # Print the current Fibonacci number
        print(current)

if __name__ == "__main__":
    main()

This program does the following:

1. It initializes the first two Fibonacci numbers (a and b) and sets the current number to be printed.
2. It defines a program_template which is a string representation of the program itself, with placeholders for the Fibonacci values.
3. The main() function checks if the --next argument is provided:
   • If --next is provided, it generates the next program by formatting the program_template with the next Fibonacci values.
   • If no argument is provided, it simply prints the current Fibonacci number.
4. The program runs the main() function when executed.

Here's how you can use this program:

1. Save it to a file, let's say fibonacci_generator.py.
2. Run it without arguments to print the current Fibonacci number:

python fibonacci_generator.py

This will print 0.

3. Run it with the --next argument to generate the next program:

python fibonacci_generator.py --next > next_fibonacci.py

This will create a new file next_fibonacci.py with the next program.

4. You can then run next_fibonacci.py to get the next Fibonacci number, or use --next again to generate the next program in the sequence.

Each generated program will print the next number in the Fibonacci sequence by default, and can generate a program for the subsequent number when run with --next.

It's pretty good. I tried it on a few mathematical questions.

First of all, a version of the standard AIW problem from the recent "Alice in Wonderland" paper. It got this right (not very surprisingly as other leading models also do, at least much of the time). Then a version of the "AIW+" problem which is much more confusing. Its answer was wrong, but its method (which it explained) was pretty much OK and I am not sure it was any wronger than I would be on average trying to answer that question in real time.

Then some more conceptual mathematical puzzles. I took them from recent videos on Michael Penn's YouTube channel. (His videos are commonly about undergraduate or easyish-olympiad-style pure mathematics. They seem unlikely to be in Claude's training data, though of course other things containing the same problems might be.)

One pretty straightforward one: how many distinct factorials can you find that all end in the same number of zeros? It wrote down the correct formula for the number of zeros, then started enumerating particular numbers and got some things wrong, tried to do pattern-spotting, and gave a hilariously wrong answer; when gently nudged, it corrected itself kinda-adequately and gave an almost-correct answer (which it corrected properly when nudged again) but I didn't get much feeling of real understanding.

Another (an exercise from Knuth's TAOCP; he rates its difficulty HM22, meaning it needs higher mathematics and should take you 25 minutes or so; it's about the relationship between two functions whose Taylor series coefficients differ by a factor H(n), the n'th harmonic number) it solved straight off and quite neatly.

Another (find all functions with (f(x)-f(y))/(x-y) = f'((x+y)/2) for all distinct x,y) it initially "solved" with a solution with a completely invalid step. When I said I couldn't follow that step, it gave a fairly neat solution that works if you assume f is real-analytic (has a Taylor series expansion everywhere). This is also the first thing that occurred to me when I thought about the problem. When asked for a solution that doesn't make that assumption, it unfortunately gave another invalid solution, and when prodded about that it gave another invalid one. Further prompting, even giving it a pretty big hint in the direction of a nice neat solution (better than Penn's :-)), didn't manage to produce a genuinely correct solution.

I rate it "not terribly good undergraduate at a good university", I think, but -- as with all these models to date -- with tragically little "self-awareness", in the sense that it'll give a wrong answer, and you'll poke it, and it'll apologize effusively and give another wrong answer, and you can repeat this several times without making it change its approach or say "sorry, it seems I'm just not smart enough to solve this one" or anything.

On the one hand, the fact that we have AI systems that can do mathematics about as well as a not-very-good undergraduate (and quite a bit faster) is fantastically impressive. On the other hand, it really does feel as if something fairly fundamental is missing. If I were teaching an actual undergraduate whose answers were like Claude's, I'd worry that there was something wrong with their brain that somehow had left them kinda able to do mathematics. I wouldn't bet heavily that just continuing down the current path won't get us to "genuinely smart people really thinking hard with actual world models" levels of intelligence in the nearish future, but I think that's still the way I'd bet.

(Of course a system that's at the "not very good undergraduate" level in everything, which I'm guessing is roughly what this is, is substantially superhuman in some important respects. And I don't intend to imply that it doesn't matter whether Anthropic are lax about what they release just because the latest thing happens not to be smart enough to be particularly dangerous.)

Claude 3.5 Sonnet solves 64% of problems on an internal agentic coding evaluation, compared to 38% for Claude 3 Opus. Our evaluation tests a model’s ability to understand an open source codebase and implement a pull request, such as a bug fix or new feature, given a natural language description of the desired improvement.

...

While Claude 3.5 Sonnet represents an improvement in capabilities over our previously released Opus model, it does not trigger the 4x effective compute threshold at which we will run the full evaluation protocol described in our Responsible Scaling Policy (RSP).

Hmmm, maybe the 4x effective compute threshold is too large given that you're getting near doubling of agentic task performance (on what I think is an eval with particularly good validity) but not hitting the threshold. 

Or maybe at the very least you should make some falsifiable predictions that might cause you to change this threshold. e.g., "If we train a model that has downstream performance (on any of some DC evals) ≥10% higher than was predicted by our primary prediction metric, we will revisit our prediction model and evaluation threshold." 

It is unknown to me whether Sonnet 3.5's performance on this agentic coding evaluation was predicted in advance at Anthropic. It seems wild to me that you can double your performance on a high validity ARA-relevant evaluation without triggering the "must evaluate" threshold; I think evaluation should probably be required in that case, and therefore, if I had written the 4x threshold, I would be reducing it. But maybe those who wrote the threshold were totally game for these sorts of capability jumps? 

Can someone create a forecasting question about which model will score better in benchmarks, Claude 3.5 Opus or GPT-5?

The race is on.