Posts
Comments
I thank y'all for rapidly replicating and extending this eval. This is the most important eval extant. Units are truly comparable, and it's directly connected to the questions of "coding for ML/AI research" and "long-horizon agency" that seem cruxy for short timelines. I did not expect @Daniel Kokotajlo to be right about the superexponentiality so quickly.
My long-timeline probability mass is increasingly dependent on "this doesn't generalize past formally verifiable domains + formally verifiable domains are insufficient for to automate AI algorithmic progress substantially" or "somehow this progress doesn't extend to the arbitrarily messy and novel real world." But it ain't looking good.
Righteous work. It would be impactful to take versions of this to mainstream media outlets: TIME and Vox have obviously proven receptive, but Bloomberg and the FT aren't deaf to the issues either. Anyone, really. The more distribution the better. Grateful to y'all for saying this out loud and clearly. To folks unfamiliar with the difficulty of alignment, you make nonobvious yet clarifying points.
Good work. Your sketches of entertainment and politics resonate as especially plausible, conditional on kinda/realistically-aligned superintelligence. "Different shades of beige." Would you be a human supremacist? From 2025, I'd like to think I would be.
May I ask what hedge fund? Or at least what kind? From what I know, the big ones (Citadel, Millennium, etc.) target and achieve a Sharpe ratio of ~2.0 for the firm as a whole. Which means individual bets must be quite weak. If you could get 100 bets with a Sharpe of 2, you could have a 20.0 Sharpe portfolio, which would be roughly Virtu-level. And they're scarcely a hedge fund; they market-make on a level that is essentially "the consistent provision of a service."
Could you explain what types of tasks lie within this "50%"?
And when you talk about "automating 50%," does this mean something more like "we all get twice as productive because the tasks we accomplish are faster," or does it mean "the models can do the relevant tasks end-to-end in a human-replacement way, and we simply no longer need attend to these tasks"?
E.g., Cursor cannot yet replace a coder, but it can enhance her productivity. However, a chatbot can entirely replace a frontline customer service representation.