DAL's Shortform 2025-01-27T21:38:55.078Z

Marx and the Machine 2025-01-15T18:33:16.789Z

Comment by DAL on DAL's Shortform · 2025-01-27T21:38:55.159Z · LW · GW

It's worth thinking through what today's DeepSeek-induced, trillion dollar-plus drop in AI related stocks means.  

There are two basic explanations for DeepSeek's success training models with a lot less compute:

1. Imitation is Easy: DeepSeek is substantially just re-treading the same ground as the other players.  They're probably training on O1 outputs, etc.  DeepSeek proves that it's easy to match breakthroughs, but not to generate them.  Further advances will still require tons of compute.
2. DeepSeek is really clever: Facing compute constraints, DeepSeek engineers were forced to find a better way to do work and they did.  That clever will likely translate into forward progress, and there's no reason it would be limited to imitation.

If #1 is true, then I think it implies that we're headed towards a big slowdown in AI progress.  The whole economic value proposition for building models just changed.  If your frontier model can be imitated at a tiny fraction of the cost after a few months, what good is it? Why would VCs invest money in your training runs?

If #2 is true, then we may be headed towards incredibly rapid AI progress, and the odds of recursively self-improving AI are much higher.  If what you really need to build better models is tons and tons of compute, then AI can't speed itself up much.  If what you need is just lots of cleverness, then it's much easier to imagine a fast takeoff.

#1 is likely better for alignment in that it will slow things down from the current frenetic pace (the possible downside is that if you can imitate a cutting edge model cheaply and easily then hostile actors may deliberately build misaligned models).  

#1 also seems to have big implications for government/legal involvement in AI.  If the private sector loses interest in funding models that can be easily imitated, then further progress will tend to rely on either: government investment (as in basic science) or aggressive IP law that allows commercialization of progress by preventing imitators (as we do in drug development).  Either of those means a much bigger role for the public sector.