DAL's Shortform

post by DAL · 2025-01-27T21:38:55.078Z · LW · GW · 4 comments

Contents

4 comments

4 comments

Comments sorted by top scores.

comment by DAL · 2025-01-27T21:38:55.159Z · LW(p) · GW(p)

It's worth thinking through what today's DeepSeek-induced, trillion dollar-plus drop in AI related stocks means.  

There are two basic explanations for DeepSeek's success training models with a lot less compute:

  1. Imitation is Easy: DeepSeek is substantially just re-treading the same ground as the other players.  They're probably training on O1 outputs, etc.  DeepSeek proves that it's easy to match breakthroughs, but not to generate them.  Further advances will still require tons of compute.
  2. DeepSeek is really clever: Facing compute constraints, DeepSeek engineers were forced to find a better way to do work and they did.  That clever will likely translate into forward progress, and there's no reason it would be limited to imitation.

If #1 is true, then I think it implies that we're headed towards a big slowdown in AI progress.  The whole economic value proposition for building models just changed.  If your frontier model can be imitated at a tiny fraction of the cost after a few months, what good is it? Why would VCs invest money in your training runs?

If #2 is true, then we may be headed towards incredibly rapid AI progress, and the odds of recursively self-improving AI are much higher.  If what you really need to build better models is tons and tons of compute, then AI can't speed itself up much.  If what you need is just lots of cleverness, then it's much easier to imagine a fast takeoff.

#1 is likely better for alignment in that it will slow things down from the current frenetic pace (the possible downside is that if you can imitate a cutting edge model cheaply and easily then hostile actors may deliberately build misaligned models).  

#1 also seems to have big implications for government/legal involvement in AI.  If the private sector loses interest in funding models that can be easily imitated, then further progress will tend to rely on either: government investment (as in basic science) or aggressive IP law that allows commercialization of progress by preventing imitators (as we do in drug development).  Either of those means a much bigger role for the public sector.

Replies from: Vladimir_Nesov
↑ comment by Vladimir_Nesov · 2025-01-28T00:39:26.399Z · LW(p) · GW(p)

training on O1 outputs

Outputs of o1 don't include reasoning traces, so not particularly useful compared to outputs of chatbot models, and very expensive, so only a modest amount can be collected.

Imitation helps with post-training, but the compute-heavy part is pretraining, and obtaining good quality with little pretraining is a novel feat that isn't known to be explainable by good post-training, or by including a lot of outputs from good models in the pretraining/annealing mix.

Replies from: gwern, hastings-greer
↑ comment by gwern · 2025-01-28T01:23:05.683Z · LW(p) · GW(p)

Outputs of o1 don't include reasoning traces, so not particularly useful compared to outputs of chatbot models, and very expensive, so only a modest amount can be collected.

It would be more precise to say outputs of o1 aren't supposed to include the reasoning traces. But in addition to the reasoning traces OA voluntarily released, people have been observing what seem to be leaks, and given that the history of LLM robustness to jailbreaks can be summarized as 'nil', it is at least conceivable that someone used a jailbreak+API to exfiltrate a bunch of traces. (Remember that Chinese companies like ByteDance have definitely been willfully abusing the OA API for the purposes of knowledge distillation/cloning and evading bans etc, in addition to a history of extremely cutthroat tactics that FANG would blanch at, so it's a priori entirely plausible that they would do such things.)

I don't believe DeepSeek has done so, but it is technically possible. (Regardless of whether anyone has done so, it is now partially moot given that r1 traces in the DS paper, and based on third party reports thus far, work so well for distillation so everyone can kickstart their own r1-clone with r1 reasoning traces and work from there. There may be more reason to try to exfiltrate o3+ traces, but OA may also decide to not bother, as users are claiming to value and/or enjoy reading the raw traces, and since the secret & capability is out, maybe there's not much point in hiding them any longer.)

↑ comment by Hastings (hastings-greer) · 2025-01-28T01:26:42.849Z · LW(p) · GW(p)

I think most of the imitation happens in the pretraining. I don't know about o1, but DeepSeek v3 is at minimum trained on a ton of 4o outputs, although they are slightly cagey about this. Just the first thing I tried, I had ChatGPT write a sock poem:

4o's poem:


Socks, the keepers of toes so warm,
Guardians of feet through life's storm.
In stripes, in polka dots, or plain,
They dance with us through joy and pain.

A cozy pair on winter’s chill,
Or mismatched mates—yet charming still.
From drawer to floor, they roam and stray,
But where one goes, the other may.

Oh, humble socks, so oft unseen,
Yet vital threads of life's routine.
A quiet comfort, soft and neat,
The unsung heroes of weary feet.
 


Then I gave v3 just the first two stanzas, and asked it to continue the poem:

v3's continuation:



They hide in shoes, they lounge in beds,
Sometimes they’re lost, or so it’s said.
Yet always there, a faithful friend,
From morning’s start to evening’s end.

They cushion steps, they soften blows,
Through every stride, their comfort shows.
Oh, humble socks, so oft unseen,
The quiet heroes of routine.

So here’s to socks, both bold and meek,
For every day, for every week.
A simple joy, a small delight,
They keep us snug, from day to night.

 

The shot in the dark guess of the "humble socks, so oft unseen... routine" couplet is a fucking soul read. v3 knows 4o, in a way that I kind of hope no model ever knows a person.