LessWrong 2.0 Reader

My bet is that this isn't an attempt to erode alignment, but is instead based on thinking that lumping together intentional bad actions with mistakes is a reasonable starting point for building safeguards. Then the report doesn't distinguish between these due to a communication failure. It could also just be a more generic communication failure.

(I don't know if I agree that lumping these together is a good starting point to start experimenting with safeguards, but it doesn't seem crazy.)

I'm kinda opposite on the timelines thing? This is probably a timeline delayer even if I thought LLM's scaled to AGI, which I don't but let's play along.

If a Pharma company could look at another company's product and copy it and release it for free with no consequences, but the product they release itself could only be marginally improved without massive investment, what does that do to the landscape?

It kills the entire industry. This HURTS anyone trying to fundraise, reckless billions will have a harder time finding their way into the hands of developers because many investors will not be happy with the possibility (already demonstrated at least twice) that someone could just read the outputs of your API available model and eat your lunch, and releases so that people have a less restricted, more customizable and cheaper alternative they can run on their own hardware. Expanding that view, how many services will hose this model for cheaper than Openai will host gpt?

Want proof? Openai has problems running a profit on its services yet has effectively cut prices (or otherwise given away more for less money) twice since deepseek came out. Is Openai so grateful for the free research that deepseek produced that they would rather have that than (probably) billions of dollars in lost revenue, added cost and thinner investment?

Being more speculative, the way in which models have converged on being basically interchangeable should be a red flag that the real growth is plateaued. Goods competing mostly via price is a sign that there's uniformity in quality, that they're mostly interchangeable.

Real model growth seems to have been discarded in favor of finding ways to make a model stare at a problem for hours at a time and come up with answers that are... Maybe human passable if the problem is easy and it's accustomed to it? All it'll cost is a million dollars per run. If that sounds like it's just brute forcing the problem, it's because it is.

Where is the real advancement? The only real advancement is inference time scaling and it doesn't look like this last reach has gotten us close to AGI. The reasoning models are less flexible, not more, the opposite of what you would think if they were actually reasoning, best case is that the reasoning is an excuse to summon a magic token or remove a toxic token.

Am I crazy? Why would this accelerate your timeline?

I just finished up a semester of head and neck anatomy. I went into it for the neuroscience, not expecting much from the other topics. I had a similar experience to you, finding many interesting things that are helpful in my day to day.

I found the Hyoid bone especially interesting. I remember my first time seeing it on a model skeleton and thinking “hu, someone attached an extra mandible to this guy… and it’s just floating. That’s weird.”

I then had my mind blown seeing it in a cadaver.

Side bar: If you ever get the chance to go into an anatomy lab and explore the cadavers I highly recommend. Take what they say about eating and hydrating before going in seriously. ~every 2 weeks someone fainted and banged their head on a metal table.

The other key takeaway for me was learning about the cranial nerves. Then having the realisation that mine have probably been compromised in some way. (Perhaps COVID?). Which has left me with the humorous and sometimes useful Specific Anosmia of not being able to smell farts.

— 

Reading the Manga Guide to statistics has been great for Anki cards. Easy image occlusion cards that make reviewing more fun.

—

I wish I liked cells at work more. Still, I was able to explain where blood comes from to a kid yesterday because of things I learned in that show.

— 

We seem to have overlapping interests. So I’d like to recommend “Curious” by Lily Serna.

Perhaps it isn’t applicable to you. But it took me a day to read, and I added 100s of Anki cards derived from it.

Lily is famous for doing mental math super quickly, and the final section of the book is just a bunch of the tricks she uses clearly written out. With a bit of thought, these tricks combine into more powerful mental movements that have sped up my mental calculations quite a bit.

The rest of the book is cool math trivia and life hacks. Very fun, a bit simplistic.

— 

I’m enjoying these media reviews, excited for the next one!

Curated. I found this story one of the best articulations of what sort of good futures you should maybe expect if things go well, with some handholds to help figure out how to relate to it. 

Some have noted in the comments "isn't this story... dystopian?". And, well, it could be. It depends a lot on how the details shake out, and philosophy-of-identity works (or how you choose to relate to it). I think we are much more likely to get a worse future than the one described here (even if things go "moderately well"), where the events in the story might happen but subtly hollow out our humanity without us noticing. But I think the story as presented is at least plausible and coherent – I think the sort of incremental uploads, and shifting from self-identifying as an individual human to something that isn't exactly a human and isn't exactly an upload, is at least one reasonable way to see one possible future.

Your specific action places most of its hope for human survival on the entities that have done the most to increase extinction risk.

if the trend toward long periods of internal-only deployment continues

Have we seen such a trend so far? I would have thought the trend to date was neutral or towards shorter period of internal-only deployment.

Tbc, not really objecting to your list of reasons [LW · GW] why this might change in the future. One thing I'd add to it is that even if calendar-time deployment delays don't change, the gap in capabilities inside vs. outside AI companies could increase a lot if AI speeds up the pace of AI progress.

ETA: Dario Amodei says "Sonnet's training was conducted 9-12 months ago". He doesn't really clarify whether he's talking the "old" or "new" 3.5. Old and new sonnet were released in mid-June and mid-October, so 7 and 3 months ago respectively. Combining the 3 vs. 7 months options with the 9-12 months range imply 2, 5, 6, or 9 months of keeping it internal. I think for GPT-4, pretraining ended in August and it was released in March, so that's 7 months from pre-training to release. So that's probably on the slower side of Claude possibilities if Dario was talking about pre-training ending 9-12 months ago. But probably faster than Claude if Dario was talking about post-training finishing that early.

Yeah, I hate to be the one to say it but... you'd be better off calculating the size of the incident in terms of government response by summing the net worth / socioeconomic power of the harmed individuals.

Be that as it may, I nevertheless feel discomfited by the fact that I have been arguing for 2026-2028 arrival of AGI for several years now, and people have been dismissing my concerns and focusing on plans for dealing with AGI in the 2030s or later.

The near-term-AGI space getting systematically neglected because it feels hard to come up with plans for is a bad pattern.

Oh, and a question. I say above:

I can imagine, on the one hand, there being a fairly direct link from a neuron being predicted to the prediction output.

Is something like this what you were getting at in other comments with respect to the model doing self-prediction by learning the identity matrix? That wasn't quite clear to me.

I think the response to 9/11 was an outlier mostly caused by the "photogenic" nature of the disaster. COVID killed over a million Americans yet we basically forgot about it once it was gone. We haven't seen much serious investment in measures to prevent a new pandemic.