LessWrong 2.0 Reader

just because it's possible in theory doesn't mean we are anywhere close to doing it

that's a good point, but then you have to explain why it would be hard to make a functional digital copy of a human given that we can make AIs like ChatGPT-o1 that are at 99th percentile human performance on most short-term tasks. What is the blocker? 

Of course this question can be settled empirically.... 

 

Huh. I first heard of Greg Egan in the context of Eliezer mentioning him as a SF writer who he liked, iirc. Kind of ironic he ended up here.

Im selling $1000 tier5 OpenAI credits at discount. DM me if interested. 

You can video call me and all my friends to reduce the probably I end up scamming you. Or vice versa I can video call your friends. We can do the transaction in tranches if we still can’t establish trust. 

Thanks to anyone that took the time to read and vote this post - regardless if it was a positive or a negative vote, I still appreciate it.

If you happen to downvote me, I'd appreciate it if you could explain the reason why: this is the second time that happens (for one of my previous posts I chose a title that was sounding like clickbait - I then corrected it), and I am curious to understand your feedback this time as well.

The reason why I write posts here from time to time is simply to be challenged and be exposed to different points of view: that cannot happen without an exchange (even if harsh).

Let me take this chance to wish you all a happy new year 2025!
Gianluca

Fascinating post! I (along with Hardik Bhatnagar and Joseph Bloom) recently completed a profile of cases of SAE latent co-occurrence in GPT2-small and Gemma-2-2b (see here [LW · GW]) and I think that this is a really compelling driver for a lot of the behaviour that we see, such as the link to SAE width. In particular, we observe a lot of cases with apparent parent-child relations between the latents (e.g. here). 

We also see a similar 'splitting' of activation strength in cases of composition e.g. we find a case where the child latents are all days of the week (e.g. 'Monday'), but the activation (of lack thereof) of the parent latent corresponds to whether there is a space in the token (e.g. ' Monday') (see here). When the parent and child are active, both have roughly half the activation strength of the child when it is active alone, which I think is similar to what you observe, although made more complex because we do not know the underlying true features in this case. If this holds in general, perhaps it would be possible to improve your method for preventing co-occurrence/absorption by looking not only for cases of splits in the activation density, but for the activation strengths between pairs of features being strongly coupled/proportional in such a manner?

Is this going to continue in 2025? I'll be visiting Berkeley from Jan 5th to Jan 17th and would like to come visit.

Frames for thinking about language models I have seen proposed at various times:

• Lookup tables. (To predict the next token, an LLM consults a vast hypothetical database containing its training data and finds matches.)
• Statistical pattern recognition machines. (To predict the next token, an LLM uses context as evidence to do Bayesian update on a prior probability distribution, then samples the posterior).
• People simulators. (To predict the next token, an LLM infers what kind of person is writing the text, then simulates that person.)
• General world models. (To predict the next token, an LLM constructs a belief estimate over the underlying context / physical reality which yielded the sequence, then simulates that world forward.)

The first two are ‘mathematical’ frames and the last two are ‘semantic’ frames. Both of these frames are likely correct to some degree and making them meet in the middle somewhere is the hard part of interpretability

Where does China fit into this picture

Unlike the West, China enjoys unconditional love from the Heavens. /j

I've added a section called "Social-instinct AGI" under the "Control the thing" heading similar to last year.

AI is getting human-level at closed-ended tasks such as math and programming, but not yet at open-ended ones. They appear to be more difficult. Perhaps evolution brute-forced open-ended tasks by creating lots of agents. In a chaotic world, we're never going to know which actions lead to a final goal, e.g. GDP growth. That's why lots of people try lots of different things.

Perhaps the only way in which AI can achieve ambitious final goals is by employing lots of slightly diverse agents. Perhaps that would almost inevitably lead to many warning shots before a successful takeover?