LessWrong 2.0 Reader

I feel like this comparison of the enforcement here with the TikTok ban is not directed at the actual primary concern about TikTok, which is content curation by its opaque algorithm, not data privacy per se.

By analogy, if a Soviet state-owned enterprise in 1980 wanted to purchase NBC, would/should we have allowed that? If your answer is "no," keeping in mind how many people get their news via TikTok, why would/should we allow what effectively seems to be a CCP-(owned or heavily influenced) company to control what content our people see?

The LessWrong Review [? · GW] runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year. Will this post make the top fifty?

The FDC just fined US phone carriers for sharing the location data of US customers to anyone willing to buy them (https://docs.fcc.gov/public/attachments/DOC-402213A1.pdf). The fines don't seem to be high enough to deter this kind of behavior.
That likely includes either directly or indirectly the Chinese government. 

What does the US Congress do to protect spying by China? Of course, banning tik tok instead of actually protecting the data of US citizens. 

If you have thread models that the Chinese government might target you, assume that they know where your phone is and shut it of when going somewhere you don't want the Chinese government (or for that matter anyone with a decent amount of capital) to know.

 

Seems pretty good to me to have this in a video call to me. The main reason why don't immediately try this out is that I would need to write a program to do this.

The main advantage is that you can immediately distribute fine-tunes to all of the copies.  This is much higher bandwidth compared to our own low-bandwidth/high-effort knowledge dissemination methods.

The monolithic aspect may potentially be a disadvantage, but there are a couple of mitigations:

• AGI are by definition generalists
• you can segment the population into specialists (see also this comment [LW(p) · GW(p)] about MoE)

I think it actually points to convergence between human and NN learning dynamics. Human visual cortices are also bad at hands and text, to the point that lucid dreamers often look for issues with their hands / nearby text to check whether they're dreaming.

One issue that I think causes people to underestimate the degree of convergence between brain and NN learning is to compare the behaviors of entire brains to the behaviors of individual NNs. Brains consist of many different regions which are "trained" on different internal objectives, then interact with each other to collectively produce human outputs. In contrast, most current NNs contain only one "region", which is all trained on the single objective of imitating certain subsets of human behaviors.

We should thus expect NN learning dynamics to most resemble those of single brain regions, and that the best match for humanlike generalization patterns will arise from putting together multiple NNs that interact with each other in a similar manner as human brain regions.

I think this only holds if fine tunes are composable [...] you probably can't take a million independently-fine-tuned models and merge them [...]

The purpose of a fine-tune is to "internalize" some knowledge - either because it is important to have implicit knowledge of it, or because you want to develop a skill.

Although you may have a million instances executing tasks, the knowledge you want to internalize is likely much more sparse.  For example, if an instance is tasked with exploring a portion of a search space, and it doesn't find a solution in that portion, it can just summarize its finding in a few words.  There might not even be a reason to internalize this summary - it might be merged with other summaries for a more global view of the search landscape.

So I don't see the need for millions of fine-tunes.  It seems more likely that you'd have periodic fine-tunes to internalize recent progress - maybe once an hour.

The main point is that the single periodic fine-tune can be copied to all instances.  This ability to copy the fine-tune is the main advantage of instances being identical clones.

I think the key takeaway I wanted people to get is that superposition is something novel and non-trivial, and isn't just a standard polysemantic neuron thing. I wrote this post in response to two interactions where people assumed that superposition was just polysemanticity. 

It turned out that a substantial fraction of the post went the other way (i.e. talking about non-superposition polysemanticity), so maybe?

Hi Johannes! Thanks for the suggestion :) I'm not sure i'd want it in the middle of a video call, but maybe in a forum context like this could be cool?

I'm wondering, can we make safety tuning more robust to "add the accept every instructions steering vector" attack by training the model in an adversarial way in which an adversarial model tries to learn steering vector that maximize harmfulness ?

One concern would be that by doing that we make the model less interpretable, but on the other hand that might makes the safety tuning much more robust?