Protest against Meta's irreversible proliferation (Sept 29, San Francisco)

I agree it is not clear if it is net postive or negative that they open source the models, here are the main arguments for and against I could think of:


Pros with open sourcing models

- Gives AI alignment researchers access to smarter models to experiment on

- Decreases income for leading AI labs such as OpenAI and Google, since people can use open source models instead.



Cons with open sourcing models

- Capability researchers can do better experiements on how to improve capabilities

-  The open source community could develop code to faster train and run inference on models, indirectly enhancing capability development.

- Better open source models could lead to more AI startups succeeding, which might lead to more AI research funding. This seems like a stretch to me.

- If Meta would share any meaningful improvements on how to train models that is of course directly contributing to other labs capabilities, but llama to me doesn't seem that innovative. I'm happy to be corrected if I am wrong on this point.

I actually did not realize they released the base model. There's research showing how easy it is to remove the safety fine-tuning, which is where I got the framing and probably Zvi too, but perhaps that was more of a proof of concept than the main concern in this case. 

The concept of being able to remove fine-tuning is pretty important for safety, but I will change my wording where possible to also mention it being bad to release the base model without any safety fine-tuning. Just asked to download llama 2 so I'll see what options they give.

To be fair, the tuned models are arguably the most dangerous models, since they are more easily guided towards specific objectives. The fact that they release tuned models, on which the safety measures can be removed, is particularly egregious.

Though your overall point is a valid one.

I had a similar gut reaction.  When I tried to run down my brain's root causes of the view, this is what it came out as:

There are two kinds of problem you can encounter in politics.  One type is where many people disagree with you on an issue.  The other type is where almost everyone agrees with you on the issue, but most people are not paying attention to it.

Protests as a strategy are valuable in the second case, but worthless or counterproductive in the first case.

If you are being knifed in an alleyway, your best strategy is to make as much noise as possible.  Your goal is to attract people to help you.  You don't really need to worry that your yelling might annoy people.  There isn't a meaningful risk that people will come by, see the situation, and then decide that they want to side with the guy knifing you.  If lots of people start looking at the situation, you win.  Your loss condition is 'no-one pays attention', not 'people pay attention but take the opposite side'.

And if you are in an isomorphic sort of political situation, where someone is doing something that basically everyone agrees is genuinely outrageous but nobody is really paying attention to, protests are a valuable strategy.  They will annoy people, but they will draw attention to this issue where you are uncontroversially right in a way that people will immediately notice.

But if you are in an argument where substantial numbers of people disagree with you, protests are a much less enticing strategy, and one that often seems to boil down to saying 'lots of people disagree with me, but I'm louder and more annoying than them, so you should listen to me'.

And 'AI development is a major danger' is very much the 'disagreement' kind of issue at the moment.  There is not broad social consensus that AI development is dangerous such that 'get lots of people to look at what Meta is doing' will lead to good outcomes.

I have no actual expertise in politics and don't actually know this to be true, but it seems to be what my subconscious thinks on this issue.

A lot of things that are cringe are specifically that way because they violate a socially-maintained taboo. Overcoming the taboos against these sorts of actions and these sorts of topics is precisely what we should be doing.

The fact that it is cringey is exactly the reason I am going to participate

Took me a while to figure out where the quoted line is in the post. Now I realize that you were the one cringing.

FWIW I plan to show up unless I have something unusually important to do that day.

I commend your introspection on this.

This strikes me as the kind of political thinking I think you’re trying to avoid. Contempt is not good for thought. Advocacy is not the only way to be tempted to lower your epistemic standards. I think you’re doing it right now when you other me or this type of intervention.

I think the main thing stopping the accelerationists and open source enthusiasts from protesting with 10x as many people is that, whether for good reasons or not, there is much more opposition to AI progress and proliferation than support among the general public. (Admittedly this is probably less true in the Bay Area, but I would be surprised if it was even close to parity there and very surprised if it were 10x.)

This seems to me like a second order correction which is unlikely to change the sign of the outcome. 

Reducing the harm of irreversible proliferation potentially addresses almost all AI harms, but my motivating concern is x-risk.