Proving it safely would be the challenging thing.

Holding organizations liable for any action they take which they do not prove is safe is an absolutely terrible idea

I think you're talking past each other. I interpret Nathan as saying that he could prove that everyone on earth has been harmed, but that he couldn't do that in a safe manner.

Thanks Quintin. That's useful. I think the general standard of holding organizations liable for any action which they do not prove to be safe is indeed a terrible idea. I do think that certain actions may carry higher implicit harms, and should be held to a higher standard of caution.

Perhaps you, or others, will give me your opinions on the following list of actions. Where is a good point, in your opinion, to 'draw the line'? Starting from what I would consider 'highly dangerous and much worse than Llama2' and going down to 'less dangerous than Llama2', here are some related actions.

1. Releasing for free a software product onto the internet explicitly engineered to help create bio-weapons. Advertising this product as containing not only necessary gene sequences and lab protocols, but also explicit instructions for avoiding government and organization safety screens. Advertising that this product shows multiple ways to create your desired bio-weapon, including using your own lab equipment or deceiving Contract Research Organizations into unwittingly assisting you. 
2. Releasing the same software product with the same information, but not mentioning to anyone what it is intended for. Because it is a software product, rather than an ML model, the information is all correct and not mixed in with hallucinations. The user only needs to navigate to the appropriate part of the app to get the information, rather than querying a language model.
3. Releasing a software product not actively intended for the above purposes, but which does happen contain all that information and can incidentally be used for those purposes. 
4. Releasing an LLM which was trained on a dataset containing this information, and can regurgitate the information accurately. Furthermore, tuning this LLM before release to be happy to help users with requests to carry out a bio-weapons project, and double-checking to make sure that the information given is accurate.
5. Releasing an LLM that happens to contain such information and could be used for such purposes, but the information is not readily available (e.g. requires jail breaking or fine tuning) and is contaminated with hallucinations which can make it difficult to know which portion of the information to trust.
6. Releasing an LLM that could be used for such purposes, but that doesn't yet contain the information. Bad actors would have to acquire the relevant information and fine-tune the model on it in order for the model to be able to effectively assist them in making a weapon of mass destruction.

Where should liability begin?

Also given the rest of the replies, I think he means that it would be challenging for a plaintiff to safely prove that Llama 2 enables terrorists to make bioweapons, not that the alleged harm is "making open-source AI without proving it safe" or such.

FWIW the heavy negative karma means that his answer is hidden by default, so that readers can't easily see why OP thinks it might make sense to bring a lawsuit against Meta, which seems bad.

Yes, that's a fair point. I should clarify that the harm I am saying has been done is "information allowing terrorists to make bioweapons is now more readily comprehensible to motivated laypersons." It's a pretty vague harm, but it sure would be nice (in my view) to be able to take action on this before a catastrophe rather than only after.

Yes, that's fair. I'm over-reacting. To me it feels very like someone is standing next to a NYC subway exit handing out free kits for making dirty nuclear bombs that say things on the side like, "A billion civilians dead with each bomb!", "Make it in your garage with a few tools from the hardware store!", "Go down in history for changing the face of the world!".

I want to stop that behavior even if nobody has yet been killed by the bombs and even if only 1 in 10 kits actually contains correct instructions.

As a bit of evidence that I'm not just totally imagining risks where there aren't any... The recent Executive Order from the US Federal Gov contains a lot of detail about improving the regulation of DNA synthesis. I claim that the reason for this is that someone accurately pointed out to them that there are gaping holes in our current oversight, and that AI makes this vulnerability much more dangerous. And their experts then agreed that this was a risk worth addressing.

For example:

" (i)    Within 180 days of the date of this order, the Director of OSTP, in consultation with the Secretary of State, the Secretary of Defense, the Attorney General, the Secretary of Commerce, the Secretary of Health and Human Services (HHS), the Secretary of Energy, the Secretary of Homeland Security, the Director of National Intelligence, and the heads of other relevant agencies as the Director of OSTP may deem appropriate, shall establish a framework, incorporating, as appropriate, existing United States Government guidance, to encourage providers of synthetic nucleic acid sequences to implement comprehensive, scalable, and verifiable synthetic nucleic acid procurement screening mechanisms, including standards and recommended incentives.  As part of this framework, the Director of OSTP shall:

               (A)  establish criteria and mechanisms for ongoing identification of biological sequences that could be used in a manner that would pose a risk to the national security of the United States; and

               (B)  determine standardized methodologies and tools for conducting and verifying the performance of sequence synthesis procurement screening, including customer screening approaches to support due diligence with respect to managing security risks posed by purchasers of biological sequences identified in subsection 4.4(b)(i)(A) of this section, and processes for the reporting of concerning activity to enforcement entities."

Thankfully, it seems like the US Federal Government is more on the same page with me about these risks than I had previously thought.

"(k)  The term “dual-use foundation model” means an AI model that is trained on broad data; generally uses self-supervision; contains at least tens of billions of parameters; is applicable across a wide range of contexts; and that exhibits, or could be easily modified to exhibit, high levels of performance at tasks that pose a serious risk to security, national economic security, national public health or safety, or any combination of those matters, such as by:

          (i)    substantially lowering the barrier of entry for non-experts to design, synthesize, acquire, or use chemical, biological, radiological, or nuclear (CBRN) weapons;

          (ii)   enabling powerful offensive cyber operations through automated vulnerability discovery and exploitation against a wide range of potential targets of cyber attacks; or

          (iii)  permitting the evasion of human control or oversight through means of deception or obfuscation.

Models meet this definition even if they are provided to end users with technical safeguards that attempt to prevent users from taking advantage of the relevant unsafe capabilities. "

Thanks RamblinDash, that's helpful. I agree that this does seem like something the government should consider taking on, rather than making sense as a claim from a particular person at this point. I suppose that if there is a catastrophe that results which can be traced back to Llama2, then the survivors would have an easier case to make.

No existing country has laws that require you to prove that everything you do is safe.

Nathan isn't saying that Meta should be sued for not proving that Llama 2 was safe before open sourcing. See this comment [LW(p) · GW(p)] and the reply.