NYT is suing OpenAI&Microsoft for alleged copyright infringement; some quick thoughts

This is great news. I particularly agree that legislators should pass new laws making it illegal to train AIs on copyrighted data without the consent of the copyright owner. This is beneficial from at least two perspectives:

1. If AI is likely to automate most human labor, then we need to build systems for redistributing wealth from AI providers to the rest of the world. One previous proposal is the  robot tax, which would offset the harms of automation borne by manufacturing workers. Another popular idea is a Universal Basic Income. Following the same philosophy as these proposals, I think the creators of copyrighted material ought to be allowed to name their price for training AI systems on their data. This would distribute some AI profits to a larger group of people who contributed to the model's capabilities, and it might slow or prevent automation in industries where workers organize to deny AI companies access to training data. In economic terms, automation would then only occur if the benefits to firms and consumers outweigh the costs to workers. This could reduce concentration of power via wealth inequality, and slow the takeoff speeds of GDP growth. 
2. For anyone concerned about existential threats from AI, restricting the supply of training data could slow AI development, leaving more time for work on technical safety and governance which would reduce x-risk. 

I think previous counterarguments [LW · GW] against this position are fairly weak. Specifically, while I agree that foundation models which are pretrained to imitate a large corpus of human-generated data are safer in many respects than RL agents trained end-to-end, I think that foundation models are clearly the most promising paradigm over the next few years, and even with restrictions on training data I don't think end-to-end RL training would quickly catch up. 

OpenAI appears to lobby against these restrictions. This makes sense if you model OpenAI as profit-maximizing. Surprisingly to me, even OpenAI employees who are concerned about x-risk have opposed restrictions, writing "We hope that US policymakers will continue to allow this area of dramatic recent innovation to proceed without undue burdens from the copyright system." I wonder if people concerned about AI risk may have been "captured" by industry on this particular issue, meaning that people have unquestioningly supported a policy because they trust the AI companies which endorse it, even though the policy might increase x-risk from AI development.  

Why do you think that

lawmakers should introduce regulations to make it an infringement

to train an AI on content that's copyrighted but publicly accessible? Should it also be illegal for people to learn from copyrighted material? If not, what are the relevant differences between humans and AIs? Are there possible future situations in which you think it would be OK for AIs to learn from copyrighted material?

Imagine, e.g., a situation where somehow it's turned out that the AIs we make are broadly comparable to humans in cognitive abilities, and they "live" alongside us in something like the way you see in some science fiction where there are humans and somewhat-human-like robots, and they learn in something like the way humans do. Would you then want humans able to learn from any materials that have been published, while the AIs have to learn only from material that has explicit "yes, AIs can learn from this" permissions attached? You may well feel that this sort of scenario is wildly improbable, and you'd probably be right, but if you would want AIs able to learn from the same things as humans in that scenario but not in more-probable ones, what is it about these scenarios that makes the difference?

If the NYT is paywalled, how did the training of ChatGPT have access to it? If OpenAI negotiated terms with NYT for access, the question is then whether ChatGPT violates those terms.

Training: it's not clear to me whether training LLMs on copyrighted content is a copyright infringement under the current US copyright law. I think lawmakers should introduce regulations to make it an infringement, but I wouldn't think the courts should consider it to be an infringement under the current laws (although I might not be familiar with all relevant case law).

My feel is that it could have been fair use as long as LLMs were just research projects, but then OpenAI started selling theirs as a product without changing their working model at all, and if you're commercializing the model, it's another story. Not sure where open models would lay here, but I still reckon probably copyright infringement since you're not using the data only internally. Would like to hear an expert's opinion on this.

I think to the extent LLMs don't preserve the wording/the creative structure, copyright doesn't provide protection; and some preservation of the structure might be fair use.

The problem is that this is a new case because it completely destroys the business model of these websites if you can have an AI agent visit them and then relay a summary to you - as it denies them the clicks (and ad visualizations) they need to pay themselves off. At which point odds are they'll just lay on paywalls even harder if they're not protected from this.

ChatGPT hallucinating false info and attributing it to NYT is outside copyright law, but seems bad and damaging.

I could imagine something like a defamation lawsuit? But it would probably have to focus on a specific case, not the general possibility of it? Again, hard to guess, this is all unexplored territory and new questions that never needed to be asked until now.

Can someone familiar with LLMs comment on how ChatGPT can memorize so many NYT texts so well?

it'd be great to see regulations making AI companies liable for all sorts of damage from their products, including attributing statements to people who've never made them.

I see a case against punishing here, in general. Consider me asking you "What did Mario say?", and you answering - in private -

"Listen to this sentence - even though it might well be totally wrong: Mario said xxxx.",

or, even more in line with the ChatGPT situation,

"From what I read, I have the somewhat vague impression that Mario said xxxx - though I might mix this up, so you may really want to double-check."

Assume Mario has not said xxxx. We still have a strong case for not, in general, punishing you for the above statement. And even if I acted badly in response to your message, so that someone gets hurt, I'd see, a priori, the main blame to fall upon me, not you.

The parallels to the case of ChatGPT^[1] suggests to extend a similar reservation about punishing to our current LLMs.

Admittedly, pragmatism is in order. If an LLM's hallucinations - despite warnings - end up creating entire groups of people attacking others due to false statements, it may be high time to reign in the AI. But the default for false attributions should not be that, not as long as the warning is clear and obvious: Do not trust it as of yet at all.

1. ^{^}

   In addition to knowing today's LLMs hallucinate, we currently even get a "ChatGPT can make mistakes. Consider checking important information." right next to its prompt.

ChatGPT isn’t a substitute for a NYT subscription. It wouldn’t work at all without browsing. It would probably get blocked with browsing enabled, both by NYT through its useragent, and by OpenAI’s “alignment.” Even if it doesn’t get blocked, it would be slower than skimming the article manually, and its output not trustable.

OTOH, NYT can spend pennies to have an AI TLDR at the top of each of their pages. They can even use their own models, as semanticscholar does. Anybody who is economical enough to prefer the much worse experience of ChatGPT, would not have paid NYT in the first place. You can bypass the paywall trivially.

In fact, why don’t NYT authors write a TLDR themselves? Most of their articles are not worth reading. Isn’t the lack of a summary an anti-user feature to artificially inflate their offering’s volume?

NYT would, if anything, benefit from LLMs potentially degrading the average quality of the competing free alternatives.

The counterfactual version of GPT4 that did not have NYT in its training is extremely unlikely to have been a worse model. It’s like removing sand from a mountain.

The whole case is an example of rent-seeking post-capitalism.