Slightly against aligning with neo-luddites

I am also worried about where ill-considered regulation could take us. I think the best hopes for alignment all start by using imitation learning to clone human-like behavior. Broad limitations on what sorts of human-produced data are usable for training will likely make the behavior cloning process less robust and make it less likely to transmit subtler dimensions of human values/cognition to the AI.

Imitation learning is the primary mechanism by which we transmit human values to current state of the art language models. Greatly restricting the pool of people whose outputs can inform the AI’s instantiation of values is both risky and (IMO) potentially unfair, since it denies many people of the opportunity for their values to influence the behaviors of the first transformative AI systems.

I agree with the sentiment that indiscriminate regulation is unlikely to have good effects.

I think the step that is missing is analysing the specific policies No-AI Art Activist are likely to advocate for, and whether it is a good idea to support it.

My current sense is that data helpful for alignment is unlikely to be public right now, and so harder copyright would not impede alignment efforts. The kind of data that I could see being useful are things like scores and direct feedback. Maybe at most things like Amazon reviews could end up being useful for toy settings.

Another aspect that the article does not touch on is that copyright enforcement could have an adverse effect. Currently there is basically no one trying to commercialize training dataset curation because enforcing copyright use is a nightmare. It is in fact a common good. I'd expect there would be more incentives to create large curated datasets if this was not the case.

Lastly, here are some examples of "no AI art" legislation I expect the movement is likely to support:

1. Removing copyright protection of AI generated images
2. Enforcing AI training data to be strictly opt-in
3. Forcing AI content to be labelled as such

Besides regulation, I also expect activists to 4) pressure companies to deboost AI made content in social medial sites.

My general impression is that 3) is slightly good for AI safety. People in the AI Safety community have advocated for it in the past, convincingly.

I'm more agnostic on 1), 2) and 4).

1 and 4 will make AI generation less profitable, but also it's somewhat confused - it's a weird double standard to apply to AI content over human made content.

2 makes training more annoying, but could lead to commercialization of datasets and more collective effort being put into building them. I also think there is a possibly a coherent moral case for it, which I'm still trying to make my mind about, regardless of the AI safety consequences.

All in all, I am confused, though I wholeheartedly agree that we should be analysing and deciding to support specific policies rather than eg the anti AI art movement as a whole.

This. I really hope LW takes this to heart, and since only the shortest timelines would agree, but longer timelines don't, and longer timelines are only somewhat less dangerous alone. AI governance does matter, but so does the technical side: Without one, you can't win very well.

Policy is hard and complex, and details matter. I also worry that the community is in the middle of failing horribly at basic coordination because it's trying to be democratic but abstract with deliberations, and unilateralist and concrete with decision making.

Abstractions like "don't ally with X on topic Y" are disconnected from the actual decision making processes that might lead to concrete advances, and I think that even the pseudo-concrete "block progress in Y," for Y being compute, or data, or whatever, fails horribly at the concreteness criterion needed for actual decision making. 

What the post does do is push for social condemnation for "collaboration with the enemy" without concrete criteria for when it is good or bad, attempting to constrain action spaces for various actors who the AI safety community is allied with, or pushing them to disassociate themselves with the community because of perceived or actual norms against certain classes of policy work.

There are 4 points of disagreement I have about this post. 

First, I think it's fundamentally based on a strawperson. 

my fundamental objection is that their specific strategy for delaying AI is not well targeted.

This post provides an argument for not adopting the "neo-luddite" agenda or not directly empowering neo-luddites. This is not an argument against allying with neo-luddites for specific purposes. I don't know of anyone who has actually advocated for the former. This is not how I would characterize Katija's post.  

Second, I think there is an inner strawperson with the example about text-to-image models. From a bird's eye view, I agree with caring very little about these models mimicking humans artistic styles. But this is not where the vast majority of tangible harm may be coming from with text-to-image models. I think that this most likely comes from non-consensual deepfakes being easy to use for targeted harassment, humiliation, and blackmail. I know you've seen the EA forum post about this [EA · GW] because you commented on it. But I'd be interested in seeing a reply to my reply to your comment on the post. 

Third, I think that this post fails to consider how certain (most?) regulations that neo-luddites would support could meaningfully slow risky things down. In general, any type of regulation that makes research and dev for risky AI technologies harder or less incentivized will in fact slow risky AI progress down. I think that the one example you bring up--text-to-image models--is a counterexample to your point. Suppose we pass a bunch of restrictive IP laws that make it more painful to research, develop, and deploy text-to-image models. That would suddenly slow down this branch of research which could concievably be useful for making riskier AI in the future (e.g. multimodal media generators), hinder revenue opportunities for companies who are speeding up risky AI progress, close off this revenue option to possible future companies who may do the same, and establish law/case law/precedent around generative models that could be set precedent or be repurposed for other types of AI later.  

Fourth, I also am not convinced by the specific argument about how indiscriminate regulation could make alignment harder. 

Suppose the neo-luddites succeed, and the US congress overhauls copyright law. A plausible consequence is that commercial AI models will only be allowed to be trained on data that was licensed very permissively, such as data that's in the public domain...Right now, if an AI org needs some data that they think will help with alignment, they can generally obtain it, unless that data is private.

This is a nitpick, but I don't actually predict this scenario would pan out. I don't think we'd realistically overhaul copyright law and have the kind of regime with datasets that you describe. But this is probably a question for policy people. There are also neo-luddite solutions that your argument would not apply to--like having legal requirements for companies to make their models "forget" certain content upon request. This would only be a hindrance to the deployer.

Ultimately though, what matters is not whether something makes certain alignment research harder. It matters how much something makes alignment research harder relative to how much it makes risky research harder. Alignment researchers are definitely the ones that are differentially data-hungry. What's a concrete, concievable story in which something like the hypothetical law you described makes things differentially harder for alignment researchers compared to capabilities researchers?

IMO it might very well be that most restrictions on data and compute are net positive. However, there are arguments in both directions.

On my model, current AI algorithms are missing some key ingredients for AGI, but they might still eventually produce AGI by learning those missing ingredients. This is similar to how biological evolution a learning algorithm which is not a GI, but it produced humans who are GIs. Such an AGI would be a mesa-optimizer, and it's liable to be unaligned regardless of the details of the outer loop (assuming an outer loop made of building blocks similar to what we have today). For example, the outer loop might be aimed at a human imitation, but the resulting mesa-optimizer is only imitating humans when it's instrumentally beneficial for it. Moreover, as in the case of evolution, this process would probably be very costly in terms of compute and data, as it is trying to "brute force" a problem for which it doesn't have an efficient algorithm. Therefore, limiting compute or data seems like a promising way to prevent this undesirable scenario.

On the other hand, the most likely path to aligned AI would be through a design that's based on solid theoretical principles. Will such a design require much data or compute compared to unaligned competitors?

Reasons to think it won't:

• Solid theoretical principles should allow improve capabilities as well as alignment.
• Intuitively, if an AI is capable enough to be transformative (given access to particular amounts of compute and data), it should be capable enough to figure out human values, assuming it is motivated to do so at the first place. Or, it should at least be capable enough to act against unaligned competition while not irreversibly destroying information about human values (in which case it can catch up on learning those later). This is similar to what Christiano calls "strategy stealing".

Reasons to think it will:

• Maybe aligning AI requires installing safe-guards that cause substantial overhead. This seems very plausible when looking at proposals such as Delegative Reiforcement Learning, which have worst regret asymptotic that "unaligned" alternatives (conventional RL). It also seems plausible when looking at proposals such as IDA or debate, which introduce another level of indirection (simulating humans) to the problem of optimizing the world that unaligned AI attacks directly (in Christiano's terminology, they fail to exploit inaccessible information). It's less clear about PreDCA [LW(p) · GW(p)], but even there alignment requires a loss function with more complex type signature than the infra-Bayesian physicalism "default", which might incur a statistical or computational penalty.
• Maybe aligning AI requires restricting ourselves to using well-understood algorithmic building blocks and not heuristic (but possibly more efficient) building blocks. Optimistically, having solid theoretic principles should allow us to roughly predict the behavior even of heuristic algorithms that are effective (because such algorithms have to be doing qualitatively the same thing as the rigorous algorithms). Pessimistically, alignment might depend on nuances that are obscured in heuristics.

We can model the situation by imagining 3 frontiers in resource space:

• The mesa-resource-frontier (MRF) is how much resources are needed to create TAI with something similar to modern algorithms, i.e. while still missing key AGI ingredients (which is necessarily unaligned).
• The direct-resource-frontier (DRF) is how much resources are needed to create TAI assuming all key algorithms, but without any attempt at alignment.
• The aligned-resource-frontier (ARF) is how much resources are needed to create aligned TAI.

We have ARF > DRF and MRF > DRF, but the relation between ARF and MRF is not clear. They might even intersect (resource space is multidimensional, we at least have data vs compute and maybe finer distinctions are important). I would still guess MRF > ARF, by and large. Assuming MRF > ARF > DRF, the ideal policy would forbid resources beyond MRF but allow resources beyond ARF. A policy that is too lax might lead to doom by the mesa-optimizer pathway. A policy that is too strict might lead to doom by making alignment infeasible. If the policy is so strict that it forces us below DRF then it buys time (which is good), but if the restrictions are then lifted gradually, it predictably leads to the region between DRF and ARF (which is bad).

Overall, the conclusion is uncertain.

Yeah, I'm worried that copyright restrictions might slow down responsible projects in Western countries vs. less responsible projects in jurisdictions that we are likely to have less influence over.

While I completely agree that care should be taken if we try to slow down AI capabilities, I think you might be overreacting in this particular case. In short: I think you're making strawmen of the people you are calling "neo-luddites" (more on that term below). I'm going to heavily cite a video that made the rounds and so I think decently reflects the views of many in the visual artist community. (FWIW, I don't agree with everything this artist says but I do think it's representative). Some details you seem to have missed:

1. I haven't heard of visual artists asking for the absolute ban of using copyrighted material in ML training data – they just think it should be opt-in and/or compensated.
2. Visual artists draw attention to the unfair double standard for visual vs audio data, that exists because the music industry has historically had tighter/more aggressive copyright law. They want the same treatment that composers/musicians get.
   1. Furthermore, I ask: is Dance Diffusion "less-aligned" than Stable Diffusion? Not clear to me how we should even evaluate that. But those data restrictions probably made Dance Diffusion more of a hassle to make (I agree with this comment [LW(p) · GW(p)] in this respect).
3. Though I imagine writers could/might react similarly to visual artists regarding use of their artistic works, I haven't heard any talk of bans on scraping the vast quantities of text data from the internet that aren't artistic works. It's a serious stretch to say the protections that are actually being called for would make text predictor models sound like "a person from over 95 years ago" or something like that.

More generally, as someone you would probably classify as a "neo-luddite," I would like to make one comment on the value of "nearly free, personalized entertainment." For reasons I don't have time to get into, I disagree with you: I don't think it is "truly massive." However, I would hope we can agree that such questions of value should be submitted to the democratic process (in the absence of a better/more fair collective decision-making process): how and to what extent we develop transformative AI (whether agentic AGI or CAIS) involves a choice in what kind of lifestyles we think should be available to people, what kind of society we want to live in/think is best or happiest. That's a political question if ever there was one. If it's not clear to you how art generation AI might deprive some people of a lifestyle they want (e.g. being an artist) see here and here for some context. Look past the persuasive language (I recommend x1.25 or 1.5 playback) and I think you'll find some arguments worth taking seriously.

Finally, see here about the term "luddite." I agree with Zapata that the label can be frustrating and mischaracterizing given its connotation of "irrational and/or blanket technophobia." Personally, I seek to reappropriate the label and embrace it so long as it is used in one of these senses [LW · GW], but I'm almost certainly more radical than the many you seem to be gesturing at as "neo-luddites."

I agree it's important to be careful about which policies we push for, but I disagree both with the general thrust of this post and the concrete example you give ("restrictions on training data are bad").

Re the concrete point: it seems like the clear first-order consequence of any strong restriction is to slow down AI capabilities. Effects on alignment are more speculative and seem weaker in expectation. For example, it may be bad if it were illegal to collect user data (eg from users of chat-gpt) for fine-tuning, but such data collection is unlikely to fall under restrictions that digital artists are lobbying for.

Re the broader point: yes, it would be bad if we just adopted whatever policy proposals other groups propose. But I don't think this is likely to happen! In a successful alliance, we would find common interests between us and other groups worried about AI, and push specifically for those. Of course it's not clear that this will work, but it seems worth trying.

I agree, and I don't think speeding up or slowing down AI is desirable due to a part of a comment by Rohin Shah:

1. It makes it easier for a future misaligned AI to take over by increasing overhangs, both via compute progress and algorithmic efficiency progress. (This is basically the same sort of argument as "Every 18 months, the minimum IQ necessary to destroy the world drops by one point.")

2. Such strategies are likely to disproportionately penalize safety-conscious actors.

(As a concrete example of (2), if you build public support, maybe the public calls for compute restrictions on AGI companies and this ends up binding the companies with AGI safety teams but not the various AI companies that are skeptical of “AGI” and “AI x-risk” and say they are just building powerful AI tools without calling it AGI.)

For me personally there's a third reason, which is that (to first approximation) I have a limited amount of resources and it seems better to spend that on the "use good alignment techniques" plan rather than the "try to not build AGI" plan. But that's specific to me.

Interesting. I know few artists and even their lawyers and not one of them see AI art as a threat — alas this might be them not having the full picture of course. And while I know that everyone can call themselves an artist, I certainly don’t want to gate-keep here, for context I’ll add that I mean friends who finished actual art schools. I know this because I use AI art in my virtual tabletop RPG sessions I play with them and they seem more excited than worried about AI. What follows is based on my casual pub discussion with them.

As for me, I don’t like my adventures to feel like a train ride so I give a great degree of freedom to my players in terms of what they can do, where they can go, with whom they can speak. During the game, as they make plans between themselves, I can use AI generators to create just-in-time art about the NPC or location they are talking about. This, together with many other tricks, allows me to up quality of my game and doesn’t take away work from artists because sheer speed required to operate here was a factor prohibiting to hire them here anyway.

However — this only works because my sessions require suspension of disbelief by default and so nobody cares about the substance of that art. After all, we all roll dice around and pretend they mean how well we wave a sword around so nobody cares if styles or themes slightly differ between sessions, it’s not an art book.

For anything that’s not just fun times with friends you will still need an artist who will curate the message, modify or merge results from multiple AI runs, fine-tune parameters and even then probably do quite a lot of digital work on the result to bring it up to standards that passes the uncanny valley or portrays exactly what movie director had in mind.

Or is AI already here that’s capable of doing those things by itself with one or two sentences from an executive and churning out a perfect result? Because I’ve worked with many models and have yet to see one that wouldn’t require further creative work to actually be good. AFAIK all award winning AI-generated content was heavily curated, not some random shots.

It feels to me like low-level art is going to be delegated to AI while artists can focus on higher forms of art rather than doing the boring things. Just like boilerplate generators in code. Or they’ll be able to do more boring things faster, just like frameworks for developers pushing out similar REST apps one after another. And base building blocks are going to become more open source while the value will be in how you connect, use and support those blocks in the end.

This may allow a lot more people to discover their creative, artistic side who couldn’t do it previously because they lacked mechanical skill to wave a brush or paint pixels.

I write this comment haphazardly so sorry if my thought process here are unpolished but overall it feels like a massive boost to creativity and a good thing for the art, if not potentially the greatest artistic boost to humanity since ever.

AI is a new brush that requires less mechanical skill than before. You must still do creative work and make Art with it.

I haven't spent much time thinking about this at all, but it's interesting to think about the speed with which regulation gets put into place for environmental issues such as climate change and HFC's ban to test how likely it is that regulation will be put in place in time to meaningfully slow down AI.

These aren't perfectly analogous since AI going wrong would likely be much worse than the worst case climate change scenarios, but the amount of time it takes to get climate regulation makes me pessimistic. However, HFC's were banned relatively quickly after realising the problem, so maybe there is some hope.

[comment deleted]

• When considering whether to delay AI, the choice before us is not merely whether to accelerate or decelerate the technology. We can choose what type of regulations are adopted, and some options are much better than others.

We (the AI Safety community/ generally alignment-concerned people/ EAs) almost definitely can't choose what type of regulations are adopted. If we're very lucky/ dedicated we might be able to get a place at the table. Everyone else at the table will be members of slightly, or very, misaligned interest groups who we have to compromise with.

Various stripes of "Neo-Luddite" and AI-x-risk people have different concerns, but this is how political alliances work. You get at the table and work out what you have in common. We can try to take a leadership role in this alliance, with safety/ alignment as our bottom line- we'll probably be a smaller interest group than the growing ranks of newly unemployed creatives, but we could be more professionalised and aware of how to enact political change. 

If we could persuade an important neo-Luddite 'KOL' to share our concerns about x-risk and alignment, this could make them a really valuable ally. This isn't too unrealistic- I suspect that, once you start feeling critical towards AI for taking your livelihood, it's much easier to see it as an existential menace. 

• Adopting the wrong AI regulations could lock us into a suboptimal regime that may be difficult or impossible to leave. So we should likely be careful not [to] endorse a proposal because it's "better than nothing" unless it's also literally the only chance we get to delay AI.

Expecting anything close to optimal regulation in the current national/ international order on the first shot is surely folly. We should endorse any proposal that is "better than nothing" while factoring potential suboptimal regime shifts into our equations.