As artificial intelligence systems increasingly generate text, images, audio, and video at scale, governments and courts around the world are confronting an uncomfortable question: when AI produces illegal content, who is responsible? The instinctive search for a single culprit—the platform, the developer, or the user—has shaped much of the public debate. Yet this framing misunderstands how AI systems actually operate. Responsibility in the age of synthetic content is not binary; it is distributed across the AI value chain.
A more productive approach is to ask how accountability should be allocated in proportion to control, capability, and proximity to harm. AI systems do not act independently. They are designed by developers, deployed by platforms, and used—sometimes maliciously—by people. Responsibility must therefore follow human agency rather than artificial output.
Illegal AI-generated content is no longer hypothetical. It includes deepfakes used for fraud and extortion, synthetic disinformation campaigns that distort elections and public trust, AI-generated child sexual abuse material, automated hate speech, and tools that lower the barrier to cybercrime. These harms are real, measurable, and increasingly transnational. Treating them as abstract ethical dilemmas risks ignoring their concrete social and security consequences.
A useful way to understand accountability is through a model of shared but differentiated responsibility, involving three primary actors.
Platforms that deploy AI systems at scale bear the greatest continuous responsibility. They control access, distribution, monetisation, and enforcement. When illegal content spreads widely, it is rarely because generation is possible, but because governance fails at scale. Platforms have a duty to conduct rigorous risk assessments, implement effective safeguards, maintain traceability through audit mechanisms, and respond swiftly to abuse. If a platform benefits from scale, it must also govern at scale.
Developers, by contrast, hold upstream and structural responsibility. They are not liable for every misuse of their systems, but they are accountable for foreseeable misuse. Secure-by-design architectures, red-teaming, abuse-resistant training practices, and transparent documentation are not optional extras. When models are released without adequate safeguards or risk disclosures, the failure is not neutral innovation; it is a governance lapse with predictable consequences.
Users remain responsible where intent and agency are clear. AI does not eliminate human intent—it amplifies it. Individuals or groups who deliberately employ AI systems for fraud, harassment, incitement, or manipulation cannot hide behind the technology they used. In such cases, existing criminal and civil liability frameworks should apply, adapted to the digital context.
Public debate often collapses this complexity into an either–or question: is the platform responsible, or the developer, or the user? This framing is misleading. Responsibility is not a zero-sum game. It should be layered, with each actor held accountable for the risks they can reasonably control. A simple guiding principle applies: responsibility increases with proximity to harm. Users are closest to the act, platforms to the spread, and developers to the design choices that make abuse easier or harder.
The challenge is compounded by the global nature of AI systems and the national character of law. Content generated in one jurisdiction can cause harm in another, routed through platforms incorporated elsewhere and built on models trained across borders. This fragmentation makes unilateral solutions inadequate. What is needed is not a single global AI law, but interoperable standards, shared definitions of high-risk synthetic content, and stronger mechanisms for cross-border regulatory and law-enforcement cooperation.
The way forward lies in moving from reactive blame to proactive governance. Liability must be tied to demonstrable negligence rather than the mere existence of capability. High-risk AI systems should undergo mandatory and ongoing risk assessments proportional to their impact. Transparency should be the default posture of AI deployment, with secrecy justified only in narrow circumstances. Law-enforcement access must be enabled through due process, not technical backdoors that weaken systemic security. Above all, public-private cooperation must replace adversarial regulation, recognising that neither governments nor industry can manage these risks alone.
AI systems have no moral agency. They cannot be punished or deterred. Responsibility rests entirely with the humans and institutions that design, deploy, and misuse them. The question facing the global community is not whether AI should be responsible, but whether those who wield power over it are prepared to accept responsibility commensurate with that power.
