AI Governance Institute
← News
Enforcement2026-09-26

Manhattan DA Seizes Dozen Deepfake Porn Sites, Targeting 1,200 Real People

What happened

The Manhattan District Attorney announced the seizure of twelve websites that generated and sold AI-created nonconsensual sexual images, according to reporting by Manhattan DA seizes a dozen celebrity deepfake websites. The sites used AI image-generation tools to produce synthetic sexual content featuring roughly 1,200 identifiable people, including celebrities, without their consent. The platforms operated at commercial scale, charging customers for access to the material before law enforcement intervened. The action follows a broader pattern of prosecutors treating synthetic-media abuse as a criminal enterprise, not merely a platform policy violation, consistent with the trajectory seen in China's Supreme Court Makes Deepfakes and AI Hallucinations Judicially Actionable and earlier enforcement patterns documented after Meta Ran Ads for Nonconsensual Deepfake App, Exposing Platform-Control Assumptions.

Why it matters

  • ·Regulators and prosecutors are no longer treating synthetic intimate-image abuse as a content-policy matter. They are treating it as a criminal distribution operation, which raises the legal exposure for any enterprise platform that generates, hosts, or pays for AI-produced image content without robust consent verification and abuse-detection controls.
  • ·The scale of harm, roughly 1,200 named individuals, illustrates that AI image tools can be weaponized against employees, executives, or customers of any organization. Enterprises without a synthetic-media abuse detection and victim-response protocol face both reputational liability and potential regulatory action under privacy and consumer protection rules.
  • ·Law enforcement seizure of commercial deepfake sites signals that payment processors, cloud infrastructure providers, and advertising platforms that support such services face secondary liability risk. Organizations operating marketplace or API monetization programs should review whether their acceptable-use policies and payment controls can prevent commercial-scale nonconsensual synthetic-media operations from running on their infrastructure.

Governance controls affected

What to do now

  • ☐Review your acceptable-use policy for AI image-generation tools to confirm it explicitly prohibits creating sexual or intimate content depicting real, identifiable people without verified consent, and verify the policy is enforced technically, not just by written terms.
  • ☐Ask your trust-and-safety or content-moderation team whether your platform has automated detection controls that flag or block AI-generated sexual content involving real people before it is published or sold.
  • ☐Confirm that your payment processing and API monetization workflows include checks that would prevent a commercial nonconsensual deepfake service from billing customers through your infrastructure.
  • ☐Establish or update your victim-notification and complaint-response procedure to address synthetic intimate-image abuse specifically, including a named owner, a response timeline, and a takedown process.
  • ☐Brief your legal and compliance team on the Manhattan DA action and ask them to assess whether your current content governance controls meet the standard that prosecutors appear to be applying to commercial AI-image platforms.

What to watch next

State and federal prosecutors are increasingly framing commercial deepfake platforms as criminal enterprises rather than platform-moderation failures, and more seizures and criminal referrals are likely to follow. Compliance teams should monitor whether Congress or additional state legislatures attach criminal liability to platforms that knowingly profit from nonconsensual synthetic intimate content, as such laws would impose direct obligations on payment, hosting, and API providers. The 41% of CISOs Suffered Deepfake Voice Attacks: Approval Controls Must Adapt trend suggests that deepfake harm is expanding beyond image content into operational fraud, meaning synthetic-media controls will need to cover multiple modalities. Teams should also track the Federal Trade Commission's consumer protection posture on synthetic media, which could expand liability beyond the operators of deepfake sites to the platforms that distribute or monetize their outputs.

Stay ahead of stories like this

Get every US AI governance development like this one, plus the rest of the week's developments. Every Thursday.

Powered by Buttondown.

Related Coverage

Research2026-09-23

Embassy-Amplified BRICS Deepfake Exposes the Pre-Publication Verification Gap

An AI-generated image falsely depicting world leaders at the BRICS summit was shared by embassy social media accounts before fact-checking caught it. Resemble AI's Deepfake Watchlist documented the incident as a case study in how synthetic media gains institutional credibility through official amplification. The incident reveals that watermark awareness does not translate into actual provenance checks before content is published.

Enforcement2026-09-22

BC Sues OpenAI Over Alleged Safety Override Before School Shooting

British Columbia filed a lawsuit against OpenAI and CEO Sam Altman in September 2026, alleging that OpenAI overrode its own human review team's recommendation to share a user's violent ChatGPT chat logs with police before the February 2026 Tumbler Ridge Secondary School shooting. The province seeks compensation for rebuilding the school and covering emergency response costs. The suit also requests a court order requiring ChatGPT to automatically terminate violent conversations.

Enforcement2026-09-14

China's Supreme Court Makes Deepfakes and AI Hallucinations Judicially Actionable

China's Supreme People's Court issued judicial guidance establishing that deepfakes, voice clones, and certain AI hallucinations can trigger criminal or civil liability when they spread false information or damage reputations. The guidance specifically identifies provider inaction after harm reports as an aggravating factor. Enterprises serving Chinese users through AI-generated content pipelines now face enforceable legal exposure, not just regulatory risk.