AI Governance Institute
← News
Enforcement2026-08-28

Court Rules Pentagon Blacklisted Anthropic Illegally Over AI Safety Restrictions

What happened

A federal judge in the Northern District of California ruled that the Pentagon had illegally blacklisted Anthropic by designating it a supply chain risk, in a case reported by The Verge. The court found the designation constituted unlawful First Amendment retaliation, triggered by Anthropic's refusal to strip policy restrictions from its AI systems prohibiting use in mass surveillance of Americans and lethal autonomous weapons applications. After Anthropic held firm on those restrictions, the Defense Department replaced it with seven other AI vendors. The ruling is notable because it treats a vendor's published acceptable use policy as a form of protected expression, not merely a commercial term subject to renegotiation under procurement pressure. This sets a precedent that government agencies cannot penalize AI vendors for maintaining safety guardrails as a condition of doing business, which reshapes the risk calculus for both vendors and the enterprises that rely on them.

Why it matters

  • ·Vendor-imposed AI safety restrictions are now more legally durable than many compliance teams have assumed. If a government agency retaliates against a vendor for maintaining acceptable use guardrails, the ruling suggests that retaliation itself may be the unlawful act, not the restriction.
  • ·Enterprises with government contracts that use AI vendors with published dual-use or weapons-related restrictions face new procurement exposure. Agencies may pressure prime contractors to substitute vendors or demand policy waivers, putting enterprise vendor selection and PRC-002 obligations in direct tension with agency expectations.
  • ·The case also sharpens the distinction between AI safety commitments that are genuinely enforced and those that yield under commercial or political pressure. Compliance teams should review whether their vendor agreements specify what happens when a vendor's acceptable use policy is challenged by a government authority, because this ruling shows that challenge is now a real and litigated risk.

Governance controls affected

What to do now

  • Audit current AI vendor contracts to identify any that include dual-use, weapons, or surveillance restrictions, and assess whether those restrictions create procurement eligibility risk under applicable government contract requirements.
  • Request written confirmation from AI vendors on whether their acceptable use policies are firm commitments or subject to waiver under government or regulatory pressure, and document those responses in your vendor risk files.
  • Brief legal counsel on the Northern District of California ruling and evaluate whether your organization's own AI policies or those of your vendors contain restrictions that could attract similar government scrutiny or retaliation.
  • Update your AI vendor due diligence questionnaire to specifically ask how vendors handle government demands to modify or remove safety-related acceptable use restrictions.
  • If your organization holds or seeks government contracts, map which AI vendors underpin which contract deliverables and assess substitution risk if a vendor maintains policy positions incompatible with agency requirements.

What to watch next

Appeals court proceedings will determine whether the First Amendment protection for vendor AI safety policies extends beyond the Northern District of California, making this a developing area of law with direct procurement implications. Compliance teams should also monitor whether other federal agencies adopt similar supply chain risk designations as a procurement tool, which would signal a broader pattern requiring proactive vendor contract governance. The interplay between this ruling and voluntary federal AI safety commitments, including those tracked under the White House Finalizes Voluntary Frontier AI Safety Testing With Top Labs process, will shape how vendors balance safety commitments against government contracting risk.

Stay ahead of stories like this

Get every US AI governance development like this one, plus the rest of the week's developments. Every Thursday.

Powered by Buttondown.

Related Coverage

Enforcement2026-08-29

Sony and Warner Sue Anthropic Over Training Data, Exposing Vendor IP Risk

Sony Music and Warner Chappell have filed a copyright infringement lawsuit against Anthropic in the US District Court for the Northern District of California, alleging that tens of thousands of protected works were used to train Claude without authorization. The complaint seeks up to $150,000 per infringed work and up to $25,000 per instance of stripped copyright metadata, with total exposure potentially reaching several billion dollars. Co-founders Dario Amodei and Benjamin Mann are named as individual defendants.

Enforcement2026-08-27

Grok CSAM Lawsuit Sets a Training Data Provenance Liability Benchmark

A federal lawsuit filed by a child sex abuse material survivor alleges that xAI trained its Grok models on CSAM identified via hash lists maintained by NCMEC and the Canadian Centre for Child Protection. The complaint also alleges that xAI's terms of service create a training pipeline that recycles public posts and model outputs without explicit exclusion categories for illegal content. Enterprise compliance teams now have a concrete litigation template against which to audit their own training data provenance and vendor due diligence controls.

Enforcement2026-09-07

DC Court Sanctions Deutsche Bank Lawyers Over AI-Hallucinated Case Citations

The District of Columbia Court of Appeals faulted lawyers representing a Deutsche Bank subsidiary after they filed a brief citing nonexistent cases apparently generated by AI. The court's rebuke highlights a direct control failure: no citation verification step and inadequate human review before submission. The incident adds to a growing body of judicial enforcement actions against AI-assisted legal work product.