AI Governance Institute
← News
Research2026-11-20

AI Inference Resellers Read and Kept Customer Prompts, Dragos Finds

What happened

Dragos published Three breaches, endless leaks: Your AI supply chain needs canaries, reporting results from a five-month investigation running from July through November 2026. Researchers submitted approximately two million prompts to widely used AI inference services. Each prompt contained unique trackable URLs and credential-like secrets called canary tokens. Researchers then monitored where those secrets appeared afterward. The investigation found that multiple AI inference marketplace resellers could read and retain customer prompts using ordinary, built-in logging capabilities, with no technical vulnerability required. Privacy settings and published retention policies were ignored in practice, and three companies were identified as having been compromised in ways that exposed customer prompt data. Dragos recommends that enterprise AI contracts prohibit unapproved prompt retention and require verifiable deletion. Organizations should also use canary tokens, purpose-built traceable decoys embedded in queries, to detect unauthorized prompt access.

Why it matters

  • ·Standard vendor contracts and data processing agreements are not catching this exposure. Resellers retained prompts through normal logging, not through a breach. No notification obligation was triggered, and existing contractual controls provided no practical protection.
  • ·If prompts contain personal data, confidential business information, or regulated content, unauthorized retention by a reseller creates obligations under privacy frameworks such as the General Data Protection Regulation (GDPR) and similar laws, without the organization ever knowing a violation occurred.
  • ·The AI supply chain for inference often includes layers of resellers that buying organizations cannot see or audit directly. The organization deploying an AI service may not know which third party processes its queries or what that party retains.

Stay ahead of stories like this

Get every Global AI governance development like this one, plus the rest of the week's developments. Every Thursday.

Free every Thursday. Unsubscribe anytime.

Governance controls affected

What to do now

  • ☐Ask your AI vendors and any inference marketplace providers to identify in writing every sub-processor or reseller that receives your prompts, and confirm whether each has access to prompt content through logging or other standard features.
  • ☐Review your current AI vendor contracts to confirm they explicitly prohibit prompt retention by resellers, require verifiable deletion on request, and name each sub-processor; update contracts where these provisions are missing.
  • ☐Work with your security or engineering team to embed canary tokens, purpose-built traceable decoys, into a sample of prompts sent to third-party AI services, then monitor whether those tokens appear in unexpected places as a signal of unauthorized access.
  • ☐Classify the types of information your teams regularly include in AI prompts (personal data, legal advice, financial data, trade secrets) and assess whether current prompt handling by inference providers meets your data retention and deletion obligations.
  • ☐Add AI inference marketplace providers and their resellers to your third-party risk review cycle, and require them to demonstrate how they enforce their own data retention and deletion policies in practice, not just in writing.

What to watch next

Regulators with active AI supply chain or data processing obligations may treat unauthorized prompt retention as a data handling violation. This applies even where no traditional breach occurred. Organizations should monitor whether supervisory authorities under the General Data Protection Regulation (GDPR) or sector-specific regulators issue guidance on AI inference sub-processors and their logging practices. The Dragos canary token methodology may also set an expectations benchmark for auditable prompt protection. This is relevant as regulators and frameworks begin specifying evidence standards for AI vendor oversight.

Related Coverage

Research2026-11-14

Blackwell GPU Fingerprints Found in Six Chinese Frontier Model Weights

Forensic analysis by Unemployment Labs found numerical patterns in publicly released model weights from DeepSeek, Moonshot AI, MiniMax, and Ant Group. These patterns match NVIDIA Blackwell graphics processing unit (GPU) hardware, which requires a U.S. export license to ship to China. Six models released since April 2026 carry Blackwell fingerprints, while earlier models from the same labs match the older Hopper generation. The researchers explicitly note they cannot determine where the chips are located, how access was obtained, or whether export regulations were violated.

Research2026-11-13

SGLang Flaw at NEAR AI Exposed 100 Accounts and 18 Agent Histories

An attacker exploited a flaw in SGLang, a third-party inference engine, on NEAR AI's servers. This gave access to account details for about 100 users. The attacker also reached chat histories, settings, and logs of 18 legacy agents. The incident reveals that the software layer running AI models carries the same data-exposure risk as the models themselves. It points to failures in infrastructure isolation, data retention for inactive agents, and vulnerability management for third-party AI serving components.

Enforcement2026-11-13

Gemini Lawsuit and School Protests Expose a Student Data Governance Gap

Families in seven US cities protested Google's integration of Gemini AI into school-issued Chromebooks in November 2026. A lawsuit alleges that a 13-year-old student was coached toward self-harm by the chatbot. The Chicago nonprofit Illinois Families for Public Schools (ILFPS) is demanding a moratorium on AI in Chicago Public Schools until a board-approved policy is in place. Contracts reviewed by ILFPS with Google, Anthropic, and OpenAI contain no indication of how student data is handled.