Frontier API Reasoning Traces Leaked 62 Live API Keys in Public Agent Logs
Source
Stealing Reasoning Traces from Proprietary LLM APIsMATS Research / ELLIS Institute Tubingen / Max Planck Institute for Intelligent Systems
What happened
A team from MATS Research, the ELLIS Institute Tubingen, and the Max Planck Institute for Intelligent Systems published Stealing Reasoning Traces from Proprietary LLM APIs, documenting two related vulnerabilities across major frontier model providers. First, encrypted chain-of-thought reasoning blocks returned by Anthropic, OpenAI, and Google APIs can be replayed across different sessions, users, and models, allowing an attacker to inject a stronger model's reasoning trace into a jailbroken weaker sibling from the same provider and recover verbatim reasoning from frontier models including Claude Opus and Gemini variants. Second, the researchers analyzed 6,708 publicly available agent trajectory datasets and reconstructed 315,320 reasoning blocks, uncovering 704 distinct privacy artifacts: 62 API keys, 33 passwords, and 24 access tokens, with 64 artifacts present only inside reasoning blocks that are never surfaced to end users. This finding builds on existing concerns about AI API data exposure, as covered in CASB and DLP Cannot See Inside AI Prompts, and confirms that the blind spot extends well beyond prompt content to include the model's own internal reasoning layer. Organizations that have shared or published agent trajectory logs for research or benchmarking purposes may have already exposed live credentials without any awareness of the leakage.
Why it matters
- ·Secrets management and DLP programs are structurally blind to this risk: because reasoning blocks are encrypted and not surfaced to end users, most enterprise tooling does not scan them, meaning live credentials can travel through and out of AI pipelines entirely undetected by existing controls.
- ·Any organization that has published agent trajectory data for research, benchmarking, or transparency purposes should treat those datasets as potentially compromised. The OWASP Top 10 for Large Language Model Applications flags sensitive information disclosure as a primary LLM risk, and this research demonstrates that the risk materializes in data artifacts most teams have never audited.
- ·The cross-session replay capability means that credential leakage is not limited to the organization whose data originally trained or prompted the model. A weaker model receiving a replayed reasoning trace from a different user or session can expose secrets from entirely separate tenants, introducing a third-party and supply chain dimension to what might otherwise be treated as an internal API security issue.
Governance controls affected
What to do now
- ☐Audit all agent trajectory logs, benchmarking datasets, and research artifacts that have been published or shared externally, and rotate any credentials that may have been present in agent inputs or outputs at the time of publication.
- ☐Extend secrets-scanning tooling to cover AI API response payloads, including any intermediate or debug outputs that capture model reasoning, not just final user-facing outputs.
- ☐Review vendor contracts with Anthropic, OpenAI, and Google to confirm whether encrypted reasoning traces are included in data-retention, cross-session isolation, and tenant separation commitments, and escalate gaps to vendor security contacts.
- ☐Update your AI API credential management policy (SEC-002) to treat reasoning-trace outputs as a potential data-loss vector, requiring that no live secrets be passed in prompts or system context to models that return chain-of-thought blocks.
- ☐Run a targeted tabletop exercise simulating the credential exfiltration scenario described in the research, specifically testing whether your incident response playbook covers credentials leaked through AI reasoning layers rather than through traditional data channels.
What to watch next
Compliance teams should monitor whether Anthropic, OpenAI, and Google issue formal security advisories or API changes in response to this research, as any patch or architectural change affecting reasoning-block handling may require reassessment of existing vendor contracts and integration configurations. Regulatory bodies with jurisdiction over data handling in AI pipelines, including the UK ICO under the UK ICO Guidance on Artificial Intelligence and Data Protection, may treat publicly exposed credentials in agent logs as notifiable incidents depending on the sensitivity of the artifacts involved. The research also reinforces pressure on enterprises to treat AI pipeline outputs as within scope for existing data-loss prevention audits, a shift that is likely to surface in forthcoming NIST AI 600-1 Generative AI Profile implementation guidance and sectoral AI risk frameworks.
Stay ahead of stories like this
Get every Global AI governance development like this one, plus the rest of the week's developments. Every Thursday.