0.1% Pretraining Poison Embeds Backdoors That Survive Clean Fine-Tuning
What happened
A team of researchers published Agentic Backdoors via Pretraining Poisoning. The paper demonstrates that an attacker can tamper with a tiny fraction of raw training data. Doing so installs a hidden backdoor that clean fine-tuning and standard safety evaluations will not detect. The attack was tested on a command-line AI agent, a type of AI system that can run instructions directly on a computer. A four-billion-parameter model achieved roughly a 92% success rate at a poison level of just 0.1%. The backdoor allows an attacker to remotely take over any developer machine running the compromised agent, without the user clicking anything or doing anything unusual. The result challenges a foundational assumption in enterprise AI procurement: that a vendor's post-training pipeline, safety fine-tuning, and evaluation reports are sufficient evidence of a model's trustworthiness.
Why it matters
- ·Training-data provenance programs that stop at the fine-tuning layer now have a documented blind spot. An adversary who contaminates raw pretraining data leaves no trace that post-training audits or benchmark results can detect. Vendor-supplied assurance evidence cannot rule out this class of threat.
- ·Organizations deploying agentic AI tools that can execute code or run commands on internal systems face elevated risk. A backdoored agent operating with developer-level access to production systems could allow a remote attacker to take over machines, exfiltrate data, or move laterally across infrastructure without triggering standard security alerts.
- ·Procurement and vendor due-diligence controls built around safety evaluations and fine-tuning disclosures are now demonstrably insufficient for high-privilege agent deployments. Compliance teams relying on a vendor's safety card or post-training audit as the primary assurance signal have no way to verify the integrity of the underlying pretraining corpus.
Stay ahead of stories like this
Get every Global AI governance development like this one, plus the rest of the week's developments. Every Thursday.
Recent issues
- Three separate incidents this week each created a new compliance action item: verify your AI vendors' data sourcing, demand agent monitoring coverage guarantees, and audit trusted app identities in your cloud environment.29 Oct
- Regulators, litigants, and attackers are all moving faster than most organizations' AI governance frameworks, shifting leverage away from deployers and toward those who can prove control, accountability, and honest disclosure.21 Oct
Free every Thursday. Unsubscribe anytime.
Governance controls affected
What to do now
- ☐Identify every AI agent in your environment that can execute code, run commands, or take actions on internal systems, and document the source of each model's pretraining data.
- ☐Ask your AI vendors to disclose whether their pretraining corpus was sourced from third parties, whether provenance logging exists at the raw-data level, and whether any independent audit has reviewed the corpus for tampering.
- ☐Review whether your current AI procurement contracts require vendors to notify you of known or suspected pretraining data integrity issues, and add that obligation if it is absent.
- ☐Restrict command-line and code-executing AI agents to isolated environments with limited system access until you can verify the integrity of the underlying model's pretraining data.
- ☐Add pretraining data integrity as a named risk in your AI risk register, and flag any agentic deployment where the vendor cannot provide verifiable pretraining provenance.
What to watch next
The research is currently a preprint and has not yet been peer-reviewed; compliance teams should monitor for formal publication and any industry or regulatory response. NIST AI Risk Management Framework (AI RMF 1.0) and Playbook and the Five Eyes Guidance on the Careful Adoption of Agentic AI Services may update their supply-chain guidance in light of findings like this one. Regulators focused on model risk may also respond. Banking supervisors building on the Interagency Revised Guidance on Model Risk Management (OCC Bulletin 2026-13, SR 26-2) may begin asking whether financial firms' model validation processes extend to pretraining data integrity.