Training Data Poisoning

Training data poisoning is an attack in which someone deliberately inserts manipulated or mislabeled examples into the data an AI model learns from, so the finished model behaves incorrectly in ways the developer did not intend. The corrupted behavior can be narrow and hard to spot, such as a model that misclassifies a specific trigger or quietly favors an attacker's content, which means ordinary testing may miss it. For enterprises, the risk sits upstream: any model trained on scraped web data, third-party datasets, or user-submitted content needs documented data sourcing, provenance checks, and post-deployment monitoring to show regulators and auditors that the model's outputs can be trusted.

1 item

ResearchGlobal2026-11-02

0.1% Pretraining Poison Embeds Backdoors That Survive Clean Fine-Tuning

Researchers have shown for the first time that contaminating as little as 0.1% of a model's pretraining dataset can implant a hidden backdoor. This backdoor persists through subsequent clean training steps. The backdoor, demonstrated in a command-line AI agent, allowed a remote attacker to take control of a developer's machine without any clicks or interaction. The finding, published in an arXiv paper by Xinyan Hu and colleagues, directly undermines the assurance value of post-training safety evaluations.

training data poisoning supply chain security agentic AI risk model integrity backdoor attacks AI procurement vendor due diligence