Backdoor Attacks

Backdoor attacks are a way of tampering with an AI model during training so that it behaves normally in most cases but produces attacker-chosen outputs when a specific hidden trigger appears in its input. A poisoned model can pass standard testing and still fail in production, which means a vendor's clean benchmark results do not fully prove the model is safe to deploy. For enterprises, this raises supply chain risk around third-party and open-source models, and it strengthens the case for provenance checks, adversarial testing, and contractual warranties from model suppliers.

1 item

ResearchGlobal2026-11-02

0.1% Pretraining Poison Embeds Backdoors That Survive Clean Fine-Tuning

Researchers have shown for the first time that contaminating as little as 0.1% of a model's pretraining dataset can implant a hidden backdoor. This backdoor persists through subsequent clean training steps. The backdoor, demonstrated in a command-line AI agent, allowed a remote attacker to take control of a developer's machine without any clicks or interaction. The finding, published in an arXiv paper by Xinyan Hu and colleagues, directly undermines the assurance value of post-training safety evaluations.

training data poisoning supply chain security agentic AI risk model integrity backdoor attacks AI procurement vendor due diligence