Backdoor AI

A backdoor in an AI model is a hidden behavior planted during training or fine-tuning that stays dormant until a specific trigger, such as an unusual phrase or pattern in the input, causes the model to act in a way its developer never intended. The model can pass ordinary testing and appear to work normally, which is why this risk matters for any organization that uses third-party or open-source models it did not train itself. For governance teams, it raises questions about model provenance, supplier due diligence, and whether testing covers adversarial triggers as well as typical use.

1 item

Research2026-10-09

Research Proves AI Models Can Hide Capabilities That No Inspection Can Find

Researchers from the University of Oxford have published a paper on arXiv showing that AI models can contain hidden capabilities activated only by a secret trigger. These hidden functions cannot be detected through weight inspection, fine-tuning, or any other currently available technique. The findings challenge the assumption that examining a model's internal structure is sufficient to verify its safety. Organizations that rely on pre-deployment model evaluation as a safety guarantee must now treat that guarantee as incomplete.

model evaluation hidden capabilities backdoor AI vendor due diligence open-weight models EU AI Act University of Oxford