Adversarial Attacks

Adversarial attacks are carefully crafted inputs designed to fool AI models into making incorrect predictions or decisions, often by adding subtle perturbations that humans would not notice. Understanding and defending against these attacks is critical for enterprise AI governance because they represent a significant security and reliability risk, particularly for high-stakes applications like fraud detection, autonomous vehicles, and medical diagnosis systems. Organizations must implement robust testing and defensive mechanisms to ensure their AI systems remain trustworthy and resistant to manipulation by malicious actors.

1 item

ResearchGlobal2026-08-25

InjecMEM Plants Persistent Agent Instructions via Single Prompt, 76.6% Success Rate

Researchers from Shanghai Jiao Tong University and Ant Group have demonstrated InjecMEM, an attack technique that injects malicious instructions into AI agent memory systems through a single ordinary interaction, without requiring direct access to the memory store. The attack persists across sessions, achieving a 76.6% success rate against the MemoryOS system. Experts warn that inference-time input and output filtering, the most common enterprise defense, does not stop this class of attack.

prompt injection memory poisoning agentic AI AI agent security adversarial attacks Ant Group red teaming