Post-Deployment Adversarial Testing Cadence
Added May 2026
Schedule recurring red-teaming (simulated attacks) of live systems by risk tier. Continue this testing after pre-deployment assessments.
Objective
Detect new weak points attackers could exploit, changes in what the system can do, and safety regressions (safeguards that worked before but now fail) that emerge after deployment. These include problems introduced by model updates, prompt changes (edits to the instructions the system is given), new user populations, and novel attack techniques discovered after go-live.
Maturity Levels
Initial
Adversarial testing occurs only pre-deployment; live systems are not retested.
Developing
Ad hoc adversarial testing occurs when incidents prompt concern; no scheduled cadence exists.
Defined
A post-deployment adversarial testing schedule is defined, tiered by system risk level; test plans are documented and results are recorded.
Managed
Test results are compared against prior runs to track safety trajectory; regressions trigger remediation before the next release cycle.
Optimizing
Adversarial test suites update continuously with newly published attack techniques; automated tests that check for regressions run on every update to a live model.
Evidence Requirements
What an auditor or assessor would expect to see for this control.
- —Post-deployment adversarial testing schedule with risk-tiered cadence documentation
- —Test plans for each live system showing attack categories covered
- —Executed test reports with findings, severity ratings, and remediation records
- —Comparison reports showing safety trajectory across test cycles
- —Evidence that test suites are updated with newly published attack techniques
Implementation Notes
Key steps
- Establish a testing cadence tied to risk tier: Critical systems tested quarterly, Significant semi-annually, Limited annually at minimum.
- Keep post-deployment test suites (sets of planned attack tests) separate from pre-deployment suites; post-deployment testing should include attack patterns that have emerged since initial deployment.
- Include in each test: prompt injection (hidden instructions that make the system ignore its rules), jailbreak attempts (tricks to bypass safety limits) using techniques published since the last test, attempts to extract confidential data, and output manipulation targeting your specific use case.
- For high-risk sectors (healthcare, financial services), include sector-specific adversarial (simulated attack) scenarios relevant to your regulatory environment.
- Treat adversarial test reports as board-reportable for Critical systems; if a new attack pattern succeeds against a live system, that is a governance event.
Example Implementation
Customer-facing AI assistant for a financial services firm
Post-Deployment Adversarial Test Report: Q2 2026
System: Customer Service AI Assistant (Critical tier) Test date: 2026-05-28 | Prior test: 2026-02-19
New attack techniques tested since last cycle:
- Many-shot jailbreaking (published Feb 2026), FAILED (blocked by guardrails)
- Crescendo multi-turn escalation, PARTIAL PASS (escalation blocked at turn 4, not turn 2 as expected)
- Financial advice boundary probing (sector-specific), PASSED
Regressions vs. prior test: None identified. New finding: Crescendo resistance degraded. Guardrail threshold requires tuning. Remediation: Guardrail update deployed 2026-05-30. Retest scheduled 2026-06-05.