13% Jailbreak Rate on OpenAI and Google Image Models Exposes Vendor Safety Gap

What happened

Why it matters

• ·Enterprises that treat vendor content filters as a reliable control now have documented evidence those filters can be bypassed at scale with widely available tools. This forces a reassessment of where content safety accountability sits and whether supplementary controls are needed.
• ·Newer model versions are more vulnerable than older ones. Routine model upgrades, including those pushed automatically by vendors, can silently worsen an organization's content risk posture. Most current NIST AI Risk Management Framework (AI RMF 1.0) and Playbook-aligned programs would not trigger an internal review for such upgrades.
• ·Organizations in regulated sectors, including media, healthcare, and financial services, face potential liability if harmful content is generated through their platforms. The 13 percent bypass rate is high enough to be reproducible at modest scale for those using AI image tools in customer-facing workflows.

Governance controls affected

What to do now

• ☐Identify every AI image generation tool your organization uses via API or embedded in products, and confirm which model version is currently active, since newer versions appear more vulnerable.
• ☐Do not rely solely on vendor content filters as your safety control: add an independent output review layer or automated content screening before AI-generated images reach end users.
• ☐Update your vendor safety assessment for OpenAI and Google image products to reflect this finding, and ask each vendor what testing they performed before releasing ChatGPT Images 2.5 and Nano Banana 2.1.
• ☐Establish a written policy requiring re-assessment of content safety controls whenever a vendor upgrades the underlying model, not only when your organization makes a deliberate configuration change.
• ☐Run your own adversarial tests against current image model versions you have deployed, using the RISE research as a reference point, and document the results for audit purposes.

What to watch next

Compliance teams should monitor whether OpenAI and Google issue formal incident disclosures. Updated safety evaluations for ChatGPT Images 2.5 and Nano Banana 2.1 should also be tracked following the White Circle findings. Regulatory bodies in the EU can inspect AI systems under the EU AI Act (Regulation (EU) 2024/1689). The ICO in the UK has already opened investigations into harmful synthetic image generation, as seen in ICO Opens Formal Grok Investigation Into Data Processing and Harmful Synthetic Images. Both may treat documented bypass rates as evidence of inadequate safety measures. Watch for enforcement signals from the FTC, which has flagged AI harm and nondisclosure as active priorities. Watch also for whether this research prompts mandatory pre-release adversarial testing requirements in pending legislation.