13% Jailbreak Rate on OpenAI and Google Image Models Exposes Vendor Safety Gap
What happened
White Circle researchers published a preprint on production image models from OpenAI and Google, including DALL-E 3 and GPT-Image-2. The models can be manipulated to produce explicit content approximately 13 percent of the time. Only commodity tools and standard programming interfaces are needed. The technique, described in coverage by Ars Technica, is called RISE, short for Red-teaming via Iterative Strategy Evolution. RISE uses one widely available AI model to craft manipulation attempts and a second to evaluate whether those attempts succeeded, requiring no frontier AI access. Subsequent testing of newer releases, ChatGPT Images 2.5 and Google's Nano Banana 2.1, found both to be significantly more vulnerable than their predecessors. Neither OpenAI nor Google identified or disclosed the regression before external researchers did.
Why it matters
- ·Enterprises that treat vendor content filters as a reliable control now have documented evidence those filters can be bypassed at scale with widely available tools. This forces a reassessment of where content safety accountability sits and whether supplementary controls are needed.
- ·Newer model versions are more vulnerable than older ones. Routine model upgrades, including those pushed automatically by vendors, can silently worsen an organization's content risk posture. Most current NIST AI Risk Management Framework (AI RMF 1.0) and Playbook-aligned programs would not trigger an internal review for such upgrades.
- ·Organizations in regulated sectors, including media, healthcare, and financial services, face potential liability if harmful content is generated through their platforms. The 13 percent bypass rate is high enough to be reproducible at modest scale for those using AI image tools in customer-facing workflows.
Stay ahead of stories like this
Get every Global AI governance development like this one, plus the rest of the week's developments. Every Thursday.
Recent issues
- Three separate incidents this week each created a new compliance action item: verify your AI vendors' data sourcing, demand agent monitoring coverage guarantees, and audit trusted app identities in your cloud environment.29 Oct
- Regulators, litigants, and attackers are all moving faster than most organizations' AI governance frameworks, shifting leverage away from deployers and toward those who can prove control, accountability, and honest disclosure.21 Oct
Free every Thursday. Unsubscribe anytime.
Governance controls affected
What to do now
- ☐Identify every AI image generation tool your organization uses via API or embedded in products, and confirm which model version is currently active, since newer versions appear more vulnerable.
- ☐Do not rely solely on vendor content filters as your safety control: add an independent output review layer or automated content screening before AI-generated images reach end users.
- ☐Update your vendor safety assessment for OpenAI and Google image products to reflect this finding, and ask each vendor what testing they performed before releasing ChatGPT Images 2.5 and Nano Banana 2.1.
- ☐Establish a written policy requiring re-assessment of content safety controls whenever a vendor upgrades the underlying model, not only when your organization makes a deliberate configuration change.
- ☐Run your own adversarial tests against current image model versions you have deployed, using the RISE research as a reference point, and document the results for audit purposes.
What to watch next
Compliance teams should monitor whether OpenAI and Google issue formal incident disclosures. Updated safety evaluations for ChatGPT Images 2.5 and Nano Banana 2.1 should also be tracked following the White Circle findings. Regulatory bodies in the EU can inspect AI systems under the EU AI Act (Regulation (EU) 2024/1689). The ICO in the UK has already opened investigations into harmful synthetic image generation, as seen in ICO Opens Formal Grok Investigation Into Data Processing and Harmful Synthetic Images. Both may treat documented bypass rates as evidence of inadequate safety measures. Watch for enforcement signals from the FTC, which has flagged AI harm and nondisclosure as active priorities. Watch also for whether this research prompts mandatory pre-release adversarial testing requirements in pending legislation.