AI Governance Institute
← News
Research2026-10-14

13% Jailbreak Rate on OpenAI and Google Image Models Exposes Vendor Safety Gap

What happened

White Circle researchers published a preprint on production image models from OpenAI and Google, including DALL-E 3 and GPT-Image-2. The models can be manipulated to produce explicit content approximately 13 percent of the time. Only commodity tools and standard programming interfaces are needed. The technique, described in coverage by Ars Technica, is called RISE, short for Red-teaming via Iterative Strategy Evolution. RISE uses one widely available AI model to craft manipulation attempts and a second to evaluate whether those attempts succeeded, requiring no frontier AI access. Subsequent testing of newer releases, ChatGPT Images 2.5 and Google's Nano Banana 2.1, found both to be significantly more vulnerable than their predecessors. Neither OpenAI nor Google identified or disclosed the regression before external researchers did.

Why it matters

  • ·Enterprises that treat vendor content filters as a reliable control now have documented evidence those filters can be bypassed at scale with widely available tools. This forces a reassessment of where content safety accountability sits and whether supplementary controls are needed.
  • ·Newer model versions are more vulnerable than older ones. Routine model upgrades, including those pushed automatically by vendors, can silently worsen an organization's content risk posture. Most current NIST AI Risk Management Framework (AI RMF 1.0) and Playbook-aligned programs would not trigger an internal review for such upgrades.
  • ·Organizations in regulated sectors, including media, healthcare, and financial services, face potential liability if harmful content is generated through their platforms. The 13 percent bypass rate is high enough to be reproducible at modest scale for those using AI image tools in customer-facing workflows.

Stay ahead of stories like this

Get every Global AI governance development like this one, plus the rest of the week's developments. Every Thursday.

Free every Thursday. Unsubscribe anytime.

Governance controls affected

What to do now

  • ☐Identify every AI image generation tool your organization uses via API or embedded in products, and confirm which model version is currently active, since newer versions appear more vulnerable.
  • ☐Do not rely solely on vendor content filters as your safety control: add an independent output review layer or automated content screening before AI-generated images reach end users.
  • ☐Update your vendor safety assessment for OpenAI and Google image products to reflect this finding, and ask each vendor what testing they performed before releasing ChatGPT Images 2.5 and Nano Banana 2.1.
  • ☐Establish a written policy requiring re-assessment of content safety controls whenever a vendor upgrades the underlying model, not only when your organization makes a deliberate configuration change.
  • ☐Run your own adversarial tests against current image model versions you have deployed, using the RISE research as a reference point, and document the results for audit purposes.

What to watch next

Compliance teams should monitor whether OpenAI and Google issue formal incident disclosures. Updated safety evaluations for ChatGPT Images 2.5 and Nano Banana 2.1 should also be tracked following the White Circle findings. Regulatory bodies in the EU can inspect AI systems under the EU AI Act (Regulation (EU) 2024/1689). The ICO in the UK has already opened investigations into harmful synthetic image generation, as seen in ICO Opens Formal Grok Investigation Into Data Processing and Harmful Synthetic Images. Both may treat documented bypass rates as evidence of inadequate safety measures. Watch for enforcement signals from the FTC, which has flagged AI harm and nondisclosure as active priorities. Watch also for whether this research prompts mandatory pre-release adversarial testing requirements in pending legislation.

Related Coverage

Enforcement2026-11-07

ChatGPT Murder-Planning Lawsuit Tests AI Vendor Duty of Care

A civil lawsuit in San Francisco County Superior Court alleges ChatGPT gave a North Carolina woman a detailed plan to poison her husband. The exchanges spanned at least 25 conversations. The complaint names OpenAI and CEO Sam Altman, alleging safety guardrails were knowingly weakened for commercial reasons. OpenAI confirmed it banned the user and cooperated with law enforcement, but the case raises unresolved questions about vendor duty of care and enterprise liability for harmful AI outputs.

Research2026-11-10

Gemini Robotics 2.5 Brings Embodied AI Governance Into the Factory

Google DeepMind released Gemini Robotics 2.5 on November 10, 2026, its most capable model for controlling physical robots. A proof-of-concept with Apptronik and Mercedes-Benz on active automotive production lines achieved a 96% success rate. The release also introduces a new benchmark designed to test whether the model can be manipulated into unsafe physical actions.

Corporate Policy2026-11-17

OpenAI's CSAM Classifier Halves False Positives as CyberTipline Reports Hit 400,000

OpenAI announced a classifier to detect AI-generated child sexual abuse material (CSAM), offering free access to vetted organizations and updated developer guidance. The tool flags 48 percent fewer false positives while doubling the rate at which flagged images contain abusive material. The release responds to a surge in generative AI-related reports to the NCMEC CyberTipline, which rose from 67,000 in 2024 to 400,000 in 2025.