Glasswing's 94.2% Accuracy Claim Conflicts with Independent Evidence
What happened
VulnCheck researcher Patrick Garrity published After 7-months, Glasswing Overwelms Maintainers with Underwhelming Findings on November 9, 2026. The report tracked seven months of Anthropic's Project Glasswing AI vulnerability disclosure program using CVE records, advisories, and Anthropic's own public disclosure ledger. Of 37,966 findings reported on the ledger as of November 2, 2026, only 7,349 have been made public and just 217 have been marked fixed. Since September, Anthropic added nearly 12,000 findings while recording only 15 additional fixes. Duplicates rose from 2 to 1,654 during that same period, nearly eight times the number of confirmed fixes. When curl maintainer Daniel Stenberg reviewed 34 of 45 Anthropic-submitted reports, 19 were rejected as non-vulnerabilities, 6 were confirmed as low severity, and 9 remained under review. Anthropic's pivot to an unreviewed open-source scanner, launched October 8, 2026, explicitly shifts triage to maintainers, with Anthropic disclosing that reports are model-generated without human review.
Why it matters
- ·Vendor capability claims used to justify procurement decisions or internal risk assessments may not hold up under independent review. Compliance teams that cited Anthropic's claimed 94.2% true positive rate or program scale as evidence of safety leadership should recheck whether those metrics were independently verified before being recorded in vendor risk files.
- ·The shift to fully model-generated, unreviewed vulnerability reports transfers work and accountability to open-source maintainers and, by extension, to any enterprise depending on those projects. This creates supply chain risk exposure when downstream software libraries absorb a flood of unverified security reports that slow patch cycles.
- ·Regulators and auditors increasingly expect organizations to substantiate AI capability claims rather than accept vendor self-reporting. Where vendor assurance programs reference AI-generated output volume or accuracy rates, those figures now require independent corroboration before they can be treated as evidence of control effectiveness.
Stay ahead of stories like this
Get every Global AI governance development like this one, plus the rest of the week's developments. Every Thursday.
Recent issues
- State-linked attackers and criminal operators are actively weaponizing AI tools as entry points into enterprise systems, turning internal platforms and open-source frameworks into live attack vectors.5 Nov
- Three separate incidents this week each created a new compliance action item: verify your AI vendors' data sourcing, demand agent monitoring coverage guarantees, and audit trusted app identities in your cloud environment.29 Oct
Free every Thursday. Unsubscribe anytime.
Governance controls affected
What to do now
- ☐Review any vendor risk assessments or audit files that cited Anthropic's Project Glasswing statistics, such as true positive rates or finding counts, and flag them for re-evaluation against the independent VulnCheck findings.
- ☐Ask your security or engineering team whether any open-source libraries your organization depends on have received bulk unreviewed vulnerability reports from AI-driven disclosure programs, and confirm whether those reports have been assessed by the maintainers.
- ☐Update your vendor due diligence questionnaire to ask AI vendors whether safety program metrics are self-reported or independently verified, and whether AI-generated outputs in those programs receive human review before disclosure.
- ☐Check whether your organization's AI governance program accepts vendor capability claims, such as accuracy rates or safety benchmark scores, without a corroboration step, and add an independent verification requirement if none exists.
- ☐Brief your legal or risk team on the gap between Anthropic's disclosed metrics and curl's on-the-ground rejection rate, so that any public statements your organization has made about Glasswing's effectiveness can be reviewed before regulatory inquiries arise.
What to watch next
Compliance teams should monitor whether Anthropic updates its disclosure ledger methodology. Key questions raised by VulnCheck concern duplicate and withdrawal accounting and how those categories affect the true positive rate calculation. Broader industry attention to AI-driven vulnerability disclosure programs is growing. Regulators focused on AI transparency under frameworks such as the EU AI Act (Regulation (EU) 2024/1689) may treat program accuracy claims as capability disclosures subject to verification requirements. Watch for whether open-source foundations or maintainer communities issue formal guidance on bulk AI-generated security reports. Such guidance would affect enterprise software supply chain governance obligations.