How do we audit our AI governance program, not just individual AI systems?

Most AI audit activity in practice tests one system against one set of requirements: does this model meet the EU AI Act's documentation standard, does this hiring tool pass a bias assessment. That work matters, but it answers a narrower question than a board or regulator asking for assurance over the governance program as a whole. Program auditing asks whether the mechanisms that are supposed to catch problems across every AI system actually work, not whether one system happens to pass.

This distinction matters because a program can pass every individual system audit and still be structurally weak, if the systems selected for audit are not representative, if the audit only checks documentation rather than enforcement, or if newly deployed systems never enter the audit cycle at all. A program audit has to test the audit process itself, not just its outputs.

A useful audit needs concrete, testable maturity criteria for each control domain, not general descriptions like "processes are informal" versus "processes are mature." For a domain like monitoring, level 1 might mean no automated drift detection exists, level 3 might mean automated alerts exist but response is manual and undocumented, and level 5 might mean automated detection, a documented response runbook, and a closed-loop record of every alert and its resolution. Vague maturity language produces vague audit findings that are hard to act on.

Score domains independently. A program can have excellent documentation practices and weak monitoring at the same time, and averaging those into one number obscures exactly the gap the audit exists to find. Present maturity as a profile across domains, not a single grade.

The single most common audit failure mode is treating a written policy as evidence that a control exists. A human-oversight checkpoint that a bug can bypass, or an approval gate that exists procedurally but was never backed by a documented accuracy threshold, will both look compliant on paper. Real incidents confirm this pattern repeatedly: content moderation systems where a technical bypass let automated enforcement reach thousands of users, and inventory systems that reached production without the validation their approval process assumed had happened.

A rigorous audit samples actual system behavior, not just policy documents, and classifies every finding into whether a control is undocumented, documented but unenforced, or documented and enforced. The middle category is where the real risk concentrates, and it is invisible to an audit that only reviews paperwork.

Audits naturally gravitate toward the highest-visibility AI systems, the ones that already have executive attention and a paper trail. The systems most likely to have gaps are the ones added to the inventory recently, adopted by a single team without broad visibility, or embedded inside a vendor product where the organization does not control the underlying model. A sampling plan that draws from the full inventory, weighted toward newer and lower-visibility systems, will surface gaps that a review of only the flagship deployments will miss entirely.