All governance templates →← How do we audit an AI system for compliance?

Implementation Kit

AI System Audit Checklist and Findings Report Template

How to audit one AI system against its obligations: a scope and methodology template, a document review checklist by tier, a bias-testing verification worksheet, a human oversight interview guide, and a findings template.

Who this is for: The internal auditor or external assessor examining a specific AI system.

Download the kit (Markdown) ↓5 artifacts. Every table also copies as CSV.

1. Audit scope and methodology template

Document

Sets what is being audited, against what, and how, before fieldwork.

Template

Agreed with the system owner before fieldwork.

System and version:
Obligations in scope: the specific regulations, internal standards, and control set
Audit questions: the 5-8 questions the audit will answer
Methods: document review, re-performance (bias tests, sample decisions), interviews, system observation
Sampling approach: how decisions and controls are selected for testing, not owner-chosen
Out of scope: and why
Timeline, team, and reporting route

Worked example

System: resume-screener v2.4.0
Obligations: EU AI Act Chapter III Section 2; NYC LL144; internal controls HOC-001/002, MON-003, ALC-001/002.
Audit questions: Is classification correct? Is bias testing run and acted on? Is human oversight meaningful in practice? Are decisions reconstructable? Is the documentation current?
Methods: document review; re-perform disparate-impact on a fresh sample; trace 25 random decisions; interview 5 reviewers; observe the review UI.
Sampling: decisions selected by the auditor via random seed from the full log; controls tested on the auditor's sample.
Out of scope: the base model vendor's internals (covered by vendor DD).
Timeline: 3 weeks; report to the Audit Committee.

Acceptance criteria

✓Scope names the specific obligations and controls, not "compliance" in general.
✓Sampling is auditor-controlled, not owner-provided.
✓The audit questions are answerable with the chosen methods.

2. Document review checklist

Spreadsheet

The documents that should exist for the system's tier, with a status per document.

Template

Document	Present	Current	Adequate
Risk assessment (pre-deployment)	Y/N	Y/N	Y/N
Model card
Data governance record
Bias / fairness evaluation
Human oversight arrangement
Technical documentation (Annex IV, if EU high-risk)
Change / version history
Monitoring plan and recent results
Incident history

Worked example

Document	Required?	Present	Current	Adequate	Notes
Risk assessment	Yes	Y	Y	Y	signed 2026-09
Model card	Yes	Y	N	Partial	still describes v2.3
Bias evaluation	Yes	Y	Y	Y	2026-08
Human oversight arrangement	Yes	N	-	-	finding: not written up
Annex IV technical doc	Yes	Y	Partial	Partial	sections 6-7 missing
Monitoring plan + results	Yes	Y	Y	Partial	monthly only; no alerting

Acceptance criteria

✓Every required document is checked for presence, currency, and adequacy separately.
✓"Present but stale" and "present but inadequate" are captured, not just present or missing.
✓Each shortfall becomes a finding.

3. Bias testing verification worksheet

Spreadsheet

The auditor re-performs, rather than relying on the owner's report.

Template

Step	Owner-reported	Auditor re-performed	Match?	Note
Groups tested
Disparate-impact ratios
Subgroup error rates
Threshold used
Remediation claimed	verify it is actually in the deployed model

Worked example

Step	Owner-reported	Auditor re-performed	Match?	Note
Groups tested	sex, race/ethnicity, age band	same, plus sex x race	partial	owner did not test the intersection
Disparate-impact ratio (sex)	0.88	0.86 on the auditor sample	close	within sampling variance
Threshold	0.80	0.80	yes
Remediation (tenure-weight cap)	applied	confirmed in the deployed config	yes

Acceptance criteria

✓The auditor runs the test on an independent sample, not the owner's dataset.
✓Claimed remediations are verified in the deployed system, not just in a report.
✓Discrepancies between reported and re-performed results are documented.

4. Human oversight interview guide

Document

Questions for reviewers that reveal whether oversight is real.

Template

Interview 3-5 reviewers separately. Open questions, then probes.

Walk me through the last case you reviewed. What did you see on screen?
How much time do you usually spend per case? What happens if you run out of time?
When did you last override the AI? What made you do it?
What would you need to override that you do not have now?
Are you measured on speed or volume? Does that affect how you review?
What are the AI's known weaknesses for this decision type?
What happens after you make a decision? Who checks it?

Worked example

Themes from 5 reviewer interviews (resume-screener):

3 of 5 could not name a recent override; "the scores are usually right".
Average self-reported time: 40-60 seconds; UI auto-advances at 60s.
All 5 are on a daily throughput target.
2 of 5 could describe a known model weakness; 3 could not.
No second check after a decision unless the candidate complains. Assessment: oversight is not currently meaningful. Corroborates the override-rate data.

Acceptance criteria

✓Reviewers are interviewed individually, not in a group with their manager.
✓Answers are corroborated against override-rate and timing data.
✓The guide probes for time pressure, throughput targets, and knowledge of model weaknesses.

5. AI audit findings template

Spreadsheet

Each finding: what, the evidence, severity, owner, and a remediation date.

Template

ID	Finding	Evidence	Obligation / control affected	Severity	Owner	Remediation	Due
A-1				high / medium / low			YYYY-MM-DD

Worked example

ID	Finding	Evidence	Affected	Severity	Owner	Remediation	Due
A-1	Human oversight is not meaningful: reviewers do not have time or authority in practice	5 interviews; override rate 4% vs 11% median; 60s auto-advance	EU AI Act Art. 14; HOC-002	High	Head of Talent	remove auto-advance; raise per-case time; drop throughput target for this queue	2026-11-15
A-2	Model card describes v2.3, model is v2.4	doc review	ALC-004	Medium	DS team	refresh on the versioning trigger	2026-10-15
A-3	Monitoring has no alerting; monthly only	monitoring config	MON-002/003	Medium	ML Platform	add thresholds + alerts	2026-11-01

Acceptance criteria

✓Each finding cites specific evidence and the exact obligation or control it breaches.
✓Severity, owner, and a due date are set for every finding.
✓Findings are tracked to closure and re-tested.

Governance controls this kit produces evidence for

Completing the artifacts above gives you a head start on the evidence requirements for these controls.

ALC-005

The whole kit is a regulatory audit-readiness exercise and its evidence for one system.

MON-003

The bias verification worksheet independently checks fairness monitoring.

HOC-003

The oversight interview guide tests whether the output review workflow is effective.

CMP-007

The document review checklist maps to EU AI Act conformity requirements.

ALC-002

Tracing sample decisions tests the high-risk audit trail.

This kit backs one playbook. Read the full guidance for the reasoning behind each artifact.

Decide what to implement next

Assess your governance gaps, then create an action plan with owners and target dates. Build and export without an account; sign in when you want to save your plan.

Start the AI governance assessment →