Question 30 of 34
What AI documentation do we actually need?
A practical guide to which AI documentation is legally required, which is best practice, and which is unnecessary overhead — organized by risk tier.
If you only do 3 things, do this:
- 1.Documentation requirements scale directly with risk tier. High-risk systems need the full stack; minimal-risk systems need a registry entry. Applying high-risk documentation standards to every system is unnecessary overhead.
- 2.Build documentation into the AI lifecycle as it happens. Documentation assembled after the fact looks different to a regulator than documentation created in real time — and often is different.
- 3.Version-control everything. A model card that has not been updated since deployment is evidence that governance was not ongoing, not evidence that it was.
The Situation
Who this is for: Compliance, legal, and engineering teams building documentation practices for AI systems
When you need this: When designing a new AI system, preparing for a regulatory exam, or responding to a customer or investor questionnaire
The Decision
What documentation is legally required for each of our AI systems, and what can we skip without creating gaps?
The Steps
- 1Apply a risk tier to each AI system — documentation requirements scale with risk
- 2For each high-risk system: create a model card, data governance record, and bias assessment before deployment
- 3For each AI-assisted decision about individuals: implement decision logging (inputs, model version, output, reviewer, final decision)
- 4Implement version control for all AI documentation: every model change triggers a documentation review
- 5Document human oversight arrangements in writing for any high-risk system
- 6Store documentation in a retrievable location: organize by system, retrievable within hours, not days
The Artifacts
- —Documentation requirements matrix (by risk tier — required vs. recommended vs. optional)
- —Model card template (purpose, training data, performance, limitations, bias evaluation)
- —Data governance record template (source, legal basis, minimization measures)
- —Bias assessment template (methodology, findings, remediation, re-test schedule)
- —Decision log schema (required fields by decision type)
- —Human oversight arrangement template
The Output
Complete, current documentation for every AI system, organized by risk tier, with version history, retrievable within hours during an examination or inquiry.
What regulations actually require
The EU AI Act requires technical documentation covering system design, training data, testing results, performance metrics, and known limitations for all high-risk AI systems. This documentation must be maintained throughout the lifecycle and provided to national authorities on request. For providers placing systems on the EU market, this is a mandatory pre-deployment requirement, not a post-deployment best practice.
GDPR Article 22 requires that data subjects receive "meaningful information about the logic involved" in automated decisions that significantly affect them. The FCRA requires adverse action notices that identify the principal reasons for adverse credit decisions. These requirements mean that explanation capability must be built into the system and documented before deployment, not improvised when a data subject makes a request.
The core documents: what each one is
The model card is the primary reference document for an AI system. It describes the model's intended use, training data sources, performance characteristics across relevant demographic groups, known limitations, and recommendations for appropriate use. It should be created during development, not after deployment, and updated with every material change.
The data governance record documents the training and evaluation datasets: their sources, the legal basis for use, the PII assessment conducted, and the data minimization measures applied. For organizations subject to the EU AI Act, this document is specifically required for high-risk systems and must demonstrate that training data is relevant, representative, and free from obvious errors.
The decision log is the audit trail for individual AI-assisted decisions. It captures, at minimum, the inputs provided to the model, the model version and configuration, the output or recommendation, any confidence scores, the human reviewer involved, and the final decision taken. For credit, employment, and other regulated decision types, these records must be retained for specific periods.
Documentation by risk tier
High-risk systems require the full documentation stack: model card, data governance record, bias assessment, human oversight arrangement, decision log, post-deployment monitoring plan, and incident log. This documentation should be treated as a single package, organized and retrievable as a unit. Regulators conducting an examination will ask for it together.
Medium-risk systems require a risk assessment with rationale, a basic model card, and a record of the business owner's sign-off. They should appear in the model registry with current metadata. The bias assessment requirement depends on whether the system makes or influences decisions about individuals — if it does, some form of fairness testing is appropriate regardless of the formal risk tier.
Low-risk systems require only a registry entry with basic metadata (system name, owner, purpose, risk tier, last reviewed). Extensive documentation for low-risk systems creates noise without signal and makes high-risk documentation harder to find and maintain.