Question 30 of 53
What AI documentation do we actually need?
By Cody Maxwell · AI Governance Institute · March 2026 · Last verified September 13, 2026
Identify legally required AI documentation by risk tier. Distinguish those duties from useful voluntary records and unnecessary paperwork.
▸Editorial status
- September 13, 2026 · Substantive update — The NIST ITL AI Program draft introduces specific requirements around public-facing AI documentation templates, external disclosures accessible to affected parties, and audit-ready records aligned with the AI RMF. The article covers documentation types and risk tiers but says nothing about publicly disclosed documentation, NIST-provided templates for external-facing disclosures, or the requirement to retain audit-ready records that can substantiate claims during third-party review. A practitioner reading this article to understand what AI documentation they need would miss this US federal guidance entirely. (AI Governance Institute pipeline)
If you only do 3 things, do this:
- 1.Documentation requirements scale directly with risk tier. High-risk systems need the full stack; minimal-risk systems need a registry entry. Applying high-risk documentation standards to every system is unnecessary overhead.
- 2.Build documentation into the AI lifecycle as it happens. Documentation assembled after the fact looks different to a regulator than documentation created in real time, and is often different.
- 3.Version-control everything. A model card that has not been updated since deployment is evidence that governance was not ongoing, not evidence that it was.
The Situation
Who this is for: Compliance, legal, and engineering teams building documentation practices for AI systems
When you need this: When designing a new AI system, preparing for a regulatory exam, or responding to a customer or investor questionnaire
The Decision
What documentation is legally required for each of our AI systems, and what can we skip without creating gaps?
The Steps
- 1Apply a risk tier to each AI system: documentation requirements scale with risk
- 2For each high-risk system: create a model card, data governance record, and bias assessment before deployment
- 3For each AI-assisted decision about individuals: implement decision logging (inputs, model version, output, reviewer, final decision)
- 4Implement version control for all AI documentation: every model change triggers a documentation review
- 5Document human oversight arrangements in writing for any high-risk system
- 6Store documentation in a retrievable location: organize by system, retrievable within hours, not days
The Artifacts
- —Documentation requirements matrix (by risk tier: required vs. recommended vs. optional)
- —Model card template (purpose, training data, performance, limitations, bias evaluation)
- —Data governance record template (source, legal basis, minimization measures)
- —Bias assessment template (methodology, findings, remediation, re-test schedule)
- —Decision log schema (required fields by decision type)
- —Human oversight arrangement template
The Output
Complete, current documentation for every AI system, organized by risk tier, with version history, retrievable within hours during an examination or inquiry.
What regulations actually require
The EU AI Act requires technical documentation covering system design, training data, testing results, performance metrics, and known limitations for all high-risk AI systems. This documentation must be maintained throughout the lifecycle and provided to national authorities on request. For providers placing systems on the EU market, this is a mandatory pre-deployment requirement, not a post-deployment best practice.
GDPR Article 22 requires that data subjects receive "meaningful information about the logic involved" in automated decisions that significantly affect them. The FCRA requires adverse action notices that identify the principal reasons for adverse credit decisions. These requirements mean that explanation capability must be built into the system and documented before deployment, not improvised when a data subject makes a request.
The core documents: what each one is
The model card is the primary reference document for an AI system. It describes the model's intended use, training data sources, performance characteristics across relevant demographic groups, known limitations, and recommendations for appropriate use. It should be created during development, not after deployment, and updated with every material change.
The data governance record documents the training and evaluation datasets: their sources, the legal basis for use, the PII assessment conducted, and the data minimization measures applied. For organizations subject to the EU AI Act, this document is specifically required for high-risk systems and must demonstrate that training data is relevant, representative, and free from obvious errors.
The decision log is the audit trail for individual AI-assisted decisions. It captures, at minimum, the inputs provided to the model, the model version and configuration, the output or recommendation, any confidence scores, the human reviewer involved, and the final decision taken. For credit, employment, and other regulated decision types, these records must be retained for specific periods.
Documentation by risk tier
High-risk systems require the full documentation stack: model card, data governance record, bias assessment, human oversight arrangement, decision log, post-deployment monitoring plan, and incident log. This documentation should be treated as a single package, organized and retrievable as a unit. Regulators conducting an examination will ask for it together.
Medium-risk systems require a risk assessment with rationale, a basic model card, and a record of the business owner's sign-off. They should appear in the model registry with current metadata. The bias assessment requirement depends on whether the system makes or influences decisions about individuals, and if it does, some form of fairness testing is appropriate regardless of the formal risk tier.
Low-risk systems require only a registry entry with basic metadata (system name, owner, purpose, risk tier, last reviewed). Extensive documentation for low-risk systems creates noise without signal and makes high-risk documentation harder to find and maintain.
NIST guidance on public-facing AI documentation
NIST's ITL AI Program has released an initial public draft providing templates and guidance for publicly disclosed AI system documentation. The templates cover system purpose, intended use, performance characteristics, and known limitations, and are designed to support developers, deployers, and procurers preparing governance records, transparency disclosures, or audit evidence. Organizations operating in the US should treat these templates as a practical starting point for structuring external-facing documentation, particularly where no other mandatory format is specified.
The guidance emphasizes two distinct documentation outputs: internal governance records sufficient to support deployment approvals, and external disclosures written in terms accessible to affected parties. Both must be backed by audit-ready records that can substantiate documentation claims under third-party or regulatory review. The draft aligns with the NIST AI RMF categories, so organizations already working to that framework can map their existing documentation practices against the new templates without starting from scratch. Public comments on the draft are open until September 16, 2026.
Governance Controls
Operational controls that implement the guidance in this playbook.
Related frameworks
Recent Coverage
News and developments relevant to this playbook topic.
Not sure where to start? Answer 3 questions and get a tailored compliance action plan.
What applies to me? →More guidance like this, every week
New playbook articles, governance controls, and the regulatory changes driving them. Every Thursday.