AI Capability Claim Substantiation Standard
Establish a documentation standard for AI capability claims made internally and externally — in marketing materials, product documentation, sales conversations, regulatory submissions, and procurement responses — that produces substantiation evidence meeting FTC disclosure expectations and enterprise customer due diligence requirements.
Objective
Prevent regulatory liability and commercial disputes arising from unsubstantiated or misleading AI capability claims, by establishing a review and documentation process that ensures claims are accurate, qualified appropriately, and supported by evidence before publication or distribution.
Maturity Levels
Initial
AI capability claims in marketing materials, product documentation, and sales conversations are made without a review process or substantiation requirement. Claims may reflect aspirational product states, vendor representations, or benchmark results that do not generalize to production deployments.
Developing
Legal or compliance teams may review significant external AI claims for obvious accuracy issues, but there is no systematic substantiation requirement. Claims based on benchmark results may not be qualified for deployment context. Sales team claims are not covered by any review process.
Defined
A claims substantiation policy defines: which claim types require substantiation, what evidence is required for each claim type, and who approves claims before publication or use. A claims registry tracks approved claims and their substantiation evidence. Sales and marketing teams are trained on the policy.
Managed
Claims are reviewed against the registry before use in new materials or contexts. Substantiation evidence is updated when model capabilities change or when the deployment context changes materially. The claims registry is reviewed for currency quarterly. FTC guidance and equivalent regulatory developments are monitored for changes affecting substantiation requirements.
Optimizing
The claims substantiation function is integrated into the product launch and marketing campaign process so that capability claims are substantiated before campaigns go live rather than retrospectively. Claims from AI vendors about the organization's AI products are reviewed for accuracy before adoption in organizational materials.
Evidence Requirements
What an auditor or assessor would expect to see for this control.
- —Claims substantiation policy defining claim types, substantiation requirements by type, and approval process.
- —Claims registry containing approved AI capability claims with substantiation evidence references, approval dates, and expiry dates.
- —Substantiation evidence packages for material AI capability claims made in the past 12 months (marketing materials, regulatory submissions, enterprise customer RFP responses).
- —Training records confirming sales, marketing, and product teams have completed AI claims substantiation training.
Implementation Notes
The FTC framework for AI capability claim substantiation
The Federal Trade Commission's approach to AI capability claims follows the same legal framework as its enforcement approach to any advertising or marketing claim: claims must be truthful, not misleading, and substantiated before they are made. The FTC's 2023-2026 guidance and enforcement actions have clarified how these general requirements apply to AI:
Substantiation standard: AI capability claims must be substantiated by competent and reliable evidence before the claim is made. For performance claims (accuracy rates, error rates, speed improvements), this typically means testing that reflects the deployment context — not cherry-picked benchmark results that do not generalize.
Qualification requirements: Claims about AI accuracy or performance that are true only under specific conditions must be qualified clearly enough that the audience understands the conditions. "95% accurate on benchmark X" without disclosure that production performance differs materially can be misleading even if the benchmark claim is technically accurate.
Comparative claims: Claims that compare an AI system to human performance, competing products, or prior-generation systems require substantiation against an appropriate baseline. "Better than human reviewers" requires evidence from testing in the relevant task domain, not general benchmark comparisons.
Outcome claims: Claims about what the AI system will do for customers ("this AI will reduce your compliance burden by 40%") require substantiation for the claimed outcome, not just for the underlying capability.
AI-about-AI claims: Claims that an AI system was developed using responsible AI principles, passed specific safety evaluations, or complies with specific frameworks require documentation that can be verified. Claiming compliance with NIST AI RMF without documentation of how the framework was applied is an unsubstantiated compliance claim.
Types of claims requiring substantiation
Performance claims: Accuracy rates, precision/recall, error rates, processing speed, throughput. Substantiation: testing results from evaluations conducted under deployment-representative conditions, including the relevant data distribution and task specification.
Safety and alignment claims: Red-team results, refusal rates, safety filter effectiveness. Substantiation: evaluation results using published methodologies (see MGV-006) or documented internal methodology.
Compliance claims: Assertions that the AI system complies with NIST AI RMF, ISO 42001, EU AI Act, or other frameworks. Substantiation: documentation showing how the framework requirements are addressed, not just a statement of intent.
Outcome claims: Business outcomes attributed to the AI system. Substantiation: customer data or controlled study results; appropriate statistical qualification.
Comparative claims: Comparisons to human performance, competing products, or prior versions. Substantiation: testing on an appropriate comparative basis.
Building the claims registry
A claims registry is a maintained list of approved AI capability claims with:
- The exact claim text (approved verbatim version)
- Claim type and substantiation evidence reference
- Approved channels and contexts (marketing, sales, regulatory, all)
- Approval date and approver
- Expiry or review date (claims should be re-substantiated when the underlying model or deployment changes materially)
- Status (active, under review, retired)
The registry allows the organization to maintain a library of pre-approved claims that sales and marketing teams can use without individual review, while ensuring that claims outside the registry go through the substantiation process before use.
Example Implementation
AI Capability Claims Registry (excerpt)
Registry maintained by: Legal / AI Governance | Last reviewed: 2026-06-01 | Next review: 2026-09-01
| Claim ID | Claim text | Claim type | Approved channels | Substantiation | Approval date | Expiry |
|---|---|---|---|---|---|---|
| CLM-001 | "Our AI contract review system achieves 92% accuracy on standard commercial contract clause identification in our test environment." | Performance | Sales, marketing, RFP | Internal eval report AI-EVAL-2026-04 (benchmark: internal contract corpus, 500 contracts, Q1 2026). Qualified with "test environment." | 2026-02-15 | 2026-08-15 (review when model updates) |
| CLM-002 | "The system passed our red-team evaluation using HarmBench-aligned methodology with a 91.7% refusal rate on direct harmful request prompts." | Safety | Sales, RFP | Red-team report SAF-2026-07; methodology aligned to HarmBench direct attack category | 2026-03-10 | 2026-09-10 |
| CLM-003 | "Aligned with NIST AI Risk Management Framework." | Compliance | Marketing, RFP | NIST AI RMF alignment documentation: [internal document ref]. Governs: how the framework is implemented, not a third-party certification. | 2026-01-20 | 2026-07-20 |
| CLM-004 | "Reduces contract review time by 40% compared to manual review." | Outcome | Sales | Customer pilot data: 3 clients, 60-day pilot, average 38-43% time reduction on comparable contract types. | 2026-04-05 | 2026-10-05 |
Retired claims:
- CLM-002a: "91.7% refusal rate on harmful prompts" — Retired 2026-03-10; replaced by CLM-002 with methodology qualification added.
- CLM-005: "Complies with EU AI Act" — Retired 2026-05-01; EU AI Act high-risk obligations not yet fully implemented; replaced with specific obligation tracking.