# AI Explainability Templates and Validation Checklist Setting and delivering the right level of explanation per system. A requirements matrix, a customer-facing adverse-action explanation template, a regulator explanation document, and a validation checklist that asks whether an explanation is actually faithful to the model. **Who this is for:** The owner deciding what explanation each AI system must produce, for whom, and how it is validated. Source playbook: https://aigovernance.com/playbook/explainability-standards-for-ai --- ## Explainability requirements matrix _System by risk tier, mapped to the explanation type and audiences it owes._ ### Template | System | Risk tier | Explanation approach | Audiences owed an explanation | Legal driver | |---|---|---|---|---| | | Minimal / Limited / High | none / global feature importance / per-decision factors / inherently interpretable | customer / regulator / internal reviewer | GDPR Art. 22 / FCRA / EU AI Act Art. 13 / none | ### Worked example | System | Risk tier | Explanation approach | Audiences | Legal driver | |---|---|---|---|---| | resume-screener | High | per-decision top factors + counterfactual; inherently interpretable scoring head | internal reviewer; candidate on request | EU AI Act Art. 13-14; local hiring transparency | | credit-decisioning | High | per-decision reason codes | customer (adverse action); regulator | FCRA / ECOA; GDPR Art. 22 | | support-copilot | Limited | none per-decision; system-level description | internal | none | ### Acceptance criteria - Every system has a stated explanation approach tied to its risk tier. - The audiences are named, and the legal driver for each is identified. - Where the law requires a specific format (FCRA reason codes, GDPR meaningful information), the matrix says so. --- ## Customer-facing explanation template _Plain-language, adverse-action-style explanation an individual can act on._ ### Template > Plain language. No model jargon. Actionable where possible. - **Decision:** what was decided - **Main reasons:** the top 3-4 factors, in everyday terms, most important first - **What would have helped:** the closest change that could have altered the outcome, if one exists - **Human involvement:** whether a person reviewed it and how to reach one - **How to contest:** the appeal or reconsideration route, with a timeframe - **Data used:** the categories of information considered, and how to request the specifics ### Worked example - **Decision:** Your application was not advanced to interview at this stage. - **Main reasons:** (1) The listed experience was below what the role requires. (2) Fewer of the role's core skills were evidenced in the application. (3) No portfolio or work samples were included. - **What would have helped:** Adding work samples and listing the core skills explicitly tends to change this outcome for similar applications. - **Human involvement:** A recruiter reviewed this decision. You can reply to this email to reach them. - **How to contest:** Request a reconsideration within 14 days by replying with any additional information. - **Data used:** Your application form and any attached documents. To request the specific factors, contact privacy@company.example. ### Acceptance criteria - The explanation uses everyday language and avoids feature names or scores. - It states whether a human was involved and how to reach one. - It gives a concrete route and timeframe to contest. - It is generated as part of the decision flow, not written by hand after a complaint. --- ## Regulator / auditor explanation document _The technical explanation a regulator expects: how the system reaches decisions, and the evidence the explanations are sound._ ### Template > One per high-risk system. Part of the documentation package. - **System and version:** - **Decision logic overview:** model type, inputs, how outputs map to decisions - **Explanation method:** inherently interpretable, or the post-hoc method (SHAP, LIME, reason codes) and its configuration - **Fidelity evidence:** how you validated that explanations reflect actual model behaviour, with results - **Worked examples:** 2-3 real decisions with their explanations - **Known limits of the explanation:** where it is approximate or can mislead - **Change control:** how explanations are kept in sync when the model changes ### Worked example - **System:** resume-screener v2.4.0 - **Decision logic:** VendorLM-4 features feed a linear scoring head; score bands map to advance / hold / review. - **Explanation method:** the scoring head is inherently interpretable (signed coefficients); per-decision output lists the top contributing features and a nearest counterfactual. - **Fidelity evidence:** because the head is linear, per-decision attributions are exact for the head; feature-embedding stability tested by perturbation (mean rank change 0.4 across 1,000 cases). - **Worked examples:** dec-...823 (score 2, factors: experience, skill match), dec-...901 (score 4), dec-...733 (override case). - **Known limits:** attributions describe the scoring head, not the upstream embedding; portfolio content is not an input, so the model cannot weigh it. - **Change control:** any change to features or the head triggers regeneration of the explanation spec and this document. ### Acceptance criteria - The document explains the actual decision logic, not a generic description of ML. - It includes evidence that explanations are faithful, not just that they are produced. - It states where the explanation is approximate or could mislead. --- ## Explanation validation checklist _Confirms an explanation method actually represents the model before you rely on it._ ### Template | Check | Result | Evidence | |---|---|---| | Removing the top-cited feature materially changes the output | Y / N | perturbation test | | Explanations are stable across near-identical inputs | Y / N | stability test | | Explanations agree with a second method (e.g. SHAP vs. permutation importance) at the global level | Y / N | comparison | | Counterfactuals offered are actually attainable and do flip the outcome when applied | Y / N | counterfactual check | | The explanation covers the whole pipeline, or its scope limit is disclosed | Y / N | scope note | | Non-experts can act on the customer-facing version (tested with real users) | Y / N | user test | ### Worked example | Check | Result | Evidence | |---|---|---| | Top-feature removal changes output | Y | mean score shift 0.7 band on removal | | Stable across near-identical inputs | Y | rank change < 1 for 96% of pairs | | Agrees with a second method globally | Y | top-5 features match permutation importance | | Counterfactuals attainable and effective | Partial | 82% flip when applied; 18% blocked by interaction effects, disclosed | | Whole-pipeline or scope disclosed | Y | scope limited to scoring head, stated | | Non-experts can act on it | Y | 9/10 test users identified a next step | ### Acceptance criteria - The method passed a perturbation or ablation test showing cited features actually drive the output. - Explanation stability was measured, not assumed. - The customer-facing version was tested with people who are not model experts. --- ## Governance controls this kit produces evidence for - **ALC-004**: The matrix, regulator document, and validation checklist are the explainability documentation for each system. - **HOC-001**: The required explanation level is derived from each system's risk classification. - **HOC-003**: Per-decision factor explanations are the information the output review workflow depends on. - **CMP-007**: The regulator explanation document supplies the transparency evidence an EU AI Act conformity assessment requires. - **ALC-002**: Worked-example explanations attached to decisions strengthen the high-risk audit trail.