Reviewer Competency Requirements
Added May 2026
Define minimum competency requirements for humans who review, approve, or override AI-generated outputs in high-risk contexts.
Objective
Ensure that human oversight is substantive rather than procedural by requiring reviewers to have the domain knowledge necessary to critically evaluate AI outputs.
Maturity Levels
Initial
No competency requirements exist; any employee can be assigned to review AI outputs.
Developing
Competency expectations are informally understood but not documented or enforced.
Defined
Written competency requirements are defined per use case, with a documented training curriculum and sign-off process.
Managed
Competency is assessed before assignment and re-evaluated periodically; records are maintained.
Optimizing
Competency framework is updated as AI capabilities evolve; training effectiveness is measured through calibration exercises.
Evidence Requirements
What an auditor or assessor would expect to see for this control.
- —Competency matrix documenting domain and AI literacy requirements per use case, version-controlled (changes tracked over time) and approved
- —Training completion certificates for all assigned reviewers, retained in HR system with completion date
- —Reviewer role assignment records confirming certification prerequisite was verified before assignment
- —Recertification status report showing current, lapsed, and upcoming-due certifications by reviewer
- —Calibration exercise (test cases with known correct answers) or assessment results used to verify competency at initial certification and recertification
Implementation Notes
Key steps
- Define competency requirements per use case, not generically, a loan underwriter reviewing AI credit decisions needs different skills than a clinician reviewing AI diagnostic suggestions.
- Distinguish between domain competency (understanding the subject matter) and AI literacy (understanding model limitations, hallucination risk (confident but false answers), and confidence signals (cues showing how sure the AI is)).
- Document training completion and maintain records, this is a key audit evidence requirement under EU AI Act Art. 26.
- For agentic systems (AI that acts on its own), include training on reading agent action logs (records of each step it took), not just final outputs.
Example Implementation
Financial services firm with AI-assisted fraud detection and credit scoring
Reviewer Competency Matrix
| Use Case | System | Domain Requirement | AI Literacy Requirement | Recertification |
|---|---|---|---|---|
| Fraud Detection | TxnGuard v3 | AML analyst Level II+ | Module 1 + Module 2 | Annual |
| Credit Scoring | LoanScoreAI | Licensed underwriter | Module 1 + Module 3 | Annual + supervisor sign-off |
| Document Extraction | DocParse | Operations analyst | Module 1 | Annual |
AI Literacy Modules:
- Understanding model outputs, confidence scores, and failure modes (required for all reviewers)
- Recognizing distribution shift and out-of-distribution inputs
- Interpreting attribution/explanation outputs for model-specific systems
Records: Training completion certificates retained in HR system; reviewer assignments blocked until current certification confirmed