AI Decision Logging
Record AI system inputs, outputs, model version, confidence scores, and contextual metadata for every decision that affects individuals or business outcomes.
Objective
Create an auditable record of AI-driven decisions to support regulatory compliance, incident investigation, and accountability.
Maturity Levels
Initial
AI decisions are not logged; only application-level errors are captured.
Developing
Some AI outputs are logged but coverage is incomplete and log format is inconsistent.
Defined
A defined logging schema captures inputs, outputs, model version, and timestamp for all in-scope AI decisions.
Managed
Logs are centralized, searchable, and reviewed regularly; completeness is verified through sampling.
Optimizing
Decision logs feed automated monitoring, drift detection, and model performance dashboards.
Evidence Requirements
What an auditor or assessor would expect to see for this control.
- —Log schema documentation defining required fields, storage type, access controls, and retention period
- —Sample log entries confirming all required fields are populated for a representative set of decisions
- —Storage integrity configuration confirming append-only or immutable log storage with appropriate access controls
- —Log completeness sampling report showing what percentage of in-scope decisions are being captured
- —Retention compliance evidence confirming logs are maintained for the required period
Implementation Notes
Key steps
- Define your minimum log record: input (or a hash if PII must be excluded), output, model version/ID, timestamp, user/session identifier, and confidence score where available.
- Log at the AI decision point, not at the application layer — application logs often omit the information needed for AI accountability.
- For high-risk systems, log the full input context including retrieved documents (in RAG systems) or tool call results (in agent systems).
- Ensure logs are immutable and access-controlled — the ability to delete or modify decision logs is a material compliance risk.
Example Implementation
Insurance company using AI to assist underwriters with risk scoring
AI Decision Log Schema — Underwriting Risk Scorer
Minimum log record (one entry per scoring request):
| Field | Example Value | Notes |
|---|---|---|
| log_id | ulid-01HXYZ... | Unique, append-only identifier |
| session_id | sess-abc123 | Links to underwriter session |
| model_id | risk-scorer-v2.4.1 | Full version including prompt hash |
| input_hash | sha256:3a9f... | SHA-256 of full input; raw input stored separately in PHI store |
| risk_score | 0.73 | Model output |
| confidence | 0.88 | Model confidence |
| top_features | ["prior_claims", "industry_code"] | Top 3 contributing features |
| timestamp | 2026-05-09T14:22:01Z | UTC |
| underwriter_id | uw-00412 | Reviewer identity |
Storage: Append-only; no delete or update; access restricted to Compliance, AI Governance, and Legal
Retention: 7 years from decision date (state insurance regulation requirement)
Control Details
- Control ID
- ALC-001
- Domain
- Audit & Logging
- Typical owner
- AI Engineering / Compliance
- Implementation effort
- Medium effort
- Agent-relevant
- Yes