AI Model Version Control
Added May 2026
Track model versions, configurations, prompts, and deployment history so that any past live setup can be recreated and compared.
Objective
Enable rollback (returning to an earlier version), reproducibility, and accountability for AI system behavior by keeping a complete version history of every file and setting that shapes the live system.
Maturity Levels
Initial
Model versions are not tracked; it is not possible to determine what model version produced a given output.
Developing
Model versions are informally noted but prompts, configurations, and deployment history are not systematically tracked.
Defined
A version control system tracks model versions, system prompts, and configuration settings with deployment timestamps.
Managed
Version history is linked to performance metrics; drops in performance can be traced to specific version changes.
Optimizing
Version control (recording every change) is automated and built into the release process; every file and setting in the live system can be recreated.
Evidence Requirements
What an auditor or assessor would expect to see for this control.
- —Model registry (the central record of models) showing every deployed version with model hash (a unique fingerprint of the model), training data reference, prompt version, and deployment timestamp
- —Source control records (from the system that logs every code change) linking model files to the code change, ticket, and approval event for each version
- —Reproducibility evidence: records confirming a past version can be rebuilt from logged files and settings
- —Change log linking each version to the change request, approval, and deployment event that created it
- —Access control settings on the model registry confirming only authorized roles can publish new versions
Implementation Notes
Key steps
- Track every change to system prompts (the standing instructions given to the model) as carefully as application code, because prompt changes can sharply alter model behavior.
- Label each release with the model version, prompt version, and a configuration hash (a unique code for the exact settings), so outputs can be traced to a specific system setup.
- Treat third-party model version changes (e.g. OpenAI model updates) as deployment events requiring testing and sign-off, not silent upgrades.
- Maintain a model card (a short fact sheet for the model) for each live model: intended use, known limitations, evaluation results, and approval status.
Example Implementation
ML platform team managing 6 production AI models across product and risk systems
Model Registry Entry: Customer Churn Predictor v2.3
| Field | Value |
|---|---|
| Model ID | churn-predictor-v2.3 |
| Base model | XGBoost 1.7.3 |
| System prompt version | N/A (non-generative) |
| Config hash | sha256:8f3a... |
| Training data version | customer-features-2026-Q1-v4 |
| Deployed to production | 2026-04-12T09:15Z |
| Deployed by | MLOps pipeline (approved by: J. Rivera, 2026-04-11) |
| Evaluation results | AUC 0.81 · Precision 0.76 · Recall 0.79 · Bias check: pass |
| Known limitations | Reduced accuracy for customers < 6 months tenure |
| Rollback target | churn-predictor-v2.2 |
| Status | Active |
Prompt tracking note: For generative models, prompt version is stored alongside model ID as a combined deployment artifact, a prompt change constitutes a new version.