Self-Hosted Open-Weight AI Model Governance

Why open-weight models need their own governance controls

Organizations that deploy open-weight AI models (models whose trained files, or weights, can be downloaded from Hugging Face, model provider repositories, or community sources) face a governance challenge distinct from vendor-hosted AI procurement. The vendor-hosted AI procurement controls (PRC domain) are designed for a procurement relationship: there is a vendor, a contract, a service level, and ongoing communication about changes.

Open-weight models have no vendor relationship. There is no contract, no service level agreement (SLA), no incident notification, and often no clear commercial entity responsible for the model. The governance risks are:

No ongoing safety assurance: A vendor-hosted AI model is supported by the vendor's ongoing safety team, red-team cycles (structured attempts to make the model misbehave), and regular updates. An open-weight model deployed in organizational infrastructure receives the organization's own infrastructure security, but the model weights themselves reflect the safety evaluation and safety training (known as RLHF) applied at the time of release. If new attack methods emerge that exploit the model, there is no vendor to patch it; the organization must update to a new model version or apply its own mitigations.

Supply chain integrity risk: Open-weight model weights downloaded from public repositories may have been tampered with. Malicious actors have demonstrated the ability to upload modified model weights to public repositories that include backdoors, trojans (hidden functions attackers can trigger later), or unsafe capabilities. Without verifying that the model files are unaltered, an organization cannot confirm that the model it deployed matches the model it believed it was deploying.

License complexity: Open-weight models are released under a variety of licenses, Apache 2.0, MIT, Llama Community License, Gemma Terms of Use, various research-only licenses, with materially different restrictions on commercial use, modification, and redistribution. Deploying a model under a license that does not permit the intended use creates legal risk. License terms for popular models have been updated over time; the organization must track which version it is using and its license terms.

Safety evaluation responsibility: For vendor-hosted models, the vendor has conducted safety evaluation and maintains safety filters. For self-hosted open-weight models, the organization is responsible for safety evaluation in its deployment environment. A model that behaved safely by default may behave differently with the organization's system prompts (standing instructions), fine-tuning (further training), or retrieval augmentation (links to internal documents).

Intake process for open-weight models

Step 1: Source verification

• Download only from the model provider's official repository or verified mirror (confirmed by matching its SHA256 hash, a file fingerprint, to the provider's published hash)
• Do not deploy model weights from unofficial community uploads without independent hash verification
• Document the download source, version, and hash verification result

Step 2: License review

• Identify the license governing the model weights
• Determine whether the intended use is permitted under the license (commercial use, modification, redistribution as applicable)
• Document the license classification: permissive commercial, restricted commercial, research-only, or custom
• Flag for legal review any model with non-standard license terms or use-case restrictions

Step 3: Model card review

• Review the model card (the developer's published summary of the model) for: training data, known limitations, intended use, out-of-scope uses, and safety evaluation results
• Document any safety concerns identified in the model card
• Check community safety research (published papers, red-team findings) for known vulnerabilities or unsafe behaviors

Step 4: Safety evaluation

• Conduct internal safety evaluation appropriate to the intended use case and risk tier (per HOC-001)
• For high-risk use cases: red-team evaluation (SAF-005) using scenarios relevant to the deployment context
• Document evaluation results and any mitigations required before deployment

Step 5: Intake record and approval

• Create intake record in AI inventory per MGV-002
• Document: model name, version, source, hash, license, model card summary, safety evaluation results, intended use, and approval
• Obtain required approvals per the intake policy (risk tier determines approval authority)

Update management for deployed open-weight models

Unlike vendor-hosted models that update automatically or through contract-managed processes, open-weight model updates require a deliberate decision. When a new model version is released:

1. Review the changelog (list of changes) and any new safety evaluations for the new version
2. Determine whether to update (new version has meaningfully improved safety or capability) or stay (current version adequate; update cost not justified)
3. If updating: repeat the intake process for the new version before replacing the deployed version
4. Document the update decision and rationale