User-Built Agent Go-Live Review
Added September 2026
Require a proportionate review before any agent built by an employee on a self-service platform goes live. Match the depth of review to what the agent can reach and do.
Objective
Stop self-service agent platforms from putting thousands of unreviewed agents into production, while keeping the review light enough that low-risk agents are approved quickly.
Maturity Levels
Initial
Employees can build and share agents on self-service platforms with no review. Nobody knows how many agents exist.
Developing
The platform keeps a list of user-built agents, but review happens only if the builder asks for it. Agents with access to sensitive data go live unreviewed.
Defined
Every user-built agent is classified into a risk tier when it is created. Low tiers get an automated check, higher tiers need a named reviewer's approval before the agent can be shared or connected to sensitive data.
Managed
Approval rates, review times, and agents blocked are tracked. Agents are re-reviewed when their connections or audience change, and inactive agents are retired on a schedule.
Optimizing
Tier rules are tuned using incident and violation data. Common agent patterns get pre-approved templates, so safe agents skip manual review entirely.
Evidence Requirements
What an auditor or assessor would expect to see for this control.
- —Documented tier criteria for user-built agents and the automated checks applied at each tier
- —Platform records showing each user-built agent's tier, review outcome, and approval date
- —Reviewer approvals for all Tier 3 and Tier 4 agents
- —Metrics on review turnaround times and agents blocked or retired
- —Re-review records for agents whose connections or audience changed
Implementation Notes
The scale problem
The US Department of Defense's GenAI.mil platform reached 50,000 user-created agents within weeks of launch. The full deployment readiness assessment (AGT-016) is built for agents that a central team designs and ships. It cannot be applied by hand to tens of thousands of agents built by staff. This control adds a tiered gate that scales.
Risk tiers
Classify each agent automatically when it is created, based on what it connects to:
- Tier 1, personal: used only by its builder, no connected data beyond the builder's own files, no actions outside the chat. It gets automated checks, with no manual review.
- Tier 2, shared: shared with a team, or reads internal documents. Automated checks plus a lightweight review by the platform team.
- Tier 3, connected: can take actions in other systems (send email, update records, call APIs) or touch sensitive data. Needs a named reviewer's approval and a register entry (AGT-027).
- Tier 4, external: interacts with customers, the public, or outside systems. Full readiness assessment under AGT-016.
What the automated checks cover
- The agent's instructions do not tell it to bypass controls or act without confirmation.
- Connected data sources match the builder's own access rights.
- The agent has a stated purpose and an owner.
- A test run against a small set of misuse prompts passes, for example requests to exfiltrate data or ignore instructions.
Keeping it light
Publish target review times, such as same day for Tier 2 and three business days for Tier 3. A slow gate pushes builders to unsanctioned tools, which is worse than the risk the gate removes. Measure violation rates before and after go-live; Microsoft's run-assert-eval work shows these rates can be measured and set as thresholds.
Example Implementation
Professional services firm rolling out a no-code agent builder to 8,000 staff
User-Built Agent Tiering: first 30 days
| Tier | Agents created | Auto-approved | Manual review | Blocked | Median review time |
|---|---|---|---|---|---|
| 1 Personal | 2,410 | 2,388 | 0 | 22 | Instant |
| 2 Shared | 612 | 540 | 72 | 9 | 4 hours |
| 3 Connected | 88 | 0 | 88 | 14 | 2 business days |
| 4 External | 6 | 0 | 6 | 2 | Routed to AGT-016 |
Most common block reason: Tier 3 agents connected to client file shares the builder could not personally access. The connector now checks the builder's own permissions before an agent is saved.