Question 29 of 34
How do we build an AI governance program from scratch?
A sequenced guide to standing up an AI governance program — from initial inventory through ongoing operations — for organizations that are starting with nothing.
If you only do 3 things, do this:
- 1.Start with the inventory, not the policy. You can't govern what you don't know exists. Run discovery first, then build governance around what you find.
- 2.Don't try to build everything at once. Get your high-risk systems under control first. A focused program that actually works is more valuable than a comprehensive one that exists only on paper.
- 3.Get executive sign-off on the program structure before you build controls. Governance without organizational authority is a compliance exercise, not a governance program.
The Situation
Who this is for: The person who just got asked to "set up AI governance" at their organization — with no existing program to build on
When you need this: Before the first high-risk AI deployment, after an incident reveals governance gaps, or when a board or investor asks
The Decision
Where do we start, and what does a functioning AI governance program look like for our organization?
The Steps
- 1Run the AI inventory: vendor contracts, employee survey, network monitoring — in parallel
- 2Establish ownership: choose a governance model, document who is responsible for each key activity, get C-suite sign-off
- 3Build the risk classification framework: define your risk tiers and the scoring rubric
- 4Write the foundational policies: acceptable use, vendor due diligence, data handling for AI, incident response
- 5Apply baseline controls to your highest-risk systems first (documentation, oversight, logging)
- 6Build the ongoing processes: monitoring, regulatory tracking, incident response workflow, training
- 7Establish a governance reporting cadence to leadership: quarterly at minimum
The Artifacts
- —AI governance program charter template
- —AI governance maturity self-assessment (where are you today?)
- —Policy library checklist (which policies to write in what order)
- —90-day governance launch plan
- —Board reporting template for AI governance (quarterly dashboard)
The Output
A documented governance program with ownership assigned, foundational policies in place, baseline controls for high-risk systems, and board-level sign-off.
The governance stack, in order
AI governance programs fail when they start with the wrong layer. Writing policies before you know what you're governing produces policies that don't fit. Building technical controls before you have risk classification produces controls without prioritization. The correct sequence is: inventory → risk classification → ownership and governance structure → policies → controls → monitoring → training.
Each layer depends on the one below it. Your policies need to reference specific systems with specific risk tiers. Your controls need to be calibrated to those tiers. Your monitoring needs to be scoped to the systems that are actually running. Starting at the top of the stack and working down is the most common governance program failure mode.
What to build in the first 90 days
Days 1–30: Run the AI inventory using all three discovery methods. Don't stop at IT-approved systems. Simultaneously, establish who owns governance: get a sponsor at the C-suite level and document a basic RACI. You cannot proceed without both.
Days 31–60: Apply a risk tier to every inventoried system using a consistent scoring rubric. Identify your top three to five high-risk systems. Write the AI acceptable use policy and the vendor due diligence standard. These are your two most important foundational policies.
Days 61–90: Apply baseline controls to your high-risk systems: technical documentation, oversight arrangement, decision logging. Build the regulatory tracking process. Present a program summary to the board or relevant executive committee and get sign-off on the governance structure.
Common failure modes
The most common failure mode is building governance theater: documentation that satisfies a checklist but does not reflect actual practice. When governance documentation describes processes that do not exist, it creates liability rather than protection. Every control you document needs to be actually operating.
The second most common failure mode is scope creep before the basics are working. Organizations that try to implement ISO 42001 certification, a model registry, a bias monitoring dashboard, and an AI ethics board simultaneously while also discovering shadow AI typically achieve none of them well. Sequencing matters. Get the inventory and risk classification right. Everything else can follow.