What AI agent security controls do we need as agent autonomy expands?

Agent security is about resisting an adversary: can an attacker manipulate this agent into misusing the credentials and permissions it already holds? Agent governance is about accountability: who decided this agent should have those permissions in the first place, and who signs off when that scope grows? Most organizations that deploy agentic AI address the first question at launch and never build a repeatable process for the second, which means every expansion of an agent's autonomy after initial deployment happens without the review it deserves.

That gap tends to widen quietly. An agent given narrow, well-scoped access for one task picks up incremental permissions over time as engineers extend it to adjacent workflows, and none of those individual extensions look risky enough on its own to trigger a formal review. The result, months later, is an agent operating with a permission set nobody explicitly approved and nobody can fully account for.

Once one agent can delegate work to another, supervise another, or act on another's output, the system needs a documented trust hierarchy. Without one, what happens when two agents receive conflicting instructions, or when a subordinate agent's output contradicts the system that spawned it, is decided by default software behavior rather than by anyone's policy decision. That is not a hypothetical edge case. Multi-agent architectures that chain delegation across several layers make it routine.

A usable trust hierarchy specifies, for every pair of agents that can interact, whose instructions take precedence, what a subordinate agent is permitted to do without checking back, and how a delegation chain gets logged so a reviewer can reconstruct which agent authorized which action after the fact. Treat this the same way you would treat a human org chart for an unusual approval: if you cannot answer who had authority in the moment, you do not have a governed system, you have a collection of agents that happen to work most of the time.

Every agent, and every sub-agent or task it spawns, needs its own identity rather than inheriting credentials from whatever process launched it. Shared or inherited credentials make it impossible to attribute an action to the specific agent instance that took it, which is precisely the information you need when something goes wrong. Non-human identities should carry an expiration and a re-certification date, the same discipline applied to any human or service account, rather than being provisioned once and left alone indefinitely.

Credential scope drift is the quieter version of the same risk. An agent's permissions were reviewed and approved at deployment, but nothing monitors whether those permissions have since expanded through routine configuration changes, a broadened OAuth scope, or a well-meaning engineer solving an unrelated problem the fastest way available. Alert on drift from the originally approved scope, not only on breach of some hard outer limit. A limit set generously enough to never trigger is not a control.

A kill switch that halts the top-level agent while its delegated sub-agents, spawned background tasks, or scheduled follow-up actions keep running is not a kill switch. It is a control that looks complete on a diagram and fails the first time it is actually needed. Multi-agent and agentic coding systems routinely spawn processes that outlive the parent agent's own execution context, and an emergency stop that does not account for that gives a false sense of containment.

Test propagation the same way you would test a disaster recovery plan: on a schedule, against a realistic scenario, and with the result documented rather than assumed. Environment isolation, running agent-executed code and tool calls inside a sandbox with no path to production systems, reduces how much damage happens in the window between an incident starting and the kill switch actually reaching every process it needs to stop.

Most agentic deployments will never approach the risk ceiling this section describes, and a standard risk classification is enough for them. But agents given cyber-offense-capable tooling, or assistance with biological or chemical design tasks, sit in a different category, and a general-purpose risk questionnaire will not surface the specific failure modes that matter at that end of the spectrum. These deployments warrant a dedicated security assessment before launch, not an accelerated version of the standard one.

Pair that assessment with a written attestation of which governance tooling is actually enforced in production: permission boundaries, audit logging, and the kill switch, named specifically, not implied by a policy document. The gap between what governance controls a program believes are in place and what is actually wired into a given deployment is where the most consequential agentic AI incidents tend to originate, and it is closeable with a document that takes an afternoon to produce and an hour to verify.