What AI agent security controls do we need as agent autonomy expands?

Agent security is about resisting an adversary: can an attacker manipulate this agent into misusing the credentials and permissions it already holds? Agent governance is about accountability: who decided this agent should have those permissions in the first place, and who signs off when that scope grows? Most organizations that deploy agentic AI address the first question at launch and never build a repeatable process for the second, which means every expansion of an agent's autonomy after initial deployment happens without the review it deserves.

That gap tends to widen quietly. An agent given narrow, well-scoped access for one task picks up incremental permissions over time as engineers extend it to adjacent workflows, and none of those individual extensions look risky enough on its own to trigger a formal review. The result, months later, is an agent operating with a permission set nobody explicitly approved and nobody can fully account for.

Once one agent can delegate work to another, supervise another, or act on another's output, the system needs a documented trust hierarchy. Without one, what happens when two agents receive conflicting instructions, or when a subordinate agent's output contradicts the system that spawned it, is decided by default software behavior rather than by anyone's policy decision. That is not a hypothetical edge case. Multi-agent architectures that chain delegation across several layers make it routine.

A usable trust hierarchy specifies, for every pair of agents that can interact, whose instructions take precedence, what a subordinate agent is permitted to do without checking back, and how a delegation chain gets logged so a reviewer can reconstruct which agent authorized which action after the fact. Treat this the same way you would treat a human org chart for an unusual approval: if you cannot answer who had authority in the moment, you do not have a governed system, you have a collection of agents that happen to work most of the time.

Every agent, and every helper agent or task it starts, needs its own identity rather than borrowing the login of whatever started it. Shared or inherited credentials make it impossible to attribute an action to the specific agent instance that took it, which is precisely the information you need when something goes wrong. Non-human identities should carry an expiration and a re-certification date, the same discipline applied to any human or automated system account, rather than being provisioned once and left alone indefinitely.

Permission creep is the quieter version of the same risk. An agent's permissions were reviewed and approved at deployment, but nothing monitors whether they have since grown. Routine settings changes, a broader connection to another app, or a well-meaning engineer solving an unrelated problem the fastest way can all expand them. Alert on drift from the originally approved scope, not only on breach of some hard outer limit. A limit set generously enough to never trigger is not a control.

A kill switch that halts the top-level agent while its delegated sub-agents, spawned background tasks, or scheduled follow-up actions keep running is not a kill switch. It is a control that looks complete on a diagram and fails the first time it is actually needed. Multi-agent systems and AI coding agents routinely start background tasks that keep running after the agent that started them has stopped, and an emergency stop that does not account for that gives a false sense of containment.

Test propagation the same way you would test a disaster recovery plan: on a schedule, against a realistic scenario, and with the result documented rather than assumed. Isolation helps too: running the agent's code and actions in a walled-off environment (a sandbox) with no path to live business systems reduces how much damage happens in the window between an incident starting and the kill switch actually reaching every process it needs to stop.

Most agentic deployments will never approach the risk ceiling this section describes, and a standard risk classification is enough for them. But agents given tools capable of hacking other systems, or help with biological or chemical design tasks, sit in a different category, and a general-purpose risk questionnaire will not surface the specific failure modes that matter at that end of the spectrum. These deployments warrant a dedicated security assessment before launch, not an accelerated version of the standard one.

Pair that assessment with a signed written statement of which governance controls are actually enforced in the live system: permission boundaries, audit logging, and the kill switch, named specifically, not implied by a policy document. The gap between what governance controls a program believes are in place and what is actually wired into a given deployment is where the most consequential agentic AI incidents tend to originate, and it is closeable with a document that takes an afternoon to produce and an hour to verify.