How to Build AI Agent Governance That Works at Runtime

Key Takeaways

• AI agent governance works when every agent has an owner, declared purpose, scoped credentials, runtime policy, and retirement path.
• Non-human identities are now the fastest-growing and least-governed class of enterprise identity, and Gartner projects that by 2028, 25% of enterprise breaches will trace back to AI agent abuse, making unmanaged agents an identity problem before a model problem.
• Static approvals miss decisions because risk appears when agents call tools, exchange data, or delegate work across systems.
• Discovery, access mapping, and runtime enforcement all have to resolve to the same acting agent, which is what makes identity the control layer.

How do you govern AI agents?

Govern AI agents by managing each agent as a non-human identity with a named owner, declared purpose, scoped credentials, and runtime decision controls. In AWS IAM, Azure Entra ID, Google Cloud IAM, GitHub Actions, Kubernetes, OAuth apps, and MCP tools, governance works when every action is checked against identity, intent, data sensitivity, and lifecycle status before execution.

• Name the agent, owner, data class, and business purpose before access is issued.
• Tie credentials and permissions to the declared task instead of a broad shared role.
• Evaluate tool calls against current context, not only against provisioning-time approval.
• Expire or revoke access when the task, ticket, owner, or lifecycle stage changes.

Quick Facts

What should teams document before an agent acts?

The useful governance record is not a form in a ticketing queue. It is the policy input that determines whether a request should run. Before an agent receives access, the record should state the owner, user population, model or runtime, connected tools, approved data classes, maximum privilege, expected behaviors, and offboarding trigger.

• Owner and escalation path, so orphaned agents cannot keep acting.
• Tool and connector list, so new SaaS or API paths are reviewed before use.
• Data classes and allowed outputs, so summaries, exports, and writes have boundaries.
• Credential type and expiry, so stale tokens do not become standing access.

That record should be tested against activity, not archived. If the agent begins calling a new API, touches a higher data class, or runs from an unexpected environment, AI agent governance needs a decision: deny, reduce scope, request human approval, or update the record with evidence.

The review cadence should also change. Quarterly attestation can confirm owners and inventory, but it cannot answer whether a running agent should execute a request now. Teams should keep the slow controls for accountability and move the access decision closer to the action. That means checking current owner status, active ticket, environment, credential age, data class, and behavior pattern inside the runtime path. If any of those signals are missing, governance should require a narrower scope or human review rather than assuming the old approval still applies. This keeps governance practical for teams that ship agents frequently because the control follows evidence instead of blocking every new use case at intake.

Why does AI agent governance fail when it stops at approval?

Approval records prove that an agent was allowed to exist. They do not prove that its current action still matches the task, data class, user request, or business owner who approved it. A support triage agent reads Salesforce cases, summarizes Slack threads, opens Jira tickets, and invokes a GitHub Actions workflow to collect logs. Four systems are touched in that sequence, each with its own idea of who is acting, and the approval that authorized any of it was granted weeks earlier. The NIST AI Risk Management Framework organizes AI risk into Govern, Map, Measure, and Manage. Those functions stay theoretical until the agent identity, its credential, its owner, and the context of the action are visible at the same moment.

Recent incidents make the point better than any framework does. The Salesloft Drift compromise showed why OAuth-connected applications belong inside governance rather than beside it: Google reported stolen OAuth tokens tied to the Drift Email integration in August 2025, and those tokens worked exactly as issued. MITRE ATLAS documents adversary techniques against AI-enabled systems. The OWASP LLM Top 10 addresses model-layer failures such as excessive agency. The OWASP NHI Top 10 addresses the machine layer, listing improper offboarding, secret leakage, and vulnerable third-party non-human identities.

Where does risk move in AI agent governance?

In practice, risk moves from the registry to the handoff: the Slack bot calls a SaaS connector, the connector calls an internal API, and the internal API acts through a service account. Traditional IAM reports the role. A SIEM reports the events. A SaaS admin console reports the installed app. Three accurate views, none of which says whether this action still serves the purpose the agent was approved for. That gap is where governance turns from a document into an operational problem.

Three things have to be true before governance can enforce anything. Every AI agent, service account, OAuth app, access token, and workload identity has to be known. Each one has to have a reach that somebody has measured, alongside a blast radius if the credential is misused. And the decision has to sit in front of execution rather than behind it, where a request can be denied, approved, reduced in scope, or escalated to a person.

Governance Controls by Execution Stage

What controls make AI agent governance enforceable?

Governance starts with the agent record rather than the model behind it. That record names the owner, purpose, identity provider entry, tool list, data classes, credential type, maximum privilege, and retirement date. On its own it is documentation. Wired into the runtime path it becomes policy, because those same fields can be evaluated as conditions: an unlisted tool, an unapproved data class, or a role assumed outside the declared boundary each becomes a reason to fail the request or send it for review.

When policy denies a request, it should say which condition it denied on: a missing owner, scope that exceeds the task, a data class outside the boundary, an expired token, behavior off the baseline, or an approval path with nobody at the end. A denial without a reason gets solved by widening the role, which is the outcome governance exists to prevent. A denial with a reason gets solved by fixing one field. That holds across AWS IAM, Azure Entra ID, Google Cloud IAM, Kubernetes service accounts, and SaaS OAuth grants.

Common Pitfalls to Avoid

• Governing the agent as a fixed application when its permissions and behavior change between runs.
• Making a broad role permanent because one workflow failed once under a narrow one.
• Signing off a tool at intake and never revisiting the connectors, scopes, or data classes added afterward.
• Running AI governance as its own program beside the NHI program, which produces two inventories and the same gaps in both.

How does Token Security approach AI agent governance?

The challenge

The surface is the problem before the policy is. An agent created in a developer tool, a custom GPT enabled by a business team, and a SaaS add-on installed in three clicks all end up holding access without passing the same gate on the way in. IAM and PAM assume a person who joins, moves, and leaves on a reviewable schedule, while machine actors call tools continuously and change what they can reach between one review and the next.

The approach

Token Security treats the agent as a non-human identity first and a model second, attaching owner, credentials, permissions, and observed behavior to that one entry. Entitlement mapping, blast-radius analysis, behavioral baselines, discovery, automated remediation, and lifecycle governance then run against it rather than against six separate records. Discover, understand, enforce describes the order that follows: establish the agent and its credentials, map reach and behavior, then narrow permissions and evaluate policy at the moment of action.

The outcome

Governance closes into a single loop across AI agents, service accounts, machine identities, OAuth apps, and every other machine actor. Token Security holds non-human identity, service account exposure, OWASP NHI Top 10 risk, blast radius, and machine-first enforcement together, which keeps AI agents inside the program that already governs the rest of the estate.

How security leaders are applying this model

Governance breaks down when the only agents on the register are the approved ones. The working version finds the agent, connects it to the access it holds, and gives the owner a fix scoped to the problem.

"Token Security gives us visibility we simply didn't have before. We can now automatically identify and control custom GPT agents running in our environment and ensure the required security level. Knowing that no AI agent is operating beyond our oversight means we can confidently accelerate our AI adoption."
Tamir Ronen, Global CISO at HiBob

That is what separates a dashboard from a control: it has to resolve to one machine identity, one entitlement, and one remediation path.

The same logic drives accountability. Once an agent or machine identity holds access, what matters is who owns it, what it can reach, and how quickly the excess can be withdrawn.

"Token Security addresses a critical issue faced by companies - the growing threat introduced by non-human identities. Our experience shows that teams need non-human identity accountability, automated remediation, and actionable insights to manage these risks."
Brian Kerr, VP Security and Trust at Klaviyo

Monitoring describes. Governance decides. It holds because accountability follows the non-human identity that acts.

Why AI agent security starts with identity

The direction of this work is better proof: which machine is acting, why, how far it can reach, and whether the action should continue. Agents collapse the distance between a request and its impact. Non-human identity governance is what puts a control point inside that gap. When identity becomes the control plane, a team can run agentic AI without also accepting access it cannot see, credentials that never expire, or autonomy with no boundary.