Back
Article

What are the approaches to AI agent security?

Poonam Dasani
Sr. Director, Identity and Access Management, Amex GBT

Too often, AI agents look like an unstoppable force that's minutes away from wreaking havoc across the organization. But take a step back, and the shape of the problem becomes more defined. There are a number of possible solutions rooted in established security practices but adapted to match the speed, scale, and unpredictability of AI agents.

When securing AI agents, there are several layers where solutions lie:

  1. Prompt and Model Defense
  2. Context and Memory Integrity
  3. The Tool and MCP Surface
  4. Identity and Privilege
  5. Runtime Enforcement and Containment
  6. What You Log and What You Detect
  7. Governance and Lifecycle

An agent has a defined execution path, and there are places along it where your controls will live. Some will predict; others will enforce. If you've been reading up on agent security, you probably noticed how much content is dedicated to prompt injection and filtering. It’s tempting to think that this is the one place where your enforcement should be the strongest. But if you look at real-world incidents, it becomes clear that prompt filtering is not sufficient.

One explanation for this discrepancy is that the closer we are to the model, the more uncharted the waters. Agent security materials tend to focus on the novel. In practice, though, things like excessive agency tend to appear much more in incident reports.

As you cannot audit the model's intent, you have to bound its capability. So let's look at where your agent controls live, their coverage and failure points, and how much they actually matter in the real world.

Deterministic or Probabilistic? You need both

First, let's disassemble an agent request:

  • An instruction arrives.
  • Context and memory are assembled around it.
  • The model forms a plan.
  • The model selects tools.
  • The model presents credentials.
  • The action takes effect.
  • A record is written.

Off the path sits the general governance that decides which agents are in and out of scope.

Now let’s think about the controls. As always, you have deterministic controls that make actions impossible, and probabilistic controls that try to judge intent with a stated error rate. They’re not interchangeable; you’ll need both to succeed.

Layer 0: What You Inherit and Cannot Inspect

You don’t control everything in your agentic stack. Similar to the Shared Responsibility Model of Cloud Computing, some things you can’t fully inspect or control directly:

  • The base model. Training data, weights, fine-tuning history, and the refusal behavior that ships with it are not under your control.
  • The inference provider. Their infrastructure, tenancy isolation, and version changes that shift your agent's behavior are also upstream of your controls.
  • Agents you do not operate. Third-party agents yours negotiates with, and the agents your SaaS vendors have already shipped into your tenant, are controlled by someone else.
  • Coarse-grained target APIs. Systems that offer admin or nothing will cap how well you can ever enforce least privilege.

What you do control about this layer is your side of a vendor-client relationship. Model versions can be pinned and recorded, for example. You can decide to self-host parts of the infrastructure. Provider due diligence also lives here.

Layer 1: Prompt and Model Defense

This is the layer with the most products and published research. It's focused on system prompt hardening and instruction hierarchy. In it, you can delimit or spotlight untrusted content, classify input to detect injection attacks, filter output to prevent data loss, and perform adversarial evaluations.

This is the only layer that sees intent. Downstream, you'll see agents performing legitimate tasks with legitimate permissions, even if the person who requested the task shouldn't have. Only the semantic layer can see that request for what it is.

That said, you must assume controls at this layer will sometimes fail, as all probabilistic controls do. Since there's no clean boundary between instruction and data inside a context window, every check is a judgment call on ambiguous text. Moreover, your attacker will get unlimited retries against a fixed target, while detection skews toward the patterns it was trained on. And a payload can be split across several inputs, stumping detection algorithms.

In the OWASP Top Ten list, these controls address ASI01 (Agent Goal Hijack) and LLM01 (Prompt Injection).

The test to apply: what bypass rate do you assume, and what can the agent do when the bypass succeeds?

Layer 2: Context and Memory Integrity

This layer has solutions that decide whether an agent can tell your instructions from someone else's. In it, you label the provenance of retrieved content, validate memory writes, allow-list retrieval sources, and keep context ephemeral by default, so that promotion to durable memory is done intentionally.

It’s the only layer that has data that indicates where a piece of text came from. Once context is assembled, a poisoned instruction and a legitimate one propagate in the same way: the tool layer sees a proper call, the identity layer sees a valid credential, and the enforcement layer sees a permitted action.

That said, assume poisoning is patient. A dedicated attacker will make the payload appear to be legitimate data when it's written. It will then fire later, often in another session under another user, which will break the link between cause and effect that your incident response depends on. Moreover, provenance labels decay between summarizations.

In OWASP's Top Ten list, the 2026 LLM list renamed System Prompt Leakage to Hidden Context Exposure, broadening it beyond the prompt to include developer instructions, retrieved policy text, and tool schemas. But it stops at static context, so persistent memory and inter-agent channels are on the agentic list, where the write side is ASI06 (Memory & Context Poisoning).

The test to apply: for any action your agent takes, can you say which parts of its context you wrote and which arrived from outside?

Layer 3: The Tool and MCP Surface

This layer determines what the agent can attempt. Your job is to:

  • Give the agent the few tools its task needs rather than a server's full catalog
  • Validate arguments and schemas on every call, allow-list servers and egress
  • Verify signatures on tool definitions.
  • Run agent-authored code in a sandbox with deny-by-default network and filesystem access.

It's the only layer that removes a capability rather than governing it. No amount of injection makes an agent do something it has no tool for, which is why limiting the catalog is worth more than checks you apply to an available call. Most of this is deterministic and enforceable.

That said, it's a layer that’s narrower than it looks. It only covers the tools you know about, whereas developer-installed MCP servers bypass your approval process. And scoping a tool tells you nothing about how far that tool can reach once called. A narrowly scoped tool with a broad credential is still broad, which is why the next layer exists.

In OWASP's Top Ten, these controls correspond to ASI02 (Tool Misuse and Exploitation); ASI04 (Agentic Supply Chain Vulnerabilities); and ASI05 (Unexpected Code Execution).

The test to apply: for each tool your agent holds, can you name the task that needed it?

Layer 4: Identity and Access

This layer determines how far a compromise can spread. Identity and Access Management (IAM/PAM) is hardly a new concept, but it applies (albeit imperfectly) to agents as well. Give each agent its own identity rather than a shared service account or a borrowed developer token, make its authority delegated, shorten credential lifetime, derive scopes from observed use, and treat revocation and a working kill switch as preconditions.

It's the only layer that still applies once every other one has failed. Ideally the layers above hold, but when they don't, what the agent was permitted to do is the whole of your exposure. It's also the one layer that gives you attribution, since an action taken under a shared account tells you nothing about which agent took it.

Worth noting just how much delegation matters here, even though there’s no accepted standard for it. An agent acting with its own broad permission set, on behalf of a user who has far less, becomes a confused deputy: the requester reaches through the agent to do something they could never do directly. Bind the agent's reach to the entitlements of the requester, or you’ll have a privilege escalation path.

That said, assume your target systems won’t play ball. Plenty of APIs offer admin or nothing, for example. OAuth scopes also tend to be wider than any single task needs because they describe what an application does rather than what a single invocation requires. And you can't right-size an agent you haven't inventoried.

In OWASP's Top Ten, this is ASI03 (Identity and Privilege Abuse).

The test to apply: if one agent identity were compromised right now, what could it reach, and how would you know which one it was?

Layer 5: Runtime Enforcement and Containment

This layer sits between an agent's decision and the decision’s effect. It’s based on a chokepoint, oftentimes a gateway or a proxy, that:

  • Sees every outbound action with its arguments and the identity behind it.
  • Sets action-level policy keyed on reversibility.
  • Requires human approval for anything irreversible
  • Caps rate/volume/spend
  • Isolates network and filesystem access
  • Segments agents so compromise doesn't chain

It's the only layer that can stop an action the agent is fully entitled to take, as it sees the attempt in context. This is also why it holds when the agent is subverted outright.

That said, it costs you latency and engineering time to classify actions by reversibility. Then there’s approval fatigue: a gate that fires forty times a day will inevitably become a rubber stamp.

In OWASP's Top Ten, this is where ASI08 (Cascading Failures) and ASI09 (Human-Agent Trust Exploitation) get answered.

The test to apply: which of your agent's actions can't be undone, and what stands between the agent and those?

Layer 6: What You Log and What You Detect

This layer is responsible for what you'll be able to say after the fact. Ideally you capture the whole chain: the instruction, the context, the plan, each tool call, the identity, the outcome, and a correlation ID that survives multi-agent hops. From there you’ll be able to baseline behavior per agent, alert on deviation, and retain everything for however long is needed.

It's the only layer that finds the mistakes in the others. Every other layer works from an assumption about what normal is, and this is where you can discover if an assumption was wrong. Broad scopes, tools that were granted erroneously, and failing agents are three of countless examples you discover in the logs.

That said, assume detection is late and noisy. First, you need to establish a baseline. Then, you still have a high volume of events and almost all of them are benign. Sometimes, the baseline is reset by model version upgrades, and you get buried in false positives.

In OWASP's Top Ten, this is ASI10 (Rogue Agents).

The test to apply: for your agent's last significant action, can you reconstruct what it was asked, what it read, what it called, and whose credential it used?

Layer 7: Governance and Lifecycle

This layer decides which of your agents are covered by the other seven. In it, you:

  • Run continuous discovery that finds shadow agents
  • Assign a named human owner to each one
  • Gate deployment on registration and approval
  • Threat model at design time
  • Review entitlements on a schedule
  • Make decommissioning actually revoke credentials and remove tool grants.

It's the only layer that isn't a control. Everything above can judge a request or constrain an action, but only for known agents. Governance should be the boring layer, but in practice, it’s the one where much of your work will exist.

That said, assume discovery won’t be able to keep up with creation. Developers and business users create agents in minutes.

The test to apply: name the owner of every agent in your environment, from a system and not from memory.

The layers side by side

Layer Enforcement What it stops Assume it fails when OWASP anchor
0. What you inherit None available to you Nothing you can verify Always. Plan for it downstream LLM04, LLM05
1. Prompt and model Probabilistic Goal hijack, malicious intent in ordinary form Attacker retries, or splits the payload across turns ASI01, LLM01
2. Context and memory Mixed Poisoned retrieval and memory writes Payload looks legitimate at write time; labels decay ASI06; LLM08 for static context only
3. Tools and MCP Mostly deterministic Tool misuse, unreviewed servers, unsandboxed code The tool is unknown to you, or holds a broad credential ASI02, ASI04, ASI05
4. Identity and privilege Deterministic Privilege abuse, confused deputy, lateral reach Target API offers admin or nothing ASI03, LLM03
5. Runtime enforcement Deterministic Irreversible actions, cascading failure, runaway spend Approval volume produces rubber-stamping ASI08, ASI09
6. Logging and detection Post-hoc Rogue behavior, slow-burn compromise No baseline yet, or the model version changed ASI10
7. Governance Procedural Unowned and undiscovered agents Creation outpaces discovery Cross-cutting

How the layers work together

Probabilistic controls can reduce how often you’ll get a subverted agent in the system. Deterministic controls will help you reduce the impact of those that do appear. Both need to work together.

The asymmetry of scale matters here a lot. Suppose a filter stops 99% of injection attempts and your agents handle ten thousand requests a day. A hundred still get through, and on them, the only question that matters is what access the agent has. When a team has to choose where the effort goes, that arithmetic favors privilege and enforcement.

Layer 0 is that same argument taken to its limit. When a model is poisoned, you get no alert and no artifact. The only thing between that and a material incident is what the agent is permitted to do. A control that works when you have zero visibility into your adversary is worth more than one that works only when you have perfect visibility.

Where to start if your agents are already running

Ordered by risk reduction against effort, for a team with agents in production and no program yet:

  1. Inventory all AI agents. You cannot scope, log, or revoke what you have not found.
  2. Separate identities. Break the shared service accounts and borrowed human tokens first.
  3. Shorten credential lifetime.
  4. Put one chokepoint in front of the irreversible actions.
  5. Turn on the action log.
  6. Add prompt-layer filtering, with a documented assumed bypass rate.
  7. Pin model versions and re-run evaluations on upgrade.
  8. Formalize ownership and the deployment gate
  9. Apply the tests, layer by layer

‍