How do you make an AI agent Secure by Design?

Who will build the next agent in your company? It’s likely not a developer. Business users are all rapidly becoming builders. However, the concepts they're learning by interacting with agent-building tools aren't necessarily security-oriented. And that's a problem.

First and foremost, it exposes the fundamental tension in the article's title. It's not just about how you, a security practitioner, can make an AI agent Secure by Design.‍

Securing agents is now a team sport, and the security team plays a different position. Instead of owning every control yourself, you carry the parts builders shouldn’t have to think about and make the rest easy. Teach people who care little about security the handful of decisions that are theirs, and make the safe path the default. Ask them to pick a design pattern or an identity model, and you’ll get nowhere.

Secure by Design principles accommodate this thinking. The first principle states that the security burden shouldn't fall solely on the customer. And builders are, confusingly, customers in this case.

In this article, we'll cover three things: the concepts, how to create guardrails for builders, and what you should ask them to do. 

What Secure by Design means

While the concept has been around for over 50 years, CISA and its partner agencies described modern Secure by Design principles in 2023, in Shifting the Balance of Cybersecurity Risk:

• Take ownership of customer security outcomes. The manufacturer carries the burden, and products are safe out of the box, without extra configuration.
• Embrace radical transparency and accountability. Share what you learn about your product's weaknesses.
• Lead from the top. Executives own security as a business requirement.

Two later documents, co-authored by CISA, apply them to AI: the Guidelines for Secure AI System Development and eful Adoption of Agentic AI Services. 

For agents, Secure by Design means that when the agent absorbs malicious input, that input should have nowhere useful to go, and when the agent errs on its own, the damage stays small and reversible.

The builders

You now have four categories of builders.

In theory, the more you offload to the security and platform team, the less is required from the business builder. That way, a business builder who just clicks through the default options still gets a relatively safe agent.

As anyone who's done agentic discovery knows, the reality is far messier. You're bound to find agents outside the purview of the professionals. 

The theory

Jerome Saltzer and Michael Schroeder of MIT published a set of design principles in 1975 in “The Protection of Information in Computer Systems”. Fifty years later, Open Security Architecture still calls them “the canonical reference” for building systems that fail safely. Here is how seven of those principles apply to AI agents:

Then there's reversibility. Drafting a memo is easily reversible; sending it is not. The same goes for sending money or deleting files. Things that are difficult to reverse should have more stringent gates in front of them.

Finally, there's the "Agents Rule of Two", authored by Meta in October 2025, which builds upon Chromium's rule of the same name.

Until prompt injection can be reliably detected and refused, an agent must have no more than two of these in a single session:

• [A] It processes untrusted input.
• [B] It can access sensitive systems or private data.
• [C] It can change state or communicate externally.

With all three, an email can tell the agent to read your inbox and forward it to the attacker. And if a task absolutely requires all three, an agent needs human approval or another reliable check, or has to start a fresh session. Treat the rule as a ceiling, not a clean bill of health: an agent with two of the three still needs least privilege.

Among the scenarios this doesn't account for in theory are rogue actions. That's when the agent causes harm with no attacker involved, by misreading a request, hallucinating a tool call, or overreaching. In practice, you also have things like approval fatigue, when a human acts as a rubber stamp.

The platform team: make the safe path the easy one

This is where CISA's first principle gets concrete. Set these properly, and the business builders will have fewer opportunities to make security mistakes.

Issue the identity for them. Ideally, the platform gives each agent an identity that is registered and bound to roles. This way, a business builder won't have to handle credentials, and the agent won't run on the builder's login. When the agent acts for a user, its effective permissions should be the intersection of the user's and the agent's, so it never exceeds either.

Widen access gradually. New agents should start with one team and a narrow scope. If something goes wrong, fewer people are affected, and it'll be easier to roll back to a known-good version.

Keep a working off switch. Confirm before launch that you can revoke a single agent: its tokens are invalidated, in-flight runs stop, and it loses access everywhere it connects.

Pro developers: build patterns that hold when the model doesn't

Code-built agents get the same platform defaults, as well as architecture decisions a no-code builder never touches.

Pick the narrowest pattern

In June 2025, researchers from ETH Zurich, IBM, Google, Microsoft, Invariant Labs, and others published six prompt injection-resistant design patterns for agents. They share one rule: once the agent has absorbed untrusted input, that input can't trigger an action with meaningful consequences.

Each pattern gives up flexibility, which is exactly the point. An agent that can do anything can be talked into anything, so it should be able to do as little as possible while doing its job.

Control the supply chain

Agents tend to have extensive supply chains, which come with their own dangers. Every MCP server, plugin, and tool is code you didn't write. Here, too, reducing flexibility is the security-minded move, and so is verifying signed provenance for the models, datasets, and MCP servers you load. 

• Allow-list tool sources. Pin versions, hash tool definitions at approval, and diff them at every session start to catch tool poisoning and rug pulls.
• Red-team in a sandbox, and run agent-written code in a sandbox in production.
• Re-test on every change. Updates to approved tools are classic opportunities for supply chain attacks, so re-verify pinned tools before accepting an update. 
• Run capability evaluations continuously on every change to the model, prompt, or tool set, and block releases on regression,, because a new model or tool can change how the agent behaves.

Hand this to your business builders

Business builders are unlikely to care about everything written above. Their checklist is more informational, designed to align them with the security team as easily as possible. Here's a sample message you can send business builders:

Before you switch your agent on

1. Say what it's for and who it's for in one sentence. Register it, so we know it exists.

2. Answer three questions:

• Does it read anything from outside the company, like customer emails, web pages, or uploaded files?
• Can it see anything you can see in our systems, or any customer, financial, HR, or other private data?
• Can it send, pay, change, delete, or post anything without you checking first?

Yes to all three? Talk to security before you turn it on. We'll help you secure it.

3. Use the approved connectors only. Don't sign the agent into our systems with your own account, and never paste a password or API key into it.

4. Keep yourself in the loop for anything irreversible.

5. Start small. Make sure it works, then give it to others.

6. Name a backup owner. If you change roles, hand the agent over, or switch it off and ask us to revoke its access.

7. If it does something odd, switch it off and tell us. You won't be in trouble for it.

8. Bringing in an agent you didn't build? Same questions, before you connect it to anything.

Lead from the top: the review gate

A senior person has to decide whether to launch based on the answers to these questions.

Agents you didn't build

Discovery will show you the fourth kind of builder: someone outside the company. 

Increasingly often, a SaaS vendor switches on an agent inside a tool you already pay for. Employees install agents on their devices or sign up for hosted personal AI agents. None of these run on your platform, so defaults don’t apply. 

You can't design someone else's agent, but you do design and own the doors it comes through. While you’re not strictly making the agent “Secure by Design”, it’s adjacent enough to be within the scope of the process. 

Then apply the Rule of Two from the outside. You can't change how the agent handles untrusted input, but you decide whether it gets reads and writes.

Some agents skip the doors entirely. As of October 2026, Muse, Grok Bot, and Dots are among those that get their own cloud computer and sign in to your tools the way an employee would, including tools with no API. 

The main control left here is conditional access: deciding how your identity provider treats a sign-in from a machine it has never seen.

That's the Secure by Design lesson in reverse. If the safe path is slower than the easiest path, people will choose the easy one. Making your platform the easy option should be paramount. CISA has a name for the customer side of all this: Secure by Demand. 

Operation and maintenance

Once the agent runs, you find out which design choices made it in. The rest of this series covers this phase:

• What to capture: What agent actions should you log?
• What to watch for: How do you detect a compromised or rogue AI agent?
• What to keep for auditors: What audit evidence do you need for AI agents?

And from CISA's transparency principle: when an agent fails, tell the builders. A brief note on what went wrong helps make the next agent better.

If you're buying, not building

When a vendor ships the agent builder itself, CISA's transparency principle is your leverage. Add these to the vendor questions in How do you extend Zero Trust to AI agents?:

• Can you limit builders to an approved list of connectors?
• Does each agent get its own identity, or does it run on the builder's login?
• What's on by default, and what can your admins lock?
• Can you revoke a single agent, including its tokens, and how fast?
• Can we export every agent action to our SIEM, with the correlation ID and the human it acted for?
• Can you provide an AI-BOM and signed provenance for the models and tools you ship?