Your AI has a threat model nobody has written down.
Classical AppSec has no section for an agent that can call your API. I go through what you have shipped and find the ways it breaks — then help you close them.
| Signal | Severity | Usually means |
|---|---|---|
| An agent can call tools nobody enumerated | High | Unbounded blast radius |
| User text reaches a prompt unescaped | High | Prompt injection is live |
| Model provider logs your customer data | High | Data leaves your boundary |
| Approval gate exists but always says yes | Moderate | Oversight theatre |
| Nobody can list what the agent did last week | Unknown | No audit trail |
Why your usual security review misses this
A penetration test looks for injection into a database. It does not look for injection into a context window, because until recently that was not a place attackers could reach.
The specific problem with LLM systems is that instructions and data arrive through the same channel. A support ticket, a PDF, a web page your agent fetches — any of them can contain text that reads as a command. If the agent holds credentials, that text now holds credentials too.
The second problem is blast radius. Most teams grant an agent the permissions a human on that team would have, because that is the easy thing to do. But a human does not retry two thousand times a minute after misreading one instruction.
What I actually do
- Map every tool your agent can call and what each one can reach, including the transitive paths nobody drew on the architecture diagram
- Attack the prompt surface with your real inputs — tickets, documents, uploads, retrieved pages — not a generic jailbreak list
- Trace where context goes: logs, traces, analytics, the model provider's retention policy, and any vector store you are writing to
- Test the approval gates by trying to get past them, and check whether anyone reads the approvals
- Review what the agent can do that cannot be undone, and whether those actions have a second pair of eyes
What you end up with
- A written threat model for your AI system, specific enough to argue with
- Findings ordered by what could realistically happen to you, with the attack path spelled out for each
- Concrete fixes — permission scoping, input handling, gate placement — not "consider implementing a policy"
- A short list of what to monitor so the next regression is caught by a test rather than a customer
How the work runs
| Stage | What happens |
|---|---|
| Week 1 | I read the code and the architecture, and we walk through what the agent is allowed to do. |
| Weeks 2 to 3 | I attack it. Prompt surface, tool permissions, data paths, gate bypass. |
| Weeks 4 to 5 | Findings written up with attack paths, then we work through the fixes together. |
| After | A short re-test on anything that was rated high, usually a few weeks later. |
Questions people ask
Is this a penetration test?
No. A pen test covers your infrastructure and web surface, and you should still have one. This covers the failure modes that exist because there is a model in the loop — prompt injection, tool permissions, context leakage. The two barely overlap.
We use a hosted model. Isn't security the provider's job?
They secure their infrastructure. They do not decide what tools your agent can call, what you put in the context window, or whether a user can talk your agent into doing something. That part is yours.
How long does it take?
Three to five weeks for most systems. A single agent with a handful of tools is closer to three; a multi-agent system with a shared tool registry is closer to five.
Will you fix things or just report them?
Both. The report is written so your team can act without me, but I would rather be in the pull requests. Most of these fixes are small once someone has named the problem.