Your output filter is not a guardrail

I was building a support agent that could actually do things. Look up an order, read a policy, issue a refund. The usual demo. And it had a guardrail on it, which I felt fine about, right up until I read a trace properly.

The trace looked like this.

1  request received          (no input screening)
2  model emits tool_use: issue_refund(order=...)
3  tool executes             (refund issued)
4  output filter runs        (text looks clean, passes)

Step four is my guardrail. Step three is money leaving the company. The safety check ran after the only irreversible thing on the path.

Nothing was broken, either. Every piece did exactly what it was built to do. The filter read the assistant's reply, found nothing unsafe in it, and passed it. It was never asked about the refund, because a text filter has no idea what a refund is. It sees a sentence.

#Why this design keeps happening

Because output filtering is the guardrail you can point to.

It's one box on the diagram, at the end, right before the user. It fires where a reviewer can watch it work. It's the last thing that happens, so it feels like the final word. You add it, the diagram looks complete, and the security review passes.

Then you give the model tools, and the shape of the problem changes underneath the diagram, and nobody redraws it. When the model could only talk, the last thing on the path was the words. Once it can call tools, the last thing that matters happens in the middle.

#Three questions, not one

There isn't one place a check belongs. There are three, and they ask different things.

Input screening runs before the model call. Should this request reach the model at all?

Output screening runs before the reply goes back. Is this text safe to return?

Tool call authorization runs before any action with side effects. May this caller take this action, in this context, right now?

Different points on the path, and different objects: a request, a string, an action. That's why the one you have does nothing for the two you don't. I had the second one and quietly assumed it covered all three.

The third one is the only one that stops a refund.

#Authorization is not a prompt

The tempting fix is to write it into the system prompt. "Only issue refunds under $50. Always confirm the order belongs to the user."

I've done this. It works in testing. It isn't a control, it's a request. The thing you're asking is the same thing the attacker is talking to, and anything that can be phrased can be rephrased. A control you can argue with is not a control.

Authorization should be the least interesting code in the system.

def authorize(call, caller):
    if call.name not in ALLOWED_TOOLS[caller.role]:
        return Deny("tool not permitted for role")
    if call.name == "issue_refund":
        if call.args["order_id"] not in orders_for(caller.id):
            return Deny("order does not belong to caller")
        if call.args["amount"] > REFUND_LIMIT[caller.role]:
            return Deny("over role limit")
    return Allow()

No model call. An allowlist, an identity check, a scope check. Boring on purpose. You want the answer to "why was this refund allowed" to be a line of code and a log entry, not an inference about what the model was probably thinking at the time. Deterministic checks are the ones you can prove and replay.

The rule I've settled on: use a model where the question genuinely needs judgment, like whether an ambiguous input is a jailbreak attempt or whether an output is toxic. Use plain code where the rule is already defined, like whether this caller may do this thing. Authorization is nearly always the second kind.

#The input you never screened

This is the one that took me longest to see.

Input screening looks at what the user sent. It does not look at what your system went and fetched.

A retrieved document arrives after the request already passed screening. A tool response arrives after that. Both get appended to the context, and the model treats both as content it should act on. If a support ticket sitting in your knowledge base contains a line addressed to the model, your input filter never saw it. It wasn't input. It was retrieval.

If your system does RAG, or has tools, this is where injection actually comes from. It needs its own control: screen retrieved content and tool output before it enters the context, using the same classifier you already run on user input. Same check, different door.

#The failure mode nobody chooses

Then there's the way all of this quietly stops working.

A screening service is a dependency like any other. It times out, it returns a 500, it gets slow under exactly the load where you need it most. So what happens to the request when the check itself fails?

You either let it through or you block it. There is no third answer. And if you didn't decide, the surrounding code decided for you, and the default is almost always this:

try:
    verdict = screen(request)
except Exception:
    verdict = ALLOW   # nobody wrote this line, it just is

That's failing open, and it's the worst outcome on the menu, because the system looks healthy the entire time. Requests keep flowing. The dashboard stays green. The guardrail is doing none of the work it was put there to do, and nothing tells you, because passing traffic is what success looks like from the outside.

You already know how to handle this. It's the same reasoning as a circuit breaker around a flaky dependency. Decide how the thing degrades, instead of inheriting whatever behavior happens to keep requests moving. On a path with side effects, fail closed and take the outage. An error your users can see beats a protection you only think you have.

Worth being precise about scope here. This is about the guardrails you build and run. The model's own trained safety behavior isn't something you configured, and it isn't what falls over when your classifier times out. The screening service is yours, so its failure direction is your decision.

#What it costs

Because none of this is free and I don't want to pretend otherwise.

Every screening point is another call or another rule evaluation on every request. A judge model on the output roughly doubles the model cost of that turn. Three gates, each with a check, a failure direction and a log line, is a lot more to build and test than one filter. Pre-action approval adds latency to everything it touches.

That's a real trade, and you make it per gate rather than all at once. Which is convenient, because the cheapest gate is the one that matters most. Authorization on a side effecting tool is a dictionary lookup and an if statement. It costs nothing, and it was the one my design was missing.

#Where I landed

Log every gate that blocks and every gate that errors. When something goes wrong you want to reconstruct the path, and "the filter passed it" is not a story you can tell an auditor.

The rest is one sentence.

Filter the text on the way out if you want to. Authorize the actions before they run. Only one of those two stands between your agent and the money.

Have you gone back and checked where the gates actually sit in an agent you already shipped? I'd like to hear what you found, especially if the answer was the same as mine.