Governed AI Agents: One Policy Model, Several Runtime Enforcement Points

A support agent gets a refund request. The message includes an account number and a phone number. To handle it, the agent needs a language model to read the request, an internal tool to look up the order, a CRM reached over MCP to add a note, and possibly a second agent to review an unusually large refund. None of that is exotic any more. The hard part is that every hop in that chain crosses a different policy boundary, and each crossing raises its own question.

• Is this customer data allowed to go to the selected model provider?
• Is the request still within its spending budget?
• May this workload use that model in this region?
• Should the CRM server receive the whole conversation?
• Can the refund tool pay out $2,000 without a human approving it?
• If the preferred model is down, does the fallback offer equivalent data residency and safety?

A policy document can answer such things in principle, and a system prompt can remind an agent of a few. Neither enforces anything consistently. That is why agent governance turns into a runtime problem.

A published framework for governed agents describes the shift well: governance sets the rules, and a gateway is one place where those rules meet live traffic. It organizes the work into five activities, Govern, Decide, Protect, Observe and Assure. A useful point inside it is that model calls, tool calls, MCP calls and agent-to-agent hops each carry different risks. That distinction supports a better design than cramming every control into one huge prompt or one huge gateway.

Let the policy travel with the request

The simplest picture is a governance envelope that accompanies an interaction through the system. It carries enough context to decide before a boundary is crossed:

• identity and tenant,
• workload and agent,
• risk tier and data classification,
• current spend and limits,
• permitted providers and regions,
• permitted tools and action scopes,
• thresholds that require human approval,
• policy version, plus a trace or decision ID.

Not every component needs every field. The envelope exists so the same kinds of question can be answered consistently wherever the agent goes.

• For a model call the relevant facts are spend, provider, region, retention and redaction.
• For a refund tool they are the user's identity, ownership of the order, the amount and the approval tier.
• For an MCP call they are the server being contacted and the data leaving the infrastructure boundary.
• For an agent-to-agent hop it is whether the receiver may see the context handed over.

So the design is not one universal filter. It is a single policy model with several enforcement points. Runtime governance becomes concrete at this stage, because the useful question is not whether a policy exists but which component is able to halt the interaction ahead of any side effect.

A small, vendor-neutral sketch

No governance platform is needed to see the pattern. The Python below is deliberately neutral and not tied to any product API. Its job is to expose the decision points.

from dataclasses import dataclass
from enum import Enum

class Decision(str, Enum):
    ALLOW = "allow"
    BLOCK = "block"
    APPROVAL = "approval"
    FALLBACK = "fallback"

@dataclass(frozen=True)
class PolicyContext:
    user_id: str
    workload: str
    risk_tier: str
    data_class: str
    spend_usd: float
    spend_limit_usd: float
    policy_version: str

def before_model_call(ctx: PolicyContext, model: str) -> Decision:
    if ctx.spend_usd >= ctx.spend_limit_usd:
        return Decision.BLOCK
    if ctx.data_class == "restricted" and model not in APPROVED_PRIVATE_MODELS:
        return Decision.FALLBACK
    return Decision.ALLOW

def before_tool_call(
    ctx: PolicyContext,
    tool: str,
    amount: float = 0,
) -> Decision:
    if tool not in TOOL_SCOPES[ctx.workload]:
        return Decision.BLOCK
    if tool == "issue_refund" and amount > 500:
        return Decision.APPROVAL
    return Decision.ALLOW

def before_mcp_call(
    ctx: PolicyContext,
    server: str,
) -> Decision:
    if server not in APPROVED_MCP_SERVERS:
        return Decision.BLOCK
    if ctx.data_class == "restricted" and server in EXTERNAL_SERVERS:
        return Decision.BLOCK
    return Decision.ALLOW

The syntax matters less than the list of things the model is never allowed to settle. It can reason about what the customer wants. It cannot rule that a $2,000 refund now fits policy, or that an unapproved MCP server is fine because the request reads innocently. Those outcomes are deterministic state changes owned by the runtime.

A production version should also emit evidence for every decision: the policy version, input scope, outcome, reason, actor, timestamp and trace ID. That evidence is what later proves which policy actually ran, as opposed to merely showing that a policy document existed.

A model gateway helps, but it is not the whole design

In that framework an LLM gateway is a shared layer for authentication, model choice, trimming context, spend policy, failure handling and decision evidence. One current product from an agent-tooling vendor makes it tangible. It entered public beta by August 2026 and offers spend and rate limits, model fallbacks, provider routing, protection for sensitive data and trace integration. Adoption for ordinary model clients is meant to be light: aim the client's base_url at the gateway, store provider keys centrally and set policy in one place.

That matters because model governance otherwise gets copied into every agent.

agent A → custom rate logic → OpenAI
agent B → custom PII logic  → Anthropic
agent C → custom fallback   → Gemini
agent D → no controls       → self-hosted model

A gateway can collapse the pile into one shared layer.

  agents
     ↓
shared model-call policy
     ↓
approved providers / hosts

Resist an easy overgeneralization here, though. The governance framework is broader than any one gateway product. It discusses tool calls, MCP calls and agent-to-agent interactions explicitly, because the biggest agent risk is often the action an agent can take and not the text it produces. The gateway's public-beta controls are mainly about model traffic. Adjacent enforcement is still needed in the agent harness, the tool runtime, the MCP boundary or a dedicated action gateway. The split is healthy, since a model gateway should not pretend it can authorize a production database change that it never sees.

Reliability counts as governance

Once a gateway sits in the critical path, governance can cause an outage of its own if failure behavior is vague. Two ideas help.

First, each workload needs an explicit fail-open or fail-closed posture. A low-risk summarization request that cannot reach the policy service might degrade gracefully. A restricted-data request that cannot prove its redaction or residency policy ran may not be allowed to fail open.

Second, a fallback model must be policy-equivalent and not merely available. Suppose the primary model was approved because it keeps data in a required region under an agreed retention policy. An automatic switch to a cheaper or more available provider that violates those terms does not add resilience. It trades a compliance breach for the appearance of staying up.

Spend controls follow the same logic. Hard caps help, but a cap with no alert and no escalation path turns governance into an arbitrary outage. A better design is a sequence.

usage approaches threshold
        ↓
      alert
        ↓
route / reduce context / downgrade model if allowed
        ↓
hard block when policy requires it
        ↓
auditable exception path if business-approved

For this reason model routing is closer to portfolio management than to sending easy prompts to a cheap model. Cost matters, and so do latency, data handling, quality, risk tier and availability.

The cost of a central control plane

Moving policy into one place spares each agent from carrying its own copy, yet it also gathers responsibility in that place. The runtime layer now stands in the path of model traffic and, in some designs, helps authorize actions, so it has to be engineered like any other critical service.

• Policy checks must run fast enough not to dominate request latency, and the policy service needs redundancy of its own.
• Price data for each provider has to stay current, and cached, batch and reasoning tokens must be counted accurately for spend caps to deserve trust.
• Redaction rules should be tested for leaks as well as false positives.
• Policy edits need versioning, rollout and rollback in the same way as application code.

A second cost is organizational. A central platform team can make safe defaults effortless, but controls cast too widely can slow product teams down. A path from visibility, through control, to assurance softens that: watch first, run new policy in warning or shadow mode, measure what happens, and block outright only where the risk warrants it. The layer should remove repeated work without becoming an opaque dependency that nobody can bypass, debug or safely modify.

Guardrails should not own consequential actions

Guardrails come in two kinds. Pattern rules suit structured secrets and identifiers. Model-based detectors cope with fuzzier categories such as unstructured personal data, jailbreak attempts or groundedness. Neither is perfect, and model-based checks are probabilistic, so they can produce both false positives and false negatives.

A sound production rule follows: content detection can lower risk, but it should never be the only control standing between an agent and a consequential side effect.

• A refund limit is better written as code than as a classifier.
• A production deployment should have an authorization check.
• A high-risk action may need human approval.
• Credentials should be scoped so an agent cannot exceed what its task requires, even if a guardrail misses something.

The framework puts the boundary the same way: consequential actions rely on deterministic limits or human approval and not on content detection by itself. It is the gap between spotting unsafe intent and containing unsafe capability. OWASP's guidance on excessive agency arrives at the same place independently, advising minimal tool functionality and permissions, with authorization enforced in downstream systems and not left to the model's own judgment. Both layers are required.

Observing and assuring

Tracing becomes far more useful once policy decisions are part of the trace.

• A blocked model request shows which spend policy fired.
• A redaction event shows which policy version applied without exposing the sensitive value.
• A tool approval is linked to the action it gated.
• A fallback records why the primary failed and why the replacement was judged acceptable.

Evaluations can then ask more than whether the model is accurate. They can test whether a smaller model keeps quality after a routing change, whether trimming context cuts cost without hurting outcomes, whether a redaction policy creates too many false positives, and whether a new fallback remains policy-equivalent.

Together these close the five-part loop. Govern sets ownership, risk and policy; Decide routes, escalates and fails over; Protect enforces at every boundary; Observe measures behavior and policy outcomes; Assure preserves lineage and tests changes. Deploying a rule does not finish the job. The job is finished when it can be shown which rule ran, what it decided, what followed, and whether the control keeps working after the system changes.

What this means for agent architecture

What production agents need is a runtime control architecture, whichever gateway is chosen. A central model gateway is a good home for spend, provider choice, redaction, rate limits and fallbacks, and it brings model optionality without touching every application. A governed agent still needs the surrounding layers:

• authorization and fixed limits on tool actions,
• trust decisions about MCP servers plus rules on what data may leave,
• minimized context and verified identity between cooperating agents,
• approval or containment for anything with serious consequences,
• a traceable policy lineage behind every decision.

Back at the refund request, the model can still reason and the agent can still act. The architecture just stops asking the model to carry the whole governance program in its prompt. Every boundary that matters gets its question answered ahead of the crossing:

• Who is asking?
• What data is moving, and where to?
• What is the workload permitted to spend?
• Which models and tools are allowed?
• What is the plan if the preferred path breaks?
• Is a human sign-off needed?
• Which policy version made the call?

That is what separates having AI policies from having governed agents.