AI Gateway
An AI gateway is the enforcement point between AI agents and the models and tools they call. Every call passes through it, so the gateway can verify the caller's identity, apply policy before execution, enforce budgets, and write the evidence record.
What an AI gateway means
The pattern is borrowed from API gateways, applied to a new kind of traffic. An agent's calls to model providers and tool servers are remote, metered, and consequential — exactly the traffic that should not flow directly from application code. The gateway sits in the path and does what an API gateway does, with agent-specific additions: it verifies the agent identity on each call (not a shared key), checks the call against the grants for that specific agent, applies content and data policies (blocking secrets and regulated data, redacting what may pass), enforces the agent's written budget — calls past the cap are refused, not warned — and records the interaction as evidence. Two gateway types cover the estate: the AI Gateway for model calls, and the MCP Gateway for tool-server traffic, with per-server guardrails like rate limits, PII redaction and approval requirements. The gateway is where "oversight outside the execution loop" becomes code: the agent cannot bypass it, because its credentials only work through it.
Frequently asked questions
What does an AI gateway do?
Verifies which agent is calling, checks the call against that agent's grants, applies data and content policy, enforces spend caps by refusing over-budget calls, and writes the evidence record — all before or during the call, not in review afterward.
Why do agents need a gateway instead of direct API access?
Direct access means policy is whatever the application code remembers to do. A gateway makes the checks architectural: the agent's credentials only function through the gateway, so identity, policy and evidence cannot be bypassed by a bug or a prompt.