Talk to us

What is an AI control plane?

Where policy for AI agents gets decided, where it gets enforced, and why it helps to treat those as separate jobs. With one reference design, a look at MCP gateways and budgets, and a worked approval.

Written by Arya Sharan · Updated

On this page
  1. Short answer
  2. Where the term comes from
  3. What a control plane decides
  4. Deciding and enforcing
  5. Gateway, framework, control plane
  6. Where an MCP gateway fits
  7. A worked approval
  8. Cost and budgets
  9. Where the data goes
  10. What it does not do
  11. Next steps
  12. Related reading
  13. Sources
Short answer

An AI control plane is a shared layer for defining and applying policy to AI agents. It connects each agent to an owner, controls which models and tools it may use, sets approval and spending rules, and brings the evidence of its actions into an audit trail.

This page explains one AI agent control plane architecture. Policy decisions and enforcement are separate roles in this design; products may combine them.

Key takeaways

  • Shared policy needs enforcement in the path of a request: at gateways, runtimes or business systems.
  • Record the agent's identity, its owner and the person who initiated the task separately.
  • An MCP gateway can check tool calls. Access to a server does not itself approve a business action.
  • Budget alerts notify people. Enforced limits block or hold calls; concurrent requests need budget coordination.
  • Governance does not make an agent accurate. Actions that bypass central controls need separate attention.

Where the term comes from

The term comes from networking. There, the control plane works out routes and policy, and the data plane moves the packets. The split carries over to AI agents well.

The work itself happens in the data plane: a call to a model, a tool call over MCP, a handoff between agents, an update to a record. The control plane sits beside that path and answers questions about it. Who is this agent? Is this call allowed? Does it need approval? What gets recorded?

The practical benefit is that policy sits outside each agent's code. Your platform team can change a permission, a budget or an approval rule without editing every agent, as long as the enforcement points pick up the change.

What a control plane decides

Most of what a control plane decides about an agent action falls under five control points:

To make those decisions the control plane needs reference data: a register of approved use cases and agents with their owners, the policies themselves, budgets and the approval rules. The AI agent governance explainer covers how teams set those rules.

A register is an inventory of what you approved. It is not discovery. Agents, API keys and MCP servers that nobody registered, sometimes called shadow AI, come under policy only once someone finds them, for example by reviewing identity provider grants, network and gateway logs, and spend on AI services. Plan that search as its own piece of work.

Deciding and enforcing are separate jobs

This distinction matters most when you are designing or buying. A control plane can hold a careful policy and still change nothing, because a decision only has effect where a call can actually be stopped.

Access-control architecture uses two terms for this, for example in NIST SP 800-207 on zero trust architecture. A policy decision point answers "is this allowed?" A policy enforcement point sits in the path of the request and acts on the answer by forwarding, holding or blocking it. We borrow the terms here to explain the idea. The design below makes no claim to conform to that publication.

Figure 1 places enforcement in systems you already run. A gateway may also make its own policy decisions; the diagram separates the roles to make them easier to inspect.

Control plane: decides
RegisterAgents, use cases, owners
Policy and approvalsAllowed actions, limits, approvers, timeouts
RecordOne correlated entry per action
Asks "is this allowed?", or applies policy distributed in advance, and gets allow, hold or deny
Enforcement points: act in the request path
Model gatewayWhich models, keys, spend
Tool or MCP gatewayWhich tools, with which arguments
Credential checksValidate the per-agent tokens an identity provider issued
Runtime egressWhich destinations the agent can reach
Target systemIts own permissions on the record
Kill switchRevoke one agent's access
In this design, every governed call crosses at least one enforcement point
Data plane: agents do the work
Agent runtimePlans steps, calls tools
ModelsInference calls
Business systemsERP, CRM, ticketing, payments
Bypass riskAgent + direct credential→Target systemOnly the target system’s own permissions apply to this path. Central approval rules are bypassed.
Figure 1. One reference design: where decisions are made and where they take effect. The control plane (teal) decides. Enforcement points (coral) act on each call. Products may combine these roles. A call that reaches a business system without crossing an enforcement point is governed only by that system's own permissions.
Figure 1 as text

One reference design, in three layers from top to bottom. Products may combine these roles.

  1. Control plane, which decides: a register of agents, use cases and owners; policy and approval rules covering allowed actions, limits, approvers and timeouts; and a record holding one correlated entry per action.
  2. Enforcement points, which ask the control plane whether a call is allowed, or apply policy it distributed in advance, and then forward, hold or block the call: the model gateway, the tool or MCP gateway, credential checks on the per-agent tokens an identity provider issued, runtime network egress, the target system's own permissions, and a kill switch that revokes one agent's access.
  3. Data plane, where the work happens: the agent runtime, the models and the business systems.

In this design, every governed call crosses at least one enforcement point. A call that does not is governed only by the target system's own permissions.

Three design consequences follow.

Remove the bypasses. If an agent holds its own API key to the payments system, it can skip the tool gateway. The payments system's own permissions for that key still apply, but your central rules on approvals, limits and records do not. Credentials should come from the identity provider, be short-lived, and be accepted only on the enforced path.

Decide how fresh policy must be. Not every call needs a live round trip to the control plane. An enforcement point can cache policy or receive it in advance, which cuts latency and keeps working during a short outage. The trade-off is staleness. Set how quickly a policy change or a revocation must reach each enforcement point, and check that it does.

Decide what happens when the control plane is unreachable. An enforcement point either blocks calls (fail closed) or lets them through (fail open), possibly on cached policy. Choose per action type before your first outage. Letting low-risk reads through may be acceptable. For a payment change, blocking is usually the safer default.

Gateway, agent framework and control plane

These three get compared often. In the reference design they do different jobs, though one product can cover more than one of them.

One request, different responsibilities

Follow the call, then ask who controls it

P
AI control planeShared policy, approvals and correlated records
Supplies the rule
Policy decision or distributed policy ↓
Agent frameworkPlan → choose tool → handle resultBuilds and runs the agent's steps
→
AI / MCP gatewayRoute and check the callSees traffic passing through it
→
Target systemApply its own permissionsReturn the operation's result
Call, decision and result → shared record
Products can combine these roles. A gateway sees its own path; a control plane depends on its enforcement points. Neither establishes output quality.
Figure 2. Framework, gateway and control-plane responsibilities on one illustrative request. This is a reference design, not a claim that every product separates these roles.
Detailed layer comparison

What each layer is for. Products vary, so check what a specific product does.

Swipe or scroll horizontally to see all columns.

LayerMain jobWhat it can seeWhere it stops
AI or MCP gatewayRoute and limit one kind of traffic: model calls or tool calls.Requests passing through it.Traffic on other paths, and the wider context of the run.
Agent frameworkBuild and run agents: planning, memory, tool wiring.The agent's own steps.Rules written in agent code are set per agent and change when the code changes, so they are hard to keep consistent across agents.
AI control planeHold shared policy, approvals and records across agents.Calls reported by its enforcement points.Calls that do not pass an enforcement point, and the quality of agent output.

In this design a gateway is one of the enforcement points. The control plane is what lets gateways, runtimes and target systems apply the same rule and write to the same record. To see the three layers side by side on a single request, read AI gateway, agent framework or control plane: which layer does your team need?

LLMOps covers operating LLM applications, including prompt and model versions, evaluations and monitoring. A control plane focuses here on permission and enforcement at the point of action. These responsibilities can overlap in a product. When evaluating a tool, check what it actually enforces, not just what category it belongs to. Evaluation results can also inform an agent's limits.

Where an MCP gateway fits

An MCP gateway is a proxy between agents and the MCP servers that expose tools. It is an architecture choice, not a required part of the protocol, and what it checks depends on how it is built. It earns its place once several agents share several MCP servers and you want one path where servers are registered, tool rules apply and calls are recorded.

Two questions are easy to merge here. In the MCP authorization specification (version 2026-07-28), authorization is an optional part of the protocol, defined for HTTP-based transports. Where it is used, a server accepts only tokens issued for it as the intended audience, and it does not pass a client's token through to the services behind it. Those rules govern access to a server. They do not settle whether this agent may change a supplier's bank details today without a manager. That is business-action approval, and it comes from your policy.

Also read: Taking MCP to production: tool design, identity, gateways and governance, for gateway hosting options and identity choices.

  1. Agent
    Calls a tool through the gatewayPresents its own short-lived token, issued for the gateway.
  2. MCP gateway
    Checks the callerValidates the token and identifies the agent. This is access, not yet approval.
  3. Control plane
    Decides allow, hold or denyAgainst policy for this agent, tool and arguments. Deny returns an error. Hold waits for a person, as in Figure 4.
  4. MCP gateway
    Forwards only on allowUses a credential issued for that MCP server rather than passing the agent's token through.
  5. MCP server
    Runs the toolValidates that the credential was issued for it.
  6. Target system
    Applies its own permissionsThe record still has its own access rules.
  7. Record
    Logs the callAllowed, held and denied attempts share one run ID with their decisions and any execution results.
Figure 3. Illustrative: one MCP tool call through a gateway. Teal is the policy decision. Coral steps act in the request path, and forwarding a call is not the same as deciding it. Step 4 applies the specification's no-pass-through rule at the gateway; how a gateway does that is a design choice. Products differ in where these checks sit.
Figure 3 as text

Illustrative sequence for one MCP tool call through a gateway. Products differ.

  1. The agent calls a tool through the gateway, presenting its own short-lived token issued for the gateway.
  2. The MCP gateway validates the token and identifies the agent. This establishes access, not approval.
  3. The control plane decides allow, hold or deny against policy for this agent, tool and arguments. Deny returns an error to the agent. Hold waits for a person, as in Figure 4.
  4. On allow only, the gateway forwards the call using a credential issued for that MCP server, rather than passing the agent's token through.
  5. The MCP server validates that the credential was issued for it and runs the tool.
  6. The target system applies its own permissions to the record.
  7. The record logs allowed, held and denied attempts under one run ID, including the policy version and any execution result. Held or denied attempts are recorded even when no downstream call runs.

A worked approval, step by step

Illustrative example

A supplier-maintenance agent receives a request to change a supplier's bank account. Policy says the agent may prepare this change and may execute it only after an accounts payable manager approves it, and approvals expire after a set time. The steps follow the reference design above. No real system or customer is described.

  1. Agent
    Calls the update toolUses its own short-lived token. The request carries the person who opened the case.
  2. Tool gateway
    Asks the control plane before forwardingSends the agent, the tool, the arguments and the initiator.
  3. Control plane
    Decides: hold for approvalThe rule for bank-detail changes needs a manager. The attempt is recorded as held.
  4. Approver
    Reviews the exact changeSees who asked, the supplier's letter, the callback check and the old and new account, partly masked.
  5. Control plane
    Binds the approval to these valuesThe approval covers this supplier and this account only, and it expires.
  6. Tool gateway
    Forwards the matching callRechecks that the arguments match the approval. The ERP applies its own permission check too.
  7. Record
    Links the whole runRequest, decision, approval, result and policy version share one run ID.
Figure 4. One approval, traced through the control plane and two enforcement points. Coral steps happen in the request path. Teal steps are decisions.
Figure 4 as text
  1. The agent calls the bank-detail update tool with its own short-lived token, carrying the identity of the person who opened the case.
  2. The tool gateway asks the control plane before forwarding, sending the agent, tool, arguments and initiator.
  3. The control plane decides to hold the call for approval, because bank-detail changes need a manager. The attempt is recorded as held.
  4. The approver reviews the exact change: who asked, the supplier's letter, the callback check, and the old and new account, partly masked.
  5. The control plane binds the approval to this supplier and this account only, with an expiry.
  6. The tool gateway forwards the call only if its arguments match the approval. The ERP applies its own permission check as well.
  7. The record links the request, decision, approval, result and policy version under one run ID.

Now change one detail. If the agent retries with a different account number, step 6 fails because the values no longer match, and a new approval is needed. If the manager does not respond in time, the call stays held or goes to a backup approver, as the approval timeout section of the governance explainer describes.

Notice where enforcement happened: at the tool gateway, twice, and again inside the ERP. The control plane never touched the payment system. It supplied the decision and kept the record, which is the kind of decision record worth retaining when an agent supports a decision.

Cost and budgets

Cost is a control question as well as a finance one. One model call looks cheap, but a task can take several calls, retries, paid tool or API calls, and a person's review before it is done. To see where money goes, tag each call with its run, its task, the agent and the team that owns it, so usage rolls up per task and per team. Then compare what a run was expected to cost with what it actually cost, failed attempts included.

Two controls are easy to confuse. A budget alert warns someone when actual or forecast spend reaches a threshold; the alert itself does not stop a call. An enforced limit is checked before a call runs and blocks or holds it. Even an enforced limit can overshoot: calls already in flight, or many running at the same moment, can each pass the check before any of them is counted. Decide how much overshoot you can accept, and whether budget should be reserved before a call starts rather than counted after it finishes.

Where the data goes

Deployment questions come up early, and the useful answer is a trace rather than a label. Follow one run and write down where each piece goes: the prompt and any retrieved context, the tool call payloads, the logs and traces, and the credentials. Running the control plane inside your own environment does not by itself mean nothing leaves it. A call to a hosted model, a telemetry exporter or an MCP server outside your network still sends data out unless egress rules stop it. For how the layer sits in the wider stack, see where an AI control plane fits in enterprise architecture.

What a control plane does not do

  • It does not make the agent right. A fully governed agent can still extract the wrong total from an invoice. Output quality needs its own evaluation.
  • It cannot see what goes around it. An agent with a direct credential, or one nobody registered, sits outside the central policy until you find it. Only the target system's own permissions still apply.
  • It cannot fix an over-permissioned backend. If the service behind a tool grants broad access, the control plane narrows what the agent requests. The grant itself still needs fixing.
  • It adds a dependency to governed calls. A live check adds latency. Cached policy adds a freshness question. Either way your team owns the fail-open or fail-closed decision.
  • It does not interpret regulation for you. It can produce records. Deciding what a regulation requires stays with your risk, compliance and legal teams.
Explainer · 1 min 36 s

What happens when an AI agent fails?

Ravi Rajappa, AI Engineer, Moring

A ticket gets created and then the call times out. Ravi walks through why a timeout is different from a failure, and why recovery paths, retry limits and a named case owner matter. These are concerns the control plane records but does not resolve for you.

↗

Next steps

To evaluate a specific implementation, ask whether it can prevent an unauthorized action, attribute the attempt to the right identities, and reconstruct what happened. The detailed control-plane evaluation guide takes those questions further. Its term Replay means reconstructing a run's history, not executing the actions again.

Sources