Key takeaways
- An AI gateway manages traffic to models and related APIs. An agent framework implements the agent’s reasoning and tool-use loop. An AI control plane manages policy, identity, authority and evidence across agents and environments.
- These layers can coexist. Choosing one does not remove the responsibilities of the others.
- Follow a real request end to end. The layer you need becomes clearer when you identify where routing, orchestration, authorization, approval and audit evidence happen.
- If a tool call can change a business system, model access controls are not enough. You also need to govern the action, the acting identity and the state against which it was approved.
An architecture diagram shows an application, a model, a few tools and a box in the middle. Depending on who drew it, that box may be called an AI gateway, an agent framework, an orchestration layer or an AI control plane.
Those labels hide different responsibilities.
A gateway sits on a request path and applies runtime rules. A framework gives developers the components used to construct an agent. A control plane holds the policies and operating state needed to govern agents across workflows and environments.
The boundaries are not standardized, and products often combine more than one layer. The useful question is therefore not which label is correct. It is which responsibility your team is missing.
This article follows one fictional request through all three layers. It is an architectural working model, not a claim that every product uses these exact boundaries.
The short version
Use this working distinction:
- AI gateway: manages traffic between applications and model or AI endpoints. Typical responsibilities include authentication, routing, rate limits, request and response policies, and usage telemetry.
- Agent framework: supplies the runtime components used to build an agent. Typical responsibilities include prompts, state, memory, planning, tool definitions, model calls and execution flow.
- AI control plane: manages policy and operating state across agents. Typical responsibilities include identity, permitted actions, approvals, configuration versions, deployment boundaries, evidence and fleet-level oversight.
The distinction resembles other software systems. API management policies run on requests and responses to authenticate, limit, transform and route traffic. Kubernetes describes its control plane as the components that make global decisions and respond to cluster events. The analogy is useful because it separates the runtime path from the layer managing desired state. It does not mean an AI control plane should copy Kubernetes architecture.
An agent framework lives elsewhere in the picture. It is the code and runtime that decide what the agent does next: call a model, retrieve context, invoke a tool, ask a person or stop.
What does an AI gateway actually do?
An AI gateway gives model traffic a controlled entry point. Instead of every application managing provider endpoints and request policies independently, applications call the gateway.
At that point, the gateway can perform functions such as:
- authenticate the calling application or workload;
- route a request to an allowed model endpoint;
- apply quotas and rate limits;
- attach or remove approved request fields;
- enforce payload-size or content rules;
- record latency, errors, token use and cost data;
- handle retries or fallbacks under defined conditions.
These are valuable controls. They give platform teams a consistent place to manage model access and observe traffic.
The limit appears when the model response causes another operation. A gateway can know that an application called a model. It may not know that the model response led an agent to create a supplier record, modify a ticket or query a production system. It may see a stream of model calls without the business meaning of the workflow.
That is why a gateway policy such as “this application can use model X” is different from an action policy such as “this procurement agent may propose a supplier-address change, but a reviewer must approve it before the system of record is updated.”
What does an agent framework do?
An agent framework helps a development team implement the agent loop. It may provide abstractions for model prompts, tool schemas, memory, state transitions, retrieval, multi-agent coordination and retries.
The Model Context Protocol architecture is useful for understanding one part of this runtime. It separates a host, clients and servers. Servers expose focused resources, prompts and tools, while the host manages coordination, permissions and user authorization decisions. An agent framework may implement or integrate with those components, but a protocol connection alone does not decide the enterprise policy for every operation.
The framework is where the agent decides what to do next. For example:
- Interpret the user’s request.
- Retrieve the supplier record.
- Compare the requested change with the available evidence.
- Draft an update.
- Request approval.
- Apply the approved operation.
- Report the outcome.
A framework can encode that flow. It can also expose hooks for human review and tracing. The remaining question is where the governing rules come from and how they stay consistent when many teams build agents.
If every agent hardcodes its own role checks, approval rules and evidence schema, the enterprise has an application-by-application control model. It may work for a pilot. It becomes difficult to review when the same identity or system appears in dozens of workflows.
What does an AI control plane do?
An AI control plane manages the operating policy around the agent rather than implementing every reasoning step inside it.
In this working model, it answers questions such as:
- Which agent version is running in this environment?
- Which identity does the agent use for each tool?
- Which operations are allowed, denied or subject to approval?
- Which person or role can approve a particular action?
- What evidence must be present before approval?
- Does the approved action still match the current system state?
- Which model, tool and data endpoints are available to the workflow?
- What record connects the request, tool calls, approvals and outcome?
The control plane does not need to replace the gateway or framework. It can supply policy and configuration to both, then retain the decision evidence that crosses them.
This is also why an AI control plane is more than a model proxy. The relevant unit is the governed agent action, not only the model request.
Follow one request through the three layers
Imagine a procurement user asks an agent to update the bank-account details on a supplier record. The example is fictional and intentionally consequential because it makes each responsibility visible.
Step one: establish the request and identity
The application authenticates the user and sends the task to the agent runtime. The control plane supplies the active agent configuration and the policy that applies to supplier-master changes.
The policy says the agent can retrieve the current record and prepare a proposed update. It cannot apply new bank details without a named reviewer from the supplier-operations team.
The framework creates the workflow state and begins the agent loop.
Step two: call the model
The framework prepares the context needed to interpret the request. The model call passes through the AI gateway.
The gateway verifies the calling workload, routes the request to an approved model endpoint, applies traffic policy and records request-level telemetry. If the selected endpoint is unavailable, it follows the configured fallback rule.
The gateway does not decide whether the supplier record may be changed. That decision belongs to the action policy.
Step three: retrieve evidence through tools
The framework calls tools to retrieve the current supplier record and the submitted evidence. If those tools are exposed through MCP, the host and server boundaries determine how capabilities are connected. The control plane determines whether this agent and user may invoke these particular read operations.
Each retrieval is recorded with the acting identity, target, parameters and result status. Sensitive values can remain masked in the general trace while remaining available to an authorized reviewer.
Step four: prepare an action
The agent compares the proposed details with the available evidence and prepares an exact update. The framework manages this reasoning and workflow state.
The control plane evaluates the proposed operation against policy. It checks the agent version, user context, supplier record, target fields and required evidence. The result is “approval required,” not a general permission to continue.
Step five: approve and recheck state
The reviewer sees the old values, proposed values, evidence and the agent’s explanation. Their approval is tied to that specific operation.
Before execution, the workflow retrieves the supplier record again. If another person has already changed it, the approved context no longer holds. The control plane returns the action for review instead of applying an approval to a different state.
Step six: execute and retain evidence
The framework invokes the update tool using the credential and scope authorized for the operation. The gateway may still mediate any model calls, but the business-system request follows its own protected path.
The record connects the initiating user, agent identity, policy decision, approver, tool request, target-system response and final state. That end-to-end evidence is the control-plane concern.
What breaks if you run AI agents without a gateway?
Without a shared gateway, each application may manage model credentials, endpoints, retries, limits and telemetry separately. That creates duplicated configuration and uneven controls. A provider change can require application-by-application work, and the platform team may lack a consistent view of model traffic.
You can still build and govern an agent without a dedicated AI gateway. The tradeoff is that gateway responsibilities must live somewhere else. A service mesh, API gateway, client library or platform service may already cover some of them.
The decision should therefore start with the gap, not the product category. If credential sprawl and model routing are the immediate problem, a gateway may be the right first layer. It will not by itself supply workflow approvals or tool-action evidence.
What breaks if you use a framework as the control plane?
A framework can implement policy inside one application. The problem appears when policy needs to span applications, business units and environments.
Several failure modes are common:
- two teams express the same approval rule differently;
- an agent version changes without a corresponding policy review;
- tool permissions are coupled to application code releases;
- evidence formats differ across frameworks;
- revoking an agent’s access requires changes in several repositories;
- reviewers cannot see one inventory of deployed agents and authority.
That does not make the framework the wrong tool. It means policy ownership and runtime construction are separate concerns. A framework remains responsible for building the agent. A control plane gives the organization one place to manage the rules that should not drift with every implementation.
Can one product cover more than one layer?
Yes. A platform may bundle a gateway, a framework and control-plane capabilities. Another architecture may combine a commercial gateway with an open-source framework and an internal policy service.
Evaluate the responsibilities individually even when one product supplies them. Ask:
- Can the gateway enforce traffic policy for every model endpoint you use?
- Can the framework express your workflow, including review and failure paths?
- Can the control plane manage identities and action policy across frameworks?
- Can evidence connect a model request to the resulting tool action?
- Can a policy change take effect without rebuilding every agent?
- Can you separate environments, business units and data boundaries?
An integrated platform can reduce operational seams. Ask which component inside it owns each responsibility, because one product name can cover all three layers while one of them goes unhandled.
Which layer should your team add first?
Use the failure you are trying to fix:
- Add or strengthen a gateway when model credentials, endpoint routing, quotas, traffic policy or request telemetry are fragmented.
- Add or standardize an agent framework when teams lack a reliable way to implement tool use, state, review steps and failure handling.
- Add a control plane when identity, authority, approvals, versioning and evidence need to remain consistent across agents and environments.
Many enterprise programs need all three, but not all at once. Start with one workflow and draw the actual request path. Name who authenticates the user, who routes the model call, who authorizes each tool operation, who approves a consequential action and who can reconstruct the outcome.
That map shows which layer is missing. If it is the control plane, Moring’s AI Control Plane manages identity, action policy, approvals and evidence across the frameworks and gateways you already run.
If you want to draw that map for one of your own workflows, the AI control plane workshop walks a team through it.
Frequently asked questions
How does an AI control plane differ from an AI gateway or agent framework?
An AI gateway applies runtime policy to model and API traffic. An agent framework implements the agent’s workflow and tool-use loop. An AI control plane manages identity, authority, policy, configuration and evidence across those runtimes. The layers can be separate or bundled.
What is the difference between MCP connectivity and an AI control plane?
MCP defines how a host connects to servers that expose resources, prompts and tools. A control plane determines which agents and identities may use those capabilities, under which policies, with which approvals and evidence. Connectivity makes a tool reachable. Governance decides whether a particular operation is allowed.
What is the difference between an AI platform and an AI control plane?
An AI platform is a broad product category that may include development, deployment, gateway, model-management and governance functions. A control plane is a specific architectural responsibility: managing desired operating state and policy across agents. A platform may contain a control plane, but the terms are not interchangeable.