Observability for AI Agents: Metrics, Traces, and Guardrails That Matter

TL;DR

• Agent observability needs to show whether a task succeeded, what it cost and which actions ran or were refused. A fast response can still be a wrong result.
• Connect outcome and cost metrics to traces of model calls, tool calls and policy checks. Capture sensitive content only where your access and retention rules allow it.
• This record supports incident investigation and governance evidence. Logging alone does not establish regulatory compliance.

An agent that burns $4,000 in a single malformed retry loop, hallucinates a customer’s financial data into a support ticket, or invokes a deprecated API seventeen times before timing out is not having a bad day. It is failing in ways your existing monitoring cannot see. Classic APM tells you the request completed in 340ms. It does not tell you the agent made three wrong decisions, violated two policies, and cost more than your entire API tier that hour.

Production AI agents require observability built for decisions, not functions. Three layers write to one shared record. Metrics answer whether the agent is working. Traces explain why it made each decision. Runtime guardrails define what it is allowed to do. Together, these layers form the audit trail regulators and auditors demand, converting operational telemetry into governance evidence for your team to assess against its controls and obligations.

Why isn’t classic APM enough for AI agents?

Application performance monitoring (APM) was built for deterministic code where the unit of work is a function call and the error budget is latency. AI agents break that model. The unit of work is a decision. A request succeeds when the agent answers the question correctly, not when the API returns 200. Failure modes multiply: a NeurIPS 2025 study of 1,600 traces across seven open-source multi-agent frameworks found failure rates between 41% and 86.7%, spanning 14 distinct modes including inter-agent misalignment, task verification breakdowns, and system design flaws. Classic APM records that a span completed. It does not record whether the agent hallucinated, exceeded its budget, or invoked the wrong tool three times before giving up.

The agent runtime is also orders of magnitude more expensive than the average microservice. Token overhead when tools attach ranges from 3.25x to 236.5x in benchmarked scenarios. A single malformed prompt can burn thousands of tokens before the first useful output. APM dashboards show throughput and error rate. They do not show cost per decision or tool-use efficiency, the metrics that determine whether an agent deployment survives its first invoice.

Which metrics actually matter in production?

Four groups define agent health. Outcome metrics measure task success: completion rate, answer correctness, user satisfaction scores. These answer the question every VP asks first: is it working? Cost metrics track spend per decision, token consumption, and tool invocation frequency. Performance metrics capture latency, queue depth, and retry count. Tool behavior metrics log tool selection accuracy, parameter validation failures, and the frequency of fallback paths.

Track these four:

• Outcome: task completion rate, answer correctness (human-labeled sample), user satisfaction score.
• Cost: spend per decision, total token count, tool invocation frequency.
• Performance: end-to-end latency (p50, p95, p99), queue depth, retry count.
• Tool behavior: tool selection accuracy, parameter validation failure rate, fallback invocation frequency.

OpenTelemetry GenAI semantic conventions define gen_ai.client.operation.duration as a required metric and specify standard dimensions for model, operation type, and error code. These conventions create a shared vocabulary. When every agent emits gen_ai.client.operation.duration tagged with gen_ai.operation.name = "invoke_agent", platform teams can aggregate cost and latency across vendors without writing custom parsers.

What should a trace record when the unit of work is a decision?

A trace should capture the observable decision sequence, including available explanations and policy results. It cannot reliably expose a model’s hidden reasoning. Start with the user prompt. Record each tool invocation as a distinct span with input parameters, output, and any available explanation for the call, clearly distinguished from a faithful account of internal model reasoning. Capture the final response and the total token count. OpenTelemetry GenAI conventions define three span types: invoke_agent, chat, and execute_tool. Each span carries attributes for model name, token counts, and finish reason. Content capture is off by default to avoid logging sensitive data, but teams must enable it selectively to debug hallucinations and misaligned outputs.

The trace also records what did not happen. When your orchestration or policy layer rejects a candidate tool, log the candidate and the rejection reason. When a retry loop exhausts its budget, log the attempt count and the final error. The MCPGAUGE framework found that LLMs using Model Context Protocol exhibit token overhead between 3.25x and 236.5x depending on tool complexity. Without per-decision traces, that overhead is invisible until the invoice arrives.

Which guardrails belong in production and where do they sit?

Runtime guardrails enforce policy at decision time. They sit between the agent and the action it wants to take. Three categories matter. Input guardrails validate the prompt before it reaches the model, rejecting malformed requests and filtering prohibited content. Output guardrails inspect the agent’s response before delivery, blocking hallucinated citations and policy violations. Tool guardrails gate each tool invocation, enforcing parameter schemas, rate limits, and access control rules.

Guardrails must write their verdicts into the same trace as the decision they govern. When a guardrail blocks a tool call, that block becomes a span in the trace with attributes for the policy name, the blocked action, and the reason. This creates a complete record. An auditor reading the trace sees not just what the agent did, but what it tried to do and why it was stopped.

How does observability become governance evidence?

Structured telemetry can support an audit trail. AI Act Article 12 requires high-risk AI systems to allow automatic event recording. Article 19 addresses providers’ retention, while Article 26 addresses deployers’ duties. Where those provisions apply, logs under their control generally need to be kept for an appropriate period of at least six months, unless other applicable law provides otherwise. A trace is useful evidence, but its completeness, access controls and retention still need to match your system’s obligations. It does not prove compliance on its own.

Collecting a trace and answering a governance question are different jobs. Your implementation must connect operational events to the relevant identities, policy versions and retention rules. That lets your team investigate which decisions involved personal data, which agents exceeded a budget, which guardrails fired and whether the required events were retained.

This is the Measure, Explain, Contain framework. Metrics measure whether the agent is working. Traces explain why it made each decision. Guardrails contain what it is allowed to do. All three write to one shared record that serves operations and compliance from the same source of truth. When an auditor asks how you prevented an agent from accessing customer financial records, you point to the trace showing the tool guardrail blocked the call, logged the policy violation, and triggered the fallback path.

Key takeaway

AI observability for production agents requires metrics, traces, and runtime guardrails writing to one unified record. Metrics answer whether it works. Traces explain why it decided. Guardrails enforce what it may do. Together, they convert operational telemetry into governance evidence.

The three layers are interdependent. Metrics without traces show that something failed but not why. Traces without guardrail verdicts explain the decision but not the policy boundary. Guardrails without metrics prove enforcement but not impact. Only the unified record answers the questions that matter: Did the agent complete the task? What did it cost? Which policies fired? Where did the reasoning break down?

The Moring AI Ops solution connects runtime controls and evidence for operations workflows. The required instrumentation and evidence mappings depend on your environment. The question is not whether your agents produce telemetry. They already do. The question is whether that telemetry can answer an auditor’s questions six months from now.

Score your agent estate against the Moring AI Ops maturity checklist at moring.ai/contact.

Sources and further reading

• Why Do Multi-Agent LLM Systems Fail?. Research study of multi-agent failure modes; its measurements describe the evaluated systems and tasks.
• Help or Hurdle? Rethinking Model Context Protocol-Augmented Large Language Models. Research introducing MCPGAUGE and evaluating tool-use performance and token overhead.
• OpenTelemetry: semantic conventions for generative AI. Public technical specification; individual conventions have their own stability status.
• European Commission: AI Act Article 12, record-keeping, Article 19, provider log retention, and Article 26, deployer obligations. Official reference text; classification and applicability require a system-specific assessment.

Editorial note: Measure, Explain, Contain is this article’s framework for connecting operational telemetry to governance evidence. It is not a regulatory certification or a claim that logging alone establishes compliance.

Frequently asked questions

About the author

Ashvath NarayanAI Engineer, Moring AIChennai

Builds the platform’s agentic primitives: the model gateway, retrieval pipelines and the observability that traces every call. Works close to the runtime where policy is enforced. Writes about gateway, framework and control-plane boundaries in practice.

LinkedIn profile →