Observability for AI Agents with OpenTelemetry in .NET

An agent gave a user a wrong answer. Why? With plain logs, you’re guessing: did it pick the wrong tool, get bad data back, or just reason poorly? An agent is a chain of decisions, and to debug it you need to see the chain. That’s what observability gives you — and for .NET agents, OpenTelemetry is the way to get it. This article instruments an agent so every run becomes a trace you can inspect.

It complements the testing guide: tests catch problems before release, observability explains them after.

Why traces, not just logs

Logs are flat lines of text. A trace is a tree of timed spans that mirrors what actually happened: the request span contains a model-call span, which contains tool-call spans, each with attributes (which tool, what arguments, how long, success or failure). For an agent, that structure is exactly the shape of the problem you’re debugging.

The good news: the .NET AI stack emits this telemetry using standard OpenTelemetry conventions, so you mostly enable it rather than hand-instrument it.

Wiring up OpenTelemetry

Add the OpenTelemetry packages and configure tracing in your agent host. Subscribe to the activity sources the AI libraries publish, and export to your backend:

using OpenTelemetry.Trace;

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddSource("Microsoft.Extensions.AI")   // model + agent activity
        .AddSource("Experimental.Microsoft.Extensions.AI")
        .AddAspNetCoreInstrumentation()          // the incoming request
        .AddHttpClientInstrumentation()          // outbound model/API calls
        .AddOtlpExporter());                     // send to your backend

Point the OTLP exporter at whatever you use — Azure Monitor, Jaeger, Grafana Tempo, Honeycomb. The value is the same everywhere: one trace per agent run, drillable down to each step.

Capturing the AI-specific detail

Generic HTTP traces tell you a model call happened; they don’t tell you what it did. The AI instrumentation adds semantic attributes that matter for agents — the model name, token counts (prompt and completion), tool/function names invoked, and finish reasons. Some of these (like full prompt/response content) are gated behind an opt-in switch because they can contain sensitive data:

// Opt in to richer AI telemetry — but mind what ends up in your traces.
AppContext.SetSwitch("Microsoft.Extensions.AI.EnableSensitiveData", true);

Enable content capture in development to debug prompts, but in production weigh it carefully: prompts and responses may contain user data, so treat traces with the same care as logs — and never let secrets land in them.

What to actually watch

Instrumentation is only useful if you look at the right signals. For agents, track:

• Tool-call accuracy and failures — which tools run, how often they error. A spike in a tool’s failures usually explains a wave of bad answers.
• Token usage per run — the single best cost signal, and an early warning when prompts bloat. (See cost control.)
• Latency, broken down — is the slow part the model, a tool, or your own code? The trace tree answers this directly.
• Turn counts — an agent looping through many tool calls to answer a simple question is a red flag worth alerting on.

From traces to alerts

Once the telemetry flows, close the loop. Build a dashboard with error rate, p95 latency, and token spend, and set alerts on the things that hurt: tool-error rate climbing, latency crossing your SLO, or token-per-request jumping (often the first sign of a prompt regression or an agent stuck in a loop). That turns “a user complained” into “we were paged before they noticed.”

Note: exact activity-source names and telemetry switches evolve as the .NET AI libraries mature. Verify against the current .NET AI docs; the approach — enable OpenTelemetry, subscribe to the AI sources, export traces, alert on tool errors and token spend — is stable and vendor-neutral.

Takeaway

You can’t operate what you can’t see, and an agent’s failures live in the chain of decisions between request and response. OpenTelemetry turns that chain into a trace: enable it, subscribe to the AI activity sources, export to your backend, and watch tool accuracy, token usage, and latency. Do it before you have traffic, not after your first incident — because the first time an agent misbehaves in production, a good trace is the difference between a five-minute fix and an afternoon of guessing.

Have a correction or a topic you want covered? Email mani.bc72@gmail.com.