Testing and Evaluating AI Agents in .NET

“How do you test something that gives a different answer every time?” is the first objection every engineer raises about shipping agents. It’s a fair worry, but it hides a false assumption: that an agent is one untestable blob. It isn’t. An agent is a deterministic shell (your tools, your routing, your guardrails) wrapped around a non-deterministic core (the model). You test each with the right tool. This article lays out a practical testing and evaluation strategy for .NET agents.

If you’re building agents with tools, this pairs with our tool-using agent tutorial.

Layer 1: Unit-test your tools (the easy win)

Your tools are ordinary C# methods. They have no LLM in them. Test them like any other code — this is where most of your real logic lives, and it’s fully deterministic:

[Fact]
public void GetOrderStatus_UnknownId_ReturnsNotFound()
{
    var result = OrderTools.GetOrderStatus("does-not-exist");
    Assert.Contains("No order found", result);
}

If your tools hit a database or API, mock those dependencies as usual. The point: the moment you push logic out of the prompt and into well-tested tools, most of your agent’s behavior becomes conventional, verifiable code. That’s a design goal, not just a testing one.

Layer 2: Test agent wiring by mocking the model

To test the agent’s plumbing — does it call the right tool, does it handle a tool error, does it respect a guardrail — you don’t want a real, paid, flaky model call. Swap in a fake IChatClient that returns a scripted response:

// A stub chat client that always "decides" to call GetOrderStatus
var fakeClient = new StubChatClient(response: ToolCall("GetOrderStatus", new { orderId = "ORD-1" }));

var agent = fakeClient.CreateAIAgent(
    name: "Test",
    instructions: "…",
    tools: [AIFunctionFactory.Create(OrderTools.GetOrderStatus)]);

var reply = await agent.RunAsync("Where is ORD-1?");
Assert.Contains("shipped", reply.ToString());

Because the framework builds on Microsoft.Extensions.AI’s IChatClient abstraction, substituting a deterministic fake is straightforward. These tests are fast, free, and run in CI — they verify your code around the model without depending on the model’s mood.

Layer 3: Evaluate the model’s actual output

The two layers above never call a real model, so they can’t tell you whether the agent’s answers are any good. That needs evaluation — a different discipline from unit testing. Instead of asserting exact strings (which non-determinism makes impossible), you score outputs against criteria over a fixed set of test cases.

Build an eval set: a collection of representative inputs with expected properties.

record EvalCase(string Input, string[] MustContain, string[] MustNotContain);

var cases = new[]
{
    new EvalCase("Where is ORD-4821?", ["ORD-4821"], ["I don't know"]),
    new EvalCase("What's your refund policy?", ["30 days"], []),
};

Then run each case through the real agent and check the properties. For fuzzy judgments (“is this answer helpful and on-topic?”), a common technique is LLM-as-judge: a second model call scores the first’s output against a rubric. The .NET ecosystem has tooling for exactly this — the Microsoft.Extensions.AI.Evaluation libraries provide evaluators for relevance, coherence, groundedness, and safety that you can run over your eval set and assert on aggregate scores.

What to actually measure

Don’t try to measure “correctness” in the abstract. Measure the things that break trust in production:

• Groundedness — does the answer stick to the tool/retrieved data, or does it invent facts?
• Tool accuracy — did it call the right tool with the right arguments?
• Refusal behavior — does it decline out-of-scope or unsafe requests?
• Format compliance — if you need JSON, is it valid JSON every time?

Track these scores over time. The real value of an eval set isn’t a one-time grade — it’s a regression net. When you change a prompt, swap a model, or upgrade the framework, re-run the evals and catch quality drops before your users do.

Where each layer runs

• Layers 1 & 2 (unit tests, mocked model) run on every commit in CI — fast and deterministic.
• Layer 3 (real-model evals) is slower and costs tokens, so run it on a schedule or before releases, not on every push. Treat a drop in eval scores like a failing test: block the release.

Note: exact type names in the evaluation libraries evolve. Verify against the current .NET AI documentation; the strategy — deterministic tests for your code, eval sets for the model’s output, tracked as a regression net — is stable regardless of API churn.

Takeaway

Non-determinism doesn’t make agents untestable; it means you test them in layers. Push logic into tools and unit-test them. Mock the model to test your wiring in CI. And evaluate the real model’s output against an eval set that doubles as a regression net. Do all three and “how do you test this?” stops being a blocker and becomes a checklist.

---

Have a correction or a topic you want covered? Email mani.bc72@gmail.com.