Microsoft.Extensions.AI: The .NET AI Abstraction, Explained

If you’ve read other guides on this site, one type keeps appearing: IChatClient. It comes from Microsoft.Extensions.AI, and it’s arguably the most important building block in the .NET AI stack — the layer that stops you from coupling your app to a single AI vendor. This article explains what it is and why it matters.

The problem it solves

Without an abstraction, calling OpenAI, Azure OpenAI, Claude, Gemini, or a local model each means a different client, different method names, and vendor-specific types threaded through your code. Switching providers — or A/B testing two — becomes a rewrite. That’s the problem Microsoft.Extensions.AI removes.

The cost is easy to underestimate, because it isn’t the client construction — that’s five lines in one file. It’s the vendor’s message and response types spreading outward from it. A method returns a provider-specific chat completion; a caller reads a property off it; a view model is shaped around that property; a test asserts on it. Six months later the coupling isn’t in your AI code at all, it’s in your controllers and your tests, and “try a different model for a week” is a two-sprint project nobody schedules.

The two core interfaces

Microsoft.Extensions.AI defines provider-agnostic interfaces, the same way Microsoft.Extensions.Logging gave .NET one logging API over many providers:

• IChatClient — send messages, get responses (streaming or not), with tools and structured output. Every chat provider implements it.
• IEmbeddingGenerator — turn text into embedding vectors, provider-agnostically.

Your code depends on the interface; a provider package supplies the implementation.

That split shows up in the packaging too, and it’s worth understanding before you add a reference. Microsoft.Extensions.AI.Abstractions contains the interfaces and the core types — ChatMessage, ChatRole, ChatResponse, ChatOptions — and nothing else. Microsoft.Extensions.AI builds on it with the middleware pipeline and the concrete behaviours. The practical rule: a library you publish should reference only the abstractions package, so consumers aren’t forced to take the implementations along with it. An application references the full package, because the middleware is the point.

Getting an actual instance is the provider package’s job. Each one ships an adapter — an extension method that wraps its own client and hands you back an IChatClient — so you install the vendor package you want, call the adapter once during startup, and never mention the vendor type again.

What using it looks like

// Your code only knows about IChatClient — not which vendor is behind it.
IChatClient chat = /* injected: OpenAI, Azure, Claude, Gemini, or Ollama */;

var response = await chat.GetResponseAsync("Explain CQRS in two sentences.");
Console.WriteLine(response);

// Streaming works the same way, regardless of provider:
await foreach (var update in chat.GetStreamingResponseAsync("Now give an example."))
    Console.Write(update);

Swap the provider in one place (usually your DI registration) and everything downstream is unchanged.

Real calls carry more than a string. The full shape takes a list of ChatMessage values with roles, and a ChatOptions describing how you want the call to behave — the parameters that are common across providers get strongly typed properties there, so temperature and output limits and tool declarations survive a provider change unmodified:

List<ChatMessage> messages =
[
    new(ChatRole.System, "You are terse. Answer in one sentence."),
    new(ChatRole.User, "What is CQRS?")
];

ChatOptions options = new() { Temperature = 0 };

var response = await chat.GetResponseAsync(messages, options);

Why it’s a big deal

• No vendor lock-in. Switch models with a config change, not a refactor.
• Easy benchmarking. Run the same code against several providers and compare on your eval set (choosing a model).
• Testability. Substitute a fake IChatClient in unit tests — no real API calls, fast and free.
• Middleware. Because it’s an interface, you can wrap it: add caching, logging, telemetry, or rate-limiting as decorators around any provider.
• It’s the foundation. The Microsoft Agent Framework and most .NET AI tooling build on these interfaces — learn them once, use them everywhere.

Middleware in practice

The decorator pattern is where it shines. Want to cache responses, add OpenTelemetry observability, or enforce a rate limit? You wrap the IChatClient in a pipeline — the same idea as ASP.NET Core middleware — so those concerns are provider-independent and composable.

ChatClientBuilder is what assembles that pipeline:

IChatClient chat = new ChatClientBuilder(innerClient)
    .UseDistributedCache(cache)
    .UseFunctionInvocation()
    .UseOpenTelemetry()
    .Build();

The order is not cosmetic, and getting it wrong produces bugs that look like model failures. Consider the cache and function invocation above. Registered in that order, the cache sits outside the tool loop, so it caches the final answer of the entire exchange — including whatever your tools returned. Ask “what’s my current balance”, get an answer, ask again a minute later, and you get the cached number rather than the live one. The model looks like it’s hallucinating stale data; the model never ran.

Flip them and the cache sits inside, caching individual model round-trips instead. That’s safer, and it also hits far less often, because every turn of a tool loop carries a different message history and therefore a different cache key. Neither placement is universally right — the point is that it’s a decision with a visible consequence, not a line ordering you can shuffle.

Telemetry has the same property in a gentler form. Outermost, your spans measure what the user experienced, and a cache hit shows up as a two-millisecond call. Innermost, they measure the provider, and cache hits are invisible. If you’re diagnosing latency you probably want both, on different layers.

Function invocation, and what it’s actually doing

UseFunctionInvocation() is the layer that does the most work, and it’s worth knowing what it automates. Tool calling is a loop: you declare tools, the model replies asking for one, you execute it, you send the result back, and the model either answers or asks for another. Written by hand that’s a while loop, JSON schema generation for each tool, argument deserialisation, and result marshalling — per provider, in each provider’s wire format.

With the pipeline, you declare tools as ordinary methods and the layer runs the loop:

ChatOptions options = new()
{
    Tools = [AIFunctionFactory.Create(GetForecast)]
};

static string GetForecast(string city) => $"Sunny in {city}.";

Two failure modes are worth pre-empting, because both are cheap to prevent and expensive to discover in production. The loop can run longer than you expect — a model that keeps calling a tool that keeps returning something unhelpful will keep going, and every iteration is a full round-trip billed against the whole accumulated history. Cap the iterations. And your tool method now runs on model-chosen arguments, which makes it an untrusted entry point: validate inputs exactly as you would for a public API endpoint, because effectively that’s what it is.

Where the abstraction leaks

“Provider swap is a config change” is true for the common path and only the common path. Messages, streaming, tool calls, structured output: those port. What doesn’t port is everything a single vendor exposes and the others don’t — Claude’s prompt caching hints, Gemini’s safety settings, provider-specific reasoning configuration. Those reach the wire through an additional-properties bag on the options rather than through typed properties, and there’s an escape hatch to the underlying provider object for cases the interface can’t express at all.

Both are the right design; a lowest-common-denominator interface that refused to let you use your provider’s best features would just be an interface people route around. But they mean the abstraction’s guarantee is narrower than the marketing version. Once a call site sets a provider-specific property, that call site is coupled — silently, because it still compiles against IChatClient.

The fix is placement, not avoidance. Keep the provider-specific bits behind a small wrapper of your own that exposes the two or three behaviours you actually depend on. Then a provider change is a rewrite of one class you knew about, rather than a search for every place someone quietly reached past the interface.

Testing against a fake

Substituting a fake IChatClient is the most immediately useful benefit, and also the easiest to do badly. A fake that returns a fixed string for every call proves your dependency injection is wired up. It proves nothing about behaviour, because the interesting cases are precisely the ones a constant can’t represent.

The cases worth encoding in test doubles are the ones that break real systems: a response containing a tool call rather than text; a response that hit the output limit and stopped mid-sentence; a response whose JSON doesn’t match the schema you asked for; a provider error surfacing as an exception mid-stream. Capture a handful of real responses once, replay them in tests, and your suite starts catching the failures that actually happen. Keep the real-provider calls for a small eval set run on a schedule rather than on every commit — that’s a different question (“is the model still good enough”) answered by a different kind of test.

When you don’t need it

Three honest exceptions:

A throwaway script or spike. If the code won’t outlive the week, the indirection buys nothing.

A product built entirely around one provider’s exclusive capability. If most of your calls reach past the interface anyway, you’ve paid for an abstraction you aren’t getting. Owning the coupling openly is better than a layer that implies portability you don’t have.

When you’re already a level up. If you’re using the Microsoft Agent Framework or similar, IChatClient is underneath you already — you don’t need to work at both levels at once.

Note: method names (GetResponseAsync, GetStreamingResponseAsync) are still stabilizing; verify against the current .NET AI docs. The design — provider-agnostic interfaces with pluggable implementations and middleware — is the durable core of .NET’s AI story.

Takeaway

Microsoft.Extensions.AI is to AI providers what Microsoft.Extensions.Logging is to loggers: one interface, many implementations. Depend on IChatClient and IEmbeddingGenerator, and you get no vendor lock-in, trivial provider swapping and benchmarking, easy testing, and composable middleware for caching, telemetry, and rate-limiting. Two things decide whether it pays off in practice: assemble the pipeline deliberately, because layer order changes behaviour rather than just style, and keep provider-specific options behind one wrapper of your own so the leaks stay in a place you can find. It’s the layer that makes every other .NET AI decision reversible — which is exactly why it shows up in nearly every guide here.