Run Your Whole AI Agent Stack Locally with Aspire (Ollama + Qdrant + Traces)

Getting a .NET AI agent running on a new machine usually looks like this: install Ollama, pull a model, run a Qdrant container, copy an appsettings.Development.json from a colleague, discover the ports collide, and then find out someone’s API key expired. Multiply by every new joiner.

Aspire collapses that into F5. It is not an AI framework — it is orchestration and observability for the supporting cast around your agent, and that turns out to be exactly the painful part.

What you get

An AppHost project describes your local topology in C#, and Aspire starts everything, wires the connection strings, and gives you a dashboard with logs, traces and metrics across all of it.

For an agent, that topology is usually:

• an LLM (Ollama locally, a hosted model in production)
• a vector store for RAG
• your API and maybe a worker
• optionally a cache and a relational database

Two properties make this worth setting up rather than clever-but-optional. First, the AppHost is C#, so it refactors, autocompletes and fails at compile time — none of which a docker-compose.yml does. Second, the dashboard shows model calls on the same trace timeline as your HTTP and database spans, which is how you discover that the “slow LLM” was actually three sequential retrieval round-trips.

The AppHost

Add the integrations you need. Aspire ships a CLI for this:

aspire add qdrant

Then describe the stack:

var builder = DistributedApplication.CreateBuilder(args);

// Local inference. Aspire runs the Ollama container and pulls the model on startup.
var ollama = builder.AddOllama("ollama")
    .WithDataVolume();                      // keep pulled models across restarts

var chatModel = ollama.AddModel("llama3");

// Vector store for retrieval.
var qdrant = builder.AddQdrant("qdrant")
    .WithLifetime(ContainerLifetime.Persistent);   // Qdrant is slow to start; keep it warm

builder.AddProject<Projects.AgentApi>("agent-api")
    .WithReference(chatModel)
    .WithReference(qdrant)
    .WaitFor(qdrant);

builder.Build().Run();

That is the entire local environment. No compose file, no port allocation, no README section titled “Local setup (please read carefully)”.

A few details that save time:

• WithDataVolume() on Ollama is not optional in practice. Without it, every restart re-downloads the model. That is several gigabytes and a lot of goodwill.
• WithLifetime(ContainerLifetime.Persistent) on Qdrant keeps the container alive between debug sessions. Qdrant’s startup is slow enough that you notice it every single time otherwise.
• WaitFor(qdrant) stops your API racing the database on startup — the classic “works on the second run” bug.
• GPU: .WithGPUSupport() for Nvidia, or .WithGPUSupport(OllamaGpuVendor.AMD). On a laptop without one, expect small models to be slow but usable for wiring work.

The consuming project

The model resource name is derived from the model you added: AddModel("llama3") under a server named "ollama" produces the resource ollama-llama3. That derived name is the connection name you use.

Worth knowing before it confuses you: the naming rules strip the tag suffix, take the last path segment of a slash-separated name, and replace dots with hyphens — so phi3.5 becomes ollama-phi3-5. If a connection name is not resolving, that transformation is usually why.

var builder = WebApplication.CreateBuilder(args);
builder.AddServiceDefaults();

// Registers IChatClient over the Ollama model resource.
builder.AddOllamaApiClient("ollama-llama3")
       .AddChatClient();

// Embeddings for retrieval — same resource, different abstraction.
builder.AddOllamaApiClient("ollama-llama3")
       .AddEmbeddingGenerator();

Need more than one model — a small fast one for classification, a larger one for answers? Register them keyed:

builder.AddKeyedOllamaApiClient(connectionName: "chat");
builder.AddKeyedOllamaApiClient(connectionName: "embeddings");

and resolve with [FromKeyedServices("chat")].

The important thing about all of this: what lands in DI is IChatClient and IEmbeddingGenerator<string, Embedding<float>> from Microsoft.Extensions.AI — the same interfaces Azure OpenAI, OpenAI and Anthropic connectors register. Nothing downstream knows or cares that a container is doing the inference. See Microsoft.Extensions.AI explained for why that abstraction is the load-bearing piece.

Your agent code does not change

Because the abstraction is the same, the agent is written once:

public sealed class SupportAgent(IChatClient chat)
{
    private readonly AIAgent _agent = chat.AsAIAgent(
        instructions: "You answer support questions from the provided context only.",
        tools: [AIFunctionFactory.Create(LookupOrder)]);

    public async Task<string> AskAsync(string question, AgentSession session)
    {
        AgentResponse response = await _agent.RunAsync(question, session);
        return response.Text;
    }
}

Local Ollama in development, Azure OpenAI in production, and this class is untouched. That is the payoff.

One caveat, stated plainly: a small local model is a way to exercise your plumbing — tool schemas, retrieval, streaming, error paths. It is not a proxy for production answer quality. Run your evaluation set against the model you will actually deploy before you believe any quality number.

And check tool support before you commit to a local model. Not every small model handles function calling reliably, and an agent whose tools silently never fire is a confusing afternoon.

The part that pays for itself: traces

AddServiceDefaults() wires OpenTelemetry across the whole app. Add instrumentation to the chat client and every model call becomes a span:

builder.Services.AddChatClient(sp =>
    sp.GetRequiredService<IChatClient>()
      .AsBuilder()
      .UseOpenTelemetry(configure: o => o.EnableSensitiveData = true)
      .UseLogging()
      .Build());

EnableSensitiveData records prompts and completions in the trace — development only, obviously; it puts user input into your telemetry. But during development it answers the question you actually have, which is not “was the model slow” but “what exactly did the model receive?” Nine times out of ten the answer explains the bad output: the retrieved chunk was wrong, or the tool result got truncated, or the system prompt lost a line.

In the dashboard those spans sit on the same timeline as your HTTP requests and Qdrant queries. Nesting makes the shape of a request obvious — and a RAG request that looked like one slow LLM call frequently turns out to be five embedding calls in a loop. For the production version of this, see observability for AI agents with OpenTelemetry.

Where Aspire stops

Be clear about the boundary. Aspire is local orchestration and dev-time observability. It is not a production runtime, not an agent framework, and not a replacement for your deployment story — you still ship to Container Apps, GKE or wherever you were already going.

Which is fine, because the problem it solves is real and specific: the distance between “clone the repo” and “the agent answers a question.” On an AI project that distance is unusually long — more moving parts than a typical CRUD service, and several of them are containers with awkward startup behaviour. Closing it to one keypress is worth an afternoon.

Takeaway

Aspire does not make your agent smarter. It makes the ten things around your agent start together, wire themselves up, and report what they are doing on one timeline. Describe Ollama and Qdrant in the AppHost, consume them as IChatClient and IEmbeddingGenerator so the same code runs against a hosted model in production, and turn on OpenTelemetry so you can see the prompt that actually reached the model. Remember the two boundaries: a small local model validates plumbing, not answer quality, and Aspire is a development-time tool — production deployment is still your own.

---

Have a correction or a topic you want covered? Email mani.bc72@gmail.com.