Foundry IQ and Agent Memory: Managed RAG and Memory for .NET Agents

If you have built a RAG pipeline for a .NET agent, you know the drill: chunk the documents, generate embeddings, stand up a vector store, tune top-K, bolt on hybrid search, and re-index when the source data changes. It works — we walked through exactly that in RAG in C# — but it is a lot of plumbing that has nothing to do with your product.

Microsoft’s answer to that plumbing is Foundry IQ, and as of Build 2026 it is a serious contender: Foundry IQ knowledge bases are generally available with SLA-backed retrieval, Foundry IQ Serverless is in public preview, and memory in Foundry Agent Service (procedural, user, and session) is in public preview too. This article explains what each piece does, what the .NET story looks like today, and — honestly — when this replaces your hand-rolled stack versus when you should keep your pgvector setup.

What Foundry IQ actually is

Foundry IQ is the unified knowledge layer for agents in Microsoft Foundry, built on Azure AI Search. The core idea: instead of wiring retrieval logic into every agent, you define a reusable knowledge base around a topic — employee policies, product docs, support content — and any number of agents connect to it through a single endpoint.

A knowledge base pulls from one or more knowledge sources, and this is where the consolidation shows. The same knowledge base can span:

• Azure Blob Storage and Azure AI Search indexes (your existing content)
• SharePoint / Work IQ (Microsoft 365 content, with permissions respected)
• Fabric IQ and OneLake (analytics-side data)
• Azure SQL and File Search
• Web IQ — live web and licensed publisher data, with sub-200 ms web grounding and zero data retention (limited access at the time of writing)
• MCP sources (private preview), so Model Context Protocol servers can feed the same layer

For indexed sources, Foundry IQ manages the full pipeline automatically: content is ingested, chunked, vectorized, and prepared for hybrid retrieval. That is three of the five steps of a classic RAG pipeline — the ones covered in our chunking strategies guide — handled for you, including re-indexing when documents change.

Agentic retrieval: the interesting part

The piece that genuinely differs from a DIY stack is the agentic retrieval engine inside each knowledge base. A conventional pipeline embeds the user’s question and runs one nearest-neighbour search. Agentic retrieval treats retrieval as a reasoning task: an LLM plans queries, decomposes the question, selects which sources to hit, runs parallel searches, and aggregates the results.

That matters most for multi-hop questions — “compare the refund policy for enterprise customers before and after the March update” — where a single vector search almost always retrieves fragments of the answer but never the whole thing. Microsoft reports an average +20-point (36%) improvement in end-to-end RAG answer quality using knowledge bases with agentic retrieval versus brute-force searching all sources at once, and up to 36% better response relevance on complex multi-hop queries.

Two retrieval modes are exposed:

• Semantic mode — fast hybrid search (vector + keyword + semantic reranking). Comparable to a well-tuned DIY pipeline, minus the maintenance.
• Agentic mode — the query-planning, multi-hop engine above. Slower and more token-hungry, with a configurable reasoning effort (minimal, low, medium), but measurably better on hard questions.

You also choose the output shape: raw extractive data (chunks you assemble into a prompt yourself, like the grounding step you already know) or answer synthesis (the service returns a composed, cited answer).

One current constraint worth knowing before you commit: Foundry IQ requires Azure OpenAI models for the agentic engine at the moment. If your agent runs on a non-Azure model, factor that in.

The .NET story today

Here is the honest state of the SDK surface as of July 2026: the first-party integration ships through the Azure AI Search Context Provider in the Microsoft Agent Framework, and the published quickstart is Python-first (agent-framework-azure-ai-search). The .NET Agent Framework shares the same architecture — context providers that inject retrieved knowledge into the agent loop — so the shape of the code is clear even where the exact C# API is still settling.

Conceptually, attaching a knowledge base to a .NET agent looks like this:

// CONCEPTUAL — verify names against the current Agent Framework packages.
// The Python provider (AzureAISearchContextProvider) defines the pattern;
// the .NET surface mirrors the same options.

var knowledge = new AzureAISearchContextProvider(
    endpoint: new Uri("https://<your-search>.search.windows.net"),
    knowledgeBaseName: "product-docs-kb",
    credential: new DefaultAzureCredential(),   // managed identity, no keys
    mode: RetrievalMode.Agentic,                // or Semantic for speed
    retrievalReasoningEffort: ReasoningEffort.Low);

AIAgent agent = chatClient.CreateAIAgent(
    name: "SupportAgent",
    instructions: "Answer from retrieved knowledge. Cite sources. " +
                  "If the knowledge base has no answer, say so.",
    contextProviders: [knowledge]);

var reply = await agent.RunAsync(
    "How did the enterprise refund window change in the March update?");

Note what is absent: no embedding client, no vector store schema, no top-K tuning, no prompt-assembly code. The context provider calls the knowledge base, the agentic engine does the planning and aggregation, and grounded context arrives in the agent’s prompt. If you prefer to stay closer to the metal, the knowledge base also exposes a REST retrieval endpoint you can call from any .NET service with HttpClient and request extractive_data, then keep your own grounding prompt — a useful migration path if you already have grounding discipline you trust.

Agent memory: three scopes, one managed store

RAG answers “what does the organization know?” Memory answers “what does the agent know from experience?” — and it is the half of the problem most teams put off. We covered the DIY approach in agent memory and state persistence in .NET; Foundry Agent Service now offers a managed alternative, in public preview, with three distinct scopes:

• Session memory — context within a conversation thread. The familiar one: what was said earlier in this chat.
• User memory — preferences and facts that persist across sessions, scoped per user (“prefers concise answers”, “is on the enterprise plan”). A scope parameter segments memory between users so one customer’s facts never leak into another’s context.
• Procedural memory — the new and most interesting scope, introduced at Build 2026. Instead of remembering what was said, the agent retains successful execution patterns: which tool sequence resolved this class of ticket, which query decomposition worked for this kind of report. On repeat tasks the agent starts from a known-good playbook instead of from scratch. Microsoft’s early Tau-bench results show +7–14% absolute task-success gains at near-baseline cost.

That last number deserves the skeptical read you should give every benchmark: it is Microsoft’s own early testing, on a specific benchmark, in preview. But the direction is credible — anyone who has watched an agent re-derive the same five-step workflow on every run can see where a 7–14% lift on repeated task families would come from.

The managed store handles extraction (deciding what is worth remembering), consolidation, and retrieval-time injection. You configure a TTL for new memory entries and control scope per user. For .NET specifically, the same caveat as retrieval applies: the official memory docs currently cover Python and REST, so today a C# service talks to memory over the REST API (the Azure.AI.Projects packages cover the surrounding Foundry surface). Conceptually:

// CONCEPTUAL — memory is in public preview; .NET currently goes via REST.
// Shape mirrors the documented REST operations, not a released C# client.

var request = new
{
    scope = $"user:{userId}",          // isolates this user's memories
    ttlSeconds = 60 * 60 * 24 * 90,    // e.g. retain for 90 days
    kinds = new[] { "user", "procedural" }
};

// Attach the memory store to the agent; the service extracts, consolidates,
// and injects relevant memories on each run — no manual prompt-stuffing.

One production concern to design for from day one: a store that writes model-derived “facts” back into future prompts is a new attack surface. Treat memory content as untrusted input, keep scopes tight, and review what the extractor is persisting — memory poisoning is a real category, not a hypothetical.

Managed vs DIY: when to switch, when to keep pgvector

This is the decision that actually matters, so no hedging — a concrete rubric.

Foundry IQ + managed memory is the right call when:

• Your knowledge lives in Microsoft 365, Fabric, OneLake, or Blob Storage — the connectors, permission sync, and sensitivity-label governance are things you do not want to rebuild, ever.
• You need document-level security (user A must not retrieve what user A cannot open in SharePoint). Getting this right DIY is genuinely hard; here it is the platform’s job.
• Your questions are multi-hop and retrieval quality — not cost — is your bottleneck. The agentic engine is the differentiator you cannot cheaply replicate.
• Your team is small and the RAG pipeline is maintenance you resent: re-indexing jobs, embedding-model migrations, chunking tweaks.
• You are already deploying agents on Foundry Agent Service, where memory arrives as configuration rather than code.

Keep your hand-rolled stack (pgvector and friends) when:

• Your data is in PostgreSQL already and single-hop retrieval works. pgvector in .NET with a tuned hybrid query is fast, cheap, and yours; a managed layer adds latency, spend, and a dependency for no measurable quality gain on easy questions.
• You need non-Azure models end to end — the agentic engine’s Azure OpenAI requirement is a hard constraint today.
• You have strict data-residency or sovereignty requirements that a self-hosted store satisfies by construction.
• Cost predictability beats retrieval quality. Agentic retrieval spends LLM tokens planning queries; on high-volume, simple-question workloads that overhead compounds. Measure both modes’ token use before you standardize — the same discipline as any production cost control.
• You depend on preview-stage features being stable: serverless and memory are public preview, Web IQ is limited access, and the .NET SDK trails Python. If your ship date is next month, build on the GA pieces only.

The pragmatic middle path many .NET teams will land on: keep pgvector for your core, latency-sensitive, single-hop retrieval, and add a Foundry IQ knowledge base for the sprawling, permission-sensitive corpus (SharePoint, tickets, wiki) where connectors and agentic retrieval earn their keep. Nothing forces a single retrieval backend — an agent can hold both as tools.

Takeaway

Foundry IQ is the strongest signal yet that RAG plumbing is becoming platform, not product: knowledge bases are GA with automated indexing and an agentic retrieval engine that measurably beats single-shot vector search on hard questions, and managed memory — session, user, and the genuinely novel procedural scope — is in preview with credible early numbers. The .NET surface is real but trailing Python, so treat exact APIs as moving and the concepts as stable. Adopt it where connectors, permissions, and multi-hop quality are your pain; keep pgvector where your retrieval is simple, cheap, and already working. The skill that transfers either way is the one you already have: knowing what good retrieval and disciplined grounding look like, so you can tell whether the managed version is actually doing it.

---

Have a correction or a topic you want covered? Email mani.bc72@gmail.com.