Build a "Chat with your PDF" App in .NET

“Chat with your PDF” is one of the most requested AI features — upload a document, ask questions, get answers grounded in that document. It’s a perfect, self-contained application of RAG. This guide builds one in .NET, end to end.

The pipeline

Four steps, two of them one-time per document:

1. Extract the text from the PDF.
2. Chunk it into passages and embed each one.
3. Retrieve the chunks most relevant to the user’s question.
4. Answer using only those chunks as context.

First: do you actually need retrieval?

Worth asking before you build any of it, because for a large share of “chat with your PDF” features the answer is no.

If the document is small enough to fit comfortably in the model’s context window — a contract, a policy document, a twenty-page report — the simplest correct implementation is to extract the text and put all of it in the prompt. No embeddings, no vector store, no chunking strategy to tune, no retrieval step that can miss. Accuracy goes up rather than down, because the model sees the whole document instead of the four passages your similarity search happened to like. If users ask several questions about the same document, prompt caching makes the repeated context cheap after the first call.

Retrieval earns its complexity when the corpus is genuinely bigger than the window, or when you have many documents and need to search across them, or when per-request cost matters more than per-request accuracy. Build the simple version first, measure where it stops working, and add retrieval at that point. Teams that start with the pipeline usually end up debugging a retrieval bug that wouldn’t exist in an application that didn’t need retrieval.

The rest of this guide assumes you’ve concluded that you do.

Step 1: Extract text from the PDF

Use a PDF library (such as PdfPig or another .NET PDF reader) to pull the text out, page by page:

using UglyToad.PdfPig;

string ExtractText(string path)
{
    using var doc = PdfDocument.Open(path);
    return string.Join("\n", doc.GetPages().Select(p => p.Text));
}

That’s the one-liner, and it has a trap in it. page.Text returns glyphs in content-stream order, which is the order the PDF happens to draw them in — frequently not the order a human reads them. On a single-column page it’s usually fine. On a two-column academic paper, an invoice, or anything with a sidebar, you get the two columns interleaved line by line, and every chunk you cut from it is nonsense that embeds and retrieves like nonsense. PdfPig ships layout-analysis helpers for exactly this, and reconstructing reading order costs one call:

using UglyToad.PdfPig;
using UglyToad.PdfPig.DocumentLayoutAnalysis.TextExtractor;

List<(int Page, string Text)> ExtractPages(string path)
{
    using var doc = PdfDocument.Open(path);
    return doc.GetPages()
        .Select(p => (Page: p.Number, Text: ContentOrderTextExtractor.GetText(p)))
        .ToList();
}

Keeping the page number alongside the text costs nothing here and is the only way you’ll be able to cite sources later. Add it now; retrofitting it means re-processing every document you’ve already ingested.

The scanned-PDF problem

The failure that generates the most support tickets: a PDF that is a photograph of a page has no text layer at all. Extraction succeeds, returns almost nothing, embedding succeeds on that nothing, retrieval returns it, and the model tells the user their document doesn’t mention the thing that’s printed on page one in bold. Nothing in the pipeline errors.

Detect it rather than discovering it from a complaint:

var pages = ExtractPages(path);
var averageChars = pages.Average(p => p.Text.Length);

if (averageChars < 100)
{
    // Almost certainly an image-only scan. Route to OCR, or reject the upload
    // with a message the user can act on — do not embed 40 characters of noise.
}

Threshold by judgement, not superstition: a genuine text page carries hundreds to thousands of characters, so anything in the low double digits means there’s no text layer. Once detected you have two honest options — run OCR (a real dependency and a real per-page cost, so decide deliberately) or reject the file and tell the user why. Silently ingesting an empty document is the only bad option.

Two smaller extraction problems worth knowing about: tables flatten into runs of numbers that lose their column meaning, so a document that’s mostly tabular will answer numeric questions badly regardless of the rest of your pipeline; and repeated headers and footers get pulled into every page, which means the same boilerplate appears in every chunk and dilutes each one’s embedding. Stripping lines that repeat on most pages is a cheap, high-value cleanup step.

Step 2: Chunk and embed

You can’t embed a whole document as one vector — retrieval would be too coarse. Split into overlapping passages and generate an embedding for each:

var chunks = ChunkText(pdfText, size: 500, overlap: 75);   // your splitter
foreach (var chunk in chunks)
{
    var vector = (await embedder.GenerateAsync(chunk)).Vector;
    store.Add(chunk, vector);   // in-memory list, or pgvector for scale
}

For a single uploaded PDF, an in-memory list is fine; for many documents, use pgvector.

Those numbers are a starting point, not an answer. A fixed character window cuts wherever it lands, so it will routinely split a sentence in half and put the subject in one chunk and the verb in the next — and a chunk starting mid-clause embeds poorly, because embeddings encode meaning and half a sentence has less of it. Overlap is the crude fix, and it isn’t free: 75 characters of overlap on 500-character chunks means roughly fifteen percent more vectors to store, embed and search, and it makes near-duplicate results more likely in your top-k.

The better move is to split on structure first and pack afterwards. Break on paragraph and heading boundaries, then combine adjacent pieces until you approach your target size, so chunks end where meaning ends. Only fall back to hard character splits for a paragraph that’s genuinely too long. And carry the metadata through, because a chunk you can’t attribute is a chunk you can’t cite:

public sealed record Chunk(
    string DocumentId,
    int Page,
    string Text,
    ReadOnlyMemory<float> Vector);

Embedding is also the slowest part of ingestion. A few hundred pages is a few hundred embedding calls, which is seconds to minutes — far too long to hold an HTTP upload open. Accept the file, return immediately, process in the background, and give the document a status the UI can poll. Batch the embedding calls rather than issuing one per chunk; the round-trip dominates.

For more on the trade-offs here, see chunking strategies for RAG.

Step 3: Retrieve the relevant chunks

When a question comes in, embed it and pull the closest chunks (semantic search):

var q = (await embedder.GenerateAsync(question)).Vector;
var top = store.NearestTo(q, k: 4);   // the 4 most relevant passages

k is the dial that most directly trades accuracy against cost, and both directions fail. Too low and the answer simply isn’t in the context, so the model either says it can’t find it (the good outcome) or fills the gap from general knowledge (the bad one). Too high and you’re paying for twenty passages on every single question while making the model’s job harder — relevant text buried among mostly-irrelevant text is measurably harder to use than the same text on its own. Start at four or five, and tune it against real questions rather than intuition.

If you’re serving more than one user or more than one document, filter inside the vector query rather than after it. Retrieving the global top-k and then discarding the chunks belonging to other tenants gives you a result set that’s sometimes empty and always smaller than you asked for — and one missed filter is a cross-tenant data leak, so it’s worth building the constraint into the store rather than into a Where clause someone can forget.

Step 4: Answer, grounded in the document

Build a prompt that gives the model only those chunks and instructs it to answer from them — and to admit when the answer isn’t there:

var context = string.Join("\n\n", top);
var prompt = $"""
    Answer the question using ONLY the context below, which comes from the user's document.
    If the answer isn't in the context, say you couldn't find it in the document.

    Context:
    {context}

    Question: {question}
    """;

var answer = await chatClient.GetResponseAsync(prompt);

That “answer only from the context, and say when you can’t” instruction is what keeps the app honest — it answers from the PDF instead of the model’s general knowledge, and it says “not in the document” instead of inventing something.

It reduces hallucination sharply. It doesn’t eliminate it, and it’s worth being clear about why: the instruction is guidance, not a constraint the system enforces. The residual failure is subtle — when the retrieved passages are nearly right, the model tends to bridge the gap with plausible general knowledge rather than declining, and the result reads exactly like a grounded answer because most of it is one.

The cheap defence is to make grounding checkable. Ask for the supporting quote as a separate field, then verify deterministically that the quote actually appears in what you retrieved:

public sealed record GroundedAnswer(string Answer, string Quote, int Page, bool FoundInDocument);

var grounded = /* structured output from the model */;

var supported = top.Any(c =>
    c.Text.Contains(grounded.Quote, StringComparison.OrdinalIgnoreCase));

if (grounded.FoundInDocument && !supported)
{
    // The model cited something that isn't in the retrieved context.
    // Flag it, or fall back to "couldn't find it" — do not present it as sourced.
}

That’s a few lines of string matching catching the one failure users are least equipped to notice themselves.

The questions this design will get wrong

Vector retrieval finds passages similar to the question. That works beautifully for “what does the contract say about termination” and badly for anything requiring completeness or counting — “how many times is the client mentioned”, “list every deadline in the document”, “summarise the whole thing”. Similarity search retrieves the top few matches, not all matches, so the model confidently answers “three” from the three chunks it was handed when the true answer is nine.

There’s no prompt that fixes this, because the information never reached the model. The options are to detect these question shapes and route them differently — a full-document pass for summarisation, a plain text search for counting — or to be explicit in the UI about what the feature answers well. Silently returning a wrong number is the worst of the three.

Making it production-grade

• Cite the source — return which chunk/page an answer came from, so users can verify.
• Cache the embeddings — embed a document once on upload, not on every question.
• Handle scanned PDFs — image-only PDFs need OCR before extraction.
• Mind the tokens — retrieving too many chunks inflates cost; tune k down.

Citation deserves the top spot on that list, because it’s the only one that changes what users can do rather than what it costs you. An answer with a page number is verifiable in five seconds; the same answer without one has to be trusted or ignored. It also changes the failure mode from “the app was wrong” to “the app pointed me at the wrong page”, which is a far more recoverable experience.

On cost, the shape is worth internalising: ingestion is a one-off charge proportional to document length, while every question costs a question embedding plus k chunks of input plus the output. Ingestion feels expensive and isn’t; the per-question input is small per call and is charged on every call forever. That’s why k matters more than chunk size for your monthly bill.

Note: PDF and embedding library APIs vary; verify against your chosen packages. The pipeline — extract, chunk, embed, retrieve, answer-from-context — is the stable, provider-independent recipe behind every “chat with your documents” feature.

Takeaway

A “chat with your PDF” app is RAG applied to one document: extract the text, chunk and embed it, retrieve the passages relevant to each question, and answer using only those passages. Ground the model firmly in the retrieved context — “answer only from this, say when you can’t” — cite sources, and cache the embeddings. The parts that decide whether it survives real users are less glamorous: reconstruct reading order instead of trusting raw extraction, detect scanned documents before ingesting emptiness, verify quoted evidence against what you actually retrieved, and be honest about the counting and summarising questions retrieval can’t answer. Get those right and it’s a genuinely useful feature you can build in an afternoon in .NET.

Next: chunking strategies for RAG to improve the answers this returns.