Embeddings in .NET, Explained

If you’ve looked into RAG or semantic search, you’ve hit the word “embeddings” — and maybe bounced off it. It’s simpler than it sounds. This article explains what embeddings actually are, how to generate them in C#, and what you do with them, without the math-heavy hand-waving.

What an embedding is

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. A model turns “I love my dog” into something like [0.02, -0.11, 0.34, ...] — often hundreds or thousands of numbers. The key property: texts with similar meaning get similar vectors. “I love my dog” and “My puppy is the best” land close together in this number-space; “quarterly tax filing” lands far away.

That’s the whole trick. Once meaning is a vector, “find similar text” becomes “find nearby vectors” — a math operation computers are fast at.

What “meaning” means here — and what it doesn’t

The word “meaning” is doing a lot of work in that description, and being precise about it saves you from a class of bug that is very hard to debug later.

An embedding captures roughly what a piece of text is about. It does not capture truth, negation, quantity or time with any reliability. “The invoice was paid” and “The invoice was not paid” are about the same thing — invoices and payment — so they land close together in vector space despite meaning opposite things. Same for “the deployment succeeded” versus “the deployment failed”, or “supported in v2” versus “removed in v2”. Retrieval will happily hand your model the passage that says the opposite of what the user asked about, and the model, having been told this is the relevant context, will often go along with it.

Two consequences follow. First, similarity is not relevance. The nearest chunk is the one most topically alike, which is usually but not always the one that answers the question. Second, for anything where a negation or a version number or a date flips the answer, retrieval alone is not enough — you need the surrounding context in the chunk, and you need an evaluation set that includes those cases. Nothing about the vectors will warn you.

Generating embeddings in C#

You call an embedding model, not a chat model. With Microsoft.Extensions.AI, you use an IEmbeddingGenerator<TInput, TEmbedding>. The interface method takes a collection of inputs and returns GeneratedEmbeddings<Embedding<float>>; accelerator extension methods cover the single-value case:

using Microsoft.Extensions.AI;

IEmbeddingGenerator<string, Embedding<float>> generator = /* e.g. an OpenAI embeddings client */;

// One value at a time
Embedding<float> embedding = await generator.GenerateAsync("I love my dog");
ReadOnlyMemory<float> vector = embedding.Vector;   // the array of numbers
Console.WriteLine($"Dimensions: {vector.Length}");

// Or straight to the vector
ReadOnlyMemory<float> v = await generator.GenerateVectorAsync("I love my dog");

That’s it — text in, vector out. You generate a vector for each chunk of text you want to make searchable, and store them.

Batch when you index. The single-value call is fine for the query side, where you embed one string per user request. It is the wrong shape for ingestion. Embedding ten thousand chunks one HTTP round trip at a time means ten thousand round trips of network latency you did not need to pay, and it burns through per-minute request quota far faster than the equivalent token volume would:

// Batched: one request carrying many inputs
GeneratedEmbeddings<Embedding<float>> batch = await generator.GenerateAsync(chunks);

// Or keep each input paired with its vector
foreach ((string Value, Embedding<float> Embedding) pair in
    await generator.GenerateAndZipAsync(chunks))
{
    Store(pair.Value, pair.Embedding.Vector);
}

GenerateAndZipAsync is worth knowing about specifically because the manual version — zipping two lists by index and trusting the order — is a bug waiting to happen the first time someone filters one list and not the other. A chunk stored against the wrong vector produces search results that are subtly, unreproducibly wrong.

Because IEmbeddingGenerator supports the same middleware pattern as IChatClient, you can wrap it too. EmbeddingGeneratorBuilder<string, Embedding<float>> gives you UseDistributedCache and UseOpenTelemetry, and DelegatingEmbeddingGenerator<TInput, TEmbedding> is the base class for your own — rate limiting during a bulk re-index being the obvious one.

Measuring similarity

To compare two vectors, you compute their cosine similarity — a number from -1 to 1 where higher means more similar. Conceptually:

static float CosineSimilarity(ReadOnlySpan<float> a, ReadOnlySpan<float> b)
{
    float dot = 0, magA = 0, magB = 0;
    for (int i = 0; i < a.Length; i++)
    {
        dot += a[i] * b[i];
        magA += a[i] * a[i];
        magB += b[i] * b[i];
    }
    return dot / (MathF.Sqrt(magA) * MathF.Sqrt(magB));
}

Don’t ship that version. .NET has a vectorized implementation in the box — TensorPrimitives.CosineSimilarity from the System.Numerics.Tensors package uses SIMD instructions and will comfortably beat a scalar loop over 1,536-element vectors:

using System.Numerics.Tensors;

float score = TensorPrimitives.CosineSimilarity(a.Span, b.Span);

Two practical notes. Cosine similarity ignores magnitude and only compares direction, which is what you want for text — a long document and a short summary of it should score as similar. And if your vectors are already unit-normalized (many embedding APIs return them that way), cosine similarity reduces to a plain dot product, which is why vector databases often expose “cosine” and “inner product” as separate distance metrics that give identical rankings on normalized data.

At scale you don’t hand-roll any of this: a vector database does approximate nearest-neighbour search for you, because comparing a query against a million vectors linearly is a scan you cannot afford per request. But the brute-force loop is genuinely the right answer below a few thousand vectors — an in-memory list and TensorPrimitives will return in single-digit milliseconds, and you have avoided an entire piece of infrastructure. Don’t reach for a vector store until you can show you need one.

Dimensions, storage and the cost you didn’t budget for

Embedding dimensionality is the number people skip past, and it is the one that shows up on the bill.

OpenAI’s text-embedding-3-small produces 1,536 dimensions by default and text-embedding-3-large produces 3,072, both accepting up to 8,192 input tokens. Stored as 32-bit floats, that is 6 KB and 12 KB per vector respectively, before index overhead. A million chunks is therefore roughly 6 GB or 12 GB of vectors — which changes a “we’ll just put it in Postgres” plan into a capacity conversation, and changes an approximate-nearest-neighbour index from optional to mandatory.

The v3 models were trained so their vectors can be truncated: you can request fewer dimensions and keep most of the retrieval quality. In Microsoft.Extensions.AI that is EmbeddingGenerationOptions.Dimensions:

var options = new EmbeddingGenerationOptions { Dimensions = 512 };
GeneratedEmbeddings<Embedding<float>> embeddings =
    await generator.GenerateAsync(chunks, options);

Halving or quartering dimensions halves or quarters your storage and speeds up every comparison. It also costs you some accuracy, and how much depends entirely on your corpus — which means it is a decision to make with an evaluation set, not from a blog post. The point is that it is a dial you have, and most teams never touch it.

The failure modes worth knowing about

Mixing models. Vectors from two different embedding models are not comparable, and — this is the dangerous part — nothing throws. If both models produce 1,536 dimensions, the arithmetic completes and returns numbers. The similarity scores are meaningless, so search quality collapses to roughly random while every health check stays green. If you re-index half a corpus with a new model and stop, you have built a system that is broken in a way no exception will tell you about. Store the model id alongside every vector so this is at least detectable.

Chunks that exceed the input limit. Feed a whole document to a model with an 8,192-token ceiling and the request is rejected. Worse is the near-miss: a chunk that fits but covers four unrelated topics produces an averaged vector that sits near nothing in particular, so it never surfaces for any query. Vectors that are too broad don’t error — they just quietly never match. Chunking strategy is where most retrieval quality is actually won or lost.

Queries that aren’t semantic. Someone searching for an order number, an error code, a SKU or an exact function name is doing a lookup, not a meaning search, and embeddings are bad at it — a product code carries almost no semantic signal, so its vector is near-arbitrary. Keyword search handles these trivially. The standard answer is hybrid retrieval: run both, and merge the results. If your users search for identifiers, an embeddings-only system will feel broken to them no matter how good the model is.

The cost of changing your mind. Switching embedding models means re-embedding your entire corpus and rebuilding the index. That is a bounded, known cost, but it is a real one, so it is worth doing a small quality comparison before you commit to a model rather than after you have indexed ten million chunks.

When not to use embeddings

If you can express the query as a filter — status, date range, owner, category — use a WHERE clause. Embeddings are for the part of the problem that structured queries cannot express, and putting them in front of a lookup that SQL already handles adds latency, cost and a source of wrongness for no gain. Similarly, if your corpus is a few dozen documents, the whole thing may fit in the model’s context window; passing it directly is simpler and more accurate than building a retrieval pipeline around it.

What you actually do with embeddings

Embeddings power a surprising range of features:

• Semantic search — find documents by meaning, not keywords (how-to here).
• RAG — retrieve the most relevant chunks to ground an agent’s answer (RAG in .NET).
• Clustering & deduplication — group similar items, find near-duplicates.
• Classification — label text by comparing it to labelled examples.
• Recommendations — “more like this.”

If your problem is “find things that mean something similar,” embeddings are the tool.

Practical notes

• Use one model consistently. Vectors from different embedding models aren’t comparable — pick one and re-embed everything if you switch.
• Chunk sensibly. Embed paragraphs or sections, not whole documents, so a match points to the right passage.
• Store the original text alongside the vector — you’ll need it when a search returns a hit.

Note: the IEmbeddingGenerator interface is still stabilizing in Microsoft.Extensions.AI. Verify against the current .NET AI docs; the concept — text becomes a meaning-vector, similarity is distance between vectors — is fundamental and unchanging.

Takeaway

An embedding turns text into a vector of numbers where similar meanings sit close together. Generate them in C# with an IEmbeddingGenerator, batch the ingestion side, compare them with TensorPrimitives.CosineSimilarity (or let a vector database do it at scale), and you have the machinery behind semantic search, RAG, clustering and recommendations. Remember the two limits that catch people out: the vectors capture topic rather than truth, so negations and version numbers slip through, and they are useless for exact identifiers — which is why serious systems run keyword search alongside. Keep one embedding model per index, store the model id next to every vector, and treat dimensionality as a dial you can turn rather than a constant you inherited.