Vector Search in SQL Server 2025 with EF Core 10: One Less Database

The standard .NET RAG architecture has an awkward shape. Your application data lives in SQL Server, and your embeddings live somewhere else — pgvector, Qdrant, Pinecone, Azure AI Search. Two stores, two connection strings, two backup policies, two things to keep consistent, and a permanent low-grade problem where the vector store knows a document exists and the relational database knows it was deleted.

SQL Server 2025 has a native vector type, and EF Core 10 maps it. For a decent number of teams that is one less database.

What it looks like

The whole feature, in the shape you will actually use it:

public class DocumentChunk
{
    public int Id { get; set; }
    public int DocumentId { get; set; }
    public string Text { get; set; } = "";

    // The dimension count is part of the column type, not a hint.
    // Change it later and it is a migration with a rewrite.
    [Column(TypeName = "vector(1536)")]
    public SqlVector<float> Embedding { get; set; }
}

And the search:

var queryVector = new SqlVector<float>(await _embeddings.EmbedAsync(question));

var hits = await db.Chunks
    .Where(c => c.TenantId == tenantId)          // ordinary relational filter
    .OrderBy(c => EF.Functions.VectorDistance("cosine", c.Embedding, queryVector))
    .Take(8)
    .ToListAsync(ct);

That is the pitch in one query. The tenant filter and the similarity ranking are the same statement, planned together. With a separate vector store you would fetch fifty candidates, hope enough of them belong to the right tenant, then filter in memory — and quietly return fewer results than you asked for whenever they did not.

Exact distance, then approximate

There are two modes and the difference matters more than the syntax suggests.

EF.Functions.VectorDistance() computes exact distances. It compares the query vector against every row that survives your WHERE clause. That is a full scan of the filtered set, which is completely fine at ten thousand chunks and completely unusable at ten million.

For scale you add a DiskANN-backed vector index, configured through the model and created by a migration:

modelBuilder.Entity<DocumentChunk>()
    .HasIndex(c => c.Embedding)
    .HasVectorIndex(m => m.DistanceMetric = "cosine");

That switches you to approximate nearest-neighbour search, and the word doing the work is approximate. The index can miss genuinely relevant rows. The recall is high and the speed-up is large, but if your evaluation set was built against exact distances, expect the numbers to move slightly when you add the index — and do not be the person who spends a day debugging a retrieval regression that was the index working as designed.

The practical sequence: build with exact distance while the corpus is small and you are still tuning chunking, because exactness removes a variable. Add the index when scan time shows up in your traces. Re-run your evaluation set afterwards and write down what changed.

Check the dimension ceiling before you choose a model

This is the constraint most likely to force a rewrite, and it is easy to hit after you have already embedded everything.

The documented limits are 1998 dimensions at single precision and 3996 at half precision. Set against the embedding models people actually reach for, that is not a generous ceiling. A 1536-dimension model fits comfortably. A 3072-dimension model does not fit in a single-precision column at full width.

There are three ways through, in order of how much you will like them:

Shorten the embedding. Several current models support requesting a reduced output dimension directly, and the quality cost of going from 3072 to 1536 is usually small — often smaller than the retrieval difference between two chunking strategies. This is the right answer most of the time.

Use half precision. Doubles the ceiling and halves the storage. You lose some numerical precision, which matters less than people expect for cosine similarity over normalised vectors, but it is a change worth measuring rather than assuming.

Pick a different model. Perfectly reasonable if you have not embedded anything yet, and expensive afterwards, because changing embedding model means re-embedding the entire corpus — every chunk, at whatever the current per-token rate is.

Decide this before the first ingest run. Embeddings for .NET developers, explained covers what you are actually trading when you shorten one.

Where it genuinely beats a separate store

Filtered search. Covered above, and it is the big one. Multi-tenant applications, permission-scoped documents, anything with a date range — these are hard to do well across two stores and trivial in one query.

Transactional consistency. Insert a document, its chunks and its embeddings in one transaction. Delete a document and the vectors go with it via a foreign key. The entire class of “the vector store still thinks this exists” bug simply does not occur, and that bug is more common than anyone admits because it fails silently — the user just gets an answer citing a document that was deleted last month.

Operations you already have. Backups, point-in-time restore, failover, monitoring, access control, your DBA. A second data store means a second set of all of that, and in most organisations the second set is worse because nobody owns it.

Hybrid search. Combining vector similarity with full-text search in one statement is well supported, and it fixes the case embeddings are genuinely weak at — exact identifiers, error codes, product SKUs. That combination is the single highest-leverage retrieval improvement available to most applications, and doing it in one engine is a lot less code than reconciling two ranked lists from two systems.

Where it does not

Very large corpora under heavy load. Purpose-built vector databases exist for a reason. At tens of millions of vectors with high concurrent query rates, dedicated stores have index structures, memory strategies and sharding that a general-purpose relational engine is not trying to compete with.

You are not on SQL Server 2025. This needs the 2025 engine. If you are on 2019 or 2022, this is a major version upgrade, and that is a bigger project than adding pgvector next to whatever you have. pgvector with .NET is the comparable move on the Postgres side.

Rapid embedding-model churn. If you expect to re-embed frequently while experimenting, a store you can drop and rebuild without a schema migration is more pleasant to iterate against. Prototype elsewhere, land here.

Your embeddings genuinely have nothing to do with your relational data. If the corpus is public documentation with no join to anything, the integration argument evaporates and it becomes a straight performance comparison — which you should expect to lose at scale. Comparing vector databases for .NET covers the alternatives on their merits.

Two practical notes

Do not select the embedding column by accident. db.Chunks.ToListAsync() drags 1536 floats per row across the wire for no reason. Project to what you need:

var hits = await db.Chunks
    .Where(c => c.TenantId == tenantId)
    .OrderBy(c => EF.Functions.VectorDistance("cosine", c.Embedding, queryVector))
    .Select(c => new { c.Id, c.Text, c.DocumentId })   // not the vector
    .Take(8)
    .ToListAsync(ct);

This is the most common performance complaint about vector columns in any database, and it is nearly always this.

Normalise your metric choice once. Cosine is the default assumption for most text embedding models and is what you almost certainly want. Whatever you choose, use the same metric in the index configuration and in the query — a mismatch does not error, it just quietly returns worse results, which is the hardest kind of bug to notice.

Takeaway

For a .NET team already running SQL Server, the interesting thing about native vector search is not that it is fast. It is that it deletes a component. One database, one transaction boundary, one backup, one set of credentials, and filtered similarity search that composes with the rest of your data instead of fighting it.

Check the dimension ceiling before you pick an embedding model, start with exact distance while the corpus is small, and add the DiskANN index when your traces say to — then re-run your evaluation set, because approximate means approximate.

If you are building retrieval on top of this, agentic RAG in .NET covers what to do when one query is not enough, and building a RAG knowledge base for a .NET agent covers the ingest side.

Note: SQL Server 2025 vector support and EF Core 10’s provider surface are both new, and index types in particular have moved through preview at different rates on SQL Server and Azure SQL. Verify the type name, the dimension ceilings and the index configuration against the EF Core SQL Server vector search documentation for your exact platform before designing a schema around them.

---

Have a correction or a topic you want covered? Email mani.bc72@gmail.com.