Agents bill by the token, and tokens scale with success. The more users your agent helps, the bigger the invoice — and because a single agent turn can involve several model round-trips (reasoning, tool calls, follow-ups), costs climb faster than request counts suggest. The good news: most agent spend is waste you can remove without hurting quality. This article is a practical playbook for controlling the cost of a production .NET agent.
Cost tracking pairs naturally with observability — you can’t optimize what you don’t measure.
Why the bill is bigger than you modelled
Most teams estimate agent cost the way they estimate API cost: requests per day multiplied by an average price per request. That model is wrong in three ways at once, and the gap between it and reality is usually a factor of several rather than a few percent.
A single user turn is not a single model call. An agent that consults two tools makes at least three round-trips — one to decide on the first tool, one to decide on the second after seeing the first result, and one to write the answer. Each of those round-trips resends the entire conversation so far, plus the system prompt, plus every tool schema. The input side is therefore quadratic-ish in tool calls rather than linear, which is why an agent that “only” makes four tool calls can cost five times what a plain chat completion does.
Output tokens are priced higher than input tokens across essentially every provider, so a verbose agent is expensive twice over: once for the tokens it generates, and again on the next turn when those tokens come back as history. A system prompt that says “explain your reasoning” is a standing charge on every future turn of that conversation.
And failures are billed. A tool call that throws, a response that fails JSON validation, a request your retry policy fires again after a timeout — the model still generated tokens, and you still pay for them. If you stream and the user closes the tab, the tokens generated up to that point are generally not refunded. Retry storms are the classic way a cost incident turns into a cost catastrophe: a downstream tool starts failing, the agent retries, each retry replays the full context, and the bill triples while the product is broken.
First: measure per-request cost
You can’t control what you can’t see. Before optimizing, make token usage visible — the AI telemetry exposes prompt and completion token counts per call. Emit them as metrics tagged by endpoint and model, and you’ll immediately spot which flows are expensive. Often one or two paths drive most of the bill, and that’s where to focus.
Record the tokens, not the money. Prices change, and a stored cost figure calculated at yesterday’s rate is a number nobody can recompute. Store input tokens, output tokens, cached tokens, model name and the flow that triggered the call; multiply by the current rate at query time. That also lets you answer the question finance eventually asks, which is not “what did we spend” but “what would we have spent on the cheaper model”.
Tag by tenant from the first day. Retrofitting a tenant dimension onto a metric after six months of data means you cannot answer the one question that matters when the bill spikes: whether this is growth or one customer doing something unusual. See counting tokens in C# for the mechanics of getting the numbers before and after a call.
1. Right-size the model (the biggest lever)
The single largest cost mistake is running a frontier model for everything. As covered in choosing a model, route simple turns — classification, extraction, short answers — to a small, cheap model and reserve the expensive one for genuinely hard reasoning. Because it’s all IChatClient, a per-request router is cheap to build and often cuts spend by more than half with no quality loss users would notice. Do this before micro-optimizing anything else.
2. Cache what repeats
Agents answer the same things over and over. Two kinds of caching help:
- Response caching — for deterministic or near-deterministic queries (FAQ-style questions, repeated lookups), cache the answer keyed on a normalized input. A cache hit costs nothing and returns instantly.
- Prompt caching — many providers now cache a stable prompt prefix (your long system prompt, tool definitions, retrieved context) so repeated calls are billed at a steep discount for the cached portion. Structure your prompts so the stable part comes first and the variable part last, and you get this largely for free.
Prompt caching has a failure mode that is almost invisible, and I have watched it cost a team a month of savings before anyone spotted it. The cache matches on an exact prefix. Put anything variable near the top of your system prompt — a timestamp, the user’s name, a request ID, a “today is {date}” line — and every request produces a different prefix, so nothing ever hits. The metric to watch is the cached-token count the provider reports; if it sits at zero while you believe caching is on, something upstream is mutating the prefix. The same applies to tool definitions serialised from a dictionary with non-deterministic ordering, which produces a different prefix on every process start.
Providers also impose a minimum cacheable prefix length and expire entries after a short idle window, so caching helps high-traffic flows and does nothing for a prompt used twice an hour. Check both figures against your provider’s current documentation before designing around them.
Response caching has its own trap, and it is worse than a wasted saving. Cache keys must include everything that changes the answer: the tenant, the user’s permissions, the model, the prompt version, and any retrieved context. Miss the tenant and you have built a data leak that presents itself as a performance win. Normalising the input is what makes the cache useful — lowercasing, trimming, collapsing whitespace — but normalise too aggressively and “cancel my order” and “cancel my order?” stop being the same question in one direction while genuinely different questions start colliding in the other. Set a short TTL and treat the cache as an optimisation you can turn off, not a source of truth.
3. Trim the context you send
You pay for every token in the prompt, on every call. Bloated context is silent, recurring waste:
- Cap conversation history. Don’t resend the entire thread forever — keep a window of recent turns and summarize older ones. A 40-turn conversation resending all 40 turns each time is paying to re-read history the model barely needs.
- Retrieve less, better. In RAG, sending the top-10 chunks when the top-3 would do triples that part of the bill. Tune K down and use re-ranking so the few chunks you send are the right ones.
- Prune tool definitions. Every tool’s schema is tokens in every request. An agent with 30 tools pays for 30 descriptions each call — expose only the tools a given flow actually needs.
4. Stop runaway loops
An agent stuck calling tools in circles can burn a fortune on a single request. Defend against it:
- Set a max-turns / max-tool-calls limit per run so a confused agent fails fast instead of spinning.
- Alert on high token-per-request — a sudden jump is often a loop or a prompt regression, and catching it early saves real money.
- Add timeouts so a slow or looping run can’t run unbounded.
5. Put a budget in place
Finally, make cost a first-class constraint, not a monthly surprise:
- Per-user or per-tenant rate limits cap how much any single caller can spend — and double as abuse protection.
- Provider-side budget alerts on your Azure OpenAI / model spend give you a backstop.
- A monthly cost dashboard next to your quality metrics keeps the trade-off honest: you want the cheapest configuration that still passes your evals.
What this costs you back
None of these levers is free, and pretending otherwise is how you end up with a cheap agent nobody wants to use.
Model routing adds a decision. If the router is itself a model call you have added latency and cost to every request, so route on cheap signals where you can — the endpoint, the flow, the input length, a classifier you already run — and escalate rather than pre-classify. The safer pattern is optimistic: send it to the small model, check the answer against a quality signal you trust, and retry on the large one when the check fails. You pay twice on the minority of requests that need it instead of paying the router on all of them. The failure mode to watch for is silent degradation: a router that quietly sends harder work to the small model as your traffic mix shifts, with quality dropping a little each week and nothing alerting.
Summarising history costs tokens to save tokens. You are paying for a model call now to avoid resending a long history later, which pays off on long conversations and loses money on short ones. Only summarise once the history is genuinely large, and never summarise on every turn.
Trimming retrieval trades cost against recall. Dropping from ten chunks to three cuts that part of the bill by seventy percent and will occasionally drop the chunk that had the answer. That is a quality decision disguised as a cost decision, which is why it belongs behind an eval rather than a hunch.
And there is a threshold below which none of this is worth doing. If your agent spend is smaller than a day of engineering time per month, the correct optimisation is to leave it alone. Build the measurement so you notice when that stops being true, then spend the effort when the number justifies it. Cost work also raises the cost of change: a heavily tuned prompt with a carefully cached prefix and three routing rules is harder to modify than one that just calls the big model, and early on the ability to change things quickly is worth more than the savings.
If you are multi-tenant, cost is also an attribution problem: multi-tenant AI agents in .NET covers per-tenant usage recording and stopping one customer consuming everyone else’s quota.
Note: provider pricing, prompt-caching behavior, and quotas change often. Verify specifics against your model provider’s current docs; the levers here — right-size the model, cache, trim context, cap loops, budget — are durable and apply across providers.
Takeaway
Agent cost isn’t fixed — it’s mostly removable waste. Measure per-request token spend first, then pull the levers in order of impact: right-size the model (the big one), cache repeated work and stable prompt prefixes, trim history and retrieval and tool schemas, cap runaway loops, and enforce per-user budgets. Done together, these routinely cut a bill by half or more while keeping quality intact — turning “the agent is getting expensive” into a number you control.
Have a correction or a topic you want covered? Email mani.bc72@gmail.com.