AAgentic.NET
.NETAzureServerlessDeployment

Serverless AI: Deploy an OpenAI Endpoint with Azure Functions

Host an AI endpoint that scales to zero and costs nothing when idle. Build and deploy a serverless OpenAI-powered API in .NET with Azure Functions.

MB
Manikandhan Balakrishnan
·3 min read

Not every AI feature needs a full service running around the clock. For an endpoint that gets sporadic traffic — a “summarize this” button, an occasional classification call — Azure Functions is ideal: it scales to zero when idle (you pay nothing) and spins up on demand. This guide deploys a serverless OpenAI-powered API in .NET.

Why serverless for AI

AI endpoints are often bursty and I/O-bound (most of the time is spent waiting on the model). That’s a great fit for Functions:

  • Scale to zero — no traffic, no cost. Perfect for internal tools or low-volume features.
  • Automatic scale-out — a burst of requests spins up more instances.
  • No servers to manage — you ship a function, not infrastructure.

For steady, high-volume traffic, a always-on container (Azure Container Apps or GKE) is often better — but for spiky workloads, serverless wins.

The function

An HTTP-triggered function that calls the model is refreshingly small:

public class SummarizeFunction
{
    private readonly IChatClient _chat;
    public SummarizeFunction(IChatClient chat) => _chat = chat;

    [Function("Summarize")]
    public async Task<HttpResponseData> Run(
        [HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req)
    {
        var input = await req.ReadFromJsonAsync<SummarizeRequest>();
        var reply = await _chat.GetResponseAsync(
            $"Summarize this in two sentences:\n\n{input!.Text}");

        var res = req.CreateResponse();
        await res.WriteAsJsonAsync(new { summary = reply.ToString() });
        return res;
    }
}

record SummarizeRequest(string Text);

Register the IChatClient in Program.cs and inject it — the same Microsoft.Extensions.AI pattern as anywhere else.

Keep the key out of the function

Don’t put your API key in app settings as plain text. Use a managed identity: give the Function App an identity and grant it access to Azure OpenAI, then authenticate with DefaultAzureCredential. No key exists to leak — the same keyless pattern from securing agents.

The one thing to watch: cold starts and timeouts

Two serverless caveats matter for AI:

  • Cold starts — an idle function takes a moment to wake. Fine for background or user-initiated actions; consider a premium plan with pre-warmed instances if you need consistently low latency.
  • Timeouts — the consumption plan caps execution time. A long, streaming generation can bump against it. For long responses, either use a plan with a higher timeout or switch to a container.

When to choose it

Reach for Functions when your AI endpoint is occasional, bursty, or event-driven (a queue message, a file upload, a button click). Choose an always-on container when it’s high-volume, latency-sensitive, or streaming long responses. Many systems use both — Functions for the spiky bits, containers for the hot path.

Note: Azure Functions APIs and plan limits change; verify against the Azure Functions docs. The design — HTTP-triggered function, injected IChatClient, managed-identity auth, mind cold starts and timeouts — is stable.

Takeaway

Azure Functions gives an AI endpoint serverless economics: scale to zero, pay per use, no servers. Inject an IChatClient, authenticate with a managed identity so there’s no key, and mind cold starts and execution timeouts. For bursty, event-driven AI features it’s the cheapest, simplest home — and it pairs naturally with always-on containers for the traffic that needs them.

Have a correction or a topic you want covered? Email mani.bc72@gmail.com.

Related articles