Running a .NET AI Agent on Google Kubernetes Engine (GKE)

Azure Container Apps is the fastest path to hosting a .NET agent, but plenty of teams live on Google Cloud, already run GKE, and want their agent alongside the rest of their microservices — same cluster, same mesh, same observability. This guide deploys a .NET agent to Google Kubernetes Engine the production way, with a focus on the two things people get wrong: identity and scaling.

It assumes you have an agent exposed over HTTP, like the one from our tool-using agent tutorial. For the Azure equivalent, see deploying to Azure Container Apps.

Step 1: Containerize and push to Artifact Registry

Use the same multi-stage Dockerfile you’d use anywhere, then push to Google Artifact Registry (GKE’s native registry):

gcloud artifacts repositories create agents \
  --repository-format=docker --location=us-central1

docker tag agent-demo \
  us-central1-docker.pkg.dev/PROJECT_ID/agents/agent-demo:v1
docker push us-central1-docker.pkg.dev/PROJECT_ID/agents/agent-demo:v1

Step 2: Identity — the part that matters most

The instinct is to create a service-account key JSON, drop it in a Kubernetes Secret, and mount it. Don’t. Exported keys are the most common cloud-credential leak, and they’re unnecessary on GKE.

Use Workload Identity instead: it lets a Kubernetes service account impersonate a Google service account, so your pod authenticates as itself with short-lived, automatically-rotated credentials — no key files anywhere. If your agent calls Vertex AI (or any Google API), this is how it should get its credentials.

# Bind the Kubernetes SA to a Google SA
gcloud iam service-accounts add-iam-policy-binding \
  agent-gsa@PROJECT_ID.iam.gserviceaccount.com \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:PROJECT_ID.svc.id.goog[default/agent-ksa]"

Then annotate the Kubernetes service account so the binding takes effect:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: agent-ksa
  annotations:
    iam.gke.io/gcp-service-account: agent-gsa@PROJECT_ID.iam.gserviceaccount.com

Grant agent-gsa only the roles the agent needs (for Vertex AI, roles/aiplatform.user). In .NET, the Google client libraries pick up these credentials automatically — your code contains no keys.

Step 3: Deployment and Service

A standard Deployment referencing the service account, with health probes and sane resource requests:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: agent-demo
spec:
  replicas: 2
  selector: { matchLabels: { app: agent-demo } }
  template:
    metadata: { labels: { app: agent-demo } }
    spec:
      serviceAccountName: agent-ksa
      containers:
        - name: agent
          image: us-central1-docker.pkg.dev/PROJECT_ID/agents/agent-demo:v1
          ports: [{ containerPort: 8080 }]
          readinessProbe: { httpGet: { path: /health, port: 8080 } }
          resources:
            requests: { cpu: "250m", memory: "256Mi" }
            limits: { cpu: "1", memory: "512Mi" }
---
apiVersion: v1
kind: Service
metadata: { name: agent-demo }
spec:
  selector: { app: agent-demo }
  ports: [{ port: 80, targetPort: 8080 }]

The /health readiness probe matters: it keeps traffic from hitting a pod before the agent has warmed up its model client.

Step 4: Expose it

Point an ingress at the Service. On GKE you can use the built-in GCE ingress, or — if you’re already running a service mesh like Istio — route the agent through your existing gateway so it inherits your mTLS, routing, and policy. Putting the agent behind the same mesh as your other services is the whole reason to choose GKE over a standalone container platform; don’t bolt on a separate ingress if you already have one.

Step 5: Autoscale on the right signal

Add a Horizontal Pod Autoscaler, but scale on the metric that reflects agent load:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: agent-demo }
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: agent-demo }
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource: { name: cpu, target: { type: Utilization, averageUtilization: 65 } }

CPU works for synchronous endpoints, but agent calls are I/O-bound (waiting on the model), so CPU may stay low while requests queue. For those workloads, scale on a custom metric like in-flight requests or queue depth via the Custom Metrics API. And cap maxReplicas to respect your model provider’s tokens-per-minute quota — scaling pods past the quota just relocates the bottleneck, exactly as it would on any platform.

Note: gcloud flags and GKE APIs shift over time; verify against the current GKE and Workload Identity docs. The architecture — Workload Identity over keys, health probes, mesh-native ingress, quota-aware autoscaling — is the durable part.

Takeaway

GKE is the right home for a .NET agent when it belongs with the rest of your Google Cloud microservices. Get two things right and the rest is standard Kubernetes: authenticate with Workload Identity so no keys ever exist, and autoscale on a signal that reflects real agent load within your model quota. Do that, and your agent is just another well-behaved workload in your cluster — observable, secured by your mesh, and scaling with demand.

---

Have a correction or a topic you want covered? Email mani.bc72@gmail.com.