Deploy

How ai-agent-eval-harness-healthtech reaches production, described at the architecture level. Production runs on Google Cloud Run (a single container instance, region us-central1) built from the same image the project’s Dockerfile produces. The deploy is a source-based Cloud Build, keyless via Workload Identity Federation, gated on the CI tag-guard, and secret-preserving. The exact operator commands live in the private operator command reference; this page is the public reference for the shape of the deployment.

Why Cloud Run

The production target is a public, always-reachable demo built from the exact image the project ships. Cloud Run, running a single container revision, gives us:

A managed HTTPS endpoint mapped to a custom domain for the live demo.
Scale-to-zero when idle and a single instance under load, so demo-scale traffic stays inside Cloud Run’s always-free allowance and real spend stays near $0 / month.
A keyless deploy: the CI job mints a short-lived Google Cloud token over Workload Identity Federation (OIDC); no service-account key is stored.
A source build that rebuilds from the shipped Dockerfile and preserves the service’s existing environment and Secret Manager bindings.

The single-instance ceiling is deliberate, not a limitation of the host: the rate limiter, the response cache, and the paused human-in-the-loop checkpointer are all in-process, so a second instance would not share them. A multi-instance deployment would need a shared store (Redis, Postgres), which is out of scope for the demo.

Deploy pipeline

The deploy is server-side and gated on a green CI matrix; no operator ships a production release by hand.

A release tag matching vX.Y.Z is pushed.
The tag-guard workflow re-runs the full CI matrix against the exact tagged commit.
The deploy workflow runs only after tag-guard completes, and only when that run concluded successfully on the tag. A red CI matrix means the deploy is skipped and nothing ships.
The deploy job checks out the exact validated commit, authenticates to Google Cloud keyless via Workload Identity Federation (OIDC), and triggers a source-based Cloud Build that rebuilds the image from the shipped Dockerfile and rolls out a new Cloud Run revision.
Because the rebuild is from source, it preserves the environment variables and Secret Manager bindings already on the service. Traffic shifts only once the new revision is healthy; a failed revision keeps traffic on the prior healthy one.

One-time setup

The keyless deploy rests on a one-time Google Cloud trust relationship, after which every release flows through the pipeline above:

Workload Identity Federation. A federation pool and provider are established for the repository, together with a least-privilege deployer service account the pool may impersonate (scoped only to what a Cloud Run source deploy needs). The provider and deployer identity are referenced from non-secret repository variables; no service-account key is ever created or stored.
Secret Manager. The completion-provider credentials - OpenAI (primary) and Anthropic (fallback and evaluation judge) - and the integration secrets (managed embeddings/reranking, the demo data layer, voice, and the edge bot check) are held in Google Secret Manager and bound to the service. Because the deploy rebuilds from source, these bindings persist across every subsequent deploy.

Runtime shape

The service reads its configuration from Cloud Run environment settings and Secret Manager bindings (the application settings map each field to an upper-cased environment variable):

Generation runs a two-provider cascade: OpenAI gpt-4o-mini (primary) falling back to Anthropic claude-haiku-4-5 on a transient upstream failure (see ADR-0002). The evaluation judge is Anthropic claude-haiku-4-5.
Embeddings default to the managed Voyage embedder when a Voyage key is present and fall back to the baked-in BAAI/bge-small-en-v1.5 otherwise, so the service keeps working with no embedding key.
Conversation state uses the in-memory MemorySaver by default (a paused HITL thread does not survive a revision restart); a durable Postgres-backed checkpointer is selected automatically when a Postgres connection string is configured.
Resilience primitives - the sliding-window rate limiter, the response cache, and the provider fallback - are per-process, which is why the service runs a single instance.
The baked-in knowledge base is ingested into an empty Chroma collection on first boot.

Streaming

The Agent Execution Graph is fed by a server-sent-events (SSE) streaming mode on /chat and /chat/resume: a request that accepts text/event-stream receives per-node execution events instead of a single JSON response. The streaming design is recorded in ADR-0009.

For the live graph to feel live, those events must reach the browser incrementally - as each node runs - rather than being buffered and delivered as one block at end of turn. The application flushes each record as the LangGraph stream yields it, but whether the edge preserves incremental delivery is a deployment property that is verified against the running revision, not assumed: a release gate confirms the records arrive spaced across the turn (graph_topology, then node_started / node_completed, then turn_completed) rather than collapsing into a post-turn dump. A release does not claim a live execution graph unless the running service demonstrably streams incrementally.

Rollback

Cloud Run retains every deployed revision, so rollback is a traffic operation, not a rebuild:

Roll back by shifting all traffic back to the prior healthy revision; this is instant and needs no CI run.
Roll forward a corrected build by reverting the offending commit on the default branch and cutting a new vX.Y.Z tag, which re-runs tag-guard and, on green, redeploys.

A failed revision never takes traffic: Cloud Run keeps serving the prior healthy revision until the new one is ready.

The image

Cloud Run runs the same Dockerfile the project ships - the image is identical in local development, CI, and production. It:

binds uvicorn to $PORT (Cloud Run injects 8080; local and Hugging Face runs default to 7860), so one image serves on whatever port the host provides,
runs as a non-root app user on a python:3.12-slim base with only ca-certificates and curl added,
writes only to /tmp and the app virtualenv, and requires no GPU.

Secondary hosts

The same image runs unchanged on other Docker-capable hosts; only the port and platform wiring differ:

Hugging Face Spaces (Docker SDK, CPU Basic free tier): a genuinely zero-cost, always-on secondary demo from the identical image, serving on port 7860 with the same baked-in knowledge base.
Render (free tier, untested): the same Dockerfile should run when the service is configured to listen on the Render-provided $PORT; this path is documented but not exercised.
Local Docker: the same image runs locally for development and for sanity-checking before a release tag.