Skip to content

feat(cli): deploy to Cloud Run GPUs with amicu apply - #361

Merged
andypalmi merged 3 commits into
mainfrom
feat/cloud-run-apply
Oct 26, 2026
Merged

andypalmi merged 3 commits into
mainfrom
feat/cloud-run-apply

Conversation

@andypalmi

@andypalmi andypalmi commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Summary

Wires the Cloud Run GPU library (amicu-gcp cloudrun) into the CLI: backend = "cloud_run" in a gcp {} hardware block now works with plan, apply, destroy, refresh and orphans, alongside Compute Engine.

  • Supervisor (amicu-gcp/src/cloudrun/supervisor.rs): CloudRunRecord (project, region, service, owner marker, secret ids) is written to state before anything is created. Then the API key (and HF_TOKEN) go to Secret Manager, regional, with accessor granted per secret to the runtime service account. Then the service is created and waited on until a revision is ready and /health answers. A re-apply tears down the recorded service first, because create adopts an owned service without changing it.
  • Teardown: the service is deleted only if it carries the owner label, a follow-up read must return 404, and then the secrets are deleted and confirmed. Only after that does the record leave state. A crash between the record and the create is cleaned up by destroy.
  • Auth: public ingress (invokerIamDisabled on our own service, no IAM policy change), vLLM's api_key as the Bearer key. Uses GatewayAuth::None, the existing variant for "the runtime key is the only guard". After the deploy, apply checks the endpoint (an anonymous request is refused, the key is accepted), and on failure it deletes the service and secrets this run created.
  • Shape: L4 (4 vCPU / 16 GiB) or RTX PRO 6000 (20 vCPU / 80 GiB), min instances 0, zonal redundancy off, startup probe on /health with a 30 min budget. scaling { max_replicas } becomes max instances (default 1), and target_concurrency becomes per-instance concurrency.
  • Diagnostics: AMICU-CAP-019 now means "a setting Cloud Run cannot honour": zone, machine_type, provisioning, max_run_duration, boot_disk_gb, allowed_source_cidrs, insecure_metadata_secrets, min_replicas > 0, scaledown_window. AMICU-CAP-016 covers a missing service_account/region and count != 1. AMICU-CAP-017 covers a GPU or region that Cloud Run does not offer (or offers by invitation only).
  • Preflight: one cheap Cloud Run read per project and region (not the Compute Engine project read), before anything is written.
  • plan shows the service name, instance shape, scaling and the GPU price while an instance runs. Provider switches know the cloud-run backend.
  • refresh: a missing service, or one whose revision failed, is marked pending. A service scaled to zero is live. orphans lists owner-labelled services in the Cloud Run GPU regions (the Cloud Run API being disabled means none), and recorded Cloud Run secrets count as tracked.
  • ServiceSpec gains project_id (amicu's project id), so the service name and owner marker use amicu's ids rather than the Google project, like the VM path.
  • examples/gcp-cloudrun-l4 (Qwen3-0.6B on one L4 in europe-west4, scale to zero) and a Cloud Run section on the GCP docs page.

Review follow-ups

  • Preflight: checks the Cloud Run Admin API, the Secret Manager API, and actAs on the runtime service account (reusing fix(providers): GCP preflight, secret teardown and idle shutdown #360's checks), all before any write.
  • Teardown: secrets_unwritten proof as on the VM path: a fresh start that provably wrote no secret can be released when Secret Manager is disabled. Written secrets keep the record.
  • Orphans: a disabled Cloud Run API (ServiceDisabled) means no services.
  • gcp { access = "public" | "iam" }: iam keeps Cloud Run's invoker check. Callers send the identity token in X-Serverless-Authorization and the key in Authorization. The check after deploy then only verifies that anonymous requests are refused. A violation of constraints/run.managed.requireInvokerIam names the policy and suggests access = "iam".
  • Cost disclosure: plan, apply and docs say that any request, even an anonymous one, can start a billed instance on a public service (capped by max instances).
  • Instance size: the profile's memory and cpu.cores are honoured above the GPU minimum; below it is AMICU-CAP-016. idle_shutdown is rejected on Cloud Run (AMICU-CAP-019), and access on a VM.

How tested

  • cargo test -p amicu-gcp -p amicu-engine -p amicu-core -p amicu-cli --locked (with HF_TOKEN unset): all pass. New tests run against an in-memory Cloud Run + Secret Manager API:

    • apply records the service and secrets ahead and stores the key before the service exists
    • a crash between the record and the create (with and without the service having been created) is cleaned up by destroy
    • a failed exposure check tears down the service and secrets
    • destroy deletes the service, confirms the 404, then deletes the secrets
    • destroy refuses a service without the owner label and keeps everything
    • refresh marks a missing service pending
    • re-apply replaces the service
    • plan output
    • orphans
    • the preflight
    • unsupported fields are refused before any write
    • the example validates and plans
  • cargo +stable clippy --workspace --all-targets -- -D warnings, cargo fmt --all --check.

  • Not yet run against Google.

  • PR title is a conventional commit

  • Linked issue above

  • No secrets or private infrastructure details in the diff

backend = "cloud_run" in a gcp {} hardware block now works with plan,
apply, destroy, refresh and orphans. A write-ahead record names the
service and its Secret Manager secrets before anything is created;
teardown checks the owner label, deletes, and confirms the service is
gone before state drops the record. The public service is guarded by the
deployment's api_key (vLLM Bearer key, stored in Secret Manager), checked
after deploy, and torn down when the check fails.
# Conflicts:
#	crates/amicu-cli/src/tests/gcp.rs
#	web/apps/web/content/docs/providers/gcp.mdx
- Preflight Cloud Run launches for the Cloud Run Admin API, the Secret
  Manager API and actAs on the runtime service account before any write.
- Record secrets_unwritten when a fresh start provably wrote no secret, so
  teardown can release that record with Secret Manager disabled; written
  secrets keep the record.
- orphans treats a disabled Cloud Run API (ServiceDisabled) as no services.
- New gcp { access = "public" | "iam" }: iam keeps Cloud Run's invoker
  check (identity token in X-Serverless-Authorization, key in
  Authorization); the requireInvokerIam org policy is named with the fix.
- Plan, apply and docs say that any request can start a billed instance on
  a public service, capped by max instances.
- Honour profile memory and cpu above the GPU minimum; below it is a
  diagnostic. idle_shutdown is Compute Engine only.
@andypalmi
andypalmi merged commit 7fb6238 into main Oct 26, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant