Skip to content

feat: run the model on a cloud GPU with outfit remote - #20

Merged
outofcoffee merged 11 commits into
mainfrom
feat/remote
Jul 29, 2026
Merged

outofcoffee merged 11 commits into
mainfrom
feat/remote

Conversation

@outofcoffee

@outofcoffee outofcoffee commented Jul 28, 2026 •

Copy link
Copy Markdown
Collaborator

Run a model too big for your laptop on a cloud GPU that only exists while you are using it, driven by the same Outfit file you would use locally.

Summary

  • Adds an outfit remote command group — start, stop, status and deploy — targeting the scale-to-zero endpoint the companion cloud-vm-llm project deploys. It drives that endpoint; it does not create it.
  • Adds a REMOTE Outfit instruction naming the file holding the endpoint's control URLs, resolved relative to the Outfit like PRESET, so a project can carry its own endpoint. Without one, a per-user config is used, so the commands still work outside a project.
  • outfit remote deploy derives what to serve from the Outfit and its preset: PROVIDER picks the engine, MODEL or the preset's hf names the weights, CONTEXT the window, ALIAS the served name, and the preset's remaining flags the engine's arguments.
  • Adds a vllm provider to the catalogue, so both self-hostable engines can be named by PROVIDER and switching between them is a one-line edit.
  • Adds an apiKeyOptional catalogue flag for a provider that works both unauthenticated and authenticated, and marks llamacpp with it — without which applying a remote llama.cpp Outfit produced a keyless config that 401s.
  • Teaches tab completion about nested commands, so outfit remote <TAB> offers its subcommands.
  • Documents the command in the user manual, and records the specification delta as the OpenSpec change add-remote-endpoint-control.

Additive throughout: an Outfit without REMOTE behaves exactly as before, and no existing command changes behaviour.

Implementation details

PROVIDER names the engine; there is deliberately no RUNNER keyword. An Outfit already says llamacpp, and outfit serve already runs llama.cpp from it — the remote case is the same statement pointed at another machine. A separate keyword would allow PROVIDER llamacpp alongside RUNNER vllm, a contradiction worth making unrepresentable. The cost is that a hosted provider cannot be deployed, which is reported as an error rather than left as a gap.

The deploy payload describes intent, not storage. It carries the engine, model, quantisation, context, served name and flags — and deliberately no storage location. The endpoint derives its own layout and fetches missing weights itself, so a change to that layout never becomes an outfit release.

One preset serves locally and deploys unchanged. Rather than a second list of remote flags, the existing preset is reused with the settings the endpoint owns (host, port, model location, API key, context, alias, metrics) removed. Two subtleties this forced: flags must be matched after alias resolution, since ngl and n-gpu-layers are the same setting, and a preset's [*] defaults are a separate layer that must be merged — missing it silently dropped ngl and jinja, which would have served the model on CPU with no tool calling.

internal/remote is the only package that reaches the network or touches AWS. It brings the first non-stdlib runtime dependency outside YAML parsing, and is reached only by this command group; every other command still runs offline with no credentials. Requests are signed with the caller's own AWS chain and need only lambda:InvokeFunctionUrl. deploy is the first request with a body, so the signature covers the payload hash and Content-Length computed from the exact bytes sent.

deploy_url is optional. A config written before deploy existed still works for the other three subcommands; only deploy complains, and it says what to add.

apiKeyOptional exists because adding the key variable alone regressed local use. The Pi harness writes a $VAR reference it resolves at run time, so a keyless local server would have received a reference to an unset variable — which makes Pi hide the provider's models. Falling back to the placeholder globally was not an option either, since for other providers that reference is deliberately resolved later. The flag marks the one case where an unset variable genuinely means "no key".

Also in this PR

  • The deployment itself, under remote/: the CDK application that outfit remote drives — an endpoint holding no instance until asked for one, launching into whichever zone has GPU capacity, and terminating itself once unused. Weights live in object storage at a location derived from engine, model and quantisation, fetched automatically when absent.
  • start reports progress. The endpoint blocks until the model is serving, so a cold start used to sit silent for minutes. Progress goes to stderr and only the exports to stdout, so eval "$(outfit remote start)" needs no filtering.
  • Keys are never written to disk. opencode's config carries an {env:VAR} reference rather than the secret, matching how Pi has always worked, and outfit harness passes the keys it can resolve to the agent it launches. A .env is read beside the Outfit being applied — it previously resolved through a build-machine path an installed binary could never find.
  • Applying a config that cannot authenticate now says so, instead of succeeding and failing later as a rejected request.
  • This repository is public, so no identifier of a deployment may be committed: scripts/check-no-cloud-identifiers.sh fails the build on account ids, ARNs, endpoint hostnames, public addresses, bucket names and resource ids, and runs first in CI.
  • CI: a separate Remote deployment workflow typechecks, tests and synthesizes remote/ when it changes.
  • Specification: the OpenSpec change is applied to the main specs and archived — four new capabilities (remote-endpoint, endpoint-lifecycle, inference-runners, model-weights) plus the requirements this work altered across seven existing ones.

outfit remote start|stop|status drives the scale-to-zero inference
instance from the cloud-vm-llm stack, via SigV4-signed calls to its
Lambda Function URLs using the standard AWS credential chain.
An Outfit can name its remote-instance config with `REMOTE <file>`,
resolved relative to the Outfit like PRESET. outfit remote
start|stop|status prefer it over the per-user remote.json, so a project
can carry its own endpoint config.
PROVIDER carries the runner, so a vLLM deployment needs its own entry
rather than borrowing openai-compatible — that keeps the llama.cpp/vLLM
cut-back symmetric and lets the provider name say which runner is meant.
@outofcoffee outofcoffee added documentation Improvements or additions to documentation enhancement New feature or request labels Jul 28, 2026
PROVIDER already names the engine, so an Outfit that says llamacpp asks
for llama.cpp — locally under `outfit serve`, in the cloud under `outfit
remote deploy`. The Outfit and its preset are the single source of
truth: MODEL or the preset's hf gives the weights as repo:QUANT, CONTEXT
or ctx-size the window, ALIAS the served name, and the preset's
remaining flags become the runner's serveArgs. Flags the cloud sets
itself are dropped, matched on canonical names so short aliases count,
and the preset's [*] globals are layered under the section so settings
written there (ngl, jinja) survive. Deploy is the first call with a
body; SigV4 covers the payload hash, so the bytes are hashed and signed
rather than assumed empty.
PROVIDER llamacpp now names the remote deployment as well as a local
server, and the remote one is started with --api-key — but the catalogue
entry had no apiKeyEnv, so applying such an Outfit produced a keyless
config that 401s.

Adding the key env alone would have regressed local use under Pi, which
writes a $VAR reference resolved at run time: a local server needs no
key, so the reference would resolve to nothing and Pi hides models with
unusable auth. apiKeyOptional marks that case, keeping Pi's keyless
placeholder when the var is unset while every other provider keeps its
reference.
The docs directory is a user manual, so the remote command needed a page
of its own alongside the others, and the Outfit guide's REMOTE example
still showed the shape from before PROVIDER selected the engine.

The OpenSpec change records the specification delta this branch
introduced: a new remote-endpoint capability, and the requirement
changes it forced on outfit files (the REMOTE keyword), the provider
catalogue (an optional API key), Pi integration (when the keyless
placeholder applies), and shell completion (subcommands).
The start endpoint blocks until the model is serving, so the only
progress callback — the one for a 503 retry — never fired on the path
that actually takes minutes: a cold start sat silent and looked hung.

It now says what it is doing up front and repeats with the elapsed time
while it waits. Progress goes to stderr and stdout carries only the
exports, so the output stays pipeable and `eval "$(outfit remote
start)"` needs no filtering.
Neither harness needs the secret in its config: opencode substitutes
{env:VAR} and Pi resolves $VAR, both when they run. Writing the resolved
key into opencode's config was justified by a global config being unable
to rely on a project-local .env — so the agent outfit launches is now
given the keys outfit can resolve, and that justification goes with it.

A .env is read beside the Outfit being applied, the same rule PRESET and
REMOTE follow, falling back to the working directory for a selection
made from flags alone. It previously resolved through runtime.Caller,
which records the path of the machine that built the binary, so an
installed outfit could never find one.

Applying a config for a remote endpoint with no resolvable key used to
succeed and then fail on the first request; it now says so, naming the
variable. A provider whose key is optional keeps Pi's keyless
placeholder only for a local endpoint, since a placeholder written for a
remote one could not be repaired by setting the variable afterwards.
The deployment outfit remote drives lived in a separate private
repository, which meant the command and the thing it commands were
versioned, reviewed and released apart. It is now a first-class part of
this project: a TypeScript CDK subproject under remote/, with its own
tests run by CI.

This repository is public, unlike the one it came from, so nothing
identifying a deployment may be committed.
scripts/check-no-cloud-identifiers.sh fails the build on an account id,
ARN, Function URL host, public address, bucket name or resource id, and
runs before anything else in CI. Deployment state stays in the
gitignored files under remote/.

Logical stack names, tag keys and SSM paths are kept as they are: they
are chosen in source and identify the deployed resources, so changing
them would orphan the running stack.
The CDK project's assertions, typecheck and a full synth now run on
their own, triggered only when remote/ changes rather than on every Go
pull request. The synth covers what the assertions cannot — bin/app.ts
wiring and esbuild bundling each Lambda — and needs no credentials,
since the stacks take their availability zones from config rather than
looking them up.

The change's OpenSpec artifacts are brought back in line with what the
branch became: start reporting progress, the missing-key warning, keys
referenced rather than embedded, and the deployment moving into this
repository under the public-repo identifier rule.
@outofcoffee
outofcoffee marked this pull request as ready for review July 29, 2026 01:34
The endpoint, its engines and its weights were described only by the code that implements them, so the rules that matter most — when an instance is torn down, what bounds a running one, where weights are looked for — were whatever the tests happened to assert. They are now requirements: an idle policy that could be changed plausibly and wrongly is the difference between minutes and hours of GPU time.

The change's deltas are applied to the main specs and the change is archived; four capabilities are new, the rest gain the requirements this work altered.
@outofcoffee
outofcoffee merged commit baf76c0 into main Jul 29, 2026
2 checks passed
@outofcoffee
outofcoffee deleted the feat/remote branch July 29, 2026 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant