Skip to content

feat(gateway): measured switch costs, /v1/berth/cost and /v1/berth/plan - #102

Merged
T-Crypt merged 6 commits into
mainfrom
T-Crypt/strata-s2-swapcost
Oct 21, 2026
Merged

T-Crypt merged 6 commits into
mainfrom
T-Crypt/strata-s2-swapcost

Conversation

@T-Crypt

@T-Crypt T-Crypt commented Oct 21, 2026

Copy link
Copy Markdown
Owner

OME QCPJEEG. Stacked on #101: merge #101 first, then retarget this to main.

  • The gateway times every start it runs, from the grant line its run prints (when the card showed free) to the engine's first answer. Juggle and drain waits are excluded. The time goes into a swapcost.Book under the tier read when the start is decided, before prefetch warms anything. The book lives in ~/.local/state/berth/swapcost.json. Failed starts and adopted engines record nothing.
  • GET /v1/berth/cost?engine=NAME: one estimate per tier; known:false means no samples, never a guess.
  • POST /v1/berth/plan: swapcost.Plan for a declared chain, with a lower-bound total, the unmeasured count, per-step prefetch and keep_parked. 400 on an empty chain, an unknown tier or an unknown engine.
  • Same key and Origin guard as the other /v1 routes. Responses use the gateway's own snake_case JSON views (swapcost types carry no tags).
  • Docs: the README examples use the median of five measured Strata loads (72.4 s) and say the gateway cannot time llama-swap's models.

Proof: build, gofmt, vet, go test ./..., go test -race -count=3 ./internal/gateway, GOOS=windows build and vet, golangci-lint v2.14.0: all green locally.

…swap book

The gateway owns one swapcost.Book (CostFile, empty keeps it in memory)
and records the switch the package defines: from the lease grant its run
printed to the engine answering, never from the request, so a wait
behind the holder or the juggle guard is not billed to the engine. The
tier comes from the position read when the start is decided: parked from
the lease table, cool or cold from the page cache's share of the
engine's prefetch files. Failed starts record nothing; book load and
save errors log WARN and the gateway keeps going.
…ed switch times

One estimate per tier, known false where the book has no samples, so a
figure it gives is one it measured. An engine berth.toml does not name
is a 404 and a missing engine parameter a 400, behind the same key and
Origin checks as the other /v1 routes.
The body names one step per engine the orchestrator plans to switch
through, with the tier it expects to find each in. The answer carries
the per-step estimate, the lower-bound total with its unmeasured count,
the per-step prefetch, and the engines worth keeping parked. A bad
body, an empty chain, an unknown tier or an engine outside the config
is a 400 naming what is wrong.
…ecorded sample

Table-driven httptest for the two routes: known and unknown estimates,
hot as the free switch, the 404 and every 400 naming what is wrong, and
the key and Origin refusals. One fake start through the test daemon and
the fake runner: exactly one sample, on the cold tier, and elapsed
minus took shows the queue before the grant stayed out of the sample.
README gateway section: what a switch costs, one curl example per
route. CHANGELOG entry under Unreleased. The gateway half of the
v0.3.x list item is done, the live-daemon half stays.
@T-Crypt
T-Crypt changed the base branch from T-Crypt/strata-s1-prefetch to main October 21, 2026 13:09
@T-Crypt
T-Crypt merged commit 1eb68dd into main Oct 21, 2026
2 checks passed
@T-Crypt T-Crypt mentioned this pull request Oct 21, 2026
@T-Crypt
T-Crypt deleted the T-Crypt/strata-s2-swapcost branch October 21, 2026 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant