Repository navigation
ci(prd-481): CI does each piece of work once - #405
Merged
Merged
Conversation
…ths they read Phase 3 of PRD-481. The audit behind it cost 2.4k runner-minutes on pipeline-cache alone, all of it on a branch whose lane had landed three weeks earlier. - pipeline-cache.yml: `push.branches` loses feat/prd-368-persistent-pipeline-cache. - integration-csg.yml: the unfiltered `push` beside `pull_request` is gone, so a commit with a PR no longer runs the proof twice. - integration-decals.yml: the per-branch `push` for fix/vq11-decal-material-lifetime is gone; PR #394 merged on 2026-10-02 and the branch outlived it. - every integration lane now skips draft pull requests, and every one of those triggers gained `ready_for_review` so the draft guard cannot silence the lane instead of cheapening it. - ci-structure.spec.ts: a workflow-file allow-list read off disk (a new workflow has to be named here to land), the push+pull_request and per-branch rules with fixture cases, and an assertion that pipeline-cache fires only on paths its own steps read. Both new assertions fail on the pre-change workflow files and pass after.
…ts; native cache key drops ci.yml Nine jobs each compiled the workspace dist (71-95 s) because they started in parallel with its producer. They now need build-artifacts and download its upload. The native-build cache key no longer hashes ci.yml, so a workflow edit stops forcing a cold 1.35 GiB Linux host build.
Phase 2's third box. Both lanes shard by a rule that knows nothing about duration, so
the run's slowest leg is decided by arithmetic nobody measured. Timestamped on run
37049488719.
test-unit, 3 -> 4 slices. `vitest --shard` sorts the file list by sha1 of its path and
slices that order, so the split is exact about file count and blind about time. The
three legs ran 163s / 285s / 472s of vitest over 397s / 712s / 709s of test time.
Replaying vitest's own partition over those per-file durations reproduces all three
assignments, and at four slices predicts 80s / 114s / 211s / 320s of vitest — three
legs inside six minutes once the 40-121s of job overhead is added.
The fourth leg cannot be, and no shard count fixes it: it carries
`create-threenative/__tests__/template-assets-compile.spec.ts` at 320s on its own, and
a shard cannot split a file. A fifth slice leaves both numbers unchanged and buys a
runner.
template-nonvisual. Per-scenario seconds on the same run, with the ~65s each leg spends
on its own setup:
shooter 26 at 32-47s, 922s total -> 4 -> 363s, 5 -> 356s, 6 -> 288s. Six.
sailing 6 at 17/163/23/75/22/44 -> 2 -> 347s.
rain 5 at 331/202/222/78/61 -> 5 -> 396s, and that is its floor: five scenarios,
one per slice, heaviest alone 331s of software rendering. It cannot reach six
minutes at any shard count, and it is not split past its scenario count to pretend
otherwise.
The box stays unticked: both lanes still have a leg over six minutes that no shard
count reaches, and its proof is a full CI run's per-job durations.
ci-structure.spec.ts holds the measured template counts (puzzle 2, sailing 2, rain 5,
shooter 6) beside the partition and scenario-count guards it already had.
196 passed; the three new pins fail on the pre-change workflow.
The board is chosen from a whole-repo tree SHA rather than from the diff. When the
candidate's tree is the tree a completed, successful `CI` run already tested, `scope`
emits `selection: reused` with that run id, every gate skips, and `ci-required` passes
only after reading the source run's own `ci-required` conclusion through the Actions
API.
- `scripts/ci-change-scope.mjs`: `findReusableRun` lists the recent successful CI runs
(`gh api`, the job's own `GITHUB_TOKEN`, `actions: read`), compares each run's
`git rev-parse <head_sha>^{tree}` with the candidate's, and fetches a commit this
clone cannot see rather than assuming it. A miss, an unreadable or uninterpretable
listing, `workflow_dispatch`, an explicit `--full`, a non-Actions run, and Sunday's
nightly all keep the full board; the miss reason is appended so a full run that never
looked cannot read like one that looked and found nothing.
- `scripts/ci-required.mjs`: a reused verdict resolves the source run by id and fails
with `CI_REQUIRED_SOURCE_UNKNOWN` / `CI_REQUIRED_SOURCE_NOT_SUCCESS` unless that run's
own ci-required job concluded success.
- `.github/workflows/ci.yml`: `merge_group` trigger (types `[checks_requested]`); the
draft guards stay true for a merge group because it carries no pull_request object;
`lint` and `supply-chain` now name the two selections they admit instead of refusing
`prose`, which would have read a reused tree as "not prose" and re-run them.
- `AGENTS.md`: "Never cache test verdicts" becomes the 2026-10-02 decision wording, and
the CLAUDE.md mirror is regenerated.
Verification: `pnpm exec vitest run scripts/__tests__/ci-needs.spec.ts` 25 passed (4 of
the 5 new cases red before the change); the five-spec lane (ci-structure, ci-needs,
ci-efficiency, sync-agent-docs, primary-docs) 218 passed; `pnpm lint` exit 0;
`pnpm typecheck` exit 0; `pnpm check:docs` 2496 links clean; `pnpm sync:agents --check`
in sync. The CI-run boxes stay unticked: nothing here has run on GitHub yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An identical tree is not the same validation. PRD-380 gives an ordinary pull request a
reduced native matrix and the planner already lets a clean develop pull request skip
`native-platforms`, so the previous contract — identical tree plus a green source
`ci-required` — let a develop pass satisfy a promotion on that tree. Reuse now requires
the source run to cover what this run proves, and any gap fails closed into the full
board.
The profile is a pure function of the run's event, its base branch and whether it owed
the native lane, and it is computed the same way for the source and for this run:
- `validationProfile` returns `{ target, native }`. `target` is `main` for a promotion or
a main push, `develop` for an ordinary pull request or the merge queue, and `other`
for anything whose target the API does not report — which ranks lowest, so an
unproven target never stands in for a proven one. `native` is `none`, `reduced` or
`full`, following the target.
- A run record in this repository reports no `base_ref`, so `sourceBase` reads a source's
target from the pull requests its commit belongs to (`commits/<sha>/pulls`); nothing
readable is no base.
- `currentRun` reads this run's own job graph (`actions/runs/$GITHUB_RUN_ID/jobs`) for its
required set — every board job and matrix leg its job graph carries, minus the ones the
plan waived, since a waived job is still listed as a skip — and for the runner class
its routing chose.
- `coverageMiss` refuses a source whose profile is weaker on either axis, that never ran
a required leg, whose leg concluded anything but `success`, or whose jobs ran on
another runner class (`hosted` or `tn-local`, from the labels and `runner_name` the
jobs API reports). A reusable workflow's `native-platforms` arrives as one check, so
the lane's own legs stay inside it.
- A `workflow_dispatch` is now excluded as a reuse source as well as a reuse target, and
a truncated job listing is a miss rather than a quietly shorter requirement set.
`ci-required` repeats the coverage check with its own reading of the source run, so a
reuse it cannot confirm is not a pass; it re-derives only what policy owes each target,
because a reuse plan records nothing about what was required.
Verification: `pnpm exec vitest run scripts/__tests__/ci-needs.spec.ts` 30 passed (5 red
before the change: the four coverage cases and the reused verdict); the five-spec lane
(ci-structure, ci-needs, ci-efficiency, sync-agent-docs, primary-docs) 223 passed;
`pnpm lint` exit 0; `pnpm typecheck` exit 0; `pnpm check:docs` 2496 links clean;
`pnpm sync:agents --check` in sync. The CI-run boxes stay unticked: nothing here has run
on GitHub yet.
Five workflow files each carried their own triggers, permissions and concurrency group for one feature's proof. The owner's rule is that no feature gets a workflow file: its proof is a job in an existing workflow. `integration-csg`, `integration-decals`, `integration-ik`, `integration-tone` and `integration-vegetation` fold into `.github/workflows/integration.yml`, one job per feature, each with the paths filter, draft guard, runner, checkout ref, timeout and artifact names it had before. - `.github/workflows/integration.yml`: one `paths` gate job reads the pull request's own changed paths once with `dorny/paths-filter` (pinned to the v4.0.3 commit) and publishes one output per lane, so a lane this commit does not touch is skipped rather than given a runner, and editing the workflow still re-runs every lane (each filter names it, as each old trigger did). The csg and decals supersession groups become one; every step, timeout, runner line and artifact name is unchanged. - `scripts/__tests__/ci-structure.spec.ts`: the workflow allow-list names one integration workflow and rejects a new `integration-*.yml` by name; a second assertion reads the merged file and requires one filter per lane, a job behind each gate, and the draft guard on every gated job. - `scripts/workspace-packages.ts`: the hand-listed-package gate counted distinct package names per *file*, so merging five files put two unrelated targeted `--filter` builds in one file and tripped it. It now counts per `run:` block — one command or one run block naming several packages is the drift this gate exists for; two targeted builds in two unrelated jobs are two builds — with a case for each direction. Verification: `pnpm exec vitest run scripts/__tests__/ci-structure.spec.ts scripts/__tests__/ci-needs.spec.ts scripts/__tests__/ci-efficiency.spec.ts scripts/__tests__/sync-agent-docs.spec.ts scripts/__tests__/primary-docs.spec.ts` 223 passed (131 in ci-structure, whose two new assertions were each red first: a lane without `needs: paths`, a lane gate without the draft guard, and an added `integration-newlane.yml` for the allow-list); the whole `scripts/__tests__` suite 1638 passed, 1 skipped; `pnpm lint` exit 0; `pnpm typecheck` exit 0; `pnpm budgets` ok; `pnpm check:docs` 2496 links clean; `pnpm sync:agents --check` in sync. For the owner: three checks change name (`test` becomes `csg`, `ik`, `vegetation` — they were three indistinguishable checks), a ~30-second gate job now appears on every pull request, and the three lanes that queued behind a superseded run are cancelled by it instead. Branch protection naming those checks has to follow.
…d-party action dorny/paths-filter was new to the repo for what one gh api call and a regex per lane do. Each lane's former paths globs translate one-to-one; spot-checked hits and misses per lane.
jonit-dev
marked this pull request as ready for review
October 2, 2026 22:15
Run 37071464562, test-unit (3/4): inside CI the workflow-only classification case turned on the live tree-reuse lookup and its reason grew an Actions API error. isolatedGitEnvironment now strips GITHUB_ACTIONS; red under GITHUB_ACTIONS=true, then 131 passed. PRD proof markers moved onto each box first line.
Phase 2's third box, measured on run 37071464562: `test-unit (4/4)` ran 481s and
`template-nonvisual (sailing, 2/2)` 467s against a six-minute leg. `rain (3/5)` 358s and `rain
(2/5)` 299s are the rest of it — every other unit and template leg was under 360s. Two changes,
each from a timestamp rather than a preference.
`template-nonvisual`. Sailing's six non-visual scenarios measured 55/247/30/98/28/59 seconds in
classifier order (fail, float-after-the-run-ends, float, sails, touch-controls, win). Two slices
put `float-after-the-run-ends` alone with `sails` and `win` — 404s of scenario work, and the same
run attributes ~63s of overhead to a slice, which is the 467s it measured. Three slices put one
heavy scenario on each and the worst at 275s of work: ~335s predicted, and 3 is the smallest count
that clears it. Six scenarios, three slices, nothing dropped.
`test-unit`. The fourth leg carried `create-threenative/__tests__/template-assets-compile.spec.ts`
at 317s on its own. `vitest --shard` sorts the file list by `sha1(path)` and slices that order: it
divides a list of files, never a file, so no shard count reaches that 317s. The spec is now two
files over the two halves of the template list, with the cases in
`template-assets-compile-cases.ts` beside them — the same 40 tests, the same names, the same
assertions, all 13 templates, and the one whole-suite assertion kept once instead of copied into
both, with `TN_TEMPLATE_ASSETS_HALF_EMPTY` failing closed where a slice could come out empty.
Replaying vitest's own partition over that run's per-file durations, which reproduces its four legs
exactly (261/429/554/630s of test time), predicts 180/240/224/137s across the four legs.
The partition is still blind to duration: the two halves' hashes land them in the two lightest
legs, and renaming either file reshuffles that. Five slices would predict a 223s worst leg for one
more runner; four is what the measurement needed.
- `.github/workflows/ci.yml`: sailing 2 -> 3 slices, with the per-scenario seconds that chose it.
- `packages/create-threenative/__tests__/template-assets-compile-cases.ts` (new): the templates,
the two halves and the three cases, once.
- `packages/create-threenative/__tests__/template-assets-compile-{1,2}.spec.ts` (new),
`template-assets-compile.spec.ts` (deleted).
- `scripts/__tests__/ci-structure.spec.ts`: the measured sailing pin, 2 -> 3.
- `docs/PRDs/CI/PRD-481-ci-does-each-piece-of-work-once.md`: the overs from 37071464562 and what
this commit changes under the shard box, which stays unticked — its proof is a full CI run.
Verification: `pnpm exec vitest run packages/create-threenative/__tests__/template-assets-compile-1.spec.ts
packages/create-threenative/__tests__/template-assets-compile-2.spec.ts` 40 passed in 2 files
(120.8s and 142.7s of file time), against the 40 the unsplit file measured in 270.6s;
`pnpm exec vitest run scripts/__tests__/ci-structure.spec.ts scripts/__tests__/ci-efficiency.spec.ts
scripts/__tests__/ci-needs.spec.ts` 208 passed; `pnpm lint` exit 0; `pnpm typecheck` exit 0. That
run's `test-unit (3/4)` failure was `ci-structure.spec.ts > classifies workflow-only changes as
full CI with no prose skip`, which passes here and is untouched by this change.
Pushing then marking ready fires a synchronize run whose payload still says draft; in the shared 'latest' group with cancel-in-progress it skipped every job and cancelled the real ready_for_review run (#403, run 37075825121). Draft runs now get a per-run group. New ci-structure case red first; PRD-481 dist-once box ticked.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR: CI does each piece of work once. One job builds the workspace dist; shards are sized under 6 min; triggers fire once; the integration lanes fold into one workflow; a tree reuses a prior verdict only when that run covered the same validation;
merge_groupis ready for the merge queue. PRD-481.What changed
pull_requestonly, skip drafts, keep their paths.pipeline-cache.ymldrops a dead branch trigger. The fiveintegration-*.ymlbecome oneintegration.yml, gated by onegh apicall (no third-party action). The allow-list rejects new per-feature workflow files.needs: build-artifactsand download its workspace instead of rebuilding it (9 builds of 71–95 s each run). The native cache key dropsci.yml. Shards: shooter 6, rain 5, sailing 2, test-unit 4.merge_grouptrigger.scopereuses a prior verdict only when all of these hold: identical tree, a completed CI source run, a full-board source, every required job and matrix leg passed, an equal-or-stronger validation profile (target tier × native tier), and the same runner class. Anything else runs the full board.AGENTS.mdtakes the reuse wording from the PRD decision.Evidence (local)
ci-needs.spec.ts30 passed: develop-pass-for-promotion → full, missing matrix leg → full, runner-class mismatch → full, one-file tree diff → full, API error → full, dispatch → full. Each case failed before its fix.ci-structure.spec.ts131 passed,workspace-packages.spec.ts17,pnpm lint,check:docs,sync:agents --check.Open (needs real runs)
template-assets-compile.spec.ts(320 s alone).integration.yml.