Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
11cafc0
docs(prds): file PRD-550, a change merges within 30 minutes
jonit-dev Oct 9, 2026
f0e8e76
feat(ci): CI speed loop, merge-latency audit and dashboard (PRD-550)
jonit-dev Oct 9, 2026
90dee0c
feat(ci): the merge queue reviews the narrowed plan (PRD-550 phase 2)
jonit-dev Oct 9, 2026
1056b16
docs(ci): refresh the CI speed dashboard with the 2026-10-09 window
jonit-dev Oct 9, 2026
481824f
fix(ci): the velocity cost measurement leaves the merge path (PRD-550…
jonit-dev Oct 9, 2026
a478510
test(ci): the template compile specs get a budget that fits their wor…
jonit-dev Oct 9, 2026
6ec34f5
docs(ci): refresh the CI speed dashboard with iteration 3
jonit-dev Oct 9, 2026
3ca7618
docs(ci): record iteration 4, PR 476 green on the phase-1 head
jonit-dev Oct 9, 2026
7789bf3
docs(ci): hourly refresh, iteration 5
jonit-dev Oct 9, 2026
aa13e02
docs(ci): hourly refresh, iteration 6
jonit-dev Oct 9, 2026
9f36ac3
docs(ci): hourly refresh, iteration 7
jonit-dev Oct 9, 2026
cbd2e66
docs(ci): hourly refresh, iteration 8
jonit-dev Oct 9, 2026
2e100ad
docs(ci): hourly refresh, iteration 9
jonit-dev Oct 9, 2026
f9a01bb
docs(ci): hourly refresh, iteration 10
jonit-dev Oct 9, 2026
f1e868d
docs(ci): hourly refresh, iteration 11
jonit-dev Oct 9, 2026
1310367
docs(ci): hourly refresh, iteration 12
jonit-dev Oct 9, 2026
6639a5b
docs(ci): hourly refresh, iteration 13
jonit-dev Oct 9, 2026
4c1bbd8
docs(ci): hourly refresh, iteration 14
jonit-dev Oct 9, 2026
4d81de7
docs(ci): hourly refresh, iteration 15
jonit-dev Oct 9, 2026
bab3a1d
Merge branch 'develop' into ci/prd-550-merge-in-30
jonit-dev Oct 9, 2026
3cfedfb
docs(ci): hourly refresh, iteration 16
jonit-dev Oct 9, 2026
8257404
test(native): name the subprocess reason when the conformance dry run…
jonit-dev Oct 9, 2026
11a932d
Revert "test(native): name the subprocess reason when the conformance…
jonit-dev Oct 9, 2026
0a4ce84
test(native): name the exit reason when the conformance dry run dies
jonit-dev Oct 9, 2026
42496fb
docs(ci): hourly refresh, iteration 17
jonit-dev Oct 9, 2026
46235b7
Merge remote-tracking branch 'origin/ci/prd-550-merge-in-30' into ci/…
jonit-dev Oct 9, 2026
bafcac5
Merge branch 'develop' into ci/prd-550-merge-in-30
jonit-dev Oct 10, 2026
a8f79b3
fix(ci): retry the attempt receipt read on connection errors
jonit-dev Oct 10, 2026
08f2539
fix(ci): drop the ci-speed-loop skill symlink so PR 476 gets a narrow…
jonit-dev Oct 10, 2026
1652670
test(native): move the conformance dry-run diagnostic out of PR 476
jonit-dev Oct 10, 2026
1c76b95
docs(ci): refresh the unmerged PR476 baseline dashboard
jonit-dev Oct 10, 2026
0d34ddc
fix(ci): fail closed for unproven merge queue targets
jonit-dev Oct 10, 2026
77bb0ee
merge: resolve PR476 against current develop
jonit-dev Oct 10, 2026
d603f28
Merge branch 'develop' into ci/prd-550-merge-in-30
jonit-dev Oct 10, 2026
d6c1af1
fix(ci): reconcile runner routing with live capacity
jonit-dev Oct 10, 2026
8143b4b
fix(ci): pin generic Integration lanes hosted and record routing uptake
jonit-dev Oct 10, 2026
15ef0b9
fix(ci): route native release-reports to the light runner lane
jonit-dev Oct 10, 2026
ec7ec8f
Merge branch 'develop' into ci/prd-550-merge-in-30
jonit-dev Oct 10, 2026
55a07f4
fix(ci): reject stale pull-request candidates before fanout
jonit-dev Oct 10, 2026
803d048
test(ci): use shared timeout for bundle isolation build
jonit-dev Oct 10, 2026
5d6092e
fix(ci): defer exposure capture imports until execution
jonit-dev Oct 10, 2026
6e49570
fix(ci): route native evidence aggregation through light runners
jonit-dev Oct 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .claude/skills/ci-speed-loop/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
name: ci-speed-loop
description: Run ThreeNative's CI speed loop — measure how long a change takes to merge, name the biggest cost from the first failing jobs and queue wait, change one thing, re-measure the same window, record the iteration and refresh the HTML dashboard. Use when asked to speed up CI or the merge queue, work PRD-550, check progress toward the 30-minute merge goal, or refresh docs/ci-speed/index.html. Not for runtime FPS (perf-loop).
---

# CI speed loop

Goal (PRD-550): a change merges within 30 minutes of its last push. One turn is one measured window,
one named cost, one change, one re-measure. Record a turn that did not move the number too, with the reason.

| Metric | Goal |
| --- | --- |
| Last push to merge, median / p90 | ≤ 30 / ≤ 45 min |
| Merge queue passes on the first attempt | ≥ 90% |
| Merge-group board wall time, median | ≤ 15 min |
| Runner queue wait, p90 | ≤ 3 min |

```sh
node scripts/ci-merge-latency.mjs --since 2026-10-09 # measure only, prints the table
node scripts/ci-speed-loop.mjs record --since 2026-10-09 --label "gating board" --notes "..." [--decision keep]
node scripts/ci-speed-loop.mjs show # the ledger
node scripts/ci-speed-loop.mjs discard --iteration 3 --reason "queue outage, 2 PRs only"
node scripts/ci-speed-loop.mjs report # regenerate the HTML only
```

`docs/ci-speed/ledger.json` is the record. `docs/ci-speed/index.html` is generated from it and opens with
`file://`. Never edit either by hand. Commit both with the change they measure.

## The turn

1. **Measure.** `record --since <date the change landed>`. The first iteration is the baseline. Do not
mix PRs from before and after a change in one window.
2. **Name the cost.** Read the "first failing job" chart and the PR table. Name one job or one wait. "CI is slow" is not an answer.
A red that repeats across unrelated PRs is a load red: fix it at its timeout source, not with a retry.
3. **Change one thing.** One PR per PRD (root `AGENTS.md`). Prove it locally first. A CI round trip costs over an hour.
4. **Re-measure the same metrics** once enough PRs merged (the PRD ACs need 10 PRs and 20 merge groups).
A window with fewer than 5 PRs is `provisional`, not `keep`.
5. **Decide.** `keep`, `reject`, `provisional`, `inconclusive` or `invalid`. A contended or outage window is `discard`ed with a reason. It stays in the table and leaves the charts.

## Rules

- A value the API cannot supply is `n/a`, never 0. A PR that skipped the queue has no enqueue time.
- Last push is the head commit's committer date, so a rebase moves it. Say so when a number looks low.
- "First attempt" counts the first merge-group run per PR. Cancelled first runs count as not passed.
- Queue wait is job created to job started over all merge-group jobs, including hosted legs. It is not the gating-only wait until PRD-550 phase 3 routes them.
- Do not claim a goal from one window. Cite the iteration id and its PR count.
32 changes: 18 additions & 14 deletions .github/workflows/integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,8 @@ jobs:
name: csg
if: ${{ needs.paths.outputs.csg == 'true' && !github.event.pull_request.draft }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
# PRD-550: generic lanes run hosted so the local build/test capacity stays free for the heavy jobs.
runs-on: ubuntu-24.04
timeout-minutes: 10
defaults:
run:
Expand Down Expand Up @@ -134,7 +135,7 @@ jobs:
name: ik
if: ${{ needs.paths.outputs.ik == 'true' && !github.event.pull_request.draft }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 10
defaults:
run:
Expand Down Expand Up @@ -166,7 +167,7 @@ jobs:
name: vegetation
if: ${{ needs.paths.outputs.vegetation == 'true' && !github.event.pull_request.draft }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 10
defaults:
run:
Expand Down Expand Up @@ -198,7 +199,7 @@ jobs:
name: decals
if: ${{ needs.paths.outputs.decals == 'true' && !github.event.pull_request.draft }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
Expand Down Expand Up @@ -314,7 +315,7 @@ jobs:
name: exposure
if: ${{ needs.paths.outputs.tone == 'true' && !github.event.pull_request.draft }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
Expand Down Expand Up @@ -358,7 +359,7 @@ jobs:
name: temporal
if: ${{ !github.event.pull_request.draft && needs.paths.outputs.temporal == 'true' }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
Expand Down Expand Up @@ -405,7 +406,7 @@ jobs:
name: velocity
if: ${{ !github.event.pull_request.draft && needs.paths.outputs.velocity == 'true' }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
Expand All @@ -426,7 +427,10 @@ jobs:
- name: Check the exact source-wired fixture types
run: pnpm exec tsc --noEmit -p examples/abyss-framework/velocity.tsconfig.json
- name: Render actual authored batch velocity and missing-history control
run: pnpm exec tsx scripts/verify-velocity-history.ts
# PRD-550 phase 1. The temporal-off cost arm lives in performance-regression.yml: it prices
# actual frame submission against split-control noise, which a runner shared by twenty jobs
# cannot hold, and its `p95NoiseExcessMs` red was other people's load.
run: pnpm exec tsx scripts/verify-velocity-history.ts --variant=history
- name: Retain runtime PNGs, readbacks and exact adapter provenance
if: always()
uses: actions/upload-artifact@v7
Expand All @@ -451,7 +455,7 @@ jobs:
name: animation
if: ${{ needs.paths.outputs.animation == 'true' && !github.event.pull_request.draft }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 25
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
Expand Down Expand Up @@ -575,7 +579,7 @@ jobs:
name: fog capture
if: ${{ needs.paths.outputs.fog == 'true' && !github.event.pull_request.draft }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
Expand Down Expand Up @@ -709,7 +713,7 @@ jobs:
name: capture
if: ${{ !github.event.pull_request.draft && needs.paths.outputs.world-capture == 'true' }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
Expand Down Expand Up @@ -759,7 +763,7 @@ jobs:
name: fluid collision
if: ${{ needs.paths.outputs.fluid-particles == 'true' && !github.event.pull_request.draft }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
Expand Down Expand Up @@ -961,7 +965,7 @@ jobs:
name: auto-exposure
if: ${{ !github.event.pull_request.draft && needs.paths.outputs.exposure == 'true' }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
Expand Down Expand Up @@ -1005,7 +1009,7 @@ jobs:
name: auto-exposure cold boot
if: ${{ !github.event.pull_request.draft && needs.paths.outputs.cold-boot == 'true' }}
needs: paths
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/native-platforms.yml
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ jobs:
name: Native collector evidence coverage
needs: [scope, android-emulator-parity, desktop-parity, desktop, ios-simulator]
if: ${{ !cancelled() && needs.scope.outputs.selection == 'full' }}
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER_LIGHT) && 'ubuntu-24.04' || vars.TN_RUNNER_LIGHT }}
timeout-minutes: 5
env:
TN_CI_SHA: ${{ needs.scope.outputs.candidate_sha }}
Expand Down Expand Up @@ -831,7 +831,9 @@ jobs:
if: >-
needs.scope.outputs.selection != 'prose' &&
inputs.ios_only == false
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER) && 'ubuntu-24.04' || vars.TN_RUNNER }}
# Report assembly only - artifact download, `pnpm install` and JSON reports, no compile or GPU
# leg (unlike desktop-parity:906) - so it joins the light lane instead of queuing behind builds.
runs-on: ${{ (github.event.pull_request.head.repo.fork || !vars.TN_RUNNER_LIGHT) && 'ubuntu-24.04' || vars.TN_RUNNER_LIGHT }}
timeout-minutes: 20
env:
TN_CI_SHA: ${{ needs.scope.outputs.candidate_sha }}
Expand Down
30 changes: 30 additions & 0 deletions .github/workflows/performance-regression.yml
Original file line number Diff line number Diff line change
Expand Up @@ -493,3 +493,33 @@ jobs:
run: |
echo "required-check-promotion requires maintainer review after calibration evidence"
echo "baseline-regeneration is a separate-reviewed-change and never runs in this gate"

velocity-cost:
name: Temporal-off velocity submission cost
# PRD-550 phase 1. This arm prices actual frame submission against split-control noise, so any
# other job on the machine reports its load as render cost. The merge path's integration lane
# therefore runs only `--variant=history`, and this measurement runs on the scheduled lane.
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v7
with:
persist-credentials: false
- uses: ./.github/actions/pnpm
- uses: actions/setup-node@v7
with:
node-version: 20
cache: pnpm
- run: pnpm install --frozen-lockfile
- uses: ./.github/actions/playwright-chromium
- name: Build this commit's public bridge and runner
run: pnpm --filter @threenative/playtest build
- name: Measure the temporal-off CPU submission cost
run: pnpm exec tsx scripts/verify-velocity-history.ts --variant=cost
- uses: actions/upload-artifact@v7
if: always()
with:
name: velocity-cost-${{ github.run_id }}-${{ github.run_attempt }}
path: artifacts/velocity-history/
retention-days: 30
if-no-files-found: warn
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ pnpm native:build # opt-in; downloads deps, compiles th
pnpm native:verify:desktop # 300 native frames + a non-blank screenshot
```

**CI.** The pre-push hook runs `pnpm ci:fast` drift checks only; it proves no runtime behaviour. `pnpm ci:local --full` runs the full board; `pnpm ci:local --affected --base origin/develop --target develop` runs what CI would select (dirty or unknown inputs fall back to full). A change touching only Markdown that no fixture consumes (not `AGENTS.md`/`CLAUDE.md`) runs no product CI job. Ordinary PRs into develop use the complete candidate diff: CI-only changes run CI contracts, exact template changes run scaffolder contracts and affected templates, and shared executable inputs keep the full safety board with starter as the default template smoke plus explicitly changed runtime/scenario kits. Template instruction and Markdown changes retain their contract lanes without adding runtime kits. Merge groups, explicit full runs, unknown inputs, main qualification and scheduled audits use fresh exhaustive qualification: all shipped templates, both golden journeys and the full supported native matrix. `ci-required` independently rejects a review plan for these events; Integration is a required reusable join on the same candidate, with each selected canonical job proven by a current-run/attempt receipt and independently checked completed-success API job; missing, skipped, cancelled, stale or unmapped receipts fail closed. Native per-leg authoritative completion and cold-path queue feasibility remain follow-ups before deferring any Android PR coverage. Template scenarios run once per selected kit; develop changes run one representative golden journey and full qualification retains both starter and platformer journeys. `ci-required` rejects missing or failed selected checks; retained contexts include `supply-chain`, `template-nonvisual`, `golden-path`, and native `desktop-parity` when applicable. Native host/platform tests run for native or shared runtime consumers, main qualification and unresolved scopes. Unsafe matrix/reusable verdict reuse stays disabled until authoritative complete coverage and per-leg routing exist. A new candidate runs fresh tests; only exact-run immutable build artifacts are shared. Details: `docs/PRDs/done/PRD-483-impact-driven-ci-without-repeated-template-setup.md` and `docs/PRDs/done/PRD-373-selective-ci-and-develop-promotion.md`.
**CI.** The pre-push hook runs `pnpm ci:fast` drift checks only; it proves no runtime behaviour. `pnpm ci:local --full` runs the full board; `pnpm ci:local --affected --base origin/develop --target develop` runs what CI would select (dirty or unknown inputs fall back to full). A change touching only Markdown that no fixture consumes (not `AGENTS.md`/`CLAUDE.md`) runs no product CI job. Ordinary PRs into develop use the complete candidate diff: CI-only changes run CI contracts, exact template changes run scaffolder contracts and affected templates, and shared executable inputs keep the full safety board with starter as the default template smoke plus explicitly changed runtime/scenario kits. Template instruction and Markdown changes retain their contract lanes without adding runtime kits. Merge groups run the same selected gating board as the PR, so a change merges within 30 minutes (PRD-550). Develop pushes, explicit full runs, unknown inputs, main qualification and scheduled audits use fresh exhaustive qualification: all shipped templates, both golden journeys and the full supported native matrix. `ci-required` independently rejects a review plan for these events; Integration is a required reusable join on the same candidate, with each selected canonical job proven by a current-run/attempt receipt and independently checked completed-success API job; missing, skipped, cancelled, stale or unmapped receipts fail closed. Native per-leg authoritative completion and cold-path queue feasibility remain follow-ups before deferring any Android PR coverage. Template scenarios run once per selected kit; develop changes run one representative golden journey and full qualification retains both starter and platformer journeys. `ci-required` rejects missing or failed selected checks; retained contexts include `supply-chain`, `template-nonvisual`, `golden-path`, and native `desktop-parity` when applicable. Native host/platform tests run for native or shared runtime consumers, main qualification and unresolved scopes. Unsafe matrix/reusable verdict reuse stays disabled until authoritative complete coverage and per-leg routing exist. A new candidate runs fresh tests; only exact-run immutable build artifacts are shared. Details: `docs/PRDs/done/PRD-483-impact-driven-ci-without-repeated-template-setup.md` and `docs/PRDs/done/PRD-373-selective-ci-and-develop-promotion.md`.
Registry commands read auth from `~/.npmrc`; pass a checkout-local untracked `.npmrc` with `npm --userconfig .npmrc <command>`. Never print either file.

TypeScript 7 `strict` (the native `tsc`); `typescript` imports use the official TypeScript 6 compatibility API for AST scripts and tsup declarations, **ESM only**; relative imports end in `.js`. Versions come from the `catalog:` in `pnpm-workspace.yaml`, templates excepted. Biome formats — never hand-format. Interfaces are `I`-prefixed; classes and type aliases are not. Unit tests: `<package>/__tests__/*.spec.ts`, vitest, node environment, DOM and GPU stubbed.
Expand Down
Loading
Loading