Skip to content

Commit 2e3e276

Browse files
ci(test-shards): widen Test Core from 6 to 8 shards (#25746)
<!-- claude-projects-attribution: {"github_login":"os-elon-musk"} --> _Requested by **Elon** · [project thread](https://claude.ai/code/project/chan_01WedDzV26nniwxv7jC88B9D?thread=cmsg_01WedDzV26nniwxv7jC88B9DWDhLxhv4cQX1gMYwSzVYWk)_ Fixes #25376 Clause-②: no Metadata-protocol: none Test Core goes from 6 to 8 shards. It was sized on a fresh merge-group timing table on current `main`, as the 2026-10-23T19:55Z unlock asked, not on the card's 2026-10-13 arithmetic. Eight shards cut the slowest shard by 10–15% on every replay below, so this clears the ~10% bar. The Test Core timeout moves 50 → 60, because the stall-guard headroom tool shows 50 is already too short at six. ## Step 1: the timing table this is sized from **Merge-group run [40155484566](https://github.com/objectstack-ai/objectstack/actions/runs/40155484566)** (PR #25730's queue build, head `c882092b33` on `main` `ec4ef94ead`, 2026-10-25 02:42Z). 59 packages were executed and 1 replayed (`@objectstack/spec`); the CLI ran as its 3 slices. Read from the `Test Core` aggregator's timing table and suite-ceiling list, and from each shard's `shard-timing-drift` line: | shard | predicted bin | measured test windows | `Run this shard's tests` step | | :-- | --: | --: | --: | | 1/6 | 809.4 s executed + spec (replayed, 1606.6 s) | 929.5 s | 666 s | | 2/6 | 2416.5 s | 3211.4 s | 1043 s | | 3/6 (CLI slice) | 2412.5 s | 2136.6 s | 1387 s | | 4/6 (CLI slice) | 2419.2 s | 3109.2 s | **1794 s** | | 5/6 (CLI slice) | 2419.5 s | 2661.2 s | 1498 s | | 6/6 | 2414.5 s | 2368.7 s | 989 s | Heaviest packages on that run, measured vs pinned: `cli` 3036.30 s vs 2899.02 s (all 3 slices, ~1012 s each), `objectql` 1044.70 vs 970.12, `runtime` 936.14 vs 848.96, `plugin-auth` 789.01 vs 405.25, `rest` 735.83 vs 876.56, `plugin-security` 666.28 vs 360.02. Spec's heaviest repo files after #25372: `build-schemas-check-mode.git-topology` 432.88 s, `.deleted-baseline` 403.48 s, `.deletion-proofs` 377.06 s. A smaller merge-group run on the same day, [40157571602](https://github.com/objectstack-ai/objectstack/actions/runs/40157571602) (21 packages executed; core, spec and types replayed), had its slowest step at 1381 s on the shard holding a CLI slice. ## Step 2: 6 vs 8, predicted Bins are not wall time. On a shard, the whole packages run four at a time, and a CLI slice then runs as a serial leg of its own. So a shard's tests step reads close to `wholeSum / 2.9 + slice`. On run 40155484566 this model lands within 6.1% of the measured step on the four slowest shards (the three slice shards and shard 2). It misreads the two light shards, which never set the wall. Each list below is replayed through the partitioner's own `planRun()` at each count: | list | slowest shard at 6 | slowest shard at 8 | change | | :-- | --: | --: | --: | | run 40155484566 (60 packages) | ~1696 s (measured 1794 s) | ~1436 s | **−15%** | | full dataset list (75 packages) | ~1491 s | ~1277 s | −14% | | run 40157571602 (24 packages) | ~1080 s (measured 1381 s) | ~967 s | −10% | The partitioner's own bins on the committed dataset (the CLI as 3 slices; `ratio` = max/mean): ``` 6 shards -> max 2489s mean 2488s ratio 1.00x 7 shards -> max 2136s mean 2133s ratio 1.00x 8 shards -> max 1868s mean 1866s ratio 1.00x 9 shards -> max 1661s mean 1659s ratio 1.00x 10 shards -> max 1607s mean 1493s ratio 1.08x <- spec's whole weight is the floor ``` **What eight cannot move:** a CLI slice runs serially for about 960–1010 s. On small runs at eight, the slice shards carry almost nothing else, so that is where the −10% stops. A ninth shard would mostly add a runner. The `1.3×` balance bound is unchanged and still met: spec's 1607 s floor sits against `1.3 × 1866 = 2426 s`. Pin 3c (CLI n = 3) is independent of the shard count and still holds. **Cost:** over the window below, job start → tests step is 1.6 min median (5.5 min p90), and the tail after the step is 0.2 min. Two more shards cost about +3.6 runner-min per run at the median. The card's maintainer directive accepts this. ## Step 3: the timeout, re-derived `node scripts/measure-stall-guard-headroom.mjs` covered the 89 completed, non-cancelled CI runs created 2026-10-24 03:00Z → 2026-10-25 03:45Z (23 merge_group, 39 pull_request, 16 push, 11 schedule; 512 `Test Core (N/6)` jobs): ``` .github/workflows/ci.yml job `test` step `Run this shard's tests` W 10m · C 20m · T 50m (job timeout-minutes) worst observed: p 2m03s + s 38m24s = 40m27s (522 observation(s), worst on `Test Core (1/6)`) undeferred p+s+W = 50m27s vs T 50m -> UNCOVERED by 0m27s deferred p+s+C = 60m27s vs T 50m -> UNCOVERED by 10m27s ``` So 50 is already short at six. The worst case is a red PR shard (run 40149547507). The healthy worst is 37m22s (p 7m28s + s 29m54s, run 40125226571 `Test Core (4/6)`, job wall 37m41s). Scaled to eight by 0.90, the smallest gain above (the safe side): | reading | at eight | needs | | :-- | --: | --: | | window verdict, p + s + 10m | 2m03s + 34m34s + 10m = 46m37s | > 46.6 | | cap-deferred verdict, p + s + 20m | 56m37s | > 56.6 | | 80% wall margin, healthy worst wall | ~34m40s | ≥ 43.3 | | #16465 red line, slowest shard ≤ 70% of timeout | ~34m40s | ≥ 49.5 | | drift ceiling, healthy worst step at 1.5× prediction | 7m28s + 26m55s × 1.5 = 47.8m | > 47.8 | The largest reading wins, as in the #22075 derivation: **60** (3m23s over the deferred path). The revert condition in `ci.yml` now reads "below 60". The first full-battery run after landing re-takes these readings on eight-shard jobs. ## What changed - `scripts/partition-test-shards.mjs`: `SHARD_COUNT` 6 → 8. Its header now carries the 6-vs-8 arithmetic above. The "#10472 asked for 6 → 8 and the answer was no" paragraph is rewritten (kept, not deleted) to say why the answer changed. - `.github/workflows/ci.yml`, Test Core: - the matrix is `1..8`; - `name: Test Core (… /8)`, `--shard …/8`, the remote-cache and prune `--site` labels, the drift `--label`, `--total 8`, and the `-of-8` artifact names; - the aggregator's `--leg "test/8:…"`; - `timeout-minutes: 60`; - the "WHY SIX" comment block is rewritten as "WHY EIGHT" / "WHY NOT MORE THAN EIGHT" with today's numbers, and the history paragraphs are kept. - `scripts/ci/select-shard-timings-run.mjs`: `SHARD_COUNT` 6 → 8, now only its matchers' default. Since the second commit, a candidate run is judged at **its own width**: `runShardWidth` reads `M` from the run's `Test Core (N/M)` job names, and the run is admitted when all `M` shards concluded success and all `M` `-of-M` summaries are retained. A run whose Test Core jobs disagree on `M`, or that has none, is refused by name. Each candidate carries `shard_count` beside its `artifact_ids`. - `.github/workflows/shard-timings-refresh.yml`: the summary download loop and the PR body's per-shard line read the shard roster from the selector's `artifact_ids` instead of spelling `1 2 3 4 5 6`. The selector only admits a run when all of that run's shards are present, so those keys are that run's roster, whatever its width. Wording that said "six" now names the count generically. - `scripts/check-shard-attestation.mjs`: the fixtures that edit the live `ci.yml` (grow the matrix, shrink the roster, the name variants) use the 8-shard spellings. The replays of measured six-shard runs (38036705318, 34007386254) read the live workflow with the six-shard name, and that substitution is asserted. The live-name pin now checks `Test Core (1..8/8)`. - `scripts/check-required-contexts.mjs`: the live-specimen battery follows the `/8` name. - `scripts/measure-test-shard-timings.mjs`: four `-of-6` spellings in comments and output text now name the width generically (`-of-<width>`, `N-of-M`), since a refresh window can mix widths. Historical run fixtures elsewhere (`scripts/pm/*`, merge-queue triage) keep the `(N/6)` names their runs really had. **No refresh transition window.** The first head of this PR judged every candidate at the live count, so `Regenerate the shard-timings dataset` rejected all 24 recent schedule runs (each a whole, green 6-shard run) as missing `7/8`, `8/8`, and exited 1 ([job 121075952437](https://github.com/objectstack-ai/objectstack/actions/runs/40162213660/job/121075952437)). That would have held for every refresh until an 8-shard schedule run existed with its one-day artifacts still retained. The dataset is per package, and the width a run was split at is not an input to any package's weight, so the fix reads each run at its own width rather than reporting NOT MEASURED through the window. Tests: a new self-test battery, `width change (#25376)` (9 cases: a whole 6-shard run is eligible under the 8-shard matrix; the same run pinned to 8 is rejected as job 121075952437 rejected it; an old-width run missing `6-of-6` or with `6/6` cancelled is refused; 6 jobs with `-of-8` summaries is refused; jobs disagreeing on `M` and a run with no Test Core job are refused by name; an unanchored `Test Core (3/8) rerun` does not move the width; `listCandidates` reports an 8-wide and a 6-wide run each with its own width). The accumulation battery gains a leg that drives the refresh workflow's real generate step over one 8-wide and two 6-wide runs, with a package measured only in each run's last shard. Both were checked to fail against the old reading: restoring the fixed-width default fails the first case, and restoring a fixed `1..8` download loop fails the accumulation leg. **Serial with #25697 / PR #25738:** #25738 landed first (`c61f75d`); this branch merges it in, with no conflict. ## Acceptance (from the card) - Partition self-test, `check-aggregator-roster`, `check:ci-filter-parity` green locally (below) and on this PR. - After landing, reported on #25376: the first full-battery run's slowest `Test Core` tests step, and max shard wall / timeout (≤ 70%, #16465). ## Local verification (head `8c254ff`) Derived with `node scripts/pm/dispatch-gates.mjs --commands` (70 families) and reconciled with `--ran`: **66 exit 0, 4 NOT MEASURED**. - NOT MEASURED (exit 3, PREREQUISITE NOT MET, they read built `dist/` and this worktree has no full build; CI runs them): `pnpm check:dts-closure`, `pnpm check:dual-build-cjs-loads`, `pnpm check:lean-entry-closure`, `pnpm check:sourcemap-no-sources-content`. - Among the green ones: `node scripts/partition-test-shards.mjs --self-test` (75 measured packages → 77 items, 8 shards, max/mean 1.00x, bins 1865.66–1867.52 s within the 2899.02 s density cap, CLI ×3), `node scripts/check-aggregator-roster.mjs` and `--self-test`, `node scripts/check-ci-filter-parity.mjs` and `--self-test`, `pnpm check:shard-attestation`, `pnpm check:required-contexts`, `pnpm check:pm-expected-skips`, `pnpm check:stall-guard-budget`, `pnpm check:stall-guard-headroom`, `node scripts/ci/select-shard-timings-run.mjs --self-test` (it executes the refresh workflow's generate step, so the new roster loop ran), `node scripts/measure-test-shard-timings.mjs --self-test`. - `node scripts/check-commit-card-trailers.mjs --range origin/main..HEAD`: OK. <details><summary>All 70 derived commands and exit codes</summary> ``` node scripts/check-aggregator-roster.mjs :: exit 0 node scripts/check-aggregator-roster.mjs --self-test :: exit 0 node scripts/check-ci-filter-parity.mjs :: exit 0 node scripts/check-ci-filter-parity.mjs --self-test :: exit 0 node scripts/check-closing-keyword-parity.mjs :: exit 0 node scripts/check-closing-keyword-parity.mjs --self-test :: exit 0 node scripts/check-comment-mask-corpus.mjs :: exit 0 node scripts/check-declaration-mirrors.mjs :: exit 0 node scripts/check-declaration-mirrors.mjs --self-test :: exit 0 node scripts/check-dts-emitted.mjs --self-test :: exit 0 node scripts/check-position-name-fold-loaders.mjs :: exit 0 node scripts/check-position-name-fold-loaders.mjs --self-test :: exit 0 node scripts/check-scripts-symbol-anchors.mjs :: exit 0 node scripts/check-scripts-symbol-anchors.mjs --self-test :: exit 0 node scripts/check-self-test-wired.mjs :: exit 0 node scripts/check-self-test-wired.mjs --self-test :: exit 0 node scripts/check-self-test-workflow-commands.mjs :: exit 0 node scripts/check-self-test-workflow-commands.mjs --self-test :: exit 0 node scripts/check-step-collectors.mjs :: exit 0 node scripts/check-step-collectors.mjs --self-test :: exit 0 node scripts/check-whole-set-label-write.mjs :: exit 0 node scripts/check-whole-set-label-write.mjs --self-test :: exit 0 node scripts/ci/assert-turbo-remote-cache.mjs --self-test :: exit 0 node scripts/ci/prune-turbo-cache.mjs --self-test :: exit 0 node scripts/ci/scheduled-full-run.mjs --self-test :: exit 0 node scripts/ci/select-shard-timings-run.mjs --self-test :: exit 0 node scripts/docs-audit/check-drift-comment.mjs :: exit 0 node scripts/measure-test-shard-timings.mjs --self-test :: exit 0 node scripts/partition-test-shards.mjs --self-test :: exit 0 node scripts/pm/bare-root-worklist.mjs --self-test :: exit 0 node scripts/pm/ci-failure.mjs --self-test :: exit 0 pnpm check:agent-test-spelling :: exit 0 pnpm check:bash32-floor :: exit 0 pnpm check:cli-command-ids :: exit 0 pnpm check:console-injection :: exit 0 pnpm check:console-sha :: exit 0 pnpm check:cross-package-test-inputs :: exit 0 pnpm check:declared-population-live :: exit 0 pnpm check:driver-memory-census :: exit 0 pnpm check:dts-closure :: exit 3 pnpm check:dual-build-cjs-loads :: exit 3 pnpm check:entry-guard :: exit 0 pnpm check:gitlink-declared :: exit 0 pnpm check:lean-entry-closure :: exit 3 pnpm check:node-version :: exit 0 pnpm check:nul-bytes :: exit 0 pnpm check:parse-guard :: exit 0 pnpm check:pm-close-cards :: exit 0 pnpm check:pm-dispatch-gates :: exit 0 pnpm check:pm-expected-skips :: exit 0 pnpm check:pm-fleet-write-dispatch :: exit 0 pnpm check:pm-fleet-write-execute :: exit 0 pnpm check:pm-fleet-write-validate :: exit 0 pnpm check:pm-governed-merges :: exit 0 pnpm check:pm-issue-transfer :: exit 0 pnpm check:pm-post-stamped :: exit 0 pnpm check:pm-record-recognisers :: exit 0 pnpm check:pnpm-acquisition :: exit 0 pnpm check:pnpm-filter-targets :: exit 0 pnpm check:pnpm-flag-spellings :: exit 0 pnpm check:ratchet-remedy-authority :: exit 0 pnpm check:refd-timer-probe :: exit 0 pnpm check:required-contexts :: exit 0 pnpm check:shard-attestation :: exit 0 pnpm check:sourcemap-no-sources-content :: exit 3 pnpm check:stall-guard-budget :: exit 0 pnpm check:stall-guard-headroom :: exit 0 pnpm check:watch-hint-literal :: exit 0 pnpm check:workflow-status-functions :: exit 0 pnpm check:workflow-step-name-quoting :: exit 0 ``` </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01M781cxsyRAqT49LvLkJKXb --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 6ad226a commit 2e3e276

8 files changed

Lines changed: 430 additions & 169 deletions

‎.github/workflows/ci.yml‎

Lines changed: 92 additions & 57 deletions
Large diffs are not rendered by default.

‎.github/workflows/shard-timings-refresh.yml‎

Lines changed: 29 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@
77
#
88
# scripts/test-shard-timings.json is the per-package duration dataset the Test
99
# Core split is binned from. It is GENERATED — scripts/measure-test-shard-
10-
# timings.mjs turns a green run's six turbo summaries into it — but until now it
11-
# was generated only when a person remembered to. It went 13 days without a
10+
# timings.mjs turns a green run's per-shard turbo summaries into it — but until
11+
# now it was generated only when a person remembered to. It went 13 days without a
1212
# refresh while ~700 test files were added, and the shard it mis-weighted was
1313
# killed by the job wall twelve times in one day and ejected from the merge queue
1414
# twice (#16173).
@@ -46,8 +46,11 @@
4646
#
4747
# HOW IT GETS ITS INPUTS, AND WHY IT RUNS HERE RATHER THAN IN AN AGENT WORKTREE
4848
# ----------------------------------------------------------------------------
49-
# The documented refresh path downloads the six `test-core-run-summary-N-of-6`
50-
# artifacts of a green run. `GET /actions/artifacts/{id}/zip` redirects to
49+
# The documented refresh path downloads the `test-core-run-summary-N-of-M`
50+
# artifacts of a green run, one per Test Core shard. M is the RUN's own width,
51+
# read from its `Test Core (N/M)` job names, so runs from before and after a
52+
# matrix resize both read (#25376).
53+
# `GET /actions/artifacts/{id}/zip` redirects to
5154
# `productionresultssa*.blob.core.windows.net`, and every agent container's
5255
# egress policy answers 403 to CONNECT for that host — reproduced four ways, on
5356
# two artifacts across two runs, on the pre-signed URL as well as the API one, on
@@ -71,8 +74,8 @@
7174
# Until #16467 a push to `main` re-ran the FULL Test Core battery, so "a green
7275
# push run on main" and "a measurement of the workspace" were the same thing.
7376
# They are not any more: a push run computes its package set with `--affected`
74-
# against `github.event.before`. Its six shards still conclude `success` and
75-
# still upload six run summaries, so NOTHING in the eligibility test would have
77+
# against `github.event.before`. Its shards still conclude `success` and
78+
# still upload every run summary, so NOTHING in the eligibility test would have
7679
# noticed — this lane would have kept regenerating the balancing dataset from
7780
# measurements of whatever the last merge happened to touch, and the coverage
7881
# check would not have caught it either (a package the affected set skipped is a
@@ -96,8 +99,8 @@
9699
# follows from the cache design rather than from luck — turbo's key is
97100
# namespaced per shard and only main pushes write it, so a package whose inputs
98101
# have not changed is a HIT, and the generator refuses hits rather than
99-
# recording a replayed ~0.1s window as a suite's cost. "Download six artifacts
100-
# from any green run" therefore measures a SLICE of the workspace. That number
102+
# recording a replayed ~0.1s window as a suite's cost. "Download the shard
103+
# artifacts from any green run" therefore measures a SLICE of the workspace. That number
101104
# should IMPROVE under the hourly run, because a push no longer seeds the whole
102105
# workspace's entries — but it is quoted as it was measured, and the next live
103106
# run is what re-measures it.
@@ -178,7 +181,7 @@
178181
# - DOWNS, in one line.
179182
# - FILES OVER BUDGET: every test file over 10 s of wall in the window, with
180183
# its package and its slowest run (#22998). The dataset holds no per-file
181-
# reading; the window's runs do, in the per-shard `test-core-timings-N-of-6`
184+
# reading; the window's runs do, in the per-shard `test-core-timings-N-of-M`
182185
# captures, which a step before the body downloads: the same runs, retained
183186
# exactly as long as their run summaries.
184187
#
@@ -432,14 +435,14 @@ jobs:
432435
# not bad luck, it is the cache design: turbo's key is namespaced per shard
433436
# and only main pushes write it, so a package whose inputs have not changed
434437
# is a HIT — and the generator refuses hits rather than recording a
435-
# replayed ~0.1s window as a suite's cost. "Download six artifacts from any
436-
# green run" therefore measures a SLICE of the workspace, never all of it.
438+
# replayed ~0.1s window as a suite's cost. "Download the shard artifacts
439+
# from any green run" therefore measures a SLICE of the workspace, never all of it.
437440
# Since #16467 the runs read here are the HOURLY `schedule` runs, which are
438441
# the full battery; the accumulation stays because the cache argument above
439442
# is unchanged by which event ran the suite.
440443
#
441-
# So runs are accumulated. Each contributes its six summaries under its own
442-
# `--run <id>` group, which is exactly the grouping #16473 added: slices are
444+
# So runs are accumulated. Each contributes its shards' summaries under its
445+
# own `--run <id>` group, which is exactly the grouping #16473 added: slices are
443446
# summed within a run and the per-run sums are medianed across runs, so a
444447
# package measured by three of these runs gets a median of three
445448
# observations rather than whichever run happened to be read last. Feeding
@@ -482,7 +485,13 @@ jobs:
482485
rm -rf "$SUMDIR"; mkdir -p "$SUMDIR"
483486
484487
OK=1
485-
for N in 1 2 3 4 5 6; do
488+
# One summary per shard the selector found: it admits a run only when
489+
# every shard OF THAT RUN concluded success with its artifact
490+
# retained, at the run's own width (select-shard-timings-run.mjs,
491+
# runShardWidth), so these keys ARE that run's roster. A count
492+
# spelled here would break the first refresh after a resize (#25376).
493+
SHARD_NS=$(node -e 'const r=JSON.parse(require("fs").readFileSync(process.env.RUNNER_TEMP+"/candidates.json","utf8"))[Number(process.argv[1])];console.log(Object.keys(r.artifact_ids).map(Number).sort((a,b)=>a-b).join(" "))' "$i")
494+
for N in $SHARD_NS; do
486495
ART=$(node -e 'const r=JSON.parse(require("fs").readFileSync(process.env.RUNNER_TEMP+"/candidates.json","utf8"))[Number(process.argv[1])];console.log(r.artifact_ids[process.argv[2]])' "$i" "$N")
487496
# curl drops the Authorization header across the redirect to the
488497
# blob host, which is correct: that URL is pre-signed and the
@@ -636,7 +645,7 @@ jobs:
636645
# The per-file half of the regression readings (#22949 ruling E, item 3).
637646
# The dataset holds no per-file durations; the window's runs do, in the
638647
# per-shard captures `report-test-timings.mjs --capture` uploads as
639-
# `test-core-timings-N-of-6`, retained 1 day like the run summaries the
648+
# `test-core-timings-N-of-M`, retained 1 day like the run summaries the
640649
# generator read. The selector listed their ids beside the summaries'
641650
# (`timing_artifact_ids`), so this is a read of the SAME runs, never a new
642651
# measurement, and it feeds the PR body alone.
@@ -822,7 +831,8 @@ jobs:
822831
const rs = JSON.parse(require("fs").readFileSync(process.env.RUNNER_TEMP + "/candidates.json", "utf8"));
823832
const r = rs.find((x) => String(x.run_id) === process.env.RUN_ID);
824833
const s = r.shard_seconds ?? {};
825-
console.log([1,2,3,4,5,6].map((n) => (s[n] == null ? "?" : `${n}: ${s[n]}s`)).join(" | "));
834+
const ns = Object.keys(r.artifact_ids ?? {}).map(Number).sort((a, b) => a - b);
835+
console.log(ns.map((n) => (s[n] == null ? "?" : `${n}: ${s[n]}s`)).join(" | "));
826836
')
827837
828838
{
@@ -851,8 +861,8 @@ jobs:
851861
done
852862
echo
853863
echo "- Newest run in the set: \`$RUN_ID\`, commit \`$HEAD_SHA\` — the date this refresh carries."
854-
echo "- Every run above had all six \`Test Core (N/6)\` jobs conclude \`success\` with its six"
855-
echo " run-summary artifacts still retained; runs that were cancelled, failed or had lost"
864+
echo "- Every run above had every \`Test Core (N/M)\` shard job conclude \`success\` with its"
865+
echo " run-summary artifact still retained; runs that were cancelled, failed or had lost"
856866
echo " their artifacts were rejected by name in the log before any of these were used."
857867
echo
858868
echo "$CARRY_LINE"
@@ -886,7 +896,7 @@ jobs:
886896
echo "## The partitioner's own pins RED on this refresh — read this before merging"
887897
echo
888898
echo "This is the designed behaviour, not a defect in the refresh: the acceptance bound is"
889-
echo "a ratio, and a package that has grown past what any six-way split can bin makes the"
899+
echo "a ratio, and a package that has grown past what the Test Core split can bin makes the"
890900
echo "pins fail with the arithmetic in the message. The remedy the partitioner names is to"
891901
echo "raise the file-level slice count for that package — ⛔ never to raise the bound, and"
892902
echo "⛔ never to hand-edit this dataset. This workflow deliberately does neither: it"

‎scripts/check-required-contexts.mjs‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2026,13 +2026,13 @@ async function selfTest() {
20262026
);
20272027

20282028
// ── (9) the shadowing collision, on the live specimen ────────────────────
2029-
// ci.yml's sharded `test` job is named `Test Core (${{ matrix.shard }}/6)`
2029+
// ci.yml's sharded `test` job is named `Test Core (${{ matrix.shard }}/8)`
20302030
// and its aggregate gate is named `Test Core`. Dropping the suffix makes two
20312031
// jobs publish one context, and the surviving conclusion is whichever
20322032
// finished last — a shard could satisfy the aggregate's required gate.
20332033
battery('(9) the shadowing collision, on the live specimen');
20342034
const collided = fixture('collide the shard name with the gate name', 'ci.yml', (s) =>
2035-
s.replace('name: Test Core (${{ matrix.shard }}/6)', 'name: Test Core'),
2035+
s.replace('name: Test Core (${{ matrix.shard }}/8)', 'name: Test Core'),
20362036
);
20372037
assert(
20382038
collided.problems.some((p) => p.includes("job 'test'") && p.includes("published by ci.yml:test-gate")),
@@ -2041,7 +2041,7 @@ async function selfTest() {
20412041
// The suffixed spelling is NOT a collision — the guard must not read a
20422042
// prefix as a clash, or ci.yml is red on main today.
20432043
assert(
2044-
baseline.problems.length === 0 && sources['ci.yml'].includes('name: Test Core (${{ matrix.shard }}/6)'),
2044+
baseline.problems.length === 0 && sources['ci.yml'].includes('name: Test Core (${{ matrix.shard }}/8)'),
20452045
'the real suffixed shard name coexists with the bare gate name (assertion 9 compares whole names, not prefixes)',
20462046
);
20472047

‎scripts/check-shard-attestation.mjs‎

Lines changed: 15 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2085,6 +2085,13 @@ async function selfTest() {
20852085
// ── #22653: the per-shard cause read itself, offline ─────────────────────
20862086
battery('#22653 the cause read, offline: one jobs list, then one annotations read per unattested shard');
20872087
const CI_TEXT = readFileSync(join(scriptRepoRoot(), '.github', 'workflows', 'ci.yml'), 'utf8');
2088+
// The runs replayed below (38036705318, 34007386254) ran the SIX-shard matrix,
2089+
// which #25376 widened to eight, so they are read against the live workflow
2090+
// with the shard name as it stood then. The live names are pinned on their own
2091+
// at the end of this battery.
2092+
const LIVE_TEST_NAME = 'name: Test Core (${{ matrix.shard }}/8)';
2093+
const CI_TEXT_SIX_SHARDS = CI_TEXT.replace(LIVE_TEST_NAME, 'name: Test Core (${{ matrix.shard }}/6)');
2094+
assert(CI_TEXT_SIX_SHARDS !== CI_TEXT, `#22653: the live ci.yml no longer carries \`${LIVE_TEST_NAME}\` — the six-shard replays below would read the live roster`);
20882095
const PARTIAL_JOBS = /\/repos\/o\/r\/actions\/runs\/38036705318\/attempts\/1\/jobs\?per_page=100&page=1$/;
20892096
const ANY_ANNOTATIONS = /\/check-runs\/\d+\/annotations\?per_page=100$/;
20902097
const shardCheckRun = (n) => 114168570850 + n;
@@ -2109,7 +2116,7 @@ async function selfTest() {
21092116
apiUrl: API,
21102117
token: 'not-a-real-token',
21112118
fetchJson,
2112-
workflowText: CI_TEXT,
2119+
workflowText: CI_TEXT_SIX_SHARDS,
21132120
...over,
21142121
});
21152122

@@ -2177,7 +2184,7 @@ async function selfTest() {
21772184
'#22653: a matrix name with no shard expression cannot locate a shard ⇒ unreadable, naming the shard',
21782185
);
21792186

2180-
assert([1, 2, 3, 4, 5, 6].every((n) => workflowJobDisplayName(CI_TEXT, 'test', n) === `Test Core (${n}/6)`), '#22653: the real ci.yml names each Test Core shard as the jobs API lists it');
2187+
assert([1, 2, 3, 4, 5, 6, 7, 8].every((n) => workflowJobDisplayName(CI_TEXT, 'test', n) === `Test Core (${n}/8)`), '#22653: the real ci.yml names each Test Core shard as the jobs API lists it');
21812188
assert([1, 2, 3].every((n) => workflowJobDisplayName(CI_TEXT, 'dogfood', n) === `Dogfood Regression Gate (${n}/3)`), '#22653: …and each dogfood shard');
21822189
assert(workflowJobDisplayName(CI_TEXT, 'dogfood-verify') === 'Dogfood Verify CLI', "#22653 control: the single leg's literal name reads as before");
21832190
assert(workflowJobDisplayName(CI_TEXT, 'test') === undefined, '#22653: a matrix template asked WITHOUT a shard is still unresolvable ⇒ undefined');
@@ -2480,8 +2487,8 @@ async function selfTest() {
24802487
// `String.replace` takes the first match. A short anchor would silently
24812488
// start mutating dogfood's matrix instead — still red, still for a
24822489
// plausible-looking reason, and no longer the assertion written here.
2483-
assert((await fixture('grow the matrix', (s) => s.replace('shard: [1, 2, 3, 4, 5, 6]', 'shard: [1, 2, 3, 4, 5, 6, 7]'))).problems.some((p) => p.includes('declares 7 shard')), 'growing the matrix without the gate ⇒ red');
2484-
assert((await fixture('shrink the roster', (s) => s.replace('--leg "test/6', '--leg "test/2'))).problems.some((p) => p.includes('expects 2 attestation')), 'shrinking the gate roster without the matrix ⇒ red');
2490+
assert((await fixture('grow the matrix', (s) => s.replace('shard: [1, 2, 3, 4, 5, 6, 7, 8]', 'shard: [1, 2, 3, 4, 5, 6, 7, 8, 9]'))).problems.some((p) => p.includes('declares 9 shard')), 'growing the matrix without the gate ⇒ red');
2491+
assert((await fixture('shrink the roster', (s) => s.replace('--leg "test/8', '--leg "test/2'))).problems.some((p) => p.includes('expects 2 attestation')), 'shrinking the gate roster without the matrix ⇒ red');
24852492
assert((await fixture('uncounted intruder', (s) => s.replace(/^jobs:$/m, 'jobs:\n intruder:\n runs-on: ubuntu-latest\n steps:\n - run: node scripts/check-shard-attestation.mjs --emit'))).problems.some((p) => p.includes('no gate counts')), 'an attestation no gate counts ⇒ red');
24862493
assert((await fixture('rename test-gate', (s) => s.replace(' test-gate:', ' test-gate-renamed:'))).problems.some((p) => p.includes("'test-gate'")), 'losing a required-context gate ⇒ red');
24872494
assert((await fixture('typo the artifact name', (s) => s.replace('name: shard-attest-test-', 'name: shard-attest-typo-'))).problems.some((p) => p.includes('cannot count')), 'an artifact name the gate cannot match ⇒ red');
@@ -2573,13 +2580,13 @@ async function selfTest() {
25732580
hasProblem(await fixture('drop the token (test-gate)', inJob('test-gate', TOKEN_LINE, '')), "gate 'test-gate'", 'GITHUB_TOKEN'),
25742581
"#22653: test-gate's verify step without GITHUB_TOKEN ⇒ red",
25752582
);
2576-
const TEST_NAME = ' name: Test Core (${{ matrix.shard }}/6)\n';
2583+
const TEST_NAME = ' name: Test Core (${{ matrix.shard }}/8)\n';
25772584
assert(
25782585
hasProblem(await fixture('a matrix name without the shard expression', (s) => s.replace(TEST_NAME, ' name: Test Core shards\n')), '#22653', "job 'test' is named"),
25792586
'#22653: a matrix name with no `${{ matrix.shard }}` ⇒ red — the platform would append the values itself, a name the reader cannot rebuild',
25802587
);
25812588
assert(
2582-
hasProblem(await fixture('a matrix name with a second expression', (s) => s.replace(TEST_NAME, ' name: Test Core (${{ matrix.shard }}/6) ${{ github.event_name }}\n')), '#22653', "job 'test' is named"),
2589+
hasProblem(await fixture('a matrix name with a second expression', (s) => s.replace(TEST_NAME, ' name: Test Core (${{ matrix.shard }}/8) ${{ github.event_name }}\n')), '#22653', "job 'test' is named"),
25832590
'#22653: any expression besides the shard ⇒ red — the text reading cannot evaluate it',
25842591
);
25852592
assert(
@@ -2591,11 +2598,11 @@ async function selfTest() {
25912598
"#22653: dogfood's matrix leg is held to the same name rule — it is asked on a partial cancel too",
25922599
);
25932600
assert(
2594-
(await fixture('the matrix name double-quoted', (s) => s.replace(TEST_NAME, ' name: "Test Core (${{ matrix.shard }}/6)"\n'))).problems.length === 0,
2601+
(await fixture('the matrix name double-quoted', (s) => s.replace(TEST_NAME, ' name: "Test Core (${{ matrix.shard }}/8)"\n'))).problems.length === 0,
25952602
'#22653 positive limb: a quoted template reads the same from text and from YAML ⇒ green',
25962603
);
25972604
assert(
2598-
(await fixture('the shard expression without inner spaces', (s) => s.replace(TEST_NAME, ' name: Test Core (${{matrix.shard}}/6)\n'))).problems.length === 0,
2605+
(await fixture('the shard expression without inner spaces', (s) => s.replace(TEST_NAME, ' name: Test Core (${{matrix.shard}}/8)\n'))).problems.length === 0,
25992606
'#22653 positive limb: `${{matrix.shard}}` is the same expression ⇒ green',
26002607
);
26012608
assert(baseline.problems.length === 0 && baseline.gateIds.includes('test-gate'), '#22653: the checked-in test-gate carries every prerequisite — the baseline is green with it counted as a gate');

0 commit comments

Comments
 (0)