Skip to content

Commit dc3e5b9

Browse files
ci: give every push to main its own concurrency group, so the next merge no longer cancels the previous merge's tests (#25454)
<!-- claude-projects-attribution: {"github_login":"os-elon-musk"} --> _Requested by **Elon** · [project thread](https://claude.ai/code/project/chan_01WedDzV26nniwxv7jC88B9D?thread=cmsg_01WedDzV26nniwxv7jC88B9DKetfGjXFjZp4c8fp4AkXHz)_ Fixes #25373 Clause-②: no Metadata-protocol: none Every push to `main` now gets its own CI and Lint concurrency group, keyed by the pushed commit. The next merge no longer cancels the previous merge's run, so each merge's own `--affected` Test Core and its lint/typecheck verdict finish on `main`. This lands as a **coverage fix**, per the maintainer's ruling on the card (batch #5 item 1, letter C, comment 6366946328). It does not make PR builds faster; the speed question moves to a sibling card on Turbo seeds vs the 10 GB Actions cache pool. ## What changed - **`ci.yml` and `lint.yml`, the workflow-level `concurrency.group`.** The last segment goes from `github.event.pull_request.number || github.ref` to `github.event.pull_request.number || github.event.after || github.ref`. `github.event.after` is set only on a push, so: - a push run's group is `ci-CI-push-<sha>`, unique per merge; - `schedule`, `workflow_dispatch` and `merge_group` carry no `after` and still fall through to `github.ref`, so they group exactly as before; - on `pull_request` the PR number still wins first, so a new push to a PR still cancels its stale run. `cancel-in-progress` stays `true`; with a per-commit push group it never fires on a push. The block comments are rewritten to the new rule. - **`scripts/ci/scheduled-full-run.mjs --check-concurrency`** (the gate `lint.yml` runs over these two keys) now pins the new invariant: - its event contexts carry each event's real run-varying fields: `github.sha`, `github.run_id`, and the payload's `after` on a push and a PR synchronize, none on schedule, dispatch or queue; - **new finding:** two pushes of two different commits must not share a group; - **positive control rewritten:** two `schedule` runs on two `main` heads, two `workflow_dispatch` runs and two heads of the same PR must still share one group. A run-unique key is still refused: `github.run_id` as before, and now also `github.sha`, which is run-unique on `schedule` too; - the self-test gains a firing control on the old shared-push key (refused, with only the push/push finding), context cases per event, and a real-file check; the battery floor moves 20 → 32. The evaluator's grammar is unchanged (context paths, quoted literals, `||`). ## Why `cancel-in-progress: false` was not enough (the card's option (a)/(b)) GitHub keeps one **running** and one **pending** run per concurrency group; a third arrival cancels the pending one. Both options on the card keep a shared push group, so they still cancel push runs. Replayed on the real arrival times of the 40 push runs below, with a 27 min CI and 16 min Lint run: | | CI push runs cancelled | Lint push runs cancelled | queue wait before start | | :-- | --: | --: | :-- | | today (shared group, cancel-in-progress) | 31 / 40 (measured) | 27 / 40 (measured) | none | | (a)/(b): shared group, no cancel-in-progress on push | 18 / 40 (replay) | 11 / 40 (replay) | CI median 4.2 min, max 25.9 | | this PR: per-commit push group | 0 / 40 | 0 / 40 | none (runs overlap) | ## Baseline (GitHub Actions API, push-to-main runs 2026-10-23 01:03Z → 11:03Z) - **CI**: 40 push runs, 31 cancelled and 9 succeeded. Cancelled runs averaged 11.7 min of wall, most of it inside "Run this shard's tests"; successful runs took a median 27.0 min. - **Lint & Type Check**: 40 push runs, 27 cancelled and 13 succeeded; successful runs took a median 16.6 min. - **The seed is not what was lost.** The three "Save Turbo cache (main only)" steps in `ci.yml` and the one in `lint.yml` are `if: always() && github.event_name == 'push'`, and `always()` runs on a cancelled job. Cancelled run 39923516471 saved all six Test Core shard seeds. Shard 4 (job 120315257857) logged "Cache saved with key: Linux-turbo-test-4-main-3306cada…" after uploading 2,135,596,699 bytes. What a cancelled run loses is its merge's Test Core verdict. ## Cost The previously cancelled runs now run to completion. Measured inputs: - one completed CI push run is 175.7 runner-min (run 39943396707, 17 jobs); - one completed Lint push run is 68.8 runner-min (run 39943396723, 14 jobs); - one cancelled CI push run had already spent 102.6 runner-min at 12.8 min of wall (run 39923516471). | | per merge | | :-- | --: | | **upper bound** (no credit for minutes a cancelled run already spent): 31/40 × 175.7 + 27/40 × 68.8 | **≈ +183 runner-min** | | **estimate** (crediting partial runs, scaled by wall time from the measured cancelled sample) | **≈ +85 runner-min** | The card estimated about 100. The repository is public and both workflows run on `ubuntu-latest`, so per the ruling the cost is runner concurrency, not billing. On concurrency, two push runs now overlap at today's cadence; the guard in the acceptance below watches the PR queue for it. ## Acceptance (rewritten by the ruling, comment 6366946328) 1. Over ≥ 10 consecutive merges to `main` after landing, every push run's own Test Core completes, and no push CI or Lint run is cancelled by its successor (Actions API: push runs with conclusion `cancelled` = 0). 2. Guard: over the next 10 PR runs, the median runner queue wait (job `started_at` − `created_at`) stays ≤ 1 min. If it does not, revert this PR. The shard-closure and Build Core speed criteria leave this card. They go with the sibling cache-pool card, which the `domain:devx` seat files. ## Risks and rollback - **Overlapping main runs.** Every cache key the push battery saves carries `github.sha` except the vendored Console dist cache (`ci.yml` ~:3112, saved only on a cache miss), which is keyed by file hashes. Two overlapping runs that both miss can race to save that key; `actions/cache/save` answers the loser with a warning ("Unable to reserve cache…"), not a failure. No push-battery job writes a ref, deploys or comments. - **Cache pool.** Push runs already saved on cancellation, so the number of seed writes per merge does not change. Completed test steps make each seed somewhat larger. The pool question itself belongs to the sibling card. - **Readers of push runs.** `scheduled-full-run.mjs --release-gate` reads completed runs at or before the version commit; more of them now conclude `success`. `check-shard-attestation.mjs` still classifies a superseded PR run by the same annotation prefix. - **Rollback.** Revert this PR. The gate change travels with it, so the revert stays green. ## Local verification (head 7da22f0) - `node scripts/ci/scheduled-full-run.mjs --self-test`: `self-test OK` (concurrency battery: 32 cases). - `node scripts/ci/scheduled-full-run.mjs --check-concurrency`: both files OK. - Firing proofs (the fix committed first, then reverted and restored): with `ci.yml`'s key put back to the shared-push shape, `--check-concurrency` exited 1 naming "two pushes to main share the concurrency group 'ci-…-push-refs/heads/main'", and `--self-test` threw. With `lint.yml` reverted the same way, `--check-concurrency` exited 1 with the same finding. Both files were restored, and the gate exited 0 again. - The derived gate list (`node scripts/pm/dispatch-gates.mjs`) was run as a node-only subset, all exit 0: `check:pm-dispatch-gates` (fast tier, 334.6 s), `check-step-collectors` and its `--self-test`, `check-ci-filter-parity`, `check-self-test-wired` and its `--self-test`, `check-self-test-workflow-commands`, `check-scripts-symbol-anchors`, `check-published-list-mirrors`, `check-declaration-mirrors`, `check-platform-checklist-watchdog`, `check-shard-attestation --self-test`, `check-adr-0087-registration --base origin/main`, `check-empty-changeset --base origin/main`, `check-comment-mask-corpus`, `check-skills-token-ratchet`, `check-aggregator-roster`, `check-closing-keyword-parity`, `pm/bare-root-worklist --self-test`, `check-nul-bytes`. The rest of the farm is left to CI. - Both workflow files parse (`yaml`), and `concurrency` reads back as written. - `check-governed-merges.mjs --branch HEAD`: GOVERNED, landing tier H (`.github/workflows/**` ×2); `scripts/ci/scheduled-full-run.mjs` is not on the register. - `skip-changeset`: no published package is touched (`.github/workflows/**`, `scripts/ci/**`). ## Acceptance notes - **File surface widened, by the seat.** The claim named `ci.yml` and `lint.yml` only. A per-commit push key needs the concurrency gate's invariant changed, so the `domain:devx` seat approved adding `scripts/ci/scheduled-full-run.mjs` on 2026-10-23 before any write. - **Premise.** The card's premise (cancellation starves the seed) was falsified before this PR opened; the evidence is comment 6364570166. That is why the PR is framed and measured as coverage. - **Branch name.** The branch is `claude/project-thread-9d8u29`, the name the session was given, not `claude/issue-25373-…`. The claim (comment 6364129096) names it. - **Serial edits.** #25374 and #25375 also edit `ci.yml`. `main` was re-read and merged before each push; neither had landed at 7da22f0. ## 维护者速读(草稿) **改了什么**:每次合并到 main 触发的 CI 和 Lint 运行,现在各自独占一个并发组,按提交区分。下一次合并不会再把上一次的运行取消掉。负责检查这条规则的门禁脚本同步改写,旧的共享写法会被它判红。 **为什么改**:今天 40 次合并里,有 31 次 CI 运行在测试中途被下一次合并取消。这些合并在 main 上就没有自己的测试结论,要等每小时一次的全量运行。缓存其实一直有存(保存步骤在取消时也会执行),所以这次改的是覆盖面,不是速度。速度问题另开一张缓存池的卡。 **风险与代价(含回滚)**:每次合并多花约 85 runner-min,最多约 183。仓库公开,用的是 GitHub 托管机,所以代价是并发,不是账单。两次 main 运行会重叠;如果之后 10 个 PR 的排队中位数超过 1 分钟,就回退本 PR。回退只需 revert,门禁改动会一起撤回。 **席位意见**:同意落地(devx 席位已在 head 7da22f0 复核,见 #25373 上的 ACCEPT 评论)。改法和裁定 C 一致:只给 push 事件按提交分组,定时、手动、合并队列和 PR 的分组不变。门禁自测在该 head 复跑通过,CI 全绿。代价是并发,有排队守卫兜底,回退成本低。 **你要做的**:本 PR 改了 `.github/workflows/**`,属于 Tier H,需要 GOVERNED_APPROVERS 中的一位批准后,席位才会翻 ready 并落地。 --- _Generated by [Claude Code](https://claude.ai/code/session_014Vs5pqqS7xcq3XeoTXPF98)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_014Vs5pqqS7xcq3XeoTXPF98 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent f7b6019 commit dc3e5b9

3 files changed

Lines changed: 183 additions & 59 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 37 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -61,32 +61,50 @@ on:
6161
# Treated as `full` by the selection script for the same reason `schedule` is.
6262
workflow_dispatch:
6363

64-
# Superseded runs on the same PR/branch waste runners and delay feedback;
65-
# cancel them. Push runs to main group by commit ref as well, so an in-flight
66-
# main run is cancelled only by a newer main push.
64+
# Superseded runs on the same PR waste runners and delay feedback; cancel them.
65+
# A push to `main` is the opposite case, and gets a group of its OWN (#25373).
6766
#
68-
# ⭐ `github.event_name` IS LOAD-BEARING IN THIS KEY, and it is the whole
69-
# reason the hourly run above can finish (#16467). On a `schedule` event
70-
# `github.event.pull_request.number` is empty and `github.ref` is
71-
# `refs/heads/main` — BYTE-IDENTICAL to what a push to `main` produces. Without
72-
# the event name in the key the hourly run and the next merge share one group,
73-
# `cancel-in-progress: true` applies, and the next merge kills the hourly run.
74-
# That is not a hypothetical: this card's own measurement is that 36 of the
75-
# last 60 push runs on `main` were already cancelled that way, at a merge
76-
# cadence that would censor most hours.
67+
# ⭐ `github.event.after` IS LOAD-BEARING IN THIS KEY. A push run is the only
68+
# run that tests ITS OWN merge: Test Core computes `--affected` against
69+
# `github.event.before` (see `on.schedule` above), so a push run that does not
70+
# finish leaves that merge's diff with no Test Core verdict on `main` until the
71+
# next hourly run. Main merges land every ~9–16 min and a push run takes ~27,
72+
# so under the old shared key `ci-CI-push-refs/heads/main` the next merge
73+
# cancelled it: 31 of 40 push runs on 2026-10-23 01:03Z–11:03Z, most of them
74+
# inside "Run this shard's tests". (The Turbo seed is NOT what was lost: the
75+
# "Save Turbo cache (main only)" steps are `if: always()` and run on a
76+
# cancelled job too.) `cancel-in-progress: false` alone does NOT fix it:
77+
# GitHub keeps ONE pending run per group and cancels it when a third arrives;
78+
# replayed on the same 40 arrivals, 18 still die. So the push segment is the
79+
# pushed commit. `github.event.after` is set on a push and absent on
80+
# `schedule`, `workflow_dispatch` and `merge_group`, which still fall through
81+
# to `github.ref`; on `pull_request` the PR number wins before it is reached.
82+
# Two push runs never share a group, so every one runs to completion. The seed
83+
# key carries `github.sha`, so two overlapping main runs never write the same
84+
# key.
7785
#
78-
# ⛔ Do not "simplify" this back to two segments. `scripts/ci/scheduled-full-
79-
# run.mjs --check-concurrency` evaluates this expression against a push-shaped
80-
# and a schedule-shaped context and reds when the two groups are equal, so the
81-
# collision cannot come back silently.
86+
# ⭐ `github.event_name` IS LOAD-BEARING TOO (#16467). A `schedule` and a
87+
# `workflow_dispatch` run on `main` both fall through to `refs/heads/main`;
88+
# without the event name they share one group and an on-demand run cancels the
89+
# hourly one (and, under any key that drops `github.event.after`, the next
90+
# merge cancels both, which is how 36 of 60 hourly-era push runs died).
8291
#
83-
# `cancel-in-progress` stays `true` for every event including `schedule`: two
84-
# hourly runs overlapping means one of them is over an hour old, and an hour-old
92+
# ⛔ Do not "simplify" either segment away. `scripts/ci/scheduled-full-run.mjs
93+
# --check-concurrency` evaluates this expression against push, schedule,
94+
# dispatch, PR and queue contexts on two different commits. It reds when two
95+
# pushes share a group, when a push shares one with a schedule or dispatch run,
96+
# or when two schedule, dispatch or same-PR runs stop sharing one (a run-unique
97+
# key such as `github.sha` or `github.run_id` would switch cancellation off for
98+
# those events too).
99+
#
100+
# `cancel-in-progress` stays `true`: with a per-commit push group it never
101+
# fires on a push, and for everything else it is the policy. Two hourly runs
102+
# overlapping means one of them is over an hour old, and an hour-old
85103
# measurement of `main` is strictly the worse of the two. Test Core's wall clock
86104
# is well inside the cadence (~39 shard-minutes across six shards), so this is
87105
# the exceptional path, not the normal one.
88106
concurrency:
89-
group: ci-${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref }}
107+
group: ci-${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.event.after || github.ref }}
90108
cancel-in-progress: true
91109

92110
# ── THE TWO SPEEDS (#22998) ──────────────────────────────────────────────────

‎.github/workflows/lint.yml‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -86,17 +86,21 @@ on:
8686
# non-required check on a queue build costs runners and blocks nothing, and
8787
# check-links.yml's own header already refuses it for that reason.
8888

89-
# Same policy as ci.yml: superseded runs on the same PR/branch waste runners
90-
# and delay feedback; cancel them. Push runs to main group by commit ref, so an
91-
# in-flight main run is cancelled only by a newer main push.
89+
# Same policy as ci.yml: superseded runs on the same PR waste runners and delay
90+
# feedback; cancel them. A push to `main` gets a group of its own, keyed by the
91+
# pushed commit (`github.event.after`, #25373), so no merge ever cancels the
92+
# previous merge's run and every merge gets its own lint and typecheck verdict
93+
# on `main` (27 of 40 push runs were cancelled by their successor on
94+
# 2026-10-23).
9295
#
9396
# ⭐ `github.event_name` is in the key for the reason ci.yml's concurrency block
94-
# states in full (#16467): on a `schedule` event the remaining two segments are
95-
# byte-identical to a push to `main`, so without it the next merge cancels the
96-
# hourly full run. `scripts/ci/scheduled-full-run.mjs --check-concurrency`
97-
# evaluates THIS expression too and reds if the two groups ever collapse again.
97+
# states in full (#16467): `schedule` and `workflow_dispatch` both fall through
98+
# to `refs/heads/main`, so without it an on-demand run cancels the hourly one.
99+
# `scripts/ci/scheduled-full-run.mjs --check-concurrency` evaluates THIS
100+
# expression too and reds if two pushes, or a push and a schedule run, share a
101+
# group, or if two schedule, dispatch or same-PR runs stop sharing one.
98102
concurrency:
99-
group: lint-${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref }}
103+
group: lint-${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.event.after || github.ref }}
100104
cancel-in-progress: true
101105

102106
# ─────────────────────────────────────────────────────────────────────────────

0 commit comments

Comments
 (0)