Skip to content

Commit 56986e9

Browse files
ci(timing gates): the two timing steps report-only on every event, schedule included (#22949 ruling E, item 1) (#23063)
Fixes #23058 Clause-②: no Both `Test Core` timing steps, "Check this shard's timing drift" and "Check the per-package suite-duration ceilings", are now report-only on every event, `schedule` included. A package over its ceiling or a shard past its drift bound prints the same table, raises `::warning::` annotations and exits 0. Ruling: #22949 letter E, item 1, the maintainer's word 「22949 同意」 (#22949 (comment)). A refusal (a missing `--event`, a malformed ceiling table or raise record, a dataset or run summary that does not parse) stays red on every event, as ACCEPT 6119677604 decided. The diagnosis the ruling rests on, in two sentences: a wall-clock reading on a shared runner measures the run's load as much as the code (the same package reads 82 s on a quiet hour and 209 s on a busy one, 2.56x, while the ceiling formula allows 2.0x of weight); the baseline window was mostly quiet-density and every event, `schedule` included, meets both densities, so busy hours red the same near-ceiling packages whatever their code does: density, not code. ## What changed - `scripts/partition-test-shards.mjs`: `TIMING_GRADED_EVENT` is removed. `timingGateMode(event)` returns `report` for every event and still throws on a missing or empty one. `renderDriftVerdict` has one DRIFT path: one `::warning::` annotation, exit 0. `timingModeNote` says the mode once. Header carries the ruling URL and the two-sentence diagnosis. The formula, the 1.3x and 1.5x constants and the dataset are untouched. - `scripts/check-test-suite-ceilings.mjs`: imports lose `TIMING_GRADED_EVENT`; `renderCeilingVerdict` has one OVER path (one `::warning::` per package over, exit 0; `REFUSED` still exits 1). Header carries the ruling URL and the diagnosis, the exit-code section is rewritten. `CEILING_FLOOR_SECONDS`, `RULED_CEILING_RAISES` and the dataset reader are untouched. `RED_STATUSES` and the local `red` are renamed `OVER_STATUSES` and `overs`, because nothing is red any more. - `.github/workflows/ci.yml`: comments only. The step names, `--event "$GITHUB_EVENT_NAME"`, `if:` and `continue-on-error` of both steps are byte-identical (checked: no non-comment line of the file changed). The drift step and the ceilings step say why no event grades; the aggregator note that called the ceilings step "a gate that DOES red the job" now says it reds only on a refusal. - Self-tests: every event is a report case. The graded-mode cases became report-mode cases, the refusal cases are unchanged, and each file gained a sweep case that fails if any tier on any event exits non-zero or raises an `::error::`. Both batteries keep their pinned case floor (8); the ceilings self-test still reads 58 cases across 9 batteries. ## The spelling of the mode `TIMING_GRADED_EVENT` is removed rather than set to a value no event matches. With a never-matching constant the exit-1 branch and its `::error::` annotation would stay in the file with nothing able to reach them, and a dead branch reads as a switch someone may flip. With the constant gone there is no graded mode to select, the self-test says it as one line (every event reads `report`), and grading a timing again is a new ruling and a new edit, not a flag. `timingGateMode` keeps its name and its refusal, which is what the ruling and ACCEPT 6119677604 name. ## Verification At head 8197058 (after merging origin/main): - `node scripts/check-test-suite-ceilings.mjs --self-test`: "self-test OK (58 cases across 9 batteries)", exit 0. - `node scripts/partition-test-shards.mjs --self-test`: "self-test OK (72 measured packages, 74 shard items, 6 shards, ...)", exit 0. - CLI on fixtures, ceilings (live dataset, `@objectstack/client` ceiling 179.74 s, fixture run 209 s): `--event` schedule, merge_group, pull_request, push and workflow_dispatch each exit 0 with exactly one `::warning::` and no `::error::`; no `--event` exits 1 with the refusal; a fixture within its ceiling exits 0 with no annotation. - CLI on fixtures, drift (`@objectstack/client` predicted 111.17 s): measured 180 s (DRIFT, 1.62x), 150 s (WARN, 1.35x) and 100 s (OK) each exit 0 on all five events, with one `::warning::` for DRIFT and WARN and none for OK; no `--event` exits 1. - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 59 commands from this diff, plus `pnpm check:doc-authoring` which I added: 56 exit 0, 4 exit 3 (below). `--ran` reconciles: 59 derived, 55 run, 4 NOT-MEASURED, 0 UNRUN. The parity checkers named in ACCEPT 6119677604 are all in the 56 green: self-test-workflow-commands, step-collectors, workflow-status-functions, required-contexts, aggregator-roster, ratchet-remedy-authority, stall-guard-budget, stall-guard-headroom, workflow-step-name-quoting, shard-attestation. So are `check:nul-bytes`, `check:pm-dispatch-gates` and `check:entry-guard`. - NOT MEASURED, exit 3 PREREQUISITE NOT MET (no `dist/` in a fresh worktree): `check:dts-closure`, `check:dual-build-cjs-loads`, `check:lean-entry-closure`, `check:sourcemap-no-sources-content`. They are Build Core's family, derived because ci.yml is on that job's path filter. This diff touches no package, so no `dist/` content can differ; Build Core builds the closure and runs them. - Reverse verification (fix committed first, each leg through `scripts/ablation-replace.mjs` in wrap mode, restore proved by blob hash equal to HEAD and an empty `git diff HEAD`): re-grading ceilings on `schedule` (`exitCode: event === 'schedule' ? 1 : 0`) turned the ceilings self-test red ("over: exit 1, verdict OVER"); the same in the DRIFT return turned the partition self-test red ("a DRIFT rendered DRIFT exit 1"); `timingGateMode` returning `grade` for `schedule` turned it red ("schedule read a mode other than report"). Direction: red in all three, the common case. These are one-time proofs; no permanent test file was added for them. - Not applicable: package `pnpm test` and `pnpm typecheck` (no package changed), changeset (workflow and tooling only, `skip-changeset`). ## Acceptance notes - Declared file surface: the claim named the comments of the two timing steps. One more comment changed, the aggregator note near `ci.yml:1470`, because it called the ceilings step "a gate that DOES red the job" and this PR makes that false. No other edit is outside the claim. - The comments in `partition-test-shards.mjs` above the new header block still say "the red" for the DRIFT bound (they carry the 1.5x measurement). One sentence in the new block says the DRIFT bound is the upper warning tier since this ruling. The self-test battery key "drift warning tier under the red" is unchanged so the pinned roster does not move. - A missing `--event` still surfaces as an uncaught throw with a stack trace, the cosmetic ACCEPT 6119677604 already recorded as non-blocking. Only the message text changed. - Regression detection for the wall moves to E2 (#23059), the dataset refresh diff and the per-file budget; until it lands a real slowdown is visible as the warning annotation only. #23007 is settled by ruling item 4 through the seat, not by this PR's merge. - Acceptance on `main` after landing: the first hourly `schedule` run shows the two steps green with their `::warning::` annotations where a package is over. That reading is the seat's, after merge. - `.github/workflows/ci.yml` is under the round's Tier H posture, so this PR stays a draft until a `GOVERNED_APPROVERS` member approves it. ## 维护者速读(草稿) ### 改了什么 主干上每小时一次的定时全量跑(`schedule`)里,`Test Core` 的两个计时检查——"单个包的测试耗时上限"和"分片耗时漂移"——不再因为超限而变红,改成只在日志里给一条黄色警告,步骤照样通过。定时、合并队列、PR、手动四类事件一视同仁。耗时公式、20 秒下限、上限提升登记表、数据集和每小时刷新的流程都没有动;`ci.yml` 里只有注释变了,步骤名、`if:`、`continue-on-error` 一个字节没动。 ### 为什么改 这是 #22949 裁决 E 的第 1 项(您的原话「22949 同意」)。同一个包在安静的小时跑 82 秒,在繁忙的小时跑 209 秒(2.56 倍),而上限公式只容许 2.0 倍;所以主干定时跑 4 次里有 3 次变红,红的是机器负载,不是代码,每次都要有人按"主干红"规则去修一个并不存在的回归。 ### 风险与代价(含回滚) 代价:计时变慢不再让构建变红,真正的变慢要等后续的 E2(#23059,数据集刷新时对比差分,外加单文件 10 秒预算)才有系统化的发现手段;在它落地前只有黄色警告可看。风险低:只改计时检查的退出码和注释,测试本身和其它门禁不变;缺少 `--event`、表格或登记记录格式错误这类"拒收"仍然是红灯,不会悄悄放行。回滚:revert 本 PR 即回到只有 `schedule` 会红的状态。 ### 席位意见 ### 你要做的 本 PR 改了 `.github/workflows/ci.yml`,按本轮的受管面姿态保持草稿,请有批准权的维护者审阅后批准,批准后由归口席位入队。合并后看第一次整点的定时跑:两个计时步骤应为绿色,超限的包带黄色警告;随后 #23007 按裁决收口。 --- _Generated by [Claude Code](https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 63d5db2 commit 56986e9

3 files changed

Lines changed: 375 additions & 295 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 60 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1081,22 +1081,32 @@ jobs:
10811081
# OK at or under 1.3x (WARN_MEASURED_OVER_PREDICTED)
10821082
# WARN past 1.3x, at or under 1.5x: a `::warning::` annotation
10831083
# naming the shard and its heaviest overshoot; green
1084-
# DRIFT past 1.5x (MAX_MEASURED_OVER_PREDICTED): red
1084+
# DRIFT past 1.5x (MAX_MEASURED_OVER_PREDICTED): a second
1085+
# `::warning::` annotation, the upper tier; green
10851086
# NOT MEASURED no executed test task carried a dataset entry; green
10861087
#
1087-
# ONE red rule and a warning under it, on the same ratio: triage ruled
1088-
# the in-tree 1.5x the red and the card's 1.3x a warning only, and took no
1089-
# second red on wall time against `timeout-minutes`. Both constants, their
1090-
# measured basis and a self-test battery each live in the script.
1091-
#
1092-
# THE RED IS A `schedule` RED ALONE (C2, both timing steps, ruling
1093-
# https://github.com/objectstack-ai/objectstack/issues/22949#issuecomment-6119186511):
1094-
# the dataset's weights are measured on the hourly `schedule` runs, so
1095-
# only those exit 1 on DRIFT. On `merge_group`, `pull_request`, `push`
1096-
# and `workflow_dispatch` the step prints the same verdict, raises the
1097-
# DRIFT as a `::warning::` annotation and exits 0. The event reaches the
1098-
# script as `--event "$GITHUB_EVENT_NAME"`, which it requires; the rule
1099-
# and its self-test live there (TIMING_GRADED_EVENT, timingGateMode).
1088+
# Two thresholds on ONE ratio, and no second rule on wall time against
1089+
# `timeout-minutes`: triage ruled the in-tree 1.5x the upper tier and the
1090+
# card's 1.3x the lower. Both constants, their measured basis and a
1091+
# self-test battery each live in the script.
1092+
#
1093+
# NO EVENT GRADES (#22949 ruling E, item 1, both timing steps:
1094+
# https://github.com/objectstack-ai/objectstack/issues/22949#issuecomment-6125775640).
1095+
# A wall-clock reading on a shared runner measures the run's load as much
1096+
# as the code: the same package reads 82 s on a quiet hour and 209 s on a
1097+
# busy one, 2.56x, past this step's 1.5x bound and the ceilings' 2.0x of
1098+
# weight. The dataset's weights come from a window that was mostly
1099+
# quiet-density and every event, `schedule` included, meets both
1100+
# densities, so busy hours put the same shards past the bound whatever
1101+
# their code does: density, not code.
1102+
# So on `schedule`, `merge_group`, `pull_request`, `push` and
1103+
# `workflow_dispatch` alike the step prints the same verdict, raises DRIFT
1104+
# as a `::warning::` annotation and exits 0. The event reaches the script
1105+
# as `--event "$GITHUB_EVENT_NAME"`, which it requires: a missing one is a
1106+
# refusal, and so is a dataset or run summary that does not parse, and a
1107+
# refusal is still a red step on every event. The rule and its self-test
1108+
# live in the script (timingGateMode); regression detection moves to the
1109+
# dataset refresh's diff (E2, #23059).
11001110
#
11011111
# EXECUTED WINDOWS ONLY. Measurements come from samplesFromSummary() in
11021112
# measure-test-shard-timings.mjs, which drops cache HITs and failed
@@ -1105,11 +1115,11 @@ jobs:
11051115
# predicted and vouch for a rotted dataset, so a shard whose packages all
11061116
# replayed is NOT MEASURED, never "fast".
11071117
#
1108-
# NO `if:` and NO `continue-on-error` (the red is the point, and the
1109-
# `schedule`-only red is the script's `--event`, so neither is needed),
1110-
# and ABOVE the attestation pair for the #6082 reason documented on the
1111-
# upload above: a failed unguarded step skips that pair, so a drift red
1112-
# also withholds this shard's attestation, the fail-closed direction. A shard
1118+
# NO `if:` and NO `continue-on-error` (the only red left here is a
1119+
# refusal, and a refusal must stay red, so neither is wanted), and ABOVE
1120+
# the attestation pair for the #6082 reason documented on the upload
1121+
# above: a failed unguarded step skips that pair, so a refusal also
1122+
# withholds this shard's attestation, the fail-closed direction. A shard
11131123
# with no packages runs no turbo and writes no summary; that is NOT
11141124
# MEASURED here rather than a usage error from the script.
11151125
#
@@ -1459,8 +1469,10 @@ jobs:
14591469
# diagnostics fault here (an artifact-service 403 is the measured one,
14601470
# #14469) must not evict a green run. The suite-duration ceiling check
14611471
# at the end of this job reads the same captures and is the one step
1462-
# here that DOES red the job, on purpose: it is a gate, not a report, and
1463-
# a capture that never arrived reads NOT MEASURED there, never red.
1472+
# here that can still red the job, on purpose and only on a refusal (a
1473+
# malformed table or raise record, no `--event`): a package over its
1474+
# ceiling is a `::warning::` on every event, and a capture that never
1475+
# arrived reads NOT MEASURED there, never red.
14641476
#
14651477
# ⛔ Steps, deliberately, NOT a new `needs:` entry: this job's
14661478
# `OS_AGGREGATOR_MEMBERS` / `OS_AGGREGATOR_NON_MEMBERS` block is asserted
@@ -1516,11 +1528,14 @@ jobs:
15161528
if-no-files-found: ignore
15171529
retention-days: 14
15181530

1519-
# ── The per-package suite-duration ratchet (#16468) ──────────────────
1531+
# ── The per-package suite-duration ceilings (#16468) ─────────────────
15201532
#
1521-
# A GATE, unlike the two reporting steps above: NO `continue-on-error`,
1522-
# so on a `schedule` run a package whose suite ran past its ceiling reds
1523-
# `Test Core`, naming the package and its slowest test files. The
1533+
# A READING, never a red (ruling E, below): a package whose suite ran
1534+
# past its ceiling is one `::warning::` annotation naming it and its
1535+
# slowest test files, and the step exits 0 on every event. Unlike the two
1536+
# reporting steps above it carries NO `continue-on-error`, because a
1537+
# REFUSAL (a malformed table or raise record, no `--event`) is not a
1538+
# timing reading and reds `Test Core` on every event. The
15241539
# ceiling is the package's slowest executed run in the refresh window
15251540
# x 1.25, generated into scripts/test-shard-timings.json (`ceilings`) by
15261541
# the refresh lane and held by every later refresh; it rises only by a
@@ -1538,19 +1553,26 @@ jobs:
15381553
# samplesFromSummary, and every vitest file line); a capture that did not
15391554
# arrive leaves its packages ungraded and says so, never red.
15401555
#
1541-
# EXIT-GRADED ON `schedule` ALONE (C2, both timing steps, ruling
1542-
# https://github.com/objectstack-ai/objectstack/issues/22949#issuecomment-6119186511):
1543-
# the ceilings come from the hourly `schedule` runs, so only those red
1544-
# this job on a package over its ceiling. On `merge_group`,
1545-
# `pull_request`, `push` and `workflow_dispatch` the step prints the same
1546-
# table, raises each package over its ceiling as a `::warning::`
1547-
# annotation and exits 0; a real slowdown merged to `main` reds the next
1548-
# hourly run. A package whose ceiling is under the absolute floor (20s,
1549-
# CEILING_FLOOR_SECONDS) prints "below floor" and is never red. The
1550-
# event reaches the script as `--event "$GITHUB_EVENT_NAME"`, which it
1551-
# requires; a refusal (a malformed table or raise record) is red on every
1552-
# event. The rule, the floor's reading and their self-tests live in the
1553-
# script.
1556+
# NO EVENT GRADES (#22949 ruling E, item 1, both timing steps:
1557+
# https://github.com/objectstack-ai/objectstack/issues/22949#issuecomment-6125775640).
1558+
# A per-package suite wall on a shared runner measures the run's load as
1559+
# much as the package: the same package reads 82 s on a quiet hour (most
1560+
# packages replayed) and 209 s on a busy one (the whole workspace
1561+
# executing), 2.56x, while the ceiling formula (window max x 1.25) allows
1562+
# 2.0x of weight. The window the ceilings come from was mostly
1563+
# quiet-density and every event, `schedule` included, meets both
1564+
# densities, so busy hours put the same near-ceiling packages over
1565+
# whatever their code does: density, not code. So on `schedule`,
1566+
# `merge_group`, `pull_request`, `push` and `workflow_dispatch` alike the
1567+
# step prints the same table, raises each package over its ceiling as a
1568+
# `::warning::` annotation and exits 0. A package whose ceiling is under
1569+
# the absolute floor (20s, CEILING_FLOOR_SECONDS) prints "below floor" and
1570+
# is never over. The event reaches the script as
1571+
# `--event "$GITHUB_EVENT_NAME"`, which it requires; a refusal (a
1572+
# malformed table or raise record, no event) is red on every event. The
1573+
# rule, the floor's reading and their self-tests live in the script;
1574+
# regression detection moves to the dataset refresh's diff (E2, #23059)
1575+
# and the per-file budget.
15541576
#
15551577
# NOT MEASURED, exit 0, while the dataset carries no `ceilings`: they are
15561578
# generated only where the run-summary artifacts are downloadable (the

0 commit comments

Comments
 (0)