Skip to content

ci: two-speed CI — affected tests on PR and queue runs, full suite and coverage on main's push and hourly schedule, spec gate reuse by objectstack sha, fifteen gate workflows folded into Repo Gates (objectui#12214) - #12232

Merged
os-zhuang merged 7 commits into
mainfrom
claude/issue-12214-two-speed-ci
Oct 12, 2026

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #12214
Refs objectui#12208 (PR-1)
Clause-②: no

Two-speed CI for objectui under the doctrine of objectstack-ai/objectstack#22998, in four commits plus a main merge, so the four diffs read separately. Dispatched on the card's claim; this PR stays a draft (a dispatched PR is the seat's to land, and the card asks for an authorised approval before it moves).

commit what
6130bef9 Tests, two speeds. pull_request / merge_group run the affected test files (new scripts/select-affected-tests.mjs); push to main and a new hourly schedule run the full suite plus the 4-shard coverage lane.
df83ebef objectui#12208 PR-1, folded in by ruling 6121816480 (letter B): the Test matrix goes 8 → 12 legs, every shard-count pin with it. Coverage stays at 4.
58610a4a Spec Main Shape Gate builds @objectstack/spec only when objectstack main moved; gains the hourly schedule run.
9573c870 Repo Gates: fifteen one-check gate workflows folded into one job, each check kept as named steps.

1. Tests: two speeds (ci.yml)

  • Fast set on pull_request and merge_group: the workspace packages the change sits in, plus every package depending on them (the set turbo ls --affected answers; the new test holds the two equal on a leaf, a mid-graph and a root package), plus an always-on set of test files the package graph cannot place: tests outside every package (scripts/**, eslint-rules/**), tests whose relative-import closure reads the disk or spawns a process, tests that reach a file outside their own package, and tests that import a workspace package their own package does not depend on (the root vitest alias map resolves every @object-ui/* name to source, declared or not).
  • Full run instead when the change reaches something every test runs through: a package.json, vitest.config.* or vite.config.* anywhere (vitest's own forceRerunTriggers default), a tsconfig*.json, a repository-root non-documentation file or patches/**, a file the root vitest config imports, or a package a broad setup file imports (the light DOM setup installs the i18n catalogues into every dom test). A package only the heavy DOM setup imports adds the dom-heavy test files (and apps/console/, whose config loads that setup) instead of forcing a full run.
  • Fails open: a selector that exits non-zero, prints no verdict or gets no base sha means the full suite.
  • Full battery: the matrix, Test (dist pins) and the Test aggregator lose their push guard; test-coverage / Test (coverage) run on push and schedule. The aggregator keeps --shards equal to the matrix width and still refuses a partial set. The width is the same twelve legs on both speeds (the affected run does not derive its own count).

H1 — measured, not assumed

  • One leaf file changed (packages/plugin-kanban/src/KanbanColumnSummary.ts), on this 4-core shared box: vitest list --changed selected 92 test files but spent 254.7 s transforming every test file's import graph before running anything, and it is blind to setup-file and disk-read edges. turbo ls --affected answered in 0.26 s (8 packages). The package graph was taken; the selector computes its own closure in about 1.5 s and the test pins its agreement with turbo.
  • Replayed over the newest 200 first-parent commits of main (each commit's own diff; the current tree supplies the always-on set): 63 (32%) would run the full suite (57 of them because i18n or a package upstream of it was affected, 4 a package.json, 1 a root file, 1 a vite.config.ts); share of test files selected: p25 41%, median 45%, p75 100%, mean 62%. A docs-only change selects the always-on floor, 622 of 4,931 tracked test files.
  • Sharding still pays on the affected speed: the floor alone is about a full shard's worth of today's files.

Before (jobs API, newest 300 completed CI runs, 2026-10-09T13:14Z → 2026-10-12T08:42Z)

event runs wall median / p90 runner-min median / p90
pull_request 137 17.3 / 18.0 min 128.0 / 139.4
merge_group 78 17.5 / 18.0 min 132.0 / 141.9
push 74 39.3 / 40.7 min 151.3 / 165.2

Test shard test step median 13.0 min (PR) / 14.5 (queue); install 0.1 min. Type Check 9.3 (PR) / 10.6 (queue). Spec Main Shape Gate 12.8 / 13.3.

After — projected here; the measured run of this PR is in the report comment on the card

  • Full suite at twelve legs: about 8/12 of today's shard step. Affected median: about 45% of that.
  • ⚠️ The PR and queue wall is then bound by Spec Main Shape Gate (12.8–13.3 min, see H3) and Type Check (9.3–10.6 min), neither of which this card changes. The card's acceptance line "a product-only PR's checks all conclude within 8 minutes" is therefore not reachable by items 1–4 alone; named as the follow-up below.
  • This PR's own diff selects mode=full (it changes lychee.toml, a root config file), so its own Test run is the full suite at twelve legs.

2. Spec Main Shape Gate (spec-main-shape-gate.yml)

  • H3 refuted on its premise. Step medians over 120 runs (2026-10-11T07:53Z → 2026-10-12T08:57Z): build 1.3 min, pack 0.3 min, forced type-check 10.6 / 11.1 min of a 12.8 / 13.3 job. Caching the spec saves at most ~1.6 min per run; the forced compile is the cost, and it is uncached by design.
  • actions/cache is branch-scoped: a pull request cannot read another pull request's cache, so a cache written by PR runs would hit only that PR's own re-runs. A pull request or queue build also runs its own branch's YAML, so what it uploads is not evidence about a spec.
  • What landed: the hourly schedule run on main publishes the packed spec plus the sha it was packed at as the artifact objectstack-spec-SHA (one day); every other run looks it up by the sha it resolved, reuses it only if a main run of this repository built it, and otherwise builds. A failed lookup or download, or an artifact recording another sha, means build. actions: read is added; nothing writes through the token; no actions/cache (the turbo-cache prohibition pin stands).
  • Hit rate, modelled on real data (objectstack main commit times vs this gate's 116 run starts, 101 objectstack commits in the window): hourly warm 22% (expected saving ≈ 0.35 min/run); a 30-min warm 35%, 15-min 53%, 10-min 64%. Reusing any earlier run's artifact would reach 44% but was rejected on provenance.
  • The real lever, not taken here: inject writes the sha and the injection time into the turbo-hashed marker, so even a sound turbo cache can never hit. A marker keyed by sha alone plus a cache of the compile is a ruling-level change.

3. Repo Gates (repo-gates.yml)

Folded (old check-run → step names CHECK — STEP): Control Byte Scan, Action Ref Convention, Pre-Install Import Graph Check, Shell Escape Residue Scan, Inert vi.mock Specifier Check, Internal Docs Link Check, Doc Fence Language Check, Doc Example Id Check, Doc Component Type Check, Skill Guide Path Check, Skill Eval Token Check, Docs Route Eager Closure Check, Skill Example Check, Doc Snippet Type Check, README Export Check. Deleted: control-bytes.yml, action-ref-convention.yml, pre-install-import-graph.yml, shell-escape-residue.yml, vi-mock-specifiers.yml, docs-links.yml, doc-fence-languages.yml, doc-example-ids.yml, doc-component-types.yml, skills-paths.yml, skill-eval-tokens.yml, docs-route-eager-closure.yml, skill-examples.yml, doc-snippet-types.yml, readme-exports.yml.

  • Same commands, same events (pull_request, push, merge_group, workflow_dispatch), no path filter. One checkout, one setup. Every gate step runs if: !cancelled() (or on the install / build it needs), so one red never hides another.
  • Install-free gates run before the install, so they stay in the pre-install import-graph population. The three type-surface gates share one build: --filter='./packages/*' in union with the two derived --build-filter closures; the derive steps now refuse an empty filter, and the build can never be unfiltered.
  • Not folded: governed-surface-guard.yml (ruled), changeset-presence.yml (its Changeset Declaration is required), spec-main-shape-gate.yml (required; item 2), changeset-guard.yml (not on the card's list).
  • scripts/dependabot-merge-gate.mjs waits for Repo Gates instead of the fifteen names; its partition test, the merge_group floor map and each gate's pin test move to the new home (install-free absence pins became order pins). The pipeline guide documents the fifteen under one Repo Gates section.

Required check-run names — byte-identical

Live read of the ruleset (GET /rules/branches/main), before this PR: Lint, Type Check, Build & E2E, Build Docs, Changeset Declaration, Test, Spec Main Shape Gate, Governed Surface Queue Guard. After: the same eight names, produced by the same workflows (lint.yml, ci.yml ×4, changeset-presence.yml, spec-main-shape-gate.yml, governed-surface-guard.yml). None of the fifteen folded names was required, so no ruleset edit is needed. merge-queue-reporting.test.ts re-derives every declared context to a merge_group-subscribed producer.

4. Lint

Unchanged (6.0 min median). It is not the critical path after items 1–3; Spec Main Shape Gate's forced compile and Type Check are, and those are the named follow-ups.

Verification (local, on 2fbe080c)

  • pnpm exec vitest run scripts/ eslint-rules/ on 2fbe080c: 192 files passed, 2 skipped; 5767 tests passed (VERDICT command-exit 0). An earlier run on the first commit had two 15 s timeouts under box load (check-vi-mock-inherit, check-i18n-dead-keys) that passed on re-run, unchanged.
  • pnpm type-check 81/81 tasks; pnpm type-check:scripts exit 0; pnpm lint 47/47 tasks (0 errors); eslint --no-inline-config over the 43 changed scripts/ files: 0 errors, 0 warnings.
  • Workflow-reading self-tests: check-pre-install-import-graph, check-required-check-set, check-governed-queue-guard, check-bash32-floor, check-cross-repo-closer-outcome — all exit 0.
  • The fifteen folded gates run live on this tree: all exit 0 (the three type gates after building every package under packages/, as the job does).
  • Ablation: deleting the main-only filter from the spec reuse lookup turns its pin red; restored from HEAD, blob hash verified.
  • YAML parse of ci.yml, spec-main-shape-gate.yml, repo-gates.yml and every edited workflow. actionlint: NOT MEASURED, not installed here.
  • node scripts/check-changeset-presence.mjs: no changeset owed (no published source changed).

Acceptance notes

  • AGENTS.md's test section still says pnpm test is what CI runs with --shard=1/4; it was already stale on the width, and a pull request's CI now runs the affected set. AGENTS.md is governed and untouched here.
  • check-governed-queue-guard.mjs --test .github/workflows/repo-gates.yml answers NOT GOVERNED: this repository's governed surfaces do not include .github/workflows/**. The PR stays a draft regardless, as a dispatched PR.
  • The submodules: true checkout option is carried over unchanged; the tree declares no submodule.

维护者速读(草稿)

改了什么:PR 与合并队列只跑受改动影响的测试(按包依赖图选取,外加读磁盘等图看不见的测试常驻);main 的每次合并与每小时定时任务跑全量测试和覆盖率。测试矩阵 8 → 12 路(#12208 第一个 PR,并入本 PR)。Spec 形状门禁按 objectstack 提交复用已构建的 spec。十五个单检查工作流并成一个 Repo Gates 作业,检查名保留为步骤名。

为什么改:您的原话「我觉得能让整个车队提速的方案就是好方案,main 很多天才会发一个版本,我只要确保发版本之前做好各种测试就可以。」—— 全量测试挪到 main,不删不减。

风险与代价(含回滚):快速集合漏掉的问题会在一小时内由 main 的全量运行变红(这是两速制的既定代价)。必需检查名逐字未变,无需改仓库设置。Spec 复用实测只省约 1.6 分钟中的命中部分(模型命中率 22%),门禁真正的耗时在强制类型检查;PR 墙钟此后受 Spec 门禁与 Type Check 约束(约 13 分钟),达不到卡上 8 分钟验收线。回滚:按倒序 revert 四个提交(第 2 个改的是第 1 个的同一批行),或整个 PR 一次 revert。

席位意见:

你要做的:审阅并批准;落地由席位执行。若要更高的 Spec 复用命中率,可裁定把定时预热改为每 10–15 分钟一次。


Generated by Claude Code

…lds, the full suite on push and an hourly schedule (objectui#12214)

A pull_request or merge_group run of the `Test (shard N/8)` matrix now runs
the test files the change affects, selected by the new
`scripts/select-affected-tests.mjs`: the workspace packages the change sits in
plus every package depending on them (turbo's affected set, held equal to
turbo by the test), plus an always-on set of test files the package graph
cannot place (outside every package, a relative closure that reads the disk or
leaves its package, an undeclared workspace import). A change that reaches
something every test runs through -- a package.json, tsconfig, vitest or vite
config, a root install/toolchain file, the root config's own imports, or a
package a broad setup file imports -- answers mode=full. The heavy DOM setup's
packages add the dom-heavy test files instead of forcing a full run. The
selector fails open: no verdict means the full suite.

`push` to main and a new hourly `schedule` carry the full battery: the matrix,
the dist-pin job and the `Test` aggregator lose their push guard, and the
coverage lane runs on push and schedule. The matrix width, the aggregator's
--shards and its refusal of a partial set are unchanged.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB
…t pin with it (objectui#12208 PR-1)

Folded into objectui#12214 by ruling 6121816480 (letter B). At eight shards the
`Test (shard N/8)` legs sat at 14.4 min median, 17.4 p95 and 20.3 max against
the 20-minute cap over the 1,728 shard jobs of the trailing week (measured on
objectui#12208). The cap is not raised, no test is skipped, the relevance gate
is not widened: the matrix goes to twelve legs, on both speeds.

Pins moved with it: the matrix and both `--shard=N/12` spellings, the job and
step names, `check-test-shard-results.mjs --shards 12` in the `Test`
aggregator, the twelve `Test (shard N/12)` rows of the Dependabot gate's
NOT_A_GATE partition and its test, the selector's wiring test, and the prose
naming the width (pipeline guide, lint.yml, three test headers). The coverage
matrix stays at four. Duration-balanced sharding is objectui#12208's PR-2.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB
…ed, and run the gate hourly on main (objectui#12214)

The Spec Main Shape Gate stays on every event. A new hourly `schedule` run on
`main` -- the gate's place in the full battery -- publishes the spec it packed
as the artifact `objectstack-spec-SHA` (with the sha recorded beside it, kept
one day). Every other run looks that artifact up at the sha it just resolved
and injects it instead of building again; the clone and the spec's own install
still run, because `inject` reads the dependencies the spec's build resolved.

Only an artifact from a `main` run of this repository is reused: a pull request
or queue build runs its own branch's YAML, so its uploads are not evidence. A
failed lookup or download, or an artifact recording another sha, means build.
The job gains `actions: read` for the lookup; nothing writes through the token.
No actions/cache is used, so the turbo-cache prohibition pin stands unchanged.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB
…job (objectui#12214)

The control-byte, action-ref, docs-link, skills-path, skill-example,
skill-eval-token, doc component/snippet/fence/example-id, pre-install import
graph, inert vi.mock, shell-escape residue, README export and docs-route eager
closure workflows each paid their own runner, checkout and setup on every pull
request, push and queue build. They are now steps of ONE job in
`.github/workflows/repo-gates.yml`: same commands, same events (pull_request,
push, merge_group, workflow_dispatch), no path filter, each step named
`CHECK — STEP` after the job name it used to report under, every gate step
`if: !cancelled()` so one red never hides another. The install-free gates run
before the install (they stay in the pre-install import-graph population); the
three gates that read built types share one build, `--filter='./packages/*'`
in union with the two derived filters, and the derive steps refuse an empty
filter.

None of the folded contexts was in the merge queue's required set (live ruleset
read: Lint, Type Check, Build & E2E, Build Docs, Changeset Declaration, Test,
Spec Main Shape Gate, Governed Surface Queue Guard), so Changeset Presence,
Spec Main Shape Gate and Governed Surface Guard stay their own workflows and
every required check-run name is unchanged. The Dependabot merge gate's
REQUIRED_CONTEXTS waits for `Repo Gates` instead of the fifteen names; its
partition, the merge_group floor map and every gate's pin test move to the new
home, with install-free absence pins rewritten as order pins. The pipeline guide
documents the fifteen under one Repo Gates section.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB
@github-actions github-actions Bot added documentation Improvements or additions to documentation configuration ci/cd tests labels Oct 12, 2026
…l untouched (objectui#12214)

The fold commit repointed one comment line in each at the folded control-byte
gate. Both files are outside this card's claimed surface, so the edits are
reverted to main's bytes; the stale pointer each now carries (a filename the
fold deleted, inside a sentence about past reasoning) is reported on the card
instead of edited here.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqNgdPZB2Jnr5HpGbaqLeB
@os-zhuang
os-zhuang marked this pull request as ready for review October 12, 2026 11:57
@os-zhuang
os-zhuang enabled auto-merge October 12, 2026 11:57
@os-zhuang
os-zhuang added this pull request to the merge queue Oct 12, 2026
Merged via the queue into main with commit 21fdeb3 Oct 12, 2026
33 of 34 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-12214-two-speed-ci branch October 12, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd configuration documentation Improvements or additions to documentation tests

Projects

None yet

2 participants