Skip to content

Commit f7b6019

Browse files
ci(test-core): a shard whose tests read no .d.ts builds its closure with OS_SKIP_DTS=1 (#25465)
<!-- claude-projects-attribution: {"github_login":"os-elon-musk"} --> _Requested by **Elon** · [project thread](https://claude.ai/code/project/chan_01WedDzV26nniwxv7jC88B9D?thread=cmsg_01WedDzV26nniwxv7jC88B9DTotpiGYVS8HCVm8u3FZDUX)_ Fixes #25375 Clause-②: no Metadata-protocol: none ## 维护者速读(草稿) **改了什么**:Test Core 六个分片在建依赖闭包前多一步「Choose this shard's declaration build」。分片里没有 `@objectstack/cli`、`@objectstack/metadata` 时,整个分片设 `OS_SKIP_DTS=1`,构建只出 JavaScript,不再生成 `.d.ts`(spec 那次 6 GB 堆的声明构建、cli 的完整 `tsc` 都省掉)。带这两个包的分片照旧。只改 `.github/workflows/ci.yml`。 **为什么改**:vitest 只加载 JS,大多数分片生成的声明文件没人读。本地 4 核冷构建同一闭包(65 个包):开声明 5m59s,关声明 1m24s。卡面原方案是六个分片全关,但实测全关会让 cli(4 个测试文件,24 例)和 metadata(1 个文件)红,原因都是 TS7016 找不到 spec / cli 的 `.d.ts`,这些测试本来就是在编译已发布的声明。所以这两个包所在的分片保留声明。 **风险与代价(含回滚)**:① 关声明的分片和 Build Core 不在同一个 turbo 哈希空间(`turbo.json` 的 `globalEnv`),读不到 Build Core 写进远端缓存的带声明产物,只能读本分片在合并队列和 main 上写的无声明产物;落地后头几轮会偏冷,之后自愈。② 以后若有新测试读已建好的 `.d.ts` 却落在关声明的分片,会以 TS7016 明确变红,修法是把它的包加进 `DTS_READERS`,不会静默放过。③ 全量跑时 cli 切 3 片、metadata 占 1 片,当前切法下只有分片 1、6 省;PR 只跑受影响包时省的分片更多。回滚:删掉这一步即可。 **席位意见**:同意落地(devx 席位已在 head e62c37b 复核,ACCEPT 见 #25375)。机制早已在 main 上(`turbo.json` 的 `globalEnv` 和各包 tsup 配置),本 PR 只决定哪些分片打开它。收窄到不含 cli、metadata 的分片是对的;漏掉的读者会以 TS7016 明确变红,不会静默放过。落地后头几轮偏冷属预期。 **你要做的**:以 GOVERNED_APPROVERS 账户审批本 PR(受管面 Tier H)。 ## What changed One new step in the `test` job, right after "Compute this shard's package set": it writes `OS_SKIP_DTS=1` to `$GITHUB_ENV` unless the shard's package list carries a package in `DTS_READERS` (`@objectstack/cli @objectstack/metadata`). Through `$GITHUB_ENV` it reaches both closure builds, the test step and the main-push seed, so all of them hash in one namespace (`turbo.json` `globalEnv`). `turbo.json` is untouched. ## The card's step 2: tests that read a `.d.ts` The card's premise "vitest never reads a `.d.ts`" is false for two packages. The static grep the card names gives 70 test files that mention `.d.ts`, so I measured instead. Under `OS_SKIP_DTS=1`, the whole Test Core list (dogfood excluded, `OS_TEST_TIERS=queue`) ran on 830a63a in a cloud container. These went red: | Package | Test file | Reads | Disposition | |:--|:--|:--|:--| | `@objectstack/cli` | `test/published-subpath-console.pin.test.ts` (6 tests) | cli's own `dist/console.d.ts` | shard keeps DTS | | `@objectstack/cli` | `test/published-subpath-hook-body.pin.test.ts` (4) | cli's own `dist/hook-body.d.ts` | shard keeps DTS | | `@objectstack/cli` | `test/scaffold-emission-typechecks.test.ts` (12) | spec's `dist/**/*.d.ts` (TS7016) | shard keeps DTS | | `@objectstack/cli` | `test/scaffold-test-script-passes.test.ts` (2) | spec's `dist/**/*.d.ts` (TS7016) | shard keeps DTS | | `@objectstack/metadata` | `src/serializers/typescript-serializer-annotation.test.ts` (1) | spec's `dist/**/*.d.ts` (TS7016) | shard keeps DTS | `@objectstack/example-showcase` was killed (exit 137) in that run because the local container ran out of memory. Run alone under `OS_SKIP_DTS=1` it passed: 37 files, 443 tests. Silent-skip check: 32 packages own a test that compiles a TS program or names a `.d.ts`. I ran them again with declarations on and compared the vitest totals per task. All 38 `test`/`test:repo` totals were identical except one. `@objectstack/spec:test` ran 21634 passed with DTS and 21632 passed plus 1 skipped without it. The difference is `scripts/root-entry-type-nameability.pin.test.ts`, which skips by its own design when spec's dist declarations are absent. That pin runs with `OS_EXPECT_ROOT_NAMEABILITY=1` against a built dist in lint.yml's `root-entry-type-nameability-pin-built-dist` step, so its coverage holds. ## Measurement Local, cloud container, 4 cores, the same 65-package closure (every package whose `build` the full Test Core list schedules), `turbo run build --force --concurrency=4`: | | Wall | Σ task time | `@objectstack/spec#build` | `@objectstack/cli#build` | |:--|--:|--:|--:|--:| | declarations on | 5m59s | 838s | 92.2s | 18.5s | | `OS_SKIP_DTS=1` | 1m24s | 146s | 47.8s | 7.6s | That is 23% of the DTS-on wall time, against the card's ceiling of 60%. Today's baseline on CI, from the card: cold closure step 3.1 to 6.0 min per shard on PR run 39891259937, about 31 runner-minutes in total. On main after the shard-timings refresh (#25409), the full list puts metadata on shard 2 and the three cli slices on shards 3 to 5, so shards 1 and 6 skip on a full run. On a PR the affected set decides. This PR's own diff selects only `@objectstack/driver-sql` (a declared ci.yml input), so its first run measures a cold DTS-off closure. Nothing in the remote cache has DTS-off hashes yet. The CI step timings from this PR, both ways, are posted as a PR comment. Replay trade-off: a skipping shard cannot replay Build Core's DTS-on remote artifacts. In steady state that costs nothing. Every package hash a merge introduces is built in the queue run's shard closures, which write the remote (`remote:rw` on `merge_group` and main pushes). That gives later PRs a DTS-off hit for every package they do not change. The packages a PR does change miss both ways, and they build faster without declarations. The cost is real only in the first runs after landing. ## Gates `node scripts/pm/dispatch-gates.mjs --commands` lists 54 commands. All 54 exit 0 on 8d58beb. `check:dts-closure` and `check:dual-build-cjs-loads` were re-run after a full DTS-on `pnpm build`, because they read a built tree. `actionlint` is not installed here. The workflow parses as YAML, and the job's step order is unchanged around the new step. ## Acceptance notes - Scope deviation from the card's step 1: two packages' shards keep their declarations, for the reason above. The other way to save on those shards is to keep only spec's and cli's declarations under a per-package variable. That needs `tsup.config.*`, cli's `build` script and `turbo.json` `globalEnv`, which is outside this card's file surface, so it is not done here. - `.github/workflows/test-nightly-tiers.yml` builds its own shard closure and is not changed here. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01VqzDQEbeHPVJoaEoutGsTi --- _Generated by [Claude Code](https://claude.ai/code/session_01VqzDQEbeHPVJoaEoutGsTi)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent dc532e8 commit f7b6019

1 file changed

Lines changed: 40 additions & 1 deletion

File tree

‎.github/workflows/ci.yml‎

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -714,6 +714,45 @@ jobs:
714714
echo 'Items on this shard (a package name, or a package plus a k/n file-level slice):'
715715
cat "$RUNNER_TEMP/shard-packages.txt"
716716
717+
# ⭐ A SHARD WHOSE TESTS READ NO `.d.ts` BUILDS ITS CLOSURE WITHOUT THEM
718+
# (#25375). vitest loads JavaScript, so on most shards every declaration
719+
# the closure emits -- spec's 6 GB-heap pass, cli's type-checking `tsc`
720+
# -- is written and never read. `OS_SKIP_DTS=1` goes to $GITHUB_ENV, so
721+
# both closure builds below, the test step and the shard's main seed all
722+
# hash in one namespace (turbo.json `globalEnv`).
723+
#
724+
# ⛔ EXCEPT a shard carrying a package in DTS_READERS, whose tests compile
725+
# against BUILT declarations and go red without them (TS7016). Measured
726+
# on 830a63a, the whole Test Core list under `OS_SKIP_DTS=1`: red only in
727+
# @objectstack/cli published-subpath-console/-hook-body.pin (its
728+
# own dist/*.d.ts), scaffold-emission-typechecks,
729+
# scaffold-test-script-passes (spec's .d.ts)
730+
# @objectstack/metadata typescript-serializer-annotation (spec's .d.ts)
731+
# A new test that reads a built `.d.ts` reds the same loud way on a
732+
# skipping shard: add its package here. Every other package's counts
733+
# matched the DTS-on run but one: spec's root-entry-type-nameability
734+
# pin skips (by its own design) and runs in lint.yml's built-dist step.
735+
#
736+
# Local cold closure, 65 packages, 4 cores, `--force`: 5m59s with
737+
# declarations, 1m24s without. The cost is replay: a skipping shard
738+
# cannot replay Build Core's DTS-on remote artifacts, only the DTS-off
739+
# ones the queue and main runs of these shards write.
740+
- name: Choose this shard's declaration build
741+
run: |
742+
DTS_READERS='@objectstack/cli @objectstack/metadata'
743+
KEEP=""
744+
while read -r PKG _; do
745+
for R in $DTS_READERS; do
746+
[ "$PKG" = "$R" ] && KEEP="$KEEP $PKG"
747+
done
748+
done < "$RUNNER_TEMP/shard-packages.txt"
749+
if [ -n "$KEEP" ]; then
750+
echo "Declarations kept: this shard tests$KEEP, which read built .d.ts."
751+
else
752+
echo "OS_SKIP_DTS=1" >> "$GITHUB_ENV"
753+
echo "Declarations skipped: OS_SKIP_DTS=1 for the rest of this shard."
754+
fi
755+
717756
# ⛔ THIS SHARD'S `^build` CLOSURE IS BUILT HERE, ON THE TURBO REMOTE
718757
# CACHE (#22077), so `Run this shard's tests` REPLAYS it from the local
719758
# cache instead of building it. Before this step the closure was built
@@ -729,7 +768,7 @@ jobs:
729768
# 3d9188502e, turbo 2.11.5, all six shards of the local partition: this
730769
# plan equals the test plan's build tasks, 32 to 63 per shard, with
731770
# identical hashes, 0 differing and 0 extra, and every one of them has
732-
# Build Core's hash. A `test` task that `dependsOn: ["build"]` (cli and
771+
# Build Core's hash -- unless the step above set OS_SKIP_DTS (#25375). A `test` task that `dependsOn: ["build"]` (cli and
733772
# metadata in turbo.json) also needs its OWN package's build. This filter
734773
# leaves that out on purpose, so a shard never builds a package its tests
735774
# do not need; the test step builds it as before. Today that is

0 commit comments

Comments
 (0)