Repository navigation
ci(test-core): a shard whose tests read no .d.ts builds its closure with OS_SKIP_DTS=1 - #25465
Conversation
…ith OS_SKIP_DTS=1 A new step after "Compute this shard's package set" writes OS_SKIP_DTS=1 to $GITHUB_ENV unless the shard carries @objectstack/cli or @objectstack/metadata, so the closure build, the slice build, the test step and the main seed hash in one namespace (turbo.json globalEnv). Those two are kept on DTS because their tests compile against built declarations: under OS_SKIP_DTS=1 the whole Test Core list on 830a63a went red only in cli's published-subpath-console/-hook-body pins and scaffold typecheck tests, and in metadata's typescript-serializer-annotation test (TS7016). Every other package's test counts matched the DTS-on run except spec's root-entry-type-nameability pin, which skips by design and runs in lint.yml's built-dist step. Local cold closure, 65 packages, 4 cores, --force: 5m59s with declarations, 1m24s without. Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VqzDQEbeHPVJoaEoutGsTi
…is PR's closure Measures the cold DTS-on closure build for the same closure the previous push built cold with OS_SKIP_DTS=1. Reverted by the next push. Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VqzDQEbeHPVJoaEoutGsTi
…RS and --force The measurement it ran is posted on the PR. ci.yml is back to 8d58beb's. Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VqzDQEbeHPVJoaEoutGsTi
|
Measurement on this PR, as the card's step 3 asks: the same closure, cold both ways. This PR's diff selects one package,
Push 1 took 40% of push 2's closure time, against the card's ceiling of 60%. The local full-closure reading in the PR body says the same thing at larger scale. The 65-package closure took 5m59s with declarations and 1m24s without. Push 2 existed only to measure. The next commit, be7daa3, reverts it, so ci.yml on head e62c37b is byte-identical to 8d58beb's. Generated by Claude Code |
Requested by Elon · project thread
Fixes #25375
Clause-②: no
Metadata-protocol: none
维护者速读(草稿)
改了什么:Test Core 六个分片在建依赖闭包前多一步「Choose this shard's declaration build」。分片里没有
@objectstack/cli、@objectstack/metadata时,整个分片设OS_SKIP_DTS=1,构建只出 JavaScript,不再生成.d.ts(spec 那次 6 GB 堆的声明构建、cli 的完整tsc都省掉)。带这两个包的分片照旧。只改.github/workflows/ci.yml。为什么改:vitest 只加载 JS,大多数分片生成的声明文件没人读。本地 4 核冷构建同一闭包(65 个包):开声明 5m59s,关声明 1m24s。卡面原方案是六个分片全关,但实测全关会让 cli(4 个测试文件,24 例)和 metadata(1 个文件)红,原因都是 TS7016 找不到 spec / cli 的
.d.ts,这些测试本来就是在编译已发布的声明。所以这两个包所在的分片保留声明。风险与代价(含回滚):① 关声明的分片和 Build Core 不在同一个 turbo 哈希空间(
turbo.json的globalEnv),读不到 Build Core 写进远端缓存的带声明产物,只能读本分片在合并队列和 main 上写的无声明产物;落地后头几轮会偏冷,之后自愈。② 以后若有新测试读已建好的.d.ts却落在关声明的分片,会以 TS7016 明确变红,修法是把它的包加进DTS_READERS,不会静默放过。③ 全量跑时 cli 切 3 片、metadata 占 1 片,当前切法下只有分片 1、6 省;PR 只跑受影响包时省的分片更多。回滚:删掉这一步即可。席位意见:同意落地(devx 席位已在 head e62c37b 复核,ACCEPT 见 #25375)。机制早已在 main 上(
turbo.json的globalEnv和各包 tsup 配置),本 PR 只决定哪些分片打开它。收窄到不含 cli、metadata 的分片是对的;漏掉的读者会以 TS7016 明确变红,不会静默放过。落地后头几轮偏冷属预期。你要做的:以 GOVERNED_APPROVERS 账户审批本 PR(受管面 Tier H)。
What changed
One new step in the
testjob, right after "Compute this shard's package set":it writes
OS_SKIP_DTS=1to$GITHUB_ENVunless the shard's package listcarries a package in
DTS_READERS(@objectstack/cli @objectstack/metadata).Through
$GITHUB_ENVit reaches both closure builds, the test step and themain-push seed, so all of them hash in one namespace (
turbo.jsonglobalEnv).turbo.jsonis untouched.The card's step 2: tests that read a
.d.tsThe card's premise "vitest never reads a
.d.ts" is false for two packages.The static grep the card names gives 70 test files that mention
.d.ts, so Imeasured instead. Under
OS_SKIP_DTS=1, the whole Test Core list (dogfoodexcluded,
OS_TEST_TIERS=queue) ran on 830a63a in a cloud container. Thesewent red:
@objectstack/clitest/published-subpath-console.pin.test.ts(6 tests)dist/console.d.ts@objectstack/clitest/published-subpath-hook-body.pin.test.ts(4)dist/hook-body.d.ts@objectstack/clitest/scaffold-emission-typechecks.test.ts(12)dist/**/*.d.ts(TS7016)@objectstack/clitest/scaffold-test-script-passes.test.ts(2)dist/**/*.d.ts(TS7016)@objectstack/metadatasrc/serializers/typescript-serializer-annotation.test.ts(1)dist/**/*.d.ts(TS7016)@objectstack/example-showcasewas killed (exit 137) in that run because thelocal container ran out of memory. Run alone under
OS_SKIP_DTS=1it passed:37 files, 443 tests.
Silent-skip check: 32 packages own a test that compiles a TS program or names a
.d.ts. I ran them again with declarations on and compared the vitest totalsper task. All 38
test/test:repototals were identical except one.@objectstack/spec:testran 21634 passed with DTS and 21632 passed plus 1skipped without it. The difference is
scripts/root-entry-type-nameability.pin.test.ts, which skips by its own designwhen spec's dist declarations are absent. That pin runs with
OS_EXPECT_ROOT_NAMEABILITY=1against a built dist in lint.yml'sroot-entry-type-nameability-pin-built-diststep, so its coverage holds.Measurement
Local, cloud container, 4 cores, the same 65-package closure (every package
whose
buildthe full Test Core list schedules),turbo run build --force --concurrency=4:@objectstack/spec#build@objectstack/cli#buildOS_SKIP_DTS=1That is 23% of the DTS-on wall time, against the card's ceiling of 60%.
Today's baseline on CI, from the card: cold closure step 3.1 to 6.0 min per
shard on PR run 39891259937, about 31 runner-minutes in total.
On main after the shard-timings refresh (#25409), the full list puts metadata
on shard 2 and the three cli slices on shards 3 to 5, so shards 1 and 6 skip
on a full run. On a PR the affected set decides. This PR's own diff selects
only
@objectstack/driver-sql(a declared ci.yml input), so its first runmeasures a cold DTS-off closure. Nothing in the remote cache has DTS-off
hashes yet. The CI step timings from this PR, both ways, are posted as a PR
comment.
Replay trade-off: a skipping shard cannot replay Build Core's DTS-on remote
artifacts. In steady state that costs nothing. Every package hash a merge
introduces is built in the queue run's shard closures, which write the remote
(
remote:rwonmerge_groupand main pushes). That gives later PRs a DTS-offhit for every package they do not change. The packages a PR does change miss
both ways, and they build faster without declarations. The cost is real only
in the first runs after landing.
Gates
node scripts/pm/dispatch-gates.mjs --commandslists 54 commands. All 54 exit 0on 8d58beb.
check:dts-closureandcheck:dual-build-cjs-loadswere re-runafter a full DTS-on
pnpm build, because they read a built tree.actionlintis not installed here. The workflow parses as YAML, and the job's step order
is unchanged around the new step.
Acceptance notes
declarations, for the reason above. The other way to save on those shards is
to keep only spec's and cli's declarations under a per-package variable.
That needs
tsup.config.*, cli'sbuildscript andturbo.jsonglobalEnv, which is outside this card's file surface, so it is not donehere.
.github/workflows/test-nightly-tiers.ymlbuilds its own shard closure andis not changed here.
🤖 Generated with Claude Code
https://claude.ai/code/session_01VqzDQEbeHPVJoaEoutGsTi
Generated by Claude Code