Repository navigation
Commit 03243fc
ci: prune each Test Core Turbo seed to its own run's tasks before saving it (#25523)
<!-- claude-projects-attribution: {"github_login":"os-elon-musk"} -->
_Requested by **Elon** · [project
thread](https://claude.ai/code/project/chan_01WedDzV26nniwxv7jC88B9D?thread=cmsg_01WedDzV26nniwxv7jC88B9DTQm7fHdg9y6VYJBP9Rc2ab)_
Fixes #25452
Clause-②: no
Metadata-protocol: none
Before each Test Core shard saves its Turbo seed on a push to `main`,
the seed is now pruned to the cache entries its own run used. The Test
Core seeds were not 2.4 GB of useful cache. They were one seed lineage
that grew by one run per merge and was never trimmed. Pruning fixes this
at the source, so the card's change (b) is the only change here. Change
(a), dropping the Build Core / Type Check seeds, and change (c), the
Next.js cache, are left alone: the measurement below shows those seeds
are 0.05 GB each, not where the pool goes.
## Step 1: the measurement (save-step and restore-step logs; the pool
API is refused from the cloud container)
**Bytes saved per merge** (push run 39982654692, `main` at `05cc3ee953`,
a completed run; read from each save step's `Sent N of N (100.0%)`
line):
| seed key prefix | per seed | seeds per merge | per merge |
| :-- | --: | --: | --: |
| `Linux-turbo-test-<1..6>-main-` | 2.410 – 2.441 GB | 6 | **14.60 GB**
|
| `Linux-turbo-build-core-main-` | 0.048 GB | 1 | 0.048 GB |
| `Linux-turbo-dogfood-<1..3>-main-` | 0.048 GB | 3 | 0.145 GB |
| `Linux-turbo-typecheck-main-` (lint.yml run 39982654477) | 0.054 GB |
1 | 0.054 GB |
| **total** | | | **≈ 14.85 GB** (Test Core = 98 %) |
Each merge writes about 1.5× the repository's 10 GB default pool. The
jobs API shows every one of the six Test Core save steps succeeding on
five more push runs from 16:23Z to 17:56Z (39986349408, 39989699728,
39994571067, 39996442855, 39999263178), including the cancelled runs.
**Eviction, seen from a PR restore.** On PR run 39999049930 (17:55Z),
`Test Core (1/6)` (job 120570291429) restored
`Linux-turbo-test-3-main-fa06169c…` (2.50 GB). That is **shard 3's**
seed, reached through the bare `Linux-turbo-test-` restore key. The push
at 17:36Z (run 39996442855) had saved a shard-1 seed, but no
`Linux-turbo-test-1-*` entry was left 19 minutes later.
**Why 2.4 GB: the seed snowballs.** A full 76-task workspace build is
0.048 GB in Build Core's seed, and a shard's closure is a subset of that
build. The restore step takes the newest seed by prefix, turbo never
evicts its local cache, and the save step stored the whole directory. So
every seed was the previous seed plus one run, and the bare-prefix
fallback carried the lineage from shard to shard. That is why all six
shards are the same size. The sizes keep growing: shard 3 was 2.441 GB
at `05cc3ee9` (16:20Z) and 2.50 GB at `fa06169c` (17:36Z). The same
growth reproduces on a two-package turbo 2.11.5 workspace: one input
change took `.turbo/cache` from 9 files to 18, and pruning to the run
summary took it back to 9 with a full `3 cached, 3 total` replay.
**Pool total and largest entries: pending.** `GET /actions/cache/usage`
and `/actions/caches` answer 403 from a cloud container ("Access to this
GitHub Actions path is not permitted through this proxy"). The devx seat
has asked the maintainer for the two reads, and they will be added here
when they arrive.
## What changed
- **`scripts/ci/prune-turbo-cache.mjs` (new).** It reads every run
summary (`--summarize`) in `--run-summaries`, plus any
`--extra-summaries`, and collects each task's `hash`. It then deletes
every `<hash>.tar.zst` / `<hash>-meta.json` / `<hash>-manifest.json` in
`.turbo/cache` whose hash no summary names, and appends
`save=true|false` to `GITHUB_OUTPUT`.
- **No readable run summary means `save=false`.** That covers a
cancelled test step or an empty shard. An extra summary alone (the
closure build's) does not count, because saving builds without test
results would make the newest seed one that replays no tests.
- A file in any other shape is **kept and reported**, never deleted, so
a turbo layout change shows up as a count and does not empty the cache.
- The script exits 0 on every run-time path, so it cannot fail the shard
or withhold its attestation. Malformed arguments exit 3.
- `--self-test` covers 25 cases across 5 batteries with a pinned battery
floor and the verdict handshake.
- **`ci.yml`, Test Core:** a new step, `Prune the Turbo cache to this
run's tasks (main only)` (`if: always() && github.event_name ==
'push'`), runs before `Save Turbo cache (main only)`. It reads
`.turbo/runs` (the test step's summary) plus
`$RUNNER_TEMP/turbo-remote-cache` (the closure build's summary, moved
there by the remote-cache assertion). The save step's `if:` gains `&&
steps.prune-turbo-cache.outputs.save == 'true'`.
- **`ci.yml`, Build Core:** a new `prune-turbo-cache self-test` step
next to `assert-turbo-remote-cache self-test`, and the script is added
to the `core` filter's guard rows (`check-ci-filter-parity` and
`check-self-test-wired` both require it).
**Kept on purpose: the bare `Linux-turbo-test-` restore key.** Once
seeds are pruned, a cross-shard restore brings at most another shard's
pruned seed, about one closure's worth. The shared build entries still
replay, and the prune removes the rest before the next save, so the
fallback can no longer carry a lineage forward. Removing it would also
take away the warm start its comment describes for a new shard number.
## Expected after
Each Test Core seed should hold its shard's closure builds plus its test
results, which is well under Build Core's 0.048 GB full build plus test
logs. A merge would then write about 0.3–0.6 GB in total instead of
about 14.85 GB. A side effect: PR shards stop downloading about 2.4 GB
in `Restore Turbo cache`, which measured 15–46 s per shard on the PR
runs above. The first push after landing still restores the old snowball
once, prunes it, and saves the small seed. The old 2.4 GB entries then
age out of the pool.
## Acceptance (from the card)
- **Before:** the per-merge table above. The pool total and its four
shares will be added when the maintainer's `gh api` reads arrive.
- **After, owed after ≥ 5 merges to `main` post-landing:** the Test Core
save-step sizes, the pool total (target ≤ 8 GB at steady state), and two
readings an hour apart. Over the next 10 PR runs whose closure matches
`main`: the median of the shard "Build this shard's dependency closure"
step ≤ 1.7 min and Build Core's build step ≤ 2.5 min. The
`assert-turbo-remote-cache` hit counts will be recorded before and
after.
- ⛔ The pool limit is not raised here. If the pool is still over budget
after this lands, the billing option goes back to the maintainer with
the numbers.
## Risks and rollback
- **Test replay on `main` depends on this seed.** `test` / `test:repo`
are off the remote cache (Build Core's PINS), so a pushed shard replays
unchanged packages' tests only from the Actions seed. The prune keeps
every hash the test step's summary names, which covers every test task
and every build it depended on, so the replay set is unchanged. A test
task that failed has no entry either way.
- **A shard whose test step left no summary saves nothing.** The
previous seed stays the newest, and the next completed push writes a
fresh one.
- **Rollback:** revert this PR. It touches no package and changes no
restore key.
## Local verification (head fcae6e3)
- `node scripts/ci/prune-turbo-cache.mjs --self-test`: OK, 25 cases
across 5 batteries. **Firing proof:** with the keep rule ablated to keep
everything (a temporary copy of the script), the self-test exits 1 with
5 failures (`every file of an unnamed hash is dropped`, `an inherited
entry no summary names is deleted`, …). The copy was deleted afterwards.
- A real turbo 2.11.5 workspace: prune to the run summary, delete
`dist/` and the summaries, re-run, and the result is `Cached: 3 cached,
3 total` with outputs restored.
- All exit 0: `check-self-test-wired` (+ `--self-test`),
`check-ci-filter-parity` (+ `--self-test`), `check-step-collectors` (+
`--self-test`), `check-self-test-workflow-commands`,
`check-published-list-mirrors`, `check-declaration-mirrors`,
`check-platform-checklist-watchdog`, `check-empty-changeset --base
origin/main`, `check-adr-0087-registration --base origin/main`,
`check-aggregator-roster`, `check-closing-keyword-parity`,
`check-comment-mask-corpus`, `check-shard-attestation --self-test`, and
`pnpm check:` `workflow-status-functions`, `workflow-step-name-quoting`,
`nul-bytes`, `merged-result`, `pm-governed-merges`, `pm-expected-skips`,
`single-claim-paths`, `pnpm-filter-targets`, `pnpm-flag-spellings`,
`scripts-symbol-anchors`, `runner-env-posture`, `node-version`,
`pnpm-acquisition`, `select-shard-packages`, `shard-attestation`,
`required-contexts`, `stall-guard-budget`, `stall-guard-headroom`,
`issue-citations`, `optional-error-sink`, `doc-authoring`. The rest of
the farm is left to CI.
- `ci.yml` parses (`yaml`); eslint is clean on the new script.
- `check-governed-merges.mjs --test .github/workflows/ci.yml
scripts/ci/prune-turbo-cache.mjs`: GOVERNED, landing tier H
(`.github/workflows/**` ×1), 331 changed lines.
- `skip-changeset`: no published package is touched
(`.github/workflows/**`, `scripts/ci/**`).
## Acceptance notes
- **Branch name.** The branch is `claude/project-thread-53z7nt`, the
name the session was given. The claim (comment 6371859957) names it.
- **Card changes (a) and (c) not taken.** Measured at 0.05 GB per seed,
they are not where the pool goes. Revisit them only if the after-reading
still exceeds 8 GB.
## 维护者速读(草稿)
**改了什么**:main 每次合并后,Test Core 的 6 个分片会各存一份 Turbo
缓存种子。现在存之前先把种子裁剪到「这次运行真正用到的任务」,其余旧条目删掉。如果测试步骤没有跑完(被取消或空分片),这次就不存。
**为什么改**:实测每份种子 2.4 GB,每次合并共写约 14.6 GB,而仓库的 Actions 缓存池只有 10
GB,结果是每合并一次,上一次的种子就被挤掉。原因是种子在滚雪球:每次都在恢复的旧种子上再加一轮,从不清理。完整构建一次的产物其实只有 0.05
GB。裁剪后预计每次合并只写 0.3–0.6 GB,PR 运行也不用再下载 2.4 GB 的种子。
**风险与代价(含回滚)**:main
上测试结果的复用只靠这份种子,所以本次运行用到的条目(含全部测试结果)一律保留,复用范围不变。没跑完的运行不存种子,旧种子继续可用。回滚只需
revert,不动任何包和恢复键。缓存池总量的读数还在等,拿到后补进正文。
**席位意见**:同意落地(devx 席位已在 head fcae6e3 复核,ACCEPT 见
#25452)。日志已证实种子在滚雪球:每次合并约 14.6 GB,远超 10 GB
的缓存池。修剪只保留本次运行用过的任务,失败模式偏保守:认不出的文件保留,没有运行摘要就不存。改动前的缓存池总量等你本机读出后补进正文,不影响审批。
**你要做的**:本 PR 改了 `.github/workflows/**`,属于 Tier H,需要 GOVERNED_APPROVERS
中的一位批准后,席位才会翻 ready 并落地。
---
_Generated by [Claude
Code](https://claude.ai/code/session_01SawdJgiuoz8deqVdM51jAE)_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01SawdJgiuoz8deqVdM51jAE
Co-authored-by: Claude <noreply@anthropic.com>1 parent 0949fe5 commit 03243fc
2 files changed
Lines changed: 410 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
294 | 294 | | |
295 | 295 | | |
296 | 296 | | |
| 297 | + | |
297 | 298 | | |
298 | 299 | | |
299 | 300 | | |
| |||
1459 | 1460 | | |
1460 | 1461 | | |
1461 | 1462 | | |
| 1463 | + | |
| 1464 | + | |
| 1465 | + | |
| 1466 | + | |
| 1467 | + | |
| 1468 | + | |
| 1469 | + | |
| 1470 | + | |
| 1471 | + | |
| 1472 | + | |
| 1473 | + | |
| 1474 | + | |
| 1475 | + | |
| 1476 | + | |
| 1477 | + | |
| 1478 | + | |
| 1479 | + | |
| 1480 | + | |
| 1481 | + | |
| 1482 | + | |
| 1483 | + | |
1462 | 1484 | | |
1463 | 1485 | | |
1464 | 1486 | | |
1465 | 1487 | | |
1466 | 1488 | | |
1467 | | - | |
| 1489 | + | |
1468 | 1490 | | |
1469 | 1491 | | |
1470 | 1492 | | |
| |||
2619 | 2641 | | |
2620 | 2642 | | |
2621 | 2643 | | |
| 2644 | + | |
| 2645 | + | |
| 2646 | + | |
| 2647 | + | |
| 2648 | + | |
| 2649 | + | |
2622 | 2650 | | |
2623 | 2651 | | |
2624 | 2652 | | |
| |||
0 commit comments