[finding] the generated shard-timings dataset has no scheduled refresh, and its documented refresh path is unreachable from an agent container #16222

scripts/test-shard-timings.json is GENERATED and nothing refreshes it on a
schedule. Its own provenance.refresh names two manual paths, and both are
weaker than they read:

1. "Download all six test-core-run-summary-N-of-6 artifacts from any green
   queue build." — measured from an agent container while working CI: the shard-timings file is stale for the CLI package — 672s predicted vs 28m46s measured against a 30-minute timeout, so Test Core shard 1/6 is one slow run from being killed on any PR touching the CLI #16173:
   GET /repos/.../actions/artifacts/{id}/zip redirects to
   productionresultssa11.blob.core.windows.net, and the session egress policy
   answers 403 to CONNECT. The artifacts existed and were unexpired; the
   channel is what is missing. Job logs are reachable through the MCP
   GitHub tooling, which is how the numbers on CI: the shard-timings file is stale for the CLI package — 672s predicted vs 28m46s measured against a 30-minute timeout, so Test Core shard 1/6 is one slow run from being killed on any PR touching the CLI #16173 were obtained, but the
   summary JSON the generator consumes is not printed to a log — only its path
   on the runner is.
2. "Or a local pnpm exec turbo run test --concurrency=4 --summarize." — a
   whole-workspace test run. The agent containers cap a foreground command at
   about ten minutes and the shared heavy-verify lock budget is 540s, and
   @objectstack/cli alone now measures 1231.52s. Not reachable either.

So the refresh is, in practice, maintainer-only and manual, for a file whose
entire design premise is that regenerating it is cheap. It was last measured
2026-08-24; thirteen days later one entry was wrong by 2.69x and a shard was
killed by the 30-minute wall.

Why this is separable from #16173

#16173 gets detection — a per-shard predicted-vs-measured drift check that
reds. Detection tells you the file has rotted; it does not un-rot it, and the
next person to see that red still has to reach a channel that is closed to
agent seats. This card is the prevention half of the question #16173 itself
poses: "Ask what makes it go stale and whether anything refreshes it on a
schedule." #16173 does not answer it and is not expected to.

The shape that looks cheapest, offered as a lead and not a ruling

The download only exists because the regeneration happens somewhere the
summaries are not. Inside a merge_group job the summaries are already on
disk — .turbo/runs/ in the workspace, which is exactly what the existing
upload step publishes. A rollup leg (or a scheduled workflow) that collects the
six shards' summaries in-runner and runs the existing generator needs no
artifact download at all, and could open a PR with the regenerated dataset.

⚠️ Whoever takes this should expect the regenerated file to red
partition-test-shards.mjs --self-test's balance pins today — measured on
#16173, substituting the true CLI weight puts the heaviest single package at
1.54x the mean against a 1.3x bound. That is those pins working as designed, and
it is a decision sitting on #16173, not a defect in the refresh path. Sequencing
matters more than usual here.

Refs: #16173 (the stale-dataset bill, and where these numbers were measured).

Generated by Claude Code