Skip to content

[measurement] compat-broad per-file CI timing (temporary, do not merge) - #3

Closed
t-k wants to merge 8 commits into
feat/compatibility-inventoryfrom
work/ci-retirement
Closed

t-k wants to merge 8 commits into
feat/compatibility-inventoryfrom
work/ci-retirement

Conversation

@t-k

@t-k t-k commented Sep 25, 2026

Copy link
Copy Markdown
Owner

Temporary draft PR to run the broad-timing workflow on the CI runner (per-file durations for duration-based compat-broad shards). It will be closed without merging; the retirement changes land on feat separately after review.

t-k added 8 commits September 25, 2026 09:05
…ist and stay pinned

closure_records.py reads every spec/compatibility/closure/*.json: each cited
repository path must exist, a #fragment must name an entry (a heading anchor in
Markdown), a <name>Path / <name>Sha256 pair must match, and cited records under
spec/ and conformance/ are pinned in record-digests.json so they cannot change
without an explicit --write in the same commit. A closure may not cite a suite
that CI no longer runs (RETIRED_SUITES in the compatibility-inventory workflow).
…on doc

The checked-in matrix cites file:line locations that moved with later code
changes (rest/mod.rs and others), so test_fs_config_surface_matrix and
test_fs_config_docs failed. Regenerated with surface_matrix.py --write and
doc_render.py --write; only the cited line numbers change.
…ee directory

test_historical_worktree_parent_accepts_the_real_repository_parent required the
developer checkout's .worktree directory and failed on every CI runner. Build a
real worktree parent under tmp_path instead; the refusal test for a symlinked
parent is unchanged.
…shard by duration

Owner decision 2026-09-25. The retired test files are listed once in the
workflow's RETIRED_SUITES with the reasons: each fails because 866bb06 moved
the shared request_bytes_preflight.py to the sandbox project (passes at
866bb06^, fails at 866bb06), because an O7/O8 shadow or runtime binding
predates later source changes, because it fails only on the CI runner, or
because it alone exceeds the 45-minute job limit. They run on demand through
workflow_dispatch (run_retired) with the O7-era history.py frozen-bytes check;
no test code or expectation changed.

compat-broad shards are planned from measured per-file durations (7 shards,
about 27 minutes for the fullest). A new required job checks the records every
closure cites and runs the closure contracts.
Artifact-backed tests refuse a dirty working tree, and the list written into
the checkout made it dirty.
fs-rules-publication/test_o5_user_token_comparator_v2.py builds a fully bound
collection per test (about 2 minutes each, 178 tests), so no file-level plan
fits the 45-minute limit. broad_shards.py now runs a file longer than 0.8 of
the budget as contiguous chunks of whole tests, rebases the collected IDs on
the repository path, and plans 22 shards (about 31 minutes for the fullest).

Also retire fs-write-limits/test_limits_03_baseline_prep.py: limits-03 takes
its project from the shared request-byte preflight, which now names the
sandbox, while baseline preparation requires fireemu-35fe6.
@t-k

t-k commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

Landed on feat/compatibility-inventory as 769f7c7..8ddfd22 (fast-forward), without the temporary per-file timing workflow (2558763 and its removal 53e36ad). The feat tree is identical to this PR's head 63003eb, which ran as compatibility-inventory run 36092961823.

@t-k t-k closed this Sep 25, 2026
@t-k
t-k deleted the work/ci-retirement branch September 25, 2026 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant