Skip to content

ci: skip the redundant Tier 2 configure on the Dell's kept build tree - #5065

Merged
zackees merged 3 commits into
masterfrom
infra/dell-configure-skip
Nov 19, 2026
Merged

zackees merged 3 commits into
masterfrom
infra/dell-configure-skip

Conversation

@zackees

@zackees zackees commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

The Dell's kept Tier 2 tree no longer restores/imports libobs module-cache archives, which restored 0/7 modules in five measured jobs and forced a second configure. Hosted runners retain their existing module cache and unconditional configure.

For kept-tree jobs, validate.sh passes its exact configure argv to ci/tier2_configure.py. An atomic stamp inside the saved tree records a SHA-256 fingerprint of argv boundaries/order, configure environment, tool paths/content, installed Ubuntu packages, toolchain/config files and the git-derived OBS version. Reuse requires matching cache/Ninja/rules contents, cached compiler/toolchain files and a working Ninja regeneration rule. Missing, changed or unverifiable inputs configure; failed configure invalidates the stamp. Preload scripts (-C), presets and --fresh always configure because their arbitrary includes may not be Ninja dependencies.

Version correctness limits the saving: OBS embeds git describe during configure. A changed version therefore forces configure, including across ordinary new commits. The first-configure saving applies to unchanged-version retries; removing the module-cache configure also benefits changed revisions. Ninja still performs CMakeLists/include/CONFIGURE_DEPENDS regeneration through cmake --build. No build/test verdicts are reused.

Local RED: original 72a8aafc2, rebased 4471bf17c, uv run --no-project python -m unittest ci.test_tier2_configure, seven errors because the helper did not yet exist. RED and GREEN are published together under the focused CI/configuration exception.

Validation:

  • Prior draft validation at 346c0cdd201e2492bca2e24da282221982d0e505 (historical evidence, refreshed below). ./push completed with 18 partial prooft Ci-Attestation records: six-target Clippy/Dylint, general lint, complete Python self-tests and four data validators. The head has no complete signed client evidence; ordinary PR CI remains required.
  • The optional host proof was stopped after 35 minutes: nextest reported 8,933 passed, two interrupted, six skipped and 13 not run. The two interrupted cases were existing native-build fixtures. Host-test, feature and GNU-link proof completion is not claimed. This local proof is incomplete; no test tier or assertion was removed from CI.
  • timeout 1200 env SOLDR_PORCELAIN=0 ./lint: passed in 1,150.0 seconds (including self-tests and Rust analysis).
  • 35 focused Python/shell tests pass: argument/environment/package/toolchain changes, missing/edited cache or graph, failed configure, changed git version, hosted behavior, module-cache exclusion, preload includes, Ninja regeneration and the real validate.sh configure block.
  • ci/test_tier2_fanout.py: 9 tests pass; ci/test_uv_run.py: 6 policy checks pass. The fingerprint hashes the running Python interpreter via sys.executable.
  • Read-only clud-review: clean after fixing the preload-script regression.
  • Small Ubuntu validation-image fixture before the final interpreter-path correction, with unmocked package/tool fingerprint: configure 3.977 s; matching-tree decision 2.0112 s. This is a small fixture, not a full OBS saving.
  • Full retained OBS-tree timing was attempted, but stalled after XCB checks and was stopped. No post-change Dell timing is claimed. Existing Dell measurements put a warm configure at 128–133 s and the expected combined opportunity at about 2–3.5 min; changed git versions conservatively retain the first configure.

This changes CI configuration only. No new workflow files, C tests, public headers, WebSocket plugin files, Rust test binaries or runtime/protocol behavior changes. New port test tiers are not applicable.

Refs #4954 #4896.

Landing validation (2026-11-19):

  • Rebased onto master b99407fb1a1d6eb76799c07076815d5b9f4898bb; no conflicts. git range-diff reports all three commits unchanged in content. This preserves master’s apt bounds (ci: bound Tier 2 apt acquisition and package steps (Refs #5213) #5221/ci: bound prepared-runtime apt and refresh missing pins once (Refs #5237) #5249), link-check split (ci: parallelize full Tier 2 cross-link checks (Refs #5224) #5231), LLVM tools reuse (ci: cache Darwin dsymutil between artifact preparations (Refs #5199) #5251) and CI Gate move (ci: run gate and cache janitor on ubuntu-latest (Refs #5256) #5264).

  • Confirmed master still configures unconditionally and has no equivalent configure fingerprint/skip; this PR is not superseded.

  • Focused configure/module-cache/build-setup suite: 35 passed in 36.976 s. Fanout: 9 passed. Apt delta, CMake cross-link, CI Gate, kept-tree and Python policy suites: 69 passed. git diff --check passed.

  • The manager explicitly lifted the draft publication hold. Full Dell timing remains unmeasured.

  • Fresh standalone timeout 1800 env SOLDR_PORCELAIN=0 ./lint passed in 581.1 s (exit 0; report parser returned unknown, confirmed final == lane lint: ok).

  • Fresh publisher client lint passed in 682.5 s, including Clippy and Dylint on all six targets; full client self-tests and data validation also passed.

  • Optional broad host proof was interrupted after 24 minutes: 2,648 passed, five SIGINT-interrupted, six skipped, 6,592 not run. No assertion failure was observed before interruption. Complete host/feature/GNU-link proof is not claimed, and ordinary PR CI remains required.

  • Rebased published head: f9260ce3ce54fe522bd45cf89c9d536442623bc1. ./push completed with 18 partial Ci-Attestation records and exact force-with-lease against 346c0cdd201e2492bca2e24da282221982d0e505. Doctests passed; interrupted host proof wrote no host-test attestation. Ordinary PR CI remains required.

@zackees zackees added the pool-active Pool worker owns this PR; automated rebases must skip it label Nov 18, 2026
@obs-rust-project-manager obs-rust-project-manager Bot moved this to In progress in obs-rust Multiview Nov 18, 2026
@zackees zackees removed the pool-active Pool worker owns this PR; automated rebases must skip it label Nov 18, 2026
@zackees zackees added the pool-active Pool worker owns this PR; automated rebases must skip it label Nov 19, 2026
Refs #4954 #4896. Local RED 72a8aaf: seven tests fail because the configure helper is missing. GREEN: 35 focused Python and shell tests pass. Reuse requires an exact configuration fingerprint and intact Ninja regeneration; hosted configuration is unchanged. Preload scripts and presets always configure. Disable the module cache only for kept-tree jobs. Preserve git-derived version correctness, even when that requires configure on changed revisions.

Ci-Attestation: {"at":1794990292,"gate":"general/all/client-lint","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"79f03a23f8b487aba466ca8f043d738b","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990293,"gate":"rust/aarch64-apple-darwin/client-clippy","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"0a2db6cc58d140a2fb92f1a0f8bdfa98","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990293,"gate":"rust/aarch64-apple-darwin/client-dylint","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"59a7399e01b603b4e69a765eefb92035","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990293,"gate":"rust/aarch64-pc-windows-msvc/client-clippy","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"6a5b4a09810755a2871e3fe3cc3c61c7","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990293,"gate":"rust/aarch64-pc-windows-msvc/client-dylint","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"7c75d382e1bafadca703aa5890f3e22c","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990292,"gate":"rust/aarch64-unknown-linux-gnu/client-clippy","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"e26837e86e626429ae2139611870bab3","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990292,"gate":"rust/aarch64-unknown-linux-gnu/client-dylint","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"e9b991593762add9eccba1e8e5c0827d","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990293,"gate":"rust/x86_64-apple-darwin/client-clippy","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"ee7b6e61409a91b740e86cec43fd95ad","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990293,"gate":"rust/x86_64-apple-darwin/client-dylint","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"53f3ad0359fab63bb22bc5674ac37119","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990293,"gate":"rust/x86_64-pc-windows-msvc/client-clippy","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"3bb7d7ae489597c0cc606219bba63a30","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990293,"gate":"rust/x86_64-pc-windows-msvc/client-dylint","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"252e8a02f59dbdb8e13207f9d5e7f81f","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990292,"gate":"rust/x86_64-unknown-linux-gnu/client-clippy","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"016ca1e84a2fc84da9b8fea67e2ace91","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1794990292,"gate":"rust/x86_64-unknown-linux-gnu/client-dylint","host":"linux-x86_64","key":"b61fd0ee84ab642cfb19323f86a08775bad0060bced52382314d8b5e450d573e","lane":"client-lint","parents":["72a8aafc2a0be2670f8ae470cf68f176978b22a1"],"secs":567,"stamp":"10b002151bff889f980ae5f187553e74","tree":"8d3547bb512531f104eb4c94d15d3929015fa9de","v":1,"via":"run"}
Ci-Attestation: {"at":1795117569,"gate":"data/all/compatibility","host":"linux-x86_64","key":"","lane":"client-checks","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":461,"stamp":"0e22f0dd81a1d2a6a5505ff34e5dd935","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117569,"gate":"data/all/flatpak-manifest","host":"linux-x86_64","key":"","lane":"client-checks","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":461,"stamp":"2593648e4218111443cfee1873068141","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117569,"gate":"data/all/qt-xml","host":"linux-x86_64","key":"","lane":"client-checks","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":461,"stamp":"df50dcbf0c66ececf3d58d6e68deb30c","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117569,"gate":"data/all/services-schema","host":"linux-x86_64","key":"","lane":"client-checks","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":461,"stamp":"ba4cfaf93dcb05a135ec282bb72dcf1c","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117098,"gate":"general/all/client-lint","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"825a16efa47a29ed8fe7d0ba19ada143","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117569,"gate":"python/all/self-test","host":"linux-x86_64","key":"","lane":"client-checks","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":461,"stamp":"afac3ef775660feab7aefd306c676b75","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117103,"gate":"rust/aarch64-apple-darwin/client-clippy","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"8f0a650135f7d0c5dcb21a2a5a407955","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117104,"gate":"rust/aarch64-apple-darwin/client-dylint","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"27f28f11a279d9cd03e56d3abd834525","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117101,"gate":"rust/aarch64-pc-windows-msvc/client-clippy","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"03edc6c83e99c5af8b1b7b8eaac0304a","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117102,"gate":"rust/aarch64-pc-windows-msvc/client-dylint","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"bc2ae9b83a6f333d3e132f8edb165a9c","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117100,"gate":"rust/aarch64-unknown-linux-gnu/client-clippy","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"8b1a061f6d86923c9fc63cbe5dd941c9","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117100,"gate":"rust/aarch64-unknown-linux-gnu/client-dylint","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"8eacdada2dc0ed8614119ba043f91967","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117102,"gate":"rust/x86_64-apple-darwin/client-clippy","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"66ff7edfb83379e4a932e450df3bfb99","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117103,"gate":"rust/x86_64-apple-darwin/client-dylint","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"e6bac8b762b163edf2b941d562c8ef37","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117100,"gate":"rust/x86_64-pc-windows-msvc/client-clippy","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"537f2f14872e00722b895ff697eecdf0","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117100,"gate":"rust/x86_64-pc-windows-msvc/client-dylint","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"1653dc2458f7dcea8ffec937685cd293","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117099,"gate":"rust/x86_64-unknown-linux-gnu/client-clippy","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"56e4b7eb77f5f8729ae49dd57521b7ba","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
Ci-Attestation: {"at":1795117099,"gate":"rust/x86_64-unknown-linux-gnu/client-dylint","host":"linux-x86_64","key":"0e218192eb7a83baab948293e6b3ed56f59347aa7c4b748e6c9ac92a6234ca0d","lane":"client-lint","parents":["794ee0927eb89a198177041c91596794f089a8c7"],"secs":682,"stamp":"ecde454efa002c556570bb721eebaf9d","tree":"3fc5f29725ee3aeba8feee277583545155f955b1","v":1,"via":"run"}
@zackees
zackees force-pushed the infra/dell-configure-skip branch from 346c0cd to f9260ce Compare November 19, 2026 20:12
@zackees
zackees marked this pull request as ready for review November 19, 2026 20:13
@zackees
zackees enabled auto-merge November 19, 2026 20:13
@zackees zackees removed the pool-active Pool worker owns this PR; automated rebases must skip it label Nov 19, 2026
@zackees
zackees added this pull request to the merge queue Nov 19, 2026
Merged via the queue into master with commit fb7d018 Nov 19, 2026
45 checks passed
@zackees
zackees deleted the infra/dell-configure-skip branch November 19, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

1 participant