perf(ci): run the web console suite in its own parallel job - #413
Merged
Conversation
It was ~170s sitting SERIALLY behind a ~1090s engine suite on the critical-path leg, for no reason beyond sharing a checkout. Run alongside instead of after, it costs zero wall-clock: setup plus suite finishes far inside the engine job it now runs beside. Expected critical path: ~1310s -> ~1160s on windows-2025. Read it off this PR's own run rather than trusting that estimate. THIS REVERSES THE PLACEMENT HALF OF BACKLOG #344 PROPOSAL 5 AND KEEPS ITS SUBSTANCE. That proposal gave the suite its own webconsole_step_timeout because two steps sharing one timeout-minutes meant the second one's cap could never fire first -- reaching it had already spent setup plus the engine suite. Moving it to its own job completes that rather than undoing it: the suite keeps its own STEP cap, gains its own JOB cap, and keeps its own margin check. The nesting invariant becomes setup(max) + webconsole_step_timeout < webconsole_job_timeout inside one job, instead of a sum of two step budgets straddling a single job cap -- strictly easier to satisfy and to check. COVERAGE IS PRESERVED BY ci-gate, NOT BY A NEW REQUIRED CONTEXT. The job is listed in ci-gate's needs; ci-gate is if: always(), fails on any needed job's failure or cancelled, and IS required. The console suite gated merges before by being a step inside the required test legs, and it must not silently stop gating them now that it is not. A new required context would instead have meant a branch-protection change and, until that landed, a PR that could never satisfy it. The five margin guards in tests/test_ci_step_margin.py are RE-SCOPED PER JOB, not relaxed: _GATED now maps job -> (gated step, step-cap key, job-cap key) and every invariant loops over it. The check step is located by its --since flag rather than its name, because "Step margin -- start the clock" also starts with "Step margin" and a name prefix would match a mark-writer and pass for the wrong step. The three web-console baseline rows kept their measurements and had their censoring reason marked historical: they were censored by the test job's cap under a topology that no longer exists. The numbers are real observations and still valid lower bounds; only the mechanism retired. Verified: ruff, ruff format, mypy strict, and all 291 tests that read ci.yml. The five new shell blocks were bash -n checked behind a positive control, because the repo's own test_workflow_shell_syntax guard cannot run on this box -- Git Bash receives a mangled Windows path, so it fails 161-of-161 locally regardless of content.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Moves the web console suite out of a second step on every
testleg and into its own parallelwebconsolejob.It was ~170s sitting serially behind a ~1090s engine suite on the critical-path leg, for no reason beyond sharing a checkout. Run alongside instead of after, it costs zero wall-clock -- setup plus suite finishes far inside the engine job it now runs beside.
Expected critical path ~1310s to ~1160s on windows-2025. Read the real figure off this PR's own run rather than trusting that estimate.
This reverses the placement half of BACKLOG #344 proposal 5, and keeps its substance
That proposal gave the suite its own
webconsole_step_timeoutbecause two steps sharing onetimeout-minutesmeant the second one's cap could never fire first -- reaching it had already spent setup plus the engine suite.Moving it to its own job completes that rather than undoing it. The suite keeps its own STEP cap, gains its own JOB cap (
webconsole_job_timeout), and keeps its own margin check. The nesting invariant becomessetup(max) + webconsole_step_timeout < webconsole_job_timeoutinside one job, instead of a sum of two step budgets straddling a single job cap -- strictly easier both to satisfy and to check.Coverage is preserved by ci-gate, not by a new required context
webconsoleis inci-gate'sneeds.ci-gateisif: always(), fails on any needed job'sfailureorcancelled, and is a required context -- the same mechanism that already gates the six path-gated legs.This mattered: the console suite gated merges before by being a step inside the required
testlegs, and it must not silently stop gating them now that it is not. A new required context would instead have meant a branch-protection change and, until that landed, a PR that could never satisfy it.The guards are re-scoped, not relaxed
_GATEDintests/test_ci_step_margin.pynow maps job to (gated step, step-cap key, job-cap key), and all five wiring invariants loop over it. None was weakened; each is applied where the steps actually live.One detail worth flagging: the check step is located by its
--sinceflag rather than its name, becauseStep margin -- start the clockalso starts withStep margin-- a name prefix would match a mark-writer and pass for the wrong step.Baseline rows
The three web-console rows keep their measurements and have their censoring reason marked historical: they were censored by the
testjob's cap under a topology that no longer exists. The numbers are real observations and still valid lower bounds; only the mechanism retired.Verification
ruff check,ruff format --check,mypystrict, and all 291 tests that readci.ymlpass.The five new shell blocks were
bash -nchecked behind a positive control (a deliberately malformed block must fail, a trivial one must pass) because the repo's owntest_workflow_shell_syntaxguard cannot run on this box -- Git Bash receives a mangled Windows path and it fails 161-of-161 locally regardless of content. That guard runs properly here in CI.