ci(docs): record the -n 8 worker-count trial, which is worse on every leg - #412
Merged
Merged
Conversation
… leg Comment-only. No functional change -- the three matrix values are untouched. Recorded so the next person does not spend a dispatch re-running it. The argument for raising the worker count is genuinely tempting: this suite is process-spawn heavy, so a worker blocked on a pwsh or git child is not holding a core, which reads as 4 workers under-subscribing 4 vCPU. Measured, it is wrong. leg -n 4 baseline (n=2) -n 8 (n=1) delta outcome ubuntu-latest 630 / 633s 731s +16% RED windows-2022 967 / 1004s 1067s +8% green windows-2025 1084 / 1092s 1366s +25% RED Slower everywhere and red on two of three. The failures are the mechanism rather than noise: they are exactly the tests that measure elapsed time or bind real ports, which is what contention degrades first. Trial branch claude/ci-worker-count-trial at 2bb7339, run 31968796353, cited so the raw data outlives the branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Comment-only change to
ci.yml. No functional change -- the threepytest_workersmatrix values are untouched.Why record a negative result
The argument for raising the xdist worker count is genuinely tempting, and someone will reach for it again: this suite is process-spawn heavy, so a worker blocked waiting on a
pwshorgitchild is not holding a core, which reads as 4 workers under-subscribing a 4-vCPU runner.Measured, it is wrong -- worse on every leg and red on two of three.
-n 4baseline (n=2)-n 8(n=1)-n 8outcomeThe
-n 4baselines are two runs agreeing within 4 percent: PR #411 and its post-merge push tomain.The failures are the mechanism, not noise
They cluster on exactly the tests that measure elapsed time or bind real ports, which is what contention degrades first:
test_outbound_batch.py's pacing floor and a super-linear-scaling ratio assertion.connscale_smoke,multishard_smoke, andsession_mail's claim-verdict test -- the three heaviest multi-process tests, the last being the slowest single test in the suite.Raw data is cited in the comment (trial branch
claude/ci-worker-count-trialat2bb733915, run31968796353) so it outlives the branch, which is being deleted.