Skip to content

fix: stop the Windows CI job timing out on integration tests - #34

Merged
SurefireStudios merged 1 commit into
mainfrom
fix/ci-windows-test-timeout
Sep 9, 2026
Merged

SurefireStudios merged 1 commit into
mainfrom
fix/ci-windows-test-timeout

Conversation

@SurefireStudios

@SurefireStudios SurefireStudios commented Sep 9, 2026

Copy link
Copy Markdown
Owner

The badge was stale, and the failure was flaky

Main is green: the run on 0f0c5be, which includes all four merged PRs, passed on both Windows and Ubuntu. The README badge was still showing the earlier failed run on 602640de.

That earlier failure was not caused by those merges. Each of the four branches passed CI on its own, and main had already failed the same way on f7242fbd on 8 September, before any of that work existed.

What is actually wrong

The Windows job fails intermittently while Ubuntu passes, on a different handful of tests each time:

run commit failures
34377620448 602640de 3, all in gate4b.test.ts
34264941497 f7242fbd 8, across gate4b.test.ts and dashboard.test.ts

Every one is Test timed out in 5000ms rather than a failed assertion. A different subset failing each run, on one operating system, with no assertion involved, is a scheduling problem.

5s is Vitest's default and was never chosen for this suite. The tests that hit it do real work rather than compute: the locked-protocol guards initialise scratch git repositories and run the CLIs as child processes, and the dashboard tests build the site and hash artifacts. Measured locally on Windows they peak at 2129ms for a single test, and GitHub's Windows runners are several times slower at process spawning and file I/O. That puts the slowest tests close enough to the ceiling to cross it on some runs and not others.

The change

testTimeout and hookTimeout set to 20s, with the reasoning recorded in the config so the next person does not have to re-derive it.

This is headroom, not permission to be slow. A genuinely hung process still fails the run, just later, and any test that actually approaches this ceiling is doing something worth looking at. Nothing about the tests themselves changes.

Full suite 552 passing, typecheck clean.

Worth knowing

This does not make the suite faster, and a slow test is still a real cost on every run. If the Windows job stays close to the edge after this, the better fix is to stop those guards spawning a fresh git repository per test and share one across the describe block, which would cut most of the 14.8s that file currently takes. That is a larger change than a flaky-CI fix should carry, so I have left it.

Note

Low Risk

Overview
This PR addresses intermittent Windows CI failures in integration tests by modifying vitest.config.ts (adding 17 lines) to configure test timeouts and/or parallelism settings appropriate for the Windows runner. The change ensures the Windows job can complete before GitHub Actions' default job timeout, since the prior failures were caused by tests exceeding the time limit rather than actual test regressions. The fix is justified because main was passing on the relevant commits and the Windows-specific timeout behavior had been flaky across multiple unrelated runs.

Written by Gitzilla for commit a875660. This will update automatically on new runs. Configure in the Gitzilla dashboard.

The Windows job has been failing intermittently on main while Ubuntu
passes, on a different handful of tests each run: three from the
locked-protocol guards on 602640d, eight across those and the dashboard
tests on f7242fb. Every one failed with "Test timed out in 5000ms"
rather than on an assertion, which is a scheduling problem, not a
regression.

5s is Vitest's default and was never chosen for this suite. The tests that
hit it do real work rather than compute: the locked-protocol guards
initialise scratch git repositories and run the CLIs as child processes,
and the dashboard tests build the site and hash artifacts. Measured on a
developer machine they peak around 2.1s each, and Windows runners are
several times slower at process spawning and file I/O, which puts the
slowest of them close enough to the ceiling to cross it some runs and not
others.

testTimeout and hookTimeout are now 20s. That is headroom rather than
permission to be slow: a genuinely hung process still fails the run, just
later, and any test approaching this ceiling is doing something worth
looking at.

Nothing about the tests themselves changes. Full suite 552 passing.
Copilot AI lite review requested due to automatic review settings September 9, 2026 17:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@SurefireStudios
SurefireStudios merged commit 6052083 into main Sep 9, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants