Observation
On cabcfc8 (main, 2026-08-06/07), four consecutive local runs of pnpm test:
| run |
result |
| 1 |
1902 pass / 4 fail |
| 2 |
1906 pass / 0 fail |
| 3 |
1906 pass / 0 fail |
| 4 |
1906 pass / 0 fail |
Same commit, no changes between runs. 1902 + 4 = 1906, so exactly four tests that failed then passed.
I did not capture run 1's output, so the four are unidentified. That is the main cost of this — the observation exists but cannot be acted on.
What is known
- CI is unaffected so far. Every CI run on the branch and on main has been green, including the Postgres-backed
Backend-TS job, which exercises strictly more than the local SQLite floor.
- Run 1 was the first suite run after a
pnpm install + pnpm typecheck in the same shell invocation, so lingering background work from install (esbuild, better-sqlite3 rebuild) is a plausible source of contention.
- The suite contains many timing-sensitive tests: several execution-diff cases run 8–14 s,
WITHOUT preparation the official artifact reads the whole roster out-of-population takes ~6–9 s, and the MEASURE/EMPLOYEE run-pipeline tests take 1–3 s each. A loaded machine is the obvious suspect.
Why it is worth tracking rather than dismissing
A suite that fails ~1-in-4 locally will eventually fail in CI, and when it does it will look like a real regression on whatever PR is unlucky. Finding it from a red CI run on an unrelated change is the expensive way.
It is also the shape this repo keeps cataloguing: a control that appears to pass reliably and does not.
Next step
Capture full output on every suite run so the next occurrence names itself:
pnpm test 2>&1 | tee /tmp/suite-$(date +%s).txt
Then the four are identifiable and this becomes a real bug report instead of an anecdote. Nothing to fix until then — guessing at candidates would be inventing a cause.
Observation
On
cabcfc8(main, 2026-08-06/07), four consecutive local runs ofpnpm test:Same commit, no changes between runs.
1902 + 4 = 1906, so exactly four tests that failed then passed.I did not capture run 1's output, so the four are unidentified. That is the main cost of this — the observation exists but cannot be acted on.
What is known
Backend-TSjob, which exercises strictly more than the local SQLite floor.pnpm install+pnpm typecheckin the same shell invocation, so lingering background work from install (esbuild, better-sqlite3 rebuild) is a plausible source of contention.WITHOUT preparation the official artifact reads the whole roster out-of-populationtakes ~6–9 s, and the MEASURE/EMPLOYEE run-pipeline tests take 1–3 s each. A loaded machine is the obvious suspect.Why it is worth tracking rather than dismissing
A suite that fails ~1-in-4 locally will eventually fail in CI, and when it does it will look like a real regression on whatever PR is unlucky. Finding it from a red CI run on an unrelated change is the expensive way.
It is also the shape this repo keeps cataloguing: a control that appears to pass reliably and does not.
Next step
Capture full output on every suite run so the next occurrence names itself:
Then the four are identifiable and this becomes a real bug report instead of an anecdote. Nothing to fix until then — guessing at candidates would be inventing a cause.