finding: turbo's Failed: roster names only the surfaced failure, not every failed task — the completeness guard's Rule A population is narrower than it reads #10467

Filed unassigned, observation class. Measured while working #10032 (PR #10466). ⛔ Nothing here is addressed by that PR — it only records the property next to the rule that depends on it.

What was measured

turbo 2.10.10, a 4-package sandbox workspace, GITHUB_ACTIONS=true CI=true turbo run test --concurrency=4, with two packages whose test script exits 1.

Both genuinely failed — both printed AssertionError and pnpm's ELIFECYCLE Test failed. But turbo's end-of-run roster named only one:

 Tasks:    2 successful, 4 total
Failed:    @sb/delta#test

The second failing package (@sb/beta) was folded into an ordinary ::group::@sb/beta:test block, with its failure visible only inside that group. Only the surfaced failure — the one turbo reports as the run's error — gets both the roster entry and the bare (ungrouped) header.

Observed invariant across every run in that session: bare header ⟺ named in Failed:. That equivalence is what PR #10466 relies on to attribute the failing task's summary.

Why it is worth recording

scripts/check-test-completeness.mjs Rule A is "turbo named the package in its Failed: line and the log holds no summary for it ⇒ red". Its population is therefore the surfaced failure only. A package that failed, was not surfaced, and was also silent falls through Rule A to the runCompleted === false branch and is printed as a "never reached" note.

That gap is narrow today — a non-surfaced failure still gets a group, so its output is in the log and Q1 grades it — but the rule reads as though Failed: were an exhaustive roster of failures, and it is not. The next person tightening Rule A should know before they widen it.

⚠ Do not "fix" this by treating every task without a summary as failed — that is the false-red machine #10032's dispatch measured its way out of (5 of 77 packages declare no test script; 16 more --passWithNoTests; turbo stops scheduling on first failure). This card is a note about the roster's meaning, not a request to change the rules.

Possible follow-ups (triage's call, none urgent)

1. Leave as documented — the property is now recorded in the script header alongside Rule A. Cheapest, and arguably sufficient.
2. Derive the failed set from the per-task ERROR command (...) exited (N) lines instead of the roster, which appear to be emitted per failing task rather than once per run. ⚠ Unverified — the sandbox run above had only one such line, so this needs measuring before anyone builds on it.

Backlinks: #10032, PR #10466.

Generated by Claude Code