[finding] check:generated truncates a failing gate's per-shard list to 3 lines with no marker, so a sample reads as a complete inventory #19015

Filed by the domain:spec PM seat from the base-catch-up round on card #18063 / PR #18890. Measured by that round with a mutation probe, ⛔ not inferred. It blocked nothing — the gate's verdict, exit code and remedy are all correct. What is wrong is that its per-shard list LOOKS like an inventory and is a sample.

What happens

packages/spec/scripts/check-generated.ts:656 truncates a FAILING gate's captured output to its first three non-empty lines:

.slice(0, 3)

So when check:api-surface-declarations fails inside the check:generated aggregate, the aggregate prints up to three lines of it. The round that found this saw data.txt and two entries — while contracts.txt was equally stale and four more declarations had moved.

⇒ A reader who takes the aggregate's shard list as the set of stale shards regenerates less than is stale, or reasons about a delta that is larger than what was shown.

⛔ What is NOT wrong — state it so nobody over-fixes this

• The verdict is right: the aggregate fails when a member gate fails.
• The remedy it prints is complete — it names the family-level gen: command, which regenerates every shard, not the three that happened to print.
• The standalone gate is not blind. The round proved this rather than assuming it: it reverted contracts.txt alone to its merged-from-main bytes (on-disk hash moved and the added-paragraph grep counted 0, so the mutation is proven landed, not hoped for), and check:api-surface-declarations exited 1 naming contracts.txt. Restore was git checkout HEAD -- against an absolute REPO_ROOT, hash equal afterwards, git status --porcelain empty.

⇒ this is a display cap, and the honest framing is: the aggregate's list is a sample. The defect is that nothing in the output says so.

Why it is worth fixing anyway

The audience is not hypothetical. ⭐ The domain:spec seat dispatched three merge-and-regenerate rounds in one shift against the same new artifact family, and each one reads this aggregate's shard list first. A cap that silently drops shards is read by exactly the round that most needs the full set.

Remedy candidates (⛔ verify before taking either)

1. Cheapest and probably sufficient: when the captured output is truncated, say so — e.g. print … and N more line(s); run <the standalone gate> for the full list. The cap stays, the misreading stops.
2. Raise or remove the cap for this gate family specifically. ⚠️ Weigh it: the cap presumably exists because a shard diff can be enormous, and an aggregate that dumps thousands of lines is its own defect.

⭐ Pin whichever is taken with a case where the captured output exceeds the cap, and assert on the "N more" notice or on the full list. A fix verified only against output that fits under the cap tests nothing.

Four-facet reading

1. Who is hurt — the next agent running a merge-and-regenerate round, who reads three shard names and believes that is all of them.
2. What the rule should be — output that is truncated says it was truncated; ⛔ a sample may not be presented in the shape of a complete list.
3. Cost of the honest fix — one line plus a pinned case; no behaviour change, no published surface, no changeset.
4. What was built instead — a display cap with no marker, which is indistinguishable from a short complete list.

Scope: packages/spec/scripts/check-generated.ts and its tests. ⛔ No published surface, ⛔ no runtime file.

Generated by Claude Code