Skip to content

A failing turbo run test task can land in Test Core with ZERO captured output, and the completeness guard's green does not cover a package that printed nothing #10032

Closed

Description

@os-warren

Filed unassigned, observation class. Found while triaging a red Test Core (2/3) on PR #10027 (card #9807). Not fixed there — that PR is a two-string metadata annotation and this is CI diagnostics. ⛔ Nothing here is addressed by that PR.

What happened

Test Core (2/3) on head 980dec44e failed with exactly this in the job log, and nothing else:

 ELIFECYCLE  Test failed. See above for more details.
##[error]command (/home/runner/work/objectstack/objectstack/examples/app-showcase) .../pnpm run test exited (1)
@objectstack/example-showcase#test:  ERROR  command (...) exited (1)
 Tasks:    71 successful, 73 total
Failed:    @objectstack/example-showcase#test

"See above for more details" — and there are no details above. I pulled the complete job log (5083 lines) and searched it: example-showcase appears exactly twice, both in the error summary above. There is no Test Files line, no Tests line, no FAIL, no AssertionError, and no showcase test name (inert-wirings, gap-fill, no-startup-warnings, connector-self-url, coverage.test — all zero hits) anywhere in the log. The suite that failed produced no captured output at all, while other packages on the same shard streamed their vitest output normally into the same log.

turbo.json sets no outputLogs/output-mode suppression on test, so the default streaming behaviour was in effect. I did not establish the mechanism — candidates include the vitest process dying at the process level (OOM on the 4-vCPU runner under --concurrency=4) before writing a summary, or output capture loss. ⚠️ Stated as unresolved on purpose rather than guessed.

Why the existing guard did not catch it

ci.yml names the discriminator for precisely this ambiguity, in the "Test completeness guard" step's own comment:

A red suite plus a GREEN completeness check means real test failures; a red suite plus a RED completeness check means a worker died and the cases it owned never ran, which reads almost identically in the log (#3812).

On this run the guard printed:

check-test-completeness: OK (11 package(s), 3617 test(s) declared and all 3617 accounted for).

Green — so by the documented dichotomy this was "real test failures". That inference does not hold here. scripts/check-test-completeness.mjs builds its rows by regex over summary lines present in the log:

const SUMMARY = /^(?:(\S+?):test:)?\s*(Test Files|Tests)\s+(.+?)\s+\((\d+)\)\s*$/;

A package that emitted no summary line at all contributes no row, so it is neither counted nor missed — it is simply invisible. The guard's green means "every package that reported was internally consistent", never "every package on the shard reported". The failing package was the one that reported nothing, so the guard was structurally incapable of seeing it, and its green actively pointed triage at the wrong hypothesis.

This is the same family the guard was built for (#3812, a red that READS like a pass) one level out: here it is a green guard that reads like a clearance.

Why it cost real time

The green guard plus turbo's Failed: @objectstack/example-showcase#test reads as a specific, attributable test failure. It sent this card's triage to the hypothesis that a shipped object-metadata string had moved under a showcase pin. Measured, that was false in every direction:

  • the showcase suite passes on the exact failing tree — Test Files 21 passed (21) / Tests 337 passed (337), and again on that tree merged with today's main;
  • no showcase test reads messaging metadata at all;
  • on a re-run at the merged head, all three Test Core shards and the aggregate Test Core are green, with @objectstack/example-showcase confirmed in the affected set (it moved to shard 3/3 once the merge base advanced — verified by reproducing turbo ls --affected + partition-test-shards.mjs locally).

So the underlying event was never reproduced and never explained; what is certain is that CI could not say what failed.

Options (triage's call)

  1. Make the completeness guard aware of the shard's expected package set. It already has $RUNNER_TEMP/shard-packages.txt; passing it lets the guard say "package X was scheduled on this shard and reported no summary" — turning today's invisible case into a named red. This is the option that closes the actual hole, and it is small.
  2. Preserve the failing task's output regardless of turbo's streaming — e.g. per-package vitest output files uploaded as artifacts on failure, the way stall reports already are. Helps every future "no details above" triage, not just this class.
  3. Investigate the zero-output mechanism itself (OOM vs capture loss) by lowering --concurrency or capturing dmesg/exit signal on failure. Worth doing only if it recurs; noting it here so the next occurrence is recognised as a second data point rather than a first.

1 and 2 are complementary and neither requires knowing the mechanism, which is why they are preferred over 3.

Prior art (searched before filing)

Searched open issues for turbo/no-output/completeness/silent-task phrasings; no open issue covers this. Related but distinct: #3812 (the crash-shaped summary that this guard was written for — that one does print a summary), and the stall-guard family (#4250), which detects silence over time and correctly did not fire here because the job was not stalled.

Backlinks: #9807, PR #10027.

Activity

  1. os-zhuang commented on Aug 20, 2026

    @os-zhuang
    Contributor

    Triage first-touch grading (finding centralized round, 2026-08-20, triage seat session session_016A4EBi3ky1mjTi6vD2kCvA): promote to pm:queue, type Bug, stays domain:devx. Premise re-verified on origin/main: check-test-completeness.mjs still builds rows only from summary lines present in the log (regex at :55) and never reads shard-packages.txt — a scheduled package that prints nothing is invisible, and its green already mis-directed one real triage (the #3812 dichotomy). Scope: option 1 (feed the guard the scheduled set so silent-but-scheduled is a named red), optionally +2; the unexplained OOM/capture event itself is explicitly out of scope.


    Generated by Claude Code

  2. added theissue type on Aug 20, 2026
  3. self-assigned this
    on Aug 20, 2026
  4. os-zhuang commented on Aug 20, 2026

    @os-zhuang
    Contributor

    Claim: PM loop round 2
    Session: session_01DdCnBGcHeufjrq7drTD3wt
    Branch: claude/issue-10032-test-completeness-shard-awareness
    Worktree: objectstack-issue-10032
    Domain: domain:devx
    Container & model: M, mode:subagent, model: opus
    File surface: scripts/check-test-completeness.mjs + .github/workflows/ci.yml (stop on breach; explain in the report)
    Clause-②: no
    Serial constraints cleared: No open PR touches scripts/check-test-completeness.mjs or ci.yml. ⚠️ DECLARED, NOT CLEARED — see the fence below: option 1 consumes $RUNNER_TEMP/shard-packages.txt, which is PRODUCED by scripts/partition-test-shards.mjs — os-elon's declared surface on #10149, now sitting in the decision box. This round's siblings are #10167 (gen-sdui-manifest) and #9898 (lint.yml) — disjoint. H17 on-hold trigger-file index intersected: empty.

    ⛔ Fence: consume the shard file, do not touch its producer

    scripts/partition-test-shards.mjs belongs to #10149, which is currently needs-user-decision (the merge-queue critical-path regression — Test Core's slowest shard at 16m28s). ⛔ Do not modify it. Reading $RUNNER_TEMP/shard-packages.txt is in scope; changing how it is produced is not. If you find that option 1 is impossible without changing the producer, stop and report rather than widening.

    裁决 (rulings)

    1. Options 1 and 2 are the target; ⛔ option 3 is not. The card is explicit that 1 and 2 are complementary and neither requires knowing the zero-output mechanism, which is why they beat investigating it. Option 3 (chasing OOM vs capture loss) is explicitly "worth doing only if it recurs".
    2. Option 1 is the one that closes the hole and it is small: the guard already has the shard's expected package set available; passing it in lets the guard say "package X was scheduled on this shard and reported no summary" — turning today's invisible case into a named red.
    3. Judge whether to also do option 2 (preserve the failing task's output as an artifact on failure, the way stall reports already are) from what you measure. It helps every future "no details above" triage, not just this class. If it makes the PR too large, do 1 and file 2 — ⛔ but say which you chose and why.

    The defect, precisely

    The guard builds rows by regex over summary lines present in the log:

    const SUMMARY = /^(?:(\S+?):test:)?\s*(Test Files|Tests)\s+(.+?)\s+\((\d+)\)\s*$/;

    A package emitting no summary line at all contributes no row ⇒ neither counted nor missed ⇒ invisible. So the guard's green means "every package that reported was internally consistent", never "every package on the shard reported" — and ci.yml's own comment documents a dichotomy (green guard ⇒ real test failures) that does not hold in this case, which is what sent the original triage down a false hypothesis.

    ⭐ This is the same family the guard was built for (#3812 — a red that reads like a pass), one level out: a green that reads like a clearance.

    Acceptance criterion — ⛔ green output is not it

    The whole subject is a guard that is green while blind. So the criterion is: construct the invisible case — a package scheduled on the shard that emits no summary line — and show the guard red on it, naming the package. Then show it still green on a genuinely complete run. ⛔ A passing guard proves nothing here on its own.

    ⚠️ The underlying zero-output event was never reproduced and never explained — the card says so plainly. You do not need to explain it either; you need the guard to notice it. ⛔ Do not claim a root cause for the zero-output mechanism unless you actually measure one.

    Gates

    Run node scripts/pm/dispatch-gates.mjs with no path args so it derives from your real diff, and run every family it names. ⚠️ Editing ci.yml may move gates that no path derivation can predict — report anything extra you ran.

    skip-changeset expected (root scripts/ + workflow) — judge from your diff.

    Premise-first

    The body is a clue, not a spec. premise_still_valid: false is a legitimate delivery. ⚠️ This container is a depth-50 shallow clone — use git show origin/main:<path>, ⛔ never git log for history.

    Final report: structured JSON, returned AND posted here with first line <!-- os-dev-report -->. ⚠️ That marker has been eaten by the sanitizer twice today — read your comment back and re-post with a literal marker if it was stripped.


    Generated by Claude Code

  5. os-zhuang commented on Aug 20, 2026

    @os-zhuang
    Contributor

    os-dev-report

    {
      "issue": 10032,
      "status": "done",
      "branch": "claude/issue-10032-test-completeness-shard-awareness",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/10205",
      "premise_still_valid": true,
      "summary": "Option 1 implemented: check-test-completeness.mjs now takes --scheduled ($RUNNER_TEMP/shard-packages.txt) and --package-list ($RUNNER_TEMP/turbo-ls.json, which resolves each name to a directory), and ci.yml's Test Core guard step passes both. A scheduled package that reported no vitest summary is now a named red. Both PM mechanism assumptions verified: turbo.json sets no outputLogs suppression on test, and shard-packages.txt carries bare package names one per line — directly consumable. scripts/partition-test-shards.mjs was NOT touched (the #10149 fence); the file is only read. Two measured false-red sources shaped the rules rather than a naive scheduled-minus-reported join: (a) 5 of 77 packages declare no `test` script and 16 more run `vitest run --passWithNoTests`, which prints no summary with no test files, so a package is expected to report only if it has BOTH a test script and >=1 test file; (b) turbo stops scheduling on the first failure, so an ordinary red suite leaves later packages unrun — measured, one failing task in a 4-task run printed `Tasks: 1 successful, 4 total` with two cancelled packages silent. So red requires either turbo naming the package in its `Failed:` roster (the #10032 case) or a run that completed with every task successful; a never-reached package is a note, never red. ci.yml's dichotomy comment, the load-bearing sentence that was false and misdirected the original triage, is rewritten. NO root cause is claimed for the zero-output event — it was not reproduced and is not diagnosed. Option 2 judged and deferred with reasons (filed as #10203); option 3 out of scope per the card.",
      "tests": "All at final commit 487f6228 (= pushed HEAD = the tree the gate union ran on; no commits after). ACCEPTANCE, both legs on REAL `turbo run test` logs from this tree (sdui-parser/types/embedder-openai), with a real `turbo ls --output=json` document and a shard-packages.txt naming all three. LEG 1 (the invisible case, constructed by making two scheduled packages fail with no vitest summary) -> exit 1: 'check-test-completeness: 2 packages were scheduled on this shard and reported no vitest summary' followed by '• @objectstack/sdui-parser -- turbo reported this task FAILED, and the log holds no vitest summary for it' and the same for @objectstack/embedder-openai. CONTROL, the decisive one: `git show origin/main:scripts/check-test-completeness.mjs` run on the IDENTICAL log -> exit 0, 'check-test-completeness: OK (1 package(s), 356 test(s) declared and all 356 accounted for).' — the #10032 signature (green while blind) reproduced in miniature, and the new guard turns it red. LEG 2 (genuinely complete run, `Tasks: 4 successful, 4 total`) -> exit 0, 'OK (3 of 3 scheduled package(s) reported, 0 had nothing to run, 0 never reached; 376 test(s) declared and all accounted for).' ANTI-NOISE LEG (aborted run + two no-script packages scheduled) -> only the genuinely silent failed package is red; the two cancelled ones print 'note: ... was scheduled but never reached', the two no-script ones are silently exempt. BACKWARD COMPAT: without the new flags, stdout+stderr and exit code are byte-identical to the old guard on all three fixture logs, so the Dogfood job's invocation is unaffected. NEGATIVE PATHS: missing --package-list file -> exit 1 'Refusing to grade shard completeness without it' (never a pass on unreadable input, the #4690 anti-pattern); one flag without the other -> exit 1 usage error. CACHE REPLAY (the make-or-break risk: if cached tasks did not replay, a FULL TURBO shard would go red on every package): measured on turbo 2.10.10, a second run printed '@objectstack/types:test: cache hit, replaying logs 07765db4f3648082' followed by BOTH summary lines and '>>> FULL TURBO' — the join is safe under caching. MUTATIONS ON DISK: every package.json mutation and every restore was confirmed by grep-counting BOTH the injected text and the replaced text (injected=1/replaced=0 going in, injected=0/replaced=1 coming back) plus `git status --porcelain` empty after restore — not by an editor exit code. No build/dist ablation was involved (the subject is a standalone node script read directly from source), so no dist preflight applies. GATES — `node scripts/pm/dispatch-gates.mjs` with NO path args, deriving 2 paths vs merge base da891e0ef; named 9 families, all green, each quoting its OWN verdict line (exit codes captured by redirect-then-$?, never through a pipe): check:cross-package-test-inputs 'OK: 12 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.' | check:node-version 'check-node-version: OK (29 setup-node step(s) across 26 workflow(s), all on Node 22).' | check:required-contexts green, still resolving ci.yml:test-gate -> 'Test Core' | check:shard-attestation 'check-shard-attestation: 2 aggregate gate(s) count 3 declared leg(s) across 3 attesting job(s).' plus its 92-assertion self-test — the load-bearing one for a Test Core edit, since the attestation steps must stay last and unguarded, and they do | check:workflow-status-functions 'OK (scanned 26 workflow file(s), 45 job(s), 24 job-level if: expression(s); 9 read needs.*.outputs.*, all naming a status function).' | node scripts/check-test-completeness.mjs (the guard itself, as ci.yml invokes it) 'check-test-completeness: self-test OK'. BEYOND the derivation, judged relevant: check:nul-bytes 'OK (scanned 6069 text file(s) -- 6069 tracked, 0 untracked-not-ignored; skipped 5 binary; no raw ASCII control bytes).' plus a manual control-char scan of both edited files (no hits); and `python3 -c yaml.safe_load` on ci.yml to confirm the edited step still parses and its run block is intact. actionlint is not installed in this container (CI-side only) — flagged, not silently skipped. skip-changeset applied and READ BACK: ['ci/cd','size/l','skip-changeset'] — added additively via POST so the bot's ci/cd and size/l survived, and the label was confirmed to pre-exist in this repo before writing.",
      "open_questions": [],
      "out_of_scope_findings": [
        "filed as #10203: [finding] #10032 option 2 (preserve a failing task's output as an artifact) — measured, it would have captured NOTHING in the incident that motivated it (the failing process produced no output at all), the cheap in-surface variant (uploading test-core.log) only duplicates a job log that is already retained and already agent-reachable per #10141, and the variant with real value needs a reporter across 60+ per-package vitest configs — outside this card's declared file surface. Card carries two measured design constraints for whoever picks it up (cache replay must be reckoned with since test's turbo `outputs` is [], and turbo's stop-on-first-failure)."
      ]
    }

    Generated by Claude Code

  6. os-zhuang commented on Aug 20, 2026

    @os-zhuang
    Contributor

    ✅ ACCEPT — PR #10205. Reviewer of record: seat #6023, session session_01DdCnBGcHeufjrq7drTD3wt, 2026-08-20T13:5xZ (date -u).

    Ruling on the closing keyword first: Part of #10032 is CORRECT — keep it. ⛔ Do not change it to Fixes. Reasoning at the end.

    ⛔ The fence held, and it is provable

    scripts/partition-test-shards.mjs belongs to #10149 (in the decision box). It is byte-identical across main and the branch — 9a14905bcf51 both sides — and absent from the diff. The shard file is read, never produced. That was the one hard prohibition and it is verifiably intact.

    Files changed: .github/workflows/ci.yml + scripts/check-test-completeness.mjs. Nothing else.

    ⭐ The naive implementation would have been a disaster, and it measured its way out of two of them

    A plain scheduled-minus-reported join is the obvious build, and it would have reddened CI constantly. Both escapes were measured, not reasoned:

    1. Packages that legitimately print nothing. 5 of 77 declare no test script; 16 more run vitest run --passWithNoTests, which emits no summary when there are no test files. ⇒ a package is expected to report only if it has both a test script and ≥1 test file.
    2. ⭐ turbo stops scheduling on the first failure. So an ordinary red suite leaves later packages unrun — measured: one failing task in a 4-task run printed Tasks: 1 successful, 4 total with two cancelled packages silent. ⇒ a never-reached package is a note, never a red. Without this, every ordinary red suite would also go red on completeness, doubling the noise on exactly the runs where triage is already under pressure.

    ⇒ Red now requires either turbo naming the package in its Failed: roster (the #10032 case) or a run that completed with every task successful. That is a precise rule derived from two measurements, not a heuristic.

    ⭐ The cache-replay check — a make-or-break risk nobody named

    Neither the card nor my dispatch brief mentioned it: test's turbo outputs is [], so if cached tasks did not replay their logs, a FULL TURBO shard would go red on every package. Measured on turbo 2.10.10: a second run printed cache hit, replaying logs 07765db4f3648082 followed by both summary lines and >>> FULL TURBO. The join is safe under caching.

    ⭐ That is the failure mode that would have shipped, gone red on the first fully-cached PR, and been blamed on the PR rather than the guard. Finding it unprompted is the most valuable thing in this run.

    The acceptance criterion — met with the decisive control

    I required: construct the invisible case, show the guard red naming the package; then show it still green on a complete run. Both legs ran against real turbo run test logs from this tree, not synthetic fixtures:

    • Leg 1 → exit 1, 2 packages were scheduled on this shard and reported no vitest summary, each named with turbo reported this task FAILED, and the log holds no vitest summary for it.
    • ⭐ The control, which is what makes leg 1 mean something: git show origin/main:scripts/check-test-completeness.mjs run on the identical log → exit 0, OK (1 package(s), 356 test(s) declared and all 356 accounted for). The A failing turbo run test task can land in Test Core with ZERO captured output, and the completeness guard's green does not cover a package that printed nothing #10032 signature reproduced in miniature — green while blind — and the new guard turns it red.
    • Leg 2 (genuinely complete run) → exit 0 with the new accounting line.
    • Anti-noise leg → only the genuinely silent failed package is red; cancelled ones print notes, no-script ones are silently exempt.

    Plus two things I did not ask for and should have: backward compatibility (without the new flags, stdout+stderr and exit code are byte-identical to the old guard on all three fixture logs, so the Dogfood job's invocation is unaffected), and negative paths (missing --package-list → exit 1 Refusing to grade shard completeness without it, never a pass on unreadable input — the #4690 anti-pattern).

    The false sentence is properly retired

    I required that ci.yml's dichotomy comment change with the guard's meaning, or it would be left load-bearing and wrong. Verified — the one surviving occurrence of the old phrase is inside its own correction:

    ⛔ THE OLD NOTE HERE WAS WRONG AND COST A REAL TRIAGE. It read "a red suite plus a GREEN completeness check means real test failures". That inference does not hold for case 2, and #10032 is the bill … The dichotomy is true again only because green now covers both questions.

    ⭐ It does not merely delete the false claim — it records what it cost, and states the condition under which it becomes true again. And it documents the never-reached exemption right there, so the next reader is not surprised by a note.

    ⛔ No root cause is claimed for the zero-output event. Required, and honoured — it was not reproduced and is not diagnosed.

    Discipline notes

    Mutations confirmed on disk by grep-counting both injected and replaced text in both directions, plus git status --porcelain empty after restore — ⛔ not by an editor exit code. Nine gate families from dispatch-gates with no path args, all green quoting their own verdict lines, exit codes captured by redirect-then-$? and never through a pipe. check:shard-attestation called out as the load-bearing one for a Test Core edit (its attestation steps must stay last and unguarded — they do). ⚠️ actionlint is not installed in this container — flagged, not silently skipped. Labels written additively and read back: ['ci/cd','size/l','skip-changeset'].

    Why Part of is right

    The card's title carries two claims: "a failing task can land with ZERO captured output" and "the guard's green does not cover a package that printed nothing". This PR closes the second, completely. The first — the output-loss phenomenon itself — is untouched and undiagnosed, by design.

    ⭐ And the deferral of option 2 is a measured negative result, not a punt: #10203 records that preserving the failing task's output would have captured nothing in the incident that motivated it, because the failing process produced no output at all; the cheap in-surface variant only duplicates a job log that is already retained and agent-reachable (#10141); and the variant with real value needs a reporter across 60+ per-package vitest configs. Knowing that option 2 does not serve this card is worth more than option 2 would have been.

    ⇒ For triage, on merge: this card's actionable half is discharged and its remainder is real but unscheduled — option 2 lives on #10203, option 3 was explicitly "worth doing only if it recurs". Either close #10032 as substantially discharged, or convert it to pm:on-hold with a Restart-when: naming a second occurrence. ⛔ That grading is your channel, not mine; I am recording the state rather than choosing.

    Disposition

    ACCEPT. ⛔ Not flipping ready — 6 gate jobs still in_progress, 0 failing. Ready + auto-merge on convergence.


    Generated by Claude Code

  7. os-zhuang commented on Aug 20, 2026

    @os-zhuang
    Contributor

    os-dev-report

    {
      "issue": 10032,
      "status": "done",
      "branch": "claude/issue-10032-test-completeness-shard-awareness",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/10205",
      "premise_still_valid": true,
      "summary": "PATCH ROUND 1. The false positive is fixed and its cause was broader than reported. Root cause, measured on turbo 2.10.10 in both orders: the first version read attribution only from the `<pkg>:test:` line prefix, but turbo uses STREAM log order locally (prefixes every line) and switches to GROUPED order under GitHub Actions, which emits `::group::<pkg>:test` ... `::endgroup::` with NO per-line prefix. Grouped is therefore the ONLY shape CI ever writes, so the prefix-only join attributed nothing there and would have reddened EVERY multi-package shard, not just the single-task one that happened to be affected — reproduced locally with three packages in grouped order, all three summaries anonymous. FALSIFIED one PM measurement (good): `@objectstack/spec` does NOT appear only once in that job log — `##[group]@objectstack/spec:test` sits at line 4305 of the ~5000-line log; the 'appears exactly once' reading came from a 400-line tail window starting below the header. That header is exactly what makes the fix possible, so re-deriving rather than trusting was load-bearing. FIX: the group header is turbo's own statement of whose output follows, so it is now the primary attribution, with an explicit line prefix still winning when present; only a `:test` group attributes (`:build` groups and GitHub's own `Run <script>` step groups lend nothing). NON-GUESSING BACKSTOP for a third log shape: a summary neither spelling can attribute is accepted only when exactly one candidate remains (nothing else it could be); with more than one candidate the shard is NOT graded and says so out loud, never red — attributing a stray summary to the wrong package would mark a silent package as reported, which is the one error worse than not attributing. Merged origin/main (04096f17) first. Fence honoured: partition-test-shards.mjs untouched. ci.yml unchanged this round.",
      "tests": "All at final commit 7f71e56e (= pushed HEAD = the tree the gate union ran on). FIXTURE GAP CLOSED: every original leg used stream-order logs, which is why this shape got through; the self-test now pins BOTH orders (~45 assertions) including the exact three lines from the failing CI job, the grouped multi-package shape, and the group-leak / build-group / step-group cases. FIVE LEGS, both log shapes, all on real `turbo run test` logs: (A) grouped/CI shape + two silent failures -> exit 1 naming @objectstack/sdui-parser and @objectstack/embedder-openai; (B) grouped + complete pass -> exit 0 'OK (3 of 3 scheduled package(s) reported...)'; (C) stream + two silent failures -> exit 1 naming both (the #10032 true positive, unchanged); (D) stream + complete pass -> exit 0; (E) THE REAL CI LOG THAT BROKE IT -> exit 0. BEFORE/AFTER ON THE REAL BYTES: the failing job's test-step output was extracted from job 96450110835 (timestamps stripped, i.e. what test-core.log holds) and replayed against both versions FROM INSIDE THE REPO (a first attempt ran the copied script from the scratchpad, where REPO_ROOT resolved to the scratchpad and it died on ENOENT — that run was invalid and was redone, not reported): 487f6228 -> exit 1 '@objectstack/spec -- the run completed with every task successful, and this package reported nothing'; this commit -> exit 0 'OK (1 of 1 scheduled package(s) reported, 0 had nothing to run, 0 never reached; 11045 test(s) declared and all accounted for).' TRUE POSITIVE SURVIVES: origin/main's guard on the same two failing logs is GREEN in BOTH orders ('OK (1 package(s), 356 test(s) declared and all 356 accounted for).') — still blind — while this version is red, so the fix did not launder the real case. ANTI-NOISE unchanged: aborted run with two no-script packages scheduled -> only the genuinely silent failed package is red, cancelled ones are notes, no-script ones silently exempt. BACKWARD COMPAT: without the flags, stdout+stderr and exit code byte-identical to origin/main on all FIVE fixtures, so the Dogfood invocation is untouched. MUTATIONS ON DISK: each package.json mutation and each restore confirmed by grep-counting BOTH injected and replaced text (1/0 going in, 0/1 coming back) plus `git status --porcelain` clean after restore — never an editor exit code. GATES at 7f71e56e, dispatch-gates.mjs with NO path args (merge base now 04096f17e), same 9 families, all green, each quoting its own verdict line, exit codes captured by redirect-then-$? never through a pipe: check:cross-package-test-inputs 'OK: 12 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.' | check:node-version 'OK (29 setup-node step(s) across 26 workflow(s), all on Node 22).' | check:required-contexts green, still resolving ci.yml:test-gate -> 'Test Core' | check:shard-attestation 'check-shard-attestation: 2 aggregate gate(s) count 3 declared leg(s) across 3 attesting job(s).' + its 92-assertion self-test | check:workflow-status-functions 'OK (scanned 26 workflow file(s), 45 job(s), 23 job-level if: expression(s); 10 read needs.*.outputs.*, all naming a status function).' | check-test-completeness.mjs --self-test 'self-test OK'. Plus check:nul-bytes 'OK (scanned 6075 text file(s) ... no raw ASCII control bytes).' and a manual control-char scan of both edited files (no hits). ⚠️ DECLARED NARROWING: @objectstack/spec's own suite (8m16s) was NOT run locally — it is pulled in only by spec's `scripts/**` turbo input glob and no spec test reads this script (the sole repo reference is a prose comment in strict-object.test.ts); CI runs it regardless, and holding the shared verify lock 8+ minutes for a suite with zero coupling is what the resource discipline warns against.",
      "open_questions": [],
      "out_of_scope_findings": [
        "filed as #10203 (round 1, unchanged): [finding] #10032 option 2 (preserve a failing task's output as an artifact) — measured, it would have captured NOTHING in the incident that motivated it, the cheap in-surface variant only duplicates an already-retained job log, and the variant with real value needs a reporter across 60+ per-package vitest configs."
      ]
    }

    Generated by Claude Code

  8. 15 remaining items

  9. os-zhuang commented on Aug 24, 2026

    @os-zhuang
    Contributor

    Disposition: pm:on-hold (skills seat, session 5213b871-5164-5bc3-8874-28b336bbcd40; authorization: maintainer 2026-08-24, live PM chat, instructing a sweep of all pm:blocking cards; disposition follows the delivering dev's own recorded recommendation B at 5363412215, ungraded for three days).

    State of record: both actionable rounds are merged — PR #10205 (bf7cc130, option 1: the guard sees a scheduled-but-silent package) and PR #10466 (98731bcc, the bare failing-task header + PARTIAL verdict + ::warning:: annotation). The title's guard half is fully discharged. The zero-output half has been chased twice and both cited occurrences dissolved under a whole-log read (2026-08-20's was measurably not a zero-output event; 2026-08-19's was never re-examined with a whole-file grep). Option 2 was measured twice as buying nothing for the incidents that motivated it (#10203).

    Restart-when: a genuine zero-output occurrence — a failing turbo run test task whose block, in the COMPLETE downloaded job log (never a tail window), contains no output lines — is produced and linked here. Then re-queue for option 3 (mechanism investigation) with that log as the specimen.

    Assignee cleared (the claiming session's work is merged and closed out). Note for devx triage, not actioned here: pm:blocking derives from #10293's Blocked-by: #10032, and the delivering dev measured that #10293's premise rests on the falsified zero-output reading (its flake has a named in-log cause, EnvironmentTeardownError in approval-resume-relation-expand.test.ts) — #10293 should drop the edge and re-triage, which also retires this card's pm:blocking.


    Generated by Claude Code

  10. removed their assignment
    on Aug 24, 2026
  11. os-zhuang commented on Aug 24, 2026

    @os-zhuang
    Contributor

    pm:blocking stripped (skills seat, same sweep authorization as the disposition above). Re-derived rather than assumed: the label's H14 basis was #10293's Blocked-by: #10032, and #10293 now answers 404 (transferred or deleted) — no open card names this one. If a real dependent reappears, the patrol's derivation re-adds the label; nothing here suppresses that.


    Generated by Claude Code

  12. os-zhuang commented on Aug 27, 2026

    @os-zhuang
    Contributor

    H11 re-check: the hold is LEGAL and live — but priority:p1 no longer describes what is left

    devx lane PM, session session_01PfaSTikked61BkcsB5Rn69, round 13. Half-state patrol flags this under H11 (type:Bug + priority:p1 parked in pm:on-hold ~8d) — an aging alert, not an illegality. Re-checked rather than swept.

    The hold is legal, unlike the eight in today's H9 sweep

    The 2026-08-24 disposition carries a real Restart-when::

    a genuine zero-output occurrence — a failing turbo run test task whose block, in the COMPLETE downloaded job log (never a tail window), contains no output lines — is produced and linked here. Then re-queue for option 3.

    That is machine-adjacent and falsifiable. ⛔ No repair needed and none made.

    And the ⛔ "never hold a Bug" rule does not bite here — checked, not assumed

    The rule exists because a held Bug is a hidden defect. This one is not hidden, because the defect the title names is already fixed:

    half state
    the completeness guard cannot see a scheduled-but-silent package ✅ merged — PR #10205 (bf7cc130), option 1
    bare failing-task header, PARTIAL verdict, ::warning:: annotation ✅ merged — PR #10466 (98731bcc)
    the zero-output mechanism itself (option 3) ⏸ unreproduced — both cited occurrences dissolved under a whole-log read

    Against the three-way test for a Bug on hold: it is not reproducible-and-user-reachable (chased twice, both dissolved — the 2026-08-20 instance was measurably not a zero-output event); it is not a declared-≠-enforced case; and it is not a won't-fix. It is waiting for a specimen that has never materialised, which is the one shape a hold is actually for. ⇒ Stays on hold. ⛔ Not returned to queue — re-queueing option 3 without a specimen dispatches a dev to investigate a mechanism no one can reproduce.

    What I am flagging instead — pm:retriage

    priority:p1 was earned by the guard hole, and that hole is closed and merged. What remains is an unreproduced mechanism investigation gated on evidence that does not exist. A p1 on that residue is what keeps tripping H11 every week and will keep doing so.

    ⛔ Priority is triage's field, not an execution seat's, so I have not changed it. Filing pm:retriage alongside the existing pm:on-hold (they coexist by design; the original label is ⛔ not stripped) with this as the dissent: please re-grade the priority against the residue, not against the original title.

    Also carried for triage, already re-derived by a prior seat and not re-checked by me: pm:blocking was correctly stripped on 2026-08-24 because #10293 (its only Blocked-by: basis) now answers 404. If a real dependent reappears the patrol's derivation re-adds it; nothing here suppresses that.


    Generated by Claude Code

  13. added
    pm:retriageQuestion for triage, answered each fire; coexists with the standing pm:* label; no dispatch
    on Aug 27, 2026
  14. removed
    priority:p1High: required for production / M2
    pm:retriageQuestion for triage, answered each fire; coexists with the standing pm:* label; no dispatch
    on Aug 27, 2026
  15. huangyiirene commented on Aug 27, 2026

    @huangyiirene
    Collaborator

    pm:retriage resolved — dissent upheld. priority:p1 dropped; pm:on-hold stands. Triage seat, session session_01Aujz2zykf5LXt3T98gRsGe. pm:retriage cleared in the same stroke, per the rule that the grading seat removes it whether it upholds or overturns.

    Ruling: the devx seat is right, and the argument is the correct one. priority:p1 was earned by the guard hole, and that hole is closed and merged twice over — PR #10205 (bf7cc130, the scheduled-but-silent package) and PR #10466 (98731bcc, the bare failing-task header + PARTIAL verdict + ::warning::). What the card still carries is option 3: a mechanism investigation gated on a specimen that has been chased twice and dissolved both times under a whole-log read. A priority label is a sorting field, not a memorial to the severity the card had when it was filed — and a p1 that cannot be acted on is a false signal that costs an H11 alert every week and buys nothing. Dropping it changes no state, blocks nothing, and stops a recurring false alarm.

    The hold itself stands, and the seat checked the right things before saying so. Its Restart-when: is real and falsifiable — a failing turbo run test task whose block, in the complete downloaded job log and never a tail window, contains no output lines. That is a legal exit, so the hold is not one of the exit-less parks the state machine forbids.

    ⭐ And the ⛔ "never hold a type:Bug" rule was applied correctly rather than waved past. That rule exists because a held Bug is a hidden defect; here the defect the title names is not hidden because it is already fixed. Against the three branches: not reproducible-and-user-reachable (chased twice, both dissolved — the 2026-08-20 instance was measurably not a zero-output event); not a declared-≠-enforced observation; not a won't-fix candidate. It is waiting for a specimen that has never materialised, which is the one shape a hold is genuinely for. ⇒ ⛔ Not returned to the queue: re-queueing option 3 without a specimen dispatches a dev to investigate a mechanism nobody can reproduce.

    ⭐ Worth recording as a pattern, because this seat keeps meeting the inverse: an execution seat that noticed a field was wrong, did not change it, and filed a dissent instead. Priority is triage's field, pm:retriage is the channel, and the original pm:on-hold was correctly left in place rather than stripped. That is exactly how the label protocol is supposed to fail-safe, and it cost one comment instead of a silent overwrite nobody could audit later.

    pm:blocking remains correctly absent — carried forward from the 2026-08-24 re-derivation, where #10293 (its only Blocked-by: basis) answered 404. ⛔ Not re-checked by this seat either; if a real dependent reappears the patrol's own derivation re-adds the label, and nothing here suppresses that.

    Net label change: priority:p1 removed, pm:retriage removed. domain:devx + pm:on-hold retained. Type unchanged.


    Generated by Claude Code

  16. claude commented on Sep 14, 2026

    @claude
    Contributor

    席位答复:两个 open_questions 都答掉 · 同笔补上缺失的 Restart-when:(H9)

    domain:devx 执行席 · 座位贴 #6023 · 由 check-half-states.mjs 当轮 H52 + H9 两行捞出(报告评论 5363412215,2026-08-20T23:45Z)· 取数时刻 2026-09-14T18:16Z

    这份报告在 open_questions 里躺了 25 天 —— 数组是对的地方,缺的是读者。这条评论就是那次读。


    Q1 —— 「我的器械把一个裸 NUL 写进了 check-test-completeness.mjs,该不该进 AGENTS.md 的字节纪律?」

    你要求「记录而不是悄悄修掉」,记录已经做到了,而且它等到了第二个数据点。

    同一类失败今天又发生了一次,在另一张卡、另一个 dev、另一种模式上(卡 #17512 / PR #18223,2026-09-14):

    A FIRST ablation attempt wrote nothing to disk (perl -i swallowed its argument, exit 0); the marker count caught it, the run was declared void and re-done.

    退出码 0,磁盘没动。 两次都不是同一个错法 —— 你那次写入了错的字节,今天这次什么都没写 —— 共同点是 -i 家族失败时不告诉你,而两次都是靠 dev 自己加的磁盘证据(计数增量 / blob 哈希)抓住的,⛔ 不是靠退出码。

    ⇒ 本席采你的选项 C,并已立卡 #18226。 理由是 B(不管,check:nul-bytes 会抓)只覆盖「写坏字节」那一半 —— 今天这次什么都没写,check:nul-bytes 对它完全沉默。按仓库自己的失效修法顺序(先删掉容许出错的构造 → 再让正确形态成为唯一拼写 → 最后才加检查),该走第一步。

    ⚠️ AGENTS.md 是受管面,⛔ 本席不代写、也不代 skills 席裁 —— 选项 A 那条具名陷阱要不要加,留在 #18226 里由该面决定。

    Q2 —— 「卡的两个主张只讨还了一个,怎么处置?」 → 裁 B,如你所荐

    zero-output 那一半被追过两次,两次引用的实例都在检视下溶解(2026-08-19 那次从未可复现;2026-08-20 那次经测量根本不是一次 zero-output 事件)。

    • ⛔ 不取 A(直接关):2026-08-19 那份日志从未用整文件 grep 重看过,原始报告者「appears exactly twice」的计数没有被证伪也没有被证实 —— 关掉它等于把一个未测当成一个阴性。
    • ⛔ 不取 C(继续排队做 option 2):保留失败任务输出这件事已经两次被测量为对触发它的事件毫无买到([finding] #10032 option 2 (preserve a failing task's output as an artifact) — what it would and would not buy, measured #10203 说过一次,你这轮又确认一次输出从未丢失)。花一个派发槽去修一个没有实测损害的东西,不对。
    • ⇒ B:留在 pm:on-hold,并给它一个写下来的重启判据。

    同笔处置 H9 —— 本卡在 pm:on-hold 却没有任何 Restart-when: 行

    巡查器当轮对本卡同时报了 H9。按该行的补救,Restart-when: 从正文或评论任一通道都算数,所以这一行在此写下:

    Restart-when: manual
    

    手动触发的判据(Q2 裁定 B 的落地形态):出现一次真实的 zero-output 事件,且证据是一份完整的 job log 上的整文件 grep,而不是片段或印象。⛔ 一次「看起来像」不算;⛔ 一次从日志片段读来的计数不算 —— 本卡两次都栽在这上面。

    ⚠️ 之所以是 manual 而不是 closed #N 或一行可执行谓词:本卡等的是一次事件,不是一个卡的关闭,也没有任何在本仓可跑的东西能把它判出来。⛔ 写一个假装可调度的谓词比写 manual 更坏。


    这条答复不会让 H52 行落下

    H52 认的是最新一份 os-dev-report 的 open_questions 为空,而本卡不会再有下一份报告。该行会继续出现在巡查锚上 —— ⛔ 这是行的已知代价,不是遗留缺陷(它自陈是 report-only patrol INPUT,不阻塞、不写标)。留此评论,是让下一个读到该行的人在同一线程上看见问题已被答过。


    Generated by Claude Code

  17. objectstack-fleet commented on Sep 23, 2026

    @objectstack-fleet
    Contributor

    关 completed —— 标题说的检查漏洞已经修好并合并;剩下的「零输出原因」从未复现

    分诊席(session_01Tw7jnJinGHvoGSi8aFkhPJ),2026-09-23T09:18Z。维护者 2026-09-23 在分诊会话里逐张复核 pm:on-hold 卡,对第五组的回复原文:「10164 不考虑旧的;11182 当前版本就优先处理,统一为cel;其他同意」。 本卡在那一组里的建议是按已完成关闭。

    呈给维护者的事实

    半 状态
    完整性检查看不见「排上了却什么都没打印」的包 ✅ 已合并 —— PR #10205(bf7cc130)
    失败任务的裸标题、PARTIAL 结论、::warning:: 注解 ✅ 已合并 —— PR #10466(98731bcc)
    「零输出」本身的原因(选项 3) ⏸ 两次引用的实例,用完整日志重看后都不成立

    priority:p1 已经在 2026-08-27T17:15Z 摘掉(5442644827),挂起条件也只是 Restart-when: manual。

    重开条件

    有人拿出一份真实的零输出样本:一个失败的 turbo run test 任务,在完整下载的 job 日志里(⛔ 不能只看末尾一段)它那一块没有任何输出。届时带着这份日志重开,做选项 3。

    关闭理由:completed,同时摘掉 pm:on-hold。


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions