Repository navigation
Commit afbf960
fix(pm): ci-failure resolves a red lane's Lane verdict to its red gates' own commands (#23246)
Fixes #23223
Clause-②: no
## What this changes
Since #23191, every gate step of a `Lint · *` / `Type Check · *` lane
runs under `continue-on-error: true`. A red gate's jobs-API `conclusion`
is therefore `success`, so the only failed step of a red lane is its
last one, `Lane verdict`. `scripts/pm/ci-failure.mjs` resolved that step
name through `stepBodies`. The name exists once per lane, so the
`substitute` block printed `node scripts/ci/lane-verdict.mjs` 12 times.
Run locally, that command exits 3 and reproduces nothing.
Now, when the failed step is `Lane verdict`, the tool reads the red gate
names from the verdict's own sentences (`gate step "NAME" (id ID)
failed`). It resolves each name through `stepBodies` to that gate's own
`run:` block. The `substitute` block prints those commands. The
verdict's own command is never offered.
Only one file changes: `scripts/pm/ci-failure.mjs` and its
`--self-test`. Nothing under `.github/workflows/**`,
`scripts/ci/lane-verdict.mjs` or `.claude/**` is touched.
- `classifyAnnotation` gains one kind, `lane-verdict`. It recognises the
five sentences that `lane-verdict.mjs`'s `renderVerdict` writes as error
annotations: `red`, `unread`, `did-not-run`, `fail-fast` and `refusal`.
For `red`, it keeps the gate name and id. Everything else still
classifies as before. `assertionStatus` and `verdictOf` read the new
kind as content, the same as the old `other`, so the exit table does not
move.
- `readLaneVerdict` takes the union of those sentences from two sources,
deduplicated: the check-run's annotations and the job log's error lines.
The log is already fetched for this family, so it costs no extra
request. GitHub's documentation says one step can create only a limited
number of error annotations. A lane with more red gates than that limit
would name only some of them in the annotations; the log names them all.
That limit is NOT measured here, and the union makes the result
independent of it.
- `resolveFailedStep`: every step except `Lane verdict` resolves by its
own name, unchanged. For `Lane verdict`, `repro` holds the red gates'
hits, each carrying `gate: { name, id }`, and `laneVerdict` holds the
full reading. Both appear in `--json`.
- `renderSubstitute` is the old inline `sayRepro`, now a pure function
so the self-test can pin the printed lines. When the verdict names no
runnable red gate, it says why, in one of three ways:
- **Names no red gate** (an unread gate, a gate that did not run, a
fail-fast step, or a refusal): it lists what the verdict did name and
says that no gate command reproduces this red.
- **NOT READ**: the annotations call failed and no log was read.
- **UNRESOLVED**, loudly: no sentence matches a known shape. The wording
may have changed, or the verdict died before printing. It never falls
back to printing the verdict command.
- `stepBodies` now filters a new `workflowSteps` scan. The output is
identical, and the existing pins still hold.
## Proof on the measured sample
Run 38373082677 (PR head `5301220aa`), check-run / job 115214618941,
`node scripts/pm/ci-failure.mjs --run 38373082677`. The tool has no
`--job` flag, so `--run` was used.
Before, on `main` at `197fbe2c04`:
```text
failed step #68 Lane verdict
...
substitute run the failing step locally — it prints the same assertion:
from .github/workflows/lint.yml, step "Lane verdict" (run:)
node scripts/ci/lane-verdict.mjs
(the same pair, 12 times in all)
```
After, on this branch:
```text
lane verdict #68 "Lane verdict" judges this lane's gates; it is not one, and its own
command reproduces nothing (run locally it exits 3). What it named:
RED "Docs anchors resolve to real headings" (id docs-anchors-resolve-to-real-headings)
1 red gate(s); each one's OWN command is the substitute below.
substitute run the failing step locally — it prints the same assertion:
from .github/workflows/lint.yml, red gate "Docs anchors resolve to real headings" (id docs-anchors-resolve-to-real-headings), named by step "Lane verdict" (run:)
pnpm check:doc-anchors
node scripts/check-system-context-census.mjs --self-test
node scripts/check-system-context-census.mjs
```
`diff before after` shows only three changes: the two `other` heads for
that sentence become `lane-verdict`, and the substitute block above
changes. The other three failing checks on that sha render
byte-identical. The walk still exits 2 (UNDETERMINED) both times,
because none of the four failing checks carries a file-anchored
assertion. The exit table is unchanged on purpose.
## Self-test, and ablation that proves it can fail
A new battery, `Lane verdict: the red gates, not the verdict (#23223)`,
has 22 cases. The `stepBodies / resolveStep` battery goes from 8 to 9
cases, and the battery floor from 14 to 15.
The battery imports `scripts/ci/lane-verdict.mjs` and drives its real
`judgeLane` and `renderVerdict`. The parser is therefore pinned to what
the producer actually prints, so a wording change there turns this
battery red. No rendered line is printed: they are workflow commands,
and this self-test runs on a runner.
The cases pin the card's three requirements:
- N red gates give N gate commands, and the verdict command is never
printed.
- An all-success lane with a red verdict (a roster mismatch: a gate with
`continue-on-error` and no `id:`) says it names no red gate. It does not
resolve to nothing, and it does not read as "resolves to no `run:`
block".
- On a pre-#23191 head, a failed gate step whose conclusion is `failure`
resolves exactly as before, line for line.
The cases also cover the measured specimen, all five sentence shapes,
the UNRESOLVED and NOT READ cases, the log/annotation union, a red gate
that is missing from the local tree, and a gate name containing quotes
and parentheses. A live pin requires that every step in the real
workflow tree that runs the verdict is named `Lane verdict` (12 of them
today). A lane that renames its verdict step therefore turns the
self-test red instead of silently bringing the defect back.
Ablations were run on committed `77480b13ce` through
`scripts/ablation-replace.mjs`. In each leg the anchor hit 1 to 0, the
blob changed, and the restore was proven blob == HEAD with an empty `git
diff HEAD`:
| leg | mutation | self-test | cases red |
|:--|:--|:--|:--|
| 1 | `resolveFailedStep` stops treating `Lane verdict` specially | exit
1 | 8 (N→N, substitute, roster mismatch, UNRESOLVED, NOT READ, missing
gate) |
| 2 | the `red` sentence pattern no longer matches | exit 1 | 8
(specimen, real-renderer parse, quoted name, N→N, union, missing gate) |
| 3 | every other step resolves to no command | exit 1 | 2 (the two
pre-A1 pins) |
My first leg-1 run crashed the harness on `r.gate.name` instead of
reporting the failure. Two accessors are now null-safe, so a regression
is reported rather than thrown (`77480b13ce`).
## Local gates, at `77480b13ce`
- `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack
--ran FILE` reports 30 derived, 30 run, 0 NOT-MEASURED, 0 UNRUN, with an
exit code recorded for every one (all 0). That count includes `pnpm
check:pm-dispatch-gates` (EXIT=0, 397 s), `node
scripts/pm/ci-failure.mjs --self-test`, `node
scripts/check-self-test-workflow-commands.mjs` (its end-to-end case runs
this self-test and finds no runner-parsed line), `pnpm check:nul-bytes`
and `pnpm check:entry-guard`.
- The rule that an edited tool's own test suites are owed: `node
scripts/pm/check-governed-merges.mjs --self-test` (EXIT=0, 500
assertions; it reads this file's unchanged `PROXY_REARM_GUARD` line).
- eslint, narrowed to this one file:
- `pnpm lint` is `eslint . --no-inline-config`, and `eslint
--print-config scripts/pm/ci-failure.mjs` resolves a config for it, so
the file is inside the lint population.
- `--no-inline-config --format json scripts/pm/ci-failure.mjs` reports 1
file, 0 errors, 0 warnings.
- The printed config has no `parserOptions.project` (`eslint.config.mjs`
never enables type-aware linting), and the two rules that apply are
per-file syntactic rules. So this diff cannot change the verdict for any
file it does not touch.
- No changeset: `scripts/pm/**` publishes nothing (`skip-changeset`).
## Acceptance notes
- A gate name that appears in more than one workflow step still resolves
to every hit. This is the existing documented `resolveStep` behaviour
("picking one would be a silent guess"). One example: `Sweep the built
closure for vanished declarations` is a step in `ci.yml` build-core and
in two `lint.yml` lanes, and the two lint steps share one `id:`. So the
annotation's id cannot narrow it either. Narrowing by job would need a
mapping from job display names to job ids, which is outside this card.
- The card's optional `references/platform-readings.md` half is not in
this PR: it is a governed `.claude/**` path, and the claim leaves it to
#23201 or a skills card. The platform reading it would record is quoted
in this file's header instead: under `continue-on-error`, a red gate's
API `conclusion` is `success` and only the steps-context `outcome` is
`failure`.
- The `Lint & Repo Gates` aggregator row (`Verify every lint lane
succeeded`) still offers its own aggregator script as the substitute. It
names the red lane in an annotation (lint lane `lint-repo-tooling`
concluded `failure`), and the lane's own row, fixed here, carries the
gate commands. Noted only.
---
_Generated by [Claude
Code](https://claude.ai/code/session_012zwsLMxQBwHUYJgPyrpEvx)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent d9d7edd commit afbf960
1 file changed
Lines changed: 519 additions & 21 deletions
0 commit comments