Conversation
The fleet's verification, gating review and Second Read run as tickets on the box, so a PR waiting on one looked the same on GitHub as a PR nobody had picked up. dario#1403 sat all morning on 2026-09-24 with green CI and a stale "Changes requested" while its verification waited on a paused seat. fleet-status.yml posts fleet/verify, fleet/review and fleet/second-read on the PR head after every push, label, review and comment: pending while a lane waits, green once it has spoken at the head, red when it said no. It reads labels, comments and reviews on a GitHub-hosted runner and never runs the PR's code. scripts/fleet-status.mjs applies the dispatcher's rules (verify first on code, verdicts count only at the head, the Second Read gates on code), and test/fleet-status.mjs pins them.
askalf
added
the
no-changelog
Claude
github-actions
Bot
added
ci
Live test: ✅ PASSHead Stages |
… the code-path edges
Verification at d001efdBase What I ran at 5a45587 (the PR head)
Rules rebuilt against the dispatcherCompared the script line by line with Two deliberate simplifications, both towards "pending", neither a hole: the script has no Boundaries the shipped tests did not cover, now on the branch (22 checks, one commit, test file only)
No row is a control: each fails under at least one mutant and the file cannot load on base. Two probes I ran and did not commit because no mutant of the shipped regex distinguishes them: CRLF review bodies ( After (d001efd)Mutant run at d001efd (
Not run here, and why
Read for behaviour outside the stated changeNothing else changes: three new files, no edits to existing ones, no runtime dependency, no Rules: reads-as-generated=covered(no folded tests; each new check names its mutant) | no-control-cases-in-the-suite=covered(base cannot load the file; every check fails under a named mutant) | mutate-the-rejected-alternatives=covered(M1..M23, M10 equivalent) | ledger-row-needs-its-fixture=covered(each row above has its own check) | comment-cites-its-own-review=unreachable(no added comments cite a review; the header cites the incident, not a gate) | test-comment-density-matches-neighbours=unreachable(new file, header matches test/opus-alias-fallback.mjs) | run-every-ci-step-not-just-the-red-one=covered(preflight, check-changelog, node --test discovery; actionlint via the fork job) | idempotence-test-asserts-only-agreement=unreachable(no repeated-call assertions) | dispatch-arm-boundary-coverage=unreachable(single implementation) | generated-release-note-voice=unreachable(no changelog bullet, no-changelog label) |
|
Live check after the label and the verification comment landed at d001efd: the |
sprayberry-redline
left a comment
sprayberry-redline
left a comment
There was a problem hiding this comment.
Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the GPT gating lane (gating review).
Verdict: request changes — public patch narration in source comments fails the first-party generated-writing gate. rule:reads-as-generated
Blocking — generated-writing tell
.github/workflows/fleet-status.yml:1-8
# Shows where a PR stands in the fleet's review lanes as three commit statuses on its head:
# without these a PR waiting on a seat looked the same as one nobody had picked up (dario#1403,
# 2026-09-24). scripts/fleet-status.mjs holds the rules, the dispatcher's, and their tests.
This is patch/incident narration rather than an operational description of the workflow. It puts the review/dispatch history into a public source file and reads as generated prose; first-party strict mode applies the generated-writing gate to new code comments. Keep only comments needed to understand how the workflow operates.
# Posts fleet review-lane commit statuses for pull requests.
I reviewed the workflow's event and permission boundaries, the status classification logic and its head-SHA checks, plus the current-head verification evidence and green CI. The implementation otherwise keeps the token scoped to reads and commit-status writes, does not execute PR code, and the added tests cover the classification edges.
sprayberry-secondread
left a comment
sprayberry-secondread
left a comment
There was a problem hiding this comment.
Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the Claude second-opinion lane (independent second read; the gating review is posted separately).
Verdict: The status logic is sound and well pinned, but one claim in the PR body is wrong: the workflow does run the PR's own copy of scripts/fleet-status.mjs. Which copy runs depends on the event.
Read at live head 95c4ff1 (the ticket named d001efd; since then the PR has one commit, 451b181, that only trims the workflow header, plus three master merges that don't touch these files). I read the full diff (3 files, +464), run 36072367939's log, the pinned checkout-with-retry@115a640 action.yml, and gh pr checks. All required checks pass, and fleet-status.mjs runs inside npm test (ok 95 - fleet-status.mjs). I could not read runtime/review-lanes.ts or tools/review-dispatch.sh (platform is not visible to this identity), so I did not check the "rules are the dispatcher's" claim against the dispatcher itself.
1. Medium: the body says the workflow "never runs the PR's code", but it runs the PR's copy of the script
.github/workflows/fleet-status.yml:34-38
- uses: askalf/checkout-with-retry@115a6407547e9711edbc2e915838d495cad9583f # v1.1.0
with:
sparse-checkout: scripts/fleet-status.mjs
sparse-checkout-cone-mode: false
persist-credentials: falseNo ref: is set, so actions/checkout uses GITHUB_REF. That ref depends on the event:
pull_requestandpull_request_review:refs/pull/N/merge, which contains the PR's version of the script.issue_comment: the default branch, which contains master's version.
Evidence: run 36072367939 (pull_request, this PR) logged HEAD is now at 572c3d4 Merge 95c4ff1dca6bc6924b4d821a6ba9aed1b25a0c82 into 9137d2f.... It then ran this PR's new scripts/fleet-status.mjs, which is not on master yet, and posted the three fleet/* statuses now on 95c4ff1. This contradicts two lines in the body: "never runs the PR's code" and "After merge, this PR's own statuses are the first live check".
Failure scenario: a later PR changes a rule in scripts/fleet-status.mjs, say the 100-file threshold. On push, its statuses are computed with the PR's rules. When askalf then posts ## Verification at <sha> (an issue_comment), master's rules recompute them. The same head gets two different rule sets depending on which event fired last. I found no security problem: fork PRs are excluded both by the job if and by the script, and a same-repo author can already edit the workflow.
Suggested fix, if the intent is "master's rules, always": pin the ref so every event runs the same file.
- uses: askalf/checkout-with-retry@115a6407547e9711edbc2e915838d495cad9583f # v1.1.0
with:
ref: ${{ github.event.repository.default_branch }}
sparse-checkout: scripts/fleet-status.mjs
sparse-checkout-cone-mode: false
persist-credentials: falseIf running the PR's copy is intended, drop "never runs the PR's code" from the body and note that issue_comment runs master's copy.
2. Low, not blocking: the header says one rule for a lineless later review, and the code and a test do another
scripts/fleet-status.mjs:18
// - On code, the Second Read gates too: its latest review at the head ends withscripts/fleet-status.mjs:83-86
if (r.login !== SECOND_READ_LOGIN || r.commitId !== facts.head) continue;
let last = null;
for (const m of (r.body ?? '').matchAll(/^SECOND READ: (READY[ \t\r]*$|NOT READY\b.*)$/gm)) last = m[1];
if (last === null) continue;test/fleet-status.mjs:171: check('a later review without the line keeps the verdict', ...)
The comment says the latest review at the head must carry the line. The code skips lineless reviews, so an older line at the same head still counts. Here is a case where the two differ. sprayberry-code/huggingface_hub#3 had a READY at a0ca259 (00:18Z), then a lineless review at that same head (00:25Z), and a fresh READY was then posted at 01:08Z. That sequence fits a pipeline that treats a lineless latest review as "no verdict at head". In that case this script would show a green fleet/second-read while the dispatcher is still waiting, which is the "something kinder" the test header rules out. I can't read review-lanes.ts to say which rule is right. Either way, the comment and the code should agree. If the dispatcher keys on the latest review, the fix is to drop the if (last === null) continue; skip, reset out for every Second Read review at the head, and flip the assertion at test line 171.
Boundaries I rebuilt from the diff
All of these are pinned by a test:
isCodePath:.mdand images (case-insensitive),docs/*.txtagainst.txtelsewhere,.gitattributes,.githubconfig against.github/actions|scriptsand script extensions (including uppercase.MJS).needsVerifyfile count: 99 and 100 files, plus the bot-branch 100-file exemption.verifiedAtHead: no label, label without comment, older comment, latest-comment-wins in both orders, another login,findingsandblockedheadings, 7-char prefix.redlineVerdict: DISMISSED and COMMENTED skipped, order at the same head, deterministic approval on code against docs.secondReadAtHead: other login carrying the line, older head,READY, mostly, last line in a body wins,NOT READYwith no reason (no trailing colon), em-dash reason stripped.fit: exactly 140 and 141 characters.
Not pinned, and I'm not treating them as findings:
- Zero changed files goes to
needsVerifyfalse, which reads "Not required". pull_request_review: editedis not a trigger, so a status can go stale if a review body is edited.
I read each assertion and found none that holds whether or not the code under test is correct.
Other notes
- The body says "37 checks". At this head the file has 59:
d001efdadded 22. - What's good: the pure
laneStatuses(facts)/ CLI split keeps every rule unit-testable without network. Fork handling is covered twice.permissions: {}at the top with job-scopedstatuses: writeis the right shape. The #1403 regression (a stale CHANGES_REQUESTED or NOT READY must not show red at a new head) is pinned directly.
SECOND READ: NOT READY — the body says the workflow "never runs the PR's code", but with no ref: the pull_request/review runs execute the PR's own scripts/fleet-status.mjs (run 36072367939, merge 572c3d4) while issue_comment runs master's copy
sprayberry-redline
left a comment
sprayberry-redline
left a comment
There was a problem hiding this comment.
Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the GPT gating lane (gating review).
Verdict: changes requested — public patch narration remains in a code comment. rule:reads-as-generated
Blocking — generated-writing tell — scripts/fleet-status.mjs:4-13
// WHY THIS EXISTS. The review lanes (the Breaker's verification, Redline's gating review, the
// Second Read) run as tickets on the fleet box, so a PR waiting on one shows nothing on GitHub.
// On 2026-09-24 dario#1403 sat all morning with green CI and a red "Changes requested" while its
// verification waited on a budget-paused seat, and nothing on the PR said so.
//
// Read-only as far as the PR goes: this looks at labels, comments and reviews and writes statuses.
This is patch-history narration in a public source comment, rather than documentation of behavior needed to maintain the program. It makes the implementation read as generated and exposes internal fleet-operational history in the shipped repository. Remove the historical explanation (and keep only concise, behavior-oriented documentation where it is needed).
// Computes fleet review-lane commit statuses from pull-request metadata.
The current head’s required CI checks are green. I also checked the head-only status logic and the workflow’s default-branch checkout change.
sprayberry-redline
left a comment
sprayberry-redline
left a comment
There was a problem hiding this comment.
Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the GPT gating lane (gating review).
Verdict: changes requested: generated patch narration remains in a newly added public test comment. rule:reads-as-generated
Blocking — test/fleet-status.mjs:3-5
// Each status must say what the dispatcher would do next at this head, never something kinder.
// A green fleet/verify on an unverified head, or a green fleet/review from a verdict on an older
// commit, is the failure that matters: it tells a reader the PR is moving when it is stuck.
This narrates the patch's desired outcome and why the new assertions are correct rather than documenting what the test fixture does. It is a generated-writing tell in a newly added public test file, which first-party strict mode makes blocking. The test names and assertions already communicate the behavior; delete this rationale rather than replacing it with another history-oriented explanation.
The live head is 4082a1e6de2e5649c9cbc994998374a3cd5d2bb2. I reviewed the 462 added lines in the workflow, status script, and unit tests. Required CI checks are green at this head; I did not run the local suite.
What's good: the status computation is factored into pure helpers with focused coverage for stale verification and review state, and the workflow uses default-branch code with narrow permissions.
sprayberry-secondread
left a comment
sprayberry-secondread
left a comment
There was a problem hiding this comment.
Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the Claude second-opinion lane (independent second read; the gating review is posted separately).
Verdict: not ready at c97b113. The new required-CI verification path has no trigger for the moment CI finishes, so fleet/verify sits on "Waiting on required CI" after CI has gone green. The PR body also still describes the label-and-comment rule, which this head no longer applies on dario.
Note: the ticket named 9c5a714, but the live head is c97b113 (two commits later), so this review is of c97b113.
1. High: fleet/verify never re-evaluates when required CI completes
.github/workflows/fleet-status.yml:6-12
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled]
pull_request_review:
types: [submitted, dismissed]
issue_comment:
types: [created, edited]scripts/fleet-status.mjs (CLI block)
if (required.length) {
try {
const statuses = (await ghAll(`/repos/${repo}/commits/${p.head.sha}/statuses`, token)).reverse()Before c97b113, every input to fleet/verify (the label and the verification comment) was covered by a trigger. This head adds a new input, the head's required checks, and that input has no trigger. dario's master ruleset requires 9 checks (build (18/20/22), validate-package-json, analyze, actionlint, live-test, test, docker-cap-drop-smoke), so the required-CI branch is the one that runs on every dario PR.
Failure scenario, on any push to a code PR:
synchronizestarts this job. It takes about 8s (statusin this PR's checks). At that pointtest(1m49s) andlive-test(2m33s) are stillin_progress, sorequiredCiStatereturns'pending'and the script postsfleet/verify = pending "Waiting on required CI at <h>".fleet/reviewandfleet/second-readpost "reads once it is verified".- CI finishes green a couple of minutes later. No
check_suite,check_run,statusorworkflow_runevent is subscribed, so the job doesn't run again. - All three lanes keep saying "waiting on CI" while CI is green. This lasts until some unrelated label, review or comment arrives. If a required check fails instead,
fleet/verifystays pending rather than going red.
This is the #1403 pattern the PR exists to remove: GitHub shows a stale state that doesn't match what the PR is actually waiting on. The unit tests can't catch it because they feed requiredCi in directly.
Suggested fix (workflow_run with the default-branch script, which already matches this workflow's model; the PR number comes from the run's head sha):
on:
# ...existing triggers...
workflow_run:
workflows: [CI, CodeQL, 'Live test (self-hosted)', actionlint] # the workflows that produce the required checks
types: [completed]Resolve the PR from github.event.workflow_run.pull_requests[0].number (or look it up by head sha, since pull_requests is empty for fork runs, which are skipped anyway), and extend the job if: to cover it. A check_suite: completed trigger would also work.
2. Medium: the PR body describes a verification rule this head no longer applies on dario
PR body:
verification is the
verifiedlabel plus askalf's latest## Verification at <sha>comment naming the head
and test/fleet-status.mjs asserts the opposite for any branch with required checks:
check('CI passed: fleet/verify green with no label or comment', passed[CONTEXTS.verify].state === 'success'scripts/fleet-status.mjs in laneStatuses:
const verified = code && (ci === 'passed' || (ci === 'none' && verifiedAtHead(facts)));dario is the only repo this ships to, and its master requires checks, so on dario the label-and-comment path is unreachable (ci is never 'none' unless the rules call fails). This PR itself shows the gap: it has no verified label and no ## Verification at comment newer than d001efd, yet at c97b113 the code would post fleet/verify = success "Required CI passed".
The body's "How to test" also still says node test/fleet-status.mjs: 37 checks, but the file now has 73. Before merge, the body should state the rule the code implements. The header comment says the rules "match the fleet dispatcher's", and the body links the statuses to tools/review-dispatch.sh / runtime/review-lanes.ts, so the body should also confirm the dispatcher now treats required CI as verification. I couldn't read the platform repo from this identity (404), so I couldn't check that. If the dispatcher still gates on label and comment, then on every dario PR fleet/verify goes green while Redline is still held for the Breaker, which is the status this PR is meant to prevent.
Suggested fix: update the body's rule paragraph, the lane table and the check count to match c97b113, and cite the dispatcher change that pairs with it.
What I checked and found sound
requiredCiState: statuses reversed so the newest wins; check-runs sorted by id so a rerun's result wins.EXPECTED/QUEUED/in_progresscount as pending,SKIPPED/NEUTRALas passed, and anything else as failed. Tests cover each of these, including a failure alongside a pending check and non-required checks (fleet/verifyisn't in the ruleset, so it can't hold itself).- Boundary rows: empty
requiredgives'none'(pinned). A missing check name gives pending (pinned). An unreadable rules call falls back to the label rule, and unreadable checks count as pending; neither turnsfleet/verifygreen by itself. Descriptions of exactly 140 and 141 characters are pinned. 99 vs 100 files is pinned, and so is the bot exemption at 100. secondReadAtHead: only reviews at the head count, the last verdict line wins, a later review with no verdict line keeps the earlier verdict,READY, mostlydoesn't count as READY, and an em-dash reason is stripped cleanly byNOT READY\W*.- The security shape is unchanged: default-branch sparse checkout of one file,
persist-credentials: false, job-scoped permissions, fork PRs skipped both in the jobif:and in the script. The PR's code is never executed. - CI is green at
c97b113(all 15 required and optional checks passed).
Not run: the test file itself (CI is the test signal). Not read: the platform dispatcher (404 for this identity).
SECOND READ: NOT READY — fleet/verify now depends on required CI, but no workflow trigger fires when CI completes, so it stays "Waiting on required CI" after CI goes green; the body also still states the label-and-comment rule
… finishes fleet/verify reads required CI, but nothing ran the workflow when CI completed, so it stayed 'Waiting on required CI' after CI went green. A workflow_run trigger on CI, CodeQL and the live test re-posts it; same-repo pull_request runs only, and it still runs the default branch's script. check_suite would not fire: GitHub does not send it for suites Actions created.
sprayberry-secondread
left a comment
sprayberry-secondread
left a comment
There was a problem hiding this comment.
Automated review from the Sprayberry Labs fleet code reviewer.
Reviewed by the Claude second-opinion lane (independent second read; the gating review is posted separately).
Verdict: one reachable gap in the new workflow_run trigger. The workflows: list leaves out one of the four workflows that produce the base branch's required checks. Everything else I checked holds.
Read at live head 894b505 (ticket named c97b113; since then 23992f8 added the workflow_run re-post and 894b505 merged master). Full diff (565 lines), the master ruleset's required checks, the on: blocks and job ids of every workflow at 894b505, and gh pr checks (all green).
Findings
1. Medium: actionlint is a required check, but its workflow is not in the workflow_run list
.github/workflows/fleet-status.yml:15-17
workflow_run:
workflows: [CI, CodeQL, 'Live test (self-hosted)']
types: [completed]The master ruleset requires build (18|20|22), validate-package-json, test, docker-cap-drop-smoke (workflow CI), analyze (CodeQL), live-test (Live test (self-hosted)) and actionlint. The last one comes from .github/workflows/actionlint.yml, whose name: is actionlint, and that name is missing from the list. 23992f8 says it re-posts "when a required-check workflow finishes", but it does that for three of the four.
Failure scenario: the actionlint job fails on something transient (its curl of the actionlint release tarball, for example) while CI is still running. When CI completes, workflow_run fires, requiredCiState sees actionlint: FAILURE and returns 'failed', so fleet/verify is posted red with "A required check failed at ". Someone re-runs actionlint and it passes. No listed workflow completes after that and nothing is pushed, labelled, reviewed or commented, so fleet/verify stays red while every required check is green. That is the stale-status problem this commit was meant to fix, on the one required check it doesn't cover. The same thing happens on a first run whenever actionlint's runner picks the job up after the other three workflows have finished. requiredCiState treats the unreported check as 'pending', so the status stays at "Waiting on required CI".
Suggested fix:
workflow_run:
workflows: [CI, CodeQL, 'Live test (self-hosted)', actionlint]
types: [completed]The header comment at :3-4 already describes this generally ("a workflow that produces required checks"), so it doesn't need to change.
2. Low (PR body): stale test count and an empty red column
The body says node test/fleet-status.mjs: "37 checks". test/fleet-status.mjs at this head has 73 check( calls. The body's table also leaves the red column blank for fleet/verify, but scripts/fleet-status.mjs:134 posts failure with "A required check failed at ${h}". Neither affects behaviour. They're claims that no longer match the diff, so worth fixing in the description.
What I checked and found sound
- Boundaries in
requiredCiState: emptyrequiredgives'none'(pinned). A required check that hasn't reported, or that is queued or in progress, givespending(pinned). Acompletedrun with a null conclusion maps to'', which gives pending.ERRORon a commit status, andcancelled/timed_out/action_requiredon a check run, givefailed. A re-run that passes overrides an earlier failure because check runs are sorted by id and the last one per name counts (pinned).skipped/neutralcount as passed, which matches how GitHub treats skipped required jobs (pinned). laneStatuseswithrequiredCi: covered by tests. Withpassed, verify is green without the label or comment. Withpending, an old label and comment don't turn it green.failedgives red. Withnone, it falls back to the label and comment. Unreadable rules count asnoneand unreadable checks aspending, so neither error path can turn verify green.secondReadAtHead:.and$in JS multiline mode treat\ras a line terminator, so the reason from a CRLF body comes out without a trailing\r. "READY, mostly" is not READY, and a later review without the line keeps the earlier verdict. All pinned.- 140-character cap: pinned at exactly 140 and at 141.
- Tests can fail: each assertion I traced compares an exact state or description string that a wrong branch would change. I didn't find any that hold whichever way the code goes.
- Security: every event runs the default branch's script (
ref: ${{ github.event.repository.default_branch }}, sparse checkout of one file,persist-credentials: false). Theworkflow_runleg requiresevent == 'pull_request'and a same-repohead_repository. The token hasstatuses: writeand read scopes only.PRcomes from the event number, and the script checks it with/^\d+$/. I found no path that runs PR code. - Concurrency is keyed on the PR number for all four event shapes. When CI and CodeQL finish close together,
cancel-in-progresskeeps the later run, and that run reads fresh state. - CI is green at
894b505: all nine required checks pass.
SECOND READ: NOT READY — the workflow_run list omits actionlint, a required check, so fleet/verify stays red or pending after an actionlint re-run goes green (fleet-status.yml:16)
actionlint is one of master's required checks, so its workflow belongs in the workflow_run list with CI, CodeQL and the live test; without it fleet/verify stayed red or pending after an actionlint re-run went green.
4 participants
What does this PR do?
The fleet's review lanes (the Breaker's verification, Redline's gating review, the Second Read) run as tickets on the box. A PR waiting on one looked the same on GitHub as a PR nobody had picked up. On 2026-09-24 dario#1403 sat all morning with green CI and a stale "Changes requested" while its verification waited on a budget-paused seat.
fleet-status.ymlposts three commit statuses on the PR head, next to build and test:fleet/verifyfleet/reviewfleet/second-readThe rules are the dispatcher's (
tools/review-dispatch.sh,runtime/review-lanes.tsin platform): on our own repos verification is the head's required CI (every check the base branch's rules require has passed); a verdict counts only at the head; on code, Redline's deterministic low-risk approval is not a verdict and the Second Read gates too.It re-runs on every push, label, review and comment, and when CI, CodeQL, the live test or actionlint finishes (
workflow_run), since each can move a lane. It runs on a GitHub-hosted runner, sparse-checks-out onlyscripts/fleet-status.mjs, installs nothing, never runs the PR's code, and its token can only read the PR and write statuses. Fork PRs are skipped; the fleet does not review them.Not in this PR:
fleet/verifyorfleet/reviewrequired is a one-line ruleset change if you want it.How to test
node test/fleet-status.mjs: 73 checks, including the feat(models): opus and opus1m fall back to Opus 5.5; opus5 pins Opus 5 (6.12.0) #1403 morning (an older CHANGES_REQUESTED and NOT READY must not show red at a new head), a stale verification comment, a verifier comment from another login, the deterministic approval on code vs docs, bot branches, and the 140-character description cap.node --test test/all.test.mjs: 242 of 243 pass. The one failure isoauth-detector.mjs, which fails the same way on master in this environment (no detectable CC binary).actionlintv1.7.1 (the CI version): clean.node scripts/preflight.mjs: clean.Checklist
npm run buildpassesnpm testpasses (offline regression test, no credentials required)src/? No:.github/,scripts/andtest/only, so this takes theno-changeloglabelproxy.ts,cc-template.tsor streaming? No