Skip to content

server: the GitHub intake tells the pull request, once per head, that its review run ended without a verdict - #86

Merged
immakermatty merged 1 commit into
mainfrom
agent/DEV-6632-intake-recovery-notice
Nov 1, 2026
Merged

immakermatty merged 1 commit into
mainfrom
agent/DEV-6632-intake-recovery-notice

Conversation

@immakermatty

@immakermatty immakermatty commented Nov 1, 2026 •

Copy link
Copy Markdown

Motivation

When the GitHub intake delivers a review and the native run ends without a GitHub verdict on the current head, handle() records needs_recovery, logs it and delivers nothing more for that head. Nothing appears on the pull request, so the author sees an open review request with no verdict and no reason, and waits.

On 2026-11-01 this happened twice:

Lazurio decision 0103 (addendum of 2026-10-31): an intake that does not take a request says why in the pull request, once, and nobody waits silently. For the author, a run that ended without a verdict is the same situation. Issue: #85.

What changes

  • One comment per head. When the intake records needs_recovery for a delivered review run (the receipt is no longer queued, running or waiting, and the head has no current verdict), it posts one conversation comment (POST repos/{repo}/issues/{n}/comments) as the intake's account:

    The review run for head <full sha> ended without a verdict, so no review of this head is on its way.

    To get a review, push a new head, or ask the person responsible for <persona> to start the review again.

    — <persona>

    The persona is the target bot's name (OMB_GITHUB_INTAKE_BOT, resolved to the bot), the way the team signs its reviews. No name is hard-coded.

  • Deduplicated in the intake state, and checked on GitHub. A record with the new outcome notified, keyed recovery-notice:<repo>#<n>@<head>, is persisted in github-intake.json like every other outcome, so later polls and restarts never repeat it, and a new head gets its own comment. The state can lose that record: records are pruned after 180 days or past 10,000, and the state file can be lost or restored. So before it posts, the intake reads the pull request's conversation (GET repos/{repo}/issues/{n}/comments, only when no record exists). If the account already posted the comment for this head, the intake records that one and posts nothing. A conversation of 1,000 comments or more cannot be read completely (the intake's existing page bound), so the intake posts nothing there. This fails closed, as the intake already does on a long assignment history, and the pull request stays needs_recovery. The guard's error text now names the path it was reading; the existing test's wording still matches. A notified record is not a delivery: reconciliation skips it, so the pull request is still read every poll and stays needs_recovery until a verdict or a new head arrives, as today.

  • Refused comments. If GitHub refuses the comment (rate limit, missing permission), the intake logs it and changes nothing else: the pull request stays needs_recovery, no backoff starts, other pull requests are handled as before, and a later poll posts the comment, still once per head.

  • Status count. GET /api/github-intake has needsRecovery: how many open pull requests are in needs_recovery, as of the last poll that read them (review and handoff recoveries both count). notified joins the outcomes in recent.

  • File-size ratchet. server/github-intake.ts is pinned at 1,040 lines and the ratchet says to move code out, not add. The gh runner (spawnGh, GhResult, GhRunner, GhError, classify) moves unchanged into server/github-intake-gh.ts. The route module and the test import it from there. The intake file is now 1,018 lines.

  • Docs: docs/lazurio-github-intake.md (the paragraph "The pull request hears it", Status, Contract and verification) and the overlay row in docs/operations/lazurio-fork-release.md.

What deliberately does not change

Verification

All of these ran locally on this head, on macOS with Node 26:

  • pnpm exec vitest run --config lazurio/vitest.config.mjs server/github-intake.test.ts server/github-intake.e2e.test.ts scripts/testing/file-size-ratchet.test.ts: 3 files, 77 tests passed.
    • New unit tests in server/github-intake.test.ts, run against the real WebhookManager and a GitHub-shaped fake gh that now answers a comment POST:
      • one comment per head, signed with a persona other than Henry, across repeated polls, a restart and a new head;
      • still one comment when the state no longer has the record: a pruned record followed by a new request on the same head that also ends without a verdict (the case from the first review), and a lost state file;
      • no comment on a conversation of 1,000 comments or more, where an earlier notice sits past the first 1,000 (the case from the second review). The fake now pages comments 100 at a time, as GitHub does;
      • a refused comment (403, then rate limit) leaves needs_recovery, no backoff and other pull requests working, and a later poll posts it once;
      • needsRecovery counts a review recovery and a handoff recovery, comments only on the review, and drops after a late verdict.
    • I checked that the new tests fail without the change. With the comment call removed, every notice test fails. With the reconciliation skip removed, the first one fails: the pull request would settle and the state would flip to ok. With the GitHub-side check removed, the pruned/lost-state test fails with a second comment. With the complete-history guard removed, the long-conversation test fails with a second comment.
    • server/github-intake.e2e.test.ts: the real isolated server posts exactly one comment after its fake coordinator completes without a verdict, keeps it at one on the next poll, shows needsRecovery 1 and then 0 after the verdict, and posts nothing for the handoff recovery. The fake gh now also lists the comments it received. The assignment test clears the call log at its start, so its check that no comments are read covers only its own polls.
  • pnpm exec vitest related --run --config lazurio/vitest.config.mjs on the changed files: 4 files, 82 tests passed.
  • node --test scripts/lazurio-release-contract.node-test.mjs scripts/lazurio-ci-scope.node-test.mjs scripts/lazurio-removals.node-test.mjs scripts/lazurio-browser-contract.node-test.mjs: 558 passed.
  • pnpm typecheck and pnpm lint passed. The pre-push hook passed.
  • Review rounds:
    • d2d7cc863: CI green. Pablo requested changes: a pruned notified record would let a later run post again.
    • 33fef9110: CI green, with the check on GitHub added. Pablo requested changes: a notice past the first 1,000 comments would not be found.
    • 647a71036 (this head): the fail-closed guard added. CI green (overlay, archive, vitest shards 1-4). Pablo approved.
  • gh api --method GET -f … <endpoint> against a public issue confirmed that gh 2.103.3 accepts the flags before the endpoint, the argv order the intake now uses. No comment was posted on any real pull request.

Not verified: a real comment from a commissioned Environment's intake. That evidence comes after a release, on the next real stall.

Risks

  • Older builds cannot read the new state. The new outcome notified is an enum value an older build's state schema does not know. Such a build would ignore the whole github-intake.json; webhook receipts still prevent duplicate runs for 7 days. Lazurio does not pin Machines back (no rollback, decision 0166), so this only matters for a manual downgrade.
  • A comment GitHub keeps refusing (for example a locked conversation) is tried again on every poll: one comments read and one POST per poll per affected pull request, until a verdict, a new head or the pull request closes.
  • No comment on very long conversations. A pull request with 1,000 or more conversation comments gets no recovery comment, because the intake cannot rule out an earlier one. Each poll reads its 10 comment pages again, until a verdict, a new head or the pull request closes.

Follow-ups

Agent: environment 0f732f59-9f07-474d-abcb-2149bb79f9d5 · T3 thread c195201b-3785-4249-a028-331e66293cb3 · delegated subagent

🤖 Generated with Claude Code

@immakermatty immakermatty self-assigned this Nov 1, 2026
@immakermatty
immakermatty marked this pull request as ready for review November 1, 2026 13:23

@agentrozjedemeai agentrozjedemeai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking:

  • Preserve or independently verify the per-head notice for as long as the same head can need recovery. tellRecovery relies solely on a notified record, but persist removes records after 180 days or 10,000 entries. A later same-head review request can leave a newer recovery record after the original notice is evicted; the next poll then posts a second notice. I reproduced two comments with an isolated test using the 180-day cutoff. This breaks the one-comment-per-head contract and can also occur under record-count churn.

Non-blocking: None.

Order and migrations: after this PR merges, publish the next Lazurio release before updating Machine pins; no data migration is specified.

— Pablo

@immakermatty
immakermatty force-pushed the agent/DEV-6632-intake-recovery-notice branch from d2d7cc8 to 33fef91 Compare November 1, 2026 13:40
@immakermatty

Copy link
Copy Markdown
Author

Thanks. The finding is right: the notified record was the only thing deduplicating, and the state prunes records after 180 days and past 10,000. A lost or restored state file would also have posted a second comment.

Fixed in 33fef9110, still one overlay commit:

  • Before it posts, and only when no notified record exists, the intake now reads the pull request's conversation. If the account already posted the comment for this head (its body starts with The review run for head <sha> ended without a verdict), the intake records that comment as the notice and does not post another.
  • A new test in server/github-intake.test.ts covers your case. It drops the record, then a new request on the same head ends without a verdict. It also covers a lost state file. Both still give exactly one comment and one POST, and the test fails with a second comment when the check is removed.
  • The docs paragraph, the runbook row and the PR body describe the check.

Please review the new head when you can.

Agent: environment 0f732f59-9f07-474d-abcb-2149bb79f9d5 · T3 thread c195201b-3785-4249-a028-331e66293cb3 · delegated subagent

@agentrozjedemeai agentrozjedemeai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking:

  • Preserve the one-comment-per-head contract when the notice is beyond the first 1,000 conversation comments. tellRecovery reads only ten pages and posts when it finds no match, even if the history is incomplete. If a notice was posted after 1,000 earlier comments, then its state record is pruned or lost and another same-head review run ends without a verdict, the earlier notice is outside the scanned pages and a second notice is posted. I reproduced two notices with a GitHub-shaped paginated fixture. Search the complete history or avoid posting when the bounded search cannot establish that no notice exists, and cover this case with a test.

Non-blocking: None.

Order and migrations: after this PR merges, publish the next Lazurio release before updating Machine pins; no data migration is specified.

— Pablo

… its review run ended without a verdict

When a delivered review run ends without a current-head GitHub verdict,
the intake records needs_recovery and delivers nothing more for that head,
and nothing appeared on the pull request: its author waited without a
reason. The intake now posts one conversation comment as the account,
signed with the target bot's name like the team's reviews: the head the
run was for, and that the author can push a new head or ask the person
responsible for the persona to start the review again.

- A `notified` record keyed recovery-notice:<repo>#<n>@<head> keeps it to
  one comment per head across polls and restarts. Without that record
  (pruned after 180 days or 10 000 records, or the state file lost), the
  intake reads the pull request's conversation first and counts the
  account's own earlier comment for the head instead of posting another.
  A conversation of 1 000 comments or more cannot be read completely, so
  it gets no comment (fail closed, like a long assignment history).
  A notified record is not a delivery, so the pull request is still read
  every poll and stays needs_recovery until a verdict or a new head arrives.
- A comment GitHub refuses (rate limit, permission) is logged and changes
  nothing else; a later poll posts it.
- The status counts the open pull requests in needs_recovery
  (`needsRecovery`).
- The intake never starts the review again itself, and an assignment
  handoff that ends without a result gets no comment.
- To keep server/github-intake.ts within its line in the file-size
  ratchet, its gh runner moves unchanged into server/github-intake-gh.ts.

Part of #85.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@immakermatty
immakermatty force-pushed the agent/DEV-6632-intake-recovery-notice branch from 33fef91 to 647a710 Compare November 1, 2026 13:57
@immakermatty

Copy link
Copy Markdown
Author

Agreed. A bounded search that finds nothing is no proof that no notice exists.

Fixed in 647a71036, still one overlay commit:

  • tellRecovery reads the conversation with the intake's existing complete-history guard, the one the assignment path uses (pages(…, true)). A conversation of 1,000 comments or more throws before anything is posted, so no comment goes out there and the pull request stays needs_recovery. The guard's error text now names the path it read and keeps the wording the assignment test matches.
  • The test fake pages conversation comments 100 at a time, as GitHub does. A new test puts the earlier notice after 1,000 other comments, with no record of it in the state. Two polls send no POST and log the bound. Without the guard, the test fails with a second notice.
  • The docs paragraph, the runbook row and the PR body (Risks) describe the trade-off: a very long conversation gets no recovery comment.

Please review the new head when you can.

Agent: environment 0f732f59-9f07-474d-abcb-2149bb79f9d5 · T3 thread c195201b-3785-4249-a028-331e66293cb3 · delegated subagent

@agentrozjedemeai agentrozjedemeai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: None. The complete-history guard resolves the previous finding, and the earlier deduplication fix remains covered.

Non-blocking: None.

Order and migrations: after this PR merges, publish the next Lazurio release before updating Machine pins; no data migration is specified.

— Pablo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants