fix!: judge every turn with band-sdk-core's turn outcome

[AlexanderZ-Band]

Summary

Every Python adapter's turn is now judged by one rule from band-sdk-core 2.8.0, and the outcome is reported honestly (Phase 2 of INT-1668, the Python sibling of INT-1672; plan attached to INT-1671).

• Complete: the turn replied, declined via band_no_reply, did real work, was settled by the adapter, or already reported a failure.
• Missing reply: anything else. It is reported once with core's missing-reply text (provider band-runtime) and the delivery is marked FAILED. No nudge, no recovery.

This fixes the silent loss in nightly 37177615499 (test_recall_and_ignore_crosstalk_in_busy_room[crewai]): a failed band_send_message, a band_send_event, then an empty completion used to be reported PROCESSED.

Part 1 — plumbing (refactor: commit)

• Pin band-sdk-core==2.8.0; the Band tool effect table and band_no_reply's definition come from core (parity tests in tests/core/test_turn_types.py).
• band.core.turn.Turn (wraps core's TurnLedger) on every AgentTools/FakeAgentTools as tools.turn; @records_turn_effects records each tool method's effect, so dispatch, direct calls, crewai's catalog, parlant and deliver_reply all count.
• send_notice / deliver_notice for adapter-originated posts (approval prompts, busy notices, /status, help, timeouts); turn.settle() when the adapter answers a message itself; relay_reply is the one gate for a model's final text. tests/framework_conformance/test_reply_boundary.py pins every reply-path call per file.
• Custom tools take a required turn and record their declared effect; ACP (external band-mcp) and Letta (external mode) record Band tools from the stream; the Slack tee shares the ledger; per-message tools bind after busy/approval checks.
• In passing: Letta read tool_return_message.tool_name (always "unknown"); it is name.

Part 2 — one judge (fix!: commit)

• The verdict in SimpleAdapter.run_judged_turn, which on_event (OneShot included) and the Slack adapter's Slack-inbound path both call; detached turns (claude_sdk, codex, opencode, cursor_acp) are judged on their normal completion path, never a finally; a cancelled, cleaned-up or interrupted turn posts nothing.
• Exempt via judges_turns: a2a, a2a_gateway, crewai_flow, BandACPServerAdapter, parlant, a static-graph= LangGraph adapter; Slack delegates; contact-hub turns are never judged.
• Every adapter's own reply judging is deleted (crewai ReplyTracker flags, strands terminal_fired, pydantic_ai tool_executed, claude_sdk replied_this_turn, copilot_sdk's no-reply RuntimeError, opencode's filler text, ACP turn_replied_in_room, letta's settle-on-call, parlant's module-global sent flags, codex settled_reply).
• BASE_INSTRUCTIONS and band_send_message's description teach band_no_reply; new docs/turn-outcome.md (runnable snippets) linked from AGENTS.md; "the platform retries" wording corrected.
• Tests: conformance probes for every judged adapter (test_turn_outcome.py + turnprobes/), runtime verdict rows through the real ExecutionContext (tests/runtime/test_turn_verdict.py), ledger tests, per-adapter verdict tests, mid-turn prompt and detached-turn flows; two new E2E smokes (test_no_reply.py, test_missing_reply_reported.py) and the regenerated expected-cells.json.

Deviations from the plan, and why

• Turn, report_unsettled_turn, judge_detached_turn and MISSING_REPLY live in band.core.turn, TURN_FAILURE_PROVIDER in band.core.protocols, and is_contact_hub_turn plus the synthetic-sender constants in band.core.types. SimpleAdapter (core) can't import band.runtime because of an import cycle (runtime/__init__ → oneshot → SimpleAdapter). records_turn_effects stays in band.runtime.turn.
• Turn.detach() always marks a turn released, and judge_detached_turn reports only a turn that is both judged and detached, so a contact-hub turn parked on a decision is never reported later.
• A shared deliver_notice (send_notice wrapped in DeliveryFailedError, like deliver_reply) for adapter posts inside provider try/except blocks.
• cursor_acp's on_interrupt settles the parked turn, so a /stop never produces a missing-reply report.
• Framework-native tools (not (InputModel, handler) pairs) keep their declared effect in strands and pydantic_ai, recorded from those adapters' existing hooks. (InputModel, handler) pairs record once, through execute_custom_tool. Agno tools and ready-made LangChain tools count as observe (documented).
• ACP relays a turn's held text runs as one reply (it used to post one message per run), and parlant relays one engine message per turn, both through relay_reply.
• Letta with auto_relay=False no longer runs its own missing-reply check: it logs and drops the final text, and the shared verdict decides the turn.
• opencode reports a reply it could not post as one send_failure from its own provider, which completes the turn, so the verdict never adds a second report.
• test_missing_reply_reported.py removes band_send_message/band_no_reply with exclude_tools so the turn is silent by construction instead of by prompt. anthropic, claude_sdk, gemini and google_adk ignore include_tools/exclude_tools (a pre-existing gap), so a model could still decline; they are N/A for that one smoke, and their verdict stays covered by the conformance probes. copilot_sdk is N/A because it relays the model's final text.

Follow-up commits

• DecisionRegistry.wait uses asyncio.timeout() instead of wait_for. On Python 3.11, wait_for drops a cancel that arrives in the same tick as the answer, so a cleaned-up turn carried on. This was pre-existing, and it surfaced as a 3.11 CI timeout in cursor_acp.

• OpenCode answers a rejected question with a decline (DECLINED_QUESTION_ANSWER) instead of calling POST /question/{id}/reject. That route carries no feedback, so OpenCode ended the turn and the model never replied. Verified live 2/2. A malformed question with nothing to answer still goes to the reject route.

• The base prompt's plain-text warning ("a turn that ends with the answer written as plain text delivers nothing") is restored.

• E2E fixtures: event-only and remember-only smoke turns end with band_no_reply, the delegating peer asks a question instead of forwarding a bare token, and the copilot successor's repeat prompt is varied.

• Review fixes:

  • Slack-inbound turns are judged again. A silent one is reported, and not re-logged as a crash.
  • A send the claude_sdk dedup wrapper suppresses records nothing: it is an earlier turn's send re-issued.
  • report_unsettled_turn logs a WARNING naming the room.
  • New tests pin each judges_turns opt-out, a Codex interrupted-turn notice, and an undeclared custom tool for six adapters.

• Cross-SDK and ACP-boundary reviews (9cafdc2):

  • OpenCode declines a question-less ask with an empty answer, and the reject route is gone.
  • RoomTurnEmitter stages out-of-process effects until the prompt closes successfully.
  • An already-reported turn failure logs at DEBUG in the runtime.
  • Guards: covered adapters must map to a real probe, and every reply-path call is pinned per file.

• Second review pass (2741048):

  • The runtime skips its own failure report only when a failure post actually landed (_turn_failure_reported), not because an adapter raised TurnResultAlreadyReported. A report that didn't post now gets the generic fallback, at every raise site (f091057).
  • The missing-reply report honors SessionConfig.report_turn_failures_to_room.
  • Claude SDK and Codex log a WARNING for the failures they report, since the runtime now logs reported failures at DEBUG.
  • Tests cover the runtime's DEBUG/ERROR choice on the live and backlog paths, and pin TurnEffect.did_work to core's ledger.
  • Probes carry relays, and the conformance suite uses core's TurnVerdict.

For reviewers

The non-obvious design choices are recorded as resolved inline comments prefixed decision: (review "Design-decision record"). Each states the choice, why, and its guard; to challenge one, reopen its thread rather than starting a new one. main is merged in (c4653ee).

Follow-ups

• INT-1677: report a turn failure only on its final delivery attempt. This was already the case on main, and it only matters when max_message_retries > 1.
• INT-1678: live band_no_reply cells for codex, opencode, cursor_acp, copilot_acp and letta, plus a live OpenCode question-decline row.
• INT-1675: unrelated E2E failures from the first full run.

Release note

BREAKING CHANGE: Every SimpleAdapter turn is now judged by band-sdk-core's turn-outcome rule.

• Adapters that silently dropped a turn now report it and mark it FAILED: langgraph, anthropic, gemini, google_adk, agno, letta, codex/ACP empty turns, and copilot_sdk empty text.
• An undeclared custom tool counts as observe.
• settles_turn_reply, is_terminal_success and missing_reply_error are removed from band.runtime.tools.
• Custom SimpleAdapter subclasses are judged too. Their own notices should use send_notice and tools.turn.settle(), because a post through send_message / deliver_reply counts as the turn's reply.
• AgentToolsProtocol gains turn and send_notice.
• BASE_INSTRUCTIONS and band_send_message's description teach band_no_reply.

Test plan

• uv run ruff check ., uv run ruff format --check ., uv run pyrefly check
• unit suite in dev --all-packages; CI's crewai and parlant job commands in dev-crewai / dev-parlant
• markdown-docs
• the two new smokes live, in the dev and dev-crewai venvs
• gh workflow run e2e.yml -f lane=all -f os=all

🤖 Generated with Claude Code

https://claude.ai/code/session_016nwuEBtdtneuDkJHcvijDQ

[linear-code]

INT-1671

INT-1677

INT-1678

[AlexanderZ-Band]

Review of the turn-outcome change: 21 findings (6 issue, 6 suggestion, 9 nit). Two confirmed defects sit on lines outside GitHub's diff, so they are here rather than inline:

• issue: src/band/integrations/claude_sdk/dedup_tools.py:175 — when a later turn's band_send_message hits the dedup cache (same text and mentions within 30 s), the wrapper returns cached_result without calling inner.send_message, so that turn's fresh AgentTools never records REPLY and on_event reports a missing reply and marks the message FAILED. Before this PR the turn was judged from the successful tool result. The in-flight join path (existing is not None) has the same gap. Proof: a repro that fails at head (TurnResultAlreadyReported: turn ended without a reply) and passes on main.
• issue: src/band/integrations/slack/adapter.py:898 — Slack-originated turns call self._inner.on_message(...) directly and never reach SimpleAdapter.on_event, now the only judge. With the brains' own judges deleted, a Slack turn that posts nothing is no longer reported (a pydantic_ai brain answering in plain text: head emits ['task', 'thought', 'task'], main emits [..., 'error']), and claude_sdk's detached report is off on this path too. Proof: a repro that fails at head and passes on main.

[bandzalkin]

Architectural interaction review: #729 with #730 / #731 — HELD

Scope: Focused review of #729's ACP stream-to-turn-ledger boundary at b6b86a7ea462, combined with the revised busy-retry behavior in #731 and backlog recovery in #730. This is not a full approval or rejection of #729's other adapters.

Constraints: The shared turn ledger must remain the canonical judge, but effects must belong to the delivery whose outcome it judges. An ACP session/prompt rejected with -32003 and data.reason=session_busy owns no remote work; session notifications received during that request can belong to an already-running autonomous turn.

Decisions: Keep the shared ledger and core effect mapping. Attempt-local staging in RoomTurnEmitter is smaller and safer than adding another verdict policy or clearing the delivery-wide ledger from the ACP retry loop. Narration can remain live.

Finding A1-729 — High, demonstrated: src/band/integrations/acp/room_emitter.py:76–78,113–126 records completed external Band-tool effects immediately into tools.turn. The unsuccessful-close guard at lines 178–182 runs too late to discard those effects. With external tool recording enabled, an autonomous band_send_message completion received before a structured busy rejection records REPLY on the new delivery. #731 then retries with a fresh collector/emitter, but the same delivery ledger still contains that REPLY. relay_reply suppresses the accepted retry's answer.

Proof: Real ACP socketpair regression test_rejected_attempt_cannot_suppress_accepted_reply: first request streams an autonomous completed band_send_message, then raises structured busy; second request streams The accepted turn's answer.. Revised #731 alone passes. The combined worktree fails: actual reply.texts == [], expected ["The accepted turn's answer."] (1 failed, 48 deselected). Command: uv run --no-sync python -m pytest tests/integrations/acp/test_client_adapter_behavior.py -q -k rejected_attempt after combining the PRs and bringing in #731's regression.

Smallest correction / architectural verdict: Stage external completed-tool effects per emitter; commit them to the canonical turn ledger only on successful prompt close, before relay_reply. Discard staged effects on rejection, timeout, or cancellation, while retaining live narration. Add the above regression for the combined behavior. HELD for combined integration until that boundary is corrected; #729 remains review-only in this fix round.

[AlexanderZ-Band]

Compared with band-sdk-typescript#306 (INT-1672), which implements the same core rule. 7 findings: 1 issue:, 4 suggestion:, 2 nit:. The issue (the empty OpenCode question) is blocking. This is posted as a comment because GitHub doesn't allow requesting changes on your own PR.

One finding falls outside the diff:

• src/band/runtime/execution.py:2171 (and :1571 for backlog): nit: a missing reply now leaves on_event as TurnResultAlreadyReported and reaches the generic logger.exception("Error processing %s"). Every silent turn, an expected verdict, therefore logs at ERROR with a traceback, on top of report_unsettled_turn's WARNING naming the room. Catching TurnResultAlreadyReported before the generic handler and logging it at WARNING or DEBUG keeps ERROR alerting for real failures.

Adopted in TS from this PR (band-sdk-typescript@d3a3147): the judged gate on detached turns, ACP renaming a call from a later update (Cursor's "MCP: tool"), ACP relaying its text runs as one answer, OpenCode custom tools resolving their room's turn, the platform's blank rule in the relay, and a live missing-reply check.

[AlexanderZ-Band]

Replies to the two review bodies, both addressed in 9cafdc2:

@bandzalkin, A1-729 (the ACP stream-to-ledger boundary): confirmed and fixed as you proposed. RoomTurnEmitter now stages completed out-of-process Band-tool effects, commits them to tools.turn only when the prompt closes successfully (before relay_reply), and discards them on rejection, timeout or cancellation. Live narration is unchanged. The emitter-level guard is test_a_failed_prompt_records_nothing_streamed_during_it, which fails on the old code: a completed band_send_message streamed during a prompt that then raises leaves the turn unreplied. Your socketpair regression test_rejected_attempt_cannot_suppress_accepted_reply needs #731's busy-retry loop, so it belongs in #731 or in the combined branch. This boundary should now let it pass.

Cross-SDK review, the out-of-diff nit (execution.py:2171 / :1571): fixed. Both generic handlers go through _log_turn_error, which logs a TurnResultAlreadyReported at DEBUG without a traceback (it was already logged where it was reported) and everything else at ERROR as before. test_a_missing_reply_is_not_logged_as_a_runtime_error guards it, mutation-checked.

Correction to my earlier comment about the dedup wrapper: following the cross-SDK review, a suppressed duplicate now records nothing, rather than counting as the current turn's reply. Details are on that thread.

[AlexanderZ-Band]

Second review pass at 9cafdc2: 18 inline comments plus 1 in this body. 5 issues, 6 suggestions, 8 nits. Posted as a comment because GitHub doesn't allow requesting changes on your own PR; the five issues are blocking.

One finding sits outside GitHub's diff (the file now matches main):

• issue: src/band/integrations/claude_sdk/dedup_tools.py:175, flagged by groups C and E. The dedup cache outlives update_inner and is keyed only by (content, mentions), and a hit now records nothing. A later turn whose model deliberately repeats an earlier answer within 30 s therefore has its post dropped, sees success, and is then reported as a missing reply and marked FAILED. Before 9cafdc2 the hit recorded REPLY, but that marked a later silent turn replied when the CLI re-issued an old send. The cache can't tell those two cases apart, so this needs a decision rather than another flip.

[AlexanderZ-Band]

Cross-SDK note from band-sdk-typescript#306's second review: a custom tool that resolves to a failure value still records its declared effect here.

invoke_validated_custom_tool (src/band/runtime/custom_tools.py:265-266, at 9cafdc2) calls turn.record(declared_effect(func) or OBSERVE) whenever the handler returns. So a REPLY or ACT tool that reports failure by its return value, such as {"ok": False, ...}, completes the turn: nothing reaches the room, and no missing reply is reported.

TS fixed this in band-sdk-typescript@3562f5b. executeCustomTool records only when the output isn't a failure value (isFailedToolOutput, the same rule its Band tools use when recording). Python could gate the same way, using whatever it already uses to tell a failed Band tool result from a successful one, so both SDKs judge such a turn alike.

[AlexanderZ-Band]

On the review-body finding (src/band/integrations/claude_sdk/dedup_tools.py:175: a dedup hit records nothing): declined, as decided. The cache can't tell a CLI re-issuing an old send apart from a model deliberately repeating an answer within 30 s. Recording REPLY on a hit marked a later silent turn as replied, which hid a real missing reply. Recording nothing reports the rare deliberate repeat as a missing reply instead. Of the two, we chose the loud failure: it shows up in the room and the logs, while a false 'replied' is silent. The file is back to main's version.

[AlexanderZ-Band]

Focused logical-bugs pass on the cycle-2 fixes found two confirmed defects, both fixed in f091057:

• issue: The F1 fix covered only the missing-reply path. claude_sdk (is_error and the turn timeout), codex (4 sites), letta and a2a also call send_failure and then raise TurnResultAlreadyReported without checking that the post landed. So when error-event posts fail, the room heard nothing. This was proven through the real ExecutionContext, live and backlog.
• issue: The RuntimeError that fix introduced escaped on Slack-inbound turns, which have no runtime, and logged a fake "background handler crashed".

Root-cause fix: _handle_turn_failure now trusts only _turn_failure_reported, which send_failure sets when the post succeeds, and no longer the exception type. unsettled_turn_error is deleted, and run_judged_turn always raises TurnResultAlreadyReported again.

Guards:

• test_the_runtime_reports_a_turn_only_when_its_report_did_not_post covers missing-reply and adapter-report × posted and post-failed × live and backlog.
• test_a_slack_turn_whose_report_failed_is_not_logged_as_a_crash.
• Mutation-checked: trusting the exception again, never falling back, always falling back, and a plain error on Slack each fail a guard.

[AlexanderZ-Band]

Design-decision record for peer review: 16 inline comments, each resolved as soon as it was posted. Each one states a choice that has already been made and checked, and why, so it doesn't come back as a new finding. To challenge a decision, reopen its thread.

Decisions with no diff line to attach to:

• claude_sdk dedup (src/band/integrations/claude_sdk/dedup_tools.py, unchanged from main): a dedup hit records nothing on the turn. The cache can't tell a CLI re-issuing an old send apart from a model repeating an answer within 30 s. Recording REPLY would hide a real missing reply, so we accepted the louder failure. Decided in the review thread.
• Retries: a turn failure is reported only on its final delivery attempt, which is main's behavior. It matters only when max_message_retries > 1. Follow-up: INT-1677.
• Live coverage gaps: INT-1678 tracks the live band_no_reply cells for codex, opencode, cursor_acp, copilot_acp and letta, plus the OpenCode question-decline row.

[bandzalkin]

Scope

Reviewed 37cef255cc425e79af40fe0f3a1c5ebf643445d9 against cc5dac33de5c7e0a5173560c5c12697a8b2e32be: the complete PR diff, affected owners/callers, adapter and protocol boundaries, tests, documentation, and all prior review discussions. Pushed the authorized corrections in 37cef255, then re-reviewed them.

Constraints

The core ledger remains the canonical judge. Effects must belong to the delivery that performed successful work; notices are not replies. Recorded decisions and explicit deferrals remain respected, including Claude dedup behavior, retry-budget reporting, and the deferred live-adapter cells.

Decisions

Keep effect recording at successful tool completion; stage external ACP effects until an accepted prompt closes. Preserve tuple/native recording separation and self-hosted MCP ownership. No additional judge or runtime dependency.

Findings and proof

Three confirmed problems were corrected before this approval:

• Explicit custom-tool failure values no longer record ACT/REPLY/DECLINE or suppress fallback, including native Pydantic AI and serialized Strands results. This addresses the existing failure-value note.
• Parlant preserves all non-preamble final segments already present in one event batch before relaying once.
• Letta correlates grouped external MCP returns by tool_call_id, recording every successful effect rather than only the first result.

Regression failures were observed before each correction. Real execution-context, native Strands, installed Parlant event-type, Letta wire-type, and socketpair ACP probes exercised success, failure, suppression, cancellation, ordering, and ownership boundaries. The earlier ACP attribution finding is corrected at this revision.

Final local verification: 6,591 passed / 156 skipped in the full non-live suite, including conformance; isolated CrewAI 105 passed / 6 skipped and Parlant 111 passed / 5 skipped; 28 markdown snippet tests passed. Repository-wide Ruff lint/format and Pyrefly passed, as did both scoped framework type checks, wheel build, and isolated extras-free wheel smoke. Baseline collection succeeded in all three dependency environments.

Verdict

APPROVED. No blocking correctness or architectural finding remains. No paid live-platform/provider run was performed, and this approval does not claim that CI for the newly pushed SHA has completed.