Skip to content

Agent: summarize tool calls as they start - #1261

Merged
AnthonyRonning merged 2 commits into
masterfrom
agent/tool-summary-at-start
Oct 27, 2026
Merged

AnthonyRonning merged 2 commits into
masterfrom
agent/tool-summary-at-start

Conversation

@AnthonyRonning

@AnthonyRonning AnthonyRonning commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Agent tool-call summaries now appear while the call runs, instead of after it finishes. Based on current master (Agent 0.0.9 release d7848049). Additive only: remote wire stays 7, management protocol stays 7.1.0, no app version bump.

What

  • Trigger: a tool row is summarized as soon as it carries the model's complete input. Native rows only get their input once the call is fully formed (ToolCallEnd / ToolExecutionStart), so a half-streamed call is never summarized.
  • Input-only operation: these summaries use a new summarize_tool_call_start operation with the merged summary model (Use normal DeepSeek without thinking for Agent summaries #1279: DeepSeek V4.1 Flash, thinking off). Its prompt:
    • asks what the call is for and what it acts on;
    • ignores incidental timeouts, sleeps, retries and output limits;
    • must never state or imply a result.
  • Unchanged:
    • the output-based summarize_tool_call (five arguments) behaves as before for callers that use it;
    • the expanded card always shows the raw output;
    • running, completed and failed statuses stay separate from the label;
    • external-agent rows keep drawing their own activity.

Row identity and storage

  • Each request carries a token in summary_inflight that maps to its row's current key.
  • When a live row (live:<run>:<occurrence>) becomes durable (stored:<task>:<epoch>:<seq>), rekey_summary moves everything tied to it together: the summary, pending dedupe, queued ids, in-flight tokens, the unsaved marker and the output fallback.
  • One model request per row, attached to the canonical row, and stored exactly once to its exact task/epoch/row. Session switches drop all of it.

Remote capability

  • Advertisement: a host advertises tool_summary_start_v1 in optional-management PeerInfo. Support follows the existing current-major rule: 7.x with the capability is supported, while 6.x and 8.x are not.
  • Local host: always the same build (exact BUILD handshake), so there's no probe.
  • Remote host: the viewer asks peer_info over its own task connection.
    • The read is bounded by 5 s, and nothing on the peer is changed by it.
    • It doesn't use the management remote_peer_info path, which also runs pairing-grant recovery.
  • Capability absent (a validated answer without it): cached for that connection. Calls are summarized from their output once they finish, exactly as before. A reconnect checks again.
  • Probe failed or timed out: not cached. That one call falls back to its output, and the next call asks again.

Validation

All of this ran on a local maple-dev-env stack (ts1261) with the local Pro fixture account; no hosted service was used. The changed source was 37894d2a plus the follow-up commit.

Unit tests

  • Pre-commit hook: cargo fmt --check, cargo clippy --workspace --all-targets -D warnings, cargo test --workspace. 2323 passed, 0 failed on 05c3c135.
  • Focused suites: 42 app tests and the runtime summarizer test, covering:
    • persistence first, then the delayed reply;
    • reply first, then persistence;
    • a queued row becoming durable while all three slots are busy;
    • output fallback under the durable key;
    • a probe failure that isn't remembered;
    • capability and major-version rules;
    • frozen fixtures: the shipped five-argument command and Agent 0.0.9's real 7.1.0 PeerInfo, which lacks the capability.

Running app (Maple Work debug build, single computer)

  • Running card: a sleep 30 call showed its summary while running.
  • Expanded card: it keeps the input and raw output.
  • Failed call: a failing ls kept its red "failed" status, and the label didn't claim success.
  • Calls beyond the three slots: five parallel calls produced exactly five requests, all labeled while running.
  • Relaunch: every summary came back from the store with 0 new requests.
  • Prompt A/B: the same binary with only the prompt swapped, both on DeepSeek, on identical prompts.
    • Both prompts gave specific labels. The old llama-era generic labels ("Runs command with 30 second timeout") are gone with either prompt under Use normal DeepSeek without thinking for Agent summaries #1279's model.
    • On a still-running failing call, the old prompt wrote "…lists a nonexistent directory path", implying the outcome. The new one wrote "Lists the contents of /definitely/missing/path-1261."

Real pairing: three isolated instances on one Mac paired through public Iroh relays, relay only with no direct IP paths. One controller (this build) and two execution peers.

  • Peer without the capability: a real build of release commit d7848049 (Agent 0.0.9).
    • Pairing and ordinary remote tasks worked, and the peer showed "Running v0.0.9".
    • While its tool call ran, the card showed only "Terminal". After completion it got the output-based label "Waited 20 seconds, then printed legacy-peer-ok."
    • Log: requested from input → waits for its output (already under its durable key) → requested once, from output.
  • Compatible updated peer: this build. "Waits 20 seconds then echoes capable-peer-ok." showed while the call was running, from one input request.
    • After the controller restarted, that summary loaded from the peer's store with 0 requests.
  • Transient failure: with a fresh connection, a watcher paused the capable peer's host daemon (SIGSTOP) 1 ms after the controller requested a summary, then resumed it after 9 s.
    • The probe timed out at 5 s, and that call fell back to its output.
    • The next call on the same connection asked again and got an input-based label, so the failure wasn't cached as "unsupported".
    • The first two attempts lost the race: the probe was answered before the pause landed. Those runs are in the notes.
  • Update status: "Refresh update status" behaved the same against the 0.0.9 peer and this build. Unpackaged builds can't check for updates, and both kept the last reported version visible.

Limits

  • Same Mac: all instances ran on one Mac against Local services, which isn't evidence from two physical machines or real NAT traversal.
  • Packaged updates: not exercised.
  • Windows: untouched and untested, per instruction.
  • Labels are model-dependent: the prompt was compared on a handful of real calls, not a broad evaluation.
  • History reload: a reload that replaces live rows without an alias can still re-ask for a row that changed key.
  • Hosted CI: not waited for, per instruction.

🤖 Generated with Claude Code

@AnthonyRonning AnthonyRonning changed the title feat(agent): summarize tool calls as they start Agent: summarize tool calls as they start Oct 24, 2026
@AnthonyRonning
AnthonyRonning force-pushed the agent/tool-summary-at-start branch from 3653164 to cb7d966 Compare October 25, 2026 09:12
@AnthonyRonning
AnthonyRonning removed this pull request from stack #1262 October 25, 2026 09:12
@AnthonyRonning
AnthonyRonning changed the base branch from codex-remote-upd-maple to master October 25, 2026 09:12
@AnthonyRonning
AnthonyRonning force-pushed the agent/tool-summary-at-start branch from cb7d966 to a7eb13d Compare October 26, 2026 02:02
@AnthonyRonning
AnthonyRonning changed the base branch from master to codex-agent-files-maple October 26, 2026 02:02
@AnthonyRonning
AnthonyRonning force-pushed the codex-agent-files-maple branch from fe495f1 to cddf1c0 Compare October 26, 2026 04:52
AnthonyRonning and others added 2 commits October 27, 2026 00:50
Request a tool call's one-line summary as soon as its row carries the
model's complete input, instead of after the tool finishes. The input-only
summary is a separate summarize_tool_call_start operation whose prompt does
not claim a result; summarize_tool_call keeps its output-based behavior for
callers that still use it. The expanded card always shows the raw output,
and the running/failed status stays separate.

Each request carries a token mapped to its row's current key. When a live
row becomes durable, its summary, pending, queued and in-flight requests,
unsaved marker and output fallback move with it, so a row is asked once,
the reply lands on the canonical row, and it is stored once to its exact
task/epoch/row.

A remote host advertises input-only summaries with the optional
tool_summary_start_v1 capability under management protocol 7.0.0. Only
validated peer information is cached per connection. When support is
missing or discovery fails, that call is summarized from its output once it
completes, and the next call checks again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tighten the input-only summary prompt: label what a tool call is for and
what it acts on, ignore incidental timeouts, sleeps, retries and output
limits, and never state or imply a result for a call that may still be
running or may fail. The output-based summarize_tool_call is unchanged.

A remote viewer now reads the host's optional capabilities with peer_info
over its own task connection, bounded by a timeout, instead of the
management remote_peer_info path, which also runs pairing-grant recovery.
Only a validated answer is cached for the connection. A failed or
timed-out probe leaves that call on the output-based summary and the next
call asks again. Management protocol stays 7.1.0; the frozen fixture is
the peer information Agent 0.0.9 reports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AnthonyRonning
AnthonyRonning force-pushed the agent/tool-summary-at-start branch from a7eb13d to 05c3c13 Compare October 27, 2026 03:44
@AnthonyRonning
AnthonyRonning removed this pull request from stack #1292 October 27, 2026 03:44
@AnthonyRonning
AnthonyRonning changed the base branch from codex-agent-files-maple to master October 27, 2026 03:44
@AnthonyRonning
AnthonyRonning marked this pull request as ready for review October 27, 2026 04:19
@AnthonyRonning
AnthonyRonning merged commit 94dcba3 into master Oct 27, 2026
@AnthonyRonning
AnthonyRonning deleted the agent/tool-summary-at-start branch October 27, 2026 04:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant