Skip to content

fix(stream): preserve tool-call identity by ID and reject invalid arguments - #39

Merged
simonsmh merged 7 commits into
simonsmh:mainfrom
summer-tt:fix/tool-call-identity-aggregation
Oct 20, 2026
Merged

simonsmh merged 7 commits into
simonsmh:mainfrom
summer-tt:fix/tool-call-identity-aggregation

Conversation

@summer-tt

Copy link
Copy Markdown
Contributor

Summary

Qoder streams tool calls as OpenAI-style deltas with id as the call's identity and
index as a continuation hint; in observed streams, different explicit IDs can reuse one
index. This implementation therefore keys identity by explicit ID and uses index only
as a correlation hint — a conservative provider rule, not a gateway guarantee.
Previously, index-keyed aggregation merged two explicit IDs and a catch {} fallback
published the survivor with {}.

Reproducer (synthetic wire deltas)

Two explicit IDs reusing index: 0:

{ tool_calls: [ { index: 0, id: "call_A", function: { name: "read", arguments: '{"path":"/a"}' } } ] }
{ tool_calls: [ { index: 0, id: "call_B", function: { name: "read", arguments: '{"path":"/b"}' } } ] }

Before, both deltas shared one index-keyed state and the survivor was published with {};
after, each call keeps its own ID and arguments.

What changed

  • ID is identity; index is a hint. Explicit IDs are tracked in a map; a new ID binds
    only to a unique provisional call and never overwrites another identity. Reusing one
    index across different IDs is the supported case; only an ID-less continuation with
    several candidates fails (Ambiguous tool-call continuation) instead of guessing.
  • Provisional and retired calls. Argument-first fragments buffer until the block
    opens; a completed call retires only after a strict JSON-object parse; an
    explicit ID can re-associate with a retired call; a conflicting later name fails.
  • Strict arguments, no {} fallback. An incremental scan finds only a possible
    root-object boundary; JSON.parse remains the grammar authority. Empty bytes stay legal
    and mean {}; non-empty malformed, non-object, or trailing-garbage input throws.
    Required-schema enforcement stays with the SDK, so {} to a tool with a required field
    becomes a validation error, not an execution.
  • Batch, ordering, lifecycle. Calls are validated before any toolcall_end; if one is
    bad, none gets a success event. Blocks open in first-identifiable-event order and keep
    direct references through the thinking-tag splice. finish_reason: "length" is
    preserved; reader cleanup runs in finally; new fragments are scanned incrementally,
    each call parsed once, with Map/Set lookups — no complexity proof for arbitrary inputs.

Validation

  • CI-covered suite: 360/360 tests pass (previous 311 retained); lint, typecheck, and
    build pass on the production revision. These run in CI for normal changes.
  • Manual offline SDK script (verify-tool-call-flow.ts): 33/33 across SDK 1.0.4,
    1.2.0, and 1.3.0 (11 each; original nine fixtures preserved). It drives the built
    package through the real host SDK with a synthetic credential and real dispatch; its
    guards are Node API guards, not an OS sandbox. New zero-byte and "{}" fixtures for a
    required-path tool yield {} in the parser, an SDK validation error on the correct
    tool-call ID, zero implementation calls, and error replay in the next request. Its
    startup isolation fix also has a real RED/GREEN.
  • Live SDK continuation smoke: One real SDK run used Pi SDK 1.3.0 with
    qoder/DeepSeek-Flash:max through the normal provider factory and SDK authentication,
    using the same unchanged normal bundle as offline validation. Across two synthetic
    in-memory fixtures and two chat requests, tool results were returned for the correct
    calls and the interaction reached the expected final stop.
  • Bound evidence: consumer/CLI checks (27 SDK cases, 6 CLI cases) ran on the
    production revision and were not re-run for the script-only commit; stream.ts and
    bundle hashes are unchanged.
  • Run manually (not wired into npm test/CI or the published package):
node node_modules/tsx/dist/cli.mjs scripts/verify-tool-call-flow.ts \
  --package-root /path/to/package \
  --sdk-root /path/to/node_modules \
  --output /path/to/report.json

Performance

Full npm test wall-clock: +1.895716% against the ≤2% gate. Parser scenarios pass the
current rule — 22/25 under 30 ms, the other three within +0.193%, +2.634%, and +3.556%
relative (per-invocation group-average median; not B−A or a per-sample claim). The
original zero-tolerance FAIL/exit 1 record is unchanged and is not relabeled as a new
pass. Local bounded workloads; no universal claim.

Risks and non-claims

  • Live verification covers only this one model/SDK path. It does not establish all-model
    coverage or general gateway-protocol behavior, and it does not cover pi-subagents'
    detached-launcher/provider path. Reused-index cases have deterministic synthetic
    coverage; this smoke does not independently verify the response delta shape.
  • Not a gateway protocol guarantee. Reusing one index across explicit IDs is the supported
    normal case; failures cover ID-less continuations with several candidates, identity
    conflicts, and invalid/incomplete or non-object JSON.
  • Earlier empty-argument or dropped-call reports can have several causes; this change does
    not attribute all of them to this defect.
  • CI runs the 360 unit tests; the SDK integration script is manual, so that flow is not
    automatically guarded.

Scope

Only three files change: src/protocol/stream.ts, src/__tests__/stream.test.ts, and
scripts/verify-tool-call-flow.ts; no dist, dependency, lockfile, build/release
configuration, or unrelated protocol changes.

@simonsmh
simonsmh merged commit ed5f18e into simonsmh:main Oct 20, 2026
1 check passed
jischeng added a commit to jischeng/pi-provider-qoder that referenced this pull request Oct 23, 2026
Sync upstream v0.6.2..v0.6.5 while keeping this fork's features, then fix the
bugs that surfaced during the review. The upstream changes and the fixes touch
the same functions, so they land together rather than as artificially split
commits that would not pass on their own.

Upstream taken: tool-call aggregation by ID instead of stream index (simonsmh#39),
combined thinking-disable signals (simonsmh#40) with the effort-less catalog exception,
bounded model-busy recovery (simonsmh#41), omp effort metadata (simonsmh#42), service-enabled
Auto retention (simonsmh#43), COSY version tracking, and wire-format tool-call IDs.
Fork behaviour preserved: multi-account providers, per-account entitlement
pre-flight, price factors, keep-alive SSE, premature-disconnect surfacing,
XML <tool_call> recovery, and always-on-thinking effort fallback.

Fixes:

1. Cross-region credential leak. The global provider prefix ("qoder") is a
   strict prefix of the CN one ("qoder-cn"), so `id.startsWith(prefix + "-")`
   matched CN slots when listing global accounts. A global catalogue refresh
   then called /model/list with CN credentials against the GLOBAL endpoint and
   wrote that account into the global cache. Added isProviderOfMode() in
   region.ts and routed both account discovery and re-registration through it.

2. System-message folding was computed and discarded. The request body still
   prepended systemText to the unfiltered list, so it carried the top-level
   prompt plus every system turn instead of one merged message.

3. Orphan `tool` messages. A result whose id no preceding assistant turn
   declared was passed through (the "no assistant to pair with" case returned
   null, which read as permission rather than a veto). Upstreams reject that
   with "tool must follow a message with tool_calls", failing every later
   request in the session. Affected: resumed mid-turn, a user turn between call
   and result, and calls the lookahead trimmed.

4. Startup could hang forever. The default export awaits automatic login, and
   the PAT exchange / token refresh had no deadline, so an upstream that
   accepted the connection and never answered blocked extension load with no
   models, no UI, and no error. Bounded via authFetchSignal() (15s, merged with
   the caller's signal); the usage fetch defaults to a 10s timeout too.

5. Multi-pane torn reads of the COSY version cache. Plain writeFileSync let a
   second process parse a half-written body, throwing and leaving that pane on
   the bundled fallback version for the rest of the day. Now write-then-rename,
   matching the model catalogue.

6. Non-object tool arguments. `<tool_call>` XML whose arguments were an array,
   scalar, null, or a non-JSON string produced a value violating
   `ToolCall.arguments: JsonObject`, so argument validation failed instead of
   the tool running with no arguments.

7. Content order shuffled around tool calls. When a stream produced text, a
   tool call, then more text, the trailing text was appended to the block
   BEFORE the call: content read ["BEFOREAFTER", toolCall] and events addressed
   indexes that no longer matched. Root cause was a cached text-block index in
   ThinkingTagParser.emitText (and the equivalent branch in stream.ts) that was
   never re-checked once tool calls appended to `content`. Present upstream in
   v0.6.5 as well; its test asserted the shuffled order, so that assertion is
   corrected and a contentIndex-consistency check added.

Also fixes provider re-registration across session replacement (issue #1): the
registered-provider set was module-level, and ESM modules stay cached, so
/resume, /new and /fork built a ModelRuntime with zero Qoder providers and the
picker fell back to the default model. Now a per-instance WeakMap cleared on
session_shutdown.

Deviating points from upstream are marked `// fork:` in the source so the next
sync can enumerate them mechanically.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants