Keep executed tool steps when a generation fails

[DavertMik]

A tool turn can execute several tool calls before the step that follows them fails. When that happens, withRetry re-ran generateText from the original messages — so the calls that had already gone through left no trace in the retry, and whatever they created on the far side got created again, once per attempt.

Seen in Langfuse trace 7d5ddba053a054f6a3725503c9569340: Fisherman created the same suite + test pair four times in 16 seconds. Each attempt POSTed both (200, 200) at steps 3–4 and then died at step 5, alternating between two Groq 400s — attempted to call tool 'json' (the model emitting a harmony pseudo-tool) and Tool choice is required, but model did not call a tool (the model writing its summary as prose under toolChoice: 'required'). The writes escaped the retry boundary; the record of them did not.

What changed

generateWithTools accumulates each completed step's response messages through onStepEnd, and on failure:

• feeds them into the next attempt's messages, so the model resumes from what it already did instead of starting over (the existing invalid-request feedback line is appended after them);
• merges them into the returned responseMessages, so invokeConversation records them on the conversation and the next loop iteration sees them too;
• returns them from the Tool choice is required swallow, which previously returned an empty message list;
• recurses from them in the context-length recovery, which previously restarted from the original messages — the same duplication hazard by a third route.

This is provider-level, so it covers every agent that runs write-effect tools, not just Fisherman.

Not in scope

When retries exhaust and generateWithTools throws AiError, executed steps still do not reach the conversation. That is moot for Fisherman (its loop stops on error) but a live hazard for Tester, which keeps iterating on the same conversation after an error. Fixing it means changing what a failed invocation returns to its caller, which is a bigger decision than this PR.

Independently of this, toolChoice: 'required' in src/ai/fisherman.ts is what turns "model finished and said so in prose" into a hard 400. Worth revisiting separately.

Testing

Three unit tests in tests/unit/provider.test.ts, each verified to fail without the change:

• executed steps carry into the retry and into responseMessages, and the tool runs once;
• the Tool choice is required swallow returns them;
• the context-length recovery reduces messages without losing them.

bun test tests/unit/ (1123 pass) and bun test tests/integration/ (82 pass, 1 skip) are green; bunx tsc --noEmit reports the same six pre-existing errors in provider.ts as main. bun run build fails identically on main in this checkout — a missing chromium-bidi in local node_modules, unrelated to the change.

🤖 Generated with Claude Code

https://claude.ai/code/session_017uAMi5ctXDCk7vbhPnMmEi