Skip to content

Fix: skip Copilot failure recalls from subagent tool hooks - #197

Open
aayank13 wants to merge 1 commit into
mainfrom
fix/copilot-subagent-recalls
Open

aayank13 wants to merge 1 commit into
mainfrom
fix/copilot-subagent-recalls

Conversation

@aayank13

Copy link
Copy Markdown
Owner

Fixes #158

Copilot CLI runs the user's hooks inside subagents too, and it fills in sessionId with the subagent's agent id (for example explore-0) in place of the root session id. A subagent hook payload has no other marker: agentName and agentId only appear in subagentStart and subagentStop, and tool payloads have no transcript path. So when a subagent ran cargo test and it failed with an error a procedure was learned from, postToolUse served that procedure to the subagent and recorded the injection under explore-0. Copilot writes the subagent's events into the parent's session-state/<root id>/events.jsonl, tagged with agentId, and the parser leaves them out of the parent's trace. No trace with the agent id is ever learned, so nothing settled that injection: its outcome was never scored and the row stayed in the store for good.

Now a shell failure from postToolUse or postToolUseFailure is only recalled when the session has its own events.jsonl under Copilot's session-state directory. That is exactly the condition for an injection to be settled later. A subagent never gets a directory of its own, since its agent id is made from the task name or the tool call id and session directories are named by the root session's UUID. A root session always has one by the time a tool finishes: Copilot starts writing events.jsonl as soon as the first user.message is emitted (with a 100 ms debounce), and a tool can only run after a model round trip that follows it. Other subagent events already do nothing: agentStop and preCompact from a subagent have no transcript, so their ingest and reminder are skipped, and sessionStart/sessionEnd don't fire in subagents.

I didn't use the shape of the session id as the signal. Root ids are UUIDs and agent ids aren't today, but agent ids come from tool call ids whose format depends on the model provider, and SDK hosts can pick their own session ids, so that check would be guessing. The trade-off of the transcript check is that someone who moves Copilot's config with --config-dir but doesn't set COPILOT_HOME gets no failure recalls, because their transcripts aren't where Trodden looks. In that setup nothing is recorded wrongly, they just miss the recall.

I left userPromptSubmitted alone on purpose. Subagents fire it too, with the agent id, so their task prompt can get a recall the same way. But the transcript check can't tell the two apart there: in a new root session Copilot fires the hook before the first user.message, so events.jsonl doesn't exist yet and the check would drop the recall for the first prompt of every new session, which is the most useful one. That needs its own fix.

To reproduce, I built the binary, used a scratch TRODDEN_HOME and COPILOT_HOME, learned a procedure by ingesting a Copilot events.jsonl where cargo test failed with E0425, the file was edited and the test then passed, and fed trodden hook copilot two postToolUse payloads with the same failing output: one with the root session id and one with a subagent id. Before the fix both got the procedure back, and the injections table had a row for the subagent id next to the root one. After the fix the subagent payload gets no reply and only the root session's injection is recorded.

The new test sends a failing shell result through postToolUse, postToolUseFailure and the PascalCase PostToolUse, checks each is recalled for the root session, then swaps in a subagent id and checks it is ignored. It fails before the fix and passes after. The existing failure test now sets up the root session's events.jsonl first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Copilot CLI: failure recalls in subagents are recorded under the child session id

1 participant