Repository navigation
feat(agentctl): add exec --json - #286
Ridwan Hoq (ridhoq) wants to merge 1 commit into
Conversation
92e30b4 to
f244e32
Compare
06a437b to
726f62b
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Replay can abort on an empty cursor, demo tests fail on non-JSON stubs, and step logs retain opaque provider reasoning that plaintext redaction cannot inspect.
5 open issues
What changed in this PR
Adds stable machine-readable output for agentctl exec, preparing scripts for the planned redesign of human-readable CLI output.
Changes:
- Stream session updates and committed records as protobuf JSON Lines with
exec --json, omitting uncommitted model chunks. - Report stdout write failures after draining the turn, with tests for JSON output and write errors.
- Migrate demo checks to JSON and document approval pauses, failure behavior and compatibility.
| File | Description |
|---|---|
examples/agentsuite-substrate/demo/driver.sh |
Parses captured replay JSON for journal checks. |
examples/agentsuite-substrate/demo.sh |
Reads exec, replay and fork results through JSON. |
docs/quickstart.md |
Documents exec JSON, examples and output guarantees. |
docs/approval-gated-tool.md |
Adds a JSON approval-pause example. |
cmd/agentctl/output.go |
Shares protobuf JSON encoding and updates flag help. |
cmd/agentctl/main.go |
Implements exec JSON streaming and write-error tracking. |
cmd/agentctl/main_test.go |
Adds JSON framing, escaping, stream-error and write-error tests. |
client/client.go |
Corrects the OnDelta documentation. |
changelog/unreleased/exec-output-errors.md |
Records stdout write-error handling. |
changelog/unreleased/exec-json.md |
Announces the exec JSON format. |
changelog/unreleased/exec-json-compat.md |
Advises scripts to migrate from human output. |
Review effort: Max
19b9fb6 to
2606acb
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved issues can mask failed executions, obstruct session recovery, and expose unrelated session metadata or raw provider payloads in demo artifacts.
7 open issues
Preserve the session UID and both errors after stdout failure · New Preserve server and journal options in replay recovery instructions · New Propagate exec failures through the JSON pipeline · New Keep raw provider records private and temporary · New Avoid logging unrelated session metadata · New Document exec JSON framing and approval pauses in spec.md · New Update install requirements for exec --json · New
5 resolved since last review
Review effort: Max
2606acb to
e52ec05
Compare
There was a problem hiding this comment.
🟡 Changes recommended
The demo test harness replaces PATH without exposing jq, so tests can fail when jq is installed outside the retained system directories.
3 open issues
7 resolved since last review
Avoid logging unrelated session metadata Keep raw provider records private and temporary Propagate exec failures through the JSON pipeline Preserve server and journal options in replay recovery instructions Preserve the session UID and both errors after stdout failure Update install requirements for exec --json Document exec JSON framing and approval pauses in spec.md
Review effort: Max
e52ec05 to
03b3604
Compare
03b3604 to
2e225c7
Compare
2e225c7 to
b186202
Compare
b186202 to
f34eab6
Compare
exec now takes --json and prints the turn as JSON Lines of protobuf ExecUpdate: the session frames and each committed record as it arrives. Streamed model chunks are left out; the OUTPUT record carries the complete response. An approval pause ends with a session frame whose pending_approval names the call. A failed stdout write no longer goes unnoticed. exec stops writing, reads the stream to its end so the turn is not cut short, and exits with an error that names the session to replay. If the stream also fails, both errors are returned. SIGPIPE behavior is unchanged. Tests and scripts that parsed the human views of exec, replay and fork now read --json, so those views can change: the daemon smoke test, delete_test, the agentsuite-substrate demo, driver and trace-turn scripts, and the Kubernetes tour. Refs agent-primitives#278 Signed-off-by: Ridwan Hoq <943533+ridhoq@users.noreply.github.com>
f34eab6 to
7b55641
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Unescaped Unicode separators in failure messages can create misleading log lines, while the quickstart’s decoded terminal escapes can let model responses manipulate the terminal display.
2 open issues
2 resolved since last review
Review effort: Max
| agentctl exec --server= --journal /tmp/qs.db --session "$SID" --input "hello again" --json | | ||
| jq -r 'select(.record.event.kind == "EVENT_OUTPUT") | [.record.event.message.parts[]?.text.text // empty] | join("")' |
| | [.message.parts[]?.text.text // empty] | join("") | gsub("\n"; "\\n")' | | ||
| python3 -c 'import sys, unicodedata | ||
| for line in sys.stdin: | ||
| print("".join(c if unicodedata.category(c)[0] != "C" else ("\\x%02x" if ord(c) < 128 else "\\u%04x") % ord(c) |
Anish Ramasekar (aramase)
left a comment
There was a problem hiding this comment.
Ridwan Hoq (@ridhoq) exec --json works as spec.md describes. I ran the built binaries embedded and remote: the session frame comes first, then committed records, with a final session frame only on park. Deltas are left out, text output is byte-identical to main, and a failed stdout write now exits 1 after draining. The new tests catch every mutation of the stream logic I tried except dropping INPUT.
Two doc changes before this goes in, both small: the SIGPIPE caveat also applies to remote turns, and the quickstart's jq -r filter prints raw model text to the terminal. Details inline; the rest are nits.
| are omitted; readers ignore update kinds they do not expect. A stream error leaves the lines already | ||
| written and exits unsuccessfully, without retrying the turn. A failed stdout write ends the command | ||
| unsuccessfully after the stream is read to its end, except that on Unix a closed pipe still ends | ||
| `exec` with SIGPIPE, which can cut an embedded turn short; readers consume the whole stream. |
There was a problem hiding this comment.
This applies to remote turns too, not only embedded. agentctl dies on SIGPIPE, which closes the Exec stream, and the server runs the turn on stream.Context() (session/service.go:630, 844). With a 3 s streaming fake model, agentctl exec --server $S --input hi --json | head -1 left EXECUTION_START, INPUT, MODEL_CALL and then EVENT_ERROR ... context canceled, with no END. --session S ... | true exited 141 before the turn ran. main's text mode behaves the same, so this isn't a regression, but the sentence reads as if remote callers are safe. Could it drop "embedded", or should exec --json ignore SIGPIPE so EPIPE takes the drain path like other write errors? Either works if the docs match.
| has already resolved it. Only a recorded `EVENT_END` shows that a turn completed. Ignore update | ||
| kinds you do not expect, since later releases may add some. If the stream fails, the lines already | ||
| written stay and the command exits with an error; it does not retry the turn. Read the stream to its | ||
| end: on Unix, closing the pipe early ends `exec` with SIGPIPE, which can cut an embedded turn short. |
There was a problem hiding this comment.
Same as spec.md:100: closing the pipe early cancels a remote turn as well. The | head -1 | jq -r .session.metadata.uid pattern this section suggests for learning the UID is exactly the case that hits it.
| agentctl list --server= --journal /tmp/qs.db --json | jq -r '.sessions[]? | .metadata.uid' | ||
| agentctl replay --server= --journal /tmp/qs.db --session "$SID" --json > records.jsonl | ||
| agentctl exec --server= --journal /tmp/qs.db --session "$SID" --input "hello again" --json | | ||
| jq -r 'select(.record.event.kind == "EVENT_OUTPUT") | [.record.event.message.parts[]?.text.text // empty] | join("")' |
There was a problem hiding this comment.
jq -r decodes the JSON escapes, so model text reaches the terminal raw. With a reply containing \u001b[8m, this filter wrote an ESC byte, where the human view prints \x1b (main.go:691-723). The echo example is fixed text, but people will copy this for a model-backed session. Could the example print JSON-quoted text (jq -c, or @json), or add a one-line note that -r is for trusted consumers only? This is Copilot's open thread here as well.
| | [.message.parts[]?.text.text // empty] | join("") | gsub("\n"; "\\n")' | | ||
| python3 -c 'import sys, unicodedata | ||
| for line in sys.stdin: | ||
| print("".join(c if unicodedata.category(c)[0] != "C" else ("\\x%02x" if ord(c) < 128 else "\\u%04x") % ord(c) |
There was a problem hiding this comment.
nit: this escapes category C only, but agentctl's needsEscape (main.go:694) also escapes Zl and Zp, so U+2028 and U+2029 pass through raw here. Escaping those too would make the comment above accurate.
| return &v1.ExecUpdate{Update: &v1.ExecUpdate_Session{Session: s}} | ||
| } | ||
| first := sessionUpdate(&v1.Session{Metadata: &v1.ResourceMetadata{Uid: "sess-json"}}) | ||
| output := &v1.ExecUpdate{Update: &v1.ExecUpdate_Record{Record: &v1.LogRecord{Seq: 2, Event: &v1.Event{ |
There was a problem hiding this comment.
nit: the stream has OUTPUT but no INPUT, so a mutation that drops only INPUT passes both the agentctl and agentsessionsd tests. An INPUT frame in the supplied and expected stream would pin "every committed record". Separately, nothing covers text-mode exec with --session: setting newSession: true at main.go:356 passes every test. A row asserting no session line would cover it.



First step toward #278: give scripts a stable way to read
exec, so the human output ofexec,replayandforkcan be reworked next without breaking them.What changes
agentctl exec --jsonprints the turn as JSON Lines, one protobufExecUpdateper line as it arrives, using the same encoding as the other--jsonoutput:session, so a script learns the UID of a sessionexeccreated.record, holding the sameLogRecordareplay --jsonline holds.sessionwhosepending_approvalnames the call.EVENT_OUTPUTcarries the full response, and nothing would read them. An opt-in flag can add them later; the docs tell scripts to ignore update kinds they don't expect.execexits with the error. No retry.execexited 0).execstops writing, reads the stream to its end so the turn isn't cut short, then exits with an error noting the records are committed. The error names the session to replay and says to use the same--serveror--journal. If the stream also fails, both errors are reported, still naming the session, and nothing claims the turn finished. This applies to both text and JSON. SIGPIPE behavior is unchanged, andTestResponseBrokenPipestill asserts it.Example
From the built-in server with the default
echoharness:Each line is one
ExecUpdate. The first and theEVENT_OUTPUTline, pretty-printed:{ "session": { "metadata": { "project": "default", "uid": "sess-2be5b546e37dbb75f6e49d15", "create_time": "2026-11-18T17:50:35.157697716Z", "update_time": "2026-11-18T17:50:35.157697716Z" }, "harness": "echo", "exec_state": "EXEC_PENDING", "compute_state": "COMPUTE_NONE", "deletion_state": "DELETION_ACTIVE" } } { "record": { "seq": "4", "prev_hash": "aa9a446a5c8271f940e83c4f0318d6e3c0bc2650f9216f10271eb641894f83d4", "content_hash": "debf5e797cd152d76b6259b05a1da99dea8ea1a7194dbeb3a887eead8b5eb349", "fence": "1", "event": { "execution_id": "7d75d32b130446a5", "kind": "EVENT_OUTPUT", "message": { "role": "assistant", "parts": [ { "text": { "text": "echo:hello" } } ] } } } }Getting just the reply:
A failed stdout write, which used to exit 0:
The quickstart's JSON section now includes a similar example.
Tests and scripts moved off text parsing
The tests and scripts that checked or parsed the human output of
exec,replayorforknow read--json, so the next PR can change that output. What's left is display only, or relies on something the next PR keeps:driver.sh's split-view highlight assumes one line per record; the next PR updates it with the layout.examples/substrate-kind/smoke.shreads thesession <uid>header, which stays.get,list,suspendanddeleteoutput stay as they are; those formats aren't changing.What changed:
cmd/agentsessionsd/lifecycle_e2e_test.go(the daemon smoke test): turn outputs, END, the fork child and the replay prefix across delete and restart are now checked on decoded records withproto.Equal, instead of text regexes and byte prefixes.cmd/agentctl/delete_test.go: the fork child comes fromfork --json, and the audit replay check usesreplay --json.examples/agentsuite-substrate/demo.shreadsexec,replayandforkthrough--json, parsed withjq, whichup.shand the README already require.saystill shows each turn, viareplay --from. Every read consumes all its input, because ajq first(...)that stops early could endreplaywith SIGPIPE underpipefail.examples/agentsuite-substrate/demo/driver.shcaptures the journal withreplay --jsonand parses it withpython3, as it already parses its other JSON.seq kindline per record instead of the JSON, which holds provider data (such as opaque reasoning bytes) that its redaction cannot inspect.mktempfile that is deleted on exit, not to the shared work directory. Only the child's UID is logged, and the parent and child comparison keeps only content hashes.content_hash(chapter 9).list --json, matched by parent and name.examples/agentsuite-substrate/demo/trace-turn.sh: the model-call hop reads a JSON journal kept outside the saved trace directory. The humanjournal.logstays for reading.examples/agentsuite-substrate/scripts_test.go: the fakeagentctlbehind thedemo.shtests now prints JSON Lines for--json.hack/demos/kubernetes-tour.sh: the tour showsSID=$(agentctl create ...)before its firstexec, and gets the fork child fromlist --json. The landing-page cast is unchanged until it is re-recorded.Docs and changelog
docs/quickstart.md: anexecrow in the JSON table, a jq example, and how pauses, chunks and failures appear. It also says the human output ofexec,replayandforkcan change.docs/approval-gated-tool.md: how a script detects the approval pause.pipefail, so a failedagentctlfails the pipeline, not justjq.exec --jsonneeds a build frommainuntil the next release; v0.4.0 has noexec --json.spec.md§1.2 defines theexec --jsonframing: the first session frame, committed records, the approval-pause frame, deltas left out, and stream and write failures. It also says the human output ofexec,replayandforkisn't a stable contract.Added(exec --json),Compatibility(human output will change; use--json),Fixed(stdout write errors).Not in this PR
Testing
gofmt,go build ./...,go vet ./..., golangci-lint (changed packages),go test -race -shuffle=on ./cmd/agentctl ./cmd/agentsessionsd ./client, theexamples/agentsuite-substratemodule'svet, tests and golangci-lint,hack/check-doc-links.py,go run ./hack/doccheck,go run ./hack/changelog check.TestExecJSON: lines round-trip to the sent updates, with and without--session; deltas are left out; escaped control and format characters; the approval-pause frame; a stream error.TestExecWriteError: text and JSON modes.exec --jsonagainst the built-in server.demo.sh,driver.sh,trace-turn.shand tour helpers and checks were exercised against real journals from the built-in server, including fork, multi-line text and checks that must fail. I could not rundemo.sh,driver.shor the tour end to end, because they need substrate or a cluster.