Skip to content

feat(agentctl): add exec --json - #286

Open
Ridwan Hoq (ridhoq) wants to merge 1 commit into
agent-primitives:mainfrom
ridhoq:feat/exec-json
Open

Ridwan Hoq (ridhoq) wants to merge 1 commit into
agent-primitives:mainfrom
ridhoq:feat/exec-json

Conversation

@ridhoq

@ridhoq Ridwan Hoq (ridhoq) commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

First step toward #278: give scripts a stable way to read exec, so the human output of exec, replay and fork can be reworked next without breaking them.

What changes

  • agentctl exec --json prints the turn as JSON Lines, one protobuf ExecUpdate per line as it arrives, using the same encoding as the other --json output:
    • The first line is a session, so a script learns the UID of a session exec created.
    • Each committed record follows as record, holding the same LogRecord a replay --json line holds.
    • A turn that pauses for approval exits 0 and ends with a session whose pending_approval names the call.
    • Streamed model chunks are left out. They aren't committed, the completed model call's EVENT_OUTPUT carries the full response, and nothing would read them. An opt-in flag can add them later; the docs tell scripts to ignore update kinds they don't expect.
    • If the stream fails, the lines already written stay and exec exits with the error. No retry.
  • Fix: a failed stdout write no longer goes unnoticed (it was discarded and exec exited 0). exec stops writing, reads the stream to its end so the turn isn't cut short, then exits with an error noting the records are committed. The error names the session to replay and says to use the same --server or --journal. If the stream also fails, both errors are reported, still naming the session, and nothing claims the turn finished. This applies to both text and JSON. SIGPIPE behavior is unchanged, and TestResponseBrokenPipe still asserts it.
  • The human output is unchanged.

Example

From the built-in server with the default echo harness:

$ agentctl exec --server= --journal /tmp/demo.db --input hello --json | jq -c '{session: .session.metadata.uid, seq: .record.seq, kind: .record.event.kind}'
{"session":"sess-2be5b546e37dbb75f6e49d15","seq":null,"kind":null}
{"session":null,"seq":"1","kind":"EVENT_EXECUTION_START"}
{"session":null,"seq":"2","kind":"EVENT_INPUT"}
{"session":null,"seq":"3","kind":"EVENT_MODEL_CALL"}
{"session":null,"seq":"4","kind":"EVENT_OUTPUT"}
{"session":null,"seq":"5","kind":"EVENT_END"}

Each line is one ExecUpdate. The first and the EVENT_OUTPUT line, pretty-printed:

{
  "session": {
    "metadata": {
      "project": "default",
      "uid": "sess-2be5b546e37dbb75f6e49d15",
      "create_time": "2026-11-18T17:50:35.157697716Z",
      "update_time": "2026-11-18T17:50:35.157697716Z"
    },
    "harness": "echo",
    "exec_state": "EXEC_PENDING",
    "compute_state": "COMPUTE_NONE",
    "deletion_state": "DELETION_ACTIVE"
  }
}
{
  "record": {
    "seq": "4",
    "prev_hash": "aa9a446a5c8271f940e83c4f0318d6e3c0bc2650f9216f10271eb641894f83d4",
    "content_hash": "debf5e797cd152d76b6259b05a1da99dea8ea1a7194dbeb3a887eead8b5eb349",
    "fence": "1",
    "event": {
      "execution_id": "7d75d32b130446a5",
      "kind": "EVENT_OUTPUT",
      "message": {
        "role": "assistant",
        "parts": [
          {
            "text": {
              "text": "echo:hello"
            }
          }
        ]
      }
    }
  }
}

Getting just the reply:

$ agentctl exec --server= --journal /tmp/demo.db --session "$SID" --input hi --json |
    jq -r 'select(.record.event.kind == "EVENT_OUTPUT") | [.record.event.message.parts[]?.text.text // empty] | join("")'
echo:hi

A failed stdout write, which used to exit 0:

$ agentctl exec --server= --journal /tmp/demo.db --session "$SID" --input x --json > /dev/full; echo "exit=$?"
agentctl: write exec output (the turn was not interrupted and its records are committed; read them with agentctl replay --session sess-bbbbc03934ce7881f1117a10 and the same --server or --journal): write /dev/stdout: no space left on device
exit=1

The quickstart's JSON section now includes a similar example.

Tests and scripts moved off text parsing

The tests and scripts that checked or parsed the human output of exec, replay or fork now read --json, so the next PR can change that output. What's left is display only, or relies on something the next PR keeps:

  • driver.sh's split-view highlight assumes one line per record; the next PR updates it with the layout.
  • examples/substrate-kind/smoke.sh reads the session <uid> header, which stays.
  • Checks on get, list, suspend and delete output stay as they are; those formats aren't changing.

What changed:

  • cmd/agentsessionsd/lifecycle_e2e_test.go (the daemon smoke test): turn outputs, END, the fork child and the replay prefix across delete and restart are now checked on decoded records with proto.Equal, instead of text regexes and byte prefixes.
  • cmd/agentctl/delete_test.go: the fork child comes from fork --json, and the audit replay check uses replay --json.
  • examples/agentsuite-substrate/demo.sh reads exec, replay and fork through --json, parsed with jq, which up.sh and the README already require. say still shows each turn, via replay --from. Every read consumes all its input, because a jq first(...) that stops early could end replay with SIGPIPE under pipefail.
  • examples/agentsuite-substrate/demo/driver.sh captures the journal with replay --json and parses it with python3, as it already parses its other JSON.
    • Its step log gets one seq kind line per record instead of the JSON, which holds provider data (such as opaque reasoning bytes) that its redaction cannot inspect.
    • The full JSON, and the session list it reads the fork child from, go to a private mktemp file that is deleted on exit, not to the shared work directory. Only the child's UID is logged, and the parent and child comparison keeps only content hashes.
    • Its checks now read the journal rather than the screen: the usage and hash chain (chapter 6), that the replay appended nothing (chapter 8), and the fork child's own answer and input, plus the shared prefix compared by content_hash (chapter 9).
    • The fork child's UID comes from list --json, matched by parent and name.
  • examples/agentsuite-substrate/demo/trace-turn.sh: the model-call hop reads a JSON journal kept outside the saved trace directory. The human journal.log stays for reading.
  • examples/agentsuite-substrate/scripts_test.go: the fake agentctl behind the demo.sh tests now prints JSON Lines for --json.
  • hack/demos/kubernetes-tour.sh: the tour shows SID=$(agentctl create ...) before its first exec, and gets the fork child from list --json. The landing-page cast is unchanged until it is re-recorded.

Docs and changelog

  • docs/quickstart.md: an exec row in the JSON table, a jq example, and how pauses, chunks and failures appear. It also says the human output of exec, replay and fork can change.
  • docs/approval-gated-tool.md: how a script detects the approval pause.
  • Both docs' jq examples set pipefail, so a failed agentctl fails the pipeline, not just jq.
  • The quickstart notes that exec --json needs a build from main until the next release; v0.4.0 has no exec --json.
  • spec.md §1.2 defines the exec --json framing: the first session frame, committed records, the approval-pause frame, deltas left out, and stream and write failures. It also says the human output of exec, replay and fork isn't a stable contract.
  • Changelog entries: Added (exec --json), Compatibility (human output will change; use --json), Fixed (stdout write errors).

Not in this PR

  • The compact human layout, which comes next.
  • Color, which follows after that.

Testing

  • gofmt, go build ./..., go vet ./..., golangci-lint (changed packages), go test -race -shuffle=on ./cmd/agentctl ./cmd/agentsessionsd ./client, the examples/agentsuite-substrate module's vet, tests and golangci-lint, hack/check-doc-links.py, go run ./hack/doccheck, go run ./hack/changelog check.
  • New tests:
    • TestExecJSON: lines round-trip to the sent updates, with and without --session; deltas are left out; escaped control and format characters; the approval-pause frame; a stream error.
    • TestExecWriteError: text and JSON modes.
  • Ran exec --json against the built-in server.
  • The new demo.sh, driver.sh, trace-turn.sh and tour helpers and checks were exercised against real journals from the built-in server, including fork, multi-line text and checks that must fail. I could not run demo.sh, driver.sh or the tour end to end, because they need substrate or a cluster.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Replay can abort on an empty cursor, demo tests fail on non-JSON stubs, and step logs retain opaque provider reasoning that plaintext redaction cannot inspect.

5 open issues
What changed in this PR

Adds stable machine-readable output for agentctl exec, preparing scripts for the planned redesign of human-readable CLI output.

Changes:

  • Stream session updates and committed records as protobuf JSON Lines with exec --json, omitting uncommitted model chunks.
  • Report stdout write failures after draining the turn, with tests for JSON output and write errors.
  • Migrate demo checks to JSON and document approval pauses, failure behavior and compatibility.
File Description
examples/​agentsuite-substrate/​demo/​driver.sh Parses captured replay JSON for journal checks.
examples/​agentsuite-substrate/​demo.sh Reads exec, replay and fork results through JSON.
docs/​quickstart.md Documents exec JSON, examples and output guarantees.
docs/​approval-gated-tool.md Adds a JSON approval-pause example.
cmd/​agentctl/​output.go Shares protobuf JSON encoding and updates flag help.
cmd/​agentctl/​main.go Implements exec JSON streaming and write-error tracking.
cmd/​agentctl/​main_test.go Adds JSON framing, escaping, stream-error and write-error tests.
client/​client.go Corrects the OnDelta documentation.
changelog/​unreleased/​exec-output-errors.md Records stdout write-error handling.
changelog/​unreleased/​exec-json.md Announces the exec JSON format.
changelog/​unreleased/​exec-json-compat.md Advises scripts to migrate from human output.

Review effort: Max

Comment thread examples/agentsuite-substrate/demo.sh Resolved
Comment thread cmd/agentctl/main.go Resolved
Comment thread cmd/agentctl/main.go Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread cmd/agentctl/main.go Resolved
Comment thread cmd/agentctl/main.go Resolved
Comment thread docs/approval-gated-tool.md Resolved
Comment thread docs/quickstart.md Resolved
Comment thread docs/quickstart.md Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread spec.md Resolved
Comment thread spec.md Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The Kubernetes tour can fail silently because stderr is suppressed, and the quickstart filter emits null lines for non-text parts and splits multipart message text.

2 open issues
3 resolved since last review

Review effort: Max

Comment thread hack/demos/kubernetes-tour.sh Resolved
Comment thread docs/quickstart.md Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The trace can misidentify an input event as a model hop, and unconditional temporary-file creation can block reset and teardown when file creation fails.

2 open issues
2 resolved since last review

Review effort: Max

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The documentation example discards the new session UID, preventing scripts from using the resulting approval request with approve or deny.

1 open issue
2 resolved since last review

Review effort: Max

Comment thread docs/approval-gated-tool.md Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Diagnostics can emit raw terminal control characters, and hash-chain validation incorrectly accepts journals with no hashes.

2 open issues
1 resolved since last review

Review effort: Max

Comment thread examples/agentsuite-substrate/demo.sh Resolved
exec now takes --json and prints the turn as JSON Lines of protobuf
ExecUpdate: the session frames and each committed record as it arrives.
Streamed model chunks are left out; the OUTPUT record carries the
complete response. An approval pause ends with a session frame whose
pending_approval names the call.

A failed stdout write no longer goes unnoticed. exec stops writing,
reads the stream to its end so the turn is not cut short, and exits
with an error that names the session to replay. If the stream also
fails, both errors are returned. SIGPIPE behavior is unchanged.

Tests and scripts that parsed the human views of exec, replay and fork
now read --json, so those views can change: the daemon smoke test,
delete_test, the agentsuite-substrate demo, driver and trace-turn
scripts, and the Kubernetes tour.

Refs agent-primitives#278

Signed-off-by: Ridwan Hoq <943533+ridhoq@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unescaped Unicode separators in failure messages can create misleading log lines, while the quickstart’s decoded terminal escapes can let model responses manipulate the terminal display.

2 open issues
2 resolved since last review

Review effort: Max

Comment thread docs/quickstart.md
Comment on lines +362 to +363
agentctl exec --server= --journal /tmp/qs.db --session "$SID" --input "hello again" --json |
jq -r 'select(.record.event.kind == "EVENT_OUTPUT") | [.record.event.message.parts[]?.text.text // empty] | join("")'
| [.message.parts[]?.text.text // empty] | join("") | gsub("\n"; "\\n")' |
python3 -c 'import sys, unicodedata
for line in sys.stdin:
print("".join(c if unicodedata.category(c)[0] != "C" else ("\\x%02x" if ord(c) < 128 else "\\u%04x") % ord(c)

@aramase Anish Ramasekar (aramase) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ridwan Hoq (@ridhoq) exec --json works as spec.md describes. I ran the built binaries embedded and remote: the session frame comes first, then committed records, with a final session frame only on park. Deltas are left out, text output is byte-identical to main, and a failed stdout write now exits 1 after draining. The new tests catch every mutation of the stream logic I tried except dropping INPUT.

Two doc changes before this goes in, both small: the SIGPIPE caveat also applies to remote turns, and the quickstart's jq -r filter prints raw model text to the terminal. Details inline; the rest are nits.

Comment thread spec.md
are omitted; readers ignore update kinds they do not expect. A stream error leaves the lines already
written and exits unsuccessfully, without retrying the turn. A failed stdout write ends the command
unsuccessfully after the stream is read to its end, except that on Unix a closed pipe still ends
`exec` with SIGPIPE, which can cut an embedded turn short; readers consume the whole stream.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This applies to remote turns too, not only embedded. agentctl dies on SIGPIPE, which closes the Exec stream, and the server runs the turn on stream.Context() (session/service.go:630, 844). With a 3 s streaming fake model, agentctl exec --server $S --input hi --json | head -1 left EXECUTION_START, INPUT, MODEL_CALL and then EVENT_ERROR ... context canceled, with no END. --session S ... | true exited 141 before the turn ran. main's text mode behaves the same, so this isn't a regression, but the sentence reads as if remote callers are safe. Could it drop "embedded", or should exec --json ignore SIGPIPE so EPIPE takes the drain path like other write errors? Either works if the docs match.

Comment thread docs/quickstart.md
has already resolved it. Only a recorded `EVENT_END` shows that a turn completed. Ignore update
kinds you do not expect, since later releases may add some. If the stream fails, the lines already
written stay and the command exits with an error; it does not retry the turn. Read the stream to its
end: on Unix, closing the pipe early ends `exec` with SIGPIPE, which can cut an embedded turn short.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as spec.md:100: closing the pipe early cancels a remote turn as well. The | head -1 | jq -r .session.metadata.uid pattern this section suggests for learning the UID is exactly the case that hits it.

Comment thread docs/quickstart.md
agentctl list --server= --journal /tmp/qs.db --json | jq -r '.sessions[]? | .metadata.uid'
agentctl replay --server= --journal /tmp/qs.db --session "$SID" --json > records.jsonl
agentctl exec --server= --journal /tmp/qs.db --session "$SID" --input "hello again" --json |
jq -r 'select(.record.event.kind == "EVENT_OUTPUT") | [.record.event.message.parts[]?.text.text // empty] | join("")'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jq -r decodes the JSON escapes, so model text reaches the terminal raw. With a reply containing \u001b[8m, this filter wrote an ESC byte, where the human view prints \x1b (main.go:691-723). The echo example is fixed text, but people will copy this for a model-backed session. Could the example print JSON-quoted text (jq -c, or @json), or add a one-line note that -r is for trusted consumers only? This is Copilot's open thread here as well.

| [.message.parts[]?.text.text // empty] | join("") | gsub("\n"; "\\n")' |
python3 -c 'import sys, unicodedata
for line in sys.stdin:
print("".join(c if unicodedata.category(c)[0] != "C" else ("\\x%02x" if ord(c) < 128 else "\\u%04x") % ord(c)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this escapes category C only, but agentctl's needsEscape (main.go:694) also escapes Zl and Zp, so U+2028 and U+2029 pass through raw here. Escaping those too would make the comment above accurate.

Comment thread cmd/agentctl/main_test.go
return &v1.ExecUpdate{Update: &v1.ExecUpdate_Session{Session: s}}
}
first := sessionUpdate(&v1.Session{Metadata: &v1.ResourceMetadata{Uid: "sess-json"}})
output := &v1.ExecUpdate{Update: &v1.ExecUpdate_Record{Record: &v1.LogRecord{Seq: 2, Event: &v1.Event{

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the stream has OUTPUT but no INPUT, so a mutation that drops only INPUT passes both the agentctl and agentsessionsd tests. An INPUT frame in the supplied and expected stream would pin "every committed record". Separately, nothing covers text-mode exec with --session: setting newSession: true at main.go:356 passes every test. A row asserting no session line would cover it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants