Run a durable agent session end to end in a few minutes: exec a turn, reconstruct it by replay with
zero model calls, fork it, suspend and resume it, and verify its provenance chain with a non-Go
verifier. Everything here uses the built-in echoagent harness and a local sqlite journal, so no
Kubernetes, no cloud, and no model key are required. Section 8 points it at a real model.
The command output below is real, captured from a live run.
- Nothing, to follow sections 1 through 7. Go 1.26 or newer only if you build from source.
- Optional, for the independent chain verifier:
buf,python3, andsqlite3.
agentctl is the client CLI; agentsessionsd is the standalone server used by the server example
in section 6 and from section 8 onwards. This walkthrough tracks main, so install both from it.
Both options below give binaries that run every step.
# Go toolchain
go install github.com/agent-primitives/sessions/cmd/agentctl@main
go install github.com/agent-primitives/sessions/cmd/agentsessionsd@mainTo build from a checkout instead:
git clone https://github.com/agent-primitives/sessions
cd sessions
go build -o agentctl ./cmd/agentctl
go build -o agentsessionsd ./cmd/agentsessionsdRelease archives and container images are published only for tagged releases. The release pinned below, v0.4.0, runs every section as shown.
# Release archive: both binaries, for linux/darwin on amd64/arm64.
# Checksums and SBOMs are published alongside it.
VERSION=v0.4.0
OS=$(uname -s | tr '[:upper:]' '[:lower:]')
ARCH=$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/')
curl -sSL "https://github.com/agent-primitives/sessions/releases/download/${VERSION}/agentsessions_${VERSION#v}_${OS}_${ARCH}.tar.gz" \
| tar xz agentctl agentsessionsdContainer images are published for the server and the harness node:
docker run --rm ghcr.io/agent-primitives/sessions/agentsessionsd:v0.4.0 --helpThe images are multi-arch, built from the tagged source with no Dockerfile. agentsessionsd is the
Sessions server; harnessnode is the in-sandbox harness the substrate manifests in
deploy/substrate/ reference. The server runs as uid 65532, so mount its journal path writable.
The examples below assume both binaries are on your PATH; if you built them into the working
directory, prefix the commands with ./.
agentctl starts an embedded Sessions gRPC server over a local unix socket, backed by a sqlite
journal on disk, and drives it as a real gRPC client. Pass --server <addr> to drive a remote
controller instead. The journal path defaults to agentsessions.db; the examples use
--journal /tmp/qs.db to keep it out of the way.
The embedded server logs only warnings and errors to stderr, keeping routine INFO diagnostics out
of CLI output. Standalone agentsessionsd logging is unchanged.
agentctl exec --journal /tmp/qs.db --input "hello world"session sess-a632fd215b69792f5eb8b359
seq=1 EVENT_EXECUTION_START
seq=2 EVENT_INPUT user: hello world
seq=3 EVENT_MODEL_CALL
seq=4 EVENT_OUTPUT assistant: echo:hello world
seq=5 EVENT_END
The first line is the new session UID. The five events are the durable, typed log of that turn: the
host's EXECUTION_START marker, the input, the mediated model call, the output, and the terminal
END. echoagent is a trivial harness whose "model" returns the input prefixed with echo:, so
the loop is real without a provider key.
Keep the UID; the next steps use it.
SID=sess-a632fd215b69792f5eb8b359 # replace with your UIDPass --session to add a turn to the same session. The sequence numbers keep climbing in the one log.
agentctl exec --journal /tmp/qs.db --session "$SID" --input "how are you"seq=6 EVENT_EXECUTION_START
seq=7 EVENT_INPUT user: how are you
seq=8 EVENT_MODEL_CALL
seq=9 EVENT_OUTPUT assistant: echo:how are you
seq=10 EVENT_END
This is the property that makes the log the source of truth. replay re-delivers the committed log.
On replay the controller serves each recorded model result from the journal and never calls the model
(invariant I1), so the reconstruction is byte-identical and free.
agentctl replay --journal /tmp/qs.db --session "$SID"seq=1 EVENT_EXECUTION_START
seq=2 EVENT_INPUT user: hello world
seq=3 EVENT_MODEL_CALL
seq=4 EVENT_OUTPUT assistant: echo:hello world
seq=5 EVENT_END
seq=6 EVENT_EXECUTION_START
seq=7 EVENT_INPUT user: how are you
seq=8 EVENT_MODEL_CALL
seq=9 EVENT_OUTPUT assistant: echo:how are you
seq=10 EVENT_END
The same durable log is what a different process reconstructs after the original one dies. For the
Kubernetes version of this (exec on one pod, delete the pod, a fresh pod replays the journal), see
hack/demo.sh.
fork --at N creates a child session that inherits the parent's log up to seq N, then evolves
independently. echoagent is a stateless-replay harness, so the child is rebuilt by replay and the
parent is untouched. (A REQUIRES_MEMORY_SNAPSHOT harness checkpoints the parent instead — see the
FAQ.) Fork at a turn's END, seq 5 here for the first turn: a
child forked inside a turn inherits it unfinished.
agentctl fork --journal /tmp/qs.db --session "$SID" --at 5child sess-0b8ad2edf255de2cb659f7e6 parent=sess-a632fd215b69792f5eb8b359 fork_seq=5 name=
A child is unnamed unless you name it. It inherits the parent's project, harness and model, but not
its label, so a listing does not fill up with rows that all read the same thing. Use --names to
label a fan-out, one comma-separated name per child:
agentctl fork --journal /tmp/qs.db --session "$SID" --at 5 --count 2 --names "optimistic,pessimistic"Replay the child: it carries the parent's first five events, plus a LIFECYCLE record marking the
fork.
CHILD=sess-0b8ad2edf255de2cb659f7e6 # replace with your child UID
agentctl replay --journal /tmp/qs.db --session "$CHILD"seq=1 EVENT_EXECUTION_START
seq=2 EVENT_INPUT user: hello world
seq=3 EVENT_MODEL_CALL
seq=4 EVENT_OUTPUT assistant: echo:hello world
seq=5 EVENT_END
seq=6 EVENT_LIFECYCLE
Now exec on the child. It diverges from the parent while the parent still ends at seq 10.
agentctl exec --journal /tmp/qs.db --session "$CHILD" --input "child-only turn"seq=7 EVENT_EXECUTION_START
seq=8 EVENT_INPUT user: child-only turn
seq=9 EVENT_MODEL_CALL
seq=10 EVENT_OUTPUT assistant: echo:child-only turn
seq=11 EVENT_END
Suspend frees compute and records the transition in the log; resume brings the session back. On the local filesystem-only backend, resume reconstructs the session by replay (no memory snapshot). On a memory-capable backend the same commands restore RAM instead. The CLI is identical either way.
Use a fresh session so the sequence numbers are easy to follow. Take its UID from create, not by
piping exec into head -1: exec streams records as they commit, and a reader that exits early
stops it mid-turn with SIGPIPE.
SID2=$(agentctl create --journal /tmp/sr.db)
agentctl exec --journal /tmp/sr.db --session "$SID2" --input "before suspend"
agentctl suspend --journal /tmp/sr.db --session "$SID2"
agentctl resume --journal /tmp/sr.db --session "$SID2"seq=1 EVENT_EXECUTION_START
seq=2 EVENT_INPUT user: before suspend
seq=3 EVENT_MODEL_CALL
seq=4 EVENT_OUTPUT assistant: echo:before suspend
seq=5 EVENT_END
session sess-... suspend execution=COMPLETED compute=COLD last_seq=6
session sess-... resume execution=COMPLETED compute=LIVE last_seq=7
The first turn is seq 1 to 5, so SUSPEND lands at seq 6 and RESUME at seq 7. Both are
LIFECYCLE events in the log, so the compute history is auditable alongside the conversation.
create registers a session without running anything, and list enumerates a project. Every
agentctl invocation opens the journal fresh, so the listing below is already proving the
restart case: nothing is held in memory between commands.
agentctl create --journal /tmp/qs.db --project acme --name triage --model echo-1
agentctl list --journal /tmp/qs.db --project acmesess-0d879482...
SESSION NAME EXECUTION COMPUTE HARNESS LAST SEQ
sess-0d879482... triage PENDING NONE echo 0
LAST SEQ being 0 is the point: the session has no events and no compute yet, and it still appears. That
is the case a listing built from the event log alone would miss.
Sessions are returned newest first and filtered on an exact --project. list walks every page for
you; --page-size only changes how many are fetched per request.
agentctl list --journal /tmp/sr.db --project defaultSESSION NAME EXECUTION COMPUTE HARNESS LAST SEQ
sess-... "" COMPLETED LIVE echo 7
compute reflects what the log implies about compute, which is why the resumed session above reads
LIVE. It is not a live probe of the backend; see
concepts.md for what that does and does not guarantee.
Execution is the independent execution axis. get and list show PENDING before any execution,
RUNNING for unfinished work, COMPLETED after END, FAILED after ERROR, or AWAITING for a valid
pending approval. RUNNING is not proof that a controller is alive.
For a quick manual check, create a session, inspect it, and execute a turn:
CHECK=$(agentctl create --journal /tmp/exec-state-check.db)
agentctl get --journal /tmp/exec-state-check.db --session "$CHECK" # Execution: PENDING
agentctl exec --journal /tmp/exec-state-check.db --session "$CHECK" --input "hello"
agentctl get --journal /tmp/exec-state-check.db --session "$CHECK" # Execution: COMPLETED
agentctl list --journal /tmp/exec-state-check.db # EXECUTION: COMPLETEDEach command reopens the same database, so this also checks persistence. With a standalone server,
replace --journal ... on the client commands with --server 127.0.0.1:8080.
This subsection needs agentctl v0.4.0 or later (see Install). v0.3.0 ignores
AGENTSESSIONS_SERVER and silently uses a local agentsessions.db; with that release, pass
--server 127.0.0.1:8080 on each command of the example below instead.
Session commands use AGENTSESSIONS_SERVER when --server is omitted. An explicit flag wins,
including --server= to select the embedded, echo-only server. An unset or empty environment
variable also selects embedded mode. A remote connection failure is an error, not a fallback to
embedded mode. Harness registry commands still require their own explicit operator --server;
they never inherit the session-server environment default. When the target comes from the
environment, the CLI prints it to stderr before making requests, including for --json.
An explicit --journal with a remote target is an error rather than an ignored flag; omit it
for remote commands, or pass --server= to use the local journal.
To try it, start a standalone server in another terminal. It needs no model key; section 8 covers the model flags. Stop it before section 8, which starts its own on the same port.
agentsessionsd --addr 127.0.0.1:8080 --journal /tmp/server.dbThen run the client commands in a subshell, so the variable does not reach later sections:
(
export AGENTSESSIONS_SERVER=127.0.0.1:8080
TRIAGE=$(agentctl create --name triage)
agentctl exec --session "$TRIAGE" --input "hello"
agentctl list
agentctl get --session "$TRIAGE"
)get prints labeled details, including the independent execution, compute and deletion states:
Session sess-...
Name triage
Project default
Harness echo
Model
Execution COMPLETED
Compute LIVE
Last seq 5
Deletion ACTIVE
Deletion requested unset
Human state values omit the enum prefix everywhere (for example, LIVE, not COMPUTE_LIVE).
Unspecified values print UNSPECIFIED, not a guessed state. Names and other text are escaped so
they cannot insert terminal controls or extra rows. Tables quote names containing spaces and show
empty names as "".
Column padding is capped at 40 characters. Longer values are printed in full but can shift
subsequent columns without expanding the other rows.
Table alignment assumes single-width characters; wide characters and combining marks can shift
columns. Use --json when character widths matter to a consumer.
For agents and scripts, use --json rather than parsing the human views:
| Commands | JSON response |
|---|---|
create, get, suspend, resume, delete |
One protobuf Session |
list |
One ListSessionsResponse, combining all pages with no next-page token |
fork |
One ForkResponse containing children |
approve, deny |
One ApproveResponse containing the original committed decision and resumed session |
replay |
JSON Lines: one complete protobuf LogRecord per line, in sequence order |
All CLI JSON uses protobuf's snake_case field names, full enum names (for example,
COMPUTE_LIVE), decimal strings for 64-bit integers and RFC3339 timestamps. Default-valued fields
and absent messages are omitted; an empty list response is {}. Whitespace and object-key order
are not a stable contract: parse the content rather than comparing bytes. The field naming
matches CHAIN-1, but CLI JSON is not the RFC 8785 canonical byte representation used for hashing.
Harness commands retain their existing JSON format.
agentctl get --server= --journal /tmp/qs.db --session "$SID" --json | jq '.compute_state'
agentctl list --server= --journal /tmp/qs.db --json | jq -r '.sessions[]? | .metadata.uid'
agentctl replay --server= --journal /tmp/qs.db --session "$SID" --json > records.jsonlReplay JSON retains full hashes, sequence numbers and protobuf field presence; --json takes
precedence over -v. It only reads recorded events, never calls a model, and does not itself verify
the chain; use agentctl verify for verification. An empty replay writes no lines. Treat output
as successful only when the command exits successfully.
Human output is uncolored; color support is deferred. Diagnostics and the deletion PENDING warning
go to stderr, including in JSON mode, so stdout remains parseable.
On Unix, unary response commands report broken stdout pipes as output errors on stderr rather
than exiting via SIGPIPE, noting that successful mutations remain committed. exec retains its
existing streaming signal behavior.
Migration: get no longer emits a single key/value line; list now has aligned columns and
headings, even when empty. Lifecycle and approval summaries use short state values and include
execution state. Switch parsers to --json. Approval JSON changes from lowerCamelCase to snake_case:
for example, decision.contentHash becomes decision.content_hash, and session.lastSeq becomes
session.last_seq. create still prints only the UID by default; fork retains its existing
default summary. Deletion still warns that PENDING is durable
intent, not resource cleanup. Approval still reports a committed decision if Resume fails; JSON
does not turn that failure into a successful response. exec output is unchanged and has no
--json flag.
The hash chain is defined on the wire proto (RFC 8785 JCS over proto3-JSON), not on Go, so any
implementation can verify it. hack/verify_chain.sh records a real journal, then runs a small Python
verifier that reproduces the Go hash, audits the full chain, and confirms a tampered record fails
closed.
This one is a script in the repository rather than a shipped binary, so it needs a checkout:
git clone https://github.com/agent-primitives/sessions
cd sessions
./hack/verify_chain.shIt reproduces a golden hash with no dependencies, audits a real chain from raw proto bytes (must pass), and tampers one record and re-audits (must fail). This is the "any auditor can verify the chain" property, demonstrated.
Everything above uses the built-in echo model, so it runs with no key. To drive a real one, serve
the Sessions API with agentsessionsd and give it an OpenAI-compatible endpoint.
export OPENAI_API_KEY=sk-...
agentsessionsd \
--addr 127.0.0.1:8080 \
--journal /tmp/real.db \
--model gpt-4o-miniThe key is read from the environment, never a flag, so it stays out of your shell history and the
process list. MODEL_API_KEY works too, and an endpoint that needs no credential works with both
unset.
Configuring a model also registers a second harness, chat, alongside echo. echo stays the
default, so ask for the conversational one when you create the session:
CHAT=$(agentctl create --server 127.0.0.1:8080 --harness chat)
agentctl exec --server 127.0.0.1:8080 --session "$CHAT" --input "name three primes"The server logs which harnesses it registered at startup (harnesses="[chat echo]").
--model-base-url points at any endpoint that accepts the chat-completions request body: a
gateway, a self-hosted server, or a proxy fronting another provider. With a key set, the server
refuses to start on a plain http URL unless its host is a loopback address, localhost, or a
name ending in .svc or .svc.cluster.local. That last allowance is a trusted-network exception:
it checks the name, not where the traffic goes, so the key and prompts still cross the cluster
network unencrypted. Use https unless you trust that network. Endpoints that differ only in the
envelope are reached with two more flags, rather than by rebuilding the server:
agentsessionsd \
--model gpt-4o \
--model-base-url https://example.com/openai/deployments/gpt-4o \
--model-path '/chat/completions?api-version=2024-10-21' \
--model-auth-header api-key--model-path covers an endpoint that scopes the model into its path or pins an API version;
--model-auth-header covers one that authenticates with something other than Authorization. An
endpoint with its own request body or signing scheme is a different protocol: implement
controller.ModelFunc for those, and every guarantee below still holds, because the host still
owns the call.
Then drive it with the same CLI, using --server instead of a local journal:
agentctl exec --server 127.0.0.1:8080 --input "name three prime numbers"The interesting part is what happens next. Replay the chat session and watch your provider dashboard:
agentctl replay --server 127.0.0.1:8080 --session "$CHAT"The turn comes back byte-for-byte, and the provider is not called. Same for fork: branching a
session costs nothing at the model, because the children inherit the recorded completions. You pay
the model once, for the live turn, and every later reconstruction is free.
That is the whole point of host-mediated model calls. Because the completion flows through the host and lands in the journal, the log is a replayable record of the run rather than a description of one.
agentctl is a thin shell over the same client package your code can use. One call runs a turn,
creating the session if you did not name one:
c, err := client.Dial("127.0.0.1:8080", client.WithProject("acme"))
if err != nil {
return err
}
defer c.Close()
turn, err := c.Exec(ctx, client.ExecOptions{Inputs: []string{"name three primes"}})
if err != nil {
return err
}
fmt.Println(turn.Session.GetMetadata().GetUid(), turn.Output)Output streams as the model produces it. OnDelta receives ephemeral chunks; OnRecord receives
committed records:
_, err = c.Exec(ctx, client.ExecOptions{
Inputs: []string{"write a haiku"},
OnDelta: func(d *v1.Delta) { fmt.Print(d.GetChunk()) },
})Deltas are transport only. They are never logged, never hash-chained, and never produced on replay,
so the journal is the same whether or not anyone watched the turn. Streaming works because the
host mediates the model call: the harness blocks on one sink.Model and never learns that
anything streamed.
turn.LastSeq is the cursor for the next turn, so opting into the single-writer check costs no
extra round trip:
next, err := c.Exec(ctx, client.ExecOptions{
Session: turn.Session.GetMetadata().GetUid(),
Inputs: []string{"and three more"},
ExpectedLastSeq: &turn.LastSeq, // stale cursor -> codes.Aborted
})ListSessions walks pagination for you, Replay and Fork return slices, and c.Sessions()
returns the generated stub for anything the SDK does not model.
DeleteSession needs v0.2.0 or later; v0.1.x binaries do not have it.
Use a fresh journal here, leaving /tmp/qs.db from the earlier examples untouched.
Deletion is irreversible for the UID. To upgrade a journal written by v0.1.x, see the
schema v4 rollout note first.
DELETE_JOURNAL=$(mktemp /tmp/qs-delete.XXXXXX)
DELETE_SID=$(agentctl create --journal "$DELETE_JOURNAL")
agentctl exec --journal "$DELETE_JOURNAL" --session "$DELETE_SID" --input "audit before deletion"
agentctl delete --journal "$DELETE_JOURNAL" --session "$DELETE_SID"
agentctl get --journal "$DELETE_JOURNAL" --session "$DELETE_SID"
agentctl replay --journal "$DELETE_JOURNAL" --session "$DELETE_SID"
agentctl list --journal "$DELETE_JOURNAL" --project defaultDelete/Get report deletion state PENDING and the committed RFC3339 request time. PENDING means durable
intent only: the SQLite allocation-attempt ledger is incomplete, not authoritative cleanup inventory.
Until #88 supplies authoritative cleanup inventory and reclamation, actors, snapshots and fork-child
pins remain retained indefinitely. No Stop or reclamation is attempted. List hides the UID, but Get and raw
Replay retain metadata and the full audit chain, including DELETE. Retry returns the original
timestamps; use Get/retry after a lost response. Exec, historical Fork, Resume, Suspend and Approve
(including exact retries) now return FAILED_PRECONDITION. Existing children remain usable. There
is no cascade, UID reuse or data erasure; UID, metadata and journal remain forever. See deletion
and admission limits before sharing a journal between hosts or
upgrading its schema.
The SDK returns the raw Session; inspect its deletion state rather than interpreting OK as cleanup:
deleted, err := c.DeleteSession(ctx, turn.Session.GetMetadata().GetUid())
if err != nil {
return err
}
fmt.Println(deleted.GetDeletionState(), deleted.GetDeletionRequestedTime())concepts.md: the nouns and why each exists.harness-authoring.md: write your own harness against theapi.HarnessSPI.architecture.md: how the core is built.substrate-conformance.md: running on real agent-substrate across both capability tiers.faq.md: common questions.