Repository navigation
Project server: one supervisor per project, many agents #103
Description
Proposal
Revision 2026-10-25 (opencode 2 pivot): upstream ≥ v2 moved this direction already —
opencode service runs a server independent of the TUI. The build-out below is now
verify-first, gap-fill second (charter §7 status note): the §7.6 ten lifecycle
assertions are driven against opencode service (task owned by
adopt-upstream-acp-and-service 1.1) before any of the "What Changes" build items are
implemented; items upstream already covers are dropped from this change with the
evidence row cited, and this change keeps only what the verification shows missing.
The DurableAgentExecution record (§7.4 amendment) remains the contract for
plugin-owned durable loop state (port-loop-crew-plugin), independent of who owns
the server.
Parent: autonomous-swarm (foundation; replaces the "depend on Go skein" assumption).
Why
skein's pattern was one supervisor, many runners. opencode-skein inverted it: every
opencode launch starts its own server inside the process (cli/cmd/tui.ts spawns a
worker per TUI; opencode serve exists but is opt-in and nothing discovers it). The
consequences are the root of several swarm problems:
- All coordination state is per process and in memory: the loop registry (
loop.ts
Ref<Map>, not persisted),PeerInbox, pending reply correlation, the repeat guard,
placement slot reservations,recentPlacements. Two sessions in two processes cannot see
each other's state, socrew-loophas to bolt on a database table, a "one queue per
directory" refusal, and a foreign-turn guard. - A loop dies when its TUI closes. Agents are tied to terminal windows, which is the
"one iTerm session per agent" cost. - A2A between opencode sessions needs sockets, a registry directory and a sidecar per
session even when both live on the same machine and project. - There is nowhere to run a coordinator that is not itself somebody's TUI.
Already true and reusable: the server loads project instances per request via the
x-opencode-directory header, so one server can serve every worktree of a repo;
opencode attach <url> [--session] already connects a TUI to a running server;
opencode run --attach does the same for CLI runs.
What Changes
- One server per repository (identity = git common dir, so every worktree shares it),
discovered through a registry record<state>/servers/<repo-key>.json
{url, pid, startedAt, authFile}; liveness = pid alive + health ping. Bound to loopback
or a unix socket, with a generated password in a 0600 file. Never unsecured. - Auto-attach.
opencodein a repo looks up the record; healthy → attach as a
client; absent or dead → start a detachedopencode serveand attach. A TUI close
detaches; it does not stop the server or the agents in it. Opt-out--standalone
(today's behaviour). Ships behindexperimental.project_server. - Agents are sessions + loops owned by the server, not processes. An agent can be
observed or driven from a TUI client, a CLI (opencode run --attach), or by no client
at all (headless).opencode agent spawn --role <persona> [--change <slug>]creates a
worktree-backed session with a loop in the server;opencode agent list|attach|stop.
The human can attach a TUI to any agent at any time — "talk to every individual"
remains, without a window per agent. - Durable loops. Loop records move from the in-memory
Refto the shared database
so a server restart resumes running agents instead of losing them (precondition for
trusting a long-lived server). - In-server A2A short-circuit. Sessions in the same server exchange messages
through the server's own inbox and status (no socket), keeping the envelope and the
trust framing identical. Sidecars stay for Claude Code and cross-repo peers, so
nothing inpeer-conversation-reliabilityis thrown away. - Cross-instance aggregation. Session status is
InstanceState, i.e. per directory,
not per process. The server exposes one aggregated view over all worktree instances
(the roster the coordinator reads). - Supervision of the supervisor. Crash containment: heavy work stays in child
processes (shell tools already are); a watchdog restarts a wedged server; the first
client to notice a dead record respawns it; recovery replays durable loops. State that
matters lives in the database and files, never only in the server's memory (the old
skein post-mortem's failure).
Non-goals
- No machine-wide singleton; the unit is the repo. Cross-repo coordination stays on A2A
sockets. - No removal of standalone mode or of the sidecar/Claude Code path.
- No new wire protocol: the existing HTTP API, SSE events and
attach.
Risks
- Single point of failure. One server crash stops every agent in the repo. Mitigation:
durable loops, restart on death,--standaloneescape hatch, a soak with a deliberate
kill (Phase 5). - Per-instance assumptions. Code that treats module-level or
InstanceStatemaps as
"the process" may mis-scope when many worktrees share a server. Mitigation: audit list
indesign.md; cross-worktree tests. - Security. A long-lived local server is an attack surface. Mitigation: loopback or
unix socket only, mandatory password, same-user file permissions. - Memory growth in a long-lived process (a known concern:
long-session-memory-retention
change). Mitigation: that change becomes a dependency for default-on.
Impact
cli/cmd/tui.ts, cli/cmd/serve.ts, cli/cmd/attach.ts, cli/cmd/run.ts, new
server/registry.ts and cli/cmd/agent.ts, loop/loop.ts (persistence), peer/*
(in-server delivery), session/status.ts (aggregation), tests.
Design notes
Design: project server
Topology
human Claude Code lead / peers
┌──────┴───────┐ │ (sidecar sockets, unchanged)
TUI client CLI client │
│ attach │ run --attach │
└──────┬────────┘ │
▼ ▼
┌────────────────────── project server (one per repo) ──────────────────────┐
│ instances: <repo>, <repo-worktree-A>, <repo-worktree-B> … (x-opencode-dir) │
│ agents = session + loop (headless unless a client attaches) │
│ in-server A2A inbox/status coordinator (design: swarm-coordinator) │
└───────────────┬────────────────────────────────────────────────────────────┘
▼ shared SQLite (sessions, durable loops, claims, grants) + files
The server is the supervisor; agents are sessions it owns; clients are optional views.
D1. Identity and discovery
repo-key = hash(realpath(git common dir)). Registry <state>/servers/<repo-key>.json:
{version, url, pid, procStart, startedAt, authFile}. A client is valid only if pid is
alive, procStart matches (guards pid reuse, same trick as the sidecar registry) and
GET /health answers. Anything else is a stale record, removed by the client that finds it.
Start race: creating the record uses an exclusive-create lock file; the loser attaches to
the winner.
Not in a git repo: fall back to the directory itself as the key.
D2. Auto-attach flow
opencode [dir]:
key = repoKey(dir)
rec = readRecord(key)
if rec healthy: attach(rec, dir)
else: spawn detached `opencode serve --repo-key key` ; wait for record; attach
Detached means the server is not a child of the TUI (own session/process group), so closing
the terminal does not kill agents. opencode server stop|status|restart for the human.
Version skew: record carries the build id; a client with a different build refuses to attach
silently and offers --standalone or server restart.
D3. Agents without processes
An agent is {sessionID, directory(worktree), persona/model, loop record, claim}.
agent spawn = create worktree via the existing Worktree service → create session in that
instance → start a loop (queue mode for a slug, or prompt). No new OS process. The
peers/roster view labels it headless until a client attaches. agent attach <id> is
opencode attach <url> --session <id> --dir <worktree>.
Capacity model unchanged: an agent still needs an inference slot; the project server just
sees all of them in one place, so the in-memory slot reservations stop being per-process lies
(for agents in this server).
D4. Durable loops
Loop records written to the shared DB on every patch; on server start, records in
running/paused/idle-watch are re-adopted: iteration continues from the recorded state
with a fresh child session (the design already runs each iteration in a child session, so
there is no in-flight turn to recover). A turn that was mid-flight at the crash is
abandoned, and its claim heartbeat lapses and is resumed per crew-loop abandoned-claim
rules. Precondition for default-on.
D5. In-server A2A
send_peer_message resolves the target first against the server's own sessions. Hit →
deliver through the in-process path (PeerInbox/SessionPrompt), same envelope, same
formatPeerMessage framing, same repeat guard. Miss → today's sidecar route. Sender identity
for in-server delivery is the session id the server itself attributes (not self-asserted),
which is strictly stronger than the socket check lead-authority uses for foreign peers. The
inbox and reply correlation maps stop being per-process for these sessions.
D6. Aggregation across worktrees
Session status/Loop/Inbox state is scoped by InstanceState (directory). The server adds a
repo-level service Roster that walks live instances and returns one list
{sessionID, directory, branch, status, loop, claim, headless}. The coordinator and peers
read only this. Audit list (each must be checked and tested with two worktrees in one server):
session/status.ts, peer/inbox.ts, peer/envelope.ts pendingReplies,
peer/repeat-guard.ts instances, peer/delegate.ts pending, local/placement.ts
reservations and recentPlacements, loop/loop.ts registry, worktree/*.
D7. Failure model
| failure | effect | recovery |
|---|---|---|
| TUI client closes | none for agents | reattach |
| server process dies | agents stop | next client respawns (D2); durable loops re-adopted (D4) |
| server wedged (event loop stalled) | agents hang | watchdog heartbeat file; client or launchd restarts |
| one agent runaway (tool output, memory) | risk to all | tools already run as child processes; per-session limits; kill agent not server |
| port/record stale | attach fails | stale record removal (D1) |
D8. Security
Loopback or unix socket only. Password mandatory for the auto-started server (random, in a
0600 authFile the client reads); serve run by hand keeps today's warning. The server
refuses non-loopback binds without an explicit flag. Peer authority from lead-authority is
unaffected: grants are verified, never trusted from the transport.
Open questions
- Scope when the operator runs sessions in several unrelated repos: one server each is
accepted; is there demand for a machine-wide "overseer" that talks to all project servers
(that is the cross-repo lead — today a Claude Code session over A2A)? - Should the default flip on, and when? Gate: soak with deliberate kill, memory retention
change landed, and no regression of the 3-process crew spike.
Tasks
Phase 0: Spikes (answers decide the rest)
- 0.1 (works; shared projectID — see findings.md) Two worktrees of this repo in ONE
opencode serveviax-opencode-directory: create
a session in each, run a prompt in each concurrently. Record what breaks. - 0.2 (NOT TESTED: needs a terminal)
opencode attach <url> --session <id>against a session whose loop is running with
no other client: does the TUI show live loop state; does closing the TUI leave the loop
running (expected yes, the loop lives in the server)? - 0.3 (sessions survive, loops do not — findings.md) Kill the server mid-loop. Record exactly what is lost (loop registry is an in-memory
Ref; expect: everything). This is the baseline D4 must beat. - 0.4 (only per-directory InstanceState needs aggregating — findings.md) Audit the D6 list: for each map, state its scope (process / instance / session) and
whether two worktrees in one server would collide or fail to see each other. - 0.5 (yes, at creation; one ~172 MB sidecar process each — findings.md) Do headless sessions register sidecars and appear in
peersfor Claude Code peers? - [~] 0.6 (partial: single-run numbers in findings.md; the 4-hour run is not done) Memory over a 4-hour, 3-agent run in one server vs three TUIs. Baseline for the
overhead argument.
Phase 1: Registry and auto-attach (behind experimental.project_server)
- 1.1
server/registry.ts: repo key, record schema (closed, validated at read), stale
detection by pid + procStart + health, exclusive-create start lock. Observed-red tests:
dead pid, reused pid, wrong schema type, concurrent start. - 1.2 Detached spawn with mandatory generated password in 0600 file; loopback/unix only;
refuse non-loopback. - 1.3
opencodeandopencode runauto-attach per D2;--standalone;server stop|status|restart; build-id mismatch handling. - 1.4 TUI close detaches without stopping the server (verify against 0.2).
Phase 2: Durable loops
- 2.1 Loop record table and write-through on every patch.
- 2.2 Re-adopt on server start; fresh child session per iteration; abandoned-turn handling.
- 2.3 Test: kill -9 the server mid-iteration; restart; loop continues; no duplicate
iteration; claim heartbeat lapses then resumes. Seen red against current code first. - 2.4 DurableAgentExecution record (charter §7.4): table + write-through on every
transition (executionId, sessionId, projectId, workItemId, status, currentStep,
attempt, heartbeatAt, leaseExpiresAt, updatedAt). Recovery reads this record plus the
actual repository state — never the chat transcript — and does not blindly rerun a
non-idempotent operation whose outcome is unknown.
Phase 3: Agents without processes
- 3.1
agent spawn|list|attach|stopover worktree + session + loop. - 3.2
headlessflag in the roster; reattach shows full history. - 3.3 Cleanup: stopping an agent releases claim and optionally the worktree.
Phase 4: In-server A2A and aggregation
-
4.0 (from the 0.5/0.6 findings) Host the per-session unix sockets inside the server, or in one multiplexing sidecar, instead of one 200 MB child process per registered session; (sessions already register at creation.)
-
4.1
Rosterservice across instances (D6) with tests over two worktrees. -
4.2 In-server delivery path in
send_peer_messageandpeers; sender identity
server-attributed; same framing and repeat guard. -
4.3 Fix each D6-audit finding that mis-scopes (one commit per map, each with a test).
-
4.4 Relax the "one queue loop per directory" refusal for crew mode, in coordination with
crew-loop1.3 (shared claims make it safe).
Phase 5: Supervision and soak
- 5.1 Watchdog heartbeat and client-side respawn.
- 5.2 Soak: three agents, deliberate kill of the server every 30 minutes for 4 hours;
assert no lost work and no duplicated commits. - 5.3 Memory comparison against 0.6; decide whether
long-session-memory-retentionmust
land before default-on. - 5.4 Decide default-on for this fork.
Plan changes
4 done · 1 dropped
Activity