Skip to content

Make running opencode instances visible to each other and to the human #25

Closed

Description

@androidand
Proposal

Why

Nothing watches the Agents. The system has good provider telemetry and zero client
telemetry, and the gap is not theoretical — it cost 18 hours on 2026-07-25.

Session ses_0691e2d30ffe1mwU1XPH5gr2mQ sat wedged for 18h48m on a half-open socket.
Throughout, every health signal was green: z4 reported in_flight: 0, busy: false
because z4 was fine — it had finished and moved on. The provider was healthy; the
client was dead. skein's get_providers_status polls providers, so it could not have
caught this, and did not.

Meanwhile four opencode processes were running concurrently on one machine across three
different projects, sharing one 17.6 GB SQLite file, and the human could not answer basic
questions about their own Agents: which instances are running, what is each working on, is
any of them stuck, which one holds the session I care about. Answering it required ps,
lsof, and log archaeology.

Two distinct problems fall out of that:

For the human — there is no single view of running work, and no way to intervene
except finding the right terminal. Autonomy without observability is what turns one
wedged turn into an 18-hour hang, and a /loop workload multiplies that by N.

For agents — instances cannot see each other at all. Two instances will happily place
subagents on the same one-slot provider, start conflicting work in the same repo, or
duplicate a task, because neither knows the other exists.

Discovery already half-exists: local/mdns.ts advertises and finds llama-swap
providers
over Bonjour. It does not advertise opencode instances. The transport is
built; nothing is announced on it.

What Changes

1. Instances announce themselves

Each opencode server advertises over the existing mDNS layer: instance id, host, pid,
API base URL, version, and the directories it has sessions open in. Peers and skein can
discover the mesh without configuration.

2. Instances publish what they are doing

An instance exposes a presence record per active session: session id, directory, agent,
model, current status (idle, busy, awaiting-permission, cancelling, stalled),
iteration count if it is a loop, and time since last token or tool event.

awaiting-permission matters disproportionately. Today a session blocked on a permission
prompt is indistinguishable from a working one unless the human is looking at that
terminal, so agents silently wait on a human who does not know they are waiting.

3. Wedge detection

An instance that has had a turn in flight with no token, tool event, or state change for
longer than a threshold marks that session stalled in its own presence record — and,
critically, keeps publishing. A wedged session must announce that it is wedged. This is
the specific signal that would have surfaced the 18-hour hang within minutes.

Presence records carry a heartbeat. A record whose heartbeat stops is reported as
unreachable rather than silently disappearing, so a crashed instance is distinguishable
from one that exited cleanly.

4. One view, and the ability to act on it

A /agents command lists every discovered instance and session with status and age, and
supports cancelling or pausing a session on another instance through that instance's
existing HTTP API.

This is the payoff for the human: the answer to "what is everything doing right now" is
one command, and the response to a wedged session is one keystroke rather than ps and
kill -9.

5. Skein integration contract

The presence payload is the opencode-side half of Skein's Agents read model. It is
metadata-only and includes an explicit owner (opencode-skein), instance and
session identifiers, optional loop identity/iteration, project, provider/model,
normalized status, event and heartbeat timestamps, and control capabilities.
Skein aggregates this with native Go Agent presence. opencode-skein remains the
authority for its sessions: a normal prompt interrupts the loop, /btw is a
non-history side channel that does not interrupt it, and pause/resume/cancel are
forwarded only when advertised as supported.

6. Supervisor-facing experience

The intended consumer is a single Supervisor conversation, not a collection of
human-managed chat windows. The Supervisor reads the Agents roster and can
delegate work to an existing session or request a new Agent when presence and
capacity allow it. The presence API therefore needs enough structured metadata
to answer which Agent owns a task, which project/repository it is working in,
whether it is available, and which controls are supported. Session transcripts
remain private to the owning Agent; the Supervisor receives progress and result
events rather than scraping conversation content.

Capabilities

New Capabilities

  • agent-presence: discovery of opencode instances, per-session status publication with
    heartbeat and wedge detection, and a cross-instance view with basic control.

Non-Goals

  • No new transport. This uses the existing mDNS discovery and the existing per-instance
    HTTP API. A push bus is agent-coordination-bus, and it is deliberately gated behind
    this change proving the data model first.
  • No Agent-to-Agent messaging or negotiation. Publishing state is not conversation.
  • No claiming or mutual exclusion — knowing an Agent exists is not coordinating with it; see
    provider-slot-leases.
  • No WAN or cross-subnet discovery. mDNS scope only; remote hosts join via explicit config.
  • Not a replacement for Skein's Agents view or Supervisor delegation. This adds the opencode-side half Skein
    structurally cannot see.

Impact

  • New: presence service in packages/opencode/src/local/, instance advertisement in
    local/mdns.ts, presence endpoints on the instance HTTP API, /agents TUI command.
  • Presence is derived from state the session layer already tracks (SessionStatus), so
    the cost is publication, not new bookkeeping.
  • Security: presence exposes directory paths and session titles on the local network.
    Advertisement must be disableable, and must be off by default on untrusted networks.

Addendum (2026-09-05)

Phases 1-2 shipped; Phases 3-4 (wedge/heartbeat detection, the /agents view, Supervisor read
path) did not. That gap turned out to matter beyond observability: /backlog's fan-out nudge and
local placement both consume presence today, with no liveness signal to filter on, which is the
most likely root cause of reports that the "free agent" list /backlog suggests is frequently
wrong (stale, busy, or unreachable entries presented as available). See Phase 6, added below, and
subagent-notification-reliability for the related delegate-and-idle investigation.

Disposition (2026-09-18)

Archived — superseded/obsolete. Phases 1–2 shipped (presence records, /agents, peers). The roster is A2A now (opencode + Claude Code, any directory); delegation/reuse is skein-pool. Stall detection and remote cancel (Phase 3, 4.1–4.3) were never built — reopen as a small change if wanted.

Tasks

Phase 1: Integrate existing instance discovery

  • 1.1 Confirm the existing server/mdns.ts HTTP advertisement is the opencode
    instance-discovery mechanism; do not add a second Agent-specific advertiser or
    transport. #discovery

    • Evidence: packages/opencode/src/server/mdns.ts publishes the running server
      over mDNS and server/server.ts owns its lifecycle.
  • 1.2 Add an Agents discovery adapter that consumes the existing opencode mDNS
    HTTP announcements and queries each instance's metadata-only /agents endpoint.
    Keep the existing llama-swap provider discovery untouched.

    • Validation: two local instances are returned as Agent sources and stale
      instances are handled without affecting provider discovery.
  • 1.3 Add a config switch for Agents discovery, defaulting to off on
    non-trusted networks

    • Mirrors the experimental.local_subagent_placement kill-switch pattern
    • Validation: disabled instance is undiscoverable and otherwise unaffected

Phase 2: Presence records

  • 2.1 Define the presence record type

    • owner, instanceID, sessionID, loopID?, directory, agent, provider, model, status, loopIteration?, lastEventAt, heartbeatAt, and explicit canPrompt, canBtw, canAbort capabilities
    • Metadata only: no prompt, tool call, or tool output fields
    • Validation: bun typecheck — zero errors
  • 2.2 Derive status from the existing session state rather than new bookkeeping

    • Source SessionStatus plus the Agent execution state; awaiting-permission from the permission layer
    • Validation: unit tests map each underlying state to the right presence status
  • 2.3 Expose presence over the instance HTTP API

    • Validation: bun run script/httpapi-exercise.ts --mode coverage --include agents.list
      exercises GET /agents successfully. The full harness remains non-green because
      14 unrelated pre-existing routes lack scenarios.
  • 2.4 Suppress content; metadata only

    • Assert no prompt text, tool output, or message content is reachable through presence
    • Validation: test inspects the serialized record for content leakage

Phase 3: Wedge detection and heartbeat

  • 3.1 Track time since last token/tool/state event per in-flight turn

    • Validation: unit test — a silent in-flight turn crosses the threshold and flips to stalled
  • 3.2 Keep publishing while stalled

    • A wedged session must not drop out of presence
    • Validation: test asserts the record persists after the threshold
  • 3.3 Heartbeat, with unreachable on lapse and withdrawal on clean exit

    • Validation: killed instance → unreachable with last-heartbeat time; clean exit → withdrawn
  • 3.4 Regression test reproducing the 2026-07-25 hang shape

    • Provider reports zero in-flight while the client still has a turn open
    • Assert: session reported stalled, provider health does not suppress it
    • Validation: new test passes

Phase 4: The Agents view

  • 4.1 /agents lists instances and sessions with status and age

    • Validation: manual — four instances across three directories all listed
  • 4.2 Cancel/pause a session on a remote instance through its API

    • Reuses the cancel path; depends on session-cancellation-integrity for cancel to be
      reliable on a wedged session
    • Validation: manual — cancelling a stalled remote session updates its status
  • 4.4 Preserve the control boundary used by Skein

    • A normal prompt is interrupting; /btw is side-channel input that does not
      enter the main history or stop a loop; pause/resume/cancel remain owner APIs
    • Validation: integration test covers a loop receiving /btw and continuing,
      then a normal prompt aborting the loop
  • 4.3 Sort so the things needing attention surface first

    • stalled and awaiting-permission above busy above idle
    • Validation: manual — a stalled session appears at the top

Phase 4A: Supervisor integration

  • 4.5 Include project, repository/worktree, Role, task/feature metadata,
    availability, and control capabilities in the Agent roster without exposing
    transcript content
  • 4.6 Add a Supervisor-facing read path for the roster and progress events;
    the Supervisor receives summaries/events while the owning Agent retains the
    full session transcript
  • 4.7 Define delegation handoff semantics: reuse an eligible existing Agent
    when possible, otherwise request a new Agent through Skein; reject stale,
    unavailable, ambiguous, or duplicate assignments

Phase 5: Verification

  • 5.1 Multi-instance soak

    • Run three instances; wedge one deliberately; confirm it is reported stalled and is
      cancellable from another instance
    • Validation: recorded run
  • 5.2 Full typecheck and test

    • Validation: bun typecheck zero errors; bun test packages/opencode --timeout 60000 green

Phase 6: Backlog fan-out roster accuracy (added 2026-09-05)

Reported symptom: /backlog's fan-out nudge (loop-spec-queue §6) and local placement
(ctx-aware-subagent-placement, role-placement-policy) suggest "free" agents that are
frequently wrong — the caller's own current session, or a peer that is actually busy/unreachable.
session-peer-awareness already excludes the caller and its descendant lineage (1.2, shipped),
so a same-session false positive after Phase 6 lands would be a regression worth its own test, not
expected behavior. The more likely cause is upstream of exclusion logic entirely: Phases 3 and 4
above — the parts of this change that give a roster entry actual liveness — were never built,
so anything consuming presence today (the fan-out nudge included) is reading a roster with no
wedge/heartbeat/staleness signal at all. An entry that looks busy-and-fine may be a session that
died 20 minutes ago.

  • 6.1 Once Phase 3 lands, verify the fan-out nudge in packages/opencode/src/loop/loop.ts
    (queue-mode brief construction) filters candidates through the same liveness/heartbeat
    status this change defines, not just raw presence existence.
    • File: packages/opencode/src/loop/loop.ts
    • Validation: unit test — a stalled/unreachable peer is never named in the fan-out nudge
  • 6.2 Perform the three outstanding "live check against a real fleet" verifications that were
    left unchecked elsewhere, now that Phase 3/4 give them something real to check against:
    session-peer-awareness task 4.2, role-placement-policy task 4.2,
    subagent-background-default task 6. Record the outcome in each change's tasks.md rather
    than only here.
    • Validation: each of the three referenced tasks is checked with a one-line evidence note, or
      reopened as a bug if the live check fails
  • 6.3 If 6.2 surfaces a real delegation-notification bug (parent idles after backgrounding a
    subagent instead of being woken by the injected result), do not fix it here — file it under
    subagent-notification-reliability, which exists specifically to investigate that failure
    mode and depends on this phase's roster accuracy to rule out "stale roster" as the cause.
    • Validation: n/a — coordination task

Unchecked items above: see Disposition in proposal.md (2026-09-18).

Plan changes

5 done

Activity

  1. androidand commented on Sep 17, 2026

    @androidand
    OwnerAuthor

    Archived 2026-09-18 — see the change's Disposition section.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions