RFC: workflow orchestration over the control plane (Connect as primary, openab-pty hosts as workers) #77

Description

Problem. An agent can already write a multi-step plan as a workflow graph: step nodes (id, agent, prompt, artifacts), repeat nodes (maxIterations, onMaxIterations, stopCondition with fileCheck / jsonPath / value), and parallel nodes (joinPolicy), the same shape as Kiro workflow JSON. Today nothing in OpenAB can run such a graph across several hosts. Every openab-pty host can already run a one-shot, non-interactive prompt (for example kiro-cli chat --no-interactive --trust-all-tools --output-format stream-json), but there is no shared view of which hosts are free and no way to hand one of them a step.

Goal. Let OpenAB Connect (the macOS client) run a workflow by assigning each step to an openab-pty host through openab-cp. Connect talks to one control plane, not to every host.

OpenAB Connect (primary)          openab-cp            openab-pty hosts (workers)
  workflow graph  ── cp/delegate ──►  router  ── fwd ──►  one-shot CLI run per step
  scheduler       ◄─ delegate_result ─        ◄─ result ─  (dial out, register + heartbeat)

This RFC asks for agreement on the shape and on one open decision (where the scheduler lives). It does not propose code yet.

Use Case

A user asks an agent in OpenAB Connect for a change that needs several steps: research, implement on two hosts in parallel, review, and repeat until a check file says pass. The agent emits the workflow graph. The user approves the whole graph once. Connect then places each step on a registered worker with the right CLI, collects results, evaluates loop and join conditions, and shows progress. Hosts behind NAT need no inbound port.

Current state on main (verified)

CP code lives in openabdev/openab; paths in this table are in that repo unless noted. Verified against openabdev/openab main at 02a8b24. Only the CP server is on main. The runtime half (OAB [control_plane] client, PromptRunner, serve_local, openab agent … CLI) lives on branch feat/cp-runtime-client-main from openabdev/openab#1528, which was auto-closed unmerged. Items marked (branch) below are verified there, not on main.

Permalinks:
https://github.com/openabdev/openab/blob/02a8b24378e1e2dc1d5e2861525d21a05693d170/crates/openab-cp/src/proto.rs#L638-L648
https://github.com/openabdev/openab/blob/02a8b24378e1e2dc1d5e2861525d21a05693d170/crates/openab-cp/src/policy.rs#L88-L135
https://github.com/openabdev/openab/blob/91c0a89f7637eb47cbd78d470df01a53e81a13bf/crates/openab-core/src/control_plane/executor.rs#L74-L83

Note: on main the "primary refuses cp/delegate" rule is a runtime-side check only. The CP registry excludes observers from selection but not primaries (registry.rs L382), so a label selector that matches a primary can route to it. Worker labels should not overlap primary labels.

Mapping

The one-way type rule fits this design: Connect only initiates (primary), hosts only serve (worker). No peer type is needed here.

Why CP instead of Connect dialing each host

• Workers dial out to the CP. Hosts need no inbound port.
• Connect holds one CP key instead of every host's admin credential.
• Placement, load, liveness and audit (cp/event) live in one place.

Missing pieces

1. CP worker in openab-pty. A PromptRunner that runs one dedicated non-interactive CLI session per delegation in the host's workspace. Result = captured output, ANSI-stripped and capped below max_result_bytes. It does not type into the user's live PTY. The runtime client itself is not on openabdev/openab main yet (feat(cp): land Agent Control Plane v1 on main openab#1528).
2. CP primary client for OpenAB Connect. Either a native Swift client of the wire protocol, or a local openab run primary driven through openab agent spawn (both need feat(cp): land Agent Control Plane v1 on main openab#1528's work to land first, or the protocol to be reimplemented).
3. Scheduler. CP only does single delegations. repeat, parallel, joinPolicy, stopCondition and onMaxIterations need a scheduler that issues delegations and evaluates conditions.
4. Artifacts between steps on different hosts. Text results are capped. Files move through git (branch per workflow run), not through CP. stopCondition.fileCheck must read from that shared repo, not a host-local path.

Open decision: where the scheduler lives (not decided here)

Limits

• Per-delegation deadline cap (default 1800 s on the CP; CLI 1..=1800 on the branch). Steps longer than 30 minutes do not fit and must be split.
• Outstanding cap: ≤256 requests per primary (branch); per-target clamp (default 16) and process-wide 4096 in-flight on the CP. Large parallel fan-out gets SATURATED; the scheduler must queue.
• Result size cap: 256 KiB default, truncated. Large outputs go through git.
• No durable CP state: registry and in-flight table are in memory. A CP restart fails in-flight steps.

Security

• A worker running with --trust-all-tools is shell-equivalent on its host. Anyone who can delegate to it can run commands there.
• Agent-generated workflows must be approved by a human as a whole graph before any step runs, so injected text in one step's output cannot add steps or change targets. Re-approval if the graph changes.
• Per-worker auth_key, bound to namespace/name, so one host can be revoked alone.
• Which labels a primary may target is not policy today (namespace-scoped only). Proposal: an allowlist per primary identity, evaluated by the CP.
• Step prompts and results appear as excerpts in cp/event unless the namespace sets metadata_only = true.

Client-side requirements (OpenAB Connect)

1. Create and manage a CP. Deploy openab-cp on a host the client manages, and mint and revoke per-worker auth_keys bound to namespace/name. Today there is no packaged image or release for openab-cp: docs/control-plane.md says "there is no packaged container image yet", and no openab-cp release tag exists; it builds from source (cargo run -p openab-cp). Keys are [[agents]] entries in CP config; whether the CP reloads them without a restart is unverified.
2. Bind a host to an existing CP at creation. The client passes the CP URL and a per-host key as a secret; the host registers on start with labels such as runtime=pty, cli=kiro. This needs a CP worker in or beside openab-pty (missing piece 1). A colocated classic openab container (FR: colocated classic openab alongside openab-pty (shared /workspace) #62) carrying a [control_plane] worker section: main has no [control_plane] config at all; the branch adds ControlPlaneConfig. Whether that worker can run next to a Discord adapter in the same process is unverified (the branch PR feat(cp): OAB runtime CP client — [control_plane] config, worker serving, headless mode (PR 3/4) openab#1471 describes a headless mode).
3. Inventory. List the workers on a CP with runtime labels and liveness. cp/list_agents exists on main (any registered client, own namespace) and returns labels and session counts; liveness = present in the registry (removed on disconnect or 45 s lease expiry), plus agent_registered / agent_deregistered events for observers. There is no last-heartbeat timestamp in the listing. openab agent list exists only on the branch.

These three are necessary but not sufficient. Running a workflow also needs: the client acting as a CP primary, a workflow approval view, the scheduler (placement still open), and progress and result views.

Phasing

• Phase 0: validate with a sidecar openab run worker next to openab-pty on two hosts, plus a primary driven by openab agent spawn. No code change, but needs a build of the feat(cp): land Agent Control Plane v1 on main openab#1528 branch.
• Phase 1: embedded worker PromptRunner in openab-pty (dedicated non-interactive session per delegation).
• Phase 2: OpenAB Connect primary plus scheduler (option A, or a thin client of B).
• Phase 3: durable scheduler, only if option B is chosen.

Acceptance criteria

Phase 0

• Two hosts register as workers with distinct labels; openab agent list from the primary shows both with labels.
• openab agent spawn --label cli=kiro --prompt … --deadline-secs 300 returns completed with output from one host.
• Two parallel spawns land on different hosts (least-loaded selection).
• Stopping one host fails its in-flight delegation as target_disconnected within the lease window.
• A worker identity cannot initiate (POLICY_DENIED).

Phase 1

• openab-pty with an opt-in [control_plane] section registers as a worker, heartbeats, and serves cp/delegate with no sidecar.
• Each delegation runs in its own non-interactive session; the user's live PTY is not typed into.
• Results are ANSI-stripped and capped below the CP's max_result_bytes; over-long output is truncated with a marker.
• cp/cancel and deadline expiry kill the session's process group.
• max_delegated_sessions is enforced; extra delegations are refused, not queued.
• The section is off by default and is not rendered by default in the K8s chart.

Related

• docs(adr): agent control plane for direct inter-agent delegation openab#1465 ADR: agent control plane (merged)
• feat(cp): openab-cp control plane — registry, router, policy (PR 1/4) openab#1469, feat(cp): observer/lobby — 3-phase design + Phase 1 protocol scaffold (PR 2/4) openab#1470 CP server and observer/lobby (merged to main)
• feat(cp): OAB runtime CP client — [control_plane] config, worker serving, headless mode (PR 3/4) openab#1471, feat(cp): add primary delegation MCP tools and CLI openab#1529 runtime client, primary MCP tools and CLI (merged into feature branches, not main)
• feat(cp): land Agent Control Plane v1 on main openab#1528 CP v1 roll-up to main (closed unmerged)
• refactor(cp): post-v1 hardening follow-ups from #1528 self-review openab#1530 post-v1 hardening follow-ups, feat(cp): scale hardening — indexed sweeps, sharded admission, borrowed lookup keys openab#1473 scale hardening
• FR: colocated classic openab alongside openab-pty (shared /workspace) #62 colocated classic openab beside openab-pty

No existing issue proposes workflow orchestration or a peer agent type.