Repository navigation
Stop concurrent instances double-booking the same provider slot #37
Description
Proposal
Why
Provider reservations are per-process and invisible to everyone else.
placement.ts holds reservations in memory with a TTL so that concurrent picks within
one process don't double-book a host. On 2026-07-25 four opencode processes were running
on this machine simultaneously. Each had its own reservation map. Nothing stopped two of
them from both observing z4 as idle — which it was, in_flight: 0, slots_total: 1 — and
both dispatching a subagent onto its single slot. The second request queues behind the
first, which is exactly the outcome placement exists to avoid.
This is the concrete form of "can I claim this provider as my sub-agent, and does that
mean it can't take other calls". Today the answer is: you can decide to claim it, and
nobody else can hear you say so.
A correction worth stating plainly. The shared SQLite file is tempting as the
coordination point — every instance on this machine has the same inode open and SQLite
gives real cross-process ACID. But it is a local file. m5 also runs opencode-skein and
does not share it. So the shared database solves same-host contention completely and
cross-host contention not at all. Any design that treats it as the fleet-wide answer is
wrong.
What already exists cross-host. skein's claim registry is genuinely distributed —
entries carry host, pid, age_secs, and it already expires them. But its claims are
scoped to changes and roles, not provider slots. The mechanism is right; the noun is
missing.
What Changes
1. Leases, not locks
A lease is a short-lived, TTL-bounded declaration of intent to use one slot on one
provider. Leases expire on their own; a crashed holder cannot wedge a provider.
Leases are deliberately advisory. Perfect distributed mutual exclusion is expensive,
and the cost of losing a race here is low — a request queues behind another. It does not
corrupt anything. Paying for consensus to avoid a queue would be the wrong trade.
2. Verify immediately before dispatch
The lease narrows the race window; it does not close it. Directly before dispatching,
placement re-reads live capacity (provider-capacity-truth) and abandons the placement if
the slot is no longer free, falling back to the existing inherit-parent path.
This makes correctness depend on an observable fact rather than on a distributed
agreement, which is the only way to get it cheaply. The lease turns a likely collision
into a rare one; the verify turns a rare collision into a fallback instead of a queue.
3. Tiered store, honest about reach
| scope | store | guarantee |
|---|---|---|
| same host | shared SQLite, unique constraint on active lease per provider slot | exact mutual exclusion |
| cross host | skein claim registry when reachable | best effort, TTL-expired |
| neither | in-memory, current behaviour | per-process only |
Each tier degrades to the one below without failing. An instance with no skein and no
peers behaves exactly as it does today. opencode-skein must remain fully functional
standalone — skein makes coordination better, never mandatory.
4. Leases are observable
Active leases are exposed alongside capacity, so a human or peer can see not just that a
slot is busy but who intends to use it and for how long. A slot that is free but leased is
a different situation from one that is simply free, and the fleet view should say so.
Capabilities
New Capabilities
provider-leases: TTL-bounded, advisory, observable claims on provider serving slots,
with a tiered store that degrades safely.
Dependencies
provider-capacity-truth— verify-before-dispatch needs a trustworthy free-slot reading.
Without it, verification would consult GPU utilisation and refuse to place on idle hosts.
Non-Goals
- No consensus protocol, no leader election, no distributed lock manager. The failure mode
being prevented is a queued request. - Not making skein a required dependency.
- Not leasing anything other than provider slots — file locks and repo/worktree contention
between instances are a separate problem. - Not changing placement's scoring policy; only adding a lease acquisition and a
pre-dispatch check around the existing decision.
Impact
- Modified:
packages/opencode/src/local/placement.ts(reservations become leases;
pre-dispatch verification), plus a lease store with the three tiers. - New: lease table in the shared database; skein claim adapter behind an interface so the
cross-host tier is swappable. - Behaviour is unchanged for a single instance with no peers — the common case must not
regress or slow down.
Disposition (2026-09-18)
Archived — superseded/obsolete. Superseded by skein-pool Phase 3: leases live on the llama-skein host that owns the slots, not in a three-tier store.
Tasks
Depends on provider-capacity-truth — pre-dispatch verification is only correct with a
trustworthy free-slot reading.
Phase 1: Lease model
-
1.1 Define the lease type and store interface
provider,slotIndex?,holderInstance,holderSession,acquiredAt,expiresAt- Interface:
acquire,renew,release,list— tier implementations swap behind it - Validation:
bun typecheck— zero errors
-
1.2 TTL expiry semantics, including renewal
- Validation: unit tests — expiry without holder action; renewal keeps the slot held
Phase 2: Same-host tier (exact)
-
2.1 Lease table in the shared database with a uniqueness constraint on active lease per provider slot
- Validation: schema migration applies cleanly against an existing database
-
2.2 Atomic acquisition
- Validation: concurrency test — four concurrent acquirers for one slot, exactly one wins
-
2.3 Expired leases do not block acquisition
- Validation: test — an expired row is superseded, not treated as held
Phase 3: Cross-host tier (best effort)
-
3.1 Adapter interface for an external claim registry
- Keep it swappable; do not hard-depend on skein
- Validation:
bun typecheck; adapter is injectable in tests
-
3.2 skein claim-registry adapter
- Map provider slot leases onto the registry's host/pid/age model
- Validation: integration test against a stub registry
-
3.3 Degrade cleanly when the registry is absent or slow
- Bounded probe budget; never block placement on it
- Validation: test — unreachable registry adds no failure and no unbounded delay
Phase 4: Wire into placement
-
4.1 Replace the in-memory reservation with a lease acquisition
local/placement.ts— preserve existing scoring and the 20s recent-placement spread- Validation:
bun test test/local/placement.test.ts --timeout 30000— existing tests pass
-
4.2 Add pre-dispatch capacity re-verification with fallback to inherit-parent
- Validation: test — slot filled between lease and dispatch results in fallback, not a queued request
-
4.3 Confirm no regression for the standalone single-instance path
- Validation: benchmark placement latency before/after; no added latency with no peers
Phase 5: Observability
-
5.1 Expose active leases alongside capacity snapshots
- Validation:
bun run test:httpapipasses; leased-but-free slots are distinguishable
- Validation:
-
5.2 Surface leases in the fleet view from
fleet-instance-presenceif that has landed- Validation: manual — a leased slot shows holder and expiry
Phase 6: Verification
-
6.1 Multi-process race on one host
- Four instances, one free slot, simultaneous placement attempts
- Validation: exactly one dispatch; three clean fallbacks; no request queues behind another
-
6.2 Full typecheck and test
- Validation:
bun typecheckzero errors;bun test packages/opencode --timeout 60000green
- Validation:
Unchecked items above: see Disposition in proposal.md (2026-09-18).
Activity
androidand commentedon Sep 17, 2026
Archived 2026-09-18 — see the change's Disposition section.