Skip to content

Proposal: Context-aware, slot-aware local sub-agent placement #4

Closed
Closed
Proposal: Context-aware, slot-aware local sub-agent placement#4
Assignees
androidand
Labels
spec:archived

Description

@androidand
androidand
opened on Jul 25, 2026
Proposal

Why

opencode prefers dispatching sub-agents (Task/Explore) to local llama-skein
providers. Two bugs make that routing send work to a provider that cannot serve
it and then double-book it (audit 2026-07-11):

  1. The scorer ignores context adequacy → biases to the worst model.
    LocalPlacement.pick/bestModel (packages/opencode/src/local/placement.ts)
    scores a candidate as
    fit_level×1000 + parent-match + resident + tokens/sec + free-VRAM.
    fit_level measures whether the model fits in VRAM, not whether its
    context is usable. A model pinned to a tiny ctx (e.g. a host's 614-token
    models, max_safe_ctx=0) fits perfectly, leaves the most free VRAM, and is
    fastest — so it wins every term the scorer measures and is selected for
    every sub-agent, which then can't fit the prompt and never returns a useful
    result. max_safe_ctx/configured_ctx are never read; the sub-agent prompt
    size is never passed into placement; the /api/fit probe is called with no
    ctx argument.

  2. No single-slot concurrency gate → double-booking. Parallel Task calls run
    with concurrency: "unbounded" (session/processor.ts) and each calls
    pick() independently. isBusy reads a point-in-time inference.busy
    snapshot; both concurrent picks see the same idle host before either
    dispatches (TOCTOU), and the only mitigation is a soft post-hoc
    RECENT_PLACEMENT_PENALTY. slots_total/in_flight exist in the client but
    are never used for scheduling. The inherit-parent fallback has no gate at
    all. Two sub-agents thus land on a single-slot (--parallel 1) provider; the
    second is queued by llama-skein with no error, and opencode only retries on
    errors with no wall-clock timeout — so it hangs indefinitely.

What

  • Context-aware selection. Placement MUST estimate the sub-agent's prompt
    size and exclude any candidate whose usable context (max_safe_ctx, or a
    per-model /api/fit?ctx= probe at the estimated size) cannot serve it. A
    model that cannot fit the prompt MUST NOT be selected regardless of VRAM fit /
    speed / free-VRAM score. Context adequacy is a hard filter, not a score term.
  • Slot-aware concurrency. Placement MUST maintain an in-process reservation
    keyed by providerID, respecting the provider's slots_total against a live
    in-flight count, held for the sub-agent's lifetime. A provider with no free
    slot MUST NOT be selected — including on the inherit-parent fallback path —
    so a single-slot provider is never double-booked within a turn. Concurrent
    picks MUST serialize their reservation decisions (close the TOCTOU gap), not
    merely apply a soft penalty.
  • Bounded wait. A sub-agent request to a local provider MUST have a
    wall-clock ceiling so a silently-queued request cannot hang forever; on
    timeout it re-places or falls back.

Non-goals

  • No display/overflow changes (separate change
    ctx-display-and-overflow-correctness).
  • No skein-supervisor changes; skein already gates one agent per provider at
    pipeline granularity but cannot see intra-process sub-agent fan-out — the fix
    must live here.

Impact

  • packages/opencode/src/local/placement.ts: prompt-size input; hard
    context-adequacy filter; per-provider reservation/semaphore honoring
    slots_total/in_flight; apply gate to the inherit path; replace/keep the
    recent-placement penalty as a tiebreak only.
  • packages/opencode/src/tool/task.ts: pass prompt-size estimate into pick;
    hold the reservation across the sub-agent run; release on completion.
  • packages/opencode/src/session/retry.ts (or the local dispatch path): a
    wall-clock ceiling on local sub-agent inference.

Disposition (2026-09-18)

Shipped — live checks moved. Shipped; task 5 (wall-clock ceiling) delivered for the delegated path by skein-pool 2.4, live checks moved to fleet-live-verification.

Tasks

  • 1. Pass an estimated sub-agent prompt size from task.ts into
    LocalPlacement.pick (promptText; estimateRequiredCtx).
  • 2. placement.ts: hard context-adequacy filter in bestModel — exclude
    any candidate whose max_safe_ctx cannot serve the estimated prompt,
    before scoring. Selection never returns a model that can't fit the prompt.
  • 3. placement.ts: per-providerID in-process reservation (reserve/
    reservedFor/freeSlots) honoring slots_total vs live in-flight;
    reserved synchronously before any await so concurrent picks serialize
    (TOCTOU closed); TTL backstop against leaks.
  • 4. task.ts: hold the reservation for the sub-agent lifetime; release via
    Effect.ensuring on completion/error/interrupt (foreground + background).
  • 5. Add a wall-clock ceiling on local sub-agent inference so a queued
    single-slot request cannot hang forever; re-place or fall back on timeout.
    (REMAINING — load-bearing. The earlier note claiming the primary hang
    cause was fixed by tasks 2–3 is WRONG: see task 9. Today the only
    backstop is SUBAGENT_TASK_TIMEOUT_MS = 10 * 60 * 1000
    (packages/opencode/src/tool/task.ts:36), which reads to a human as
    "hung forever". Bound it far tighter and re-place on timeout.)
  • 6. Tests (test/local/placement.test.ts): tiny-ctx model never selected
    even when resident/perfect/fastest; no adequate model → null;
    freeSlots slot/in-flight/reservation accounting; estimateRequiredCtx.
    (Note: concurrency serialization is covered at the freeSlots+reservation
    unit level, not a full two-pick integration test.)
  • 7. bun run typecheck (0 errors) + bun test (10 pass) green.
  • 8. Design-first codegen: regenerated the opencode TS llama-skein client
    (bun run build:llama-skein-client) from the updated spec; the new
    unknown fit_level forced FIT_RANK to handle it (ranked 0 / excluded).
    Committed a9112d3.

Reopened 2026-07-25 — the inherit-parent path was never gated

The proposal mandates the slot gate "including on the inherit-parent fallback
path". That was not implemented, so the reported session hang is still live.

  • 9. Gate the inherit-parent path. packages/opencode/src/tool/task.ts:190-210
    skips placement entirely when !provider || next.model || session || local_subagent_placement === false, and when LocalPlacement.pick
    returns null it falls through to model = ... ?? inherited — with no
    busy/free-slot preflight    . The sub-agent is then dispatched to the exact
    single-slot provider the parent is currently occupying, where it queues
    invisibly behind the parent. placement.ts:216 filters candidates with
    .filter((info) => info.id !== input.parent.providerID), so the slot gate
    at :232 (if (freeSlots(...) <= 0) continue) never evaluates the parent.
    Apply the same freeSlots check before inheriting; if the parent has no
    free slot, fail fast or fall back rather than inherit.
    Validation: a test where the parent provider has 0 free slots asserts the
    sub-agent is NOT dispatched to it.
    Fixed: changed capacity === "no-slot" to capacity !== "free" in
    task.ts — blocks both "no-slot" and "unknown" (unreachable parent).
  • 10. Do not let a failed probe degrade into the unguarded path.
    placement.ts:218-221 uses a 1.5 s probe budget and treats probe failure
    as null → inherit. A provider that is up but unreachable on
    /api/hardware therefore lands on the very path task 9 fixes. Treat probe
    failure as "unknown, do not inherit" rather than "inherit".
    Validation: test with the probe stubbed to fail asserts no inherit-dispatch.
    Fixed: the capacity !== "free" change in task 9 also covers this —
    "unknown" (returned when probe fails) now blocks inheritance. Tests updated
    to document the contract.
  • 11. Consume llama-skein readiness once available. model-failure-state
    (llama-skein) adds a failed state, a retained last_error, and a
    /health body reporting per-model state plus provider-level busy.
    Preflight that before dispatch instead of inferring readiness from
    reachability. Until it lands, tasks 5/9/10 are the caller-side mitigation.
    Validation: with a provider reporting failed, no sub-agent is dispatched
    to it.
  • 12. Header timeout coverage. packages/opencode/src/provider/provider.ts:58,65
    defines LOCAL_PROVIDER_HEADER_TIMEOUT_DEFAULT = 180_000 and
    LOCAL_PROVIDER_CHUNK_TIMEOUT_DEFAULT = 120_000, applied at :2058-2070
    but gated on model.api.npm === "@ai-sdk/openai-compatible" — a local
    provider registered under any other adapter gets no header timeout at all.
    Apply the timeouts by provider locality, not by npm adapter identity.
    Validation: a local provider on a different adapter still gets both timeouts.

Unchecked items above: see Disposition in proposal.md (2026-09-18).

Plan changes

9 done

Activity

  1. androidand
    added
    specsync
    stage:active
    on Jul 25, 2026
  2. androidand
    self-assigned this
    on Jul 25, 2026
  3. androidand
    added a commit that references this issue on Jul 25, 2026

    fix(task): refuse to queue a subagent behind its own parent (#5)

    0821e6d
  4. androidand
    added
    spec:archived
    and removed
    specsync
    stage:active
    on Sep 17, 2026

  5. androidand commented on Sep 17, 2026

    @androidand
    androidand
    on Sep 17, 2026
    OwnerAuthor

    Archived 2026-09-18 — see the change's Disposition section.

  6. androidand
    closed this as completedon Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

spec:archived

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions