diff --git a/.agents/skills/harness-adapters/SKILL.md b/.agents/skills/harness-adapters/SKILL.md index bcc5c87a6a4..886bc49664a 100644 --- a/.agents/skills/harness-adapters/SKILL.md +++ b/.agents/skills/harness-adapters/SKILL.md @@ -33,7 +33,7 @@ For recovery and control, use the exact `harness=` in `state/.meta`; never i Deliver lifecycle actions only through `../../../bin/fm-control.sh interrupt|exit|relaunch`. Never type an interrupt key or exit command through `fm-send`, where routing-marked lifecycle text becomes chat. Trust handling is complete only when inspection proves the target started processing its instructions; delivery success alone is not proof. -Muse, Gemini, Devin, Command Code, Polytoken, and Cortex are verified only for crewmate and scout work, never a secondmate or primary. +Muse, Gemini, Devin, Command Code, Polytoken, Cortex, Kiro, and Hermes are verified only for crewmate and scout work, never a secondmate or primary. ## Detection @@ -97,7 +97,9 @@ A new tool remains undispatchable until the `verify` plan, its harness entry, ev "devin": "references/harness/devin.md", "commandcode": "references/harness/commandcode.md", "polytoken": "references/harness/polytoken.md", - "cortex": "references/harness/cortex.md" + "cortex": "references/harness/cortex.md", + "kiro": "references/harness/kiro.md", + "hermes": "references/harness/hermes.md" } } ``` diff --git a/.agents/skills/harness-adapters/references/common/control-and-recovery.md b/.agents/skills/harness-adapters/references/common/control-and-recovery.md index 2967f3136f1..d92cb883b41 100644 --- a/.agents/skills/harness-adapters/references/common/control-and-recovery.md +++ b/.agents/skills/harness-adapters/references/common/control-and-recovery.md @@ -20,6 +20,7 @@ Each supported harness handles its folder-trust gate differently, and the tool r For Claude, load `references/harness/claude.md`; its workspace-trust section owns the non-key-answerable gate and spawn-time pre-registration for every spawn kind. agy gates every fresh worktree too; the spawn pre-registers it in agy's own store the same way, and a strict post-launch gate answers any dialog that still renders before the spawn reports success. Cursor suppresses its dialog with launch-time `--trust`, and Muse suppresses its own with `--yolo`. +Kiro suppresses its `--trust-all-tools` confirmation modal with a `chat.disableTrustAllConfirmation` setting the spawn seeds into the per-task `KIRO_HOME`; `references/harness/kiro.md` owns that recipe and where the decision persists. Grok renders a folder-trust gate in a linked worktree, and `references/harness/grok.md` owns how to verify the worker's real location, answer it, and where the decision persists; the project picker is a separate dialog that stays absent when the spawn starts in a git root. Pi gates the fresh-worktree case too, but unlike Claude its dialog is answered with Enter, and `references/harness/pi.md` owns that recipe and where the decision persists. Codex shows a directory-trust dialog on the first run for a repository root. diff --git a/.agents/skills/harness-adapters/references/harness/claude.md b/.agents/skills/harness-adapters/references/harness/claude.md index 1e8e284022c..e72d0958c54 100644 --- a/.agents/skills/harness-adapters/references/harness/claude.md +++ b/.agents/skills/harness-adapters/references/harness/claude.md @@ -23,7 +23,7 @@ Claude gates a folder it has never seen behind an interactive workspace-trust di Every Claude spawn therefore pre-registers its launch directory before launch, and the dialog does not appear: the task worktree for a ship or scout, and the home itself for a `--secondmate` spawn, in either seeded shape (a leased worktree or a standalone clone). A second, separate dialog - "Allow external CLAUDE.md file imports?" - renders whenever a loaded CLAUDE.md chain reaches outside the project tree, which every crewmate's does through the captain's own `~/.claude/CLAUDE.md` importing `~/.claude/RTK.md`. -`--setting-sources project,local` (the minimal worker tool surface) does not suppress it either, and it gates the session exactly like the trust dialog: cursor on "No, disable external imports", no way to move the selection from firstmate's steering plane. +`--setting-sources project,local`, which only `../../../bin/fm-live-lab.sh` passes (an `fm-spawn.sh` worker loads the user settings too), does not suppress it either, and it gates the session exactly like the trust dialog: cursor on "No, disable external imports", no way to move the selection from firstmate's steering plane. `../../../bin/fm-claude-trust.sh` records `hasTrustDialogAccepted` for both the worktree and its primary checkout in `${CLAUDE_CONFIG_DIR:-$HOME}/.claude.json`, where a home's worker account pin decides `CLAUDE_CONFIG_DIR` (`../../../docs/configuration.md` "Worker account pin"), for a ship or scout spawn; a secondmate spawn registers only its own home entry, since a secondmate home has no separate primary-checkout entry to carry import consent forward from. For a ship or scout spawn, the external-imports flags (`hasClaudeMdExternalIncludesApproved`, `hasClaudeMdExternalIncludesWarningShown`) are carried forward alongside the trust flag only when the primary checkout's project entry already carries an explicit `hasClaudeMdExternalIncludesApproved===true` from a prior interactive session - the common first-spawn case is a project claude has never been asked about, so those two flags are left unwritten and the import dialog still renders, even though trust registers normally. diff --git a/.agents/skills/harness-adapters/references/harness/codex.md b/.agents/skills/harness-adapters/references/harness/codex.md index 3542431d969..7dd042385a2 100644 --- a/.agents/skills/harness-adapters/references/harness/codex.md +++ b/.agents/skills/harness-adapters/references/harness/codex.md @@ -45,14 +45,16 @@ This is why `$no-mistakes` reaches a Codex worker instead of being consumed by t ## Primary integration The primary integration was verified on 2026-07-08 with codex-cli 0.142.1. -The firstmate primary's `.codex/hooks.json` registers a Stop hook that pipes Codex's payload to `../../../bin/fm-turnend-guard.sh`. +The firstmate primary's `.codex/hooks.json` registers a Stop hook that pipes Codex's payload to `../../../bin/fm-turnend-guard.sh`, through `../../../bin/fm-codex-stop-park.sh --guard` when that script is present. Codex Stop hooks preserve exit status 2 and stderr to block, and expose `stop_hook_active` for the same one-block loop safety used by the guard's default mode. The Stop payload includes `cwd`, but the tracked hook does not use it to choose the guard executable. Codex runs the Stop command with process PWD set to the hook-loaded project root, while no `CODEX_PROJECT_DIR`, `CODEX_WORKSPACE_ROOT`, or `CODEX_CWD` root variable is set. The tracked hook anchors to `pwd -P`, verifies that root is Firstmate-shaped and hook-bearing, and then invokes the guard with the original payload. -Codex's primary watcher protocol is `../../../bin/fm-watch-checkpoint.sh --seconds "${FM_CODEX_WATCH_CHECKPOINT:-180}"`, not `../../../bin/fm-watch-arm.sh`. -Codex cannot reason while a foreground tool call is running, so the checkpoint is deliberately foreground and bounded to return control regularly for user messages and queued notifications. -In a home with `config/supervision-host` and no `config/supervision-host-off` the checkpoint runs the supervision host instead of the watcher, with Claude's print mode as its headless engine, and holds for at least an hour while away; [`supervision-host.md`](../../../../../docs/supervision-host.md) owns the host and that bound. +Codex CLI 0.163.0 supports the native asynchronous Stop park registered by `bin/fm-codex-stop-park.sh` in a home that opts in with [`config/codex-stop-park`](../../../../../docs/configuration.md#codex-native-stop-park-configcodex-stop-park), verified in an isolated real CLI lab on 2026-10-19. +The hook waits outside the model and delivers actionable wakes with `codex queue --thread`, including in embedded mode without a shared app-server daemon. +The script owns version and native-session selection; [`codex.md`](../../../../../docs/supervision-protocols/codex.md) owns the operating protocol and failure fallback. +Homes without the opt-in, unverified versions, and sessions without the native registration keep [`codex-checkpoint.md`](../../../../../docs/supervision-protocols/codex-checkpoint.md), the unchanged bounded foreground protocol. +The [supervision host](../../../../../docs/supervision-host.md) retains its existing posture decisions on both paths. Codex's PreToolUse watcher-arm seatbelt blocks directly through its project hook. diff --git a/.agents/skills/harness-adapters/references/harness/hermes.md b/.agents/skills/harness-adapters/references/harness/hermes.md new file mode 100644 index 00000000000..5b15f366fc4 --- /dev/null +++ b/.agents/skills/harness-adapters/references/harness/hermes.md @@ -0,0 +1,37 @@ +# Hermes Agent + +Hermes is a spawned crewmate/scout TUI adapter on tmux only. +Primary and secondmate supervision, other runtime backends, desktop/web surfaces, and messaging bridges are unsupported and refused. +[Hermes verification](../../../../../docs/verification/hermes.md) owns dated vendor evidence and the live guard. + +## Operating facts + +| Fact | Value | +|---|---| +| Launch and profile | The interactive `hermes` CLI TUI, with native per-spawn profile selection; [`fm-spawn.sh`](../../../../../bin/fm-spawn.sh)'s header owns flags, omission, and relaunch mechanics. | +| Model and effort | The selected profile owns both axes. Firstmate refuses a non-default model override and records but omits effort. Native profile discovery is `hermes -p config get model --json` and `config get agent.reasoning_effort --json`; `/model` is the native interactive selection surface. No provider or catalog membership is inferred from a model name. | +| Busy and turn end | Native `pre_llm_call` and `on_session_end`, through [`fm-hermes-hook.sh`](../../../../../bin/fm-hermes-hook.sh) and the generation-bound busy writer; only the native `tui` platform is accepted, and the turn id binds the close across session-id rotation. | +| Hook carrier | A private native managed-scope overlay staged by [`fm-hermes-lib.sh`](../../../../../bin/fm-hermes-lib.sh); existing profile hooks are preserved, existing managed policy is refused, and no profile SOUL or configuration is edited. | +| Autonomy and hooks | The launch grants native tool autonomy and native shell-hook acceptance, including existing configured hooks. The startup guard confirms only a complete visible telemetry offer with `No thanks` already selected; native Hermes records that decline. Other first-run setup still requires a prepared profile. | +| Interrupt | Double Escape, only with a proven empty composer because the same chord discards drafts; idle double Escape preserves the worker. Ctrl+C is not used because it can exit an idle TUI. | +| Exit and relaunch | `/quit` exits to the pane shell. Relaunch starts from the stored task brief with the recorded profile and home; it does not attach to a gateway or resume an unrelated native session. | +| Identity | `FM_HERMES_HARNESS=hermes` at the launch boundary plus structural Hermes ancestry; [`fm-hermes-process-lib.sh`](../../../../../bin/fm-hermes-process-lib.sh) owns anchored Node TUI / Python module process identity. | +| Composer | Native `❯`, profile-prefixed or the default profile's unprefixed prompt, beneath a status rule; on tmux a pane showing `❯` is checked for Hermes identity before the generic cursor read, so only the Hermes classifier answers for Hermes panes; synthetic busy-hint cursor and bright drafts are distinguished by [`fm-composer-lib.sh`](../../../../../bin/fm-composer-lib.sh). Unrecognized themes, wraps, dialogs, or unreadable identity defer delivery. | +| Skills | Use ordinary text pointing at the installed skill's `SKILL.md`; native slash skill invocation has not been verified. | + +## Profile safety + +The selected profile's own running gateway is accepted only while idle: its native `gateway_state.json` must name the live gateway process, report `gateway_state: running` with no restart requested, `active_agents: 0` and no active work, and carry a heartbeat no older than Hermes's own 120-second staleness bound. +A mid-turn, starting, draining, or transitional gateway is refused, as is a live gateway, held native runtime lock, or active systemd unit without that idle status, and any unreadable or malformed gateway state. +A live default-root gateway, or a multiplexer launched from another home that serves the profile, is refused because its activity cannot be attributed to this profile. +An existing managed scope is refused before hooks are staged. +Firstmate does not stop, restart, or message the gateway. +The [verification record](../../../../../docs/verification/hermes.md) owns the idle-coexistence evidence and its limits. +The check is a launch preflight, not a lifetime reservation: the gateway may begin a turn after dispatch, and overlapping turns on one profile were not verified. + +## Limits + +The TUI's `--query-file` path did not submit the initial prompt on the verified version; the spawn uses the native interactive query path instead. +Early provider/build errors can leave the spawn-owned busy record open when the native end hook does not run; inspect the visible pane and use the ordinary recovery procedure. +User profile settings, plugins, tools, and inference credentials remain native Hermes concerns. +No primary Stop guard, pre-tool protection, watcher protocol, secondmate hook, or Herdr lifecycle was verified. diff --git a/.agents/skills/harness-adapters/references/harness/kiro.md b/.agents/skills/harness-adapters/references/harness/kiro.md new file mode 100644 index 00000000000..619d47ecc6e --- /dev/null +++ b/.agents/skills/harness-adapters/references/harness/kiro.md @@ -0,0 +1,65 @@ +# Kiro CLI (V2 engine) + +Kiro's `kiro-cli` TUI, verified end to end on 2026-09-13 with kiro-cli 2.21.4 on Amazon Linux 2 through tmux, on the V2 agent engine only. +It was re-verified on 2026-10-29 with kiro-cli 2.35.1 on Linux through tmux, with a Herdr composer probe, on Auto and on `glm-5`. +Verified as a CREWMATE and SCOUT adapter only; `../../../../../bin/fm-spawn.sh` refuses a secondmate launch on it because `../../../../../docs/supervision-protocols/` carries no kiro wake protocol. +`../../../../../docs/verification/kiro.md` owns how every fact below was established and what is still unproven. + +**V2 only.** The launch pins `--agent-engine v2`, the AL2-supported engine. The v3/KAS engine is out of scope and untested here: it is unsupported on this Amazon Linux 2 host and its hooks are not yet at parity, so carry no v3 behaviour into this adapter. + +## Operating facts + +| Fact | Value | +|---|---| +| Binary | Absolute `kiro-cli` from `PATH`, refused if absent; the installed command is a toolbox/aim-sandbox wrapper whose foreground process name is exactly `kiro-cli`. | +| Launch | `kiro-cli chat --agent-engine v2 --agent firstmate --model --effort --trust-all-tools ""` with the resolved absolute binary and `KIRO_HOME` relocated onto the per-task home; the positional brief auto-submits with no extra Enter. No launch-then-confirm gate: like claude, the pane just launches. | +| Busy state | Claude-shaped V2 agent-config hooks are the ONLY source: `userPromptSubmit` opens a turn and `stop` closes it, written as the `kiro-hook` record in `../../../../../bin/fm-busy-lib.sh`. A kiro task with no record classifies `unknown missing`; the classifier never reads the pane for kiro. | +| Rendered tail | Busy composer footer is `Kiro is working · · Type to steer · Ctrl+S to queue` (2.35.1 added the elapsed time); idle shows `Trust All Tools active ... /quit to exit` or the `ask a question or describe a task` placeholder. The harness-named `Kiro is working` literal is matched, never the bare `esc to cancel` token kiro shares with agy. It is a DELIVERY guard in `../../../../../bin/fm-composer-lib.sh`, never a worker state source, and its one reachable consumer is the harness-less union in `FM_DELIVERY_BUSY_REGEX_DEFAULT` that the tmux submit core reads to acknowledge a submit; `FM_DELIVERY_KIRO_BUSY_REGEX_DEFAULT` is registered per the fleet convention and has no caller today. | +| Turn end | The `stop` hook keeps the `state/.turn-ended` notification touch. Like claude, `stop` does NOT fire on a manual Escape interrupt; kiro V2 exposes no verified StopFailure/SessionEnd equivalent, so an abnormal turn end leaves the record busy until the next `userPromptSubmit`, and the supervisor reads that record as provably working. | +| Exit | `/quit`, one Enter; the process exits and prints `Session ended.` then `Resume with: kiro-cli --resume-id `. | +| Interrupt | Single `Escape`, which prints a `Cancelled ...` row and leaves an idle composer with no repollution, so no clear key follows. | +| Skill | No verified slash-skill form; use natural language. | +| Autonomy | `--trust-all-tools` auto-approves tool calls; its otherwise blocking confirmation modal is suppressed by the seeded `chat.disableTrustAllConfirmation` setting (see Trust below). | +| Marker | None; a live tool subprocess carries no `KIRO_*` identity variable. `KIRO_HOME` is a firstmate-set config-relocation path, not an identity. | +| Resume | The interactive-exit line prints `--resume-id `, and sessions live under the per-task `KIRO_HOME`, so `--resume-id` (or `--resume` for the most recent from the cwd) resumes with context. Firstmate does not automate resume: the control plane's deterministic `relaunch` re-reads the durable brief, matching every other adapter. | +| Model | `--model ` with a bare catalog `model_id` from `kiro-cli chat --agent-engine v2 --list-models -f json` (for example `claude-opus-5`, `auto`); `bin/fm-spawn.sh` refuses a requested id a reachable listing omits. The listing is a remote fetch, so the probe runs stdin-detached under the shared hard bound and an unreachable, hung, or `model_id`-less listing launches unvalidated with a notice. | +| Effort | `--effort low\|medium\|high\|xhigh\|max`; the full shared vocabulary passes through (unlike agy, which omits `xhigh`/`max`). | +| Composer | Bare `›` (U+203A) agent-glyph row, the same glyph codex draws. The `ask a question or describe a task` placeholder is near-gray at luminance 158, drawn as truecolor `38;2;158;158;158` on a truecolor pane and as the palette index `38;5;247` - the identical grey - on a pane with no `COLORTERM`. The shared ghost strip removes both under a near-achromatic ceiling it applies only when the caller names the recorded harness `kiro` (`FM_COMPOSER_HARNESS`, owned by `fm_composer_kiro_scope` in `../../../../../bin/fm-composer-lib.sh`), so a real idle pane reads `empty` in either encoding (verified against a live capture with tmux's own descriptor). Before that ceiling existed the placeholder survived the strip and the pane read `pending`, which suppressed every steer; `../../../../../docs/verification/kiro.md` owns that history. kiro-cli 2.28.0 draws the same row in the default foreground, which no strip removes, so the kiro-scoped `FM_COMPOSER_HINTED_IDLE_RE_DEFAULT` matches the whole row by its `↵` submit hint, which typed input never renders, and reads it `empty`; a cursorless read also stops there, so the `/copy to clipboard` footer below it is never taken as wrapped input. kiro registers no entry in the fleet-wide idle-placeholder set, which changes no verdict here. | + +## Out-of-tree agent config (the turn-end wiring) + +`--agent` is name-only: a config path is rejected, and discovery is the global `KIRO_HOME/agents/` dir plus the workspace `/.kiro/agents/` dir. +The workspace dir is inside the disposable worktree and must never be written, so `../../../../../bin/fm-spawn.sh` writes a firstmate-owned per-task agent config at `state/.kiro-home/agents/firstmate.json` and reaches it by relocating `KIRO_HOME` onto `state/.kiro-home` on the launch command, the gemini shape (a dedicated per-task config outside the project, reached through a config-pointer). +That same per-task home holds `settings/cli.json` seeding `chat.disableTrustAllConfirmation`, which suppresses the `--trust-all-tools` modal (the only blocker on a fresh launch). +Relocating `KIRO_HOME` moves the whole global config root (agents, settings, sessions), and on Amazon Linux 2 it leaves auth alone because auth lives in the XDG data dir (`~/.local/share/kiro-cli`), so the worker uses the operator's real sign-in while nothing in the captain's real `~/.kiro` is touched. +That path does not exist on macOS and where auth lives there is unestablished, so the untouched-by-relocation premise does not carry to macOS; see Credential precondition below. +Because the relocation is wholesale rather than gemini's additive single-file layer, the per-task home does not inherit the operator's global MCP servers, skills, or agents; the per-task agent config declares `tools: ["*"]` and the crewmate relies on built-in tools. +Each hook `command` in that config is a single-token absolute path to a script the spawn generates beside it under `state/.kiro-home/hooks/`, so a hook runs the same whether kiro execs the command directly or hands it to a shell; the busy event, its `|| true` tolerance of a refused stale generation, and the turn-end touch all live inside the script. +No single token can carry whitespace or a shell-special character (quotes, `$`, backtick, `;`, `&`, `|`, `<`, `>`, parentheses, `*`, `?`, brackets, braces, `!`, backslash) under both interpretations, so a canonical kiro spawn refuses before launch when the resolved `state/.kiro-home` path contains one, naming that path and asking for a `FM_HOME` or `FM_STATE_OVERRIDE` free of them; without the hooks kiro has no other state source, so a pane that could never clear its busy record is never started. +`../../../../../bin/fm-control-lib.sh` lists the agent config file and both hook scripts as retirement paths so a relaunch retires the incarnation's hooks; `../../../../../bin/fm-teardown.sh` removes the whole `state/.kiro-home` directory. + +## Credential precondition + +A verified kiro worker ran under a signed-in account with no key export, on Amazon Linux 2. +macOS is unestablished: the Amazon Linux 2 auth path does not exist there, and where kiro carries auth on macOS is not something this adapter can determine, because determining it means probing a credential store that belongs to the operator. +So a kiro worker spawned on macOS may hit an interactive auth prompt, which makes the harness unusable unattended on that platform; `../../../../../docs/verification/kiro.md` owns the gap and its cost. +Treat any auth prompt or refusal as a credential blocker under `../../../../../AGENTS.md` section 9, fix the environment, and retire the endpoint rather than typing into it. + +## Detection + +Detected by ancestry alone: `../../../../../bin/fm-harness.sh` matches the anchored process name `kiro-cli`, never `*kiro*`. +No environment marker is promoted, and kiro does not clear an inherited `CLAUDECODE`, so the spawn clears foreign markers at the launch boundary and the ancestry arm decides. +kiro is deliberately absent from the primary-capable session-lock vocabulary in `../../../../../bin/fm-session-lock-lib.sh`, where the other crewmate-only adapters are also absent: a crewmate-only adapter must never own a home session lock. + +## Worker busy state and turn end + +`../../../../../bin/fm-spawn.sh` arms a busy generation for kiro and embeds it in the hook commands, exactly like claude and gemini. +`fm_busy_classify` returns the `kiro-hook` record when a valid one exists and `unknown missing` when none does; it has no kiro pane arm, so a rendered footer never classifies a kiro worker. +An abnormal turn end therefore leaves the record busy until the next `userPromptSubmit` re-opens it, and the watcher reads that record as provably working. +Teardown removes the per-task home and retires the busy generation. + +## Primary integration + +Unsupported and unverified. +`../../../../../docs/supervision-protocols/` carries no kiro protocol, no turn-end guard adapter exists for it, and this adapter verified only the crewmate-side launch, busy state, interrupt, exit, and resume on the V2 engine. +`references/common/primary-hooks.md`'s unsupported-boundary rule applies: never invent a wake protocol from a similar TUI. diff --git a/.agents/skills/harness-adapters/references/harness/omp.md b/.agents/skills/harness-adapters/references/harness/omp.md index 13721916513..e475698504b 100644 --- a/.agents/skills/harness-adapters/references/harness/omp.md +++ b/.agents/skills/harness-adapters/references/harness/omp.md @@ -54,4 +54,3 @@ omp has no asynchronous Stop-hook equivalent, so the Claude auto-arm model does The Pi supervision branch does not run on omp; without the supervision host every actionable wake is delivered to main, and in a home with `config/supervision-host` and no `config/supervision-host-off` the watch extension spawns the host instead of the arm, with Claude's print mode as its headless engine ([`supervision-host.md`](../../../../../docs/supervision-host.md)). Launch a primary with plain `omp` inside the home (`FM_OMP_HARNESS=omp omp` when starting from a Claude pane); `../../../bin/fm-session-start.sh` prints `OMP_WATCH_EXTENSION: not loaded` when the running session has not loaded both tracked extensions. `FM_OMP_LIVE_E2E=1 ../../../tests/fm-omp-primary-live-e2e.test.sh` is the opt-in live guard; `../../../tests/fm-omp-harness.test.sh` is the portable regression. -A secondmate registered with `remote=1` in `data/secondmates.md`, spawned through the ordinary `../../../bin/fm-spawn.sh --secondmate` path, is refused on omp until a remote host verifies it, as is `../../../bin/fm-remote-secondmate-control.sh launch`; there is no `--remote` flag. diff --git a/.agents/skills/harness-adapters/references/harness/pi.md b/.agents/skills/harness-adapters/references/harness/pi.md index 3a84c2f8b44..2e00655b489 100644 --- a/.agents/skills/harness-adapters/references/harness/pi.md +++ b/.agents/skills/harness-adapters/references/harness/pi.md @@ -40,6 +40,7 @@ For unattended seeded-secondmate launches, `../../../bin/fm-spawn.sh --help` own `../../../bin/fm-spawn.sh` keeps the worker turn-end extension in `state/`, outside the worktree, because project-local extension files worsen the trust gate and pollute the project. The extension listens for Pi's `turn_end` event, not `agent_end`, so supervision is notified after each completed turn rather than only when the whole run exits. Native-harness progress uses the separate generation-bound marker owned by `../../../bin/fm-busy-event.sh`; it never fabricates Pi turn completion. +Optional foreground model-stream observation is version-gated separately; `../../../docs/configuration.md` (Environment variables) owns activation and `../../../docs/verification/runtime-backends.md` (Pi worker model-stream observation) owns the real-Pi evidence. Pi sets `PI_CODING_AGENT=true` for its children as its harness-detection marker. ## Primary integration diff --git a/.agents/skills/operational-home-layout/SKILL.md b/.agents/skills/operational-home-layout/SKILL.md index ffe330f6f34..237d89a1362 100644 --- a/.agents/skills/operational-home-layout/SKILL.md +++ b/.agents/skills/operational-home-layout/SKILL.md @@ -53,7 +53,7 @@ config/herdr-lifecycle-spawn-guard optional presence flag opting this home into config/fleet-ledger optional presence flag opting this home in to the default-off fleet activity ledger state/fleet-ledger.jsonl that outside tools can follow; LOCAL, gitignored, and not inherited; see docs/fleet-ledger.md config/firstmate-role optional LOCAL, gitignored firstmate-repository publication role; see docs/configuration.md "Firstmate role" for operator opt-in and secondmate inheritance config/wait-no-turns optional presence flag opting this home into default-off waiting-worker behavior (brief waiting section, foreground pipeline drive, pending-reply hold, one fire-and-forget retry ring); LOCAL, gitignored, and not inherited; see docs/configuration.md "Waiting worker spends no turns" -config/pr-health-monitor optional presence flag opting this home into the default-off monitor of its owned PRs, which wakes once per condition per latest commit on a definite merge conflict or a failing required check and never changes a PR; LOCAL, gitignored, and not inherited; see docs/configuration.md "PR health monitor" +config/pr-health-monitor optional presence flag opting this home into the default-off monitor of its owned PRs; LOCAL, gitignored, and not inherited; see docs/configuration.md "PR health monitor" config/contribution-review-bots optional exact GitHub-login allowlist for review-bot signals on owned pull requests; LOCAL, gitignored, and not inherited; see docs/configuration.md "Contribution review bots" config/pipeline-spend optional presence flag opting this home in to default-off per-task no-mistakes spend recording in data/pipeline-spend.jsonl; LOCAL, gitignored, and not inherited; see docs/configuration.md "No-mistakes pipeline spend" config/nm-home-resolver optional one-line absolute path of a site-provided executable that maps a task copy to its no-mistakes home for host-side reads when NM_HOME is unset; LOCAL, gitignored, and not inherited; see docs/configuration.md "No-mistakes home resolver" @@ -70,6 +70,7 @@ config/spawn-base-primary-checkout optional presence flag opting this home into config/cmux-socket-password optional cmux control-socket password; LOCAL, gitignored; read fresh on every cmux CLI call and passed through without ever overriding an operator's own ambient CMUX_SOCKET_PASSWORD when absent (docs/cmux-backend.md "Setup") config/wedge-alarm optional away-mode wedge-alarm active-alert directives; LOCAL, gitignored; absent means auto (macOS Notification Center when available); see docs/wedge-alarm.md config/watched-tools.json optional list of the tools this home depends on, read by the update check armed with bin/fm-tool-update-check.sh; LOCAL, gitignored, firstmate-maintained but human-editable, and NOT inherited by secondmate homes; see docs/configuration.md "Watched tool updates" +config/self-update-source optional /updatefirstmate source override; LOCAL, gitignored, and not inherited; an unusable file stops the update, so report it to the captain and never edit or remove it; see docs/configuration.md "Self-update source" config/watch.env optional home-local watcher tuning as exported shell assignments; LOCAL, gitignored, never removed by bootstrap; loaded by bin/fm-watch-arm.sh before x-mode.env; see docs/configuration.md "Relay (.env)" config/usage-warner optional Claude usage-window thresholds for the opt-in macOS threshold warner armed with bin/fm-usage-warner.sh; LOCAL, gitignored, and NOT inherited by secondmate homes; see docs/configuration.md "Local Claude usage warner" config/x-mode.env generated Relay watcher cadence; LOCAL, gitignored; source before arming watcher when present @@ -90,12 +91,15 @@ state/ runtime records and signals; gitignored .turn-ended touched by turn-end hooks .progress touched for observed native-harness activity inside one Pi turn; bin/fm-busy-event.sh owns its generation binding and bin/fm-watch.sh reads it beside turn-ended for the busy-age bound only, never as a completed turn .busy-state .busy-gen semantic busy-state record (one line, atomically replaced) and its per-incarnation gen sidecar; bin/fm-busy-event.sh is the only writer and bin/fm-busy-lib.sh owns the record format and classification; arming again replaces the previous incarnation so late events carrying its gen are rejected as stale; removed by retire and teardown + .model-stream .model-stream-order .model-progress opt-in Pi foreground model-stream evidence, the request ordering record that admits only a strictly newer open and fences progress and close to the request it names even after that evidence clears, and the separate normal-model progress anchor the opted-in busy-age bound reads; written only by bin/fm-busy-event.sh, with bin/fm-busy-lib.sh owning the record format and trust; removed by arm, retire, and teardown; docs/configuration.md "Environment variables" owns activation .grok-turnend-token firstmate-owned grok hook registry token for the task; removed by teardown .kimi-turnend-token firstmate-owned Kimi hook registry token for the task; removed by teardown .gemini-settings.json firstmate-owned per-task Gemini settings carrying the busy-state and turn-end hooks, reached through GEMINI_CLI_SYSTEM_SETTINGS_PATH so nothing is written into the project's own .gemini/; removed by teardown .claude-settings.json firstmate-owned per-task Claude settings for a secondmate (busy-state hooks plus the session keys), passed through --settings so the mate home's own .claude/settings.local.json is never edited; written by fm-spawn; removed by relaunch retirement and teardown .devin-config.json firstmate-owned per-task Devin config (mode 600 snapshot of the user config plus the busy-state and turn-end hooks) passed through --config so no user or project config is edited; bin/fm-devin-config.sh owns it; removed by teardown .cortex-plugin/ firstmate-owned per-task Cortex Code plugin directory (mode 700; busy-state hooks and Herdr pane registration) passed through --plugin-dir so no user or project config is edited; bin/fm-cortex-plugin.sh owns it; removed by teardown + .kiro-home/ firstmate-owned per-task Kiro CLI home, reached by relocating KIRO_HOME, holding the agent config whose hook scripts carry the busy-state and turn-end wiring plus the trust setting and this task's sessions, so nothing is written into the project's own .kiro/; removed whole by teardown + .hermes-hooks/ firstmate-owned private Hermes managed-scope overlay carrying the busy-state and turn-end hooks, passed through HERMES_MANAGED_DIR so no profile SOUL or configuration is edited; bin/fm-hermes-lib.sh owns it; removed by teardown .muse-session muse busy-source binding (sessions root plus task worktree) written by fm-spawn; removed by teardown .cursor-session cursor busy-source binding (projects root, task worktree, prior conversations) written by fm-spawn; removed by teardown .git-hooks/ per-task git hooksPath for the commit attribution policy and push guards; written by fm-spawn, removed by teardown (bin/fm-git-strip-ai-trailers.sh); docs/configuration.md "Commit attribution" owns selection @@ -126,7 +130,7 @@ state/ runtime records and signals; gitignored .mail-seen .mail-woken .mail-retry .mail-retry-pos .mail-turn .mail-seen.lock mail-plane poll cursor, emission journal, transient-fetch retry set, retry-scan position, contended-slot turn flag, and overlapping-poll lock; written only by bin/fm-mail.sh (mail schema: docs/configuration.md "Mail plane") startup-growth.check.sh generated daily startup-growth poll shim with its .check-trust binding and .startup-growth-check report record; see docs/configuration.md "Daily startup growth check" for lifecycle and record-reset consequences pr-health.check.sh generated owned-PR health poll shim and its .check-trust binding; present only after bin/fm-pr-health.sh arm, which config/pr-health-monitor gates (docs/configuration.md "PR health monitor") - pr-health.json the health monitor's fm-pr-health.v1 record: per-PR fingerprints of the conditions already reported and the task branch-to-PR lookup cache, replaced by rename under .pr-health.lock; bin/fm-pr-health.sh owns the format + pr-health.json the health monitor's fm-pr-health.v1 record: per-PR fingerprints of the conditions already reported, the task branch-to-PR lookup cache, and the GitHub notifications cursor, replaced by rename under .pr-health.lock; bin/fm-pr-health.sh owns the format pending-replies/ parent-owned secondmate pending-reply records (correlation id, delivery vs reply, recovery, escalation); fm-pending-reply-lib.sh t3code-sends/ T3 send request-id retention and accepted delivery outcomes; docs/t3code-backend.md "Current lifecycle and safety" owns the window and files procevent/ registered process-to-event sources, one private record per canonical source id; written only by bin/fm-procevent.sh, and their presence alone keeps supervision required (`process-event-sources` skill) @@ -156,7 +160,7 @@ state/ runtime records and signals; gitignored .rex-session.lock Rex backend lock serializing a spawn's or reclaim's session choice and tab creation against killing an emptied per-home session; bin/backends/rex.sh owns it; never touch .claude-autoarm.lock .claude-autoarm-epoch .claude-autoarm-failure-notified .claude-autoarm-failure-alarmed .turnend-claude-blocks .turnend-claude-blocks.lock Claude Stop auto-arm single-flight, epoch, failure-episode, attended-alarm, guard-budget, and budget-lock records; never touch .cursor-park-owner .cursor-park-owner.lock .turnend-cursor-blocks Cursor stop-hook owner record, publication and commit lock, and bounded repair-nag budget; never touch - .hash-* .count-* .stale-* .stale-since-* .churn-since-* .paused-* .wedge-escalations-* .wedge-permanent-* .wedge-terminal-queued-* .wedge-rollback-failed-* .dead-reported-* .writing-* .validating-* .guard-deferrals-* .waiting-* .seen-* .hb-surfaced-* .last-* .heartbeat-streak .secondmate-liveness-tick .secondmate-liveness-*.lock* watcher internals; never touch. The three wedge-cap markers are written only while FM_WEDGE_MAX_ESCALATIONS is above 0, and are safe to delete (rm STATE/.wedge-permanent- is the immediate-re-engagement escape hatch; docs/configuration.md owns that cap's release boundary): .wedge-permanent- silences that window for FM_CAP_HORIZON_SECS, .wedge-rollback-failed- silences it for FM_ROLLBACK_SENTINEL_TTL_SECS, and .wedge-terminal-queued- marks the one episode that already queued its terminal row; a cap release drops all three, and only the first two re-engage a window when deleted + .hash-* .count-* .stale-* .stale-since-* .churn-since-* .paused-* .wedge-escalations-* .wedge-permanent-* .wedge-terminal-queued-* .wedge-rollback-failed-* .model-stream-episode-* .dead-reported-* .writing-* .validating-* .guard-deferrals-* .waiting-* .seen-* .hb-surfaced-* .last-* .heartbeat-streak .secondmate-liveness-tick .secondmate-liveness-*.lock* watcher internals; never touch. The three wedge-cap markers are written only while FM_WEDGE_MAX_ESCALATIONS is above 0, and are safe to delete (rm STATE/.wedge-permanent- is the immediate-re-engagement escape hatch; docs/configuration.md owns that cap's release boundary): .wedge-permanent- silences that window for FM_CAP_HORIZON_SECS, .wedge-rollback-failed- silences it for FM_ROLLBACK_SENTINEL_TTL_SECS, and .wedge-terminal-queued- marks the one episode that already queued its terminal row; a cap release drops all three, and only the first two re-engage a window when deleted .secondmate-relaunch- .secondmate-relaunch-bound- durable relaunch history and parked-bound state; never touch (bin/fm-secondmate-liveness-lib.sh owns the ledger contract) .watch-triage.log watcher's absorbed-wake debug log (size-capped); never relied on, safe to delete lane-capacity.json this home's published lane facts, refreshed only while config/lane-capacity exists; written only by bin/fm-capacity.sh --publish, whose header owns the document diff --git a/.agents/skills/quota-array-dispatch/SKILL.md b/.agents/skills/quota-array-dispatch/SKILL.md index e25c52e0e50..a8b18c89c91 100644 --- a/.agents/skills/quota-array-dispatch/SKILL.md +++ b/.agents/skills/quota-array-dispatch/SKILL.md @@ -16,6 +16,14 @@ This skill is the single owner of the completion-aware profile-array selection p Do not add a daemon, opaque composite score, routing wrapper, hard-coded model-specific policy, or producer-side route recommendation. The [worker helper](../../../bin/fm-quota-choose.sh) and [typed resolver](../../../docs/configuration.md#typed-dispatch-resolution-env-fm_dispatch_provider-fm_dispatch_model) own their deterministic mapping boundaries. +## Shared-quota opt-out + +When the home's `config/crew-dispatch.json` declares a top-level `"shared_quota": true`, this home draws every provider's usage from one single shared budget, so per-provider quota rows do not describe what is left, and this skill's quota snapshot, the quota parts of the eligibility and runway gates, and `spendPriority` ranking are off by home declaration. +Make no `quota-axi` call, apply no quota row, quota floor, runway gate, or `spendPriority` rank, and choose the matched rule's first declared profile that its authoritative catalog supports and that meets the required reasoning class; catalog eligibility and reasoning-class fit still apply. +`preferred_provider` is not a quota signal, so it still applies: `preferred_provider` still narrows the pool; declared order then picks within it. +Account for every candidate as always, stating in place of quota evidence that quota ranking is off by home declaration. +The declared value must be a boolean; anything else is an actionable configuration error. + ## Worker-side quota helper The canonical shell helper for a worker that has already performed its model-selection reasoning and now needs to pick the first viable candidate is `bin/fm-quota-choose.sh`. @@ -35,6 +43,7 @@ It never removes this skill's authority, and its `ambiguous`, `escalate`, and `e ## Read the default TOON +When the home declares the [shared-quota opt-out](#shared-quota-opt-out), skip this section's quota reading and every quota-derived check built on it (quota rows, floors, runway, and `spendPriority`); catalog eligibility and reasoning-class fit still apply. Start each intake by running `quota-axi` once with no `--json`, and reuse that TOON for every candidate. Post-consolidation quota-axi (the floor owned by `bin/fm-quota-axi-lib.sh`) puts `spendPriority` in the default `quota[]` block beside `effectivePercentRemaining`, `runway`, `confidence`, `limitedBy`, and `resetsAt`. Sparse `exhaustion[]` carries finite-runway seconds only for `projected_exhaustion` and `exhausted_now`. @@ -68,6 +77,7 @@ A model the catalog does not list is concrete contradictory evidence: block that Apply quota at the granularity the vendor actually supplies. A provider-level or `all_models`/`all_products` scope bounds every model you established in that family within the candidate's matched account, including one with no window of its own. A named-model or named-product scope is an additional bound for that model alone. +A scope the vendor reports outside those names, such as an Antigravity tier, bounds a candidate only when its profile declares that exact scope as `quota_scope`; never infer the tier from the model name. Match the candidate to its `quota[]` row by that established provider, its `accountKey` when the snapshot is schema 6 (a Pi lane's auth provider id such as `openai-codex-work`, or `codex-home` for native Codex including Pi's `codex-native/` adapter, then the `default` row, else unmeasured; never a row picked by position, never rows summed across accounts), and scope; a stale, auth-required, or unmeasurable scope is named in `attention[]` instead of a fabricated number. For a Codex profile with a named `seat`, resolve that named seat with `bin/fm-dock.sh resolve --seat --harness codex` on the snapshot's host and match exactly one row whose credential home, with a leading `~/` expanded to that host's OS user home, is its resolved physical directory plus `/auth.json`; each seat maps to its own quota account, and if no unique row exists, report it as unmeasured rather than using the ambient or another seat's Codex account. Pass that same directory to `fm-spawn.sh` as `--seat-home` with the selected `--seat`, so a changed dock binding refuses the launch instead of starting under a home that was not measured. diff --git a/.agents/skills/secondmate-provisioning/SKILL.md b/.agents/skills/secondmate-provisioning/SKILL.md index 70cd7709a17..1cd9cb6834d 100644 --- a/.agents/skills/secondmate-provisioning/SKILL.md +++ b/.agents/skills/secondmate-provisioning/SKILL.md @@ -32,7 +32,7 @@ Each registry entry stays concise and single-line: the summary is one sentence n Natural-language summary and `scope:` text may contain parentheses and semicolons; keep the generated `(home: ...; scope: ...; projects: ...; added ...)` suffix intact so operational consumers resolve its explicit field markers. The `home:` path points to the seeded home containing `data/charter.md`; no extra registry pointer field is needed. For a remote route, `host:` is an OpenSSH config alias and `root:` is that host's separate tracked Firstmate code root. -A remote second-mate agent always runs on the Herdr backend and every seed, launch, and liveness relaunch first gates its host on `bin/fm-remote-doctor.sh` readiness, so an unready host refuses with that doctor's own gap text rather than half-creating a route; the workers that second mate supervises keep the home's ordinary backend selection. +See [remote launch and recovery](../../../docs/remote-secondmates.md#launch-or-recover) for backend selection and readiness (Herdr remains the default; Rex requires explicit selection and a ready doctor verdict). This release places whole secondmate homes remotely and never individual workers. [`docs/remote-secondmates.md`](../../../docs/remote-secondmates.md) owns current operator setup and transport behavior. The home-seeded `data/charter.md` is the sole owner of boilerplate idle-by-default behavior, the normal delegation lifecycle, and standard escalation contracts, so point to that charter rather than restating those contracts in the registry entry. @@ -89,7 +89,7 @@ Select the registered mate's id when resolving its pin; [`bin/fm-harness.sh`](.. This section is the single owner of the secondmate sync and inherited-local-material propagation contract; `AGENTS.md` sections 3 and 4 point here. Before a local launch, `fm-spawn.sh --secondmate` locally fast-forwards the home to the primary firstmate checkout's current default-branch commit when it is safe, or reconciles a clean divergence whose complete local result is already present there (e.g. after a squash merge) with `reset --keep`; dirty, uniquely diverged, or in-flight homes launch unchanged with a warning, and a genuine divergence gets the same durable reconciliation record `bin/fm-ff-lib.sh` writes for `/updatefirstmate`. The locked session-start deferred network stage runs the same bootstrap sweep for every live local secondmate home, discovered from `state/.meta` records with `kind=secondmate` (`data/secondmates.md` only backfills `home=` for older records). -That no-fetch path is a purely local fast-forward or redundant-divergence reconcile of tracked files, never an origin fetch, and it never touches the gitignored operational dirs, so a secondmate's backlog, projects, and in-flight work are never disturbed; a linked worktree advances immediately, while a standalone clone that lacks the target receives firstmate updates through `/updatefirstmate`'s origin refresh. +That no-fetch path is a purely local fast-forward or redundant-divergence reconcile of tracked files, never an origin fetch, and it never touches the gitignored operational dirs, so a secondmate's backlog, projects, and in-flight work are never disturbed; a linked worktree advances immediately, while a standalone clone that lacks the target receives firstmate updates through `/updatefirstmate`. A remote launch and the deferred bootstrap sweep hand the configured host the primary's own default-branch commit and ask it to fast-forward, or reconcile a redundant divergence, the persistent home to exactly that commit, under the same clean, ancestry, and branch guards a local home gets. A remote home is a standalone clone on another machine, so that host imports the one commit it was given - already present, else from that host's own Firstmate copy without moving it, else from the home's origin - and skips with an actionable reason when none of them holds it, which is what an unpushed primary commit looks like from there. Neither path moves the host's Firstmate copy, and the host-local launch never re-targets that copy after the parent has already synced the home. diff --git a/.agents/skills/ship-landing/SKILL.md b/.agents/skills/ship-landing/SKILL.md index 29f845a05df..7575427cf36 100644 --- a/.agents/skills/ship-landing/SKILL.md +++ b/.agents/skills/ship-landing/SKILL.md @@ -10,6 +10,7 @@ metadata: For PR-based ship tasks, the ready signal depends on mode: `no-mistakes` reports `done [at=]: PR checks green` after CI is green, while `direct-PR` reports `done [at=]: PR ` after opening the PR, each only for a non-draft PR; a lane that deliberately holds a draft declares a wait instead, and `bin/fm-pr-check.sh` refuses to arm merge monitoring on a draft. Run `bin/fm-pr-check.sh ` with the URL copied from that ready signal or the resolved checks-green `fm-crew-state.sh` line - it records `pr=` and the forge's `pr_head=` when available in the task's meta and arms the watcher's merge poll. +With `config/pr-register-scan` present, the watcher's check sweep also records a PR that a `done:` line names when no `pr=` exists, and reports a refused registration as a `check: pr-register:` notification; handle that refusal as you would a refusal from a manual run. `bin/fm-dod-lib.sh` owns the named-head gate on that ready signal: a ship `done:` whose named head exists only in the worker's disposable copy is not ready (`bin/fm-crew-state.sh` reports blocked, `bin/fm-pr-check.sh` refuses to register, and a secondmate does not publish that done upstream). That blocked reading is the gate working, not a stuck worker, so steer the worker on the commit the refusal names rather than waiting. A `forge=external` task reports `done [at=]: review uploaded for review` instead: run `bin/fm-review-check.sh ` rather than `bin/fm-pr-check.sh`, which arms no merge watch, then always escalate the review URL to the captain whatever the yolo setting, because firstmate can neither observe nor perform its approval or merge. diff --git a/.agents/skills/stuck-crewmate-recovery/SKILL.md b/.agents/skills/stuck-crewmate-recovery/SKILL.md index ede521355ff..5d3c0172d82 100644 --- a/.agents/skills/stuck-crewmate-recovery/SKILL.md +++ b/.agents/skills/stuck-crewmate-recovery/SKILL.md @@ -39,7 +39,7 @@ Read the targeted current state with `bin/fm-crew-state.sh ` before deciding A no-mistakes run matched to the crew's branch and current code remains authoritative when the endpoint is dead: handle a terminal or parked run through the normal lifecycle, and keep supervising an active run instead of creating a duplicate worker. When no authoritative run accounts for the task, inspect only its recorded backend and worktree inventory. -Use `treehouse status` for treehouse-backed tmux, herdr, zellij, cmux, Paseo, Rex, t3code, or tern tasks, and use the recorded `orca_worktree_id=` and `terminal=` for Orca tasks. +Use `treehouse status` for treehouse-backed tmux, herdr, zellij, cmux, Paseo, Rex, t3code, tern, or hc tasks, and use the recorded `orca_worktree_id=` and `terminal=` for Orca tasks. Do not sweep another home's endpoints or infer ownership from a matching window label. Before relaunch, prove that no live agent still owns the recorded task and that the existing worktree remains available. diff --git a/.agents/skills/updatefirstmate/SKILL.md b/.agents/skills/updatefirstmate/SKILL.md index eb7a204cdfd..5b83a4926ac 100644 --- a/.agents/skills/updatefirstmate/SKILL.md +++ b/.agents/skills/updatefirstmate/SKILL.md @@ -1,7 +1,7 @@ --- name: updatefirstmate description: >- - Self-update a running firstmate and its secondmates to the latest from origin. + Self-update a running firstmate and its secondmates to the latest from origin, or from a configured self-update source. Use when the captain invokes /updatefirstmate (e.g. "/updatefirstmate", "update firstmate", "pull the latest firstmate"). user-invocable: true metadata: @@ -28,6 +28,7 @@ The only live mates that do not restart are the ones whose home the update pass **One-time rollout note:** the update that carries this change is still executed by the previous release, which restarts only the mates whose `AGENTS.md` or `.agents/skills/` moved on that pass. After it completes, run `bin/fm-secondmate-restart.sh ...` once with every live second mate ID, not only the ones that release named; later updates follow the normal flow below. +The primary's source is its runtime branch's tracking remote unless this home has `config/self-update-source`, which names one remote branch for the primary to follow instead; secondmates follow the primary's resulting commit either way, and [`docs/configuration.md`](../../../docs/configuration.md#self-update-source-configself-update-source) owns that contract. The primary update is fast-forward only, while each secondmate uses the same guarded convergence path plus one narrow recovery for squash-merged local history. By default, the runtime branch remains the origin/HEAD branch with the historical main/master fallback. Set the validated repository git config key `firstmate.runtimeBranch` to name a local runtime branch such as `house`; its configured branch remote and merge ref supply the update target. @@ -46,8 +47,9 @@ This touches only the firstmate repo and its own worktrees, never anything under ```sh bin/fm-update.sh ``` - It fast-forwards this firstmate repo's configured runtime branch from its tracking remote, then updates every registered local or remote secondmate home to the resulting pinned commit through its placement-specific guarded path. + It fast-forwards this firstmate repo's configured runtime branch from its tracking remote, or from the configured self-update source, then updates every registered local or remote secondmate home to the resulting pinned commit through its placement-specific guarded path. If the primary update is skipped, propagation stops. + When a source is configured it prints a `self-update source:` line first, and an unusable config file stops the whole update before anything moves; report that file to the captain rather than editing or removing it. It prints one status line per target (`updated ..` / `reconciled redundant divergence ..` / `already current` / `skipped: `), followed by three action lines that tell you exactly what to do next: - `reread-firstmate: yes|no` - `restart-secondmates: fm-...|none` diff --git a/.codex/hooks.json b/.codex/hooks.json index 92c5093a575..9c68bb2c91c 100644 --- a/.codex/hooks.json +++ b/.codex/hooks.json @@ -5,7 +5,7 @@ "hooks": [ { "type": "command", - "command": "bash -lc 'payload=$(cat 2>/dev/null || true); [ -n \"$payload\" ] || exit 0; command -v jq >/dev/null 2>&1 || exit 0; root=$(pwd -P) || exit 0; [ -x \"$root/bin/fm-sessionstart-run.sh\" ] || exit 0; [ -f \"$root/AGENTS.md\" ] || exit 0; [ -f \"$root/.codex/hooks.json\" ] || exit 0; jq -e \"any(.hooks.SessionStart[]?.hooks[]?.command?; type == \\\"string\\\" and contains(\\\"fm-sessionstart-run.sh\\\"))\" \"$root/.codex/hooks.json\" >/dev/null 2>&1 || exit 0; printf \"%s\" \"$payload\" | \"$root/bin/fm-sessionstart-run.sh\"'", + "command": "bash -lc 'payload=$(cat 2>/dev/null || true); [ -n \"$payload\" ] || exit 0; command -v jq >/dev/null 2>&1 || exit 0; root=$(pwd -P) || exit 0; [ -x \"$root/bin/fm-sessionstart-run.sh\" ] || exit 0; [ -f \"$root/AGENTS.md\" ] || exit 0; [ -f \"$root/.codex/hooks.json\" ] || exit 0; jq -e \"any(.hooks.SessionStart[]?.hooks[]?.command?; type == \\\"string\\\" and contains(\\\"fm-sessionstart-run.sh\\\"))\" \"$root/.codex/hooks.json\" >/dev/null 2>&1 || exit 0; if [ -x \"$root/bin/fm-codex-stop-park.sh\" ]; then printf \"%s\" \"$payload\" | \"$root/bin/fm-codex-stop-park.sh\" --session-start; else printf \"%s\" \"$payload\" | \"$root/bin/fm-sessionstart-run.sh\"; fi'", "timeout": 180 } ] @@ -33,8 +33,14 @@ "hooks": [ { "type": "command", - "command": "bash -lc 'payload=$(cat 2>/dev/null || true); [ -n \"$payload\" ] || exit 0; command -v jq >/dev/null 2>&1 || exit 0; root=$(pwd -P) || exit 0; [ -x \"$root/bin/fm-turnend-guard.sh\" ] || exit 0; [ -f \"$root/AGENTS.md\" ] || exit 0; [ -f \"$root/.codex/hooks.json\" ] || exit 0; jq -e \"any(.hooks.Stop[]?.hooks[]?.command?; type == \\\"string\\\" and contains(\\\"fm-turnend-guard.sh\\\"))\" \"$root/.codex/hooks.json\" >/dev/null 2>&1 || exit 0; printf \"%s\" \"$payload\" | \"$root/bin/fm-turnend-guard.sh\"'", + "command": "bash -lc 'payload=$(cat 2>/dev/null || true); [ -n \"$payload\" ] || exit 0; command -v jq >/dev/null 2>&1 || exit 0; root=$(pwd -P) || exit 0; [ -x \"$root/bin/fm-turnend-guard.sh\" ] || exit 0; [ -f \"$root/AGENTS.md\" ] || exit 0; [ -f \"$root/.codex/hooks.json\" ] || exit 0; jq -e \"any(.hooks.Stop[]?.hooks[]?.command?; type == \\\"string\\\" and contains(\\\"fm-turnend-guard.sh\\\"))\" \"$root/.codex/hooks.json\" >/dev/null 2>&1 || exit 0; if [ -x \"$root/bin/fm-codex-stop-park.sh\" ]; then printf \"%s\" \"$payload\" | \"$root/bin/fm-codex-stop-park.sh\" --guard; else printf \"%s\" \"$payload\" | \"$root/bin/fm-turnend-guard.sh\"; fi'", "timeout": 30 + }, + { + "type": "command", + "command": "bash -lc 'root=$(pwd -P) || exit 0; [ -x \"$root/bin/fm-codex-stop-park.sh\" ] || exit 0; [ -f \"$root/AGENTS.md\" ] || exit 0; exec \"$root/bin/fm-codex-stop-park.sh\"'", + "async": true, + "timeout": 2147483647 } ] } diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 263b270bfb1..80d7d8b1fb8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -399,6 +399,94 @@ jobs: ${{ runner.temp }}/fm-herdr/default-server.log if-no-files-found: warn + # Required real-herdr-core (`hc`) lane: pinned install, serial hc family, + # hard-fail on "skip: hc not found". Mirrors tests-herdr's posture: the + # lab (bin/fm-hc-lab.sh) gives each session a private core, a fleet-state + # tripwire, and a serve/hold daemon sweep at teardown. + tests-hc: + name: ${{ (github.event.action == 'labeled' && github.event.label.name != 'full-ci') && 'Not run for label - ' || '' }}Behavior tests (herdr-core) + runs-on: ubuntu-24.04 + if: ${{ !(github.event_name == 'push' && vars.FM_CI_SKIP_PUSH == '1') && !(github.event.action == 'labeled' && github.event.label.name != 'full-ci') && (github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository || github.event.pull_request.author_association == 'OWNER' || github.event.pull_request.author_association == 'MEMBER' || github.event.pull_request.author_association == 'COLLABORATOR' || contains(github.event.pull_request.labels.*.name, 'full-ci')) }} + # Heavy tier (see the timeout policy above): the last-resort job backstop. + # The family-run step below owns the hang tripwire, so a wedged suite fails + # there with the always() cleanup and timing upload still running. + timeout-minutes: 75 + steps: + - uses: actions/checkout@v6 + with: + fetch-depth: 0 + - name: Require tools for real herdr-core + run: | + set -eu + command -v jq >/dev/null || { echo "::error::jq is required"; exit 1; } + command -v curl >/dev/null || { echo "::error::curl is required"; exit 1; } + command -v tar >/dev/null || { echo "::error::tar is required"; exit 1; } + command -v sha256sum >/dev/null || { echo "::error::sha256sum is required"; exit 1; } + jq --version + - name: Install pinned herdr-core + run: | + set -eu + bin/fm-install-hc.sh "$RUNNER_TEMP/bin" + echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH" + - name: Install pinned Treehouse + run: | + set -eu + bin/fm-install-treehouse.sh "$RUNNER_TEMP/bin" + echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH" + - name: Assert herdr-core pin and Treehouse presence + run: | + set -eu + command -v herdr-core >/dev/null || { echo "::error::herdr-core not on PATH after install"; exit 1; } + command -v treehouse >/dev/null || { echo "::error::treehouse not on PATH after install"; exit 1; } + herdr-core --version + treehouse --version + version=$(herdr-core --version | awk '{print $2; exit}') + [ "$version" = "0.1.14" ] || { + echo "::error::expected exact herdr-core pin 0.1.14, got ${version:-}" + exit 1 + } + - name: Snapshot pre-suite hc lab sessions + run: | + set -eu + mkdir -p "$RUNNER_TEMP/fm-hc" + bin/fm-hc-ci-cleanup.sh snapshot "$RUNNER_TEMP/fm-hc/sessions-before.txt" + - name: Run hc family (serial, required) + id: run-real-hc-family + # Heavy tier step tripwire: above the healthy wall and far below the + # job backstop, so a hang fails this step and cleanup runs. + timeout-minutes: 20 + run: | + set -eu + mkdir -p "$RUNNER_TEMP/fm-test" + # Pin the same resolved binary for the adapter, lab, and cleanup. + export FM_HC_BIN + FM_HC_BIN=$(command -v herdr-core) + # Serial only. Fail if any script reports hc not found. Live + # harness credential tests stay outside this family (opt-in env only). + bin/fm-test-run.sh --family hc \ + --fail-on-gate-skip 'hc not found' \ + --json "$RUNNER_TEMP/fm-test/fm-test-timing-hc.json" + - name: Cleanup job-owned hc lab sessions + id: cleanup-hc-lab-sessions + if: always() + run: | + set -eu + export FM_HC_BIN + FM_HC_BIN=$(command -v herdr-core) + bin/fm-hc-ci-cleanup.sh teardown "$RUNNER_TEMP/fm-hc/sessions-before.txt" || { + echo "::warning::lab session cleanup reported a problem; see logs" + exit 1 + } + - name: Upload hc timing and diagnostics + if: always() + uses: actions/upload-artifact@v4 + with: + name: fm-test-timing-hc + path: | + ${{ runner.temp }}/fm-test/fm-test-timing-hc.json + ${{ runner.temp }}/fm-hc/sessions-before.txt + if-no-files-found: warn + # Aggregate per-lane timing into one summary artifact for critical-path review. tests-timing-aggregate: name: ${{ (github.event.action == 'labeled' && github.event.label.name != 'full-ci') && 'Not run for label - ' || '' }}Behavior timing aggregate @@ -410,6 +498,7 @@ jobs: - tests-portable-parallel-2 - tests-portable-serial - tests-herdr + - tests-hc if: ${{ always() && !(github.event_name == 'push' && vars.FM_CI_SKIP_PUSH == '1') && !(github.event.action == 'labeled' && github.event.label.name != 'full-ci') && (github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository || github.event.pull_request.author_association == 'OWNER' || github.event.pull_request.author_association == 'MEMBER' || github.event.pull_request.author_association == 'COLLABORATOR' || contains(github.event.pull_request.labels.*.name, 'full-ci')) }} steps: - uses: actions/checkout@v6 @@ -547,6 +636,7 @@ jobs: - tests-portable-parallel-2 - tests-portable-serial - tests-herdr + - tests-hc - macos-stock-bash - invariants if: ${{ always() && !(github.event_name == 'push' && vars.FM_CI_SKIP_PUSH == '1') && !(github.event.action == 'labeled' && github.event.label.name != 'full-ci') }} diff --git a/.gitignore b/.gitignore index 2c9b9a4ec59..757032467b5 100644 --- a/.gitignore +++ b/.gitignore @@ -15,5 +15,6 @@ __pycache__/ *.pyc .env config/ +.claude/settings.local.json .tools/ diff --git a/.pi/extensions/lib/fm-primary-read-repetition-policy.ts b/.pi/extensions/lib/fm-primary-read-repetition-policy.ts index 8a9fe7b8980..a9fad29c2c7 100644 --- a/.pi/extensions/lib/fm-primary-read-repetition-policy.ts +++ b/.pi/extensions/lib/fm-primary-read-repetition-policy.ts @@ -115,10 +115,10 @@ function simpleRead(words: string[], root: string, remote: boolean): boolean { return args[0] === "-n" && /^\d+(?:,\d+)?p$/.test(args[1] ?? "") && args.slice(2).every(path); } if (name === "cut") return args.length === 1 && /^-c[1-9]\d*-[1-9]\d*$/.test(args[0]); - if (name === "nl") return args.every((arg) => /^-b[atn]$/.test(arg) || path(arg)); + if (name === "nl") return args.every((arg) => /^-b[atn]$/.test(arg) || /^-v\d+$/.test(arg) || path(arg)); if (name === "wc") return args.every((arg) => /^-[clmwL]+$/.test(arg) || path(arg)); if (name === "od") { - return args.every((arg) => /^-A[nodx]$/.test(arg) || /^-t(?:x[1248]|[cu])+$/.test(arg) || path(arg)); + return args.every((arg) => /^-A[nodx]$/.test(arg) || arg === "-c" || /^-t(?:x[1248]|[cu])+$/.test(arg) || path(arg)); } if (name === "grep") { // GNU grep's options read/filter only; refuse unfamiliar options anyway. diff --git a/AGENTS.md b/AGENTS.md index 5b318eae645..64958101ea1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -89,7 +89,7 @@ Load `session-start-recovery` when the digest reports unfinished checks, actiona ## 4. Harness and runtime dispatch - Load `harness-adapters` before every spawn or recovery and before trust handling, skill invocation, interrupt, exit, resume, or adapter verification. -- The verified harnesses are `claude`, `codex`, `opencode`, `pi`, `pi-signed`, `grok`, `kimi`, `cursor`, `omp`, and `agy`, plus `muse`, `gemini`, `rovo`, `devin`, `commandcode` (Command Code), `polytoken`, and `cortex` for crewmates and scouts only; never dispatch on an unverified adapter. +- The verified harnesses are `claude`, `codex`, `opencode`, `pi`, `pi-signed`, `grok`, `kimi`, `cursor`, `omp`, and `agy`, plus `muse`, `gemini`, `rovo`, `devin`, `commandcode` (Command Code), `polytoken`, `cortex`, `kiro`, and `hermes` for crewmates and scouts only; never dispatch on an unverified adapter. - If static `config/crew-harness` or `config/secondmate-harness` names an unverified adapter, report it and fall back only to a verified adapter rather than launching it. - Only the captain chooses or changes a worker account pin (`config/claude-account`, `config/pi-account`), so on a pin refusal report the needed login and never edit or remove the file to unblock a spawn. @@ -99,6 +99,7 @@ Routing precedence is an explicit per-task captain override, then the best-fit c Firstmate alone resolves a matched profile array: begin with `quota-axi`'s default TOON at that intake, using the skill's narrow TOON-then-`--json` fallback only for genuine ambiguity, evaluate every configured candidate against that current output, and choose under the skill's `preferred_provider` procedure with inspectable `spendPriority` as the one quota-perspective ranker after its eligibility, reasoning-class, and runway-feasibility gates. Account for every candidate with the catalog evidence, provider relationship, applicable quota and authentication facts, remaining uncertainty, fit and reasoning class, and the spendPriority and runway evidence used in selection; never omit a candidate, guess, fall back silently, or call the result quota-informed without them. Establish model support and provider family from that harness's own authoritative catalog, then apply the [account and scope matching rules in `quota-array-dispatch`](.agents/skills/quota-array-dispatch/SKILL.md#1-eligibility). +When the home's `config/crew-dispatch.json` declares `"shared_quota": true`, make no quota-axi call and apply no quota gate or ranking: choose by the matched rule's declared profile order and reasoning class (`preferred_provider` still narrows the pool; declared order then picks within it), accounting for every candidate with quota ranking off by home declaration (schema: `docs/configuration.md` "Crew dispatch profiles"). Missing model-level quota, a missing authentication source, unmeasurable headroom, or unmodeled authentication is disclosed uncertainty that keeps a candidate eligible, never a credential or login escalation. Only concrete contradictory evidence blocks a candidate, such as an authoritative catalog proving the model unsupported or proof that the credential selected for that surface is unusable; never infer a credential store, provider family, or quota mapping from a harness, model, or source name, and never launch another harness's CLI to judge a candidate. Preserve malformed profile configuration as an actionable error rather than selecting around it. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6d0b84febf0..6be27b4d91a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -156,10 +156,11 @@ Some of those loops are also catching a transient rather than waiting for a sett Discover tests by listing `tests/*.test.sh`: each is a self-contained bash script named `.test.sh`, and its header comment describes what it covers, so pass one to `bin/fm-test-run.sh` to focus on a subject with canonical timing output. Shared test helpers live in `tests/lib.sh` (reporters, temp roots, git fixtures), `tests/fixtures.sh` (fake toolchain and spawn-world builders), `tests/wake-helpers.sh`, `tests/secondmate-helpers.sh`, `tests/supervision-host-helpers.sh` (stub-engine fixtures for the split supervision-host suites), `tests/watch-triage-helpers.sh` (shared fixtures for the split watch-triage suites), and `tests/git-config-helpers.sh` (fixture Git isolation from the host's global and system configuration and from an inherited repository location such as `GIT_DIR`, already sourced by `tests/lib.sh` and `tests/herdr-test-safety.sh`; a suite that sources neither must source it itself before its first Git operation so a direct invocation stays isolated). Source those instead of copying a fake toolchain into a new suite. +Resolve every host tool a fixture execs with `command -v` instead of hardcoding a `/bin` path, and pass the result to the fake-tool shim either interpolated into the shim body (`tests/fm-backlog-atomicity.test.sh`) or through an environment variable (`tests/fm-watcher-lock.test.sh`): `/bin` holds only `sh` on some supported hosts, so a shim that execs a hardcoded `/bin/ln`, `/bin/rm`, or `/bin/ps` names a path that does not exist. The same host makes a `sleep` renamed through `exec -a` dispatch on `argv[0]` and exit at once, so a holder that only needs a distinguishing name gets it as a plain argument instead; keep `exec -a` only where `argv[0]` naming is itself the subject. A fixture may shorten a production timeout to keep a failure path prompt, but never below what the real work inside that window costs on a loaded machine: a fork, an exec, a lock acquisition, a beacon publication, or a first-poll check. Where a case's assertion is not about the timeout itself, give that window headroom over the measured loaded cost, and bound the test's own waiting with iteration-counted poll loops, which stretch under load where a wall-clock budget does not. Supervision host and Stop-hook hand-back waits are the exception: use `wait_until_deadline` (a 60s wall-clock budget well above the loaded hand-back cost) so a timeout dumps `host.out`, `hook.out`/`hook.err`, the supervision-host log, health file, and a process tree. -Tests that need a real optional backend or an explicit opt-in (real herdr/zellij/cmux/paseo/rex/tern smoke tests, the live Pi regression) skip themselves and print the tool or environment gate needed to enable them, so the portable suite remains safe on machines without those tools. +Tests that need a real optional backend or an explicit opt-in (real herdr/zellij/cmux/paseo/rex/tern/hc smoke tests, the live Pi regression) skip themselves and print the tool or environment gate needed to enable them, so the portable suite remains safe on machines without those tools. The complete [`tests/fm-watch-triage-pause.test.sh`](tests/fm-watch-triage-pause.test.sh) suite requires compatible `tasks-axi` on `PATH`, as specified in [Backlog backend](docs/configuration.md#backlog-backend-taskstoml--configbacklog-backend). [`tests/rex-test-safety.sh`](tests/rex-test-safety.sh) owns the real-Rex suites' private-server isolation and capability gates. The [Herdr backend guide](docs/herdr-backend.md#destructive-lab-safety) owns the lane's isolation boundary, while [runtime backend verification](docs/verification/runtime-backends.md#herdr) owns active empirical evidence; live harness credential tests remain opt-in. diff --git a/README.md b/README.md index 54d8971ae80..e54bb30e3d0 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ Launching a supported harness inside it for your primary session instantiates yo - **Event-driven, zero-token supervision** - a bash watcher sleeps on the fleet and wakes the first mate only when something needs you; verified primary harnesses also get a turn-end backstop that blocks or follows up on a blind stop when work is under way and supervision is not live. - **Optional Relay** - opt in with one local `.env` pairing token so firstmate can answer your public mentions on X and Discord alike, act on normal reversible mention requests through the same lifecycle as chat requests, acknowledge spawned work, and post up to three public-safe completion follow-ups within seven days for genuine milestones and the final outcome without changing non-Relay behavior; a final reply promised in a thread becomes durable state that is reconciled from disk, so a restart or a compacted conversation cannot lose it; dry-run preview records would-be replies and dismissals locally before go-live. - **Strict project boundary** - the first mate is read-only over your projects except for the narrow guarded and captain-approved operations authorized by [hard rule 1](AGENTS.md#1-identity-and-prime-directives), including fleet sync's guarded safe branch pruning; crewmates make every other project change behind the configured merge authority. -- **Restart-proof** - all state lives on disk and in the active session backend (tmux by hard default, herdr, rex, or cmux when selected or auto-detected, zellij/orca/paseo/t3code/tern when explicitly selected); the next session reconciles after a restart, while ordinary supervision recovers confirmed-dead, unpaused secondmate agents without waiting for one. +- **Restart-proof** - all state lives on disk and in the active session backend (see [backend selection](docs/configuration.md#backend-selection-order)); the next session reconciles after a restart, while ordinary supervision recovers confirmed-dead, unpaused secondmate agents without waiting for one. Full detail on every feature lives in [docs/architecture.md](docs/architecture.md). @@ -74,7 +74,7 @@ All three have verified turn-end guard paths when launched with their documented Pick whichever one matches your subscription and workflow. Oh My Pi (`omp`), a Pi fork, is verified as a primary with the same extension-owned watcher model as Pi and a stronger turn-end guard: its blocking `session_stop` hook compels a continuation instead of requesting one. -Codex and OpenCode are also verified and supported as primary harnesses; Codex uses bounded foreground checkpoints, and OpenCode uses a TUI plugin, so both carry more harness-specific supervision tradeoffs than the three co-primaries. +Codex and OpenCode are also verified and supported as primary harnesses; Codex uses bounded foreground checkpoints unless a home opts in to its model-free native Stop park, and OpenCode uses a TUI plugin, so both carry more harness-specific supervision tradeoffs than the three co-primaries. Cursor Agent CLI is verified as a primary too, using a tracked project-scope `.cursor/hooks.json` whose `stop` hook parks on the watcher between turns, closest in shape to Claude Code's. Launch it with `--trust`, or none of its project hooks load; it also has no turn-end hook in headless `cursor-agent -p`, so run the primary session interactively. Antigravity CLI (`agy`) is verified as a primary too, using a tracked project-local `.agents/hooks.json` whose Stop hook parks on the watcher between turns like Cursor's. @@ -146,7 +146,7 @@ Pi's `/supervision-model` command pins a cheaper model and a shallower reasoning ### More backends -Setup guides for tmux (the default) and every other supported backend (herdr, zellij, Orca, cmux, Paseo, Rex, T3 Code, tern) are linked in [Documentation](#documentation) below. +Setup guides for tmux (the default) and every other supported backend (herdr, zellij, Orca, cmux, Paseo, Rex, T3 Code, tern, hc) are linked in [Documentation](#documentation) below. ## How It Works @@ -236,6 +236,7 @@ Firstmate's skills live in two separate places with different audiences: - [docs/rex-backend.md](docs/rex-backend.md) - current setup, auto-detection, block-id routing, the absence proof, and limits for the experimental Rex backend. - [docs/t3code-backend.md](docs/t3code-backend.md) - current setup, sign-in, and limits for the experimental T3 Code backend. - [docs/tern-backend.md](docs/tern-backend.md) - current setup, surfaces capture, native-state degradation, and limits for the experimental tern backend. +- [docs/hc-backend.md](docs/hc-backend.md) - current setup, task labels, and limits for the experimental hc (herdr-core) backend. - [docs/codex-app-backend.md](docs/codex-app-backend.md) - the current blocked Codex App backend boundary and rollout contract. - [docs/verification/runtime-backends.md](docs/verification/runtime-backends.md) - active maintainer verification for runtime backend guarantees. - [docs/gerrit-forge-integration.md](docs/gerrit-forge-integration.md) - maintainer architecture for the forge axis: why change-shaped review is not a forge variant, the mode/forge/shape composition test, and where responsibility for forge mechanics sits. diff --git a/bin/backends/hc.sh b/bin/backends/hc.sh new file mode 100644 index 00000000000..424a57b6647 --- /dev/null +++ b/bin/backends/hc.sh @@ -0,0 +1,500 @@ +#!/usr/bin/env bash +# bin/backends/hc.sh - the herdr-core (`hc`) session-provider adapter (EXPERIMENTAL). +# +# herdr-core is the terminal core under Herdr.app: a per-user daemon that keeps +# terminals alive and answers a JSON CLI (`hc run|get|ls|read|send|keys|kill|rm`). +# It is not Herdr 1 (`herdr`, bin/backends/herdr.sh); the two share no socket, +# no id space, and no CLI. Like tmux, hc is a session provider ONLY: the +# worktree provider stays treehouse, driven by fm-spawn.sh through the same +# send/current-path primitives it uses for tmux. docs/hc-backend.md owns the +# user-facing mechanics and limits; this header owns the adapter's design. +# Sourced only through bin/fm-backend.sh's fm_backend_source in normal +# operation; the unit tests source it directly. +# +# docs/hc-backend.md "Runtime auto-detection" owns selection; an open app +# window alone never selects hc. fm_backend_detect owns the check order. +# +# Target string shape: ":
". is the hc session the +# terminal lives in (HC_SESSION at spawn time, `default` when unset) and +#
is hc's own terminal address (`t7`). Every call re-exports exactly +# that HC_SESSION, so a target keeps naming the core it was created on even when +# the caller's ambient HC_SESSION differs. Measured on hc 0.1.13: HC_SESSION +# wins over an inherited HC_TERMINAL_SOCKET, so a firstmate running inside an hc +# terminal still reaches the recorded core. +# +# Identity: each task terminal carries the labels firstmate.task= +# and firstmate.home= (bin/fm-backend-hometag-lib.sh). Operations that +# receive an expected label refuse a terminal whose labels do not match, so a +# recorded address can never steer input into another home's terminal. hc +# names are not used: they allow only 1-32 of [a-z0-9_-], narrower than task ids. +# +# Visibility in Herdr.app: the app shows a terminal that carries +# herdr.space= and herdr.tab= labels, and an unknown id becomes a new +# space or tab. create_task sets herdr.space=FM_HC_SPACE (default +# fm-) and herdr.tab=, so a home's tasks group in one +# space with one tab each. Naming the space needs the app window and is not done. +# +# Measured limits on hc 0.1.13 and 0.1.14 (docs/hc-backend.md "Limits"): +# - `hc send` always presses Enter after pasting. There is no paste without +# submit, so hc cannot type without submitting. `hc send --wait` reports +# the turn the send started natively, which is what send_text_submit reads. +# - hc reports one foreground process per terminal (`foreground` in +# `hc get`), not a process group; when that process is a launcher, +# agent_state also reads the rest of its process group from local `ps`. +# - hc has no event push this adapter consumes; supervision polls like tmux. +# +# Binary gate: FM_HC_BIN wins, else `herdr-core`, else `hc` (the release +# asset's name and its CLI alias). ` --version` must report `hc x.y.z` +# at or above FM_BACKEND_HC_MIN_VERSION, and container_ensure additionally +# proves the session speaks protocol generation 1 through `hello`. + +# shellcheck source=bin/fm-backend-hometag-lib.sh +. "$FM_BACKEND_LIB_DIR/fm-backend-hometag-lib.sh" +# shellcheck source=bin/fm-composer-lib.sh +. "$FM_BACKEND_LIB_DIR/fm-composer-lib.sh" +# shellcheck source=bin/fm-agent-process-lib.sh +. "$FM_BACKEND_LIB_DIR/fm-agent-process-lib.sh" + +# The verified floor: the oldest herdr-core release every adapter surface +# (including `send --wait` and the `hello` handshake) was exercised against. +FM_BACKEND_HC_MIN_VERSION=0.1.14 + +# The one protocol generation this adapter speaks. +FM_BACKEND_HC_GENERATION=1 + +# The hc session a new task is created on: the caller's HC_SESSION, else hc's +# own default session name. +fm_backend_hc_session() { + printf '%s' "${HC_SESSION:-default}" +} + +# fm_backend_hc_bin: the resolved herdr-core binary path, or failure. The +# release asset's own name is `herdr-core`; `hc` is its CLI alias. FM_HC_BIN +# wins so tests and the lab can pin a build; version_check still proves the +# resolved file is herdr-core and no other tool. +fm_backend_hc_bin() { + if [ -n "${FM_HC_BIN:-}" ]; then + [ -x "$FM_HC_BIN" ] || return 1 + printf '%s' "$FM_HC_BIN" + return 0 + fi + command -v herdr-core 2>/dev/null || command -v hc 2>/dev/null +} + +# fm_backend_hc_version_check: refuse loudly on a missing, misnamed, or +# older-than-verified binary. ` --version` must print `hc x.y.z`. +fm_backend_hc_version_check() { + local bin raw ver major rest minor patch cur min fmaj fmin fpat + bin=$(fm_backend_hc_bin) || { + echo "error: backend=hc selected but no 'herdr-core' or 'hc' binary was found on PATH (install Herdr Core per docs/hc-backend.md or set FM_HC_BIN)" >&2 + return 1 + } + raw=$("$bin" --version 2>/dev/null) \ + || { echo "error: '$bin --version' failed; is herdr-core installed correctly?" >&2; return 1; } + case "$raw" in + 'hc '[0-9]*.[0-9]*.[0-9]*) + ver=${raw#hc } + ver=${ver%%[[:space:]]*} + ;; + *) + echo "error: expected 'hc x.y.z' from '$bin --version' but got '$raw'; refusing to use a binary that is not Herdr Core" >&2 + return 1 + ;; + esac + major=${ver%%.*} + rest=${ver#*.} + minor=${rest%%.*} + case "$rest" in + *.*) patch=${rest#*.} ;; + *) patch=0 ;; + esac + patch=${patch%%[^0-9]*} + case "$major" in ''|*[!0-9]*) major=0 ;; esac + case "$minor" in ''|*[!0-9]*) minor=0 ;; esac + case "$patch" in ''|*[!0-9]*) patch=0 ;; esac + IFS=. read -r fmaj fmin fpat <&2 + return 1 + fi + return 0 +} + +# fm_backend_hc_env_run: run the resolved binary with exactly 's env +# pins. Ambient HC_TERMINAL_* routing markers are always stripped. An +# inherited HC_SOCKET is stripped because hc lets it win over HC_SESSION; +# FM_HC_SOCKET is the deliberate pin used by tests and the lab instead. +fm_backend_hc_env_run() { # + local session=$1 bin + shift + bin=$(fm_backend_hc_bin) || { + echo "error: backend=hc selected but no 'herdr-core' or 'hc' binary was found on PATH (install Herdr Core per docs/hc-backend.md or set FM_HC_BIN)" >&2 + return 1 + } + if [ -n "${FM_HC_SOCKET:-}" ]; then + env -u HC_TERMINAL_ID -u HC_TERMINAL_SOCKET -u HC_SCOPE \ + HC_SESSION="$session" HC_SOCKET="$FM_HC_SOCKET" "$bin" "$@" + else + env -u HC_TERMINAL_ID -u HC_TERMINAL_SOCKET -u HC_SCOPE -u HC_SOCKET \ + HC_SESSION="$session" "$bin" "$@" + fi +} + +# fm_backend_hc_cli: run hc against exactly . +# An invalid or empty session is refused: an empty HC_SESSION would let hc +# fall back to whatever core the caller's environment names. stdin is +# detached so hc never consumes a caller's loop input. +fm_backend_hc_cli() { # + local session=$1 + shift + fm_backend_hc_atom_valid "$session" || { echo "error: refusing hc session '$session'" >&2; return 1; } + fm_backend_hc_env_run "$session" "$@" + case "$1" in + ''|-*|*[!A-Za-z0-9._-]*) return 1 ;; + esac +} + +fm_backend_hc_address_valid() { # + case "$1" in + t*) ;; + *) return 1 ;; + esac + case "${1#t}" in + ''|*[!0123456789abcdefghijklmnopqrstuvwxyz]*) return 1 ;; + esac +} + +# fm_backend_hc_parse_target: split ":
" into +# FM_BACKEND_HC_SESSION and FM_BACKEND_HC_ADDRESS, or return 1. +fm_backend_hc_parse_target() { # + local target=${1:-} + FM_BACKEND_HC_SESSION= + FM_BACKEND_HC_ADDRESS= + case "$target" in + *:*:*|:*|*:) return 1 ;; + *:*) ;; + *) return 1 ;; + esac + fm_backend_hc_atom_valid "${target%%:*}" || return 1 + fm_backend_hc_address_valid "${target#*:}" || return 1 + FM_BACKEND_HC_SESSION=${target%%:*} + FM_BACKEND_HC_ADDRESS=${target#*:} +} + +fm_backend_hc_scoped_label() { # + printf '%s-%s' "$(fm_backend_hometag)" "$1" +} + +# fm_backend_hc_get: `hc get --json` of the parsed target. Status 3 is hc's +# structured not_found; any other failure is an unreadable core. +fm_backend_hc_get() { # + fm_backend_hc_parse_target "$1" || return 1 + fm_backend_hc_cli "$FM_BACKEND_HC_SESSION" get --json "$FM_BACKEND_HC_ADDRESS" 2>/dev/null +} + +# fm_backend_hc_endpoint_state: classify against a fresh `hc get`. +# Prints exactly one of: ready, finished, absent, foreign, unreadable. A +# terminal is `foreign` when [expected-label] is given and its labels do not +# carry this home's labels for it; `*` accepts any task label of this home. +fm_backend_hc_endpoint_state() { # [expected-label] + local target=$1 expected=${2:-} json rc fields state task home + fm_backend_hc_parse_target "$target" || { printf 'unreadable'; return 0; } + json=$(fm_backend_hc_get "$target") + rc=$? + if [ "$rc" -eq 3 ]; then printf 'absent'; return 0; fi + [ "$rc" -eq 0 ] || { printf 'unreadable'; return 0; } + fields=$(printf '%s' "$json" | jq -r '[.state // "", .labels["firstmate.task"] // "", .labels["firstmate.home"] // ""] | @tsv' 2>/dev/null) || { + printf 'unreadable' + return 0 + } + IFS=$'\t' read -r state task home < [expected-label] + [ "$(fm_backend_hc_endpoint_state "$1" "${2:-}")" = ready ] && fm_backend_hc_parse_target "$1" +} + +# fm_backend_hc_hello: the generation handshake for over `hc +# connect`. Prints the negotiated generation, nothing on failure. +fm_backend_hc_hello() { # + local session=$1 out gen + out=$(printf '{"jsonrpc":"2.0","method":"hello","params":{"generation":%s},"id":1}\n' \ + "$FM_BACKEND_HC_GENERATION" \ + | fm_backend_hc_env_run "$session" connect --session "$session" 2>/dev/null) || return 1 + gen=$(printf '%s' "$out" | jq -r '.result.generation // empty' 2>/dev/null) + [ -n "$gen" ] || return 1 + printf '%s' "$gen" +} + +# fm_backend_hc_container_ensure: prove the binary is herdr-core at the +# verified floor, make sure the session's core answers (hc starts its own +# core on first use) and speaks protocol generation 1, then print the +# session name. +fm_backend_hc_container_ensure() { + local session gen + fm_backend_hc_version_check || return 1 + command -v jq >/dev/null 2>&1 \ + || { echo "error: backend=hc selected but 'jq' is not installed (required to parse hc's JSON output)" >&2; return 1; } + session=$(fm_backend_hc_session) + fm_backend_hc_atom_valid "$session" || { echo "error: refusing hc session name '$session'" >&2; return 1; } + fm_backend_hc_cli "$session" server status --json >/dev/null 2>&1 \ + || { echo "error: 'hc server status' failed for session '$session'; is herdr-core installed and startable?" >&2; return 1; } + gen=$(fm_backend_hc_hello "$session") || { + echo "error: herdr-core session '$session' did not answer the hello handshake; is the core reachable?" >&2 + return 1 + } + if [ "$gen" != "$FM_BACKEND_HC_GENERATION" ]; then + echo "error: herdr-core session '$session' speaks protocol generation '$gen', but this adapter requires generation $FM_BACKEND_HC_GENERATION" >&2 + return 1 + fi + printf '%s' "$session" +} + +# fm_backend_hc_create_task: start a login shell terminal in for +# , refusing a live duplicate in this home. Prints the address. +# --no-parent and --outlive-parent keep the terminal independent of whichever +# hc terminal firstmate itself runs in; --no-notify keeps hc from typing notes +# into any parent. +fm_backend_hc_create_task() { # + local session=$1 label=$2 cwd=$3 home scoped dup out address + home=$(fm_backend_hometag) + scoped=$(fm_backend_hc_scoped_label "$label") + dup=$(fm_backend_hc_cli "$session" ls --json --label "firstmate.task=$label" --label "firstmate.home=$home" 2>/dev/null) \ + || { echo "error: 'hc ls' failed for session '$session'; cannot create $label" >&2; return 1; } + if [ -n "$(printf '%s' "$dup" | jq -r '.terminals[]?.address' 2>/dev/null)" ]; then + echo "error: hc terminal for $label already exists in session '$session'" >&2 + return 1 + fi + out=$(fm_backend_hc_cli "$session" run --bg --json --no-parent --outlive-parent --no-notify \ + --cwd "$cwd" \ + --label "firstmate.task=$label" --label "firstmate.home=$home" \ + --label "herdr.space=${FM_HC_SPACE:-fm-$home}" --label "herdr.tab=$scoped" 2>&1) \ + || { echo "error: hc run failed for $label: $out" >&2; return 1; } + address=$(printf '%s' "$out" | jq -r '.address // empty' 2>/dev/null) + fm_backend_hc_address_valid "$address" || { echo "error: hc run did not return a terminal address for $label (got: $out)" >&2; return 1; } + printf '%s\n' "$address" +} + +# fm_backend_hc_capture: the screen plus of history, the same window +# tmux's `capture-pane -S -` returns. +fm_backend_hc_capture() { # [expected-label] + local lines=${2:-200} + fm_backend_hc_target_ready "$1" "${3:-}" || return 1 + case "$lines" in ''|*[!0-9]*) lines=200 ;; esac + fm_backend_hc_cli "$FM_BACKEND_HC_SESSION" read --lines "$lines" "$FM_BACKEND_HC_ADDRESS" 2>/dev/null +} + +# fm_backend_hc_visible_capture: the screen only (`--lines 0`). +fm_backend_hc_visible_capture() { # [expected-label] + fm_backend_hc_target_ready "$1" "${2:-}" || return 1 + fm_backend_hc_cli "$FM_BACKEND_HC_SESSION" read --lines 0 "$FM_BACKEND_HC_ADDRESS" 2>/dev/null +} + +# fm_backend_hc_normalize_key: firstmate's key vocabulary onto hc key names. +fm_backend_hc_normalize_key() { # + case "$1" in + Enter|enter|C-m) printf 'enter' ;; + Escape|escape|Esc|esc) printf 'esc' ;; + C-c|c-c|ctrl+c|Ctrl+c|Ctrl+C|ctrl-c) printf 'ctrl+c' ;; + C-u|c-u|ctrl+u|Ctrl+u|Ctrl+U|ctrl-u) printf 'ctrl+u' ;; + Up) printf 'up' ;; + Down) printf 'down' ;; + Tab) printf 'tab' ;; + *) printf '%s' "$1" ;; + esac +} + +fm_backend_hc_send_key() { # [expected-label] + fm_backend_hc_target_ready "$1" "${3:-}" || return 1 + fm_backend_hc_cli "$FM_BACKEND_HC_SESSION" keys --json "$FM_BACKEND_HC_ADDRESS" \ + -- "$(fm_backend_hc_normalize_key "$2")" >/dev/null 2>&1 +} + +# fm_backend_hc_send_text_line: paste one line and submit it (`hc send`). The +# `--` delimiter keeps flag-looking text from being parsed as options. +fm_backend_hc_send_text_line() { # [expected-label] + fm_backend_hc_target_ready "$1" "${3:-}" || return 1 + fm_backend_hc_cli "$FM_BACKEND_HC_SESSION" send --json "$FM_BACKEND_HC_ADDRESS" -- "$2" >/dev/null 2>&1 +} + +fm_backend_hc_current_path() { # [expected-label] + local json + fm_backend_hc_target_ready "$1" "${2:-}" || return 0 + json=$(fm_backend_hc_get "$1") || return 0 + printf '%s' "$json" | jq -r '.foreground.cwd // .cwd // empty' 2>/dev/null +} + +fm_backend_hc_composer_caps() { + printf 'styled=0\ncursor=0\nidentity=0\nrows=%s\n' "$FM_COMPOSER_CAPTURE_LINES" +} + +fm_backend_hc_composer_state() { # [expected-label] -> empty|pending|pending-unproven|unknown + local cap verdict + cap=$(fm_backend_hc_visible_capture "$1" "${2:-}") || { printf 'unknown'; return 0; } + verdict=$(fm_composer_classify_screen "$(fm_backend_hc_composer_caps)" "$cap") + [ "$verdict" != need-identity ] || verdict=unknown + printf '%s' "$verdict" +} + +# fm_backend_hc_send_text_submit: one `hc send --wait`, which pastes, presses +# Enter, and reports the turn it started. Echoes +# empty|pending|pending-unproven|unknown|send-failed, a subset of the +# proof-carrying submit vocabulary. +# +# A started turn or a still-working wait confirms the submit natively, which +# no composer text can fake. A program that never reports status answers +# started:null once the bounded wait times out; the send still typed and +# entered, so the verdict falls back to the composer screen, and only a +# proven pending composer gets Enter retries through the shared loop. A +# blocked terminal refuses with exit 9 before typing - unknown, with a +# diagnostic and no retries, never forced. The other structured refusals +# (3 not_found, 11 finished) and any other failure are send-failed. +fm_backend_hc_send_text_submit() { # [expected-label] + local target=$1 text=$2 retries=$3 sleep_s=$4 settle=$5 expected=${6:-} + local wait_secs out rc started wait_status verdict + case "$retries" in ''|*[!0-9]*) retries=1 ;; esac + case "$sleep_s" in ''|*[!0-9.]*) sleep_s=0.4 ;; esac + case "$settle" in ''|*[!0-9.]*) settle=0 ;; esac + fm_backend_hc_target_ready "$target" "$expected" || { printf 'send-failed'; return 0; } + wait_secs=$(awk -v r="$retries" -v s="$sleep_s" 'BEGIN { v = r * s; if (v < 1) v = 1; printf "%d", (v == int(v) ? v : int(v) + 1) }' 2>/dev/null) + case "$wait_secs" in ''|*[!0-9]*|0) wait_secs=1 ;; esac + out=$(fm_backend_hc_cli "$FM_BACKEND_HC_SESSION" send --json "$FM_BACKEND_HC_ADDRESS" \ + --wait --timeout "$wait_secs" -- "$text" 2>&1) + rc=$? + case "$rc" in + 0) ;; + 9) + echo "error: hc send refused $target - the terminal is blocked waiting on input (never forced): $out" >&2 + printf 'unknown' + return 0 + ;; + *) printf 'send-failed'; return 0 ;; + esac + started=$(printf '%s' "$out" | jq -r '.started // empty' 2>/dev/null) + wait_status=$(printf '%s' "$out" | jq -r '.wait.status // empty' 2>/dev/null) + if [ "$started" = true ] || [ "$wait_status" = working ]; then + printf 'empty' + return 0 + fi + [ "$settle" = 0 ] || sleep "$settle" + verdict=$(fm_backend_hc_composer_state "$target" "$expected") + case "$verdict" in + pending|pending-unproven) + fm_composer_submit_retry_core fm_backend_hc_send_key fm_backend_hc_composer_state \ + "$target" "$retries" "$sleep_s" "$expected" + ;; + *) printf '%s' "$verdict" ;; + esac +} + +# fm_backend_hc_busy_state: hc's native agent status. `working` is busy; the +# settled words are idle; a terminal with no reported status is unknown. +fm_backend_hc_busy_state() { # + local json + json=$(fm_backend_hc_get "$1") || { printf 'unknown'; return 0; } + case "$(printf '%s' "$json" | jq -r '.status.state // empty' 2>/dev/null)" in + working) printf 'busy' ;; + idle|done) printf 'idle' ;; + *) printf 'unknown' ;; + esac +} + +# fm_backend_hc_kill: end the program and remove the terminal, then prove it is +# gone. Already absent is a silent success; a terminal still present, a core +# that cannot be read, or a terminal without this home's labels is a refusal. +fm_backend_hc_kill() { # [unused] [expected-label] + local target=$1 expected=${3:-'*'} state + fm_backend_hc_parse_target "$target" || return 1 + state=$(fm_backend_hc_endpoint_state "$target" "$expected") + case "$state" in + absent) return 0 ;; + foreign) echo "error: hc terminal $target does not carry this task's labels; refusing to close it" >&2; return 1 ;; + unreadable) echo "error: hc terminal $target could not be read, so it was not closed" >&2; return 1 ;; + esac + [ "$state" != ready ] || fm_backend_hc_cli "$FM_BACKEND_HC_SESSION" kill --json "$FM_BACKEND_HC_ADDRESS" >/dev/null 2>&1 || true + fm_backend_hc_cli "$FM_BACKEND_HC_SESSION" rm --json "$FM_BACKEND_HC_ADDRESS" >/dev/null 2>&1 || true + state=$(fm_backend_hc_endpoint_state "$target") + case "$state" in + absent) return 0 ;; + unreadable) echo "error: hc terminal $target could not be read after its close, so whether it survived is unknown" >&2 ;; + *) echo "error: hc terminal $target is still present after its close" >&2 ;; + esac + return 1 +} + +# fm_backend_hc_agent_state: recovery-grade state (vocabulary owned by +# bin/fm-backend.sh's fm_backend_agent_state). A structured not_found or a +# finished terminal is `missing`: an hc terminal whose program ended can never +# host the agent again, exactly like a closed tmux window. A live terminal is +# classified from hc's foreground process through the shared classifier. +fm_backend_hc_agent_state() { # + local json rc fields state name argv0 args pid class + fm_backend_hc_parse_target "${1:-}" || { printf 'unreadable'; return 0; } + json=$(fm_backend_hc_get "$1") + rc=$? + if [ "$rc" -eq 3 ]; then printf 'missing'; return 0; fi + [ "$rc" -eq 0 ] || { printf 'unreadable'; return 0; } + fields=$(printf '%s' "$json" | jq -r '[.state // "", .foreground.name // "", .foreground.argv[0] // "", + ((.foreground.argv // []) | join(" ")), (.foreground.pid // "" | tostring)] | @tsv' 2>/dev/null) || { + printf 'unreadable' + return 0 + } + IFS=$'\t' read -r state name argv0 args pid <'s process +# group classifies as a harness. Positive evidence only. +fm_backend_hc_group_has_agent() { # + local pid=$1 pgid member mpgid comm args + case "$pid" in ''|*[!0123456789]*) return 1 ;; esac + pgid=$(LC_ALL=C ps -o pgid= -p "$pid" 2>/dev/null | tr -d ' ') || return 1 + [ -n "$pgid" ] || return 1 + while read -r member mpgid comm; do + [ "$mpgid" = "$pgid" ] && [ "$member" != "$pid" ] || continue + args=$(LC_ALL=C ps -o args= -p "$member" 2>/dev/null) || continue + [ "$(fm_agent_process_classify "$comm" "${args%% *}" "$args" "$member")" = agent ] && return 0 + done </dev/null) +EOF + return 1 +} diff --git a/bin/backends/herdr.sh b/bin/backends/herdr.sh index 6f419404dff..bc1b1911506 100644 --- a/bin/backends/herdr.sh +++ b/bin/backends/herdr.sh @@ -524,7 +524,7 @@ fm_backend_herdr_client_select() { # [force] # fm_backend_herdr_tool_check: refuse loudly if herdr or jq is missing. fm_backend_herdr_tool_check() { - command -v herdr >/dev/null 2>&1 || { echo "error: backend=herdr selected but the 'herdr' CLI is not installed (https://herdr.dev) (dual-licensed AGPL-3.0-or-later/commercial)" >&2; return 1; } + command -v herdr >/dev/null 2>&1 || { echo "error: backend=herdr selected but the 'herdr' CLI is not installed (https://herdr.dev) (Apache-2.0)" >&2; return 1; } command -v jq >/dev/null 2>&1 || { echo "error: backend=herdr selected but 'jq' is not installed (required to parse herdr's JSON output)" >&2; return 1; } return 0 } @@ -1901,7 +1901,14 @@ fm_backend_herdr_launcher_identity() { # | .result.pane.workspace_id ' 2>/dev/null) if [ -z "$tab" ] || [ -z "$workspace" ]; then - echo "error: herdr launcher pane '$pane' returned an ambiguous tab or workspace identity in session '$session'; refusing to place a worker without its exact parent workspace" >&2 + # Not an ambiguous identity. A destroyed pane id never reaches here: herdr + # 0.9.3 answers `pane get` for it with pane_not_found and the "could not + # be read" refusal above fires. This branch covers an id herdr resolves to + # a different pane (a legacy id form such as `1-1` answers as `w1:p1`), so + # the exact-id filter keeps nothing, or a record naming no tab or + # workspace. Call the id stale and name the three injected values a live + # shell has to refresh, instead of blaming an ambiguity that does not exist. + echo "error: herdr launcher pane '$pane' is stale - in session '$session' that pane no longer exists or names no tab or workspace; refresh HERDR_PANE_ID, HERDR_TAB_ID and HERDR_WORKSPACE_ID in this shell to the current pane's values; refusing to place a worker without its exact parent workspace" >&2 return 1 fi @@ -2409,40 +2416,50 @@ fm_backend_herdr_pane_process_state() { # # for fm_backend_herdr_pane_process_state, which owns the verdict contract and # the settle retry. fm_backend_herdr_pane_process_state_sample() { # - local session=$1 pane_id=$2 info shell_pid count i pid name argv0 args verdict + local session=$1 pane_id=$2 info tag shell_pid pid name argv0 args local others=0 ps_bin rows info=$(fm_backend_herdr_cli "$session" pane process-info --pane "$pane_id" 2>/dev/null) \ || { printf 'unreadable'; return 0; } - printf '%s' "$info" | jq -e --arg pane "$pane_id" ' - .result.type == "pane_process_info" - and .result.process_info.pane_id == $pane - ' >/dev/null 2>&1 || { printf 'unreadable'; return 0; } - shell_pid=$(printf '%s' "$info" | jq -er \ - '.result.process_info.shell_pid | select(type == "number" and . > 1) | floor' 2>/dev/null) \ - || { printf 'unreadable'; return 0; } - count=$(printf '%s' "$info" | jq -er \ - '.result.process_info.foreground_processes | select(type == "array") | length' 2>/dev/null) \ - || { printf 'unreadable'; return 0; } - i=0 - while [ "$i" -lt "$count" ]; do - pid=$(printf '%s' "$info" | jq -r --argjson i "$i" \ - '.result.process_info.foreground_processes[$i].pid | select(type == "number") | floor' 2>/dev/null) - name=$(printf '%s' "$info" | jq -r --argjson i "$i" \ - '.result.process_info.foreground_processes[$i].name // empty' 2>/dev/null) - argv0=$(printf '%s' "$info" | jq -r --argjson i "$i" ' - .result.process_info.foreground_processes[$i] as $p - | (($p.argv // [])[0]) // $p.argv0 // empty' 2>/dev/null) - args=$(printf '%s' "$info" | jq -r --argjson i "$i" ' - .result.process_info.foreground_processes[$i] as $p - | $p.cmdline // (($p.argv // []) | join(" ")) // empty' 2>/dev/null) - verdict=$(fm_agent_process_classify "$name" "$argv0" "$args" "$pid") - case "$verdict" in - agent) printf 'agent'; return 0 ;; - shell) ;; - *) others=$((others + 1)) ;; - esac - i=$((i + 1)) - done + + # One jq pass validates the response and emits NUL-separated fields: `ok`, + # the shell pid, then pid, name, argv0 and args for each foreground process. + # The watcher classifies every busy pane on every poll, so a jq process per + # field per process made one fleet scan outrun the watcher heartbeat grace. + # NUL keeps args with spaces, tabs or newlines intact. A field that is + # missing or malformed reads as empty, and an invalid response emits nothing. + { + if ! { IFS= read -r -d '' tag && [ "$tag" = ok ] && IFS= read -r -d '' shell_pid; }; then + printf 'unreadable' + return 0 + fi + while IFS= read -r -d '' pid && IFS= read -r -d '' name \ + && IFS= read -r -d '' argv0 && IFS= read -r -d '' args; do + case "$(fm_agent_process_classify "$name" "$argv0" "$args" "$pid")" in + agent) printf 'agent'; return 0 ;; + shell) ;; + *) others=$((others + 1)) ;; + esac + done + } < <(printf '%s' "$info" | jq -j --arg pane "$pane_id" ' + def field(f): [try f catch empty] + | if length == 0 then "" + else .[0] | (if type == "string" then . else tojson end) | gsub("\u0000"; "") | sub("\n+$"; "") + end; + .result as $r + | if ($r.type == "pane_process_info" and $r.process_info.pane_id == $pane) | not then empty + else + ($r.process_info.shell_pid | select(type == "number" and . > 1) | floor) as $shell + | ($r.process_info.foreground_processes | select(type == "array")) as $procs + | ["ok", ($shell | tostring)] + + [$procs[] as $p | $p | ( + field(.pid | select(type == "number") | floor), + field(.name // empty), + field((($p.argv // [])[0]) // $p.argv0 // empty), + field($p.cmdline // (($p.argv // []) | join(" ")) // empty) + )] + | map(. + "\u0000") | add + end + ' 2>/dev/null) # Nothing in the foreground is a harness. A foreground that is not purely # shells is already `other`, whatever else the pane holds. Before calling a @@ -2957,13 +2974,22 @@ fm_backend_herdr_projection_create_task() { # - local session=$1 task_pane=$2 seeded_pane=$3 - [ -z "$task_pane" ] || fm_backend_herdr_projection_close_pane_focus_preserving "$session" "$task_pane" || true + local session=$1 task_pane=$2 seeded_pane=$3 task_closed=1 + if [ -n "$task_pane" ]; then + if fm_backend_herdr_projection_close_pane_focus_preserving "$session" "$task_pane" || + [ "$(fm_backend_herdr_pane_presence_state "$session" "$task_pane")" = dead ]; then + task_closed=0 + fi + fi if [ -n "$seeded_pane" ] && [ "$seeded_pane" != "$task_pane" ]; then fm_backend_herdr_projection_close_pane_focus_preserving "$session" "$seeded_pane" || true fi + return "$task_closed" } # fm_backend_herdr_projection_parent_workspace_exact: resolve one exact parent diff --git a/bin/backends/rex.sh b/bin/backends/rex.sh index 27564aee49c..0b30dac085b 100644 --- a/bin/backends/rex.sh +++ b/bin/backends/rex.sh @@ -42,8 +42,9 @@ # line, 3 target not found (or an ambiguous label, which full ids never are), # 4 server unreachable. Passive reads pass --autostart=false so a probe never # starts a server; spawn, cleanup, and the control plane's absence recheck let -# the CLI start a missing personal server, because a stopped server ends every -# session on it and a started one then answers "not found" with authority. +# the CLI start a missing personal server so it can prove whether the recorded +# block restored as a shell or is authoritatively missing. +# FM_BACKEND_REX_NO_AUTOSTART=1 withholds that start (fm_backend_rex_cli). # # Viewport: Rex has no viewport-only read, so the visible viewport is DERIVED # as the last `size.rows` lines of the styled `format` read, which always ends @@ -113,17 +114,26 @@ fm_backend_rex_tool_check() { return 0 } -# fm_backend_rex_cli: one CLI call that may start a missing personal server. +# fm_backend_rex_cli: one CLI call that may start a missing personal server, +# unless FM_BACKEND_REX_NO_AUTOSTART=1 prevents calls from starting one. A remote second mate's host-local control plane sets it +# (bin/fm-remote-secondmate-control.sh): there an automatic start creates an +# exit-empty server, and only remote readiness may start the persistent one. fm_backend_rex_cli() { # local bin bin=$(fm_backend_rex_bin) || return 1 - "$bin" "$@" + if [ "${FM_BACKEND_REX_NO_AUTOSTART:-}" = 1 ]; then + "$bin" --autostart=false "$@" + else + "$bin" "$@" + fi } # fm_backend_rex_read: one CLI call that never starts a server; an unreachable # server exits 4 instead. fm_backend_rex_read() { # - fm_backend_rex_cli --autostart=false "$@" + local bin + bin=$(fm_backend_rex_bin) || return 1 + "$bin" --autostart=false "$@" } # fm_backend_rex_failure_class: classify a failed call from its exit status @@ -984,8 +994,8 @@ fm_backend_rex_send_text_submit() { # < } # fm_backend_rex_block_presence: present|missing|unknown for the target block, -# read through a call that may start a missing personal server, so a stopped -# server resolves to the authoritative not-found its fresh start returns. +# read through a call that may start a missing personal server. A restored block +# is present; only a structured not-found proves it missing. fm_backend_rex_block_presence() { # local err rc fm_backend_rex_parse_target "$1" || { printf 'unknown'; return 0; } @@ -1054,11 +1064,9 @@ fm_backend_rex_retire_session() { # # fm_backend_rex_endpoint_absence_recheck: the control plane's absence proof # (bin/fm-control-lib.sh's fm_control_endpoint_absence_verdict owns the -# argument). Rex ids are never reused and a session never survives its -# server, so a structured not-found from the server the task was created on -# proves the block is gone. The proof therefore requires the endpoint this -# process reaches (starting a stopped personal server) to equal the recorded -# one. Prints missing|dead|alive|unproven. +# argument). Require the recorded server before accepting block presence or +# process evidence; server-stop restoration alone proves neither absence nor +# a live agent (docs/rex-backend.md). Prints missing|dead|alive|unproven. fm_backend_rex_endpoint_absence_recheck() { # local target=$1 recorded=${2:-} now state [ -n "$recorded" ] || { printf 'unproven'; return 0; } diff --git a/bin/backends/tmux.sh b/bin/backends/tmux.sh index f3547259af9..d4bcdf22e77 100644 --- a/bin/backends/tmux.sh +++ b/bin/backends/tmux.sh @@ -506,11 +506,11 @@ $argv0s EOF # Preserve argv boundaries where the platform exposes them. This is needed - # when the Gemini script path contains whitespace, which flattened ps output - # cannot represent unambiguously. + # when a Gemini or Hermes script path contains whitespace, which flattened + # ps output cannot represent unambiguously. while IFS= read -r pid; do [ -n "$pid" ] || continue - if fm_gemini_pid_is_gemini "$pid"; then + if fm_gemini_pid_is_gemini "$pid" || fm_hermes_process_matches '' "$pid"; then printf 'alive' return 0 fi @@ -522,7 +522,7 @@ EOF # evidence only - a bare interpreter still reaches the negative verdicts. while IFS= read -r name; do [ -n "$name" ] || continue - if fm_gemini_args_are_gemini "$name"; then + if fm_gemini_args_are_gemini "$name" || fm_hermes_process_matches "$name"; then printf 'alive' return 0 fi diff --git a/bin/fm-agent-process-lib.sh b/bin/fm-agent-process-lib.sh index edc5446075d..90d4ef83902 100644 --- a/bin/fm-agent-process-lib.sh +++ b/bin/fm-agent-process-lib.sh @@ -19,6 +19,8 @@ _FM_AGENT_PROCESS_LIB_DIR=${BASH_SOURCE[0]%/*} . "${_FM_AGENT_PROCESS_LIB_DIR:-/}/fm-session-lock-lib.sh" # shellcheck source=bin/fm-gemini-lib.sh . "${_FM_AGENT_PROCESS_LIB_DIR:-/}/fm-gemini-lib.sh" +# shellcheck source=bin/fm-hermes-process-lib.sh +. "${_FM_AGENT_PROCESS_LIB_DIR:-/}/fm-hermes-process-lib.sh" unset _FM_AGENT_PROCESS_LIB_DIR # fm_agent_process_classify_name: the single owner of the process-name @@ -56,7 +58,13 @@ fm_agent_process_classify_name() { # [argv0] -> agent|shell|other # daemon), so a `*polytoken*` glob never claims an unrelated command. # cortex (Cortex Code) is anchored too (verified, cortex 1.1.104: a # Bun-compiled single binary, comm=cortex). - agy|devin|command-code|polytoken|cortex) printf 'agent' ;; + agy|devin|command-code|polytoken|cortex|hermes) printf 'agent' ;; + # kiro (Kiro CLI) is anchored for the same reason as muse, omp, and agy: its + # live foreground process name is the bare word `kiro-cli` (verified, + # kiro-cli 2.21.4: a toolbox/aim-sandbox wrapper whose comm and tmux + # #{pane_current_command} both read kiro-cli), and a glob would claim + # unrelated commands containing that fragment. + kiro-cli) printf 'agent' ;; zsh|bash|sh|dash|ash|ksh|mksh|tcsh|csh|fish) printf 'shell' ;; *) if fm_harness_path_name "$path" >/dev/null || fm_harness_path_name "$argv0" >/dev/null; then @@ -106,6 +114,10 @@ fm_agent_process_classify() { # [pid] -> agent|shell|othe else by_argv0=$by_name fi + if fm_hermes_process_matches "$args" "$pid"; then + printf 'agent' + return 0 + fi if [ -n "$pid" ] && fm_gemini_pid_is_gemini "$pid"; then printf 'agent' return 0 diff --git a/bin/fm-backend.sh b/bin/fm-backend.sh index 04876ba0572..fa48e4c9df8 100644 --- a/bin/fm-backend.sh +++ b/bin/fm-backend.sh @@ -44,6 +44,9 @@ # auto-detection. Tern stamps TERM_PROGRAM=tern and TERN_PANE into every block # it spawns (a tmux server or Herdr client started inside a block inherits # them), so an inherited marker is not a selection; see docs/tern-backend.md. +# bin/backends/hc.sh adds herdr-core (`hc`, the core under Herdr.app), also +# EXPERIMENTAL, spawn-capable, and session-provider-only; see +# docs/hc-backend.md "Runtime auto-detection" for its selection contract. # Codex App is intentionally not in the known set yet. # docs/codex-app-backend.md owns that blocked backend contract. # @@ -52,7 +55,7 @@ # `backend=tmux` for a default-backend task, so existing and newly spawned # default-path metas stay byte-identical. Only a task spawned on a non-tmux # spawn-capable backend, currently herdr, zellij, orca, cmux, paseo, rex, or -# t3code, or tern, carries an explicit `backend=` line. +# t3code, tern, or hc, carries an explicit `backend=` line. # # Event-source framing (herdr-addendum "Events as the core abstraction"): a # backend's supervision surface is conceptually an EVENT SOURCE - it produces @@ -95,8 +98,10 @@ FM_BACKEND_CONFIG_DIR="${FM_CONFIG_OVERRIDE:-$FM_HOME/config}" # (docs/tern-backend.md); docs/verification/runtime-backends.md records what # was exercised against a real Tern. # codex-app remains deliberately absent; see docs/codex-app-backend.md. -FM_BACKEND_KNOWN="tmux herdr zellij orca cmux paseo rex t3code tern" -FM_BACKEND_SPAWN="tmux herdr zellij orca cmux paseo rex t3code tern" +# hc is EXPERIMENTAL and spawn-capable, session-provider-only like tmux +# (docs/hc-backend.md). +FM_BACKEND_KNOWN="tmux herdr zellij orca cmux paseo rex t3code tern hc" +FM_BACKEND_SPAWN="tmux herdr zellij orca cmux paseo rex t3code tern hc" # fm_backend_list_contains: whitespace-delimited membership without relying on # shell word splitting. fm-backend.sh is normally sourced by bash scripts, but @@ -169,9 +174,48 @@ fm_backend_is_known() { # # scrubbed entirely (no bundle id to inherit); NOT usable from inside # tmux, where the tmux server reparents to launchd and the chain never # reaches cmux - which is fine, because $TMUX already won there. +# The Herdr app (herdr-app, the native window over herdr-core) is checked +# LAST, after every marker a layer of this process's own runtime could set: +# herdr-core strips everything but terminal-identity vars when it starts a +# terminal, so an app-created terminal can never legitimately carry $TMUX, +# HERDR_ENV, Rex's pair, or a cmux marker - and a marker set that IS +# contradictory (e.g. cmux markers inherited by the whole core through an app +# launched from a cmux tab) keeps the earlier check's verdict instead of +# forcing a new switch. Exactly one app signal selects hc (docs/hc-backend.md +# "Runtime auto-detection" owns the contract): +# app-terminal: this process's own terminal carries a non-empty +# herdr.space label in its core's record - the exact condition the app +# projects a terminal into its window (herdr-app 0.1.11 window.rs +# groups by non-empty herdr.space and stamps it, with herdr.tab, on +# every terminal it creates). The claim is verified from the record, +# not from env: an app terminal's own env is deliberately identical to +# any other hc terminal's (the app asks for TERM_PROGRAM=herdr in +# terminal.run's env, and the core's child_env normalizes it to +# "ghostty" for every terminal, app-created or not). The read-only +# `hc get` runs only behind a live unix-connect probe of +# HC_TERMINAL_SOCKET, because `hc` STARTS a core on contact with a +# missing socket - a bare call could create state. The terminal's +# socket must also be the very core a spawn reaches - the adapter's +# ${HC_SESSION:-default} endpoint (fm_backend_detect_hc_core_sock), +# probed live rather than started, because the adapter's +# container_ensure STARTS a missing core - so a terminal on some other +# core never steers the spawn. +# A live herdr-app.sock or an open window is deliberately NOT a signal: +# every process on the host sees it, so it cannot tell a firstmate inside +# an app terminal from one on a bare shell while the window happens to be +# open elsewhere - verified live against the real app, where marker-less +# processes selected hc only because a window existed. The toolchain gate +# still applies: the adapter's own version gate (an executable FM_HC_BIN, +# or herdr-core/hc on PATH reporting `hc x.y.z` at or above its verified +# minimum - other tools ship an `hc` name and an older core would only +# fail the spawn's container check later), plus jq. A live app-terminal +# candidate failing it is NOT a detection: the probe records +# app-terminal-unusable in FM_BACKEND_DETECT_SIGNAL so fm_backend_resolve +# can say why it did not switch. # Callers needing the winning signal read FM_BACKEND_DETECT_SIGNAL (set to -# TMUX, HERDR_ENV, TERM_PROGRAM, CMUX_WORKSPACE_ID, bundle-id, or ancestry) and -# FM_BACKEND_DETECTED after a direct (non-command-substitution) call. +# TMUX, HERDR_ENV, TERM_PROGRAM, CMUX_WORKSPACE_ID, bundle-id, ancestry, +# or app-terminal) and FM_BACKEND_DETECTED after a direct +# (non-command-substitution) call. FM_BACKEND_CMUX_BUNDLE_ID="com.cmuxterm.app" fm_backend_detect() { @@ -206,6 +250,11 @@ fm_backend_detect() { printf 'cmux' return 0 fi + if fm_backend_detect_herdr_app; then + FM_BACKEND_DETECTED=hc + printf 'hc' + return 0 + fi # Paseo is deliberately NOT detected here. Its PASEO_* markers and the # sh.paseo.desktop bundle id reach every descendant process, including a # tmux server started from a Paseo tab, so their presence never proves the @@ -214,6 +263,100 @@ fm_backend_detect() { # every descendant process of a Tern block, including a tmux server or Herdr # client started inside it, so their presence never proves the captain chose # Tern (docs/tern-backend.md "Selection is explicit"). + # Bare hc markers without the Herdr app claim above are not a selection + # either: HC_TERMINAL_ID reaches every process an hc terminal starts + # (docs/hc-backend.md "Runtime auto-detection"). + return 1 +} + +# fm_backend_unix_sock_live: a process is accepting unix-socket connections +# at right now - a real connect, not liveness by file shape: a stale +# socket file refuses, an absent one never binds. python3 is required +# (firstmate's toolchain already relies on it); nc's -U/-z flags are +# unreliable for this on macOS. +fm_backend_unix_sock_live() { # + command -v python3 >/dev/null 2>&1 || return 1 + [ -S "$1" ] || return 1 + python3 - "$1" <<'PY' >/dev/null 2>&1 +import socket, sys +s = socket.socket(socket.AF_UNIX) +s.settimeout(0.75) +try: + s.connect(sys.argv[1]) +except OSError: + sys.exit(1) +PY +} + +# fm_backend_detect_hc_usable: an hc selection can only succeed when the +# adapter's own binary gate passes - fm_backend_hc_version_check +# (bin/backends/hc.sh, loaded through fm_backend_source exactly as spawn +# loads it) resolves FM_HC_BIN or herdr-core/hc on PATH and requires a +# self-identifying `hc x.y.z` at or above FM_BACKEND_HC_MIN_VERSION, since +# unrelated tools ship an `hc` name and an older core would only be refused +# by the spawn's container gate later - plus jq, which every adapter +# surface needs. Checking here keeps a detected app signal from silently +# steering a spawn into an hc the adapter would abort on. +fm_backend_detect_hc_usable() { + command -v jq >/dev/null 2>&1 || return 1 + fm_backend_source hc >/dev/null 2>&1 || return 1 + fm_backend_hc_version_check >/dev/null 2>&1 +} + +# fm_backend_detect_hc_core_sock: the socket of the core the hc adapter +# reaches for a new spawn, resolved WITHOUT contacting hc (any hc call starts +# a missing core). FM_HC_SOCKET is the adapter's deliberate pin; otherwise +# hc's own layout for ${HC_SESSION:-default}, measured on hc 0.1.14: +# $XDG_RUNTIME_DIR/herdr-core/.sock when XDG_RUNTIME_DIR is set, +# else ${XDG_STATE_HOME:-$HOME/.local/state}/herdr-core//.sock. +# A layout this misses only fails the liveness probe - no switch, never a +# wrong or newly started core. +fm_backend_detect_hc_core_sock() { + local session=${HC_SESSION:-default} + if [ -n "${FM_HC_SOCKET:-}" ]; then + printf '%s' "$FM_HC_SOCKET" + elif [ -n "${XDG_RUNTIME_DIR:-}" ]; then + printf '%s/herdr-core/%s.sock' "$XDG_RUNTIME_DIR" "$session" + else + printf '%s/herdr-core/%s/%s.sock' "${XDG_STATE_HOME:-$HOME/.local/state}" "$session" "$session" + fi +} + +# fm_backend_detect_herdr_app_terminal: this process's own terminal record +# carries a non-empty herdr.space label, the exact condition the app uses to +# show a terminal in the window. The caller has already proven +# HC_TERMINAL_SOCKET live (a bare `hc get` against a missing socket starts a +# core) and the toolchain usable, which also sourced the adapter, so +# fm_backend_hc_bin is available here. A get failure or empty label is a miss. +fm_backend_detect_herdr_app_terminal() { + local bin space + bin=$(fm_backend_hc_bin) || return 1 + # `env -u HC_SESSION` lets the pinned socket alone name the endpoint; an + # inherited session could point at another core. + space=$(env -u HC_SESSION HC_SOCKET="$HC_TERMINAL_SOCKET" "$bin" get "$HC_TERMINAL_ID" --json 2>/dev/null \ + | jq -r '.labels["herdr.space"] // ""' 2>/dev/null) || return 1 + [ -n "$space" ] +} + +# fm_backend_detect_herdr_app: the single Herdr app signal documented on +# fm_backend_detect. On a detection sets FM_BACKEND_DETECT_SIGNAL to +# app-terminal and returns 0. A live app-terminal candidate with no usable +# hc toolchain sets app-terminal-unusable and returns 1; with no such +# candidate it leaves the globals alone and returns 1. +fm_backend_detect_herdr_app() { + local core + if [ -n "${HC_TERMINAL_ID:-}" ] && [ -n "${HC_TERMINAL_SOCKET:-}" ] \ + && fm_backend_unix_sock_live "$HC_TERMINAL_SOCKET"; then + if ! fm_backend_detect_hc_usable; then + FM_BACKEND_DETECT_SIGNAL=app-terminal-unusable + return 1 + fi + core=$(fm_backend_detect_hc_core_sock) + if [ "$HC_TERMINAL_SOCKET" -ef "$core" ] && fm_backend_detect_herdr_app_terminal; then + FM_BACKEND_DETECT_SIGNAL=app-terminal + return 0 + fi + fi return 1 } @@ -279,7 +422,7 @@ fm_backend_detect_cmux_app_is_ancestor() { } # fm_backend_explicit_name: the explicitly selected backend, FM_BACKEND env -# then config/backend; returns 1 when neither selects one. fm_backend_name and +# then config/backend; returns 1 when neither selects one. fm_backend_resolve and # the away-mode t3code discovery gate (bin/fm-supervisor-target-lib.sh) share it. fm_backend_explicit_name() { local line v @@ -298,39 +441,66 @@ fm_backend_explicit_name() { return 1 } -# fm_backend_name: resolve the ACTIVE backend for a NEW spawn, absent an -# explicit per-task override. Precedence: FM_BACKEND env, then config/backend -# (a single word on its first non-empty line, mirroring config/crew-harness), -# then runtime auto-detection (fm_backend_detect), then default tmux. A -# per-task `--backend` flag is parsed by the caller (fm-spawn.sh) and takes -# precedence over this resolution entirely; it is not read here. Auto-detect -# fires only when nothing was explicitly configured, so an explicit setting -# always wins. Auto-detected herdr and rex stay silent like tmux. Auto-detected -# cmux prints a loud stderr notice because it remains experimental. The cmux +# fm_backend_resolve: resolve the ACTIVE backend for a NEW spawn into +# FM_BACKEND_RESOLVED, absent an explicit per-task override. Precedence: +# FM_BACKEND env, then config/backend (a single word on its first non-empty +# line, mirroring config/crew-harness), then runtime auto-detection +# (fm_backend_detect), then default tmux. A per-task `--backend` flag is +# parsed by the caller (fm-spawn.sh) and takes precedence over this +# resolution entirely; it is not read here. Auto-detect fires only when +# nothing was explicitly configured, so an explicit setting always wins and +# leaves FM_BACKEND_DETECTED/FM_BACKEND_DETECT_SIGNAL empty. Called directly +# (not in a command substitution), the one detection that chose the backend +# also leaves its signal globals for the caller's provenance record. +# Auto-detected herdr and rex stay silent like tmux. Auto-detected cmux +# prints a loud stderr notice because it remains experimental. The cmux # notice names the winning signal, so a fallback-detected cmux (bundle id or # ancestry, after the claude wrapper stripped CMUX_WORKSPACE_ID) is visibly -# distinct from the primary-marker case. +# distinct from the primary-marker case. An hc selection from the Herdr app +# signal prints the same class of notice; a live app-terminal candidate that +# cannot select (no usable toolchain) is not a detection, and a notice names +# the missing piece instead of switching. # Paseo and tern are never auto-detected, so they only ever come out of this # function from FM_BACKEND or config/backend. -fm_backend_name() { - local detected marker - fm_backend_explicit_name && return 0 - # Called directly (not in a command substitution) so the detect signal - # globals survive into the notice below. +fm_backend_resolve() { + local marker toolchain + FM_BACKEND_DETECTED="" + FM_BACKEND_DETECT_SIGNAL="" + FM_BACKEND_RESOLVED=$(fm_backend_explicit_name) && return 0 if fm_backend_detect >/dev/null; then - detected=$FM_BACKEND_DETECTED - if [ "$detected" = cmux ]; then - case "$FM_BACKEND_DETECT_SIGNAL" in - bundle-id) marker="FALLBACK signal __CFBundleIdentifier=$FM_BACKEND_CMUX_BUNDLE_ID; CMUX_WORKSPACE_ID absent, stripped by cmux's bundled claude wrapper" ;; - ancestry) marker="FALLBACK signal process-ancestry reaching the running cmux app; CMUX_WORKSPACE_ID absent, stripped by cmux's bundled claude wrapper" ;; - *) marker="CMUX_WORKSPACE_ID" ;; - esac - echo "NOTICE: auto-detected cmux runtime ($marker) - spawning into the EXPERIMENTAL cmux backend. Set config/backend or pass --backend tmux to opt out." >&2 - fi - printf '%s' "$detected" + FM_BACKEND_RESOLVED=$FM_BACKEND_DETECTED + case "$FM_BACKEND_RESOLVED" in + cmux) + case "$FM_BACKEND_DETECT_SIGNAL" in + bundle-id) marker="FALLBACK signal __CFBundleIdentifier=$FM_BACKEND_CMUX_BUNDLE_ID; CMUX_WORKSPACE_ID absent, stripped by cmux's bundled claude wrapper" ;; + ancestry) marker="FALLBACK signal process-ancestry reaching the running cmux app; CMUX_WORKSPACE_ID absent, stripped by cmux's bundled claude wrapper" ;; + *) marker="CMUX_WORKSPACE_ID" ;; + esac + echo "NOTICE: auto-detected cmux runtime ($marker) - spawning into the EXPERIMENTAL cmux backend. Set config/backend or pass --backend tmux to opt out." >&2 + ;; + hc) + case "$FM_BACKEND_DETECT_SIGNAL" in + app-terminal) marker="a Herdr-app-projected terminal (${HC_SESSION:-default}:${HC_TERMINAL_ID:-?} carries a herdr.space label)" ;; + *) marker="$FM_BACKEND_DETECT_SIGNAL" ;; + esac + echo "NOTICE: auto-detected $marker - spawning into the EXPERIMENTAL hc backend with Herdr app presentation (herdr.space/herdr.tab labels). Set config/backend or pass --backend to opt out." >&2 + ;; + esac return 0 fi - printf 'tmux' + toolchain="no usable hc toolchain (an executable FM_HC_BIN, or herdr-core/hc on PATH reporting 'hc x.y.z' at or above the verified minimum ${FM_BACKEND_HC_MIN_VERSION:-version}, and jq)" + case "$FM_BACKEND_DETECT_SIGNAL" in + app-terminal-unusable) + echo "NOTICE: running in a live hc terminal (${HC_SESSION:-default}:${HC_TERMINAL_ID:-?}) but $toolchain can verify a Herdr app projection; staying on the default backend. See docs/hc-backend.md." >&2 + ;; + esac + FM_BACKEND_RESOLVED=tmux +} + +# fm_backend_name: fm_backend_resolve for callers that only need the name. +fm_backend_name() { + fm_backend_resolve + printf '%s' "$FM_BACKEND_RESOLVED" } # fm_backend_validate: refuse an unknown backend LOUDLY. Silent on success. @@ -376,6 +546,7 @@ fm_backend_required_tools() { # paseo) printf '%s' 'paseo jq treehouse' ;; rex) printf '%s' 'rex jq treehouse' ;; tern) printf '%s' 'tern jq treehouse' ;; + hc) printf '%s' 'hc jq treehouse' ;; orca) printf '%s' 'orca' ;; t3code) printf '%s' 'node treehouse' ;; *) return 1 ;; @@ -399,6 +570,10 @@ fm_backend_required_tool_available() { # fm_backend_source rex >/dev/null 2>&1 || return 1 fm_backend_rex_bin >/dev/null 2>&1 ;; + hc:hc) + fm_backend_source hc >/dev/null 2>&1 || return 1 + fm_backend_hc_bin >/dev/null 2>&1 + ;; *) command -v "$tool" >/dev/null 2>&1 ;; esac } @@ -503,6 +678,39 @@ fm_backend_endpoint_u64_valid() { # [ "${#1}" -le 20 ] } +# The repair path printed with the worktree-identity refusal below, so a record +# that cannot name its copy is a repairable state rather than a dead end. +# One path serves all three refused shapes, because each one ends in the same +# place: the record has to carry exactly one worktree= line, whether that means +# adding it, filling it in, or dropping all but the right one of several. +# It names only evidence this system writes by itself: the project's own +# worktree list, which maps the task's recorded branch back to its copy, and the +# record snapshot bin/fm-control.sh preserves before a relaunch. +# No command here restores the line: only the operator can say which copy a task +# holds, and a one-shot rewrite would make a damaged record convenient to keep. +# Restoring it stays safe when the pool slot has since been handed to another +# task: bin/fm-teardown.sh reads that slot's own claim (bin/fm-wake-lib.sh's +# fm_treehouse_slot_owner_marker) and then touches nothing under it, while a +# slot carrying no claim carries no such proof and leaves the worktree listing +# as the only evidence. +fm_backend_worktree_identity_repair_hint() { # + local meta=$1 prior project branch + prior="${meta%.meta}.control-relaunch.meta-prior" + project=$(fm_meta_get "$meta" project) + branch=$(fm_meta_get "$meta" branch) + echo "The record needs exactly one worktree= line naming the copy this task holds: add it, fill it in, or keep only the right one of several, then re-run." + if [ -n "$project" ]; then + # Quoted, so the printed command survives a project path with spaces and + # can be copied and run exactly as it reads. + echo "Find the copy checked out on this task's recorded branch${branch:+ $branch} with:" + echo "git -C $(printf '%q' "$project") worktree list" + fi + if [ -f "$prior" ] && [ ! -L "$prior" ]; then + echo "The record as it stood before the last relaunch still carries that line at $prior." + fi + echo "A pool slot handed on to another task since is still safe to name: cleanup reads that slot's own .fm-slot-owner claim and then leaves the slot untouched, while a slot with no claim is proved only by that worktree listing." +} + fm_backend_validate_task_endpoint() { # local meta=$1 id=$2 backend_count backend window worktree project binding_count binding local session pane recorded_session workspace tab terminal worktree_id surface rex_window rex_server thread block @@ -522,6 +730,7 @@ fm_backend_validate_task_endpoint() { # } worktree=$(fm_backend_meta_exact_value "$meta" worktree) || { echo "REFUSED: task $id has a missing, empty, or ambiguous worktree identity; preserving task state." >&2 + fm_backend_worktree_identity_repair_hint "$meta" >&2 return 1 } project=$(fm_backend_meta_exact_value "$meta" project) || { @@ -712,6 +921,21 @@ fm_backend_validate_task_endpoint() { # return 1 fi ;; + hc) + [ "$binding" = "$id" ] || { + echo "REFUSED: hc endpoint metadata for task $id lacks an exact task binding; preserving task state." >&2 + return 1 + } + recorded_session=$(fm_backend_meta_exact_value "$meta" hc_session) || recorded_session= + terminal=$(fm_backend_meta_exact_value "$meta" hc_terminal) || terminal= + if [ -z "$recorded_session" ] || [ -z "$terminal" ] \ + || [ "$window" != "$recorded_session:$terminal" ] \ + || ! fm_backend_endpoint_atom_valid "$recorded_session" \ + || ! fm_backend_endpoint_atom_valid "$terminal"; then + echo "REFUSED: hc endpoint metadata for task $id is malformed or inconsistent; preserving task state." >&2 + return 1 + fi + ;; esac # shellcheck disable=SC2034 # Output globals are consumed by sourcing callers. FM_BACKEND_VALIDATED_BACKEND=$backend @@ -820,10 +1044,10 @@ fm_backend_source() { # # word-split an unquoted expansion, so a space-separated string is one path. case "$name" in tmux) - set -- fm-tmux-lib.sh fm-composer-lib.sh fm-cursor-lib.sh fm-session-lock-lib.sh fm-agent-process-lib.sh fm-gemini-lib.sh + set -- fm-tmux-lib.sh fm-composer-lib.sh fm-cursor-lib.sh fm-session-lock-lib.sh fm-agent-process-lib.sh fm-gemini-lib.sh fm-hermes-process-lib.sh ;; herdr) - set -- fm-composer-lib.sh fm-transition-lib.sh fm-agent-process-lib.sh fm-session-lock-lib.sh fm-gemini-lib.sh + set -- fm-composer-lib.sh fm-transition-lib.sh fm-agent-process-lib.sh fm-session-lock-lib.sh fm-gemini-lib.sh fm-hermes-process-lib.sh ;; zellij) set -- fm-backend-hometag-lib.sh fm-composer-lib.sh @@ -835,13 +1059,16 @@ fm_backend_source() { # set -- fm-backend-hometag-lib.sh fm-composer-lib.sh ;; rex) - set -- fm-backend-hometag-lib.sh fm-composer-lib.sh fm-agent-process-lib.sh fm-session-lock-lib.sh fm-gemini-lib.sh + set -- fm-backend-hometag-lib.sh fm-composer-lib.sh fm-agent-process-lib.sh fm-session-lock-lib.sh fm-gemini-lib.sh fm-hermes-process-lib.sh ;; t3code) set -- fm-composer-lib.sh fm-transition-lib.sh ;; tern) - set -- fm-backend-hometag-lib.sh fm-composer-lib.sh fm-timeout-lib.sh fm-agent-process-lib.sh fm-session-lock-lib.sh fm-gemini-lib.sh fm-cursor-lib.sh + set -- fm-backend-hometag-lib.sh fm-composer-lib.sh fm-timeout-lib.sh fm-agent-process-lib.sh fm-session-lock-lib.sh fm-gemini-lib.sh fm-hermes-process-lib.sh fm-cursor-lib.sh + ;; + hc) + set -- fm-backend-hometag-lib.sh fm-composer-lib.sh fm-agent-process-lib.sh fm-session-lock-lib.sh fm-gemini-lib.sh fm-hermes-process-lib.sh fm-cursor-lib.sh ;; *) return 1 @@ -916,6 +1143,13 @@ fm_backend_source() { # _FM_BACKEND_TERN_SOURCED=1 fi ;; + hc) + if [ -z "${_FM_BACKEND_HC_SOURCED:-}" ]; then + # shellcheck source=/dev/null + . "$adapter" || return 1 + _FM_BACKEND_HC_SOURCED=1 + fi + ;; esac } @@ -992,6 +1226,7 @@ fm_backend_capture() { # [expected-label] rex) fm_backend_rex_capture "$@" ;; t3code) fm_backend_t3code_capture "$@" ;; tern) fm_backend_tern_capture "$@" ;; + hc) fm_backend_hc_capture "$@" ;; *) echo "error: no capture implementation for backend '$backend'" >&2; return 1 ;; esac } @@ -1008,7 +1243,8 @@ fm_backend_capture() { # [expected-label] # (bin/backends/rex.sh's header owns the derivation and its verified basis). # tern reads the viewport with `tern capture --surfaces` and no `--scrollback` # (trimmed to the block's rows); its own capture opts into history with the flag. -FM_BACKEND_VISIBLE_CAPTURE="tmux herdr zellij rex tern" +# hc reads the screen alone with `hc read --lines 0`. +FM_BACKEND_VISIBLE_CAPTURE="tmux herdr zellij rex tern hc" # fm_backend_visible_capture_supported: whether can read the visible # viewport WITHOUT scrollback. Callers that must not mistake a scrolled-away @@ -1046,6 +1282,7 @@ fm_backend_send_key() { # [expected-label] rex) fm_backend_rex_send_key "$@" ;; t3code) fm_backend_t3code_send_key "$@" ;; tern) fm_backend_tern_send_key "$@" ;; + hc) fm_backend_hc_send_key "$@" ;; *) echo "error: no send-key implementation for backend '$backend'" >&2; return 1 ;; esac } @@ -1090,6 +1327,7 @@ fm_backend_send_text_submit() { # &2; rc=1 ;; esac fm_composer_dialog_sink_release @@ -1126,6 +1364,7 @@ fm_backend_kill() { # rex) fm_backend_rex_kill "$@" ;; t3code) fm_backend_t3code_kill "$1" ;; tern) fm_backend_tern_kill "$@" ;; + hc) fm_backend_hc_kill "$@" ;; *) echo "error: no kill implementation for backend '$backend'" >&2; return 1 ;; esac } @@ -1166,6 +1405,7 @@ fm_backend_busy_state() { # rex) fm_backend_rex_busy_state "$@" ;; t3code) fm_backend_t3code_busy_state "$@" ;; tern) fm_backend_tern_busy_state "$@" ;; + hc) fm_backend_hc_busy_state "$@" ;; *) printf 'unknown' ;; esac } @@ -1196,6 +1436,7 @@ fm_backend_composer_state() { # [expected-label] -> empty|pe zellij) fm_backend_zellij_composer_state "$@" ;; t3code) fm_backend_t3code_composer_state "$@" ;; tern) fm_backend_tern_composer_state "$@" ;; + hc) fm_backend_hc_composer_state "$@" ;; *) printf 'unknown' ;; esac } @@ -1266,6 +1507,10 @@ fm_backend_target_exists() { # [expected-label] fm_backend_source tern || return 1 fm_backend_tern_target_ready "$target" "$expected_label" ;; + hc) + fm_backend_source hc || return 1 + fm_backend_hc_target_ready "$target" "$expected_label" + ;; *) return 1 ;; @@ -1300,6 +1545,8 @@ fm_backend_target_exists() { # [expected-label] # Tern answers through the same process classifier from `tern process`, but # bin/fm-control-lib.sh's fm_control_backend_state_verified does not list it, so # no control-plane verb treats a Tern verdict as proof that an agent stopped. +# hc classifies the one foreground process `hc get` reports through the same +# classifier and is likewise absent from fm_control_backend_state_verified. fm_backend_agent_state() { # local backend=$1 target=$2 fm_backend_source "$backend" || { printf 'unverified'; return 0; } @@ -1309,6 +1556,7 @@ fm_backend_agent_state() { # rex) fm_backend_rex_agent_state "$target" ;; t3code) fm_backend_t3code_agent_state "$target" ;; tern) fm_backend_tern_agent_state "$target" ;; + hc) fm_backend_hc_agent_state "$target" ;; *) printf 'unverified' ;; esac } diff --git a/bin/fm-backlog-transition-lib.sh b/bin/fm-backlog-transition-lib.sh index 97a079d7cf6..c264067ae36 100644 --- a/bin/fm-backlog-transition-lib.sh +++ b/bin/fm-backlog-transition-lib.sh @@ -42,10 +42,10 @@ # with it the completion links, so a process killed between the two halves would # leave nothing to reconstruct the close from. It writes # `state/.backlog-close` first, and removes it once the close lands. -# The writer and replay share one complete-record validator, and teardown stages -# that record before destructive cleanup, so it never publishes or acts on a close -# replay would reject. The validator pins the data path to this home's configured -# root before any recovery mutation, then re-runs exactly that close. +# The writer and replay share one complete-record validator, so teardown never +# publishes or acts on a close replay would reject. The validator pins the data +# path to this home's configured root before any recovery mutation, then re-runs +# exactly that close. # `tasks-axi done` on an already-closed task backfills links # without moving the close date, so replay is idempotent. Spawn needs no marker: # it publishes the meta first, so a crash diff --git a/bin/fm-bootstrap.sh b/bin/fm-bootstrap.sh index 6f24b00567c..325d43ac62e 100755 --- a/bin/fm-bootstrap.sh +++ b/bin/fm-bootstrap.sh @@ -216,6 +216,8 @@ DATA="${FM_DATA_OVERRIDE:-$FM_HOME/data}" . "$SCRIPT_DIR/fm-remote-readiness-lib.sh" # shellcheck source=bin/fm-timeout-lib.sh disable=SC1091 . "$SCRIPT_DIR/fm-timeout-lib.sh" +# shellcheck source=bin/fm-credential-source-lib.sh disable=SC1091 +. "$SCRIPT_DIR/fm-credential-source-lib.sh" # Shared secondmate endpoint probe + guarded relaunch; the watcher's poll tick # drives the same library so session start and ordinary supervision recover # from identical evidence through an identical path. @@ -869,6 +871,7 @@ install_cmd() { case "$1" in tmux|node|git|gh|curl|jq|orca|zellij) echo "brew install $1 # or the platform's package manager" ;; cmux) echo "brew install --cask cmux # or see https://cmux.com" ;; + hc) echo "bin/fm-install-hc.sh && add to PATH # or see docs/hc-backend.md" ;; treehouse) echo "curl -fsSL https://kunchenguid.github.io/treehouse/install.sh | sh" ;; no-mistakes) echo "curl -fsSL https://raw.githubusercontent.com/kunchenguid/no-mistakes/main/docs/install.sh | sh" ;; gh-axi|chrome-devtools-axi|lavish-axi) echo "npm install -g $1 && $1 setup hooks" ;; @@ -1154,7 +1157,7 @@ EOF } crew_dispatch_validate() { - local file err verified_harnesses typed_key typed_active=false + local file cred err verified_harnesses typed_key typed_active=false file="$CONFIG/crew-dispatch.json" [ -f "$file" ] || return 0 if ! command -v jq >/dev/null 2>&1; then @@ -1167,11 +1170,17 @@ crew_dispatch_validate() { fi typed_key=$TYPESAFE_API_KEY_PRIVATE [ -n "$typed_key" ] || typed_key=$(fmx_env_get TYPESAFE_API_KEY "$FM_HOME/.env") - [ -z "$typed_key" ] || typed_active=true + cred="$CONFIG/dispatch-credential-sources.json" + if [ -n "$typed_key" ]; then + typed_active=true + elif [ -f "$cred" ] && fm_credsrc_check "$cred" typesafe openrouter >/dev/null && + fm_credsrc_declares "$cred" typesafe; then + typed_active=true + fi if $typed_active; then verified_harnesses=$(fm_control_harnesses | jq -Rsc 'split("\n") | map(select(length > 0))') else - verified_harnesses='["claude","codex","opencode","pi","pi-signed","grok","kimi","cursor","agy","muse","rovo","omp","devin","commandcode","polytoken","cortex"]' + verified_harnesses='["claude","codex","opencode","pi","pi-signed","grok","kimi","cursor","agy","muse","rovo","omp","devin","commandcode","polytoken","cortex","kiro","hermes"]' fi err=$(jq -r --argjson typed "$typed_active" --argjson verified_harnesses "$verified_harnesses" --arg provider_re "$FM_QUOTA_PROVIDER_ID_RE" --arg seat_re "^${FM_DOCK_SEAT_NAME_RE}\\z" ' def seat_ok($s): ($s | type) == "string" and ($s | test($seat_re)); @@ -1185,6 +1194,7 @@ crew_dispatch_validate() { elif $h == "codex" then ((["low","medium","high","xhigh"] | index($e)) != null or ($e == "max" and $m == "gpt-5.6-luna")) elif $h == "grok" then (["low","medium","high"] | index($e)) elif $h == "agy" then (["low","medium","high"] | index($e)) + elif $h == "kiro" then (["low","medium","high","xhigh","max"] | index($e)) elif $h == "pi" or $h == "pi-signed" or $h == "omp" then (["low","medium","high","xhigh","max"] | index($e)) elif $h == "muse" then (["low","medium","high","xhigh","max"] | index($e)) elif $h == "rovo" then (["low","medium","high","max"] | index($e)) @@ -1218,6 +1228,8 @@ crew_dispatch_validate() { end); def malformed_profile_floors($items): ($items | any(has("floor") and floor_bad(.floor; false))); + def malformed_quota_scopes($items): + ($items | any(has("quota_scope") and (((.quota_scope | type) != "string") or (.quota_scope | length) == 0))); def bad_efforts: configured_profiles | map({h: .harness, m: .model, e: .effort}) @@ -1241,6 +1253,7 @@ crew_dispatch_validate() { if $typed then "use profile model and effort must be non-empty strings, and provider must match ^[a-z0-9]+(-[a-z0-9]+)*\\z when present" else "use profile model and effort must be non-empty strings when present" end + elif $typed and malformed_quota_scopes([(.rules // [])[]? | profiles(.use?)[]?]) then "use profile quota_scope must be a non-empty string when present" elif $typed and malformed_profile_floors([(.rules // [])[]? | profiles(.use?)[]?]) then "use profile floor needs scope and min_percent 0..100" elif $typed and ([(.rules // [])[]? | select(has("approval") and .approval != "captain")] | length > 0) then "approval must be \"captain\" when present" elif $typed and ([(.rules // [])[]? | select(has("floor") and floor_bad(.floor; true))] | length > 0) then "rule floor needs scope, min_percent 0..100, and provider matching ^[a-z0-9]+(-[a-z0-9]+)*\\z" @@ -1248,6 +1261,7 @@ crew_dispatch_validate() { elif [(.rules // [])[]? | select(has("select") and ((.select? | type) != "string" or (.select | length) == 0))] | length > 0 then "select must be a non-empty string" elif [(.rules // [])[]? | .select? // empty | select(. != "quota-balanced")] | length > 0 then "unknown select: " + ([ (.rules // [])[]? | .select? // empty | select(. != "quota-balanced") ] | unique | join(", ")) + elif has("shared_quota") and (.shared_quota | type) != "boolean" then "shared_quota must be true or false when present" elif has("default") and ((.default | type) != "object" and (.default | type) != "array") then "default must be a profile object or non-empty profile array" elif has("default") and ((.default | type) == "array" and (.default | length) == 0) then "default needs at least one profile" elif has("default") and ([profiles(.default)[]? | select(type != "object")] | length) > 0 then "each default profile must be an object" @@ -1258,6 +1272,7 @@ crew_dispatch_validate() { if $typed then "default profile model and effort must be non-empty strings, and provider must match ^[a-z0-9]+(-[a-z0-9]+)*\\z when present" else "default profile model and effort must be non-empty strings when present" end + elif $typed and has("default") and malformed_quota_scopes([profiles(.default)[]?]) then "default profile quota_scope must be a non-empty string when present" elif $typed and has("default") and malformed_profile_floors([profiles(.default)[]?]) then "default profile floor needs scope and min_percent 0..100" else (configured_profiles diff --git a/bin/fm-busy-event.sh b/bin/fm-busy-event.sh index 76f80272086..3f8a24fdd09 100755 --- a/bin/fm-busy-event.sh +++ b/bin/fm-busy-event.sh @@ -17,7 +17,9 @@ # --source S --event E [--turn-ended] [--if-seq N] # Append one lifecycle event: validate the gen against the armed # sidecar and optional --if-seq, advance seq under the lock, and atomically -# replace the record. +# replace the record. A non-busy event also clears active stream evidence +# and closes any request ordering record, so a settled or unknown worker +# never leaves an observation standing. # --turn-ended also touches state/.turn-ended, the watcher # notification, under the same lock, so a writer whose gen was refused, # or whose incarnation is replaced right after, never signals the @@ -40,6 +42,13 @@ # an old task from retiring a newly armed incarnation. A missing sidecar # is already retired, so any orphan record is removed idempotently. # +# stream --gen G --request N +# Mutate the optional Pi stream contract owned by fm-busy-lib.sh. +# Open requires a strictly newer request; open and progress both require +# a busy pi-ext record, and progress and close must name the recorded +# request. Close removes active evidence before persisting closed +# ordering, so a persistence failure cannot leave observation active. +# # Exit codes: 0 applied; 1 refused (stale gen, unarmed task, lock timeout, # invalid input); 2 usage. Adapter hook command lines append `|| true` so a # refusal never breaks the harness's own lifecycle. @@ -51,6 +60,7 @@ usage: fm-busy-event.sh arm [--state busy|idle|unknown] [--source S] [--event E] fm-busy-event.sh apply (--gen G | --current-gen) --source S --event E [--turn-ended] [--if-seq N] fm-busy-event.sh progress --gen G + fm-busy-event.sh stream --gen G --request N fm-busy-event.sh retire (--gen G | --current-gen) See the header comment for the full contract. EOF @@ -63,7 +73,7 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" CMD=${1:-} case "$CMD" in - arm|apply|progress|retire) shift ;; + arm|apply|progress|stream|retire) shift ;; *) usage ;; esac @@ -81,6 +91,8 @@ TURN_ENDED=0 IF_SEQ= SOURCE= EVENT= +STREAM_OP= +REQUEST= if [ "$CMD" = apply ]; then NEW_STATE=${1:-} case "$NEW_STATE" in busy|idle|unknown) shift ;; *) usage ;; esac @@ -88,6 +100,9 @@ elif [ "$CMD" = arm ]; then NEW_STATE=busy SOURCE=fm-spawn EVENT=launch-brief +elif [ "$CMD" = stream ]; then + STREAM_OP=${1:-} + case "$STREAM_OP" in open|progress|close) shift ;; *) usage ;; esac fi while [ $# -gt 0 ]; do case "$1" in @@ -103,6 +118,7 @@ while [ $# -gt 0 ]; do case "$IF_SEQ" in ''|*[!0-9]*) usage ;; esac shift 2 || usage ;; + --request) REQUEST=${2:-}; shift 2 || usage ;; *) usage ;; esac done @@ -113,6 +129,10 @@ if [ "$CMD" = apply ] || [ "$CMD" = arm ]; then fi [ "$CMD" != progress ] || [ "$USE_CURRENT_GEN" = 0 ] || usage +if [ "$CMD" = stream ]; then + [ "$USE_CURRENT_GEN" = 0 ] || usage + fm_busy_stream_uint_valid "$REQUEST" || usage +fi REC=$(fm_busy_record_path "$STATE" "$ID") GEN_FILE=$(fm_busy_gen_path "$STATE" "$ID") @@ -166,6 +186,23 @@ write_record() { # mv -f "$tmp" "$REC" } +clear_stream() { + local order order_gen request phase + # Active observation must disappear even when closing-order persistence fails. + # The retained order still denies duplicate open, and progress needs a marker. + rm -f "$STATE/$ID.model-stream" || return 1 + if order=$(fm_busy_stream_order_read "$STATE" "$ID"); then + read -r order_gen request phase <<< "$order" + printf 'v1 gen=%s request=%s phase=closed\n' "$order_gen" "$request" > "$STATE/$ID.model-stream-order.tmp.$$" && + mv -f "$STATE/$ID.model-stream-order.tmp.$$" "$STATE/$ID.model-stream-order" || return 1 + fi + return 0 +} + +clear_stream_incarnation() { + rm -f "$STATE/$ID.model-stream" "$STATE/$ID.model-stream-order" "$STATE/$ID.model-progress" +} + old_umask=$(umask) umask 077 @@ -174,7 +211,7 @@ if [ "$CMD" = arm ]; then lock_acquire || exit 1 { printf '%s\n' "$GEN" > "$GEN_FILE.tmp.$$" && mv -f "$GEN_FILE.tmp.$$" "$GEN_FILE" \ - && write_record "$GEN" 1 && rm -f "$STATE/$ID.progress" + && write_record "$GEN" 1 && rm -f "$STATE/$ID.progress" && clear_stream_incarnation } || { lock_release; umask "$old_umask"; echo "error: arm failed for $ID" >&2; exit 1; } lock_release umask "$old_umask" @@ -197,12 +234,12 @@ fi lock_acquire || { umask "$old_umask"; exit 1; } CURRENT=$(fm_busy_current_gen "$STATE" "$ID") || { if [ "$CMD" = retire ] && [ ! -e "$GEN_FILE" ] && [ ! -L "$GEN_FILE" ]; then - rm -f "$REC" "$STATE/$ID.progress" || { + if ! { rm -f "$REC" "$STATE/$ID.progress" && clear_stream_incarnation; }; then lock_release umask "$old_umask" echo "error: busy-state retirement failed for $ID" >&2 exit 1 - } + fi lock_release umask "$old_umask" exit 0 @@ -222,12 +259,12 @@ if [ "$GEN" != "$CURRENT" ]; then exit 1 fi if [ "$CMD" = retire ]; then - rm -f "$GEN_FILE" "$REC" "$STATE/$ID.progress" || { + if ! { rm -f "$GEN_FILE" "$REC" "$STATE/$ID.progress" && clear_stream_incarnation; }; then lock_release umask "$old_umask" echo "error: busy-state retirement failed for $ID" >&2 exit 1 - } + fi lock_release umask "$old_umask" exit 0 @@ -238,6 +275,49 @@ if [ "$CMD" = progress ]; then umask "$old_umask" exit 0 fi +if [ "$CMD" = stream ]; then + order_gen=$GEN old_request=0 phase=closed rev=1 + if [ -e "$STATE/$ID.model-stream-order" ]; then + order=$(fm_busy_stream_order_read "$STATE" "$ID") || { lock_release; exit 1; } + read -r order_gen old_request phase <<< "$order" + [ "$order_gen" = "$GEN" ] || { lock_release; exit 1; } + fi + if [ "$STREAM_OP" = open ]; then + [ "$REQUEST" -gt "$old_request" ] || { lock_release; exit 1; } + else + [ "$REQUEST" = "$old_request" ] || { lock_release; exit 1; } + fi + if [ "$STREAM_OP" = close ]; then + clear_stream || { lock_release; exit 1; } + lock_release + exit 0 + fi + case "$(fm_busy_record_read "$STATE" "$ID")" in 'busy pi-ext '*) : ;; *) lock_release; exit 1 ;; esac + if [ "$STREAM_OP" = progress ]; then + [ "$phase" = active ] || { lock_release; exit 1; } + snapshot=$(fm_busy_stream_record_read "$STATE/$ID.model-stream") || { lock_release; exit 1; } + read -r stream_gen stream_request old_rev _ <<< "$snapshot" + [ "$stream_gen" = "$GEN" ] && [ "$stream_request" = "$REQUEST" ] || { lock_release; exit 1; } + rev=$((old_rev + 1)) + else + if ! { printf 'v1 gen=%s request=%s phase=active\n' "$GEN" "$REQUEST" > "$STATE/$ID.model-stream-order.tmp.$$" && + mv -f "$STATE/$ID.model-stream-order.tmp.$$" "$STATE/$ID.model-stream-order"; }; then + lock_release; exit 1 + fi + fi + if ! { printf 'v1 gen=%s request=%s rev=%s ts=%s\n' "$GEN" "$REQUEST" "$rev" "$(date +%s)" > "$STATE/$ID.model-stream.tmp.$$" && + mv -f "$STATE/$ID.model-stream.tmp.$$" "$STATE/$ID.model-stream"; }; then + lock_release; exit 1 + fi + if [ "$STREAM_OP" = progress ]; then + if ! { cp "$STATE/$ID.model-stream" "$STATE/$ID.model-progress.tmp.$$" && + mv -f "$STATE/$ID.model-progress.tmp.$$" "$STATE/$ID.model-progress"; }; then + lock_release; exit 1 + fi + fi + lock_release + exit 0 +fi OLD_SEQ=0 if [ -f "$REC" ]; then old_line=$(head -n 1 "$REC" 2>/dev/null || true) @@ -264,6 +344,9 @@ write_record "$GEN" $((OLD_SEQ + 1)) || { echo "error: record write failed for $ID" >&2 exit 1 } +if [ "$NEW_STATE" != busy ]; then + clear_stream || { lock_release; exit 1; } +fi umask "$old_umask" # Best effort: the record already settled, so a failed touch never refuses it. [ "$TURN_ENDED" = 0 ] || touch "$STATE/$ID.turn-ended" 2>/dev/null || true diff --git a/bin/fm-busy-lib.sh b/bin/fm-busy-lib.sh index b24e7b1843b..0125f88f743 100755 --- a/bin/fm-busy-lib.sh +++ b/bin/fm-busy-lib.sh @@ -44,6 +44,10 @@ # cortex-hook Cortex Code per-task plugin hooks (UserPromptSubmit opens; # Stop, which also fires on an Escape interrupt, and # SessionEnd close) +# kiro-hook Kiro V2 agent-config hooks (userPromptSubmit opens; stop +# closes) - kiro's ONLY state source; its per-turn stop hook +# also keeps the turn-ended notification touch +# hermes-hook Hermes native TUI pre_llm_call opens; on_session_end closes # codex-hook, codex-appserver reserved: Codex, gated by # fm_busy_codex_semantic_source # kimi-wire, kimi-hook reserved: standalone Kimi, gated by fm_busy_kimi_verified @@ -100,16 +104,25 @@ # a real busy verdict once any hook has posted, and it defers to whatever # harness-specific trust pre-registration already exists (fm-claude-trust.sh, # GEMINI_CLI_TRUST_WORKSPACE) to stop the dialog from appearing at all. -# Apart from the launch-prompt backstop above, Grok, Rovo, and AGY are the ONLY -# rendered-text busy fallbacks that survive the redesign, because none of their -# structured lifecycles was credited-live-verified +# Apart from the launch-prompt backstop above, Grok, Rovo, and AGY are the only +# harnesses whose rendered text is their SOLE classification, because none of +# their structured lifecycles was credited-live-verified # in the approved audit (Rovo's clean ACP stopReason lives outside the TUI # path firstmate drives, see references/harness/rovo.md; agy 1.2.0 exposes no # hook surface at all, see references/harness/agy.md); each is scoped to -# its own harness= and can never classify another adapter. The delivery -# guards in bin/fm-composer-lib.sh match rendered footers for submit -# acknowledgement and away-mode supervisor injection only; neither is a -# recorded worker state source. +# its own harness= and can never classify another adapter. +# Kiro is different: its kiro-hook record above is its ONLY state source (a +# live-verified per-turn userPromptSubmit/stop pair, references/harness/kiro.md), +# and it has no rendered-tail arm here. Its `Kiro is working` footer is matched +# only by the delivery guards in bin/fm-composer-lib.sh, never as a recorded +# worker state. The one reachable consumer today is the harness-less union entry +# in FM_DELIVERY_BUSY_REGEX_DEFAULT, which the tmux submit core reads for submit +# acknowledgement; FM_DELIVERY_KIRO_BUSY_REGEX_DEFAULT is registered per the +# fleet convention that every verified harness declares its own signature, and +# has no caller today. So a kiro task with no record classifies unknown +# missing, and an abnormal turn end (no StopFailure/SessionEnd equivalent +# exists on kiro V2) leaves the record busy until the next userPromptSubmit +# re-opens it - docs/verification/kiro.md owns that disclosure. # # The muse pull source is semantic, not rendered: it folds muse's own durable # session event log. It has no writer, no arm, and no gen, because @@ -204,6 +217,85 @@ fm_busy_gen_path() { # printf '%s/%s.busy-gen' "$1" "$2" } +# Optional Pi model-stream contract. The busy writer owns every mutation under +# its incarnation lock. model-stream is active evidence; model-stream-order +# retains the latest request and active/closed phase after the marker is cleared. +# order: v1 gen= request= phase= +# stream / model-progress: v1 gen= request= rev= ts= +# Requests strictly increase within a generation. Progress/close must match the +# open request; closed and older requests cannot reopen or refresh evidence. +# model-progress is separate from native progress and is read ONLY by opted-in +# watchers. Missing, malformed, locked, idle, foreign, or stale evidence conveys +# no model-phase observation, leaving the general busy bound applicable. +fm_busy_pi_stream_version_supported() { + [ "${1:-}" = 1.2.0 ] +} + +fm_busy_stream_seconds() { # ; off/invalid prints 0 + local value=${1:-} + case "$value" in ''|*[!0-9]*) printf '0'; return ;; esac + [ "${#value}" -le 9 ] || { printf '0'; return; } + printf '%s' "$((10#$value))" +} + +fm_busy_stream_uint_valid() { + case "${1:-}" in ''|0*|*[!0-9]*) return 1 ;; esac + [ "${#1}" -le 15 ] +} + +fm_busy_stream_order_read() { # + local line version gen request phase extra + line=$(cat "$1/$2.model-stream-order" 2>/dev/null) || return 1 + read -r version gen request phase extra <<< "$line" + [ "$version" = v1 ] && [ -z "$extra" ] || return 1 + gen=${gen#gen=} request=${request#request=} phase=${phase#phase=} + fm_busy_token_valid "$gen" && fm_busy_stream_uint_valid "$request" || return 1 + case "$phase" in active|closed) : ;; *) return 1 ;; esac + [ "$line" = "v1 gen=$gen request=$request phase=$phase" ] || return 1 + printf '%s %s %s' "$gen" "$request" "$phase" +} + +fm_busy_stream_record_read() { # + local line version gen request rev ts extra + line=$(cat "$1" 2>/dev/null) || return 1 + read -r version gen request rev ts extra <<< "$line" + [ "$version" = v1 ] && [ -z "$extra" ] || return 1 + gen=${gen#gen=} request=${request#request=} rev=${rev#rev=} ts=${ts#ts=} + fm_busy_token_valid "$gen" && fm_busy_stream_uint_valid "$request" && + fm_busy_stream_uint_valid "$rev" && fm_busy_stream_uint_valid "$ts" || return 1 + [ "$line" = "v1 gen=$gen request=$request rev=$rev ts=$ts" ] || return 1 + printf '%s %s %s %s' "$gen" "$request" "$rev" "$ts" +} + +fm_busy_stream_read() { # ; gen request rev ts + local state=$1 id=$2 harness=$3 snapshot gen request rev ts order current + case "$harness" in pi|pi-signed) : ;; *) return 1 ;; esac + [ ! -e "$state/$id.busy-state.lock" ] || return 1 + current=$(fm_busy_current_gen "$state" "$id") || return 1 + order=$(fm_busy_stream_order_read "$state" "$id") || return 1 + snapshot=$(fm_busy_stream_record_read "$state/$id.model-stream") || return 1 + read -r gen request rev ts <<< "$snapshot" + [ "$gen" = "$current" ] && [ "$order" = "$gen $request active" ] || return 1 + case "$(fm_busy_record_read "$state" "$id")" in 'busy pi-ext '*) : ;; *) return 1 ;; esac + [ ! -e "$state/$id.busy-state.lock" ] && + [ "$(fm_busy_current_gen "$state" "$id")" = "$current" ] && + [ "$(fm_busy_stream_order_read "$state" "$id")" = "$order" ] && + [ "$(fm_busy_stream_record_read "$state/$id.model-stream")" = "$snapshot" ] || return 1 + printf '%s' "$snapshot" +} + +fm_busy_model_progress_ts() { # + local snapshot gen request rev ts current + case "$3" in pi|pi-signed) : ;; *) return 1 ;; esac + [ ! -e "$1/$2.busy-state.lock" ] || return 1 + current=$(fm_busy_current_gen "$1" "$2") || return 1 + snapshot=$(fm_busy_stream_record_read "$1/$2.model-progress") || return 1 + read -r gen request rev ts <<< "$snapshot" + [ "$gen" = "$current" ] && [ ! -e "$1/$2.busy-state.lock" ] && + [ "$(fm_busy_current_gen "$1" "$2")" = "$current" ] || return 1 + printf '%s' "$ts" +} + # fm_busy_token_valid: conservative token charset shared by gen, source, and # event fields. Anything else is malformed. fm_busy_token_valid() { # @@ -247,8 +339,10 @@ fm_busy_sources_for_harness() { # gemini*) adapter=gemini-hook ;; devin) adapter=devin-hook ;; cortex) adapter=cortex-hook ;; + hermes) adapter=hermes-hook ;; commandcode) adapter=commandcode-mod ;; polytoken) adapter=polytoken-hook ;; + kiro) adapter=kiro-hook ;; pi|pi-signed) adapter=pi-ext ;; omp) adapter=omp-ext ;; kimi*) diff --git a/bin/fm-captain-hold.sh b/bin/fm-captain-hold.sh index 235727028fc..c290a4b3219 100755 --- a/bin/fm-captain-hold.sh +++ b/bin/fm-captain-hold.sh @@ -64,7 +64,13 @@ # tasks-axi --archive-body). It closes a question with `tasks-axi done` - or, # with `--release`, lifts the hold with `tasks-axi unhold` so a captain-gated # WORK item resumes without closing - and restores resolution-first body -# ordering. An exact retry also completes unfinished ordering normalization and +# ordering. `--release` is attended-only: while the away-posture record exists +# (state/.afk-contract, or the daemon flag state/.afk, read as away; a quiet +# record is a present captain) it is refused before anything is written, so an +# unattended supervisor cannot free held work with its own words and then steer +# it; the captain releases the hold in an attended session. A closing answer +# without `--release` is not gated by posture. An exact retry also completes +# unfinished ordering normalization and # is idempotent only when its requested close mode # matches the newest record; a changed decision or a mode mismatch is rejected. # A re-held task may record a new answer on top. On a task already closed outside this script, @@ -83,7 +89,9 @@ # identically no matter which channel the answer arrived on. The key IS the # task id - no identity arithmetic. The optional fourth field selects the close: # empty or `done` completes the task, `release` lifts the hold so held work -# resumes; anything else is skipped. A key that names no task, a task that is +# resumes; anything else is skipped. A `release` line while the away-posture +# record exists is reported as `refused:` under the same attended-only rule as +# `answer --release` and feeds nothing. A key that names no task, a task that is # not held for the captain, or a task already closed is reported as `skipped:` # and feeds nothing. A replayed delivery whose answer digest and requested # close mode both match the newest record is reported `closed:` and is a no-op; @@ -138,7 +146,7 @@ # `--none` is an explicit semantic attestation that the just-reviewed surface # has no unresolved captain call, and is refused while the origin still has an # open keyed status decision. With a non-empty inventory, every listed task is -# verified durable (captain-held, or carrying a recorded resolution), +# verified durable (docs/captain-hold-lifecycle.md owns the inventory checks), # is never the origin itself, and, when `hold --origin` recorded one, was held # for this origin; a hold with no recorded origin is accepted on durability # alone and named in the output, @@ -240,6 +248,9 @@ DATA="${FM_DATA_OVERRIDE:-$FM_HOME/data}" # shellcheck source=bin/fm-parent-channel-lib.sh # shellcheck disable=SC1091 . "$SCRIPT_DIR/fm-parent-channel-lib.sh" +# shellcheck source=bin/fm-afk-contract.sh +# shellcheck disable=SC1091 +. "$SCRIPT_DIR/fm-afk-contract.sh" PARENT_HOLD_PUBLISHED=0 publish_parent_hold() { # @@ -266,6 +277,7 @@ captain_hold_cleanup() { fm_lock_release "$CAPTAIN_CONTROL_LOCK" || true CAPTAIN_CONTROL_LOCK_HELD=0 fi + fm_afk_contract_lock_release || true } trap captain_hold_cleanup EXIT @@ -526,6 +538,10 @@ resolution_block() { # # Durable state of one captain call: an active captain hold (annotations # surviving even when a date gate has expired) or a recorded captain answer. +# Without captain annotations, an open task's leading stamp can signal an +# incomplete re-hold or unfinished release normalization; the old record alone +# cannot vouch for that state. Done tasks retain their recorded resolution as +# evidence, and verify_entry_durable separately checks the stored origin. verify_hold_durable() { # local id=$1 show state hold_kind body archived_status task_show "$id" || { @@ -540,10 +556,14 @@ verify_hold_durable() { # state=$(show_field "$show" state) hold_kind=$(show_field_value "$show" hold_kind) body=$(show_field "$show" body) - if body_has_resolution_record "$body"; then + if [ "$state" != "done" ] && [ "$hold_kind" = captain ]; then return 0 fi - if [ "$state" != "done" ] && [ "$hold_kind" = captain ]; then + if body_has_resolution_record "$body"; then + if [ "$state" != "done" ] \ + && [ -n "$(body_hold_set_timestamp "$(decode_shown_value "$body")")" ]; then + fail "captain-held task $id carries a recorded answer from an earlier call, but its newer hold never completed; re-run the hold or answer it again" + fi return 0 fi fail "captain-held task $id is neither held for the captain nor closed with a recorded captain answer" @@ -835,7 +855,7 @@ task_identity() { printf '%s' "$id" } -write_hold_origin() { # +write_hold_origin() { # local id=$1 body=$2 origin=$3 stamp rest new_body tmp body=$(decode_shown_value "$body") \ || fail "could not decode the existing body for $id" @@ -844,10 +864,7 @@ write_hold_origin() { # || fail "task $id lost its hold-set stamp before its origin was recorded" rest=$(printf '%s\n' "$body" | sed 1d | awk '!/^Captain hold origin: /' \ | awk 'NF || started { started = 1; print }') - new_body=$stamp - if [ -n "$origin" ]; then - new_body=$(printf '%s\nCaptain hold origin: %s' "$stamp" "$origin") - fi + new_body=$(printf '%s\nCaptain hold origin: %s' "$stamp" "$origin") if [ -n "$rest" ]; then new_body=$(printf '%s\n\n%s' "$new_body" "$rest") fi @@ -859,7 +876,25 @@ write_hold_origin() { # fi if ! tasks_axi update "$id" --body-file "$tmp" >/dev/null; then rm -f -- "$tmp" - fail "could not record the hold origin on $id" + printf 'fm-captain-hold: could not record the hold origin on %s\n' "$id" >&2 + return 1 + fi + rm -f -- "$tmp" +} + +# Best-effort restoration removes a refused hold attempt's stamp while +# preserving the prior body; the new origin has not been published yet. +# tasks-axi cannot write an empty body, and a task that began empty carries no +# earlier answer for a stale stamp to mislabel, so that case keeps its body. +restore_hold_body() { # + local id=$1 body tmp + body=$(decode_shown_value "$2") || return 1 + [ -n "$body" ] || return 0 + tmp=$(umask 077; mktemp "${TMPDIR:-/tmp}/fm-captain-hold-restore.XXXXXX") || return 1 + if ! printf '%s\n' "$body" > "$tmp" || ! tasks_axi update "$id" --body-file "$tmp" >/dev/null; then + rm -f -- "$tmp" + printf 'fm-captain-hold: could not restore the body of %s after an incomplete hold\n' "$id" >&2 + return 1 fi rm -f -- "$tmp" } @@ -1049,7 +1084,7 @@ verify_entry_durable() { # ; prints " &2; exit 2; } shift while [ "$#" -gt 0 ]; do @@ -1122,16 +1157,17 @@ command_hold() { # Publish the timestamp before the captain-hold annotation. A concurrent # snapshot may see the harmless stamp by itself, but can never see a newly # held task without the timestamp that defines this hold lifecycle's age. + # Resolve the origin identity before stamping or holding. A lookup failure + # leaves an existing task untouched, but a missing row was already created. + if [ -n "$origin" ]; then + origin=$(task_identity "$origin") || exit $? + fi task_show_or_fail "$id" "task $id disappeared before recording its hold-set stamp" + original_body=$(show_field "$show" body) write_hold_set_stamp "$id" "$(show_field "$show" body)" "$hold_set" "$preserve_hold_set" task_show_or_fail "$id" "task $id disappeared while recording its hold-set stamp" [ -n "$(body_hold_set_timestamp "$(show_field_value "$show" body)")" ] \ || fail "task $id did not retain its hold-set stamp" - if [ -n "$origin" ]; then - origin=$(task_identity "$origin") || exit $? - previous_origin=$(body_hold_origin "$(show_field_value "$show" body)") - write_hold_origin "$id" "$(show_field "$show" body)" "$origin" || exit $? - fi if [ -n "$until" ]; then tasks_axi hold "$id" --reason "$stored_reason" --kind captain --until "$until" >/dev/null \ || hold_status=$? @@ -1140,21 +1176,22 @@ command_hold() { || hold_status=$? fi if [ "$hold_status" -ne 0 ]; then - # A refused re-hold must not associate the previous hold or answer with a - # new origin. Restore the old line verbatim, without resolving it again. - if [ -n "$origin" ]; then - write_hold_origin "$id" "$(show_field "$show" body)" "$previous_origin" || exit $? - fi + # Restore the body without resolving its old origin again. The new origin + # is still unpublished even if this best-effort restoration fails. + restore_hold_body "$id" "$original_body" || true fail "could not hold task $id for the captain" fi task_show "$id" || fail "task $id disappeared while holding it" show=$TASK_SHOW_OUTPUT hold_kind=$(show_field_value "$show" hold_kind) [ "$hold_kind" = captain ] || fail "task $id did not retain its captain hold" - occurrence=$(( $(resolution_record_count "$(show_field "$show" body)") + 1 )) [ -n "$(body_hold_set_timestamp "$(show_field_value "$show" body)")" ] \ || fail "task $id lost its hold-set stamp while being held" + occurrence=$(( $(resolution_record_count "$(show_field "$show" body)") + 1 )) publish_parent_hold "$id" "$occurrence" needs-decision "$reason" + if [ -n "$origin" ]; then + write_hold_origin "$id" "$(show_field "$show" body)" "$origin" || exit $? + fi printf '%s\n' "$id" } @@ -1219,6 +1256,20 @@ apply_pending_retained_artifact() { # esac } +# A captain hold is released only in an attended session. The away daemon flag +# (state/.afk, bin/fm-wake-lib.sh fm_afk_mode) and the posture record +# (state/.afk-contract, bin/fm-afk-contract.sh mode) are both read: either one +# reading away refuses, a quiet record is a present captain, and no record is +# attended. The check runs before any write, so a refusal leaves the hold and +# the body untouched. +RELEASE_AWAY_REASON='the away-posture record exists, and a captain hold is released only by the captain in an attended session; retry --release after the return archives the record' +release_attended() { + if [ -e "$STATE/.afk" ] && [ "$(fm_afk_mode "$STATE")" = away ]; then + return 1 + fi + ! fm_afk_contract_away_present "$STATE" +} + close_answered() { # if [ "$2" = 1 ]; then tasks_axi unhold "$1" >/dev/null @@ -1263,8 +1314,15 @@ command_answer() { shift done validate_slug task-id "$id" + if [ "$release" = 1 ] && ! release_attended; then + fail "task $id stays held: $RELEASE_AWAY_REASON" + fi load_decision "$decision_file" acquire_task_control_lock "$id" + if [ "$release" = 1 ]; then + fm_afk_contract_lock_hold "$STATE" || fail "task $id stays held: the away-posture record is locked" + release_attended || fail "task $id stays held: $RELEASE_AWAY_REASON" + fi require_tasks_axi task_show "$id" || fail "captain-held task $id is absent from this home's configured backlog (data directory $DATA)" show=$TASK_SHOW_OUTPUT @@ -1514,6 +1572,11 @@ command_answers() { continue ;; esac + if [ "$release_flag" = --release ] && ! release_attended; then + printf 'refused: %s (%s)\n' "$key" "$RELEASE_AWAY_REASON" + skipped=$((skipped + 1)) + continue + fi resolve_rc=0 id=$(resolve_entry "$origin" "$key" 2>"$err") || resolve_rc=$? id=${id%% *} @@ -1878,7 +1941,7 @@ reconcile_note() { command_complete() { local origin=${1:-} meta previous='' supplied='' keys='' entry key status_file open has_meta=0 transfer_rc transfers=() resolved - local resolved_how attested_by_prefix='' origin_state unrecorded_origin='' + local resolved_how attested_by_prefix='' origin_state unrecorded_origin='' tmp [ "$#" -ge 2 ] || { usage >&2; exit 2; } validate_slug origin-id "$origin" shift @@ -1933,7 +1996,17 @@ EOF if [ "$has_meta" = 1 ]; then if [ "$(meta_value "$meta" decisions_reviewed)" != 1 ] || [ "$previous" != "$keys" ]; then - printf 'decisions_reviewed=1\ndecision_keys=%s\n' "$keys" >> "$meta" + tmp=$(mktemp "$STATE/.$origin.meta.XXXXXX") \ + || fail "cannot stage the completion record for $origin" + if ! awk -v keys="$keys" ' + /^(decisions_reviewed|decision_keys)=/ { next } + /^pr=/ && !done { print "decisions_reviewed=1"; print "decision_keys=" keys; done = 1 } + { print } + END { if (!done) { print "decisions_reviewed=1"; print "decision_keys=" keys } } + ' "$meta" > "$tmp" || ! mv -f -- "$tmp" "$meta"; then + rm -f -- "$tmp" + fail "cannot record completion for $origin" + fi fi fm_lock_release "$CAPTAIN_META_LOCK" CAPTAIN_META_LOCK_HELD=0 diff --git a/bin/fm-cd-command-policy.mjs b/bin/fm-cd-command-policy.mjs index 41adcc612bf..55b287cba07 100755 --- a/bin/fm-cd-command-policy.mjs +++ b/bin/fm-cd-command-policy.mjs @@ -9,11 +9,20 @@ // scoping to the real primary checkout lives in the bin/fm-cd-pretool-check.sh // transport, not here. See docs/cd-guard.md for the full contract. // +// One target is exempt: a `cd` whose single literal absolute argument resolves +// to exactly the home passed as --home under both path rules: logical (`..` +// collapsed lexically, Bash's default) and physical (symlinks followed component +// by component, as under `set -P`). Either rule leaving the home, or failing to +// resolve, denies. That cd cannot move the shell out of the home, so it is allowed. A relative, expanded, tilde, +// glob, or option-bearing target is never resolved and stays denied, as does +// every pushd and popd. Without --home no target is exempt. +// // The shell tokenizer and command-position analysis are imported from // bin/fm-arm-command-policy.mjs, the sole owner of firstmate's shell // classification, so this guard never duplicates shell lexing. This policy // never evaluates, expands, sources, or runs any byte of the submitted command; -// it inspects lexical command positions only. +// it inspects lexical command positions only, and resolves the one literal +// absolute path it compares against the home. import { Lexer, splitProgram, commandPosition } from "./fm-arm-command-policy.mjs"; import { realpathSync } from "node:fs"; @@ -68,7 +77,32 @@ function hasCommandQueryPrefix(position) { return false; } -function decision(command) { +function resolvedPaths(path) { + try { + return [realpathSync(path), realpathSync.native(path)]; + } catch { + return null; + } +} + +// The exempt shape is exactly `cd `, optionally after +// `--`. A word carrying an expansion, substitution, glob, or tilde has a value +// this policy cannot know without evaluating it, so it is never resolved. +function cdTargetsHome(command, position, wordIndex, home) { + if (!home || command.value !== "cd") return false; + const args = position.words.slice(wordIndex + 1); + if (args[0]?.value === "--" && args[0].literal) args.shift(); + if (args.length !== 1) return false; + const [target] = args; + if (!target.literal || target.subs.length > 0 || target.unquotedExpansion) return false; + if (!target.value.startsWith("/")) return false; + const resolvedTarget = resolvedPaths(target.value); + const resolvedHome = resolvedPaths(home); + if (resolvedTarget === null || resolvedHome === null) return false; + return resolvedTarget.every((path) => path === resolvedHome[1]); +} + +function decision(command, { home = "" } = {}) { const lexed = new Lexer(command).tokenize(); // Fail open on syntax this classifier cannot tokenize. The cd-guard's threat // model is agent mistakes - an accidental bare `cd projects/foo` always @@ -94,13 +128,14 @@ function decision(command) { if (!command) continue; if (!CD_BUILTINS.has(command.value)) continue; if (position.wrappers.some((wrapper) => FORKING_WRAPPERS.has(wrapper))) continue; + if (cdTargetsHome(command, position, wordIndex, home)) continue; return deny("persistent-cd"); } return { decision: "allow" }; } function parseArguments(argv) { - const result = { command: "", commandSet: false }; + const result = { command: "", commandSet: false, home: "" }; for (let i = 0; i < argv.length; i += 1) { const name = argv[i]; if (name === "--command") { @@ -115,6 +150,16 @@ function parseArguments(argv) { result.commandSet = true; continue; } + if (name === "--home") { + if (i + 1 >= argv.length) throw new Error("--home requires a value"); + result.home = argv[i + 1]; + i += 1; + continue; + } + if (name.startsWith("--home=")) { + result.home = name.slice("--home=".length); + continue; + } throw new Error(`unknown argument: ${name}`); } return result; @@ -137,7 +182,7 @@ if (invokedDirectly()) { if (!args.commandSet || !args.command) { process.stdout.write("allow\n"); } else { - const result = decision(args.command); + const result = decision(args.command, { home: args.home }); if (result.decision === "allow") { process.stdout.write("allow\n"); } else { diff --git a/bin/fm-cd-pretool-check.sh b/bin/fm-cd-pretool-check.sh index 26d0c919fd4..4752cc0052c 100755 --- a/bin/fm-cd-pretool-check.sh +++ b/bin/fm-cd-pretool-check.sh @@ -57,6 +57,8 @@ Claude/Codex tool_input.command). Fires only in the real primary firstmate checkout; it is a silent no-op in a crewmate/scout task worktree or any non-firstmate repo. Exits 0 to allow and 2 to deny a persistent top-level cwd change. +A cd whose single literal absolute target resolves to this checkout itself is +allowed, because it cannot move the shell out of the home. The deny reason is written to stderr, with a Grok decision object on stdout unless --claude is supplied. With --cursor, a deny is Cursor's own decision object on stdout and exit 0, @@ -171,7 +173,7 @@ POLICY="$FM_ROOT/bin/fm-cd-command-policy.mjs" command -v node >/dev/null 2>&1 || exit 0 [ -f "$POLICY" ] || exit 0 -POLICY_OUTPUT=$(node "$POLICY" --command "$CMD" 2>/dev/null) || exit 0 +POLICY_OUTPUT=$(node "$POLICY" --home "$FM_ROOT" --command "$CMD" 2>/dev/null) || exit 0 [ -n "$POLICY_OUTPUT" ] || exit 0 TAB=$(printf '\t') diff --git a/bin/fm-classify-lib.sh b/bin/fm-classify-lib.sh index e00b6536149..00318586fa6 100755 --- a/bin/fm-classify-lib.sh +++ b/bin/fm-classify-lib.sh @@ -1407,23 +1407,46 @@ FM_OPEN_DECISIONS_FOLD_VERSION=10 # Portable device:inode identity for the rotation/recreation check below. _fm_open_decisions_file_ident() { # -> strongest available identity - local f=$1 epoch birth ident record + local f=$1 if [ -n "${FM_STATUS_IDENTITY_READER:-}" ]; then "$FM_STATUS_IDENTITY_READER" "$f" return fi - # Capture device, inode, and birth time together: this runs for every cursor - # read and three independent stats both cost forks and can straddle rotation. + _fm_status_stat_fields "$f" || return 1 + printf '%s' "$_FM_STAT_IDENTITY" +} + +# One stat exec sets _FM_STAT_IDENTITY and _FM_STAT_SIZE for , because +# the drain reads identity and size of every status file several times and +# each exec is slow on a loaded host. +_fm_status_stat_fields() { # + local out dev_ino epoch birth rest if [ "$_FM_CLASSIFY_UNAME_S" = Darwin ]; then - record=$(LC_ALL=C /usr/bin/stat -f $'%d:%i\t%B\t%FB' "$f" 2>/dev/null) || return 1 + out=$(LC_ALL=C /usr/bin/stat -f '%d:%i%t%z%t%B%t%FB' "$1" 2>/dev/null) || return 1 else - record=$(LC_ALL=C stat -c $'%d:%i\t%W\t%w' "$f" 2>/dev/null) || return 1 + out=$(LC_ALL=C stat -c $'%d:%i\t%s\t%W\t%w' "$1" 2>/dev/null) || return 1 fi - case "$record" in *$'\n'*) return 1 ;; esac - IFS=$'\t' read -r ident epoch birth <<< "$record" + dev_ino=${out%%$'\t'*}; rest=${out#*$'\t'} + _FM_STAT_SIZE=${rest%%$'\t'*}; rest=${rest#*$'\t'} + epoch=${rest%%$'\t'*}; birth=${rest#*$'\t'} [ "$epoch" != 0 ] || birth='' - case "$ident$birth" in *$'\t'*|*$'\n'*|'') return 1 ;; esac - if [ -n "$birth" ]; then printf 'strong:%s:%s' "$ident" "$birth"; else printf 'weak:%s' "$ident"; fi + case "$dev_ino$birth" in *$'\t'*|*$'\n'*|'') return 1 ;; esac + if [ -n "$birth" ]; then + _FM_STAT_IDENTITY="strong:$dev_ino:$birth" + else + _FM_STAT_IDENTITY="weak:$dev_ino" + fi +} + +# Set _FM_STAT_IDENTITY and _FM_STAT_SIZE for one status file with one stat +# exec. Injected test readers are honored, identity first and size second. +_fm_status_read_ident_size() { # + if [ -n "${FM_STATUS_IDENTITY_READER:-}" ] || [ -n "${FM_STATUS_SIZE_READER:-}" ]; then + _FM_STAT_IDENTITY=$(_fm_open_decisions_file_ident "$1") || return 1 + _FM_STAT_SIZE=$(_fm_status_file_size "$1") || return 1 + return 0 + fi + _fm_status_stat_fields "$1" } _fm_status_file_size() { # @@ -1535,10 +1558,9 @@ status_open_decisions_incremental() { # [] [ # A stat/size-read failure is a genuine I/O error, not "the file is empty" - # report the already-trusted persisted set unchanged rather than risking a # silent invalidation that would wipe it. - cur_ident=$(_fm_open_decisions_file_ident "$f") || { printf '%s' "$trusted_open"; return 0; } + _fm_status_read_ident_size "$f" || { printf '%s' "$trusted_open"; return 0; } + cur_ident=$_FM_STAT_IDENTITY actual_size=$_FM_STAT_SIZE [ -n "$cur_ident" ] || { printf '%s' "$trusted_open"; return 0; } - actual_size=$(_fm_status_file_size "$f") \ - || { printf '%s' "$trusted_open"; return 0; } actual_size=${actual_size//[[:space:]]/} case "$actual_size" in ''|*[!0-9]*) printf '%s' "$trusted_open"; return 0 ;; esac if [ -n "$captured_end" ]; then @@ -1862,9 +1884,8 @@ EOF offset=0 ident=$(_fm_open_decisions_file_ident "$f") || return 1 fi - cur_ident=$(_fm_open_decisions_file_ident "$f") || return 1 - size=$(_fm_status_file_size "$f") || return 1 - size=${size//[[:space:]]/} + _fm_status_read_ident_size "$f" || return 1 + cur_ident=$_FM_STAT_IDENTITY size=${_FM_STAT_SIZE//[[:space:]]/} case "$size:$offset" in *[!0-9:]*) return 1 ;; esac if [ "$ident" != "$cur_ident" ] || [ "$offset" -gt "$size" ]; then offset=0; fi printf '%s' "$offset" @@ -1895,9 +1916,8 @@ status_outcome_backstop_cursor_offset() { # case "$presented:$row_backstop" in *[!0-9:]*) return 1 ;; esac [ -n "$presented" ] && [ -n "$ident" ] || return 1 if [ "$row_task" = "$task" ]; then - current=$(_fm_open_decisions_file_ident "$f") || return 1 - size=$(_fm_status_file_size "$f") || return 1 - size=${size//[[:space:]]/} + _fm_status_read_ident_size "$f" || return 1 + current=$_FM_STAT_IDENTITY size=${_FM_STAT_SIZE//[[:space:]]/} case "$size" in ''|*[!0-9]*) return 1 ;; esac [ "$ident" = "$current" ] || { printf '0'; return 0; } backstop=${row_backstop:-0} @@ -2217,9 +2237,8 @@ status_commit_presentation_snapshot() { # [ -n "$ident" ] || { rm -f "$tmp"; return 1; } f="$state/$task.status" [ -f "$f" ] && [ -r "$f" ] && [ ! -L "$f" ] || { rm -f "$tmp"; return 1; } - cur_ident=$(_fm_open_decisions_file_ident "$f") || { rm -f "$tmp"; return 1; } - size=$(_fm_status_file_size "$f") || { rm -f "$tmp"; return 1; } - size=${size//[[:space:]]/} + _fm_status_read_ident_size "$f" || { rm -f "$tmp"; return 1; } + cur_ident=$_FM_STAT_IDENTITY size=${_FM_STAT_SIZE//[[:space:]]/} case "$size" in ''|*[!0-9]*) rm -f "$tmp"; return 1 ;; esac [ "$cur_ident" = "$ident" ] && [ "$endpoint" -le "$size" ] \ || { rm -f "$tmp"; return 1; } @@ -2327,10 +2346,9 @@ status_open_decisions_cursor_offset() { # return 1 fi fi - cur_ident=$(_fm_open_decisions_file_ident "$f") || return 1 + _fm_status_read_ident_size "$f" || return 1 + cur_ident=$_FM_STAT_IDENTITY size=${_FM_STAT_SIZE//[[:space:]]/} [ -n "$cur_ident" ] || return 1 - size=$(_fm_status_file_size "$f") || return 1 - size=${size//[[:space:]]/} case "$size" in ''|*[!0-9]*) return 1 ;; esac if [ -z "$version" ] || [ -z "$ident" ] || [ "$ident" != "$cur_ident" ] || [ "$offset" -gt "$size" ]; then offset=0 diff --git a/bin/fm-claude-stop-autoarm.sh b/bin/fm-claude-stop-autoarm.sh index 9ae647a3e0e..f90a0695072 100755 --- a/bin/fm-claude-stop-autoarm.sh +++ b/bin/fm-claude-stop-autoarm.sh @@ -93,8 +93,16 @@ # fresh watcher predicate and retried a bounded number of times in this # hook. Only an exhausted failure with no verified watcher emits one # last-resort notice per failure episode; later consecutive failures still -# exit 2 to guarantee the next Stop-owned retry without repeating notice, -# until the synchronous guard has consumed its attended fail-open. +# exit 2 with a one-line suppression reason to guarantee the next +# Stop-owned retry without repeating notice, until the episode's attended +# fail-open is consumed. Two writers own that attended alarm +# (state/.claude-autoarm-failure-alarmed) under the same owner-then-budget +# lock order: the synchronous guard's terminal_fail_open, and this hook's +# own terminal failure - the exhausted continuation is the one participant +# that knows no automatic continuation remains, so once this session's +# consumed block count is over the guard's budget it commits the alarm in +# the same owned critical section as its failed-suppressed write and +# delivers the one loud final rewake; later generations exit 0. # # The epoch ledger state/.claude-autoarm-epoch records the latest claim # generation and outcome, and binds rewake outcomes to the session-lock pid and @@ -144,6 +152,16 @@ CONFIG="${FM_CONFIG_OVERRIDE:-$FM_HOME/config}" OWNER_LOCK="$STATE/.claude-autoarm.lock" FAILURE_NOTICE="$STATE/.claude-autoarm-failure-notified" FAILURE_ALARM="$STATE/.claude-autoarm-failure-alarmed" +# Only this invocation's commit must survive an interruption as a final rewake. +AUTOARM_COMMITTED_ALARM=0 +# The guard's attended-block budget (bin/fm-turnend-guard.sh --claude owns the +# accounting): the auto-arm reads this session's consumed count when its own +# terminal continuation failure is reached, to decide whether it must also +# commit the episode's attended alarm itself. +BUDGET_FILE="$STATE/.turnend-claude-blocks" +BUDGET_LOCK="$STATE/.turnend-claude-blocks.lock" +BLOCK_BUDGET=${FM_CLAUDE_TURNEND_BLOCK_BUDGET:-3} +case "$BLOCK_BUDGET" in ''|*[!0-9]*|0) BLOCK_BUDGET=3 ;; esac AUTOARM_ATTEMPTS=${FM_CLAUDE_AUTOARM_ATTEMPTS:-2} case "$AUTOARM_ATTEMPTS" in 1|2|3) : ;; @@ -290,6 +308,125 @@ autoarm_record() { # fm_autoarm_write_owned "$STATE" "$MY_GEN" "$1" >/dev/null 2>&1 || true } +# Commit this generation's terminal failed-suppressed outcome together with +# the failure episode's attended alarm, when the episode has already burned +# this session's whole attended block budget. The exhausted continuation is +# the one participant that knows no automatic continuation remains, so the +# auto-arm is the second writer of .claude-autoarm-failure-alarmed beside +# bin/fm-turnend-guard.sh --claude's terminal_fail_open. +# +# The decision and the commit share one critical section under the documented +# lock order - owner lock, then budget lock, the same order terminal_fail_open +# uses. Holding the budget lock across the marker write excludes +# fm_failure_episode_reset, so an alarm can never be orphaned into a reset +# episode, and holding the owner lock makes the ledger write and the alarm +# create one atomic section (_fm_autoarm_write_owned_locked). The open-claim +# exclusion that keeps the guard's alarm from firing mid-generation is already +# satisfied: this generation's own claim is the open one, and its terminal +# failure is the proof the continuation is gone. +# +# Returns 0 when the alarm was committed, 2 when a verified healthy watcher +# appeared inside the hold (the caller resets and closes clean instead), and +# 1 on any other refusal or uncertainty. +autoarm_commit_attended_alarm() { + local session_id budget_session budget_count rc=1 i=0 + fm_lock_try_acquire "$OWNER_LOCK" || return 1 + while ! fm_lock_try_acquire "$BUDGET_LOCK"; do + if [ "$i" -ge 20 ]; then + fm_lock_release "$OWNER_LOCK" + return 1 + fi + sleep 0.02 + i=$((i + 1)) + done + if fm_autoarm_ledger_read "$STATE" \ + && [ "$FM_AUTOARM_GEN" = "$MY_GEN" ] \ + && [ "$FM_AUTOARM_OWNER" = "${BASHPID:-$$}" ] \ + && [ -e "$FAILURE_NOTICE" ] \ + && [ ! -e "$FAILURE_ALARM" ] \ + && [ ! -e "$STATE/.afk" ]; then + if fm_watcher_healthy "$STATE" "$SCRIPT_DIR/fm-watch.sh" "$GRACE" "$FM_HOME"; then + rc=2 + else + session_id=$(printf '%s' "$PAYLOAD" | jq -r '.session_id // "unknown"' 2>/dev/null || printf 'unknown') + budget_session=$(sed -n '1s/^session=//p' "$BUDGET_FILE" 2>/dev/null || true) + budget_count=$(sed -n '2s/^count=//p' "$BUDGET_FILE" 2>/dev/null || true) + case "$budget_count" in ''|*[!0-9]*) budget_count=0 ;; esac + if [ "$budget_session" = "$session_id" ] \ + && [ "$budget_count" -gt "$BLOCK_BUDGET" ] \ + && _fm_autoarm_write_owned_locked "$STATE" "$MY_GEN" failed-suppressed "$FAILURE_ALARM"; then + AUTOARM_COMMITTED_ALARM=1 + rc=0 + fi + fi + fi + fm_lock_release "$BUDGET_LOCK" + fm_lock_release "$OWNER_LOCK" + return "$rc" +} + +autoarm_alarm_banner() { + local need_desc + if [ "${FM_SUP_IN_FLIGHT:-0}" -gt 0 ]; then + need_desc="$FM_SUP_IN_FLIGHT task(s) in flight" + elif [ "${FM_SUP_SOURCES:-0}" -gt 0 ]; then + need_desc="$FM_SUP_SOURCES process-event source(s) registered" + elif [ "${FM_SUP_CHECKS:-0}" -gt 0 ]; then + need_desc="$FM_SUP_CHECKS registered custom check(s)" + else + need_desc="X-mode relay polling active" + fi + { + printf 'FIRSTMATE SUPERVISION IS GENUINELY DOWN: %s - the Stop-owned auto-arm exhausted its bounded retries and one failure notice, no live watcher or automatic continuation exists, and this session block budget is exhausted.\n' "$need_desc" + printf 'Keep this session attended and diagnose the automatic Stop-hook and watcher startup before relying on unattended supervision; do not launch a manual background arm from this notice.\n' + } >&2 +} + +# The one terminal exhaustion decision every failed-suppressed continuation +# goes through - the ordinary exhausted failure, a lost supervision-host +# hand-back, an interruption, and a refused healthy-watcher reset alike - so +# none of them can commit another retry once the episode's attended alarm is +# due (bin/fm-turnend-guard.sh --claude steps aside for every open claim, so a +# path that skipped this decision could burn the block budget forever with no +# participant alarming). In order: commit the attended alarm with the one loud +# final rewake; close clean when a verified healthy watcher lets the episode +# reset; otherwise commit failed-suppressed. says why this continuation +# is suppressed, and the appended clause states the outcome the commit actually +# decided: an already-alarmed episode schedules no Stop-owned retry (exit 0), +# anything else keeps it alive (exit 2). Text already on stderr, such as the +# host's actionable wake lines, is delivered with whichever rewake wins. +autoarm_exit_suppressed() { # + local cause=$1 rc + autoarm_commit_attended_alarm + rc=$? + if [ "$rc" -eq 0 ]; then + autoarm_alarm_banner + [ -z "${OUT:-}" ] || rm -f "$OUT" 2>/dev/null || true + exit 2 + fi + if [ "$rc" -eq 2 ]; then + fm_autoarm_reset_owned "$STATE" "$MY_GEN" >/dev/null 2>&1 + rc=$? + [ "$rc" -ne 0 ] || autoarm_record clean + if [ "$rc" -ne 1 ]; then + [ -z "${OUT:-}" ] || rm -f "$OUT" 2>/dev/null || true + exit 0 + fi + cause='firstmate watcher auto-arm could not clear the resolved failure episode - a verified healthy watcher is running, but the episode reset was refused' + fi + if ! autoarm_commit failed-suppressed; then + [ -z "${OUT:-}" ] || rm -f "$OUT" 2>/dev/null || true + exit 0 + fi + [ -z "${OUT:-}" ] || rm -f "$OUT" 2>/dev/null || true + if [ -e "$FAILURE_ALARM" ]; then + printf '%s; the attended failure alarm is already committed for this episode, so this suppressed continuation repeats no operator notice and schedules no Stop-owned retry.\n' "$cause" >&2 + exit 0 + fi + printf '%s; no operator notice repeats in this episode, and this suppressed continuation keeps the Stop-owned retry alive.\n' "$cause" >&2 + exit 2 +} + # Claude terminates the complete async-hook process tree when the configured # hook timeout expires. The arm is intentionally allowed to follow a healthy # watcher until its next wake, so that wait cannot be shortened without adding @@ -308,7 +445,12 @@ handle_autoarm_signal() { wait "$ARM_PID" 2>/dev/null || true fi [ -z "${OUT:-}" ] || rm -f "$OUT" 2>/dev/null || true + if [ "$AUTOARM_COMMITTED_ALARM" -eq 1 ]; then + autoarm_alarm_banner + exit 2 + fi if [ -e "$FAILURE_ALARM" ]; then + printf 'firstmate watcher auto-arm interrupted by %s after the attended alarm - the failure episode is already alarmed, so this suppressed continuation emits no further rewake.\n' "$signal" >&2 autoarm_record failed-suppressed exit 0 fi @@ -318,8 +460,7 @@ handle_autoarm_signal() { autoarm_commit failed "$FAILURE_NOTICE" && exit 2 exit 0 fi - autoarm_commit failed-suppressed && exit 2 - exit 0 + autoarm_exit_suppressed "firstmate watcher auto-arm interrupted by $signal - the failure episode notice was already consumed" } trap 'handle_autoarm_signal HUP' HUP @@ -389,12 +530,13 @@ run_arm() { # # successor receives the closed arm's pid as FM_WATCH_PREDECESSOR_ARM_PID; it # must outlive this hook's exit, so it is detached three ways: nohup, stdio # away from the hook's pipes, and its own process group. Its one status line -# is awaited within the arm's own confirmation budget plus slack. Sets +# is awaited within its own short successor budget plus slack - deliberately +# not the arm's fresh-child backstop, which may run far longer. Sets # SUCCESSOR_FAILURE to the banner line for an unconfirmed successor. SUCCESSOR_FAILURE= start_handling_successor() { # local out pid deadline budget line monitor_was_on=0 - budget=${FM_ARM_CONFIRM_TIMEOUT:-30} + budget=${FM_ARM_SUCCESSOR_CONFIRM_TIMEOUT:-30} case "$budget" in ''|*[!0-9]*) budget=30 ;; esac if ! out=$(mktemp "$STATE/.claude-autoarm-successor.XXXXXX"); then SUCCESSOR_FAILURE='The handling successor did not confirm a live watcher (its output file could not be created); this handling turn runs uncovered until the next turn end re-arms.' @@ -523,18 +665,13 @@ if [ "$HEALTHY" -eq 1 ]; then [ -z "$OUT" ] || rm -f "$OUT" 2>/dev/null || true exit 0 fi - if autoarm_commit failed-suppressed; then - [ -z "$OUT" ] || rm -f "$OUT" 2>/dev/null || true - [ -e "$FAILURE_ALARM" ] && exit 0 - exit 2 - fi - [ -z "$OUT" ] || rm -f "$OUT" 2>/dev/null || true - exit 0 + autoarm_exit_suppressed 'firstmate watcher auto-arm could not clear the resolved failure episode - a verified healthy watcher is running, but the episode reset was refused' fi -# After the synchronous guard has consumed the episode's attended fail-open, +# After either alarm writer has consumed the episode's attended fail-open, # do not create another exit-2 continuation that could defeat it. if [ -e "$FAILURE_ALARM" ]; then + printf 'firstmate watcher auto-arm suppressed - the attended failure alarm is already committed for this episode, so this suppressed continuation emits no further rewake.\n' >&2 autoarm_record failed-suppressed [ -z "$OUT" ] || rm -f "$OUT" 2>/dev/null || true exit 0 @@ -573,14 +710,10 @@ if [ "$ACTIONABLE" -eq 1 ]; then && fm_recovery_marker_snapshot "$STATE/.watcher-down" \ && [[ "$FM_RECOVERY_MARKER_TOKEN" == pending:handling:* || "$FM_RECOVERY_MARKER_TOKEN" == announced:handling:* ]] \ && ! fm_watcher_healthy "$STATE" "$SCRIPT_DIR/fm-watch.sh" "$GRACE" "$FM_HOME"; then - LOST_HANDBACK_COMMITTED=0 - if [ ! -e "$FAILURE_NOTICE" ]; then - printf 'firstmate watcher auto-arm FAILED - the supervision host returned an actionable wake, but its rewake could not be committed.\n' >&2 - autoarm_commit failed "$FAILURE_NOTICE" && LOST_HANDBACK_COMMITTED=1 - else - autoarm_commit failed-suppressed && LOST_HANDBACK_COMMITTED=1 - fi - if [ "$LOST_HANDBACK_COMMITTED" -eq 1 ]; then + [ ! -e "$FAILURE_NOTICE" ] \ + || autoarm_exit_suppressed 'firstmate watcher auto-arm hand-back still uncommitted - the supervision host returned an actionable wake, but its rewake could not be committed, and the failure episode notice was already consumed' + printf 'firstmate watcher auto-arm FAILED - the supervision host returned an actionable wake, but its rewake could not be committed.\n' >&2 + if autoarm_commit failed "$FAILURE_NOTICE"; then [ -z "$OUT" ] || rm -f "$OUT" 2>/dev/null || true exit 2 fi @@ -589,11 +722,9 @@ if [ "$ACTIONABLE" -eq 1 ]; then exit 0 fi -# Notify only once for this continuous failure episode; every later invocation -# still exits 2 so Claude must continue into another Stop-owned retry without -# creating a repeated operator notice or manual-arm loop. The notice marker -# commits in the same owned critical section as the winning failed write, so a -# losing generation can neither consume nor deliver it. +# The notice marker commits in the same owned critical section as the winning +# failed write, so a losing generation can neither consume nor deliver it. +# Later failures use autoarm_exit_suppressed's exhaustion decision below. if [ ! -e "$FAILURE_NOTICE" ]; then if ! fm_autoarm_still_owner "$STATE" "$MY_GEN"; then [ -z "$OUT" ] || rm -f "$OUT" 2>/dev/null || true @@ -612,9 +743,9 @@ if [ ! -e "$FAILURE_NOTICE" ]; then [ -z "$OUT" ] || rm -f "$OUT" 2>/dev/null || true exit 0 fi -if autoarm_commit failed-suppressed; then - [ -z "$OUT" ] || rm -f "$OUT" 2>/dev/null || true - exit 2 -fi -[ -z "$OUT" ] || rm -f "$OUT" 2>/dev/null || true -exit 0 +# Every later failure in the episode goes through the shared exhaustion +# decision, which commits the attended alarm once this session's block budget +# is burned (bin/fm-turnend-guard.sh --claude's terminal_fail_open is the other +# alarm writer; docs/turnend-guard.md names both and the owner-then-budget lock +# order they share). +autoarm_exit_suppressed 'firstmate watcher auto-arm still down - the automatic supervision mechanism remains broken in the recorded failure episode, whose notice was already consumed' diff --git a/bin/fm-claude-trust.sh b/bin/fm-claude-trust.sh index 8fc3fb6b88a..532655efbe4 100755 --- a/bin/fm-claude-trust.sh +++ b/bin/fm-claude-trust.sh @@ -31,10 +31,11 @@ # it ever reads the brief. Registering the trust before launch is the only # control that reaches an interactive pane. The same reasoning covers Claude # Code's separate "Allow external CLAUDE.md file imports?" dialog, which -# `--setting-sources project,local` (firstmate PR 10's minimal worker tool -# surface) stopped suppressing: it renders whenever a loaded CLAUDE.md chain -# reaches outside the project tree - which every crewmate's does, through the -# captain's own `~/.claude/CLAUDE.md` importing `~/.claude/RTK.md` - and it is +# `--setting-sources project,local` (passed only by bin/fm-live-lab.sh; an +# fm-spawn.sh worker loads the user settings too) stopped suppressing: it +# renders whenever a loaded CLAUDE.md chain reaches outside the project +# tree - which every crewmate's does, through the captain's own +# `~/.claude/CLAUDE.md` importing `~/.claude/RTK.md` - and it is # gated the same fail-closed way as trust: cursor on "No, disable", no arrow # navigation from firstmate's steering plane. Only worktree mode reaches this # second dialog's flags: a secondmate home has no separate "project" entry to diff --git a/bin/fm-codex-stop-park.sh b/bin/fm-codex-stop-park.sh new file mode 100755 index 00000000000..a70cfb2d4ac --- /dev/null +++ b/bin/fm-codex-stop-park.sh @@ -0,0 +1,309 @@ +#!/usr/bin/env bash +# Codex 0.163.0 model-free supervision. Native SessionStart registers the +# session; asynchronous Stop owns one foreground arm and queues only a real +# wake through `codex queue --thread`. This works in embedded CLI mode without +# an app-server daemon. No terminal injection or separate event selector. +# +# Usage: fm-codex-stop-park.sh --session-start|--guard|--capable +# fm-codex-stop-park.sh (asynchronous Stop hook) +# Hook modes read the native JSON payload on stdin. --capable is read-only. +# The park is opt-in: only a home with config/codex-stop-park +# (docs/configuration.md owns it) registers or parks. Without it --session-start +# only runs the session start, --guard only runs the turn-end guard, the +# asynchronous Stop hook exits at once, and --capable fails. +# Registration also requires the native transcript's codex-tui originator; +# headless exec sessions have no persistent interactive input surface. +# Homes without the opt-in, unsupported versions, manual session starts, +# workers, and foreign session owners retain the existing foreground +# checkpoint and turn-end guard. +# +# state/.codex-park-session binds selection to the native session and fleet +# lock PID. state/.codex-park.lock is the shared lock library's single-flight +# claim. state/.codex-park.json records the claim's PID identity, session, +# outcome and timestamp; it never acknowledges or alters the wake queue. +# The arm remains the owner of watcher identity, recovery generations and +# successor following. Quiet polls never return to the model. +# +# Async hooks do not themselves wake Codex. A bounded native queue submission +# does; delivery failure latches this session to foreground fallback and stays +# visible to the next synchronous Stop guard. When supervision stops being needed +# with a final result still queued, that result is delivered the same way before +# the park exits. One arm failure is reported and +# retried; a second consecutive failure also latches fallback. A dead park is +# reported when the next Stop replaces its stale claim. Signals terminate only +# the tracked arm child; fm_exec_timed also reaps it after owner death on Perl +# hosts. FM_CODEX_PARK_OWNER_PID and FM_CODEX_PARK_OWNER_IDENTITY bind the +# watcher itself to this hook: its poll loop exits when that identity is gone, +# even if the native harness killed the timeout wrapper too. No other owner +# sets these variables, and no process is signalled by that self-expiry check. +# The native hook timeout is deliberately longer than a process lifetime +# in ordinary use; a hook timeout is a failure, never a quiet polling clock. +set -u + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" +STATE="${FM_STATE_OVERRIDE:-$FM_HOME/state}" +CONFIG="${FM_CONFIG_OVERRIDE:-$FM_HOME/config}" +# shellcheck source=bin/fm-primary-scope-lib.sh +. "$SCRIPT_DIR/fm-primary-scope-lib.sh" +# shellcheck source=bin/fm-session-lock-lib.sh +. "$SCRIPT_DIR/fm-session-lock-lib.sh" +# shellcheck source=bin/fm-supervision-lib.sh +. "$SCRIPT_DIR/fm-supervision-lib.sh" +# shellcheck source=bin/fm-wake-lib.sh +. "$SCRIPT_DIR/fm-wake-lib.sh" +# shellcheck source=bin/fm-timeout-lib.sh +. "$SCRIPT_DIR/fm-timeout-lib.sh" +# shellcheck source=bin/fm-supervision-engine-lib.sh +. "$SCRIPT_DIR/fm-supervision-engine-lib.sh" + +MODE=${1:-stop} +SESSION_FILE="$STATE/.codex-park-session" +CLAIM="$STATE/.codex-park.lock" +RECORD="$STATE/.codex-park.json" +FALLBACK="$STATE/.codex-park-fallback" +DELIVERY_TIMEOUT=20 +SESSION= +CHILD= +OUT= +OWNED=0 +CODEX_COMMAND= + +opted_in() { [ -f "$CONFIG/codex-stop-park" ]; } + +verified_cli() { + local pid executable + command -v jq >/dev/null 2>&1 || return 1 + [ "$("$SCRIPT_DIR/fm-harness.sh" 2>/dev/null)" = codex ] || return 1 + # Version the running native ancestor, not a possibly upgraded PATH shim. + while IFS= read -r pid; do + if [ -L "/proc/$pid/exe" ]; then + executable=$(readlink "/proc/$pid/exe") + else + executable=$(ps -o comm= -p "$pid" 2>/dev/null | sed 's/^[[:space:]]*//') + fi + case "$executable" in */codex) ;; *) continue ;; esac + [ -x "$executable" ] || return 1 + [ "$("$executable" --version 2>/dev/null)" = 'codex-cli 0.163.0' ] || return 1 + CODEX_COMMAND=$executable + return 0 + done < <(fm_harness_ancestry_pids) + return 1 +} + +registered() { + local registered_session registered_pid + [ -f "$SESSION_FILE" ] && [ ! -L "$SESSION_FILE" ] || return 1 + registered_session=$(sed -n '1p' "$SESSION_FILE") + registered_pid=$(sed -n '2p' "$SESSION_FILE") + [ -n "$registered_session" ] && [ "$registered_pid" = "$(cat "$STATE/.lock" 2>/dev/null)" ] || return 1 + [ -z "$SESSION" ] || [ "$SESSION" = "$registered_session" ] || return 1 + [ "$(cat "$FALLBACK" 2>/dev/null)" != "$registered_session" ] +} + +capable() { + opted_in && fm_primary_scope_matches "$FM_ROOT" "$STATE" && verified_cli \ + && fm_session_lock_owned_by_self "$STATE" || return 1 + # Only the native SessionStart wrapper supplies this before its receipt is + # published; the digest has acquired the session lock before rendering. + [ -n "${FM_CODEX_PARK_SESSION:-}" ] || registered +} + +park_alive() { + local pid identity anchor + [ -f "$RECORD" ] || return 1 + pid=$(jq -r '.pid // empty' "$RECORD" 2>/dev/null) + [ "$pid" = "$(cat "$CLAIM/pid" 2>/dev/null)" ] || return 1 + fm_pid_alive "$pid" || return 1 + identity=$(fm_pid_identity "$pid") || return 1 + anchor=$(cat "$STATE/.lock" 2>/dev/null) || return 1 + jq -e --arg session "$SESSION" --arg identity "$identity" --arg anchor "$anchor" \ + '.session == $session and .identity == $identity and .anchor == $anchor' "$RECORD" >/dev/null 2>&1 +} + +record() { + local outcome=$1 temp="$RECORD.tmp.$$" + jq -n --arg session "$SESSION" --arg identity "$IDENTITY" \ + --arg anchor "$ANCHOR" --arg outcome "$outcome" --argjson pid "$$" \ + --argjson at "$(date +%s)" \ + '{session:$session,identity:$identity,anchor:$anchor,pid:$pid,at:$at,outcome:$outcome}' > "$temp" \ + && mv -f "$temp" "$RECORD" +} + +fallback() { printf '%s\n' "$SESSION" > "$FALLBACK"; } + +# The home no longer needs a watcher. A final result queued as the last source +# finished (for example an exhausted-quota alert) still needs its model turn. +settle_idle() { + if [ -s "$STATE/.wake-queue" ]; then + record delivering || exit 1 + if deliver 'Firstmate watcher wake: a final result is queued. Run bin/fm-wake-drain.sh first. Handle every presented wake, open decision and unread status, then run its exact WAKE_ACK_REQUIRED command. Supervision has ended; do not run a foreground checkpoint.'; then + record delivered + exit 0 + fi + record delivery-failed; fallback; exit 1 + fi + record idle + exit 0 +} + +deliver() { + local message=$1 + fm_session_lock_owned_by_self "$STATE" && registered || return 1 + fm_run_timed "$DELIVERY_TIMEOUT" "$CODEX_COMMAND" queue --thread "$SESSION" --message "$message" > "$STATE/.codex-park-delivery.tmp.$$" 2>&1 + local rc=$? + rm -f "$STATE/.codex-park-delivery.tmp.$$" + return "$rc" +} + +# shellcheck disable=SC2329 # EXIT trap owns cleanup. +cleanup() { + if [ -n "$CHILD" ]; then + kill -TERM "$CHILD" 2>/dev/null || true + wait "$CHILD" 2>/dev/null || true + fi + [ -z "$OUT" ] || rm -f "$OUT" + [ "$OWNED" -eq 0 ] || fm_lock_release "$CLAIM" +} + +case "$MODE" in + --capable) capable; exit $? ;; + --session-start|--guard|stop) ;; + -h|--help) + echo 'Usage: fm-codex-stop-park.sh [--session-start|--guard|--capable]' + exit 0 ;; + *) echo "error: unknown argument: $MODE" >&2; exit 2 ;; +esac +PAYLOAD=$(cat) +SESSION=$(printf '%s' "$PAYLOAD" | jq -r '.session_id // empty' 2>/dev/null) +case "$SESSION" in ''|*[!A-Za-z0-9-]*) SESSION= ;; esac + +if [ "$MODE" = --session-start ]; then + transcript=$(printf '%s' "$PAYLOAD" | jq -r '.transcript_path // empty' 2>/dev/null) + if [ -n "$SESSION" ] && opted_in && [ "$(cat "$FALLBACK" 2>/dev/null)" != "$SESSION" ] \ + && fm_primary_root_matches "$FM_ROOT" && verified_cli \ + && [ -f "$transcript" ] \ + && head -n 1 "$transcript" | jq -e --arg session "$SESSION" \ + '.type == "session_meta" and .payload.id == $session and .payload.originator == "codex-tui"' >/dev/null 2>&1; then + export FM_CODEX_PARK_SESSION="$SESSION" + fi + printf '%s' "$PAYLOAD" | "$SCRIPT_DIR/fm-sessionstart-run.sh" + rc=$? + if [ -n "${FM_CODEX_PARK_SESSION:-}" ] && fm_session_lock_owned_by_self "$STATE"; then + printf '%s\n%s\n' "$SESSION" "$(cat "$STATE/.lock")" > "$SESSION_FILE.tmp.$$" \ + && mv -f "$SESSION_FILE.tmp.$$" "$SESSION_FILE" + fi + exit "$rc" +fi + +if [ "$MODE" = --guard ]; then + if [ -n "$SESSION" ] && capable && [ ! -e "$STATE/.afk" ] \ + && fm_supervision_needed "$STATE"; then + # Startup grace for the async hook to publish its claim: 50 x 0.2s, about 10s. + # That is 5x the earlier ~2s grace, under which retained native lab sessions + # recorded no fallback latch; an observation bound, not a measured + # hook-start to claim-publication latency. + for _ in $(seq 50); do + park_alive && exit 0 + # A queued wake owns the immediate handoff to the next model turn. + if jq -e --arg session "$SESSION" --argjson now "$(date +%s)" \ + '.session == $session and .outcome == "delivered" and ($now - .at) < 5' \ + "$RECORD" >/dev/null 2>&1; then exit 0; fi + sleep 0.2 + done + fallback + echo 'Codex watcher park failed to establish a live owner. Use the bounded foreground checkpoint protocol for this session; inspect state/.codex-park.json.' >&2 + printf '%s' "$PAYLOAD" | jq -e '.stop_hook_active == true' >/dev/null 2>&1 || exit 2 + elif [ -n "$SESSION" ] && [ "$(cat "$FALLBACK" 2>/dev/null)" = "$SESSION" ] \ + && [ ! -e "$STATE/.afk" ] && fm_supervision_needed "$STATE"; then + echo 'Codex watcher park is in foreground fallback after a failure; inspect state/.codex-park.json and resume bounded foreground checkpoints.' >&2 + printf '%s' "$PAYLOAD" | jq -e '.stop_hook_active == true' >/dev/null 2>&1 || exit 2 + fi + printf '%s' "$PAYLOAD" | "$SCRIPT_DIR/fm-turnend-guard.sh" + exit $? +fi + +[ -n "$SESSION" ] && capable || exit 0 +[ ! -e "$STATE/.afk" ] && fm_supervision_needed "$STATE" || exit 0 +if ! fm_lock_try_acquire "$CLAIM"; then + # A park delivering this session's wake holds the claim until its EXIT + # cleanup; the woken turn's Stop can arrive before the queue call returns. + # Wait up to DELIVERY_TIMEOUT plus a 5s margin, in 0.1s steps, to rearm. + holder=$FM_LOCK_HELD_PID + jq -e --arg session "$SESSION" --arg pid "$holder" \ + '.session == $session and (.outcome == "delivering" or .outcome == "delivered") + and (.pid | tostring) == $pid' \ + "$RECORD" >/dev/null 2>&1 && fm_pid_alive "$holder" || exit 0 + for _ in $(seq $(((DELIVERY_TIMEOUT + 5) * 10))); do + [ -e "$CLAIM" ] || [ -L "$CLAIM" ] || break + sleep 0.1 + done + fm_lock_try_acquire "$CLAIM" || exit 0 +fi +OWNED=1 +trap cleanup EXIT +trap 'record interrupted; exit 0' HUP INT TERM +ANCHOR=$(cat "$STATE/.lock") +IDENTITY=$(fm_pid_identity "$$") || exit 1 +previous=$(jq -r '.outcome // empty' "$RECORD" 2>/dev/null || true) +previous_session=$(jq -r '.session // empty' "$RECORD" 2>/dev/null || true) +# A SIGKILL cannot run EXIT cleanup. The exclusive claim proves no previous +# park is still using these private captures; retain only the small receipt. +rm -f "$STATE"/.codex-park-output.* "$STATE"/.codex-park-delivery.tmp.* "$RECORD".tmp.* +record arming || exit 1 +if [ "$previous_session" = "$SESSION" ] \ + && { [ "$previous" = waiting ] || [ "$previous" = arming ] || [ "$previous" = delivering ] \ + || [ "$previous" = interrupted ]; }; then + if ! deliver 'Firstmate Codex park recovered a dead or interrupted owner. Drain the durable wake queue and inspect state/.codex-park.json; automatic supervision is restarting.'; then + record delivery-failed; fallback; exit 1 + fi +fi +OUT=$(mktemp "$STATE/.codex-park-output.XXXXXX") || exit 1 +attempt=0 +while [ "$attempt" -lt 2 ]; do + registered && fm_session_lock_owned_by_self "$STATE" || exit 0 + if [ -e "$STATE/.afk" ]; then record idle; exit 0; fi + fm_supervision_status "$STATE" + [ "$FM_SUP_ACTIVE" = true ] || settle_idle + arm=("$SCRIPT_DIR/fm-watch-arm.sh") + if fm_supervision_host_enabled "$CONFIG" codex; then + arm=("$SCRIPT_DIR/fm-supervision-host.sh" park) + fi + # Tracked child, always waited/reaped. The timeout owner also reaps after a + # SIGKILL of this hook; this is not a detached shell background watcher. + (export FM_EXEC_TIMED_OWNER_PID=$$ FM_SUPERVISION_HOST_PRIMARY=codex FM_CODEX_MODEL_FREE_PARK=1 + export FM_CODEX_PARK_OWNER_PID=$$ FM_CODEX_PARK_OWNER_IDENTITY=$IDENTITY + fm_exec_timed 2147483000 5 "${arm[@]}") > "$OUT" 2>&1 & + CHILD=$! + record waiting || exit 1 + while fm_pid_alive "$CHILD"; do + if [ -e "$STATE/.afk" ] || ! fm_session_lock_owned_by_self "$STATE" \ + || ! registered; then + record idle + exit 0 + fi + fm_supervision_status "$STATE" + [ "$FM_SUP_ACTIVE" = true ] || settle_idle + sleep 1 + done + wait "$CHILD"; rc=$? + CHILD= + if [ "$rc" -eq 0 ] && grep -Eq '^(signal:|stale:|check:|heartbeat($|:)|supervision-host:)' "$OUT"; then + wake=$(grep -E '^(signal:|stale:|check:|heartbeat($|:)|supervision-host:)' "$OUT") + record delivering || exit 1 + if deliver "Firstmate watcher wake: $wake +Run bin/fm-wake-drain.sh first. Handle every presented wake, open decision and unread status, then run its exact WAKE_ACK_REQUIRED command. The next Stop re-arms automatically; do not run a foreground checkpoint."; then + record delivered + exit 0 + fi + record delivery-failed; fallback; exit 1 + fi + attempt=$((attempt + 1)) + if ! deliver "Firstmate Codex park failed (arm exit $rc, attempt $attempt/2). Drain first and inspect state/.codex-park.json. Automatic recovery retries once; a repeated failure selects the bounded foreground checkpoint protocol."; then + record delivery-failed; fallback; exit 1 + fi +done +record failed +fallback +exit 1 diff --git a/bin/fm-composer-lib.sh b/bin/fm-composer-lib.sh index db8d17af48a..d92509e52d5 100644 --- a/bin/fm-composer-lib.sh +++ b/bin/fm-composer-lib.sh @@ -271,6 +271,17 @@ fm_composer_normalize_trim_var() { # printf -v "$__fmnt_name" '%s' "$__fmnt_text" } +# fm_composer_kiro_scope: 0 when the caller has named the pane's harness as kiro +# through FM_COMPOSER_HARNESS, the one input that turns on the kiro-only composer +# rules (the near-gray ghost ceiling in fm_composer_strip_ghost and the hinted +# idle row, FM_COMPOSER_HINTED_IDLE_RE_DEFAULT). A caller that knows its target's +# recorded harness sets it for its reads (bin/fm-send.sh, bin/fm-control.sh, the +# watcher's and process-event re-rings); an unset or other value leaves every +# read exactly as it was before kiro existed. +fm_composer_kiro_scope() { + [ "${FM_COMPOSER_HARNESS:-}" = kiro ] +} + # fm_composer_strip_ghost: the ONE fleet-wide ANSI-aware extractor of "real typed # content" from a captured, styled composer row. Reads the styled line on stdin # (from `tmux capture-pane -e`, `herdr pane read --format ansi`, or @@ -291,9 +302,9 @@ fm_composer_normalize_trim_var() { # # fleet reality, where real typed input is bright and only de-emphasised UI # is dark; the SGR-2 signal above stays theme-independent. A 256-colour # foreground (38;5;n) is NOT luminance-tested - it is palette-dependent and -# no fleet harness uses it for ghost text, so it is kept (real text wins: -# under-stripping merely defers, which the max-defer alarm surfaces, while -# over-stripping would inject over real input). +# no fleet harness except kiro (below) uses it for ghost text, so it is kept +# (real text wins: under-stripping merely defers, which the max-defer alarm +# surfaces, while over-stripping would inject over real input). # muse draws its `⟩` prompt glyph in truecolor 38;2;90;160;255, luminance # ~149.9 (verified, muse 0.1.0-R708.1); its spread keeps it from being stripped # at any FM_COMPOSER_GHOST_LUMA_MAX. The bare-glyph fallback below still @@ -306,12 +317,45 @@ fm_composer_normalize_trim_var() { # # ceiling to reach it would over-strip other harnesses, so only a screen that the # backend's identity probe attributes to Command Code is read with a fixed # ceiling of 160 in its place (fm_composer_classify_screen). +# +# KIRO ONLY: when the caller names the pane's harness as kiro +# (FM_COMPOSER_HARNESS=kiro, see fm_composer_kiro_scope), a NEAR-ACHROMATIC run +# gets a higher ceiling, because kiro draws its idle placeholder in a grey above +# the 128 default. Every other harness, and a read with no harness named, keeps +# exactly the rules above: the wider ceiling read Claude's grey slash-command +# input (38;2;112;112;112, which the herdr payload proof reads with +# FM_COMPOSER_GHOST_LUMA_MAX=0) as a placeholder, so it is not a fleet default. +# FM_COMPOSER_GHOST_GRAY_LUMA_MAX (default 180) applies when +# FM_COMPOSER_GHOST_GRAY_SPREAD_MAX (default 12) covers the run's channel +# spread (max channel minus min channel); every other run keeps the 128 default. +# The gray ceiling only ever RAISES the applicable ceiling. A near-gray run takes +# the higher of the two, so an operator who raises FM_COMPOSER_GHOST_LUMA_MAX +# above 180 still gets that wider strip on gray text as the knob documents. +# kiro ghost 38;2;158;158;158 luminance 158.0 spread 0 -> stripped +# grey real text 38;2;206;207;210 luminance 207.0 spread 4 -> kept +# muse real glyph 38;2;90;160;255 luminance 149.9 spread 165 -> kept +# The same scope also luminance-tests a 256-colour foreground, because kiro +# draws the same grey in whichever encoding the terminal advertises: +# 38;2;158;158;158 on a truecolor pane and 38;5;247 - the identical grey, xterm +# level 158 - on a pane with no COLORTERM. Only the 232-255 greyscale ramp +# (level 8 + (n-232)*10) is tested, because it is the one index range whose RGB +# is fixed by definition rather than by a terminal theme. The 6x6x6 cube's +# r==g==b diagonal is arithmetically grey too and stays untested: kiro has not +# been measured drawing ghost text there, and widening the predicate is how the +# palette-dependence problem the 38;5 carve-out exists for gets in. A chromatic +# index is kept untested rather than converted, and indices 0-15 stay untested +# because every terminal theme remaps them. The base 30-37 / 90-97 foregrounds +# still just end a dark run. # The dim/faint and dark-foreground states are tracked together as "de-emphasis"; # codes are processed left to right within a sequence, so "ESC[0;2m" reads as dim. # LC_ALL=C makes awk walk bytes, so multibyte glyphs (e.g. ❯) and de-emphasised # runs alike pass through or drop intact without locale-dependent classes. fm_composer_strip_ghost() { - LC_ALL=C awk -v lumamax="${FM_COMPOSER_GHOST_LUMA_MAX:-128}" ' + local grayscope=0 + fm_composer_kiro_scope && grayscope=1 + LC_ALL=C awk -v lumamax="${FM_COMPOSER_GHOST_LUMA_MAX:-128}" -v grayscope="$grayscope" \ + -v graylumamax="${FM_COMPOSER_GHOST_GRAY_LUMA_MAX:-180}" \ + -v grayspreadmax="${FM_COMPOSER_GHOST_GRAY_SPREAD_MAX:-12}" ' function sgr_code(v, b) { b = v sub(/:.*/, "", b) @@ -328,25 +372,65 @@ fm_composer_strip_ghost() { if (code == "2") return p + 4 return p + 1 } - # muted_dark: 1 when r,g,b is below lumamax AND a muted near-grey (channel - # spread under 96); a saturated colour is emphasis, never ghost text. - function muted_dark(r, g, b, lumamax, hi, lo) { + # ceiling_for: the luminance ceiling that applies to one truecolor run. + # Outside the kiro scope (grayscope 0) that is always lumamax. Inside it, a + # NEAR-ACHROMATIC run (max channel minus min channel within grayspreadmax) + # takes the HIGHER of graylumamax and lumamax, so the gray ceiling only ever + # raises the applicable ceiling and never reduces an operator-set one. + # Anything more saturated keeps lumamax. + function ceiling_for(r, g, b, hi, lo) { + if (!grayscope) return lumamax hi = r; if (g > hi) hi = g; if (b > hi) hi = b lo = r; if (g < lo) lo = g; if (b < lo) lo = b - return ((299*r + 587*g + 114*b) / 1000 < lumamax && hi - lo < 96) ? 1 : 0 + if ((hi - lo) > grayspreadmax) return lumamax + return (graylumamax > lumamax) ? graylumamax : lumamax + } + # muted_dark: 1 when r,g,b is below the ceiling that applies to it AND a + # muted near-grey (channel spread under 96); a saturated colour is emphasis, + # never ghost text. + function muted_dark(r, g, b, hi, lo) { + hi = r; if (g > hi) hi = g; if (b > hi) hi = b + lo = r; if (g < lo) lo = g; if (b < lo) lo = b + return ((299*r + 587*g + 114*b) / 1000 < ceiling_for(r, g, b) && hi - lo < 96) ? 1 : 0 + } + # palette_gray_level: the channel level of a 256-colour index in the 232-255 + # greyscale ramp, and -1 for EVERY other index and for every index outside + # the kiro scope. A ramp entry has r == g == b, so its level IS its luminance. + # + # The ramp alone is deliberate, not an oversight. It is the only index range + # whose RGB is fixed by definition rather than by a theme, so it needs no + # palette guessing. The 6x6x6 cube diagonal is arithmetically grey too, but + # it is NOT tested here: nothing has measured a harness drawing ghost text + # there, and widening this predicate is how the palette-dependence problem + # the 38;5 carve-out warns about gets in. The test asserting an index outside + # 232-255 is untouched exists to hold that line. + function palette_gray_level(n) { + if (!grayscope) return -1 + if (n >= 232 && n <= 255) return 8 + (n - 232) * 10 + return -1 + } + function gray_is_dark(lv) { + return (lv >= 0 && lv < ceiling_for(lv, lv, lv)) ? 1 : 0 } # fg38_is_dark: 1 when the SGR 38 foreground starting at param p is a - # TRUECOLOR (38;2 / 38:2) muted dark colour; 0 otherwise (a 38;5 palette - # colour, a bright or saturated truecolor, or a malformed run). - function fg38_is_dark(a, p, k, lumamax, spec, nf, f) { + # TRUECOLOR (38;2 / 38:2) muted dark colour, or in the kiro scope a PALETTE + # GREY (38;5 / 38:5) below the ceiling that applies to it; 0 otherwise (any + # other palette index, a bright or saturated colour, or a malformed run). + function fg38_is_dark(a, p, k, spec, nf, f) { spec = a[p] if (index(spec, ":") > 0) { # colon form: whole colour in a[p] nf = split(spec, f, ":") + if (f[2] == "5" && nf >= 3) return gray_is_dark(palette_gray_level(f[nf] + 0)) if (f[2] != "2" || nf < 5) return 0 - return muted_dark(f[nf - 2] + 0, f[nf - 1] + 0, f[nf] + 0, lumamax) + return muted_dark(f[nf - 2] + 0, f[nf - 1] + 0, f[nf] + 0) + } + if (p + 1 > k) return 0 + if (a[p + 1] == "5") { + if (p + 2 > k) return 0 + return gray_is_dark(palette_gray_level(a[p + 2] + 0)) } - if (p + 1 > k || a[p + 1] != "2" || p + 4 > k) return 0 - return muted_dark(a[p + 2] + 0, a[p + 3] + 0, a[p + 4] + 0, lumamax) + if (a[p + 1] != "2" || p + 4 > k) return 0 + return muted_dark(a[p + 2] + 0, a[p + 3] + 0, a[p + 4] + 0) } { line = $0; out = ""; dim = 0; darkfg = 0; n = length(line); i = 1 @@ -367,7 +451,7 @@ fm_composer_strip_ghost() { for (p = 1; p <= k; p++) { v = a[p]; code = sgr_code(v) if (code == "38") { - darkfg = fg38_is_dark(a, p, k, lumamax) + darkfg = fg38_is_dark(a, p, k) p = skip_color_payload(a, p, k) } else if (code == "48" || code == "58") { p = skip_color_payload(a, p, k) @@ -411,6 +495,7 @@ fm_composer_strip_ghost() { # devin: "esc twice to interrupt" and its "❭ Guide Devin while it works" working composer; # cortex: "(esc to interrupt)" on its spinner row and "Type to queue message..." in its working composer; # commandcode: "esc to interrupt" beside a bullet-framed elapsed cell. +# kiro: "Kiro is working". # Claude's current spinner has a rotating glyph and word, but every active-turn # line has an ellipsis followed by a parenthesized elapsed duration. Keep this # signature separate from the shared default because that shape is not generic @@ -434,8 +519,14 @@ fm_composer_strip_ghost() { # agy's `esc to cancel` is part of the union for the same reason: an explicit # tmux agy endpoint reaches the submit core with no recorded harness, and its # bare `>` composer verdict is `unknown`, so the busy footer is the only -# turn-started acknowledgement that path can read. -FM_DELIVERY_BUSY_REGEX_DEFAULT='esc (to )?interrupt|Working(\.\.\.|…)|Ctrl\+c:cancel|ctrl\+c to stop|esc[[:space:]]+to[[:space:]]+cancel|esc twice to interrupt|^[[:space:]]*❭ Guide Devin while it works$' +# turn-started acknowledgement that path can read. kiro's `Kiro is working` is +# in the union for two reasons of its own: the footer renders ON the composer +# row, so a mid-turn composer read is `pending` and only a busy read lets +# fm_composer_queued_enter_verdict convert that to `empty`; and the submit core +# takes its pre-typing baseline with no harness, so without the literal an +# already-busy kiro pane reads `idle` and wrongly arms the idle-to-busy +# confirmation. +FM_DELIVERY_BUSY_REGEX_DEFAULT='esc (to )?interrupt|Working(\.\.\.|…)|Ctrl\+c:cancel|ctrl\+c to stop|esc[[:space:]]+to[[:space:]]+cancel|esc twice to interrupt|^[[:space:]]*❭ Guide Devin while it works$|^[[:space:]]*›[[:space:]]+Kiro is working' FM_DELIVERY_CLAUDE_BUSY_REGEX_DEFAULT='esc to interrupt|…[[:space:]]+\([0-9]+[smh]' # Devin 3000.11.1: the working composer and interrupt hint are independent # delivery signals. Neither is used as semantic worker-state evidence. @@ -498,6 +589,18 @@ FM_DELIVERY_AGY_BUSY_REGEX_DEFAULT='esc[[:space:]]+to[[:space:]]+cancel' # It is deliberately not part of the harness-less union above, so a pane with # no recorded harness never reads it as busy. FM_DELIVERY_POLYTOKEN_BUSY_REGEX_DEFAULT='^[[:space:]]*Running for [0-9]+([.][0-9]+)?s' +# kiro (Kiro CLI) renders its composer footer as `› Kiro is working · Type to +# steer · Ctrl+S to queue` while a turn runs, and an idle footer of +# `Trust All Tools active ... /quit to exit` or the `ask a question or describe +# a task` placeholder otherwise (verified live, kiro-cli 2.21.4). The +# harness-named `Kiro is working` literal is matched rather than the bare +# `esc to cancel` token kiro also renders in its tool-call region and shares with +# agy, so echoed worker output cannot fake an acknowledgement. Delivery guard +# only; recorded worker state comes from the kiro-hook record in +# bin/fm-busy-lib.sh, which never consults this footer. The match is anchored to +# the footer row (`›` glyph first), so prose that merely names the phrase, such +# as a grep of these docs in another harness's pane, is never read as busy. +FM_DELIVERY_KIRO_BUSY_REGEX_DEFAULT='^[[:space:]]*›[[:space:]]+Kiro is working' FM_DELIVERY_KIMI_BUSY_REGEX_DEFAULT='^[[:space:]]*(🌑|🌒|🌓|🌔|🌕|🌖|🌗|🌘)[[:space:]]+·[[:space:]]+' fm_busy_lines_match() { # [harness] @@ -510,6 +613,7 @@ fm_busy_lines_match() { # [harness] claude) regex=$FM_DELIVERY_CLAUDE_BUSY_REGEX_DEFAULT ;; devin) regex=$FM_DELIVERY_DEVIN_BUSY_REGEX_DEFAULT ;; cortex) regex=$FM_DELIVERY_CORTEX_BUSY_REGEX_DEFAULT ;; + hermes) regex='^[[:space:]]*─.*·[[:space:]]*[0-9]+([.][0-9]+)?s[[:space:]]*│|^[[:space:]]*([a-z0-9][a-z0-9_-]*[[:space:]]+)?❯[[:space:]]+Ctrl[+]C to interrupt…' ;; commandcode) regex=$FM_DELIVERY_COMMANDCODE_BUSY_REGEX_DEFAULT ;; codex) regex=$FM_DELIVERY_CODEX_BUSY_REGEX_DEFAULT ;; opencode) regex=$FM_DELIVERY_OPENCODE_BUSY_REGEX_DEFAULT ;; @@ -518,6 +622,7 @@ fm_busy_lines_match() { # [harness] grok) regex=$FM_DELIVERY_GROK_BUSY_REGEX_DEFAULT ;; agy) regex=$FM_DELIVERY_AGY_BUSY_REGEX_DEFAULT ;; polytoken) regex=$FM_DELIVERY_POLYTOKEN_BUSY_REGEX_DEFAULT ;; + kiro) regex=$FM_DELIVERY_KIRO_BUSY_REGEX_DEFAULT ;; kimi) regex=$FM_DELIVERY_KIMI_BUSY_REGEX_DEFAULT ;; cursor) regex=$FM_DELIVERY_CURSOR_BUSY_REGEX_DEFAULT ;; '') regex=$FM_DELIVERY_BUSY_REGEX_DEFAULT ;; @@ -562,10 +667,21 @@ FM_COMPOSER_PI_PROMPT_GLYPHS=$(printf '%s\n' '>') # Command Code renders the anchored `Ask your question...` # after its `❯` glyph (verified live, Command Code 1.74.1); its truecolor is too # bright for the default ghost ceiling, so fm_composer_classify_screen reads its -# screen with a raised one. FM_COMPOSER_IDLE_RE overrides for an unverified harness; +# screen with a raised one. kiro's bare `›` composer needs no entry here: see +# FM_COMPOSER_HINTED_IDLE_RE_DEFAULT below. FM_COMPOSER_IDLE_RE overrides for an unverified harness; # matching is case-insensitive. FM_COMPOSER_IDLE_RE_DEFAULT='^Type a message\.\.\.$|^Ask anything(\.\.\.|…)|^Plan, search, build anything$|^Add a follow-up$|^Ask Devin to build features, fix bugs, or work on your code$|^Ask your question\.\.\.$|^Type your message\.\.\.$|^Type to queue message\.\.\.$' +# KIRO ONLY (fm_composer_kiro_scope): idle placeholders that carry a submit hint +# typed input never renders, so the whole anchored row proves an empty composer +# whatever its colour. Every other harness ignores this set. kiro-cli +# 2.21.x draws `ask a question or describe a task ↵` in near-gray, which the +# ghost strip removes; 2.28.0 draws the same row in the default foreground, +# which no stripper can remove. Typed text, even the placeholder words, renders +# with no `↵` (verified live, kiro-cli 2.28.0), so this match reads `empty` even +# on a styled capture where an FM_COMPOSER_IDLE_RE collision reads `pending`. +FM_COMPOSER_HINTED_IDLE_RE_DEFAULT='^ask a question or describe a task ↵$' + # Opencode draws a mode/model footer line INSIDE its left-bar composer # ("Build · GPT-5.5 Fast OpenAI · high"). It is composer furniture, not typed # text, and only the run's LAST row is ever matched against it. @@ -811,6 +927,10 @@ fm_composer_classify_content() { # [idle_re] [idle_case] [ fi fm_composer_normalize_trim_var content [ -n "$content" ] || { printf 'empty'; return 0; } + if [ -n "$idle_re" ] && fm_composer_kiro_scope \ + && fm_composer_idle_matches "$content" "$FM_COMPOSER_HINTED_IDLE_RE_DEFAULT" sensitive; then + printf 'empty'; return 0 + fi fm_composer_idle_matches "$content" "$idle_re" "$idle_case" && idle_collision=1 # Ghost stripping can leave a REMNANT of an idle placeholder rather than # emptying it, because a terminal draws the cell under its cursor in reverse @@ -2095,7 +2215,19 @@ _fm_composer_select_cursorless() { fi if [ "$FM_COMPOSER_SELECTED_KIND" = bare ]; then next=$((FM_COMPOSER_SELECTED_LAST + 1)) - while :; do + # In the kiro scope a hinted idle placeholder is the whole composer, so the + # row below it (kiro's `/copy to clipboard` footer) is never wrapped input. + if fm_composer_kiro_scope; then + trimmed=$(_fm_composer_screen_row "$FM_COMPOSER_SELECTED_FIRST" "$plain") + fm_composer_normalize_trim_var trimmed + if fm_composer_leading_agent_glyph_var glyph "$trimmed"; then + trimmed=${trimmed#*"$glyph"} + fm_composer_normalize_trim_var trimmed + fm_composer_idle_matches "$trimmed" "$FM_COMPOSER_HINTED_IDLE_RE_DEFAULT" sensitive \ + && next=-1 + fi + fi + while [ "$next" -ge 0 ]; do raw=$(_fm_composer_screen_row "$next" "$plain") trimmed=$raw fm_composer_normalize_trim_var trimmed @@ -2309,6 +2441,32 @@ fm_composer_dialog_sink_release() { fi } +# Hermes TUI 0.21.7: profile-prefixed ❯ prompt immediately below the status +# rule; placeholder text is dim, while typed drafts are bright. Identity is +# required, and dialogs, wrapped drafts, non-ready status and unstyled reads +# remain unproven. The hook record, never this row, owns semantic busy state. +_fm_composer_hermes_state() { # + local screen=$1 styled=$2 stripped + [ "$styled" = 1 ] || { printf 'unknown'; return; } + # The busy hint uses xterm orange 178, above the shared dim threshold. + # Strip only the captured native synthetic cursor chip plus that exact hint; + # the same words typed in the normal bright input color remain a draft. + local esc=$'\033' hint + hint="${esc}[38;5;231m${esc}[48;5;178mC${esc}[38;5;178m${esc}[49mtrl+C to interrupt…${esc}[39m" + screen=${screen//"$hint"/} + stripped=$(printf '%s\n' "$screen" | fm_composer_strip_ghost | fm_composer_strip_ansi) + printf '%s\n' "$stripped" | awk ' + { rows[NR]=$0 } + END { + last=NR; while (last>0 && rows[last] ~ /^[[:space:]]*$/) last-- + # Never interpret a palette or arbitrary output as the composer. + if (last<2 || rows[last] !~ /^[[:space:]]*([a-z0-9][a-z0-9_-]*[[:space:]]+)?❯/) { print "unknown"; exit } + if (rows[last-1] !~ /^[[:space:]]*─[[:space:]]+.*│.*│/) { print "unknown"; exit } + text=rows[last]; sub(/^[[:space:]]*([a-z0-9][a-z0-9_-]*[[:space:]]+)?❯[[:space:]]*/, "", text) + if (text ~ /^[[:space:]]*$/) print "empty"; else print "pending" + }' +} + fm_composer_classify_screen() { # [cursor_row] [identity] local caps=$1 screen=$2 cy=${3:-} identity=${4:-} local styled=0 cursor=0 has_identity=0 compact=0 cortex_proven=0 cortex_ask=0 kv plain @@ -2344,6 +2502,10 @@ EOF if [ -n "$cy" ]; then case "$cy" in *[!0-9]*) printf 'unknown'; return 0 ;; esac fi + if [ "$has_identity" = 1 ] && [ "${identity%%$'\t'*}" = hermes ]; then + _fm_composer_hermes_state "$screen" "$styled" + return + fi plain=$(printf '%s\n' "$screen" | fm_composer_strip_ansi) _fm_composer_scan_screen "$plain" "$cy" '' "$cortex_proven" # The lazy Cortex identity pass: only a caller that asked for it diff --git a/bin/fm-contributions.sh b/bin/fm-contributions.sh index e79a7422737..f4aa63d9aea 100755 --- a/bin/fm-contributions.sh +++ b/bin/fm-contributions.sh @@ -25,7 +25,8 @@ # task already has a record for that URL and fails when that record is # retired, so a retirement is never undone by rediscovery. # owned prints the supported PR URLs this home owns as a JSON array of -# {url,tasks,final}, where final means a saved observation is merged or closed. +# {url,tasks,recorded,final}, where recorded names the tasks holding a durable +# record for the URL and final means a saved observation is merged or closed. # It is read-only and never contacts a forge. # Only canonical PR and issue URLs on the exact github.com host are supported; # other accepted URLs remain visibly unmeasured. @@ -282,66 +283,12 @@ jq_lib() { # jq options/program via final argument } READ_SAVED_MAX_BYTES=1048576 -READ_SAVED_BATCH=256 - -# The per-file reference check: size, one parsed valid record, task identity. -read_saved_one() { # file task - if [ "$(wc -c < "$1")" -gt "$READ_SAVED_MAX_BYTES" ] \ - || ! jq_lib -ne --slurpfile record "$1" '($record | length) == 1 and ($record[0] | valid_record)' >/dev/null 2>&1; then - ERRORS=$((ERRORS + 1)) - return 0 - fi - # A file's task identity must match its durable directory, not arbitrary JSON. - if ! jq -e --arg task "$2" '.task == $task' "$1" >/dev/null; then - ERRORS=$((ERRORS + 1)); return 0 - fi - jq -c . "$1" >> "$TMP/saved.jsonl" -} - -# Apply read_saved_one's checks to a batch of files in one jq process. Each -# --slurpfile parses its file independently, exactly as the per-file check does, -# and the -e verdicts are mirrored as "last output is neither false nor null". -# Any failure of the batch itself - a malformed file, a missing file, or an -# evaluation error - replays the whole batch through read_saved_one, so every -# rejection keeps the reference path's exact outcome. -read_saved_batch() { # start count - local i rejected - local -a args=() - for ((i = 0; i < $2; i++)); do - args+=(--slurpfile "f$i" "${SAVED_FILES[$1 + i]}" --arg "t$i" "${SAVED_TASKS[$1 + i]}") - done - if jq_lib -nc --argjson n "$2" "${args[@]}" ' - [range(0; $n) as $i - | if ($ARGS.named | has("f\($i)")) then . else error("batch file missing") end - | $ARGS.named["f\($i)"] as $record | $ARGS.named["t\($i)"] as $task - | ([($record | length) == 1 and ($record[0] | valid_record)] | last) as $valid - | if $valid == null or $valid == false then null - else ([$record[0].task == $task] | last) as $owned - | if $owned == null or $owned == false then null else $record[0] end - end] - | (map(select(. == null)) | length), (.[] | select(. != null))' \ - > "$TMP/batch.jsonl" 2>/dev/null && IFS= read -r rejected < "$TMP/batch.jsonl"; then - case "$rejected" in - ''|*[!0-9]*) ;; - *) - tail -n +2 "$TMP/batch.jsonl" >> "$TMP/saved.jsonl" || fail 'contribution records could not be collected' - ERRORS=$((ERRORS + rejected)) - return 0 - ;; - esac - fi - for ((i = 0; i < $2; i++)); do - read_saved_one "${SAVED_FILES[$1 + i]}" "${SAVED_TASKS[$1 + i]}" - done -} +READ_SAVED_BATCH=100 read_saved() { - local file dir task path start count - local -a candidates=() - : > "$TMP/saved.jsonl" + local file dir task path start n=0 i + local -a candidates=() files=() tasks=() args=() part=() ERRORS=0 - SAVED_FILES=() - SAVED_TASKS=() if [ -L "$DATA" ]; then ERRORS=1; printf '[]\n' > "$TMP/saved.json"; return 0 fi @@ -355,33 +302,48 @@ read_saved() { # A path find could read as an operand never starts with a dash. case "$file" in /*) candidates+=("$file") ;; *) candidates+=("./$file") ;; esac done + [ "${#candidates[@]}" -gt 0 ] || { printf '[]\n' > "$TMP/saved.json"; return 0; } # One find pass, in place of a wc per file, prints every file within the size # bound in operand order; each unprinted file is oversized. If find cannot - # answer for every file, each one takes the reference check instead. - if [ "${#candidates[@]}" -gt 0 ] \ - && find "${candidates[@]}" -prune ! -size +"$READ_SAVED_MAX_BYTES"c -print0 > "$TMP/sized" 2>/dev/null; then - while IFS= read -r -d '' path; do - fm_dirname_to dir "$path" - fm_basename_to task "$dir" - SAVED_FILES+=("$path") - SAVED_TASKS+=("$task") - done < "$TMP/sized" - ERRORS=$((ERRORS + ${#candidates[@]} - ${#SAVED_FILES[@]})) + # answer for every file, each one takes its own size check instead. + if find "${candidates[@]}" -prune ! -size +"$READ_SAVED_MAX_BYTES"c -print0 > "$TMP/sized" 2>/dev/null; then + while IFS= read -r -d '' path; do files+=("$path"); done < "$TMP/sized" else - for path in ${candidates[@]+"${candidates[@]}"}; do - fm_dirname_to dir "$path" - fm_basename_to task "$dir" - read_saved_one "$path" "$task" + for path in "${candidates[@]}"; do + [ "$(wc -c < "$path")" -gt "$READ_SAVED_MAX_BYTES" ] || files+=("$path") done fi - start=0 - while [ "$start" -lt "${#SAVED_FILES[@]}" ]; do - count=$(( ${#SAVED_FILES[@]} - start )) - [ "$count" -le "$READ_SAVED_BATCH" ] || count=$READ_SAVED_BATCH - read_saved_batch "$start" "$count" - start=$((start + count)) + ERRORS=$((ERRORS + ${#candidates[@]} - ${#files[@]})) + for path in ${files[@]+"${files[@]}"}; do + fm_dirname_to dir "$path" + fm_basename_to task "$dir" + tasks+=("$task") + done + [ "${#files[@]}" -gt 0 ] || { printf '[]\n' > "$TMP/saved.json"; return 0; } + # Validate records in bounded jq batches: one process per file outgrows the + # watcher's check timeout, and one process for all files outgrows ARG_MAX. + # Each file is read raw and parsed on its own, so a malformed record is + # rejected inside its batch rather than failing the batch around it. + for ((start = 0; start < ${#files[@]}; start += READ_SAVED_BATCH)); do + part=("${tasks[@]:start:READ_SAVED_BATCH}") + printf '%s\0' "${part[@]}" | jq -Rs 'split("\u0000")[:-1]' > "$TMP/tasks.json" + args=() + for i in "${!part[@]}"; do args+=(--rawfile "rec$i" "${files[start + i]}"); done + jq -L "$SCRIPT_DIR" -n --slurpfile tasks "$TMP/tasks.json" "${args[@]}" 'include "fm-contributions"; + [range(0; $tasks[0] | length) as $i + | [try ($ARGS.named["rec\($i)"] | fromjson) catch empty] as $parsed + | if ($parsed | length) == 1 + and (try ($parsed[0] | valid_record) catch false) + and $parsed[0].task == $tasks[0][$i] + then $parsed[0] else null end] as $checked + | {records: [$checked[] | select(. != null)], errors: ($checked | map(select(. == null)) | length)}' \ + > "$TMP/checked.$n.json" || fail 'contribution records could not be collected' + n=$((n + 1)) done - jq -s . "$TMP/saved.jsonl" > "$TMP/saved.json" + jq -s '{records: (map(.records) | add), errors: (map(.errors) | add)}' "$TMP"/checked.*.json > "$TMP/checked.json" + rm -f "$TMP"/checked.[0-9]*.json + ERRORS=$((ERRORS + $(jq '.errors' "$TMP/checked.json"))) + jq '.records' "$TMP/checked.json" > "$TMP/saved.json" } get_input() { @@ -847,23 +809,25 @@ poll() { [ "$ERRORS" -eq 0 ] || printf 'contributions: %s unreadable durable record(s)\n' "$ERRORS" clear_unsupported_errors # Only supported URLs enter final settlement and the forge-read rotation. - # One line per distinct URL: the URL, then every owning task. + # One line per distinct URL: its settlement, the URL, then every owning task. + # A contribution with a final observation is not re-read for any owner; it + # is settled once every owner holds that final observation without an error. jq_lib -nr --slurpfile input "$TMP/input.json" --slurpfile saved "$TMP/saved.json" ' known($input[0];$saved[0]) | map(select(.url | supported_url)) | group_by(.url) | map({url:.[0].url,tasks:(map(.task) | unique)}) - | .[] | [.url] + .tasks | @tsv' > "$TMP/known.tsv" + | .[] | . as $known + | [$known.tasks[] as $task | [$saved[0][] | select(.task == $task) | .records[] | select(.url == $known.url)] | first + | if . != null and (.observation.state | IN("merged","closed")) then .error == null else null end] as $final + | [(if all($final[]; .) then "settled" elif any($final[]; . != null) then "final" else "live" end), $known.url] + + $known.tasks | @tsv' > "$TMP/known.tsv" : > "$TMP/live.tsv" while IFS=$'\t' read -r -a row; do - [ "${#row[@]}" -ge 2 ] || continue - url=${row[0]} - # A contribution with a final observation is not re-read for any owner. - if jq -ne --slurpfile saved "$TMP/saved.json" --arg url "$url" --args \ - 'any($ARGS.positional[] as $task | [$saved[0][] | select(.task == $task) | .records[] | select(.url == $url)] | first; - . != null and (.observation.state | IN("merged","closed")))' "${row[@]:1}" >/dev/null; then - settle_final "$url" "${row[@]:1}" - else - (IFS=$'\t'; printf '%s\n' "${row[*]}") >> "$TMP/live.tsv" - fi + [ "${#row[@]}" -ge 3 ] || continue + case "${row[0]}" in + settled) ;; + final) settle_final "${row[@]:1}" ;; + *) (IFS=$'\t'; printf '%s\n' "${row[*]:1}") >> "$TMP/live.tsv" ;; + esac done < "$TMP/known.tsv" jq -Rnr --argjson bucket "$((EPOCH / 300))" ' [inputs] | if length == 0 then . else ($bucket % length) as $offset | .[$offset:] + .[:$offset] end @@ -978,7 +942,8 @@ case "${1:-}" in jq_lib -n --slurpfile input "$TMP/input.json" --slurpfile saved "$TMP/saved.json" ' [$saved[0][] | .records[] | select(.error == null and (.observation.state | IN("merged","closed"))) | .url] as $final | known($input[0];$saved[0]) | map(select((.url | supported_url) and (.url | contains("/pull/")))) - | group_by(.url) | map({url:.[0].url,tasks:(map(.task) | unique),final:(.[0].url | IN($final[]))})' + | group_by(.url) | map(.[0].url as $url | {url:$url,tasks:(map(.task) | unique), + recorded:([$saved[0][] | select(any(.records[]; .url == $url)) | .task] | unique),final:($url | IN($final[]))})' ;; enroll) [ "$#" -eq 3 ] || fail 'enroll needs a task and a PR URL' diff --git a/bin/fm-control-lib.sh b/bin/fm-control-lib.sh index 9328eec64c0..7c566925eaa 100644 --- a/bin/fm-control-lib.sh +++ b/bin/fm-control-lib.sh @@ -64,7 +64,7 @@ fm_control_verb_allowed() { # # section 4's verified-adapter list; an unverified adapter is refused rather # than guessed at, exactly as a spawn on it would be. fm_control_harnesses() { - printf '%s\n' claude codex opencode pi pi-signed grok kimi cursor gemini muse rovo omp agy devin commandcode polytoken cortex + printf '%s\n' claude codex opencode pi pi-signed grok kimi cursor gemini muse rovo omp agy devin commandcode polytoken cortex kiro hermes } fm_control_harness_supported() { # @@ -83,7 +83,7 @@ fm_control_harness_supported() { # # `pi-signed` are exact because a `pi*` prefix would swallow the signed adapter, # `omp` is exact because an `omp*` prefix would claim unrelated commands, `agy` # is exact for the same reason on an even shorter name, `devin` and -# `commandcode` are exact as well, and an unrecognized value returns nonzero rather than being guessed into a family. +# `commandcode` are exact as well, `kiro` is exact because a `kiro*` prefix has no verified variant to admit, and an unrecognized value returns nonzero rather than being guessed into a family. fm_control_harness_family() { # case "${1-}" in pi) printf 'pi' ;; @@ -94,6 +94,8 @@ fm_control_harness_family() { # commandcode) printf 'commandcode' ;; polytoken) printf 'polytoken' ;; cortex) printf 'cortex' ;; + hermes) printf 'hermes' ;; + kiro) printf 'kiro' ;; claude*) printf 'claude' ;; codex*) printf 'codex' ;; opencode*) printf 'opencode' ;; @@ -108,7 +110,7 @@ fm_control_harness_family() { # } # Which task kinds an adapter is verified to run. muse, gemini, rovo, devin, -# commandcode, polytoken, and cortex are crewmate/scout adapters only: none has a primary +# commandcode, polytoken, cortex, and kiro are crewmate/scout adapters only: none has a primary # supervision protocol, and bin/fm-spawn.sh refuses a --secondmate launch on any of them. The control # plane asks this BEFORE it stops anything, so an incompatible relaunch target is # refused while the current agent is still running rather than after it has @@ -117,7 +119,8 @@ fm_control_harness_supports_kind() { # local harness=${1-} kind=${2-} fm_control_harness_supported "$harness" || return 1 case "$harness" in - muse|gemini|rovo|devin|commandcode|polytoken|cortex) [ "$kind" != secondmate ] || return 1 ;; + hermes) case "$kind" in ship|scout) ;; *) return 1 ;; esac ;; + muse|gemini|rovo|devin|commandcode|polytoken|cortex|kiro) [ "$kind" != secondmate ] || return 1 ;; esac return 0 } @@ -134,7 +137,7 @@ fm_control_harness_supports_kind() { # fm_control_interrupt_transport() { # case "${1-}" in polytoken) printf 'rest' ;; - claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|muse|rovo|agy|devin|commandcode) printf 'key' ;; + claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|muse|rovo|agy|devin|commandcode|hermes) printf 'key' ;; *) return 1 ;; esac } @@ -148,25 +151,27 @@ fm_control_interrupt_transport() { # # with an idle composer and no repollution (verified live, agy 1.2.0 through # Herdr). omp (Oh My Pi) shares Pi's single Escape, empty composer # afterwards, and /quit exit (verified omp 18.1.2 in a PTY, re-verified 18.1.11 -# through Herdr). cortex cancels on a single Escape, printing the Interrupted row +# through Herdr). kiro cancels on a single Escape, printing a `Cancelled ...` +# row with an idle composer and no repollution (verified live, kiro-cli 2.21.4). +# cortex cancels on a single Escape, printing the Interrupted row # with an idle composer and firing its Stop hook; an Escape pressed on an idle # composer does nothing (verified live, cortex 1.1.104). fm_control_interrupt_key() { # case "${1-}" in - claude|codex|opencode|pi|pi-signed|omp|kimi|cursor|gemini|muse|rovo|agy|devin|commandcode|cortex) printf 'Escape' ;; + claude|codex|opencode|pi|pi-signed|omp|kimi|cursor|gemini|muse|rovo|agy|devin|commandcode|cortex|kiro|hermes) printf 'Escape' ;; grok) printf 'C-c' ;; *) return 1 ;; esac } -# How many times the interrupt key must be delivered. OpenCode and Devin need a double -# Escape; every other verified adapter interrupts on a single press. Command Code +# How many times the interrupt key must be delivered. OpenCode, Devin and Hermes +# need a double Escape; every other verified adapter interrupts on a single press. Command Code # cancels a running turn on one Escape (verified live, Command Code 1.74.1), and a # second press is exactly what opens its rewind picker, so it gets one. fm_control_interrupt_repeat() { # case "${1-}" in - opencode|devin) printf '2' ;; - claude|codex|pi|pi-signed|omp|grok|kimi|cursor|gemini|muse|rovo|agy|commandcode|cortex) printf '1' ;; + opencode|devin|hermes) printf '2' ;; + claude|codex|pi|pi-signed|omp|grok|kimi|cursor|gemini|muse|rovo|agy|commandcode|cortex|kiro) printf '1' ;; *) return 1 ;; esac } @@ -185,7 +190,7 @@ fm_control_interrupt_repeat() { # fm_control_interrupt_arm_signal() { # case "${1-}" in devin) printf '%s' 'esc again to interrupt' ;; - claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|muse|rovo|agy|commandcode|cortex) ;; + claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|muse|rovo|agy|commandcode|cortex|kiro|hermes) ;; *) return 1 ;; esac } @@ -200,7 +205,7 @@ fm_control_interrupt_arm_signal() { # fm_control_interrupt_press_gap() { # case "${1-}" in devin|commandcode) printf '0.5' ;; - claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|muse|rovo|agy|cortex) printf '0.2' ;; + claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|muse|rovo|agy|cortex|kiro|hermes) printf '0.2' ;; *) return 1 ;; esac } @@ -223,7 +228,7 @@ fm_control_interrupt_hazard_signal() { # # a hazard surface on its path; the /quit exit opens only the slash palette, # where Enter accepts the preselected /quit row (verified live). polytoken) ;; - claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|muse|rovo|agy|cortex) ;; + claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|muse|rovo|agy|cortex|kiro|hermes) ;; *) return 1 ;; esac } @@ -238,13 +243,16 @@ fm_control_interrupt_hazard_signal() { # # follow-up` placeholder, so it needs no clear key. gemini was checked the # same way and also does not repollute: after a single Escape it prints # `Request cancelled.` and its composer shows only the `Type your message -# or @path/to/file` placeholder. Prints the key or nothing; +# or @path/to/file` placeholder. kiro was checked the same way and also does not +# repollute: after a single Escape it prints a `Cancelled ...` row and its +# composer shows only the `ask a question or describe a task` placeholder +# (verified, kiro-cli 2.21.4). Prints the key or nothing; # a harness with no verified mechanics returns nonzero, matching the tables # above. fm_control_interrupt_clear_key() { # case "${1-}" in muse) printf 'C-u' ;; - claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|rovo|agy|devin|commandcode|cortex) ;; + claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|rovo|agy|devin|commandcode|cortex|kiro|hermes) ;; *) return 1 ;; esac } @@ -259,7 +267,10 @@ fm_control_interrupt_ack_source() { # # rovo's TUI prints "Agent cancelled" on Escape, but for parity with # claude/cursor this stays 'none': the ack is a rendered string, not a # recorded state source, and rovo has no busy wiring to confirm against. - claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|rovo|agy|devin|commandcode|cortex) printf 'none' ;; + # kiro's stop hook fires on normal completion but, like claude's, not on a + # manual Escape, and its `Cancelled ...` row is likewise rendered rather than + # recorded, so fm-control preserves the adapter-owned busy record instead. + claude|codex|opencode|pi|pi-signed|omp|grok|kimi|cursor|gemini|rovo|agy|devin|commandcode|cortex|kiro|hermes) printf 'none' ;; *) return 1 ;; esac } @@ -280,7 +291,7 @@ fm_control_exit_command() { # [opencode-major] esac ;; claude|grok|kimi|cursor|muse|rovo) printf '/exit' ;; - codex|pi|pi-signed|omp|gemini|agy|devin|commandcode|polytoken|cortex) printf '/quit' ;; + codex|pi|pi-signed|omp|gemini|agy|devin|commandcode|polytoken|cortex|kiro|hermes) printf '/quit' ;; *) return 1 ;; esac } @@ -331,7 +342,7 @@ fm_control_relaunch_resume_flag() { # fm_control_backend_supports_key() { # local backend=${1-} key=${2-} case "$backend" in - tmux|herdr|zellij|cmux|paseo|rex|tern) + tmux|herdr|zellij|cmux|paseo|rex|tern|hc) case "$key" in Escape|Enter|C-c|C-u) return 0 ;; esac ;; orca) @@ -389,7 +400,7 @@ fm_control_backend_relaunch_supported() { # # fm_control_endpoint_absence_verdict before a recovery verb (exit, relaunch) # acts or refuses. `missing` always does. A Rex `unreadable` does too: Rex's # passive reads never start a server, so a stopped server - which ends every -# session on it - reads as unreachable rather than not-found, and only the +# agent process on it - reads as unreachable rather than not-found, and only the # recheck (which may start the personal server) can turn that into a proof. # Passive readers keep the raw state. fm_control_absence_recheck_applies() { # @@ -431,13 +442,15 @@ fm_control_absence_recheck_applies() { # # passes `--session `, so the recheck starts and reads the session # the RECORD names, through that session's own socket. The answer is about # the task's endpoint and nothing else. -# rex CAN prove it. Rex never reuses an id, a session never survives its -# server, and a missing target is a structured not-found (exit 3) distinct +# rex CAN prove it. Rex never reuses an id, and a missing target is a +# structured not-found (exit 3) distinct # from an unreachable server (exit 4). So a not-found from the server the # task was CREATED on proves the block gone. The record names that server # (rex_server, from the task's [meta]); the recheck refuses unless the # server this process reaches - starting a stopped personal server, which # then answers with authority - is that one. +# Server-stop restoration is documented in docs/rex-backend.md; a restored +# block is classified from its current process rather than assumed missing. # t3code has a native absence verdict: `missing` means archived or not found # on the verified server, while an unreachable server reads `unreadable`. # The proof re-reads the thread, but exit and relaunch refuse before it: @@ -547,6 +560,8 @@ fm_control_harness_wiring_paths() { # [ki # the project, and nothing global is installed. gemini) printf '%s\n' "$state/$id.gemini-settings.json" ;; devin) printf '%s\n' "$state/$id.devin-config.json" ;; + # Hermes reaches its private native managed overlay through launch env. + hermes) printf '%s\n' "$state/$id.hermes-hooks/config.yaml" ;; # cortex's lifecycle hooks live in a firstmate-owned plugin directory the # launch reaches through --plugin-dir. Only its two files are retired here # (this table's consumer removes files); teardown removes the directory. @@ -579,6 +594,19 @@ fm_control_harness_wiring_paths() { # [ki done printf '%s\n' "$state/$id.polytoken-busy.sh" ;; + # kiro's busy-state and turn-end hooks live in a firstmate-owned per-task + # agent config the launch reaches through a relocated KIRO_HOME, at + # /.kiro-home/agents/firstmate.json, whose two hook commands are + # generated scripts beside it under hooks/. All three carry the incarnation's + # gen, so all three are retired; the sibling settings/sessions carry no gen + # and are cleared wholesale by teardown's rm -rf of .kiro-home. Nothing + # is written into the worktree, whose own .kiro/ belongs to the project, and + # nothing in the captain's real ~/.kiro is touched. + kiro) + printf '%s\n' "$state/$id.kiro-home/agents/firstmate.json" + printf '%s\n' "$state/$id.kiro-home/hooks/user-prompt-submit" + printf '%s\n' "$state/$id.kiro-home/hooks/stop" + ;; esac } diff --git a/bin/fm-control.sh b/bin/fm-control.sh index 0e0a318d076..806b7aee90b 100755 --- a/bin/fm-control.sh +++ b/bin/fm-control.sh @@ -401,6 +401,10 @@ HARNESS=$(fm_control_harness_family "$RECORDED_HARNESS") \ || die "task $ID records harness '${RECORDED_HARNESS:-none}', which has no verified control mechanics; fm-control refuses to guess an interrupt key or exit command" fm_control_harness_supported "$HARNESS" \ || die "task $ID records harness '${RECORDED_HARNESS:-none}', which has no verified control mechanics; fm-control refuses to guess an interrupt key or exit command" +# The recorded harness scopes the kiro-only composer rules for every composer +# read this control action makes (fm_composer_kiro_scope in bin/fm-composer-lib.sh). +# shellcheck disable=SC2034 +FM_COMPOSER_HARNESS=$HARNESS fm_backend_validate "$BACKEND" || exit "$CONTROL_FAILURE_EXIT" if [ "$BACKEND" = t3code ]; then @@ -497,6 +501,9 @@ dismiss_interrupt_hazard() { # # further press is sent. Its hazard surface is then closed before returning. send_interrupt_keys() { local key repeat clear arm hazard gap i=0 + if [ "$HARNESS" = hermes ] && [ "$(fm_backend_composer_state "$BACKEND" "$T")" != empty ]; then + die 'Hermes double-Escape clears an unsent draft; interrupt requires a proven empty composer' + fi key=$(fm_control_interrupt_key "$HARNESS") repeat=$(fm_control_interrupt_repeat "$HARNESS") clear=$(fm_control_interrupt_clear_key "$HARNESS") @@ -1064,6 +1071,24 @@ resolve_relaunch_profile() { fm_worker_account_codex_check "$seat_home" codex || return 1 RELAUNCH_SEAT_BINDING=$binding fi + if [ "$TARGET_HARNESS" = hermes ]; then + [ "$BACKEND" = tmux ] || die 'Hermes is verified on tmux only; refusing before the current worker stops' + [ "$TARGET_MODEL" = default ] || die 'Hermes models are profile-owned; relaunch requires the default model axis' + # shellcheck source=bin/fm-hermes-lib.sh + . "$SCRIPT_DIR/fm-hermes-lib.sh" + fm_hermes_managed_check || return 1 + local hermes_bin hermes_home hermes_profile + hermes_bin=$(command -v hermes) || die 'hermes executable not found on PATH' + hermes_profile= + if [ "$PRIOR_HARNESS" = hermes ]; then + hermes_home=$(fm_meta_get "$META" hermes_home) + hermes_profile=$(fm_meta_get "$META" hermes_profile) + else + hermes_home=$(fm_hermes_profile_home "$hermes_bin" '') || return 1 + fi + [ -n "$hermes_home" ] || die 'recorded Hermes profile home is missing; refusing an unbound replacement' + fm_hermes_gateway_check "$hermes_home" "$hermes_profile" || return 1 + fi # The launch owner applies this home's worker account pin too, but only after # the old agent has been stopped, so a pin that no longer resolves or is # signed out must refuse here, while nothing has changed yet. diff --git a/bin/fm-credential-source-lib.sh b/bin/fm-credential-source-lib.sh new file mode 100644 index 00000000000..a58f4fe56a0 --- /dev/null +++ b/bin/fm-credential-source-lib.sh @@ -0,0 +1,139 @@ +# shellcheck shell=bash +# Declared credential sources for provider keys. +# Usage: . bin/fm-credential-source-lib.sh +# +# This file is the single owner of the credential-source declaration: a JSON +# object keyed by provider whose values say WHERE that provider's key lives and +# never hold the key itself, for example +# {"typesafe": {"source": "macos-keychain", "service": "typesafe-api-key"}} +# bin/fm-dispatch-resolve.sh reads it only after the environment and .env have +# supplied no key, and bin/fm-bootstrap.sh only checks whether it declares a +# typesafe source; docs/configuration.md "Typed dispatch resolution" owns the +# file's location and the operator contract. +# +# A source is an adapter: FM_CREDSRC_SCHEMA names each adapter and its fields, +# fm_credsrc_supported says whether this host can run it, and fm_credsrc_read +# fetches the key through it. macos-keychain is the first. A further adapter +# adds one schema entry and one arm to those two functions; nothing here knows +# which provider is asking, so the shared surface binds to no one OS or vendor. +# +# Secret containment: a key leaves fm_credsrc_read on stdout for the caller's +# command substitution and nowhere else. It is never an argument, an +# environment variable, a file, or a log line. An adapter passes only the +# fields that identify the item to its tool, and discards the tool's stderr +# because that can echo what it was given. fm_credsrc_supported never runs the +# tool, so asking whether a source is available retrieves no credential. + +# shellcheck source=bin/fm-timeout-lib.sh +. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/fm-timeout-lib.sh" + +# Adapter name -> the fields it takes. A declaration holds "source" plus +# exactly these fields, so no field exists for a credential value to sit in. +FM_CREDSRC_SCHEMA='{"macos-keychain":{"required":["service"],"optional":["account"]}}' + +# Seconds one adapter read may take, so an access prompt nobody answers cannot +# hold the caller. +FM_CREDSRC_TIMEOUT=5 + +# fm_credsrc_supported +# Succeed when names an adapter this host can run. It checks the host +# and that the adapter's tool is on PATH, and runs nothing else. +fm_credsrc_supported() { + case "${1:-}" in + macos-keychain) + [ "$(uname -s 2>/dev/null)" = Darwin ] && command -v security >/dev/null 2>&1 + ;; + *) return 1 ;; + esac +} + +# fm_credsrc_check ... +# Succeed, printing nothing, when is one JSON object whose keys are all +# among the named providers and whose values are well-formed declarations. +# Otherwise print the first problem on one line and fail. A problem names the +# provider, the field, or the allowed values; it never prints a declared value. +fm_credsrc_check() { + local file=$1 providers problem + shift + providers=$(printf '%s\n' "$@" | jq -Rsc 'split("\n") | map(select(length > 0))') || return 1 + problem=$(jq -rs --argjson schema "$FM_CREDSRC_SCHEMA" --argjson providers "$providers" ' + def flat: explode | map(if . < 32 or . == 127 then 63 else . end) | implode; + def plain: type == "string" and length > 0 and (explode | all(. >= 32 and . != 127)); + def entry($name; $d): + if ($d | type) != "object" then "\($name) must be an object with a source" + elif ($d | has("source") | not) then "\($name) needs a source" + elif ($d.source | type) != "string" or ($schema | has($d.source) | not) then + "\($name) source must be one of \($schema | keys | join(", "))" + else + $schema[$d.source] as $f + | (["source"] + $f.required + $f.optional) as $allowed + | ([$d | keys_unsorted[] | select(. as $k | $allowed | index($k) | not)]) as $extra + | ([$f.required[] | . as $k | select(($d[$k] | plain) | not)]) as $bad_required + | ([$f.optional[] | . as $k | select($d | has($k)) | select(($d[$k] | plain) | not)]) as $bad_optional + | if ($extra | length) > 0 then + "\($name) has unknown field \($extra[0] | flat) (allowed: \($allowed | join(", "))); credential values do not belong in this file" + elif ($bad_required | length) > 0 then + "\($name) \($bad_required[0]) must be a non-empty string without control characters" + elif ($bad_optional | length) > 0 then + "\($name) \($bad_optional[0]) must be a non-empty string without control characters when present" + else empty end + end; + if length != 1 then "must hold exactly one JSON object" + elif (.[0] | type) != "object" then "top-level value must be an object keyed by provider" + else + .[0] as $doc + | ([$doc | keys_unsorted[] | select(. as $k | $providers | index($k) | not)]) as $unknown + | if ($unknown | length) > 0 then + "unknown provider \($unknown[0] | flat) (expected \($providers | join(" or ")))" + else first(($doc | to_entries[] | entry(.key; .value)), empty) + end + end' "$file" 2>/dev/null) || problem='not JSON' + if [ -z "$problem" ]; then + return 0 + fi + printf '%s\n' "$problem" + return 1 +} + +# fm_credsrc_declares +# Succeed when declares a source for . +fm_credsrc_declares() { + jq -e --arg p "$2" 'has($p)' "$1" >/dev/null 2>&1 +} + +# fm_credsrc_field +# Print one declared field of 's source, or nothing when it is absent. +# Call only on a file fm_credsrc_check accepted, which keeps every value on one +# printable line. +fm_credsrc_field() { + jq -r --arg p "$2" --arg f "$3" '.[$p][$f] // empty' "$1" +} + +# fm_credsrc_read +# Print the key 's declared source yields. Status 0 means a non-empty +# read; fm_timed_out accepts the status when the source did not answer within +# its bound; any other status means the source holds no readable key. Call only +# on a file fm_credsrc_check accepted and fm_credsrc_supported agreed to run. +fm_credsrc_read() { + local src service account + src=$(fm_credsrc_field "$1" "$2" source) + case "$src" in + macos-keychain) + service=$(fm_credsrc_field "$1" "$2" service) + account=$(fm_credsrc_field "$1" "$2" account) + fm_credsrc_macos_keychain "$service" "$account" + ;; + *) return 1 ;; + esac +} + +# macos-keychain: one generic-password read, bounded by FM_CREDSRC_TIMEOUT. +# Only the service and, when declared, the account reach argv; the key comes +# back on stdout. +fm_credsrc_macos_keychain() { # [] + if [ -n "${2:-}" ]; then + fm_run_timed "$FM_CREDSRC_TIMEOUT" security find-generic-password -s "$1" -a "$2" -w 2>/dev/null /dev/null · source: · # -# Logic, in order: +# A run-step reading whose run came from inventory selection ends its detail +# with a `run: ` field naming that run, and a terminal one appends +# `pr=` after it. The field is always the last clause of the detail so the +# separator never splits a sentence. The supersession verdict labels the same id +# `earlier run: `, and a verdict taken from the ledger's own row carries no +# run id at all, because there the id belongs to a run the verdict did not come +# from (rule 2b). +# +# Logic, numbered for reference; rules 2b and 3 each record that rule 3's +# status-log reconciliation is applied before rule 2b: # 1. Resolve worktree + backend target + kind from state/.meta. A meta # recording remote_host= is a remote secondmate: its worktree and endpoint # live on that host, so the local worktree and pane reads are skipped and @@ -127,6 +136,55 @@ # FAILED record whose daemon an explicit probe proves down reads unknown, # never failed: an instrument failure must not read as work failure # (nm_daemon_probe_down). +# 2b. A TERMINAL reading - failed OR done - is authoritative only +# while nothing newer contradicts it, because head identity binds whatever +# run last sat on this head, not necessarily this branch's current one - +# and a stale terminal reading here is promoted into a captain-facing +# terminal outcome by bin/fm-inactive-reconcile.sh. A false SUCCESS is the +# worse of the two mistakes, because nobody goes looking, so the currency +# test is the same in both directions: a later run on this branch that +# cannot be bound here is reported as unknown - history, but no proof of +# the present. A FAILED reading is additionally answered from a newer +# record where one states a terminal outcome for this worktree: a later +# COMPLETED run the ledger proves is current, or the ledger's own FAILED +# verdict; a later CANCELLED row carries no verdict, so it leaves the +# failure unprovable (unknown) like any other unbindable row. A DONE +# reading claims no such verdict, because unknown is already the whole +# answer a contradicted success may publish. +# Because the ledger has no run ID, +# an equal head does not identify a rerun; a terminal row supplies newer +# truth when its observable status, head, or PR differs. Nothing else does, +# so a real failure is never hidden. The whole rule needs two records to +# compare, so it applies only to a run attributed from `axi status`: a +# coarse reading IS the ledger's own answer for this worktree, and a record +# cannot supersede itself. +# A self-declared pause or done in the status log is NOT one of those +# records. The reader once ordered such a declaration against the ledger's +# date column, let a later one supersede the failure, and published the +# ordering it believed it had proven as a detail field. Those rules were +# removed after repeated defects: the date column stamps a run's START, so +# it can never prove a declaration newer than the moment the run finished, +# and every narrowing of the rule left another shape in which a real +# failure was hidden or a crew's mid-run word was minted as a terminal +# verdict. The reader now claims no ordering between the log and the run: +# where a later ledger row leaves the current state unprovable it answers +# unknown, and otherwise the failed reading stands. +# Independently of outcome, a terminal reading publishes a PR only when the +# current-run evidence still attributes that reading, or when the ledger +# offers no row for this branch at all: an absent row is absence of +# contradicting evidence, not disproof, and one record cannot be +# authoritative for the verdict while being untrusted for the link that +# makes it actionable (the long-inactive crews bin/fm-inactive-reconcile.sh +# scans routinely have runs outside the listing window). A row that EXISTS +# and cannot be bound still publishes no PR. +# Every answer here is reached only AFTER rule 3 below has reconciled the +# status log, so rule 3's daemon-socket-down override outranks all of them, +# not merely a plainly attributed run record: an instrument failure must not +# read as work failure. A verdict taken from the ledger row also names that +# row's status and PR together and names the attributed run only as the +# reading that row could not be bound to, so a single line never labels one +# run's outcome with another run's identity, and never asserts a second, +# earlier run the unbindable row does not prove. # 3. Reconcile the status log through fm-classify-lib.sh's status_current_line: # open decisions survive unrelated events and continuation prose cannot # hide a declaration. Ship/scout terminal declarations supersede stale log @@ -136,7 +194,9 @@ # agree, and are reported as parked. A `blocked:` line that reports a # refused or missing daemon socket remains blocked even if an attributed # run record is stale or terminal, for as long as that blocker is still the -# log's latest event. The same holds for any open decision when the run +# log's latest event; this reconciliation runs BEFORE rule 2b, so that +# blocker also outranks rule 2b's supersession and unknown answers. The +# same holds for any open decision when the run # record itself is UNVERIFIED (its daemon answered down): the crew saw its # gate or blocker first hand, so needs-decision stays parked and blocked # stays blocked, with the unverified record named as the reason. @@ -211,6 +271,14 @@ emit() { # [detail] exit 0 } +emit_run() { # [pr] + local detail=$2 + case "$1" in + done|failed) [ -z "${3:-}" ] || detail="$detail${SEP}pr=$3" ;; + esac + emit "$1" run-step "$detail" +} + # --- meta resolution -------------------------------------------------------- [ -f "$META" ] || emit unknown none "no metadata for $ID" @@ -961,6 +1029,18 @@ nm_ci_checks_state() { nm_runs_list() { nm_run runs --limit "$FM_CREW_STATE_RUNS_LIMIT" } +# The ledger is fetched at most once per read into RUNS_LIST: the coarse +# attribution fallback and the terminal-failed supersession cross-check below +# both need it, and neither is worth a second bounded CLI call. Assigns rather +# than prints, so the cache survives (a command substitution would fetch into a +# subshell every time). +RUNS_LIST="" +RUNS_LIST_FETCHED=0 +runs_list_once() { + [ "$RUNS_LIST_FETCHED" = 1 ] && return 0 + RUNS_LIST=$(nm_runs_list) + RUNS_LIST_FETCHED=1 +} # CREW_BRANCH is empty at detached HEAD (a just-spawned crew, or a scout's # scratch worktree); with no branch there is no run to attribute to this crew. @@ -1128,7 +1208,8 @@ if [ "$KIND" = ship ] && [ -n "$CREW_BRANCH" ] && command -v no-mistakes >/dev/n # Without run ids, contradictory liveness cannot prove precedence. # A live replacement also needs an id-addressed status read: a bare # "running" row cannot tell working from waiting at a gate. - ledger_status=$(fm_nm_runs_status_for_worktree "$WT" "$CREW_BRANCH" "$(nm_runs_list)") + runs_list_once + ledger_status=$(fm_nm_runs_status_for_worktree "$WT" "$CREW_BRANCH" "$RUNS_LIST") if fm_nm_run_is_active "$RUN_OUT"; then if [ "$(fm_nm_run_status_class "$ledger_status")" = terminal ]; then emit unknown run-step "run records disagree; run ids: $(strip_quotes "$(nm_field id)"), competing identity unavailable" @@ -1150,7 +1231,8 @@ if [ "$KIND" = ship ] && [ -n "$CREW_BRANCH" ] && command -v no-mistakes >/dev/n # `[ -n "$RUN_OUT" ]`: an empty/timed-out primary call means the CLI # itself did not respond, so retrying it immediately with a second # bounded call would just double the wait for no better answer. - COARSE_STATUS=$(fm_nm_runs_status_for_worktree "$WT" "$CREW_BRANCH" "$(nm_runs_list)") + runs_list_once + COARSE_STATUS=$(fm_nm_runs_status_for_worktree "$WT" "$CREW_BRANCH" "$RUNS_LIST") if [ -n "$COARSE_STATUS" ]; then HAVE_RUN=1 # A branch-matching answer the strict rule rejected is this branch's @@ -1170,6 +1252,7 @@ fi if [ "$HAVE_RUN" = 1 ]; then RUN_STATE=working RUN_DETAIL="" + RUN_PR="" CI_STEP_STATUS="" CI_LOG_STATE="" RUN_STATUS="" @@ -1204,7 +1287,17 @@ if [ "$HAVE_RUN" = 1 ]; then else strip_quotes "$(nm_field status)" status RUN_STATUS=$status + RUN_ID=$(strip_quotes "$(nm_field id)") + RUN_HEAD=$(strip_quotes "$(nm_field head)") + # The pull request THIS run opened, if it reached its pr step at all. A run + # that never opened one reports no pr field, and that absence is meaningful: + # it is what keeps a terminal outcome from being lent an older task's PR. + RUN_PR=$(strip_quotes "$(nm_field pr)") strip_quotes "$(nm_field outcome)" outcome + case "$status" in + ci|running|fixing|awaiting_approval|fix_review) ATTRIBUTED_LEDGER_STATUS=running ;; + *) ATTRIBUTED_LEDGER_STATUS=$status ;; + esac awaiting=$(printf '%s\n' "$RUN_OUT" | grep -E "$FM_NM_AWAITING_AGENT_RE" | head -1 || true) gate_status=$(nm_gate_status) has_gate=0 @@ -1300,6 +1393,38 @@ if [ "$HAVE_RUN" = 1 ]; then fi fi + TERMINAL_PR="" + case "$RUN_STATE" in + done|failed) + runs_list_once + NEWEST_ROW=$(fm_nm_runs_newest_for_branch "$CREW_BRANCH" "$RUNS_LIST") + NEWEST_STATUS=${NEWEST_ROW%%|*} + NEWEST_REST=${NEWEST_ROW#*|} + NEWEST_SHA=${NEWEST_REST%%|*} + NEWEST_PR=${NEWEST_REST#*|} + NEWEST_ROW_AGREES=0 + LEDGER_STATUS=$(fm_nm_runs_status_for_worktree "$WT" "$CREW_BRANCH" "$RUNS_LIST") + if [ "$RUN_SOURCE" = full ] && [ -n "$RUN_ID" ] \ + && [ "$(strip_quotes "$(nm_field branch)")" = "$CREW_BRANCH" ] \ + && [ "$NEWEST_STATUS" = "$ATTRIBUTED_LEDGER_STATUS" ] \ + && [ "$NEWEST_PR" = "$RUN_PR" ]; then + if [ -n "$NEWEST_SHA" ] && [ -n "$RUN_HEAD" ]; then + case "$RUN_HEAD" in + "$NEWEST_SHA"*) NEWEST_ROW_AGREES=1 ;; + *) case "$NEWEST_SHA" in "$RUN_HEAD"*) NEWEST_ROW_AGREES=1 ;; esac ;; + esac + fi + fi + if [ "$RUN_SOURCE" = coarse ]; then + case "$LEDGER_STATUS" in + completed|failed|cancelled) TERMINAL_PR=$NEWEST_PR ;; + esac + elif [ "$NEWEST_ROW_AGREES" = 1 ] || [ -z "$NEWEST_STATUS" ]; then + TERMINAL_PR=$RUN_PR + fi + ;; + esac + # Reconcile the status log. A needs-decision/blocked log line that the run-step # has moved past (anything but a genuinely parked run) is deterministically # stale: the gate resolved and the run resumed or finished. @@ -1348,8 +1473,42 @@ if [ "$HAVE_RUN" = 1 ]; then ;; esac - [ -z "$SELECTED_RUN_ID" ] || RUN_DETAIL="$RUN_DETAIL${SEP}run: $SELECTED_RUN_ID" - emit "$RUN_STATE" run-step "$RUN_DETAIL" + READING_DETAIL=$RUN_DETAIL + RUN_ID_FIELD="" + [ -z "$SELECTED_RUN_ID" ] || RUN_ID_FIELD="${SEP}run: $SELECTED_RUN_ID" + RUN_DETAIL="$RUN_DETAIL$RUN_ID_FIELD" + + # Apply the terminal-reading precedence contract owned by header rule 2b. + if { [ "$RUN_STATE" = "failed" ] || [ "$RUN_STATE" = "done" ]; } \ + && [ "$RUN_SOURCE" != coarse ]; then + if [ "$RUN_STATE" = "failed" ]; then + case "$LEDGER_STATUS" in + completed) + SUPERSEDED_DETAIL="run superseded by a newer completed run on this branch (earlier $READING_DETAIL)" + [ -z "$SELECTED_RUN_ID" ] \ + || SUPERSEDED_DETAIL="$SUPERSEDED_DETAIL${SEP}earlier run: $SELECTED_RUN_ID" + TERMINAL_PR=$NEWEST_PR + emit_run "done" "$SUPERSEDED_DETAIL" "$TERMINAL_PR" + ;; + failed) + if [ "$NEWEST_ROW_AGREES" != 1 ]; then + LEDGER_DETAIL="run $LEDGER_STATUS from the ledger's row for this worktree; the $READING_DETAIL reading could not be bound to it" + TERMINAL_PR=$NEWEST_PR + emit_run failed "$LEDGER_DETAIL" "$TERMINAL_PR" + fi + ;; + esac + fi + case "$NEWEST_STATUS" in + '') ;; + *) + [ "$NEWEST_ROW_AGREES" = 1 ] || emit unknown run-step \ + "the newest $NEWEST_STATUS ledger row on this branch cannot be bound to the $READING_DETAIL; current state not provable here$RUN_ID_FIELD" + ;; + esac + fi + + emit_run "$RUN_STATE" "$RUN_DETAIL" "$TERMINAL_PR" fi # --- fallback: no run attributed to this crew ------------------------------ diff --git a/bin/fm-dispatch-resolve.sh b/bin/fm-dispatch-resolve.sh index 336b6e91a28..29ac60a40dd 100755 --- a/bin/fm-dispatch-resolve.sh +++ b/bin/fm-dispatch-resolve.sh @@ -4,6 +4,7 @@ # # Usage: # fm-dispatch-resolve.sh [--project ] +# fm-dispatch-resolve.sh --supports-credential-source # # Opt-in gate: FM_DISPATCH_PROVIDER and FM_DISPATCH_MODEL select the provider # and model from the environment or $FM_HOME/.env (environment wins). With no @@ -13,9 +14,33 @@ # (http:// or https://) and posts to /v1/systemone; its optional # FM_DISPATCH_API_KEY, when absent, means no Authorization header. The openai # provider maps the System One request to OpenAI's Decisions API and maps its -# answer back before validation. The selected key lives in a non-exported -# shell variable and reaches curl through fd 3, never argv, logs, or child -# environments. +# answer back before validation. The typesafe and openrouter keys come from the +# environment, else $FM_HOME/.env, else the credential source declared for that +# provider (below); the openai and systemone keys come from the environment or +# $FM_HOME/.env only. The selected key lives in a non-exported shell variable +# and reaches curl through fd 3, never argv, logs, or child environments. +# +# Credential source declaration: the optional $CONFIG/dispatch-credential-sources.json +# ($CONFIG is FM_CONFIG_OVERRIDE, else $FM_HOME/config) is a JSON object keyed +# by provider (typesafe or openrouter). Each value says where that provider's +# key lives and holds no key: {"source": "macos-keychain", "service": "", +# "account": ""}, where account is optional and no other field is +# accepted. bin/fm-credential-source-lib.sh owns the schema and its adapters; +# macos-keychain reads the generic-password item with `security +# find-generic-password -s [-a ] -w`, so only the service +# and account reach argv. The file is read only when neither the environment +# nor .env supplied a key, and only the selected provider's entry is used. +# With no entry for that provider and no key the tool is off as above. An +# unreadable or invalid file exits 2 naming the problem, never a value. A +# declared source this host cannot run, or that yields no key within +# 5 seconds, is an error outcome ending "nothing sent", with no network or +# quota call, so it is not mistaken for an unconfigured resolver. +# +# Capability probe: --supports-credential-source prints "supported" +# and exits 0, or prints "unsupported" and exits 1, for a source name such as +# macos-keychain on this host. It reads no declaration, retrieves no +# credential, and runs no credential tool, so a configuration installer can +# refuse an unsupported activation before it writes anything. # # What it does when on with at least one rule: one POST with the project name and the brief's # `## Captain's intent` and `## Firstmate spec` sections, tagged when it is a @@ -26,7 +51,7 @@ # after that is jq: the confidence floor (0.6 on the answer confidence, or a # rule's declared `min_confidence` on that rule's probability, falling to the # most probable other option that clears its own floor), the rule's declared -# `approval` and `floor`, each profile's declared `provider` and `floor`, the +# `approval` and `floor`, each profile's declared `provider`, `quota_scope`, and `floor`, the # quota rows from ONE quota-axi --json snapshot (schema 5 or 6; each # candidate binds to one row through quota_row in # bin/fm-quota-axi-lib.sh, so a Pi lane such as openai-codex-work/... @@ -38,6 +63,17 @@ # The model never sees quota, catalogs, approvals, preferred_provider, # confidence floors, `why`, or `use`. With no rules, it returns a non-clear # result so firstmate keeps using the existing intake. +# +# Shared-quota opt-out: an optional top-level "shared_quota": true in the +# rules file declares this home draws every provider's usage from one single +# shared budget, so per-provider quota rows do not describe what is left. +# With it set, this tool makes no quota-axi call, applies no quota floor or +# runway gate, ranks nothing by spendPriority, and resolves the matched +# rule's `use` profiles in declared order; preferred_provider still narrows +# the pool; declared order then picks within it. Every candidate line still +# prints, stating quota ranking is off by home declaration. Without the +# setting, behavior is unchanged. The declared value must be a boolean; +# anything else is an actionable configuration error. # docs/configuration.md "Crew dispatch profiles" owns the declared fields and # "Typed dispatch resolution" owns this tool's operator contract. # @@ -60,16 +96,19 @@ # reason: # preferred_provider: # candidate: : provider=.. scope=.. remaining=..% spendPriority=.. runway=.. -> eligible | eligible, unranked: | not eligible: +# (with the shared-quota opt-out: : provider=.. -> eligible (quota ranking off: home declares a single shared quota)) # profile: --harness [--model ] [--effort ] [--seat ] [--seat-home ] (status clear only) # clear -> pass the profile line to fm-spawn.sh unless you state a reason to override # ambiguous -> confidence below the floor; decide as today from the probabilities # escalate -> the rule requires captain approval, no candidate is rankable, or a genuine tie -# error -> API, network, response, or quota-axi failure; decide as today +# error -> API, network, response, quota-axi, or declared credential source failure; decide as today # Every outcome exits 0 so an intake is never blocked by this tool. # Exit 2 only for a usage or configuration error (unreadable brief, an -# existing unreadable rules file, malformed rules, missing jq, an unknown -# provider, a selected non-typesafe provider without a model, or systemone -# without a valid base URL), which is actionable, never selected around. +# existing unreadable rules file, malformed rules, an unreadable or invalid +# credential source declaration, missing jq, an unknown provider, a selected +# non-typesafe provider without a model, or systemone without a valid base +# URL), which is actionable, never selected around. +# The capability probe is the one other exit: 1 for an unsupported source. # # Environment: # FM_DISPATCH_PROVIDER, FM_DISPATCH_MODEL, FM_DISPATCH_BASE_URL, @@ -111,6 +150,8 @@ CONFIG="${FM_CONFIG_OVERRIDE:-$FM_HOME/config}" . "$SCRIPT_DIR/fm-timing-lib.sh" # shellcheck source=bin/fm-brief-heading-lib.sh . "$SCRIPT_DIR/fm-brief-heading-lib.sh" +# shellcheck source=bin/fm-credential-source-lib.sh +. "$SCRIPT_DIR/fm-credential-source-lib.sh" CONFIDENCE_FLOOR=0.6 TS_MODEL=jev-latest @@ -128,6 +169,12 @@ no_rules() { printf 'dispatch-resolve:\n status: escalate\n reason: no rules to match\n' exit 0 } +emit_error() { + local reason=$1 + echo "dispatch-resolve: error ($reason)" >&2 + printf 'dispatch-resolve:\n status: error\n reason: %s\n' "$reason" + exit 0 +} usage() { awk ' NR == 1 { next } @@ -138,9 +185,14 @@ usage() { BRIEF='' PROJECT='' RULES_PATH="$CONFIG/crew-dispatch.json" RULES='' NEVER_SEND_PATH="$CONFIG/dispatch-never-send" +CRED_PATH="$CONFIG/dispatch-credential-sources.json" while [ $# -gt 0 ]; do case "$1" in --project) [ $# -ge 2 ] || die "--project needs a value"; PROJECT=$2; shift 2 ;; + --supports-credential-source) + [ $# -ge 2 ] || die "--supports-credential-source needs a value" + if fm_credsrc_supported "$2"; then echo supported; exit 0; fi + echo unsupported; exit 1 ;; -h|--help) usage; exit 0 ;; -*) die "unknown flag $1" ;; *) [ -z "$BRIEF" ] || die "one brief file only"; BRIEF=$1; shift ;; @@ -148,6 +200,47 @@ while [ $# -gt 0 ]; do done # ---- opt-in gate --------------------------------------------------------------- +# declared_key : set the variable from the source +# $CRED_PATH declares for . Returns 1 when none is declared, which +# leaves the off gate in charge. Past that point nothing is mistaken for an +# unconfigured resolver: an unreadable or invalid declaration is a +# configuration error, and a declared source this host cannot run or that +# yields no key is an error outcome. Nothing is sent in either case. +declared_key() { + local provider=$1 src problem value rc + [ -e "$CRED_PATH" ] || [ -L "$CRED_PATH" ] || return 1 + [ -r "$CRED_PATH" ] || die "credential source declaration not readable: $CRED_PATH" + command -v jq >/dev/null 2>&1 || die "jq required" + problem=$(fm_credsrc_check "$CRED_PATH" typesafe openrouter) \ + || die "malformed credential source declaration: $CRED_PATH - $problem" + fm_credsrc_declares "$CRED_PATH" "$provider" || return 1 + src=$(fm_credsrc_field "$CRED_PATH" "$provider" source) + fm_credsrc_supported "$src" \ + || emit_error "credential source $src declared for $provider is not supported on this host; nothing sent" + value=$(fm_credsrc_read "$CRED_PATH" "$provider"); rc=$? + if [ "$rc" -ne 0 ] || [ -z "$value" ]; then + if fm_timed_out "$rc"; then + emit_error "credential source $src declared for $provider did not answer in time; nothing sent" + fi + emit_error "credential source $src declared for $provider is unavailable; nothing sent" + fi + printf -v "$2" '%s' "$value" +} + +# select_key : the selected provider's +# key is the environment copy already in the variable, else KEY_NAME in +# $FM_HOME/.env, else the source declared for the provider. With none of them +# the tool is off, exactly as it was before declarations existed. +select_key() { + local provider=$1 name=$2 var=$3 + [ -n "${!var}" ] || printf -v "$var" '%s' "$(fmx_env_get "$name" "$FM_HOME/.env")" + [ -n "${!var}" ] || declared_key "$provider" "$var" + if [ -z "${!var}" ]; then + echo "dispatch-resolve: off ($name absent from the environment and $FM_HOME/.env)" >&2 + exit 0 + fi +} + DISPATCH_PROVIDER=${FM_DISPATCH_PROVIDER:-} [ -n "$DISPATCH_PROVIDER" ] || DISPATCH_PROVIDER=$(fmx_env_get FM_DISPATCH_PROVIDER "$FM_HOME/.env") DISPATCH_MODEL=${FM_DISPATCH_MODEL:-} @@ -156,27 +249,15 @@ case "$DISPATCH_PROVIDER" in '') DISPATCH_PROVIDER=typesafe DISPATCH_MODEL=$TS_MODEL - [ -n "$TYPESAFE_API_KEY_PRIVATE" ] || TYPESAFE_API_KEY_PRIVATE=$(fmx_env_get TYPESAFE_API_KEY "$FM_HOME/.env") - if [ -z "$TYPESAFE_API_KEY_PRIVATE" ]; then - echo "dispatch-resolve: off (TYPESAFE_API_KEY absent from the environment and $FM_HOME/.env)" >&2 - exit 0 - fi + select_key typesafe TYPESAFE_API_KEY TYPESAFE_API_KEY_PRIVATE ;; typesafe) [ -n "$DISPATCH_MODEL" ] || DISPATCH_MODEL=$TS_MODEL - [ -n "$TYPESAFE_API_KEY_PRIVATE" ] || TYPESAFE_API_KEY_PRIVATE=$(fmx_env_get TYPESAFE_API_KEY "$FM_HOME/.env") - if [ -z "$TYPESAFE_API_KEY_PRIVATE" ]; then - echo "dispatch-resolve: off (TYPESAFE_API_KEY absent from the environment and $FM_HOME/.env)" >&2 - exit 0 - fi + select_key typesafe TYPESAFE_API_KEY TYPESAFE_API_KEY_PRIVATE ;; openrouter) [ -n "$DISPATCH_MODEL" ] || die "FM_DISPATCH_MODEL required when FM_DISPATCH_PROVIDER=openrouter" - [ -n "$OPENROUTER_API_KEY_PRIVATE" ] || OPENROUTER_API_KEY_PRIVATE=$(fmx_env_get OPENROUTER_API_KEY "$FM_HOME/.env") - if [ -z "$OPENROUTER_API_KEY_PRIVATE" ]; then - echo "dispatch-resolve: off (OPENROUTER_API_KEY absent from the environment and $FM_HOME/.env)" >&2 - exit 0 - fi + select_key openrouter OPENROUTER_API_KEY OPENROUTER_API_KEY_PRIVATE ;; systemone) [ -n "$DISPATCH_MODEL" ] || die "FM_DISPATCH_MODEL required when FM_DISPATCH_PROVIDER=systemone" @@ -259,7 +340,7 @@ rules_err=$(jq -r --argjson verified_harnesses "$VERIFIED_HARNESSES" --arg provi ($items | map([.harness, (.model // null), (.effort // null), (.seat // null)] | @json)) as $keys | ($keys | length) != ($keys | unique | length); def invalid_seats($items): - [$items[] | select(has("seat") and ((seat_ok(.seat) | not) or .harness != "codex")) | (.seat | tostring)] | unique; + [$items[] | select(type == "object" and has("seat") and ((seat_ok(.seat) | not) or .harness != "codex")) | (.seat | tostring)] | unique; if type != "object" then "top-level value must be an object" elif has("preferred_provider") and (provider_id(.preferred_provider) | not) then "preferred_provider must be a lowercase provider ID (for example codex)" elif has("rules") and (.rules | type) != "array" then "rules must be an array" @@ -271,9 +352,11 @@ rules_err=$(jq -r --argjson verified_harnesses "$VERIFIED_HARNESSES" --arg provi elif any((.rules // [])[]; has("select") and ((.select | type) != "string" or (.select | length) == 0)) then "select must be a non-empty string" elif any((.rules // [])[]; has("select") and .select != "quota-balanced") then "unknown select: " + ([.rules[] | select(has("select") and .select != "quota-balanced") | .select] | unique | join(", ")) + elif has("shared_quota") and (.shared_quota | type) != "boolean" then "shared_quota must be true or false when present" elif any((.rules // [])[]; has("floor") and floor_bad(.floor; true)) then "rule floor needs scope, min_percent 0..100, and provider matching ^[a-z0-9]+(-[a-z0-9]+)*\\z" elif (invalid_seats([(.rules // [])[] | profiles(.use)[]]) | length) > 0 then "unsupported use profile seat (safe name on codex): " + (invalid_seats([(.rules // [])[] | profiles(.use)[]]) | join(", ")) + elif any((.rules // [])[] | profiles(.use)[]; type == "object" and has("quota_scope") and ((.quota_scope | type) != "string" or (.quota_scope | length) == 0)) then "each use profile quota_scope must be a non-empty string when present" elif any((.rules // [])[] | profiles(.use)[]; profile_bad(.)) then "each use profile needs harness; model, effort, seat, and floor must be well formed, and provider must match ^[a-z0-9]+(-[a-z0-9]+)*\\z when present" elif any((.rules // [])[]; duplicate_profiles(profiles(.use))) then "each rule use must not contain duplicate harness, model, and effort profiles" elif any((.rules // [])[] | profiles(.use)[]; (verified(.harness) | not)) then "each use profile must name a verified harness" @@ -281,6 +364,7 @@ rules_err=$(jq -r --argjson verified_harnesses "$VERIFIED_HARNESSES" --arg provi elif has("default") and (profiles(.default) | length) == 0 then "default must be a profile object or non-empty profile array" elif has("default") and (invalid_seats(profiles(.default)) | length) > 0 then "unsupported default profile seat (safe name on codex): " + (invalid_seats(profiles(.default)) | join(", ")) + elif has("default") and any(profiles(.default)[]; type == "object" and has("quota_scope") and ((.quota_scope | type) != "string" or (.quota_scope | length) == 0)) then "each default profile quota_scope must be a non-empty string when present" elif has("default") and any(profiles(.default)[]; profile_bad(.)) then "each default profile needs harness; model, effort, seat, and floor must be well formed, and provider must match ^[a-z0-9]+(-[a-z0-9]+)*\\z when present" elif has("default") and duplicate_profiles(profiles(.default)) then "default must not contain duplicate harness, model, and effort profiles" elif has("default") and any(profiles(.default)[]; (verified(.harness) | not)) then "each default profile must name a verified harness" @@ -337,13 +421,6 @@ done < <(jq -r ' RULE_COUNT=$(jq -r '(.rules // []) | length' "$RULES") -emit_error() { - local reason=$1 - echo "dispatch-resolve: error ($reason)" >&2 - printf 'dispatch-resolve:\n status: error\n reason: %s\n' "$reason" - exit 0 -} - if [ "$RULE_COUNT" -eq 0 ]; then no_rules fi @@ -472,11 +549,11 @@ command -v curl >/dev/null 2>&1 || emit_error "curl not installed" if [ "$DISPATCH_PROVIDER" = openai ]; then jq -e ' (.answers | type) == "array" and - ([.answers[] | select(type == "object" and .name == "rule")]) as $rule | + (([.answers[] | select(type == "object" and .name == "rule")]) as $rule | ($rule | length) == 1 and $rule[0].type == "choice" and ($rule[0].probabilities | type) == "array" and all($rule[0].probabilities[]; type == "object" and (.value | type) == "string") and - ($rule[0].probabilities | map(.value) | length) == ($rule[0].probabilities | map(.value) | unique | length)' \ + ($rule[0].probabilities | map(.value) | length) == ($rule[0].probabilities | map(.value) | unique | length))' \ "$RESP_FILE" >/dev/null 2>&1 || emit_error "response is not a rule Choice answer" if ! jq --arg model "$DISPATCH_MODEL" ' ([.answers[] | select(type == "object" and .name == "rule")][0]) as $a | @@ -509,13 +586,21 @@ jq -e --slurpfile rules "$RULES" ' (.usage.output_tokens | type) == "number"))' \ "$RESP_FILE" >/dev/null 2>&1 || emit_error "response is not a rule Choice answer" -# ---- quota evidence: one quota-axi --json snapshot ----------------------------- -command -v quota-axi >/dev/null 2>&1 || emit_error "quota-axi not installed" -quota-axi --json > "$QUOTA" 2>/dev/null || emit_error "quota-axi --json failed" -fm_quota_json_valid < "$QUOTA" || emit_error "quota-axi --json returned an invalid snapshot" +# ---- quota evidence: one quota-axi --json snapshot, skipped entirely under the +# ---- shared-quota opt-out (a stub keeps the slurpfile contract intact) --------- +SQ=false +SHARED_QUOTA=$(jq -r '.shared_quota // false' "$RULES" 2>/dev/null) || SHARED_QUOTA=false +if [ "$SHARED_QUOTA" = true ]; then + printf '{"providers":[]}\n' > "$QUOTA" + SQ=true +else + command -v quota-axi >/dev/null 2>&1 || emit_error "quota-axi not installed" + quota-axi --json > "$QUOTA" 2>/dev/null || emit_error "quota-axi --json failed" + fm_quota_json_valid < "$QUOTA" || emit_error "quota-axi --json returned an invalid snapshot" +fi # ---- resolution: declared gates + quota evidence + argmax, all in jq ------------ -RESULT=$(jq -n --arg floor "$CONFIDENCE_FLOOR" --argjson lat "$LAT_MS" --arg none_criterion "$DEFAULT_WHEN" --argjson pmap "$PMAP" \ +RESULT=$(jq -n --arg floor "$CONFIDENCE_FLOOR" --argjson lat "$LAT_MS" --arg none_criterion "$DEFAULT_WHEN" --argjson pmap "$PMAP" --argjson sq "$SQ" \ --argjson seat_homes "$SEAT_HOMES" \ --slurpfile resp "$RESP_FILE" --slurpfile rules "$RULES" --slurpfile quota "$QUOTA" "$FM_QUOTA_ROW_JQ"' ($resp[0]) as $r | ($rules[0]) as $cfg | ($quota[0]) as $q | ($r.answers.rule) as $a | @@ -527,11 +612,12 @@ RESULT=$(jq -n --arg floor "$CONFIDENCE_FLOOR" --argjson lat "$LAT_MS" --arg non def lane_of($c): if $c.seat != null then "seat:" + $seat_homes[$c.seat] else quota_lane($c.harness; $c.model) end; def measured($p; $lane): (prov($p; $lane) != null and (["known", "partial"] | index(prov($p; $lane).quotaSemantics.status)) != null); - def applicable($p; $lane; $m): + def applicable($p; $lane; $m; $qs): (bare($m)) as $bare | [rows($p; $lane)[] | select( .scope == "all_models" or .scope == "all_products" or - ($m != "" and (.scope == ("model:" + $bare) or .scope == ("product:" + $bare))) + ($m != "" and (.scope == ("model:" + $bare) or .scope == ("product:" + $bare))) or + ($qs != null and .scope == $qs) )]; def floor_state($f; $p; $lane): if $f == null then "none" @@ -546,14 +632,16 @@ RESULT=$(jq -n --arg floor "$CONFIDENCE_FLOOR" --argjson lat "$LAT_MS" --arg non $rows | map({scope, status, pct: (.effectivePercentRemaining // null), runway: (.runway.status // null), spendPriority: (.selection.spendPriority // null)}); def evaluate($c): (provider_of($c)) as $p | (lane_of($c)) as $lane | - if $p == null then {profile: $c, eligible: false, reason: "no provider family for harness \($c.harness); declare provider on the profile"} + if $sq then + {profile: $c, provider: $p, eligible: true, sq: true, reason: "ok"} + elif $p == null then {profile: $c, eligible: false, reason: "no provider family for harness \($c.harness); declare provider on the profile"} elif prov($p; $lane) == null then {profile: $c, provider: $p, eligible: true, unranked: true, reason: (if any($q.providers[]; .provider == $p) then "provider \($p) has no quota row for account \(if $lane == "" then "default" elif ($lane | startswith("seat:")) then $c.seat else $lane end)" else "provider \($p) not in the quota snapshot" end)} else - (applicable($p; $lane; ($c.model // ""))) as $rows | + (applicable($p; $lane; ($c.model // ""); ($c.quota_scope // null))) as $rows | (evidence($rows)) as $bounds | (floor_state($c.floor; $p; $lane)) as $profile_floor_state | if any($rows[]; (.runway.status // "") == "exhausted_now") then @@ -571,6 +659,8 @@ RESULT=$(jq -n --arg floor "$CONFIDENCE_FLOOR" --argjson lat "$LAT_MS" --arg non elif (measured($p; $lane) | not) then ($rows | first) as $row | {profile: $c, provider: $p, bounds: $bounds, scope: ($row.scope // null), pct: ($row.effectivePercentRemaining // null), runway: ($row.runway.status // null), eligible: true, unranked: true, unknown: true, reason: "provider \($p) unmeasured (\(prov($p; $lane).quotaSemantics.status))"} + elif ($c.quota_scope != null and (any($rows[]; .scope == $c.quota_scope) | not)) then + {profile: $c, provider: $p, bounds: $bounds, scope: $c.quota_scope, eligible: true, unranked: true, unknown: true, reason: "declared quota_scope \($c.quota_scope) has no quota row for provider \($p): not rankable"} elif ($rows | length) == 0 then {profile: $c, provider: $p, bounds: $bounds, eligible: true, unranked: true, unknown: true, reason: "no applicable quota row for provider \($p)"} elif $profile_floor_state == "unknown" then @@ -614,7 +704,7 @@ RESULT=$(jq -n --arg floor "$CONFIDENCE_FLOOR" --argjson lat "$LAT_MS" --arg non end) as $fb | (if $fb.to then $fb.to else $picked end) as $choice | (rule_at($choice)) as $rule | - (if $rule == null then "none" else floor_state($rule.floor; $rule.floor.provider; "") end) as $rule_floor_state | + (if $sq or $rule == null then "none" else floor_state($rule.floor; $rule.floor.provider; "") end) as $rule_floor_state | (if $choice != "default" and $rule == null then [] elif $rule == null then profiles($cfg.default // null) else profiles($rule.use) @@ -660,8 +750,8 @@ RESULT=$(jq -n --arg floor "$CONFIDENCE_FLOOR" --argjson lat "$LAT_MS" --arg non (if ($preferred | length) > 0 then " in preferred provider \($cfg.preferred_provider); uncertainty does not authorize fallback" else "" end)), note: $sel.note, candidates: $cands} else - ($elig | max_by(.spendPriority)) as $best | - ([$elig[] | select(.spendPriority == $best.spendPriority)] | length) as $ties | + (if $sq then ($elig | first) else ($elig | max_by(.spendPriority)) end) as $best | + (if $sq then 1 else ([$elig[] | select(.spendPriority == $best.spendPriority)] | length) end) as $ties | if $ties > 1 then $ev + {status: "escalate", reason: "genuine spendPriority tie", note: $sel.note, candidates: $cands} else $ev + {status: "clear", note: $sel.note, candidates: $cands, chosen: $best} + (if ($unranked | length) > 0 then @@ -689,7 +779,7 @@ TEXT=$(jq -r --argjson seat_homes "$SEAT_HOMES" ' + (if .provider then " provider=\(.provider | flat)" else "" end) + (if .scope then " scope=\(.scope | flat) remaining=\(show(.pct))% spendPriority=\(show(.spendPriority)) runway=\(show(.runway))" else "" end) + (if (.bounds // [] | length) > 1 then " bounds=" + ([.bounds[] | "\(.scope | flat):\(show(.pct))%/\((.runway // .status) | flat)"] | join(",")) else "" end) - + " -> " + (if .unranked then "eligible, unranked: \(.reason | flat): disclosed uncertainty" elif .eligible then "eligible" else "not eligible: \(.reason | flat)" end)), + + " -> " + (if .unranked then "eligible, unranked: \(.reason | flat): disclosed uncertainty" elif .sq then "eligible (quota ranking off: home declares a single shared quota)" elif .eligible then "eligible" else "not eligible: \(.reason | flat)" end)), (if .chosen then " profile: --harness \(.chosen.profile.harness | shell_arg)" + (if .chosen.profile.model then " --model \(.chosen.profile.model | shell_arg)" else "" end) + (if .chosen.profile.effort then " --effort \(.chosen.profile.effort | shell_arg)" else "" end) diff --git a/bin/fm-ff-lib.sh b/bin/fm-ff-lib.sh index aa2234c06d3..221e917229b 100644 --- a/bin/fm-ff-lib.sh +++ b/bin/fm-ff-lib.sh @@ -5,7 +5,11 @@ # This is the one implementation of "advance a firstmate checkout to a base by a # clean fast-forward, never forcing, merging, or stashing" used by every sync # path: -# - /updatefirstmate (bin/fm-update.sh) pulls from origin: base_mode "origin". +# - /updatefirstmate (bin/fm-update.sh) pulls the runtime branch from its +# tracking remote: base_mode "tracking". With an optional +# config/self-update-source, it instead fetches that one remote branch for the +# primary and passes it as a local base; its secondmates follow the primary's +# resulting commit either way (bin/fm-update.sh owns it). # - the local-HEAD secondmate sync (bin/fm-spawn.sh on launch, bin/fm-bootstrap.sh # on startup) follows the PRIMARY checkout's current default-branch commit: # base_mode is that local commit, with NO fetch and no origin dependency. @@ -373,13 +377,20 @@ live_secondmate_meta_records() { # dependency (the local-HEAD secondmate sync). The commit must # already exist in the target's object store, which it always does # for a worktree of this same repo; a standalone clone that lacks -# it is skipped rather than fetched. +# it is skipped rather than fetched, unless the caller sets +# FF_IMPORT_FROM to a local repository path that holds it. Then +# that one commit is imported from there first (objects only, +# no ref of the target moves), the same read-only import the +# remote home sync uses. /updatefirstmate sets it to the primary +# checkout when config/self-update-source is in use, because no +# origin fetch then refreshes a standalone clone. # Guards are identical in both modes: never force/merge/stash; skip a dirty or # wrong-branch target and leave its work untouched. An optional secondmate id # enables the content-equivalent divergence proof and durable marker described # in this file's header. FF_STATUS="" FF_INSTR="" +FF_IMPORT_FROM="" ff_target() { local dir=$1 label=$2 base_mode=$3 allow_detached=${4:-no} ignore_seed_marker=${5:-no} local secondmate_id=${6:-} reconciliation_state=${7:-} @@ -429,6 +440,10 @@ ff_target() { fi else base="$base_mode" + if [ -n "$FF_IMPORT_FROM" ] \ + && ! git -C "$dir" rev-parse --verify --quiet "$base^{commit}" >/dev/null; then + git -C "$dir" fetch --quiet --no-tags -- "$FF_IMPORT_FROM" "$base" >/dev/null 2>&1 || true + fi fi if ! git -C "$dir" rev-parse --verify --quiet "$base^{commit}" >/dev/null; then diff --git a/bin/fm-harness.sh b/bin/fm-harness.sh index 1c6a0569546..9d89c8a0a9b 100755 --- a/bin/fm-harness.sh +++ b/bin/fm-harness.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # Detect the agent harness this process tree runs on. -# Usage: fm-harness.sh print own harness: claude|codex|opencode|pi|pi-signed|grok|kimi|cursor|gemini|muse|rovo|omp|agy|devin|commandcode|polytoken|cortex|unknown +# Usage: fm-harness.sh print own harness: claude|codex|opencode|pi|pi-signed|grok|kimi|cursor|gemini|muse|rovo|omp|agy|devin|commandcode|polytoken|cortex|kiro|hermes|unknown # fm-harness.sh crew print the effective CREWMATE harness # (config/crew-harness; "default" resolves to own) # fm-harness.sh secondmate [id] print the harness the PRIMARY uses to launch @@ -79,11 +79,16 @@ CONFIG="${FM_CONFIG_OVERRIDE:-$FM_HOME/config}" . "$SCRIPT_DIR/fm-cursor-lib.sh" # shellcheck source=bin/fm-gemini-lib.sh . "$SCRIPT_DIR/fm-gemini-lib.sh" +# shellcheck source=bin/fm-hermes-process-lib.sh +. "${SCRIPT_DIR:-/}/fm-hermes-process-lib.sh" # Print the harness named by a verified environment marker, or nothing when no # marker is present. Markers only report what the environment CLAIMS; detect_own # decides whether that claim survives contradicting ancestry. harness_marker() { + if [ "${FM_HERMES_HARNESS:-}" = hermes ] && [ "$(harness_ancestry | awk '{print $2}')" = hermes ]; then + echo hermes; return + fi # Cursor is tested BEFORE claude, deliberately. cursor-agent does NOT clear an # inherited CLAUDECODE, so a cursor session started by hand from a claude # primary carries BOTH markers and whichever is tested first wins. This @@ -146,7 +151,7 @@ harness_marker() { # identified, and any rule that must be RELIABLE under grok has to test the hook # markers too (see .claude/settings.json Stop entries, docs/turnend-guard.md). [ "${GROK_AGENT:-}" = "1" ] && { echo grok; return; } - # codex, opencode, kimi, muse, agy, devin, commandcode, polytoken, and cortex publish no harness-identity marker at all, so + # codex, opencode, kimi, muse, agy, devin, commandcode, polytoken, cortex, and kiro publish no harness-identity marker at all, so # they are never named here and are identified by ancestry alone. That is the # whole reason a foreign marker must not outrank ancestry: with markers winning # unconditionally, any retained CLAUDECODE would silently rename one of them. @@ -192,6 +197,11 @@ harness_process_verdict() { # echo "comm gemini" return fi + args=$(ps -o args= -p "$pid" 2>/dev/null) || args= + if fm_hermes_process_matches "$args" "$pid"; then + echo "comm hermes" + return + fi case "$(basename -- "$comm")" in # gemini precedes claude here for the same precedence reason as the # marker layer above, so a gemini worker under a claude primary is never @@ -261,11 +271,24 @@ harness_process_verdict() { # # marker a parent harness could not also carry, so like devin it is # detected by anchored ancestry alone. cortex) echo "comm cortex"; return ;; + hermes) echo "comm hermes"; return ;; # Command Code runs as a node bundle that sets its own process title, so # the kernel process name is exactly `command-code` (verified, Command Code # 1.74.1: comm=command-code and the rewritten argv is the same word). # Anchored, so an unrelated command containing that fragment never matches. command-code) echo "comm commandcode"; return ;; + # kiro (Kiro CLI) publishes no harness-identity marker of its own (a live + # 2.21.4 tool subprocess carries no KIRO_* identity variable; KIRO_HOME is + # a firstmate-set config-relocation path, not an identity), so like muse + # and agy it is detected by ancestry alone. The installed command is a + # toolbox/aim-sandbox wrapper whose foreground process name is exactly + # `kiro-cli` (verified, kiro-cli 2.21.4: `ps -o comm=` and tmux + # #{pane_current_command} both report kiro-cli; the aim-sandbox and the + # compiled bun/node binary run as descendants). Anchored, never *kiro*, so + # unrelated commands cannot be misread as this harness. kiro is a + # crewmate/scout adapter only and is never a primary or secondmate, so + # this arm only lets a kiro crewmate identify its own harness. + kiro-cli) echo "comm kiro"; return ;; node*|python*) # Bare interpreter: match the harness name in its script path. args=$(ps -o args= -p "$pid" 2>/dev/null) @@ -424,7 +447,7 @@ supervision_primary_pin() { local pin=${FM_SUPERVISION_PRIMARY_HARNESS:-} [ "${FM_SUPERVISION_ACTOR:-}" = branch ] && [ -n "$pin" ] || return 0 case "$pin" in - claude|codex|opencode|pi|pi-signed|grok|kimi|cursor|gemini|muse|rovo|omp|agy|devin|commandcode|polytoken|cortex) + claude|codex|opencode|pi|pi-signed|grok|kimi|cursor|gemini|muse|rovo|omp|agy|devin|commandcode|polytoken|cortex|hermes) printf '%s\n' "$pin" ;; *) diff --git a/bin/fm-hc-ci-cleanup.sh b/bin/fm-hc-ci-cleanup.sh new file mode 100755 index 00000000000..da1f382eb06 --- /dev/null +++ b/bin/fm-hc-ci-cleanup.sh @@ -0,0 +1,178 @@ +#!/usr/bin/env bash +# fm-hc-ci-cleanup.sh - bounded cleanup of CI-owned herdr-core (`hc`) lab +# sessions. +# +# Snapshot the lab session inventory before the real-hc suite, then at job +# end only stop/remove session directories that: +# 1. match the guarded fm-hclab-* name pattern under the lab root, +# 2. were not present in the pre-suite snapshot (job-proven ownership). +# +# bin/fm-hc-lab.sh owns the session directory and socket layout. +# Cleanup requests `hc server stop` under the session's own pinned env, then +# sweeps its session-marked processes before removing its directory and sockets. +# Unlike normal lab teardown, this job-end fallback does not verify the fleet +# tripwire, so it is not evidence that real-core state stayed unchanged. +# +# A missing binary skips only the stop request; process and file cleanup +# still run. Surviving session processes or directory-removal failures exit +# non-zero. +# +# Usage: +# fm-hc-ci-cleanup.sh snapshot +# fm-hc-ci-cleanup.sh teardown +set -eu + +die() { + printf 'fm-hc-ci-cleanup.sh: %s\n' "$*" >&2 + exit 1 +} + +log() { + printf 'fm-hc-ci-cleanup.sh: %s\n' "$*" >&2 +} + +LAB_DIR=${FM_HC_LAB_DIR:-${TMPDIR:-/tmp}/fm-hc-lab-${UID}} +SOCK_DIR=${FM_HC_LAB_SOCK_DIR:-/tmp/fm-hc-lab-${UID}} + +cmd=${1:-} +path=${2:-} +[ -n "$cmd" ] && [ -n "$path" ] || die "usage: fm-hc-ci-cleanup.sh snapshot|teardown " + +is_lab_name() { + local name=$1 + [[ "$name" =~ ^fm-hclab-[a-zA-Z0-9][a-zA-Z0-9_-]*$ ]] +} + +# One basename per directory, computed in the shell so the inventory does not +# depend on multi-operand basename (BSD/GNU differ). A find failure is a real +# error, never an empty list. +list_lab_names() { + local found d + [ -d "$LAB_DIR" ] || return 0 + found=$(find "$LAB_DIR" -mindepth 1 -maxdepth 1 -type d -name 'fm-hclab-*' -print) || return 1 + [ -n "$found" ] || return 0 + printf '%s\n' "$found" | while IFS= read -r d; do + printf '%s\n' "${d##*/}" + done | LC_ALL=C sort +} + +# hc_ci_stop: `hc server stop` under the lab session's own pinned env, +# resolving the binary exactly like bin/fm-hc-lab.sh (FM_HC_BIN, then +# herdr-core, then hc). A missing binary is a no-op. +hc_ci_stop() { # + local name=$1 dir=$2 sock=$3 bin='' + if [ -n "${FM_HC_BIN:-}" ] && [ -x "$FM_HC_BIN" ]; then + bin=$FM_HC_BIN + elif command -v herdr-core >/dev/null 2>&1; then + bin=$(command -v herdr-core) + elif command -v hc >/dev/null 2>&1; then + bin=$(command -v hc) + fi + [ -n "$bin" ] || return 0 + ( unset HC_TERMINAL_ID HC_TERMINAL_SOCKET HC_SCOPE HC_SESSION HC_SOCKET + XDG_STATE_HOME="$dir/env/xdg-state" \ + XDG_CONFIG_HOME="$dir/env/xdg-config" \ + XDG_DATA_HOME="$dir/env/xdg-data" \ + HC_SESSION="$name" HC_SOCKET="$sock" \ + "$bin" server stop --json /dev/null 2>&1 ) || true +} + +# hc_ci_caller_pids: the sweeper's own pid, its ancestor chain, and its +# process group (as a `g` token). A caller that exported HC_SESSION +# spreads that env token across its whole process tree - including this +# sweep's own ps/grep/awk transients - which must never be signaled. Session +# daemons are setsid'd (reparented to init), so they can never share the +# caller's process group. +HC_CI_CALLER_PIDS="" +hc_ci_caller_pids() { + [ -n "$HC_CI_CALLER_PIDS" ] && return 0 + local pid=$PPID pgid + pgid=$(ps -o pgid= -p $$ 2>/dev/null | tr -d ' ') + HC_CI_CALLER_PIDS=" $$ g${pgid:-0}" + while [ -n "$pid" ] && [ "$pid" -gt 1 ] 2>/dev/null; do + HC_CI_CALLER_PIDS="$HC_CI_CALLER_PIDS $pid" + pid=$(ps -o ppid= -p "$pid" 2>/dev/null | tr -d ' ') + done +} + +# hc_ci_session_pids: pids of processes carrying HC_SESSION= in their +# environment (the session's serve/hold daemons and children). The match is +# anchored on env-token boundaries on both sides so "fm-hclab-a" can never +# match "fm-hclab-ab" or a longer assignment. Same-uid env scan works on macOS +# (BSD ps) and Linux (procps) alike; the caller's process tree is excluded. +hc_ci_session_pids() { # + local name=$1 + hc_ci_caller_pids + # shellcheck disable=SC2009 # the match is on the process ENVIRONMENT (HC_SESSION), which pgrep -f cannot read. + ps ewwax -o pid=,ppid=,pgid=,command= 2>/dev/null \ + | grep -E "(^|[[:space:]])HC_SESSION=${name}([[:space:]]|$)" \ + | awk -v excl="$HC_CI_CALLER_PIDS" \ + 'index(excl, " "$1" ") == 0 && index(excl, " "$2" ") == 0 && index(excl, " g"$3" ") == 0 { print $1 }' +} + +case "$cmd" in + snapshot) + list_lab_names >"$path" || die "failed to write lab session snapshot to $path" + log "wrote lab session snapshot to $path ($(wc -l <"$path" | tr -d ' ') names)" + ;; + teardown) + [ -f "$path" ] || die "snapshot file not found: $path" + names=$(list_lab_names) || die "could not inventory lab sessions" + failed=0 + while IFS= read -r name; do + [ -n "$name" ] || continue + is_lab_name "$name" || { + log "refusing non-lab name from candidate set: $name" + failed=1 + continue + } + if grep -qxF "$name" "$path"; then + continue + fi + dir=$LAB_DIR/$name + sock=$SOCK_DIR/$name.sock + [ -e "$dir" ] || continue + log "stopping job-owned lab core session $name" + hc_ci_stop "$name" "$dir" "$sock" + sleep 0.3 + # `server stop` closes the listener but the session's serve/hold + # daemons stay by design; sweep every process still carrying the + # session's HC_SESSION env marker (same-uid env scan, macOS and Linux). + for sig in TERM TERM TERM TERM KILL KILL KILL KILL; do + pids=$(hc_ci_session_pids "$name") + [ -z "$pids" ] && break + printf '%s\n' "$pids" | while IFS= read -r p; do + [ -n "$p" ] && kill -"$sig" "$p" 2>/dev/null || true + done + sleep 0.3 + done + # Signaled daemons take a while to finish exiting - measured ~30-60s + # while they drain terminal state. Keep polling well past that + # envelope before counting them survivors. + i=0 + while [ "$i" -lt 180 ]; do + pids=$(hc_ci_session_pids "$name") + [ -z "$pids" ] && break + sleep 0.5 + i=$((i + 1)) + done + pids=$(hc_ci_session_pids "$name") + if [ -n "$pids" ]; then + log "session processes for $name did not exit: $pids" + failed=1 + continue + fi + if rm -rf "$dir"; then + rm -f "$sock" "$SOCK_DIR/$name".holder.*.sock 2>/dev/null || true + log "removed job-owned lab core session $name" + else + log "failed to remove lab session dir $dir" + failed=1 + fi + done <<< "$names" + [ "$failed" -eq 0 ] || die "one or more job-owned lab sessions could not be cleaned" + ;; + *) + die "unknown command: $cmd (use snapshot or teardown)" + ;; +esac diff --git a/bin/fm-hc-lab.sh b/bin/fm-hc-lab.sh new file mode 100755 index 00000000000..dbf779fe178 --- /dev/null +++ b/bin/fm-hc-lab.sh @@ -0,0 +1,419 @@ +#!/usr/bin/env bash +# Provision and operate an isolated Herdr Core (`hc`) lab session without +# risking the user's own core state. +# +# Usage: +# fm-hc-lab.sh name