Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 48 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,42 @@ Notable changes to the Swarm plugin. Format follows

## [Unreleased]

### Added
- Per-repo port ranges: `git config swarm.portStart` / `swarm.portSpan`. Two
repos swarming at once both started at 4100, and the only override was
environment-only, which an action-invoked fan-out pane never inherits. An
exported `HERDR_SWARM_PORT_START`/`_SPAN` still wins.

### Fixed
- Merge winner could skip — and, on a clean worktree, archive and so kill — an
agent that was working again. Which slots were "finished" came from the
compare table, and a `marker` finish is never cleared. It now runs `settle`
after the merge and re-reads compare: a slot is skipped only if it is still
finished and its tip and dirty count are what you saw. Anything else is left
running and named.
- Merge winner checked base drift against the base it had just re-read, so a
base that moved after you confirmed merged without the drift warning. It now
checks against the base current at confirm, and refuses if it moved.
- Archive, discard and abort no longer act on a slot whose `validate` hook is
still running. The hook runs with the repo lock released and could write
ignored files after the cleanup inventory was approved, or after the
discard snapshot.
- A timeout that is not whole seconds (`HERDR_SWARM_CLONE_TIMEOUT=2m`, `0900`,
`08`) was found only after the timed command had started: the arithmetic
error ended the caller, leaving `cp` (or the validate hook) running with no
deadline and, for validate, its per-slot lock dropped. Timeouts are now
parsed as decimal before anything starts, and fan-out refuses a bad clone or
setup timeout before creating any worktree.
- `validate` exited 30 (base drift) when the *slot* moved while its hook ran.
It now exits 36 (refused); 30 still means only that the base moved.
- Compare could rank a slot `passed` on a validation record that the strict
candidate handoff rejects (one without the hook's own `validate` check):
the two had separate copies of the identity rule. Both now use
`isSwarmValidationRecord` from `pr-handoff.mjs`.
- A `validate` that times out in the harvest pane now names
`HERDR_SWARM_VALIDATE_TIMEOUT`, the setting that sets its budget, not the
generic step timeout.

## [0.5.0] — 2026-10-04

The "pick the winner" release: Swarm now checks each slot, notices when a
Expand Down Expand Up @@ -70,24 +106,32 @@ merging. Exercised live end to end on Herdr 0.8.2 (see `docs/readiness.md`).
gets `HERDR_SWARM_RUN_ID`, `_SLOT`, `_PORT_BASE`, `_PORT_SPAN`: in
`setup.sh` always, in the agent via `pane split --env` on Herdr 0.7.5+, and
as a port-range line in the task file. Ports default to 10 per slot from
4100 and are recorded on the manifest row.
4100 and are recorded on the manifest row. Settings:
`HERDR_SWARM_PORT_START` / `_SPAN`, `HERDR_SWARM_CLONE_MODE=copy` (full copy
where copy-on-write is unavailable), `HERDR_SWARM_CLONE_TIMEOUT` (default
120s), and a `clone-paths` file in the plugin config dir. Clone temps
(`.swarm-clone.*`) are added to `info/exclude`.
- Compare view (`w` in the harvest pane; scriptable as the read-only
`compare` verb). It ranks running slots by validation of their current
tip, then commits, then finished, never by diff size, and shows commits,
files, +/-, dirty count, and the files each pair of slots both changed.
`d` shows the full diff between two slots' tips in git's pager. `m` merges
a chosen winner through the normal merge, and only if it lands, skips and
archives the other running slots (branches kept; anything that won't
archive cleanly is left and named).
- Finish detection: the status pane runs a new `settle` verb every 10s. It
archive cleanly is left and named). The merge verb takes an optional third
argument, `merge <slot> <base-sha> [expected-slot-tip]`: it refuses unless
the slot is still at that SHA, then merges the SHA, not the branch name.
- Finish detection: the status pane runs a new `settle` verb every 10s
(`HERDR_SWARM_SETTLE_INTERVAL_MS`). It
records a slot as finished when its agent creates `.swarm-done` (now asked
for in the task file's standing instructions, and excluded from `git status`
and from ignored-file cleanup approval like `.swarm-task.md`). It also does
so when the slot pane is back at a bare shell (via Herdr `pane process-info`).
Finished slots show `finished`, flip their plugin-reported state to `idle`,
and get one Herdr notification per run when all are done. Opt-in
auto-validate (`HERDR_SWARM_AUTO_VALIDATE=1` or a config-dir
`auto-validate` file) runs `validate` on each settled slot.
`auto-validate` file) runs `validate` on each finished slot. An empty
`.swarm-done` is exempt from that approval; one with content is not.
- `validate <slot>` harvest verb (`v`, then slot in the pane) runs the
operator's `validate.sh` hook against a slot's clean HEAD and records a
SHA-bound checks file. The strict candidate handoff now produces its own
Expand Down
36 changes: 36 additions & 0 deletions CONCEPTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,22 @@ The Manifest is what makes a crashed Run recoverable rather than orphaned. It
is archived rather than deleted when a Run ends, because it remains the record
that lets cleanup resolve what a Run left behind.

### Finished
A running Slot whose agent has stopped working on its task, recorded as
`finished: {at, reason}` on the Slot while its status stays `running`. It is
evidence for the operator, not a lifecycle state: it never merges, skips or
archives anything by itself. Two reasons, and they differ in strength:
`marker` (the agent created the Finish marker) is final; `exited` (the pane is
back at its bare shell) is reversible, and is cleared if the agent is seen
busy again. Anything that acts on Finished re-reads it at the moment it acts,
because the operator may have given the agent more work since.

### Finish marker
The empty `.swarm-done` file an agent creates in its worktree root to declare
it is done, as the task file asks. Excluded from git status like the task
file. Only an empty marker is exempt from the ignored-file approval at archive;
one with content is agent output like any other file.

### Preset
A named agent configuration a Slot runs. Presets are what make fan-out
agent-agnostic: any command can be a Slot's agent, not a fixed list.
Expand Down Expand Up @@ -99,6 +115,20 @@ The base branch moving between the moment a merge was previewed and the moment
it would land. Detected drift sends the Slot back to re-preview rather than
merging against a base the user never reviewed.

### Compared tip
The exact commit a Slot's branch pointed at when the compare view read it.
Picking a winner pins this SHA: the merge refuses if the branch has moved and
merges the SHA itself, not the branch name, so what lands is what was
compared.

### Validation result
The checks file the `validate` verb writes for one Slot, bound to the HEAD it
ran against. Because it sits at a predictable path, a file there counts only if
it proves it is Swarm's own record for exactly this Run and Slot, with the
hook's own `validate` check present. One rule decides that, shared by the
compare ranking and the strict candidate handoff. A result for an older commit
is stale, never a pass.

### Snapshot
A copy of a Slot's uncommitted work — tracked and untracked — written to a
plugin-owned ref before any discard. Discard is the only operation that
Expand Down Expand Up @@ -127,3 +157,9 @@ since the leftovers may hold committed work that never landed.
- "Archive" applies to two different things and both are intentional: a Slot is
archived (worktree removed, branch kept), and a Manifest is archived (kept as
a recovery record rather than deleted). Neither means "delete."
- "Settle" has an older and a newer meaning. In the Slot lifecycle, a settled
Slot is one whose agent has stopped (merged, skipped and failed come after
it). The `settle` verb and its `settled<TAB>slot<TAB>reason` output line are
Finish detection: they record Finished on a Slot that stays `running`. The
verb name is a scripting contract and is kept; in prose, say Finished for
the verb's result.
54 changes: 40 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -453,7 +453,8 @@ Where copy-on-write isn't available, behaviour depends on the platform:
- **macOS:** `cp -c` silently falls back to a full copy when it can't clone,
for example on a non-APFS volume or with worktrees on another volume.

`HERDR_SWARM_CLONE_TIMEOUT` (default 120 s) bounds each clone. A timeout stops
`HERDR_SWARM_CLONE_TIMEOUT` (default 120 s, whole seconds — `2m` refuses the
fan-out before any worktree is created) bounds each clone. A timeout stops
`cp` and everything it started, and removes the partial copy. A clone failure
never fails the slot. Cloning happens before `setup.sh`, so the hook can build
on it.
Expand Down Expand Up @@ -487,7 +488,23 @@ Each slot gets `HERDR_SWARM_RUN_ID`, `HERDR_SWARM_SLOT`,
Ports start at `HERDR_SWARM_PORT_START` (default 4100), with
`HERDR_SWARM_PORT_SPAN` (default 10) per slot: slot 1 gets 4100–4109, slot 2
gets 4110–4119. This is a convention the agents are told, not an OS
reservation. A range that doesn't fit in 1024–65535 refuses the fan-out. The same applies to repo hooks during harvest: a hook that
reservation. A range that doesn't fit in 1024–65535 refuses the fan-out.

Ranges are not coordinated across repos: two repos swarming at the same time
both start at 4100. Give each repo its own range with git config, which a
fan-out pane reads even though it never inherits your shell's environment:

```sh
git config swarm.portStart 5100 # in the second repo
git config swarm.portSpan 10 # optional
```

An exported `HERDR_SWARM_PORT_START`/`_SPAN` still overrides git config.

### Fresh-worktree hooks

Fresh worktrees lack gitignored files that the clone list doesn't cover
(`.env`, build output). The same applies to repo hooks during harvest: a hook that
shells into `node_modules/.bin` fails in the plugin-owned merge worktree —
`HERDR_SWARM_HARVEST_WT_NO_HOOKS=1` disables hooks in that worktree *only*
(never in your tree). To automate worktree prep, drop a `setup.sh` in the
Expand Down Expand Up @@ -627,8 +644,9 @@ In the harvest pane, `w` opens the compare view, which replaces the slot list
with one ranked row per slot. The same data is scriptable as
`bash scripts/harvest-step.sh compare`, which emits one
`compare_slot<TAB>{json}` per slot and one
`compare_overlap<TAB>a<TAB>b<TAB>n<TAB>[files]` per candidate pair. It is
read-only.
`compare_overlap<TAB>a<TAB>b<TAB>n<TAB>[files]` per candidate pair, where
`n` is `-1` when either slot's diff could not be read (overlap unknown, not
zero) and `[files]` lists at most 10 of the shared files. It is read-only.

| Column | Source |
| --- | --- |
Expand All @@ -640,8 +658,9 @@ read-only.

**Ranking** is an order, not a score: checks (passed > not run > stale >
failed), then having commits, then finished, then slot number. Diff size is
shown but never ranked, because a smaller change isn't a better one. Settled
slots (merged, skipped) are listed without a rank.
shown but never ranked, because a smaller change isn't a better one. Slots
that are no longer running (merged, skipped, failed) are listed without a
rank.

In the compare view:

Expand All @@ -650,21 +669,25 @@ In the compare view:
merge-tree shell.
- `m`, then a slot digit, then `y`: **merge the winner and skip the rest**.
The winner is merged **at the exact tip the table showed**: if its agent
committed since, the merge refuses and you re-open compare. Otherwise it
goes through the normal merge: re-previewed and drift-checked. If the merge
would now land in your checked-out branch and the confirm didn't say so,
you are asked again. **Only if that merge lands** are the other running
slots whose agent has *finished* marked skipped and archived. Slots still
committed since, the merge refuses and you re-open compare. If the base
moved after you confirmed, it refuses too. Otherwise it goes through the
normal merge: re-previewed and drift-checked. If the merge would now land
in your checked-out branch and the confirm didn't say so, you are asked
again. **Only if that merge lands** are the other running slots whose agent
has *finished* marked skipped and archived. "Finished" is re-checked at
that moment, not taken from the table: finish detection runs again, and a
slot whose tip or uncommitted changes differ from what the table showed is
treated as working (you may have given its agent a follow-up). Slots still
working are left running and named. Branches are always kept. A worktree
that won't archive cleanly (dirty, ignored files, a working agent) is left
in place and named in the banner. If that's the winner's, its approval
prompt stays open. A winner that isn't clean, or a merge that conflicts,
skips nothing.
- `r` refreshes; `b`/Esc goes back.

Scripted, the same pin is `harvest-step.sh merge <slot> <base-sha>
<slot-tip>`: it refuses unless the slot is still at `<slot-tip>`, then merges
that SHA rather than the branch name.
- `r` refreshes; `b`/Esc goes back.

## Finish detection

Expand Down Expand Up @@ -696,7 +719,7 @@ yourself. A marker with content in it is treated as agent data, so archive
needs the usual ignored-file approval to remove it.

**Auto-validate (opt-in).** With `HERDR_SWARM_AUTO_VALIDATE=1`, or an empty
`auto-validate` file in the plugin config dir, a slot that settles also starts
`auto-validate` file in the plugin config dir, a slot that finishes also starts
a detached `validate` for that slot, if `validate.sh` exists (log:
`auto-validate-<run>-s<slot>.log` in the state dir). Use the file when the
status pane is opened through the plugin action, because action-launched panes
Expand Down Expand Up @@ -725,7 +748,7 @@ npm test >/dev/null 2>&1 && echo "unit-tests passed" >>"$HERDR_SWARM_CHECKS_FILE
reserved line records the whole result as failed rather than trusting part
of it.
- A dirty slot refuses before the hook runs. If the slot is dirty or its HEAD
moved when the hook finishes, nothing is recorded (exit 36 or 30). A hook
moved when the hook finishes, nothing is recorded (exit 36). A hook
that leaves build output behind needs that output gitignored.
- Passing and failing hooks both exit 0 with a
`validated<TAB>slot<TAB>sha<TAB>passed|failed<TAB>path` record.
Expand All @@ -738,6 +761,9 @@ npm test >/dev/null 2>&1 && echo "unit-tests passed" >>"$HERDR_SWARM_CHECKS_FILE
sent TERM and then KILL. A dev server or test worker it started therefore
can't outlive it. A timeout is recorded as failed. A process that calls
`setsid` to leave its group is beyond reach.
- While a slot's hook runs, archive, discard and abort refuse to touch that
slot's worktree (abort keeps it and says so): the hook may still be writing
files there that no cleanup approval or snapshot has seen.
- Only one validate runs per slot at a time; a second is refused (exit 36).
- `candidate-status` and `publish-candidate-pr` use this result when
`HERDR_SWARM_CANDIDATE_VALIDATION_FILE` is unset. In that case the file must
Expand Down
Loading
Loading