Skip to content

feat(cli): dune ps finds the CLI and task wrappers and says "not listening"; new dune stop - #37

Merged
zumbrunn merged 8 commits into
mainfrom
feat/ps-shim-notlistening-stop
Oct 27, 2026
Merged

zumbrunn merged 8 commits into
mainfrom
feat/ps-shim-notlistening-stop

Conversation

@zumbrunn

@zumbrunn zumbrunn commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Fixes #46

Why

Found while cleaning up a machine running ~45 dune processes: dune ps listed 19 of them as "port unknown — is lsof installed?" and missed the one server that was actually serving (started through the dune shim as cli.ts dev).

dune ps (read-only)

  • Matching. Besides the generated main.ts dev|serve, it now matches the dune CLI run directly (cli.ts dev|serve via the global shim or a checkout, jsr:@dune/core@x/cli serve) and deno task dev|serve wrappers. A bare cli.ts or deno task could be any project, so those are weak matches that count only when a config/site.yaml is found via --root, PWD or the live cwd.
  • One instance per process tree. A server's wrapper, re-exec child and esbuild workers are grouped under the topmost matching process from a single ps -eo pid,ppid,uid,lstart,... snapshot (drops pgrep). Only the current user's processes are considered. --root is used to find the site; the command line is shown.
  • "Not listening" instead of a misleading hint: listening / not-listening (lsof ran, found nothing) / unknown (lsof can't be spawned).
  • Confirmed vs. unconfirmed. An instance is confirmed when its own command line points at a dune site: the directory of the main.ts it runs, its --root, or (for the dune CLI proper) its working directory, each holding a config/site.yaml. Merely running from inside a site directory confirms nothing, since any tool started from a site repo inherits that. A command line that only looks like dune (any Deno project's main.ts serve, a path containing "dune") is still listed but marked unconfirmed. (feat(cli): dune ps probes health and reads the instance registry #53 adds the instance registry as a second source of confirmation.)

dune stop (new)

Stopping used to be flags on ps; it is its own command now, so ps never prompts or signals anything.

dune stop <pid>        the instance that process belongs to (top process or any child)
dune stop --port <n>   the instance listening on that port
dune stop --idle       confirmed instances running but not listening for >= 2 minutes
dune stop --all        every confirmed instance dune ps lists
        [--force]      SIGKILL what ignores SIGTERM after 10 s
        [--yes, -y]    skip the confirmation
  • A target is required. A bare dune stop prints usage and exits 2; two targets together is also a usage error.
  • Only what it can verify. --all and --idle act on confirmed instances only; an unconfirmed one must be named by pid (with a warning), and the prompt shows every instance's full command line. Only your own processes are touched (our uid is read from our own row in the process table, so no --allow-sys). A signal the OS refuses is reported as "not permitted (owned by another user)"; one Deno's permissions refuse as "re-run with --allow-run"; a missing --allow-run is reported as such rather than as "no instances".
  • No stale-pid kills. Each pid's start time is captured when the instance is found and re-checked right before every SIGTERM and SIGKILL, so a pid that exited and was reused while the prompt was open is never signalled (it is reported as "not signalled").
  • Not servers that are still starting. --idle leaves out a not-listening instance younger than 2 minutes (indexing a big site takes a while), and says so.
  • SIGTERM by default; a process that ignores it is reported with a hint, and only --force escalates to SIGKILL (which can orphan a wrapper's child). It stops the whole process tree, including any other tool a deno task wrapper started; the usage text says so.
  • Asks first; with no terminal and no --yes it refuses. Never signals itself or its ancestors.
  • Exit codes: 0 done (or nothing to do for --idle/--all), 1 a named pid/port matches nothing or something would not stop or was refused, 2 usage error.

Code layout

src/cli/instances.ts (discovery, shared), ps.ts (listing), stop.ts (selection, confirmation, signalling). Tests split the same way, with shared fixtures in tests/cli/ps_helpers.ts.

Verification

  • deno lint src/, deno check; deno test -A --unstable-kv tests/cli: 404 passed (content_delete_test is a pre-existing flake on main, being fixed separately)
  • Real processes: discovery by every launch shape; confirmed vs. unconfirmed (an unrelated main.ts started from inside a site directory is not confirmed; the site's own main.ts started from elsewhere is); dune ps under --allow-run only (no --allow-sys); a kill refused by Deno's permissions reported as needing --allow-run; start times recorded; whole tree stopped; a SIGTERM-ignoring process is left alone without --force and SIGKILLed with it; a mismatching or missing recorded start time means the process is never signalled, even with --force; a refused signal (pid 1, skipped as root) is reported as denied; never signals itself.
  • Pure tables for target selection (usage errors, pid by child, --port, --idle with unconfirmed/young/unknown-state instances, --all), and the command flow against injected fakes so no flow test can stop a real instance.
  • Real CLI against a live fake-dune server and idle ones: ps lists both; bare stop prints usage; stop --idle with no terminal refuses; stop by pid/port stops only that tree; stop 424242 says no such instance.

Known limits

Docs: duneorg/dune-docs#12.

🤖 Generated with Claude Code

…stening", adds --stop

Found while cleaning up a machine running ~45 dune processes: `dune ps` listed
19 of them as "port unknown — is lsof installed?" and missed the one server
that was actually serving.

- Matching: besides the generated `main.ts dev|serve`, match the dune CLI run
  directly (`cli.ts dev|serve` via the global shim or a checkout,
  `jsr:@dune/core@x/cli serve`) and `deno task dev|serve` wrappers. A bare
  `cli.ts` / `deno task` could be any project, so those are "weak" matches
  that count only when a config/site.yaml is found via --root, PWD or live cwd.
- One instance per process tree: wrapper, re-exec child and esbuild workers
  are grouped under the topmost matching process, from one `ps -eo
  pid,ppid,...` snapshot (drops the pgrep dependency). `--root` on the
  command line is used to find the site, resolved against the process's cwd
  when relative.
- Wording: `portStatus` distinguishes "listening", "not-listening" (lsof ran,
  found nothing: a hung or leftover process) and "unknown" (lsof can't be
  spawned). lsof exits 1 on no matches, so availability is checked by
  spawning it, not by its exit code.
- `--stop`: SIGTERM to every pid in every listed instance tree, waits, reports
  survivors (exit code 1). Asks first; --yes skips the prompt; without a
  terminal and without --yes it refuses. Never signals this process or its
  ancestors, whatever the caller passes. It stops everything it lists, so it
  is documented as such.

Tests: pure matching/--root parsing tables; real processes for the CLI-style
invocation (absolute and relative --root), a not-listening process, a
`deno task` tree reported once, and stopInstances really stopping a tree and
refusing to signal itself; the command flow runs against injected fakes so no
test can ever stop a real instance. Mutation-checked: disabling tree grouping
and the weak-match filter each fail exactly their test. Verified on the real
machine: the previously-missed iofc dev server is now listed with title, root,
port and children, and `ps --stop` without --yes/TTY refuses and kills nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…alone

`dune ps --stop` is all-or-nothing, and on a shared machine that includes
servers you want: while building --stop, a live dev server sat next to ~45
hung leftovers. --idle narrows both the listing and --stop to instances that
are running but "not listening" (hung or leftover), so
`dune ps --stop --idle` clears the debris and leaves anything serving alone.
`dune ps --idle` alone is the preview.

An instance whose port state is unknown (no lsof) is never assumed idle: with
nothing provably idle, --idle says so and --stop does nothing (exit 1). When
nothing is idle it says "No idle instances" rather than prompting. The
confirmation names the idle count and the process count.

Tests (fakes only, so nothing real can be stopped): the listing shows only
idle instances and counts the live ones left out; --stop --idle hands the
stopper only idle instances and not even a live one's child pids; it still
refuses without --yes and a terminal; the nothing-idle and lsof-unavailable
cases. Disabling the filter fails 4 of them, including the safety one. Also
run for real on the machine: with a live and an idle fixture next to a live
server from another session, `ps --stop --idle --yes` stopped only the idle
fixture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zumbrunn

Copy link
Copy Markdown
Member Author

Added the selective form I flagged as a follow-up: dune ps --idle (commit 3d6c156).

  • dune ps --idle lists only instances that are running but not listening; dune ps --stop --idle stops just those and leaves live servers alone. Plain --stop is unchanged and still stops everything listed.
  • An instance whose port state is unknown (no lsof) is never assumed idle: with nothing provably idle, --idle says so and --stop does nothing (exit 1).
  • The confirmation names the idle count and process count; with nothing idle it says "No idle instances" instead of prompting.
  • Tests are fakes-only (no test can stop a real instance): the listing, --stop --idle handing the stopper only idle instances (not even a live one's child pids), the refusal without --yes/TTY, the nothing-idle and lsof-unavailable cases. Disabling the filter fails 4 of them, including the safety one.
  • Run for real: with a live and an idle fixture next to a live server from another session, ps --stop --idle --yes stopped only the idle fixture; the other two stayed up.
  • 1900 tests pass; deno task check clean. CHANGELOG, README and the docs PR (docs(cli): dune ps finds the dune CLI and task wrappers, reports "not listening"; new dune stop dune-docs#12) are updated.

A process can ignore SIGTERM indefinitely. A `dune serve` closed its listener
on the first SIGTERM and then stayed alive until it was sent a second one, so
"SIGTERM, then report survivors" left exactly the idle processes this command
exists to clear, and made a plain `--stop` on a live serve end in "still
running".

stopInstances() now sends SIGTERM, waits up to a 10 s grace period (for real
servers draining in-flight requests), then SIGKILLs whatever is still alive and
waits briefly for it to go. StopResult gains `killed` (the subset of `stopped`
that needed the SIGKILL) so the escalation is never silent; `survivors` now
means "alive even after SIGKILL" and is the only thing that makes the exit code
non-zero. Still never signals this process or its ancestors.

Tests: a real process that swallows SIGTERM is escalated (killed, stopped, no
survivors, died by SIGKILL), a cooperative process is not SIGKILLed, and the
command's output names the killed pids and says nothing about SIGKILL when
there were none. Removing the escalation fails the stubborn-process test.
Also reproduced the original situation for real: a serve that got one SIGTERM
sat idle, `ps --idle` listed it, and `ps --stop --idle --yes` cleared it while
a live server from another session stayed up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zumbrunn

Copy link
Copy Markdown
Member Author

Added the SIGKILL escalation (commit follows 3d6c156).

stopInstances() now sends SIGTERM, waits a 10 s grace period (for real servers draining in-flight requests), then SIGKILLs whatever is still alive. The result gains killed (the pids that needed it), so the escalation is never silent; survivors now means "alive even after SIGKILL" and is the only thing that makes the exit code non-zero. It still never signals itself or its ancestors.

Why: a dune serve closed its listener on the first SIGTERM and then stayed alive until sent a second one (root-caused to a ref'd plugin-admin timer, see duneorg/plugin-admin#8 and #38), so plain SIGTERM left exactly the idle processes this command exists to clear.

  • A real process that swallows SIGTERM is escalated (died by SIGKILL, reported as killed); a cooperative one is not SIGKILLed; removing the escalation fails the stubborn-process test.
  • Reproduced the original situation for real: a serve that got one SIGTERM sat idle, dune ps --idle listed it, and dune ps --stop --idle --yes cleared it while the live iofc server stayed up.
  • 1903 tests pass; deno task check clean. CHANGELOG and the docs PR (docs(cli): dune ps finds the dune CLI and task wrappers, reports "not listening"; new dune stop dune-docs#12) are updated.

`dune ps` is read-only again. Stopping is its own command with a required
target (a pid, --port, --idle or --all), SIGTERM by default and SIGKILL only
with --force, so stopping every server on a machine has to be asked for by
name and a kill that can orphan a wrapper's child is opt-in.

Code is split along the same line: instances.ts (discovery, shared), ps.ts
(listing), stop.ts (selection, confirmation, signalling). Tests follow, with the
shared fixtures in ps_helpers.ts.

Refs #46

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@zumbrunn zumbrunn changed the title feat(cli): dune ps finds the dune CLI and task wrappers, says "not listening", adds --stop feat(cli): dune ps finds the CLI and task wrappers and says "not listening"; new dune stop Oct 26, 2026
zumbrunn and others added 2 commits October 26, 2026 23:01
Review of #37:
- --all/--idle act only on confirmed instances (a config/site.yaml at the site
  root). A command line that merely looks like dune (any Deno project's
  `main.ts serve`, a path containing "dune") is still listed but has to be named
  by pid, with a warning.
- Only the current user's processes are listed and signalled; a refused signal
  (EPERM) is reported as "not permitted", not as "ignored SIGTERM".
- Every pid's start time is captured when the instance is found and re-checked
  right before each SIGTERM and SIGKILL, so a pid that exited and was reused
  while the confirmation prompt was open is never signalled.
- --idle leaves out a not-listening instance younger than 2 minutes: it may
  still be starting (indexing a big site).
- Help and docs say that the whole process tree is stopped.

Refs #46

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Second-pass review of #37:
- "confirmed" now needs evidence from the command line itself (the directory of
  the main.ts it runs, --root, or the dune CLI's cwd), not merely that the
  process runs from inside a site directory, which any tool started from a site
  repo inherits. stop prints each instance's full command line before asking.
- Our own uid is read from our own row in the process table instead of
  Deno.uid(), which needs --allow-sys; without it `dune ps` used to throw.
- A missing --allow-run now says so (instead of "no instances" or "already
  gone"), and Deno.errors.NotCapable on a kill is reported separately from
  EPERM (another user's process) and from a pid that already exited.

Refs #46

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
zumbrunn added a commit that referenced this pull request Oct 27, 2026
…ealth

Brings the stricter 'confirmed' (registry records also confirm) and the
permission handling from #37.
@zumbrunn
zumbrunn merged commit c3f0bcd into main Oct 27, 2026
7 checks passed
zumbrunn added a commit that referenced this pull request Oct 27, 2026
* feat(cli): dune ps finds the dune CLI and task wrappers, says "not listening", adds --stop

Found while cleaning up a machine running ~45 dune processes: `dune ps` listed
19 of them as "port unknown — is lsof installed?" and missed the one server
that was actually serving.

- Matching: besides the generated `main.ts dev|serve`, match the dune CLI run
  directly (`cli.ts dev|serve` via the global shim or a checkout,
  `jsr:@dune/core@x/cli serve`) and `deno task dev|serve` wrappers. A bare
  `cli.ts` / `deno task` could be any project, so those are "weak" matches
  that count only when a config/site.yaml is found via --root, PWD or live cwd.
- One instance per process tree: wrapper, re-exec child and esbuild workers
  are grouped under the topmost matching process, from one `ps -eo
  pid,ppid,...` snapshot (drops the pgrep dependency). `--root` on the
  command line is used to find the site, resolved against the process's cwd
  when relative.
- Wording: `portStatus` distinguishes "listening", "not-listening" (lsof ran,
  found nothing: a hung or leftover process) and "unknown" (lsof can't be
  spawned). lsof exits 1 on no matches, so availability is checked by
  spawning it, not by its exit code.
- `--stop`: SIGTERM to every pid in every listed instance tree, waits, reports
  survivors (exit code 1). Asks first; --yes skips the prompt; without a
  terminal and without --yes it refuses. Never signals this process or its
  ancestors, whatever the caller passes. It stops everything it lists, so it
  is documented as such.

Tests: pure matching/--root parsing tables; real processes for the CLI-style
invocation (absolute and relative --root), a not-listening process, a
`deno task` tree reported once, and stopInstances really stopping a tree and
refusing to signal itself; the command flow runs against injected fakes so no
test can ever stop a real instance. Mutation-checked: disabling tree grouping
and the weak-match filter each fail exactly their test. Verified on the real
machine: the previously-missed iofc dev server is now listed with title, root,
port and children, and `ps --stop` without --yes/TTY refuses and kills nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(cli): dune ps --idle, a selective stop that leaves live servers alone

`dune ps --stop` is all-or-nothing, and on a shared machine that includes
servers you want: while building --stop, a live dev server sat next to ~45
hung leftovers. --idle narrows both the listing and --stop to instances that
are running but "not listening" (hung or leftover), so
`dune ps --stop --idle` clears the debris and leaves anything serving alone.
`dune ps --idle` alone is the preview.

An instance whose port state is unknown (no lsof) is never assumed idle: with
nothing provably idle, --idle says so and --stop does nothing (exit 1). When
nothing is idle it says "No idle instances" rather than prompting. The
confirmation names the idle count and the process count.

Tests (fakes only, so nothing real can be stopped): the listing shows only
idle instances and counts the live ones left out; --stop --idle hands the
stopper only idle instances and not even a live one's child pids; it still
refuses without --yes and a terminal; the nothing-idle and lsof-unavailable
cases. Disabling the filter fails 4 of them, including the safety one. Also
run for real on the machine: with a live and an idle fixture next to a live
server from another session, `ps --stop --idle --yes` stopped only the idle
fixture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): dune ps --stop escalates to SIGKILL after a grace period

A process can ignore SIGTERM indefinitely. A `dune serve` closed its listener
on the first SIGTERM and then stayed alive until it was sent a second one, so
"SIGTERM, then report survivors" left exactly the idle processes this command
exists to clear, and made a plain `--stop` on a live serve end in "still
running".

stopInstances() now sends SIGTERM, waits up to a 10 s grace period (for real
servers draining in-flight requests), then SIGKILLs whatever is still alive and
waits briefly for it to go. StopResult gains `killed` (the subset of `stopped`
that needed the SIGKILL) so the escalation is never silent; `survivors` now
means "alive even after SIGKILL" and is the only thing that makes the exit code
non-zero. Still never signals this process or its ancestors.

Tests: a real process that swallows SIGTERM is escalated (killed, stopped, no
survivors, died by SIGKILL), a cooperative process is not SIGKILLed, and the
command's output names the killed pids and says nothing about SIGKILL when
there were none. Removing the escalation fails the stubborn-process test.
Also reproduced the original situation for real: a serve that got one SIGTERM
sat idle, `ps --idle` listed it, and `ps --stop --idle --yes` cleared it while
a live server from another session stayed up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(runtime): event-loop watchdog for dev and serve

One synchronous loop blocks the only thread serving requests, so every URL
stops answering at once with nothing in the logs. A Worker now expects a
heartbeat from the main thread; after 15s (DUNE_WATCHDOG_TIMEOUT_MS) of
silence it writes a report to <site>/.dune/hangs/ (in-flight requests, plus a
native stack sample on macOS) and SIGKILLs the process so a supervisor can
restart it. DUNE_WATCHDOG=warn reports only, =off disables.

SIGKILL rather than Deno.exit(): an exit requested from a Worker only ends the
Worker, the spinning main thread carries on. A Worker also can't be unref'd, so
serve's graceful-shutdown paths stop it explicitly.

The main thread additionally logs a warning when its own timer ran over a
second late.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* feat(runtime): instance registry written by dev and serve

Each running instance records itself in ~/.dune/instances/<pid>.json (pid,
process start time, mode, root, title, port, version) and removes the record on
a graceful exit. Readers verify a record against the live process (alive and
same start time, since pids are reused) and prune stale ones. Groundwork for
`dune ps` to list instances from what each one says about itself instead of
parsing command lines and lsof; nothing reads it yet.

Refs #42

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* refactor(cli): stopping moves out of `dune ps` into `dune stop`

`dune ps` is read-only again. Stopping is its own command with a required
target (a pid, --port, --idle or --all), SIGTERM by default and SIGKILL only
with --force, so stopping every server on a machine has to be asked for by
name and a kill that can orphan a wrapper's child is opt-in.

Code is split along the same line: instances.ts (discovery, shared), ps.ts
(listing), stop.ts (selection, confirmation, signalling). Tests follow, with the
shared fixtures in ps_helpers.ts.

Refs #46

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* feat(cli): dune ps probes health and reads the instance registry

Every listening instance gets a /health/live request (3 s timeout, any status
counts); one that accepts the connection but never answers is shown as NOT
RESPONDING, the signature of a blocked event loop that lsof can't see. Instances
that registered themselves are listed even when their command line doesn't look
like dune, and show their version. `dune stop --idle` also takes instances that
are listening but not responding.

Refs #42

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(runtime): watchdog no longer kills what is merely paused or legitimately busy

Review of #51 and a test of my own turned up false-kill cases and a kill that
could fail to happen:

- A process suspended and resumed (laptop sleep, Ctrl-Z/SIGSTOP) was killed on
  resume: confirmed by SIGSTOPping a watched process for 4s. The stall clock is
  now monotonic (performance.now()), and a gap in the Worker's own ticks resets
  it instead of firing.
- Startup gets its own, much longer limit (2 min) until the server calls
  ready(), so a big site's initial indexing can't put a supervised process into
  a kill/restart loop.
- watchdogProgress() (rate-limited heartbeat for long synchronous loops) and
  watchdogGrace() (capped, auto-expiring, nestable busy window), exported as
  @dune/core/watchdog.
- Under --inspect/--inspect-brk the default becomes warn.
- The kill no longer depends on the report: writing it is raced against a
  timeout, and a failed Deno.kill is logged instead of silencing the watchdog.
- Reports are pruned to the newest 20, and path segments over 24 characters are
  masked before they are written.

Refs #42

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(cli): dune stop only acts on what it can verify

Review of #37:
- --all/--idle act only on confirmed instances (a config/site.yaml at the site
  root). A command line that merely looks like dune (any Deno project's
  `main.ts serve`, a path containing "dune") is still listed but has to be named
  by pid, with a warning.
- Only the current user's processes are listed and signalled; a refused signal
  (EPERM) is reported as "not permitted", not as "ignored SIGTERM".
- Every pid's start time is captured when the instance is found and re-checked
  right before each SIGTERM and SIGKILL, so a pid that exited and was reused
  while the confirmation prompt was open is never signalled.
- --idle leaves out a not-listening instance younger than 2 minutes: it may
  still be starting (indexing a big site).
- Help and docs say that the whole process tree is stopped.

Refs #46

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix(cli): stricter "confirmed", no --allow-sys, honest permission errors

Second-pass review of #37:
- "confirmed" now needs evidence from the command line itself (the directory of
  the main.ts it runs, --root, or the dune CLI's cwd), not merely that the
  process runs from inside a site directory, which any tool started from a site
  repo inherits. stop prints each instance's full command line before asking.
- Our own uid is read from our own row in the process table instead of
  Deno.uid(), which needs --allow-sys; without it `dune ps` used to throw.
- A missing --allow-run now says so (instead of "no instances" or "already
  gone"), and Deno.errors.NotCapable on a kill is reported separately from
  EPERM (another user's process) and from a pid that already exited.

Refs #46

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* chore: drop test-run drift from deno.lock

* docs(changelog): #53 entries on top of main's [Unreleased]

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dune ps misses the CLI shim and task wrappers; no way to stop what it lists

1 participant