Terminal lease hardening

CHANGELOG.md

@@ -270,6 +270,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- Commands no longer fail with "Error reclaiming terminal control: ... no such process"
+  when several mshell processes share one terminal, as under parallel build runners
+  (`redo`, `make -j`) or when a script is backgrounded.
+  mshell now transfers terminal control only when it is itself the terminal's current
+  foreground process group (the same gate bash and fish use), and a hand-back to a
+  previous foreground group that has since exited falls back to mshell's own group
+  instead of failing the command.
+  A reclaim problem is now at most a warning on stderr; the command's own exit code always stands.
+- A fast pipeline whose processes finished before mshell could transfer terminal
+  control is no longer killed and reported as failed; the transfer is skipped,
+  since the work is already done.
+  Restoring terminal modes is now also protected from `SIGTTOU`,
+  which could previously stop the shell mid-cleanup when another process group owned the terminal.
+  A failure while closing the retained pipeline terminal handle is likewise now a warning,
+  never a failure of a pipeline whose commands succeeded.
 - Attempted to formally improve the semantics of job control and terminal control on both Linux and Windows.
   Should fix potential bugs when running TUI programs from within mshell scripts.
 - On Windows, a command name containing a forward slash (e.g. `./script.msh`)

ai/terminal-control/ASSUMPTIONS.md

@@ -17,6 +17,12 @@ system versions change.
 - A shell foregrounds a job with `tcsetpgrp()`, observes stopped children with
   `waitpid(..., WUNTRACED)`, reclaims the terminal, and foregrounds a stopped job
   before sending `SIGCONT`.
+- A process-group ID passed to `tcsetpgrp()` is a snapshot, not a stable handle:
+  the group can cease to exist at any time.  POSIX documents `EPERM` for a group
+  ID with no member in the caller's session; Linux's `TIOCSPGRP` returns `ESRCH`
+  for a nonexistent group (observed in production, 2026-08-13, when restoring a
+  sibling shell's exited child group).  Terminal hand-back must therefore treat
+  a dead target as an expected outcome, not an exceptional command failure.
 
 Sources:
 

ai/terminal-control/ERRNO_AUDIT.md

@@ -0,0 +1,143 @@
+# Errno audit of the terminal/job-control syscall surface
+
+Date: 2026-08-13.
+Sources: Linux man-pages on the development machine (man-pages 6.x), POSIX
+Issue 8, observed production behavior, and kernel behavior where the man pages
+are incomplete.  Method: enumerate every syscall the controller makes, list
+every documented (and observed-undocumented) errno, and trace each to an
+explicit policy in code and, where applicable, a transition in the TLA+
+models.  A row with no policy is a finding.
+
+Verdicts: **OK** (explicit, correct policy), **FIXED** (gap found by this
+audit, fixed 2026-08-13), **REC** (works, but a better policy is recommended
+below), **N/A** (cannot occur at this call site, with reasoning).
+
+## POSIX backend
+
+### tcsetpgrp — `TIOCSPGRP` (`SetForegroundProcessGroup`, `RestoreForegroundProcessGroup`)
+
+| Errno | Trigger | Policy | Verdict |
+| --- | --- | --- | --- |
+| `ESRCH` (undocumented) | Target pgid fully reaped.  Linux's `TIOCSPGRP` returns this; the tcsetpgrp man page does not list it.  Observed in production 2026-08-13. | Acquire: skip the handoff (nothing left to foreground).  Release: fall back to own group, warn only. | FIXED / OK |
+| `EPERM` | Target pgid valid but not in caller's session. | Acquire: kill child, fail command.  Release: fallback to own group (own group cannot be `EPERM`). | REC (R2) / OK |
+| `ENOTTY` | fd no longer the caller's controlling terminal (hangup or dissociation after the resolve-time `CanControlTerminal` probe). | Acquire: kill child, fail command.  Release: fallback also fails → warn, unblock input gate; later commands re-probe and skip. | REC (R2) / OK |
+| `EBADF`, `EINVAL` | Bad fd / bad pgid value.  Program error; fd is held via a duplicated handle in pipelines. | Same as `ENOTTY` paths. | OK |
+| SIGTTOU (not an errno) | Caller is in a background group and does not ignore/block SIGTTOU: default action stops the shell. | Both call sites wrapped in `IgnoreSignalsForJobControl`. | OK |
+
+### tcgetpgrp — `TIOCGPGRP` (`CanControlTerminal`, `ShellOwnsTerminal`, first step of `SetForegroundProcessGroup`)
+
+| Errno | Trigger | Policy | Verdict |
+| --- | --- | --- | --- |
+| `ENOTTY`, `EBADF` | Not the controlling terminal / bad fd. | Resolve probe: endpoint is nil, no transaction.  Ownership gate: reports non-owner, transaction skipped.  Inside acquire: acquire fails (see tcsetpgrp rows). | OK |
+
+Read-only; does not raise SIGTTOU.
+
+### kill(-pgid, SIGCONT) — `ContinueProcessGroup`
+
+| Errno | Trigger | Policy | Verdict |
+| --- | --- | --- | --- |
+| `ESRCH` | Group fully reaped between `tcsetpgrp` and `SIGCONT`.  A group whose members are zombies still exists (man kill: an existing process may be a zombie), so the single-command path — which reaps only after acquisition — cannot hit this; pipelines reap concurrently in per-stage goroutines and can. | Roll back the handoff, then skip: return no lease, no error.  The job already finished. | FIXED |
+| `EPERM` | Cannot signal any member.  POSIX exempts `SIGCONT` within the sender's session, and the child group is always in our session. | Falls into the generic continue-failure path (rollback + fail). | N/A in practice, policy exists |
+| `EINVAL` | Bad signal number. | Impossible (constant `SIGCONT`). | N/A |
+
+### kill(-pgid, SIGKILL) — `KillProcessGroup`
+
+Called only on already-failed launch/acquisition paths; the return value is
+deliberately ignored at every call site (best-effort cleanup; `ESRCH` here
+means the work is already done).  **OK** — now documented as intentional.
+
+### tcgetattr — `TCGETS` (`CaptureTerminalMode`)
+
+| Errno | Trigger | Policy | Verdict |
+| --- | --- | --- | --- |
+| `ENOTTY`, `EBADF` | Terminal hung up between resolve and acquire. | Acquire fails before any state is touched; rollback is trivial.  Evaluator kills the child and fails the command. | REC (R2) |
+
+Read-only; does not raise SIGTTOU.
+
+### tcsetattr — `TCSETSF` via `term.Restore` (`posixTerminalModeSnapshot.Restore`)
+
+| Errno | Trigger | Policy | Verdict |
+| --- | --- | --- | --- |
+| SIGTTOU (not an errno) | POSIX: `tcsetattr` from a background process group raises SIGTTOU; default action stops the process.  Reachable whenever the restore runs while another group owns the terminal (e.g. Release handed the terminal back to a live foreign group recorded during the steal window). | Restore is now wrapped in `IgnoreSignalsForJobControl`, like the tcsetpgrp calls. | FIXED |
+| `ENOTTY`, `EBADF`, `EIO`, `EINTR`, `EINVAL` | Terminal gone or hung up. | Release: warn; input gate stays blocked only when the terminal is otherwise alive (mode restored wrong is a real input hazard); on total terminal loss the gate is released. | OK |
+| Partial success (returns 0) | man termios: `tcsetattr` "returns success if any of the requested changes could be carried out." | No policy possible without a verify-readback; accepted contract gap, recorded here. | OK (accepted) |
+
+### setpgid — child-side via `SysProcAttr` (leader `Pgid=0`, followers join leader)
+
+| Errno | Trigger | Policy | Verdict |
+| --- | --- | --- | --- |
+| `EPERM` (target group does not exist) | Follower joins after the leader's group died. | Prevented structurally: the leader waits for every stage to launch before it can be reaped (launch barrier).  Residual failure surfaces as that stage's `cmd.Start` error → per-stage exit code. | OK |
+| `EACCES`, `ESRCH`, `EINVAL` | Post-exec / wrong pid / negative pgid. | Not reachable from the child-side pre-exec call with Go's contract. | N/A |
+
+### waitpid — via `os/exec.Cmd.Wait`
+
+`EINTR` is retried inside the Go runtime; `ECHILD` cannot occur because Go
+owns reaping for started commands.  Non-`ExitError` failures map to
+`ExitStartUnknown` with a printed diagnostic.  **OK**.
+
+### dup / close — `DuplicateTerminalHandle`, `CloseTerminalHandle` (pipeline terminal retention)
+
+| Errno | Trigger | Policy | Verdict |
+| --- | --- | --- | --- |
+| dup: `EMFILE`, `ENOMEM`, `EBADF` | fd exhaustion / bad fd. | `registerTerminal` fails before the stage starts; command fails fast, nothing orphaned. | OK |
+| close: `EBADF`, `EINTR`, `EIO` | On Linux the fd is closed even when `close` reports `EINTR`/`EIO`; the code correctly does not retry. | `closeTerminal` error is a stderr warning; the pipeline's own result stands. | FIXED (was R1) |
+
+### SIGTTOU/SIGTTIN protection — `IgnoreSignalsForJobControl`
+
+`signal.Ignore`/`signal.Reset` are process-wide and not reentrant: a nested
+wrap would drop protection at the inner `Reset`.  All current uses are
+sequential and disjoint (verified).  Recorded as a constraint for future code.
+
+## Windows backend
+
+The direct-console backend has a much smaller failure surface: the
+"foreground" is an in-process marker, so `RestoreForegroundProcessGroup` is
+infallible and `ESRCH`/SIGTTOU have no analogue.
+
+- `SetConsoleCtrlHandler` (inside `SetForegroundProcessGroup`): failure fails
+  acquisition; `KillProcessGroup` is a no-op, so only the immediate child is
+  killed — already documented in the code as awaiting Job Object support.
+- `GetConsoleMode` capture skips per-handle failures and errors only when no
+  handle yields a mode; `SetConsoleMode` restore applies every saved mode and
+  reports the first failure.  Reasonable; console modes are shared state that
+  other processes can also change (see the closed-world note in
+  MODEL_RESULTS.md).
+
+## Findings summary
+
+- **F1 (documented):** Linux `TIOCSPGRP` returns `ESRCH` for a reaped group;
+  the man page omits it.  Recorded in ASSUMPTIONS.md.
+- **F2 (fixed):** `term.Restore` (`tcsetattr`) ran without SIGTTOU protection;
+  a release that handed the terminal to a live foreign group could stop the
+  shell mid-`Release`.  Now wrapped like the tcsetpgrp calls.
+- **F3 (fixed):** a fast pipeline whose stages were reaped concurrently could
+  be fully gone before acquisition; the resulting `ESRCH` from
+  `tcsetpgrp`/`SIGCONT` killed and failed a job whose children exited 0.
+  `ESRCH` during acquisition now means "already finished, run without a
+  handoff" (with rollback where the handoff partially happened).
+
+Resolutions (maintainer decisions 2026-08-13):
+
+- **R1 (applied):** `closeTerminal` failure is now a warning; a successful
+  pipeline is never failed over it.
+- **R2 (decided 2026-08-13: keep the kill):** non-`ESRCH` acquisition
+  failures (`ENOTTY`/`EIO` after a hangup in the resolve→acquire window,
+  `EPERM`) kill the child and fail the command.  bash degrades to running the
+  job without foreground ownership — but bash can afford to: it has a job
+  table and `WUNTRACED` waits, so a child that later stops on `SIGTTIN`
+  becomes a recoverable stopped job.  mshell's synchronous `cmd.Wait` would
+  hang forever on such a child, and a silent hang is worse than a wrong kill.
+  The kill also approximates `SIGHUP` semantics for children in their own
+  process groups when the terminal dies, and never leaves orphans.
+  Alternatives considered and declined for now: degrade only on
+  dead-terminal errnos (`ENOTTY`/`EBADF`/`EIO`, provably hang-free because
+  reads return `EIO` instead of raising `SIGTTIN`) — rejected as an extra
+  policy fork whose benefit (a rare race) does not outweigh orphaned
+  processes; full bash-style degrade — deferred.  **Revisit when the
+  `jobs`/`fg`/`WUNTRACED` milestone lands, which makes full degrade safe.**
+- **R3 (applied, modeled):** `POSIXTerminalControl.tla` now has a `reaped`
+  state distinct from zombie `exited`, early `ProcExits`, concurrent
+  `ReapProc`, the `~GroupDead` kernel contract on `GiveTerminal`, and the
+  `GiveTerminalTargetGone` skip transition; invariant
+  `UnsupervisedJobNeverOwnsTerminal` verified non-vacuous by mutation.  See
+  MODEL_RESULTS.md.

ai/terminal-control/IMPLEMENTATION_STATUS.md

@@ -1,9 +1,57 @@
 # Implementation status
 
-Last updated: 2026-08-09.
+Last updated: 2026-08-13.
 
 ## Implemented
 
+### 2026-08-13 parallel-shell hardening
+
+Parallel `redo` builds exposed a cross-process race the 2026-08-09 milestone
+missed: with several mshells sharing one PTY, each shell's recorded "previous
+foreground process group" could be a sibling's transient child group, dead by
+release time, so the restoring `tcsetpgrp` failed with ESRCH and the command
+was failed even though its child exited 0.  Changes:
+
+- Acquisition is gated on `tcgetpgrp(tty) == getpgrp()` (the bash/fish gate).
+  A shell that is not the terminal's current foreground owner skips the
+  foreground transaction entirely; its child runs as an ordinary background
+  process group.  On Windows the gate is always open because the foreground
+  state is an in-process Ctrl-C marker, not shared kernel ownership.
+- A failed hand-back to the recorded previous group falls back to restoring
+  the shell's own process group (the group bash and fish restore).  Only if
+  both targets fail does `Release` report an error.
+- Reclaim errors are warnings on stderr, never command failures; the child's
+  exit status always stands.  This supersedes the earlier "restore the exact
+  previous foreground marker or fail" behavior.
+- Policy change: after a double restore failure the shell input gate is now
+  released rather than held.  Both targets failing means the terminal itself
+  is unusable (for example a closed PTY); holding the gate wedged every later
+  command behind a terminal that no longer exists.  Mode-restore failure with
+  a live terminal still blocks shell input as before.
+
+### 2026-08-13 errno audit
+
+ERRNO_AUDIT.md traces every errno of every syscall the controller makes to an
+explicit policy or a finding.  Two further gaps found and fixed:
+
+- `term.Restore` (`tcsetattr`) ran without SIGTTOU protection and could stop
+  the shell when the terminal had been handed back to a live foreign group;
+  it is now wrapped in `IgnoreSignalsForJobControl` like the tcsetpgrp calls.
+- A fast pipeline whose stages were reaped concurrently could have its whole
+  process group vanish before acquisition; the resulting ESRCH killed and
+  failed a job whose children exited 0.  Acquisition-time ESRCH now means
+  "already finished": roll back any partial handoff and run without a lease.
+
+All three audit recommendations are resolved: R1 applied (closeTerminal
+failure is a warning, never a pipeline failure), R3 modeled
+(reaped-vs-zombie states in POSIXTerminalControl.tla), and R2 decided as
+keep-the-kill — a non-ESRCH acquisition failure kills the child rather than
+risking an unrecoverable SIGTTIN hang, since mshell has no stopped-job
+recovery yet.  R2 must be revisited when the `jobs`/`fg`/`WUNTRACED`
+milestone lands; the full trade-off is in ERRNO_AUDIT.md.
+
+### 2026-08-09 milestone
+
 - Resolved stdin/stdout/stderr metadata remains attached to the `exec.Cmd`
   streams after redirects and merges are applied.
 - Terminal selection distinguishes an arbitrary TTY from the session's
@@ -38,6 +86,20 @@ Last updated: 2026-08-09.
   isolation mechanism for interactive background jobs, not the foreground
   handoff mechanism.
 
+Verification added with the hardening:
+
+- A PTY integration test starts six shells sharing one PTY, each running a
+  staggered short external command, and requires all six to succeed.  Before
+  the fix it reproduced the production failure exactly (three to four of six
+  failing with "no such process" on reclaim).
+- Controller unit tests cover the skipped acquisition for a non-owner shell,
+  the fallback restore order (dead previous group, then own group), and the
+  combined error when both hand-back targets fail.
+- `POSIXTerminalControl.tla` now models the foreign owner, the steal window,
+  the dying previous-owner group, the ownership gate, and the fallback; all
+  six bounded TLC configurations pass, including the new shared-terminal
+  profile.  See MODEL_RESULTS.md.
+
 ## Verification currently passing
 
 - Controller unit tests inject mode capture, acquisition, continue, rollback,