Skip to content

fix(host): read a terminal session next to the destroy marker as destroyed - #284

Merged
ThomasK33 merged 37 commits into
mainfrom
fix/265-marker-repair
Nov 3, 2026
Merged

ThomasK33 merged 37 commits into
mainfrom
fix/265-marker-repair

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Why

ThomasK33 asked to work every deferred follow-up of the first autopilot round until none is left. This PR closes #265, the follow-ups after #256. Since #256, recordDestroyed writes an empty <session>/destroyed marker before its destroyed write, and a reconcile that wrote over that record writes destroyed again right after its own write. Three gaps stayed open, and all three reproduce on main 71d1b0c2:

  1. A killed reconcile leaves failed. A reconcile killed between its own write and the repair write left failed, after destroy --force had printed destroyed: true. Every later reconcile skipped the terminal manifest, so the session stayed failed.
  2. The reader window. A reader between those two writes saw failed, and scanHome counted the session. home list --all and the dashboard picker then showed one extra session, and after a killed reconcile they kept showing it.
  3. A stale marker after a failed destroy. A destroy of an ended session that failed after its marker write left the marker next to exited or failed, and nothing repaired it.

What changed

  • src/host/lifecycle.ts
    • reconcileSession first checks a terminal manifest other than destroyed for the marker. If the marker is there, writeMarkedDestroyed writes destroyed from that copy, with hostPid and childPid null, the exit code kept and no failure fields. The reconcile then unlinks the socket, which a destroy whose write failed left, removes the empty Home socket folder and returns. Only a destroy writes the marker, after it stopped the session and right before its destroyed write, so the marker means that destroy chose destroyed.
    • listSessions reconciled only active sessions. It now also reconciles a terminal session that is not destroyed when the marker exists, so list heals items 1 and 3 itself. When that repair throws, list reads the manifest again and shows it: destroyed when only the socket cleanup after the write failed, and the copy it read before when the write itself failed, for example on a full disk, as before this PR.
    • inspect calls reconcileSession when the stored record is terminal, not destroyed, and has the marker, so inspect shows destroyed as list does. A running session with a live host and the marker stays running: the marker repairs only an ended session.
  • src/storage/homeScope.ts. scanHome skips a session whose folder holds the marker when its status is terminal, the same rule as the repair. It asks its ScanHomeDependencies.pathExists for the marker. home list and the dashboard picker stay read-only and no longer count such a session.
  • The lifecycle check gets a new guard, ReconcileRepairsMarked.
    • The model writes destroyed at the reconcile's first read when the marker is next to a terminal manifest other than destroyed.
    • facts.mjs reads the guard, and writeMarkedDestroyed is covered.
    • The new witness WitnessMarkedRepaired shows that the branch is reachable: one reconcile's first read repairs the failed that the other reconcile wrote.
    • mutant-ReconcileRepairsMarked expects pass. The model has no crash of a reconciling process, so the repair after the reconcile's own write always runs there. The unit tests cover the crash. SKILL.md says so.
  • docs/USAGE.md, src/host/README.md, src/storage/README.md, src/dashboard/README.md and src/cli/commands/home/README.md describe the repair, which commands run it (list, inspect, gc) and the home list count.

Declined, as approved by the root: deleting the marker (item 4). A reconcile that read before a destroy's write can still write after it, and no process knows when the last such reconcile has finished. So deleting the marker would reopen the #217 race. The marker is one empty file, and gc removes it with the session folder.

Behavior changes for CLI and JSON users

  • No JSON shape, error code or exit code changes.
  • A session that a destroy marked reads destroyed after the next list, also when a reconcile was killed in the middle or when that destroy failed after its marker write. Before, such a session stayed failed or exited. gc repairs it too before it removes the folder.
  • home list and the dashboard picker no longer count such a session.

Validation

  • Red first. At 35bce9d8 (tests only), 3 tests failed: the two reconcileSession repair tests and the scanHome marker test. The control test (an exited session without the marker stays exited) passed. At 7ddc041e, the list test failed. All pass on the head.

  • Live repro with the CLI, a killed host and a held rename (REPLAY_HOLD), each run in an isolated AGENT_TTY_HOME and XDG_STATE_HOME:

    Item main 71d1b0c2 head aea5277e
    1. destroy --force prints destroyed: true, then the reconcile is killed before its repair write. The next list shows failed destroyed
    2. home list --all while the repair write is held {"activeSessions":0,"totalSessions":1} home not listed (no visible session)
    3. destroy of an exited session fails after the marker (STORAGE_WRITE_ERROR). The next list shows exited destroyed
  • Lifecycle check (.agents/skills/verify-session-lifecycle/scripts/check.sh --update-coverage): all 66 runs as expected. current explored 24,208,708 distinct states, current-no-token 23,413,352. WitnessMarkedRepaired is reached. coverage.json is synced at cb4b7f83.

  • CI-shaped gate on aea5277e: typecheck, lint and format pass. Unit 1736/1736. Integration shards 2 and 4 and e2e shards 1 and 3 pass. Three files failed under the 7 parallel shards and pass alone on the head: backend-selection.test.ts and export-fixtures.test.ts (12/12 together), and run.test.ts (14/14, plus 3/3 runs of detects session exit during wait before timing out). backend-selection and the export-fixtures color-grid test fail the same way on main under that load.

  • Round 2 (Codex round 1 on aea5277e): red 2c47894e (2 tests failed: the marked repair left the socket, and list --all dropped a marked failed session whose repair write failed), fix 8a6695fd, model c5482b20 (the repair goes on to the socket unlink), facts.mjs d875e821 (the guard pattern includes the unlink), coverage cb4b7f83. Unit tests of src/host, src/storage and the commands: 368/368. Alone on d875e821: lifecycle.test.ts 58/58, non-reaping-init.test.ts 3/3, socket-directory.test.ts 9/9. A full local gate on d875e821 ran at host load 130 to 180 and timed out in files that this change does not reach, so CI on the head is the full gate.

  • Round 3 (Codex round 2 on cb4b7f83): red 749795da (list showed failed when the socket cleanup threw after the repair write), fix e845dd71. Unit 1739/1739. facts.mjs drift 0, so check.sh did not run again.

  • Round 4 (Codex round 3 on e845dd71, plus the root verifier notes N1 to N3): red 35ae60d5 (inspect left a marked failed session failed, and scanHome read the marker from the real filesystem), fix 18336a12. Model b8b8dbb4 splits the marked repair into its read ("r_marked") and its write (ReconcileWritesMarked). b23d1214 pins the terminal-status rule (a running session with a live host and the marker stays running; I checked that the test fails without the rule) and adds a red test for scanHome, fixed in 6e0a595c. Docs b08dad39. Coverage ad17bf61. Lifecycle check: all 66 runs as expected, current explored 24,866,000 distinct states, current-no-token 24,005,824. Unit 1743/1743. lifecycle integration 58/58 and cli integration 65/65.

  • Round 5 (Codex round 4 on ad17bf61, security clean): red d22e3df4 (inspect failed when the marked repair threw, though list showed the session), fix 61e81f1c (inspect reads the manifest again, as list does). Docs eaae86e0 (the scanHome README line says "ended", USAGE and the commands README name the inspect fallback). facts.mjs drift 0, so check.sh did not run again. Unit 1745/1745. lifecycle and cli integration 123/123. Typecheck, lint and format pass.

  • Merge of main 307c0e99 (f7aad5b8, counts 3868d3b0), no Codex round. Codex round 5 on eaae86e0 was clean in both lanes. Main's fix(host): freeze a program group that no scan shows before the proven kill #283 changed the model, check.sh, facts.mjs and SKILL.md, and so did this PR. Conflicts: the guard list in SessionLifecycle.tla (both new guards kept), the run list in check.sh, two SKILL.md hunks and two src/host/README.md hunks (merged word by word), and coverage.json (rewritten by check.sh --update-coverage). check.sh on the merge: all 68 runs as expected, current 24,866,000 distinct states and current-no-token 24,005,824. SKILL.md now counts 68 runs and thirty guards. Unit 1888/1888. lifecycle and cli integration 138/138. Typecheck, lint and format pass.

  • Merge of main 142f7793 (fix(host): keep sockets under a per-uid root and remove a dead host socket #288, 2371636a) and two follow-ups from the root's advisor (90480ffa head). Conflicts: the SKILL.md Scope line (only the state counts, now from this run), a USAGE hunk (merged word by word) and coverage.json. Follow-up 1: removeDeadHostSocket returns when the host check throws, so it removes neither socket of a host it cannot judge (red 7e259ab7: destroy removed the old socket under /tmp/agent-tty when process.kill(pid, 0) threw EINVAL; fix 30c71732). Follow-up 2: USAGE limit 2 names umask 002, under which an older version made /tmp/agent-tty mode 0775, which the old-socket check refuses until chmod g-w (f452642d, with the host README). check.sh on the head: all 68 runs as expected, current 25,534,376 distinct states and current-no-token 24,675,744 (SKILL.md updated). facts.mjs drift 0. Unit 1922/1922. lifecycle, cli, socket-directory and socket-permissions integration 153/153. Typecheck, lint and format pass.

  • Merge of main 374937f9 (fix(host): read a group out of the CLI's pid space as running #291, fix(release): raise the Communique response token limit so release notes are not cut off #295; merge 8b2863b2, coverage 91d7f744), no Codex round (root's GO for a merge-only delta). Conflicts: the check.sh run list and a src/host/README.md hunk (merged word by word and checked against both sides), the SKILL.md counts (now 71 runs, CHECK_JOBS=71, eight runs that explore the whole model, thirty-one guards) and coverage.json. check.sh on the merge: all 71 runs as expected, current 25,534,376 distinct states and current-no-token 24,675,744. facts.mjs drift 0. Unit 1929/1929. lifecycle, cli and socket-directory integration 153/153. Typecheck, lint and format pass.

  • Deferred to the root's parking issue: Codex round 6's P2 (the marked repair unlinks only the current socket, not the old /tmp/agent-tty one).

Closes #265


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-03T17:53:03.981918Z 90480ff Manual request
🔒 Security Review ✅ Completed 2026-11-03T17:52:22.659441Z 90480ff Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: aea5277ed1

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aea5277ed1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/host/lifecycle.ts Resolved
Comment thread src/host/lifecycle.ts Resolved
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb4b7f839c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/host/lifecycle.ts Resolved
@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: cb4b7f839c

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: e845dd71d0

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: eaae86e069

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: eaae86e069

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

# Conflicts:
#	.agents/skills/verify-session-lifecycle/SKILL.md
#	.agents/skills/verify-session-lifecycle/coverage.json
#	.agents/skills/verify-session-lifecycle/model/SessionLifecycle.tla
#	.agents/skills/verify-session-lifecycle/scripts/check.sh
#	src/host/README.md
# Conflicts:
#	.agents/skills/verify-session-lifecycle/SKILL.md
#	.agents/skills/verify-session-lifecycle/coverage.json
#	docs/USAGE.md
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 90480ffae1

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 90480ffae1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/host/lifecycle.ts Resolved
# Conflicts:
#	.agents/skills/verify-session-lifecycle/SKILL.md
#	.agents/skills/verify-session-lifecycle/coverage.json
#	.agents/skills/verify-session-lifecycle/scripts/check.sh
#	src/host/README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reconcile/destroy: follow-ups after #256 (killed reconcile leaves failed, reader window, stale marker)

1 participant