Skip to content

An interrupted turn is dropped without a signal and its receipt is never reclaimed #42

Description

@MPIsaac-Per

Problem

A turn interrupted after dispatch begins is lost silently, and the receipt it leaves behind is never removed.

markDispatchStarted sets dispatchStartedAt before the first external side effect. If the process dies between that point and completeEvent / failEvent, the receipt stays status: "claimed" with the marker set. From there:

  1. Never replayed. activeRecoverableReceipts (src/state/store.ts:2004) requires dispatchStartedAt === undefined, so startup recovery skips it. This is correct as designed: the turn may already have posted activities to Linear, and replaying it would double-run.
  2. Never removed. prune (src/state/store.ts:1913) only removes receipts where isTerminal(status) holds, and claimed is not terminal. The maxEntries eviction also only considers terminal receipts, so the store can pass its cap and stay there. One entry accumulates per interrupted turn, permanently.
  3. Never reported. Nothing outside the store reads dispatchStartedAt. /healthz returns a bare ok.

In Linear the session simply goes quiet. A queued session and an abandoned one look identical, which is the manual liveness check documented in CLAUDE.md. Observed on 2026-08-19 when the service was restarted with two turns active.

claimEvent already distinguishes these two cases (src/state/store.ts:622 and :657): a foreign claim with no dispatch marker transfers and re-runs, a foreign claim with the marker set is answered ambiguous and not dispatched. Both branches only fire when a redelivery arrives. Linear got its 200, so no redelivery is coming, and nothing else ever looks at the receipt.

Proposed change

A startup sweep that terminalizes abandoned dispatches and tells Linear the turn ended. Recovery semantics do not change: an interrupted turn is still never replayed.

Ownership. failEvent cannot be reused as-is. It routes through terminalize, which calls ownedActiveClaim and requires claim.ownerId === this.ownerId; ownerId defaults to a fresh randomUUID() per store instance, so a restarted process is never the owner of its predecessor's claim. This needs a distinct store method for foreign-owned stranded claims.

Liveness. Terminalizing any foreign-owned claim would let a second process abandon a first process's live turns if both ever shared a state directory. Persist process identity on the claim at markDispatchStarted, reusing the identity source behind the lock owner record (lockProcessIdentity / lockBootIdentity, checked the same way removeAbandonedLock does), and sweep only claims whose owning process is provably gone. A startup-only sweep guarded by a single-instance assumption was considered and rejected: the assumption is not enforced anywhere in code.

  1. Persist owning process identity on the claim when the dispatch marker is set.
  2. Add a store method that lists non-terminal receipts carrying dispatchStartedAt whose owning process is gone, and one that marks them failed with the existing AmbiguousDispatch error class. Terminal status makes them prunable through the path that already exists.
  3. Run the sweep at startup before recoverAcceptedIngressPass (src/server.ts:961). Log bounded identifiers and a count, never prompt or issue content.
  4. Post one activity per abandoned session saying the turn was interrupted and was not retried. { type: "error", body } matches how the server already reports terminal failure (src/server.ts:1711). Use getOrCreateActivityId so a sweep that itself crashes and re-runs does not post twice.

Acceptance criteria

  • A receipt left claimed with dispatchStartedAt set by a dead process reaches failed / AmbiguousDispatch on the next startup.
  • That receipt is pruned once past retention, and counts toward maxEntries eviction.
  • It is still never dispatched, before or after the sweep.
  • A claim whose owning process is alive is left untouched, including one owned by the current process.
  • Exactly one activity is posted per abandoned session, including when the sweep runs twice.
  • Logs carry identifiers, counts, and error classes only.
  • npm run typecheck && npm test pass.

Notes

Frequency is low, since it needs a crash or a restart mid-turn. The unbounded growth in the state file is what makes it worth fixing rather than tolerating.

Split out of #8, which is otherwise delivered.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions