feat: durable message lifecycle from admission to execution

[Astro-Han]

Summary

This PR makes accepted messages survive any Runtime Host crash cut without being lost, revived, or executed twice. It replaces Draft PR #3633 from a fresh main; none of #3633's commits were cherry-picked.

The central simplification is that lifecycle names are classifications of existing durable facts, not rows in a second state machine:

flowchart TD
  S[turn.message.submit] --> A[Accepted<br/>pending admission payload]
  A -->|retract before handoff| C1[Cancelled<br/>minimal identity tombstone]
  A -->|durable root-source or steering proof| H[HandedOff<br/>proof + transcript<br/>pending payload deleted]
  H -->|model_call_attempt_recorded| E[Executed<br/>derived from provider proof]
  H -->|terminal Stop without provider request| C2[Cancelled<br/>derived from terminal proof]

Loading

queued, leased, and in_flight exist only in Host memory and are rebuilt from Accepted admissions after restart.

Authority and lifecycle

• message_admissions owns only accepted drafts: canonical content, placement, order, edit, promotion, and the data needed to restart.
• Queued Messages belong to the durable Session, not to the transient Client connection that submitted them. A terminal queue forms one ordered successor root even when several Clients contributed Messages.
• Connection-local tools are not silently carried into a queued successor. Durable Session bindings survive; lost or ambiguous ephemeral bindings degrade through the existing Client Capability contract.
• Pre-handoff retract replaces the draft with a minimal cancellation tombstone, preventing the same message identity from being revived without retaining its full payload.
• Root admission and immutable steering events are the handoff proofs. One SQLite transaction materializes transcript history and deletes the accepted payload.
• Normal terminal handling and startup recovery call the same handoff materializer.
• model_call_attempt_recorded is already the durable Executed proof; terminal root facts are already the durable Stop proof. No copied message lifecycle column or unsettled-message scan remains.
• Durable Session capability recovery is derived from the durable external root execution contract, not a Host-memory marker.
• Same-Host operation retries replay in-memory outcomes. Old-Host retries use durable root/steering proof; the obsolete SQLite Host-Epoch receipt tables and their failure state machine are removed.
• RuntimeKernel's embedded queue/fallback methods are removed, leaving Runtime Host as the only message authority.

Root handoff

sequenceDiagram
  participant Q as Accepted Session queue
  participant O as Message settlement owner
  participant R as Root admission
  participant T as Transcript
  participant H as Runtime Host
  Q->>O: terminal queue in canonical order
  O->>R: admit one successor with all sources
  R-->>O: durable root-source proof
  O->>T: materialize each source once
  O->>Q: delete handed-off drafts atomically
  O-->>H: activate successor from durable root

Loading

Recovery first classifies every admission against durable proof and then rebuilds memory only from the surviving Accepted set. The pre-classification snapshot is never reused as queue authority.

Crash cuts

1. Before admission commits: no message exists.
2. After admission, before proof: one Accepted draft exists and no transcript row exists; restart reconstructs it.
3. After root/steering proof, before handoff materialization: recovery uses that proof to run the same handoff transaction.
4. After handoff: transcript exists once and the pending payload is gone, so restart cannot requeue it.
5. After provider request: the existing model-call event proves execution; no message-state write is required.
6. After pre-handoff retract: only the minimal tombstone remains; the message cannot reappear in queue or history.
7. With proof-owned and still-pending Messages mixed in one active Run: recovery materializes the proved subset and projects only the surviving pending subset.

Scope

Included: atomic admission/root transcript composition, canonical Skill preparation, edit/reorder/promotion/retract, successor handoff, terminal handling, restart recovery, size boundaries, durable capability derivation, ledger repair provenance, and removal of duplicate queue/receipt authority.

Excluded: CLI/Desktop projection refactors and Side Conversation UI/hooks.

Verification

Only affected tests were run locally; the full repository suite was not run locally.

• Exact pushed head: dbad1b463.
• Build: @maka/runtime-host.
• Client Capability, Message Coordinator, and Root Turn Coordinator: 117 tests passed.
• Real SQLite + Runtime Host/UDS message and queue composition: 16 tests passed.
• Fault coverage includes Host crash after queue admission, restart after atomic root admission, mixed durable-proof/pending recovery, exact old-Host retry proof, explicit retract across connections, one ordered multi-Client successor, and transcript de-duplication.
• Earlier unchanged storage/runtime lifecycle slices on this branch also passed their focused tests; no full-repository test was run locally.

CI for this exact head is pending.

Review follow-up

• Active recovery now rebuilds from the post-classification pending set, so a proof-owned steering Message cannot be revived from a stale admission snapshot.
• Per-Client successor partitioning and its connection identity state were removed. Queued Messages form one Session-owned successor, eliminating cross-generation survivor ownership and ordering states.
• Failed terminal roots cannot strand copied handed_off rows because those rows and unsettled scans no longer exist.
• Settled Messages do not retain a second full payload: handoff atomically writes transcript history and deletes the accepted draft.

AI use

This PR was implemented with Codex assistance. The design, repository decisions, code changes, affected-test selection, review analysis, simplification audit, and verification were directed and checked against the repository's durable authorities and production Runtime Host composition.

[M4n5ter]

English

I found two correctness issues on exact head 2ea21a68950502615f5e109ce914a7af94a97140: one makes durable follow-up reordering fail for every real permutation, and the other rejects otherwise valid 32–49 KiB messages as an internal failure. The exact-head hosted test check is green, but its queue test uses an in-memory lifecycle stub for reorder and does not exercise either storage boundary.

简体中文

我在 exact head 2ea21a68950502615f5e109ce914a7af94a97140 上确认了两个正确性问题：持久化 follow-up 队列的任何实际换序都会失败；另一个问题会把本来合法的 32–49 KiB 消息错误地变成内部错误。当前 head 的托管 test 检查是绿色，但队列测试对重排使用了内存 lifecycle stub，没有覆盖这两个真实存储边界。

[Astro-Han]

I reviewed this head and found blocking issues that need to be addressed before merge.

[P1] Recovery can replay raw /skill text instead of the prepared skill invocation

Idle submit persists raw modelContent first (message-coordinator.ts:906-918) and only later expands the skill in the Host (root-turn-coordinator.ts:1028-1077). If the Host exits between those steps, recovery replays the persisted raw content without re-running skill preparation, producing a root that the normal path would have wrapped.

[P1] Rejected idle submit leaves a phantom user message in the transcript

The admission and user transcript are written together, but later start/admission can still fail (skill blocked, oversized, binding failure). The cancellation only flips lifecycle state, not the transcript — a failed send remains visible and retries create duplicates.

[P2] Message reorder with identical content is rejected as a conflict

The metadata store compares target order byte-for-byte, so any non-trivial reorder is treated as a conflict. Existing tests pass only because reorders in those fixtures are no-ops.

[P2] Admission envelope can exceed storage limits undetected

Submitted and prepared payloads are checked separately, but the combined admission envelope (64 KiB limit) is not pre-validated. Inputs in the 32–49 KiB range pass early gates yet fail at admission, surfacing as an internal failure.

CI on 2ea21a689505 is test: success. These issues are independent of CI and require fixes before approval. Heads verified at time of review.

简体中文

存在恢复路径与 transcript 残留等阻断问题，需修复后重审。

[Astro-Han]

I verified a blocking issue in this head.

[P2] Retracted messages that never reached the Host remain visible in the transcript

When a queued Accepted message is retracted during reload, the lifecycle row is moved to cancelled but the session_messages user row and catalog preview inserted at admission stay visible. TUI expects still-queued text to return to the editor and only consumed messages to stay in history — after reload the old row is still paged as consumed steering, and resubmission leaves a duplicate.

Repro: next_turn submit → queue.retract → reopen. Lifecycle cancelled but readMessages still returns the user row with steeringEventId.

Fix: for accepted -> cancelled (never handed off) hide/remove the transcript row and recompute the preview with a lifecycle-aware projection; keep the row only for handed_off -> cancelled where Runtime actually consumed it. Cover reload + edit/resubmit.

Previous skill-recovery P2 is closed in this head. Checks on 490ffb31a4 are test: success.

简体中文

撤回未送达消息仍留在历史视图，需区分未送达与已消费后取消。

[Astro-Han]

I verified the previous blocking finding still applies in this head.

[P2] Retracted Accepted messages remain visible

sqlite-session-metadata-store.ts:1605-1648 inserts the transcript row on Accepted, but cancelMessageAdmissions at :1935-1963 only flips lifecycle to cancelled without hiding rows that never reached the Host. After reload the never-handed-off row is still rendered, and resubmission leaves a duplicate.

Head f01aa233cd vs previous 490ffb31. Checks on f01aa233cd are test: success dependent on exact-head run.

简体中文

该阻断在该头仍成立。

[Astro-Han]

Update on 48b162326f:

The previous retracted-message P2 is now closed. Admission now commits a canonical draft only, and markMessagesHandedOff atomically materializes the transcript and advances lifecycle within a single SQLite transaction. Pre-handoff retract therefore leaves no transcript row to leak, while post-handoff cancel correctly retains history.

No remaining blocking issues. Previous Skill-recovery P2 remains closed.

Checks on 48b16232 are test: success.

简体中文

该阻断已在该头闭合。

[Astro-Han]

Update on 48b162326f: two blocking issues found in independent re-review.

[P2] Terminal root without provider-request proof leaves handed-off messages unsettled

A failed root before any model_call_attempt leaves handed_off rows in listUnsettledMessages() forever, causing O(N²) recovery scans on restart.

[P2] Settled admissions permanently retain a full message copy

After handoff, message_admissions keeps content_json (48 KiB) even though the transcript already holds the text, permanently doubling storage for executed messages.

Checks on 48b16232 are test: success, but these lifecycle issues remain independent of CI.

简体中文

该头存在终态行为与存储双重阻断。

[M4n5ter]

English

I found two blocking issues on exact head 48b162326f39ac1c83d7ebd8277a33ffd56ddba3: a failed terminal root can leave handed-off messages permanently unsettled, and every settled admission permanently retains a second full copy of the message payload. The hosted test check is green; I also ran 156 focused tests and four package builds, then reproduced both issues through the production coordinator/storage seams with real SQLite. The previous reorder and admission-envelope findings are closed on this head.

简体中文

我在 exact head 48b162326f39ac1c83d7ebd8277a33ffd56ddba3 上确认了两个阻断问题：失败终态的 root 可能让已 handoff 的消息永久处于未结算状态；每条已结算 admission 还会永久保留第二份完整消息 payload。托管 test 检查是绿色；我也运行了 156 个聚焦测试和四个 package build，并通过 production coordinator/storage seam 与真实 SQLite 复现了两个问题。此前的重排与 admission-envelope 问题在该 head 上已经闭合。

[M4n5ter]

English

On exact head d9c700103336f2b39d40667f16b89a8f0bf56cf7, the two previous P2 findings are closed: proof-owned handoff now removes the pending admission atomically with transcript materialization, and settled messages no longer retain a second full payload. I found one new P2 in the mixed restart-recovery path: a message already owned by a durable steering proof can be requeued when another admission is still pending. The hosted test check is currently queued; locally, four package builds, all 42 message-coordinator tests, and 108 storage/runtime/root focused tests passed, but the existing recovery test covers only the all-proved case.

简体中文

在 exact head d9c700103336f2b39d40667f16b89a8f0bf56cf7 上，之前的两个 P2 已闭合：由 durable proof 接管的 handoff 现在会在 materialize transcript 的同一事务中删除 pending admission，已结算消息也不再保留第二份完整 payload。我发现一个新的 P2：重启恢复同时包含“已有 durable steering proof”和“仍 pending”两类 admission 时，已经被 proof 接管的消息会再次入队。托管 test 当前仍在排队；本地四个 package build、42 个 message-coordinator 测试以及 108 个 storage/runtime/root 聚焦测试全部通过，但现有恢复测试只覆盖全部消息都有 proof 的情况。

[M4n5ter]

Reviewed exact head 19de5eb7ca13c6fb27f5c4598df5d9c55019604d.

The previous mixed-recovery P2 is closed: active recovery now rebuilds only from the post-classification pending set, and the regression covers one proof-owned steering admission beside one genuinely pending admission. I also checked the follow-up refactor that removes connection-owned queue partitions: queued messages now form one ordered Session successor, while missing or ambiguous connection-local capabilities degrade through the existing capability contract instead of being silently rebound to a submitting Client.

Locally, the Runtime Host build and 129 focused Message Coordinator, Client Capability, Root Turn, and production queue tests passed. The only current-main overlap is a Client Capability test file; a clean merge tree was produced. I found no remaining P0–P3 issue in the reviewed delta.

The hosted test check for this exact head is still queued, so this approval does not claim that the merge gate is complete.