Parent epic: #337
Delivery step 5; depends on filesystem pending-message implementation #339 and stage contracts #344 (building on the memory baseline #338). Followed by file-backed DLR #340.
Goal
Implement file-backed selected-router and routed-work stages for the single-process outbound SMS profile, then enable and document all supported file-pending/file-or-memory stage combinations. Keep memory/memory/memory as the non-durable default after this issue is delivered; all durable profiles, including file/file/file, require explicit configuration.
Scope
- Implement the file-backed selected-router backlog so a bounded eligibility/priority selection has a recoverable committed result. After restart, refill bounded runtime router queues from that result without repeating selection for already staged messages.
- Implement recoverable recorded routing results (all copied destinations), per-destination progress and completion. Restore completed branches without resending them; unfinished branches may repeat provider submission.
- Keep pending source records until all destinations and required existing PostgreSQL DLR handoffs are terminal. For a file-selected/memory-routed profile, retain selected records until safe completion so recovery reroutes rather than loses in-flight work.
- Reconcile interrupted writes/transitions using stable IDs and a documented durable stage representation. A partial batch or interrupted route handoff must not be mistaken for completed work. File write/index/journal choices belong here, not in the architecture contract.
- Use bounded in-memory prefetch; avoid loading the whole spool or rescanning every pending message on each refill. Define the selection index/scan strategy, recovery behavior and limits for a large backlog.
- Preserve exclusive ownership of the local spool and process/container restart semantics when its persistent volume survives.
Configuration and supported profiles
Support pending=file with both router-queue=file|memory and routed-work=file|memory, plus the explicit memory/memory/memory compatibility profile from #339. Validate configurations at startup; do not silently substitute memory on storage failure. Fail closed on unavailable/corrupt committed state. All three outbound selectors default to memory when omitted. memory/memory/memory remains the non-durable default at the end of this issue; file-backed profiles are opt-in.
Verification and acceptance criteria
- Forced restart tests cover selection publication, router dequeue, route handoff, worker dequeue, copied-route partial completion, provider ambiguity, and file-selected/memory-routed and memory-selected/file-routed recovery.
- A durable selected batch survives restart without selection rerun; durable destinations resume without rerouting; volatile stages repeat the documented work without losing an acknowledged message.
- Metrics and readiness reflect selected backends and unsafe transition failures without per-message/path labels; logs report recoverable counts and actionable errors without SMS payloads.
- JVM/native container restart coverage with the spool volume retained; deployment and operational docs explain stage tradeoffs, volume requirements, and at-least-once behavior.
Out of scope
PostgreSQL stage implementations (#333), DLR backend replacement (#340/#334), durable retries and provider outcome checkpoints, multi-process spool sharing, host/power-loss guarantees, and exactly-once submission.
Related: architecture #338; pending contract #343; stage contracts #344; pending file implementation #339.
Parent epic: #337
Delivery step 5; depends on filesystem pending-message implementation #339 and stage contracts #344 (building on the memory baseline #338). Followed by file-backed DLR #340.
Goal
Implement file-backed selected-router and routed-work stages for the single-process outbound SMS profile, then enable and document all supported file-pending/file-or-memory stage combinations. Keep
memory/memory/memoryas the non-durable default after this issue is delivered; all durable profiles, includingfile/file/file, require explicit configuration.Scope
Configuration and supported profiles
Support
pending=filewith bothrouter-queue=file|memoryandrouted-work=file|memory, plus the explicitmemory/memory/memorycompatibility profile from #339. Validate configurations at startup; do not silently substitute memory on storage failure. Fail closed on unavailable/corrupt committed state. All three outbound selectors default tomemorywhen omitted.memory/memory/memoryremains the non-durable default at the end of this issue; file-backed profiles are opt-in.Verification and acceptance criteria
Out of scope
PostgreSQL stage implementations (#333), DLR backend replacement (#340/#334), durable retries and provider outcome checkpoints, multi-process spool sharing, host/power-loss guarantees, and exactly-once submission.
Related: architecture #338; pending contract #343; stage contracts #344; pending file implementation #339.