Skip to content

feat(storage): persist selected-router and routed-work state on filesystem #345

Description

@lykakis

Parent epic: #337
Delivery step 5; depends on filesystem pending-message implementation #339 and stage contracts #344 (building on the memory baseline #338). Followed by file-backed DLR #340.

Goal

Implement file-backed selected-router and routed-work stages for the single-process outbound SMS profile, then enable and document all supported file-pending/file-or-memory stage combinations. Keep memory/memory/memory as the non-durable default after this issue is delivered; all durable profiles, including file/file/file, require explicit configuration.

Scope

  • Implement the file-backed selected-router backlog so a bounded eligibility/priority selection has a recoverable committed result. After restart, refill bounded runtime router queues from that result without repeating selection for already staged messages.
  • Implement recoverable recorded routing results (all copied destinations), per-destination progress and completion. Restore completed branches without resending them; unfinished branches may repeat provider submission.
  • Keep pending source records until all destinations and required existing PostgreSQL DLR handoffs are terminal. For a file-selected/memory-routed profile, retain selected records until safe completion so recovery reroutes rather than loses in-flight work.
  • Reconcile interrupted writes/transitions using stable IDs and a documented durable stage representation. A partial batch or interrupted route handoff must not be mistaken for completed work. File write/index/journal choices belong here, not in the architecture contract.
  • Use bounded in-memory prefetch; avoid loading the whole spool or rescanning every pending message on each refill. Define the selection index/scan strategy, recovery behavior and limits for a large backlog.
  • Preserve exclusive ownership of the local spool and process/container restart semantics when its persistent volume survives.

Configuration and supported profiles

Support pending=file with both router-queue=file|memory and routed-work=file|memory, plus the explicit memory/memory/memory compatibility profile from #339. Validate configurations at startup; do not silently substitute memory on storage failure. Fail closed on unavailable/corrupt committed state. All three outbound selectors default to memory when omitted. memory/memory/memory remains the non-durable default at the end of this issue; file-backed profiles are opt-in.

Verification and acceptance criteria

  • Forced restart tests cover selection publication, router dequeue, route handoff, worker dequeue, copied-route partial completion, provider ambiguity, and file-selected/memory-routed and memory-selected/file-routed recovery.
  • A durable selected batch survives restart without selection rerun; durable destinations resume without rerouting; volatile stages repeat the documented work without losing an acknowledged message.
  • Metrics and readiness reflect selected backends and unsafe transition failures without per-message/path labels; logs report recoverable counts and actionable errors without SMS payloads.
  • JVM/native container restart coverage with the spool volume retained; deployment and operational docs explain stage tradeoffs, volume requirements, and at-least-once behavior.

Out of scope

PostgreSQL stage implementations (#333), DLR backend replacement (#340/#334), durable retries and provider outcome checkpoints, multi-process spool sharing, host/power-loss guarantees, and exactly-once submission.

Related: architecture #338; pending contract #343; stage contracts #344; pending file implementation #339.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions