feat(storage): define outbound stage abstractions and memory baseline

[lykakis]

Parent epic: #337
Delivery step 1; prerequisite for focused contracts #343 and #344, filesystem work #339/#345, and future PostgreSQL work #333.

Problem

Current HTTP/SMPP ingress acknowledges process-local queue insertion. Router and worker queues destructively dequeue messages; retries, multipart state, and asynchronous provider work are process-local. A durable source spool alone preserves acceptance, but resuming an expensive selected batch or a recorded destination requires separately defined storage boundaries. The existing code also injects concrete InMemoryQueueProvider in ingress, routing, worker startup and resource wiring.

Goal

Define the backend-neutral outbound storage architecture and deliver its first working memory/memory/memory vertical slice. Add the initial interfaces/classes, lifecycle coordinator/wiring, global backend selection and startup validation, and memory-backed implementations using the existing non-durable behavior. This first slice makes the abstraction executable and easy to review; #343 and #344 deepen the contracts and tests before physical backends are implemented.

Architecture and initial interfaces

Define three globally selected, startup-only stage roles, with public configuration semantics equivalent to:

sendium.sms.pending.backend=memory|file|postgresql
sendium.sms.router-queue.backend=memory|file|postgresql
sendium.sms.routed-work.backend=memory|file|postgresql

The API should expose pending admission/completion, bounded selection into a router backlog, destination recording/completion, lifecycle coordination, backend identity/readiness, and actionable failures, without leaking SQL, paths, CBOR, or physical queue classes. Choose clear interfaces, backend registration/selection, and a single startup profile validator; avoid building speculative durable operations or codecs in this issue.

accepted pending -> selected for router queue -> routed to destination(s) -> terminal completion

• The accepted pending source remains authoritative until all destinations and required existing DLR handoff are terminal.
• Selection chooses bounded eligible, priority-aware work; when persisted in a later backend, its committed batch is recoverable without repeating expensive selection. Taking work from an in-memory router queue is not a separate persisted state in this single-process design.
• Routing records all chosen destinations, including copied routes. When persisted in a later backend, unfinished destinations resume without rerouting.
• On graceful shutdown, return messages taken for routing but not yet assigned a destination to the router queue. Messages already routed remain with their chosen worker queue; drain unfinished in-flight worker work back to that queue. Before shutdown completes, persist each queue when its configured backend is durable so restart resumes at the correct stage. Memory-backed stages retain their documented non-durable behavior.
• Document how HTTP/SMPP admission, multipart, routing/filters, worker dispatch, provider outcome and existing PostgreSQL DlrStorage cross the abstraction. Startup recovery, capacity/backpressure and ownership failures must be visible.

First working profile and rollout

• Implement/adapt the three memory stage components and wire them end-to-end through the new interfaces/coordinator while preserving current protocol behavior and the explicitly non-durable nature of this mode. Keep the initial implementation small; detailed pending/multipart/codec contract work is in feat(storage): define the outbound pending-message contract and CBOR envelope #343 and detailed selection/routed-work semantics in feat(storage): define selected-router and routed-work contracts #344.
• memory/memory/memory is the only supported profile at this step. Log the effective profile and a prominent non-durable warning at startup.
• Reject any requested combination that has no implementation, including file/file/memory, during startup with a clear error naming the unsupported profile and supported choices. Never replace a requested file/PostgreSQL backend with memory, even with a warning.
• Support future documented profiles only as their implementation issues land: feat(storage): implement filesystem pending-message store and ingress #339 adds file/memory/memory; feat(storage): persist selected-router and routed-work state on filesystem #345 adds the other planned file-pending/file-or-memory stage mixes as opt-in profiles. All three outbound selectors default to memory when omitted; memory/memory/memory remains the non-durable default throughout delivery. feat(storage): implement PostgreSQL outbound stage backends #333 adds explicitly supported PostgreSQL combinations. Backend changes with outstanding durable records need a drain/migration rule; they must never silently strand work.

Pending / selected router / routed work	Future restart guarantee
memory/memory/memory	Non-durable; available in this issue
file/memory/memory	Recover accepted work; reselect and reroute (#339)
file/file/memory	Recover selected batch; reroute (#345)
file/memory/file	Reselect not-yet-routed work; recover destinations (#345)
file/file/file	Recover selected batch and destinations (#345)

Durable profiles require surviving storage and remain at least once: provider submission before terminal completion may repeat. Retry count/delay, host/power-loss recovery and provider outcome checkpoints are not part of the initial lifecycle.

Deliverables and acceptance criteria

• A concise architecture description/state diagram and explicit supported-profile/guarantee table accompany reviewable interfaces and memory implementations.
• Ingress, routing and worker setup use the abstractions instead of depending directly on a concrete queue provider for backend choice; normal HTTP/SMPP -> router -> worker processing still works in the memory profile.
• Startup validation tests prove explicit memory starts and unsupported file/file/memory, partial-file and PostgreSQL requests fail before admission. Startup logs identify the effective non-durable profile; there is no silent fallback.
• The first slice has meaningful smoke/behavior tests for a successful message and copied routing without duplicating the detailed contract and codec suites owned by feat(storage): define the outbound pending-message contract and CBOR envelope #343/feat(storage): define selected-router and routed-work contracts #344.
• Responsibilities for source retention, bounded queue refill, stage transitions and later backend integration are clear enough for feat(storage): define the outbound pending-message contract and CBOR envelope #343/feat(storage): define selected-router and routed-work contracts #344 to refine independently.

Out of scope

Detailed pending/multipart admission contract, versioned CBOR schema and tests (#343); detailed durable batch selection/routed-work transition and failure-boundary tests (#344); physical file formats and backends (#339/#345), PostgreSQL schema/queries (#333), DLR backend redesign, multi-process leases, and exactly-once provider delivery.