Skip to content

feat(storage): define selected-router and routed-work contracts #344

Description

@lykakis

Parent epic: #337
Delivery step 3; depends on the architecture and working memory baseline #338 and pending-message contract #343.
Blocks filesystem router/routed-state implementation #345 and PostgreSQL stage implementations #333.

Goal

Define backend-neutral contracts and focused behavioral tests for a bounded selected-router backlog and recorded routed destinations. Their backends are globally configurable, independently from pending-message storage, as specified in #338. Refine and test the initial interfaces and memory variants delivered by #338; physical file and PostgreSQL implementations belong to backend issues.

Selected-router backlog

  • Select an eligible, priority-aware bounded batch of still-pending messages and durably record the selected membership when the router-queue backend is durable. Define the selection inputs, ordering requirements within a batch, and what a committed batch means; do not imply strict global priority order among batches/new arrivals.
  • Refill a bounded in-memory router scheduling queue from selected records. A router thread taking a message from that runtime queue is not a distinct persisted stage in the single-process design.
  • On restart, restore committed selected items without repeating the expensive selection; when this stage is memory-backed, reselect eligible pending items that have no durable later-stage state.
  • Interrupted batch publication must have a defined recoverable outcome (committed membership or eligible for selection again) without dropping items or treating incomplete work as committed.

Routed destinations

  • Record the full routing result, including copied-route destinations, before allowing a durable selected record to be retired. Support per-destination terminal completion; keep the source pending record until all required branches and DLR handoffs complete.
  • A durable routed-work backend restores recorded destinations without rerouting them; a memory-backed backend may reroute after restart. Provider-generated multipart PDUs may repeat after a crash when provider outcome is not checkpointed.
  • When the selected state is durable but routed work is memory-backed, retain enough selected state until terminal completion to recover safely. When the selection state is memory-backed but routed work is durable, recovery must avoid reselecting already-routed work.
  • Define stable IDs and precedence/reconciliation for interrupted transitions across backends; never assume a file/database or file/memory transition is one atomic transaction. Retain the authoritative pending source until safe completion.

Tests and acceptance criteria

  • Reusable tests for batch selection, bounded refill, crash before/after selection commit, router dequeue, route publication, copied-route partial completion, and terminal deletion; exercise the existing memory implementation from feat(storage): define outbound stage abstractions and memory baseline #338 where applicable.
  • Tests for file/memory stage combinations using contract fixtures or fault-injectable fakes; test physical crash recovery in the backend issues.
  • Runtime queues remain bounded projections, not authoritative sources of work. Stage failures prevent unsafe completion; no silent fallback changes the configured restart guarantee.
  • The interfaces describe lifecycle operations and error behavior without exposing SQL, paths, CBOR, or physical queue internals.

Out of scope

Reimplementing the memory baseline from #338, file layout/journaling, PostgreSQL queries and indexes, multi-process leases/fencing, durable retry timing/counters, exactly-once provider submissions, and per-provider-PDU outcome checkpoints.

Related: epic #337; architecture #338; pending/CBOR #343; filesystem pending #339; filesystem stages #345; PostgreSQL #333.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions