Skip to content

[Hold for next release] Shared queue worker loop in QueueBase - #584

Draft
niemyjski wants to merge 12 commits into
mainfrom
feature/queue-worker-loop
Draft

niemyjski wants to merge 12 commits into
mainfrom
feature/queue-worker-loop

Conversation

@niemyjski

@niemyjski niemyjski commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Moves queue worker error handling into QueueBase.StartWorker, used by InMemoryQueue and opt-in providers. Dequeue failures back off, handler failures attempt abandonment, and settlement failures keep the worker alive. Adds protected worker error counters without changing IQueue.

Validation at 7f62004c (2026-10-23): Release build and changed-file formatting passed; core tests 1,952 passed, 13 skipped, zero failures; three worker regressions passed. PR CI and push CI passed. Main be10785c is integrated; #579 is already merged.

Hold for next release. Current provider runtime validation against this exact core/TestHarness pair is pending; historical provider runs are not current proof. Inherited sample MessagePack/Aspire warnings remain; the MessagePack advisory is tracked separately in #570.

Verification and implementation details

Shared regressions cover dequeue failure, abandonment failure, and cancellation without worker errors. Existing provider candidates are Foundatio.Redis#192 (1b54fcbe), Foundatio.AWS#430 (e6596ab8), Foundatio.AzureStorage#101 (31ad05f1, current and legacy queues), and Foundatio.AzureServiceBus#105 (12989e19). Each registers all three shared tests. Isolated source-reference validation is pending the serialized resource slot.

The push workflow's CI package publication step succeeded; release publication was skipped. No release or main merge was performed.

Adds QueueEntryOptions.GroupId, IQueueEntry.GroupId and SharedQueueOptions.GroupIdResolver. The resolver runs before Enqueuing handlers, never mutates caller options, and TraceState no longer overwrites caller properties. Shared harness tests and docs included.
…claims

- QueueEntryOptions.GroupId stores empty as null, so an Enqueuing handler can no
  longer hand providers an empty group id; QueueBase only copies options when the
  resolver supplies a value, and keeps the caller's Properties comparer
- The unsupported-provider debug message no longer claims every provider stores
  the value, since older or custom providers may drop it
- Docs and skill no longer list Azure Service Bus as changing delivery: it sends
  SessionId but cannot receive from session-enabled queues yet
EnqueueAsync now copies the caller's options once (including Properties with its comparer) and applies the GroupId resolver plus Activity CorrelationId/TraceState to the copy before EnqueueImplAsync. Previously only the resolver path copied, so a reused options instance kept the first message's CorrelationId and TraceState, and Enqueuing handler edits leaked back to the caller. Providers can now validate the final options before Enqueuing handlers run.

Move provider-neutral tests into QueueTestBase: Enqueuing handler clearing GroupId, reused options isolation, and Enqueued entry GroupId.

Restore CI package publishing for non-main branches so provider PRs can consume a nightly that matches this branch.
… SQS docs

The shared test now checks only what every provider can honor: the caller's options are unchanged and a second enqueue does not fail. CorrelationId round-trips are already covered by other tests, so legacy Azure Storage no longer needs to skip it. Docs now say UniqueId is sent to SQS only on FIFO queues, and describe FIFO dead-letter naming and deletion when no dead-letter queue exists.
The existing retry test used a zero delay, so providers that re-send the message for a delayed retry (Service Bus) never exercised GroupId on that path. The same assertion now runs with a one second retry delay.
…on-safe error handling

Every provider copied its own StartWorkingImpl loop and the copies drifted: InMemory stopped on a null or failed dequeue, and SQS/Service Bus/Azure Storage stopped when dequeue or abandon threw. QueueBase.StartWorker gives every provider one loop: cancellation stops it without an error, dequeue failures back off exponentially with jitter, handler failures abandon the entry, and abandon/auto-complete run through the resilience policy without stopping the worker. Adds shared QueueTestBase fault-injection tests and docs.
@niemyjski
niemyjski force-pushed the feature/queue-worker-loop branch from 4ff67ef to cd4f359 Compare October 8, 2026 21:34
Base automatically changed from feature/queue-group-id to main October 14, 2026 01:54

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant