Skip to content

Fix BackFiller settlement, NNTP session reuse, and connection pool lifecycle - #248

Merged
enteriaza merged 3 commits into
masterfrom
backfiller-fixes
Oct 25, 2026
Merged

enteriaza merged 3 commits into
masterfrom
backfiller-fixes

Conversation

@enteriaza

Copy link
Copy Markdown
Collaborator

Description

Summary

This PR improves BackFiller correctness and reliability by correcting terminal work settlement, preserving reusable NNTP sessions after selected ARTICLE responses, enforcing connection pool capacity throughout session disposal, and strengthening operational diagnostics.

It also adds regression coverage and a gated end-to-end stress harness.

Changes

1. Correct terminal work settlement

  • ACK and publish correlated failure replies for ProviderFailure, UnexpectedFailure, and RetentionRejected, preventing completed work from being repeatedly redelivered without a client reply.
  • Preserve requeue behaviour for Cancelled work.
  • Handle mandatory unroutable replies explicitly: raise RabbitMqPublicationUnroutableException and terminally NACK without requeue, recording ReplyUnroutableTerminal.

2. Improve NNTP ARTICLE response handling

  • Keep sessions reusable after ARTICLE responses with status codes 403, 500, 501, and 503, provided the connection remains synchronised.
  • Classify 451 as ArticleNotFound.
  • Preserve retirement behaviour for responses and failures that require the session to be retired.
  • Improve exception details to support diagnosis of provider failures.

3. Enforce connection pool capacity

  • Retain the pool's created-session count until disposal completes.
  • Wait for capacity notifications when the pool is full.
  • Base capacity decisions on _created rather than _live, preventing disposal in progress from prematurely freeing capacity.

4. Improve operational diagnostics

  • Add stable ConnectionId correlation.
  • Record structured session-retirement causes.
  • Enrich article-processing events with structured outcome information.

5. Expand verification and address CodeQL findings

  • Add regression tests covering settlement, NNTP response handling, and pool lifecycle behaviour.
  • Add an opt-in, gated end-to-end Stress harness.
  • Remediate CodeQL findings in the affected BackFiller scope, including Stress harness path handling and exception-handler analysis.
  • Update supporting documentation and architecture contracts.

Verification

  • Release x64 build succeeded.
  • 184 focused tests passed.
  • 1,448 ArticleWork/NNTP tests passed, excluding Stress/DualSoak.
  • Final local CodeQL analysis reported zero unsuppressed findings in the analysed BackFiller, Common, and BackFiller test scope. Fifteen findings remain under documented CodeQL suppressions.

Review notes

  • The settlement changes intentionally alter how terminal failures and unroutable replies are handled.
  • The Stress harness is opt-in and gated; it is not intended to run as part of ordinary tests unless explicitly enabled.
  • CodeQL results apply to the analysed scope; StorageService was not included in this audit.

ACK and reply ProviderFailure, UnexpectedFailure, and RetentionRejected
so completed work is not redelivered without a client reply. Handle
mandatory unroutable replies as terminal NACK without requeue.

Keep NNTP sessions reusable for ARTICLE 403/500/501/503, map 451 to
ArticleNotFound, and retain pool capacity until disposal completes.
Add connectionId and retirement-cause diagnostics, the gated E2E Stress
harness, regression tests, and related CodeQL remediations.
@enteriaza enteriaza self-assigned this Oct 25, 2026
@enteriaza enteriaza added the bug Something isn't working label Oct 25, 2026
Skip host-specific corpus assertions when the required Windows corpus
is unavailable. Replace generic Stress exception handlers with typed
handling, ensure reply-client cleanup in finally, and fix null-flow
assertions flagged by CodeQL.
Document all applicable production, test, helper, and Stress
declarations, including private and internal members. Correct stale
documentation and close missing XML documentation gaps.

Record the namespace documentation limitation (CS1587) in the
audit report. Release x64 builds pass with zero warnings or errors.
@enteriaza
enteriaza merged commit 4727fd3 into master Oct 25, 2026
7 checks passed
@enteriaza
enteriaza deleted the backfiller-fixes branch October 25, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants