Skip to content

Log dropped subscription events#1271

Merged
brandur merged 4 commits into
riverqueue:masterfrom
peter941221:peter/subscription-drop-observability
Jun 3, 2026
Merged

Log dropped subscription events#1271
brandur merged 4 commits into
riverqueue:masterfrom
peter941221:peter/subscription-drop-observability

Conversation

@peter941221
Copy link
Copy Markdown
Contributor

  1. Failure mechanism

Subscription delivery already uses non-blocking sends and the public docs already warn that slow consumers can lose events. Right now, when a subscriber channel fills up, subscription_manager drops the event silently. That leaves you with no signal that a subscription fell behind or which kind of event was lost.

  1. Semantic change

This adds a warn log when a job or queue subscription event is dropped because the subscriber buffer is full.

The log includes event_kind so you can tell what was lost.

  1. Why this design

I kept the existing delivery semantics. This does not add retries, change buffering, or block producers behind a slow subscriber. It only turns a known lossy path into something observable.

  1. Testing

  2. Added Test_SubscriptionManager/LogsDroppedQueueEvents in subscription_manager_test.go.

  3. Ran go test . -run Test_SubscriptionManager/LogsDroppedQueueEvents -count=1.

  4. Ran go test . -run TestDoesNotExist -count=1 as a compile/load smoke check.

@peter941221 peter941221 marked this pull request as ready for review June 3, 2026 11:54
Copy link
Copy Markdown
Contributor

@brandur brandur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this isn't a bad idea. Want to add a changelog?

@peter941221
Copy link
Copy Markdown
Contributor Author

Added a changelog entry under Unreleased -> Changed.

@brandur brandur merged commit 1ca3420 into riverqueue:master Jun 3, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants