Repository navigation
fix(rpc): stop resubscribing subscription IDs as queries after reconnect - #1623
Draft
Claudius-Maginificent wants to merge 2 commits into
Draft
Claudius-Maginificent wants to merge 2 commits into
Claudius-Maginificent wants to merge 2 commits into
Conversation
wsEvents stored each event's subscription ID as an extra key in its subscription map, and resubscribed every key on reconnect except one equal to the subscription's latest ID. Tenderdash sends events without a subscription ID, so the empty key was resubscribed as a query: the node rejected it, and each rejection made the event loop resubscribe again a second later, stalling event delivery. Upstreams sending fresh IDs per session leaked one key per reconnect, resubscribed the same way. Nothing looks subscriptions up by ID, so drop the ID keys; the alias write was also the event loop's only map write under a read lock. <sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
|
🕓 Review not started yet because this PR is a draft.
Commit 69cb77a. Normal review starts when eligible; priority review starts as soon as a slot is available. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR: After a WebSocket reconnect, the HTTP RPC client re-sent a bogus empty subscription that the node rejects, and then retried every second forever, stalling event delivery and filling logs with
WS error.User story
As a developer using
rpc/client/httpto subscribe to node events, directly or through the light client proxy, I want my subscriptions restored cleanly when the connection to the node drops and comes back, so events keep flowing without errors.Scenario
Base flow
An application subscribes to node events (for example new blocks) through the HTTP RPC client. The node restarts or the network blips, and the client reconnects.
Actual behavior
Besides each query, the client stored the subscription ID from incoming events as an extra subscription key. Tenderdash sends events with an empty subscription ID, so after the first event the client tracked an extra key
"". On every reconnect it sentsubscribe "", which the node rejects (failed to parse query: offset 0: EOF). Each rejection makes the client wait a second and resubscribe everything again, so the loop never ends: events are delayed andWS erroris logged every second. Upstreams that do send subscription IDs leaked one entry per reconnect and later resubscribed those IDs as queries.Expected behavior
After a reconnect, the client resubscribes exactly the queries it was asked to subscribe to, and events keep arriving with no errors.
Detailed discussion
What was done
rpc/client/http/ws.go: subscriptions are tracked by query only. Removed theidfield, the ID alias written byeventListener(also its only map write underRLock), the alias delete inUnsubscribe, and the alias skip inredoSubscriptionsAfter. Nothing looked subscriptions up by ID: event delivery is keyed byresult.Query, and subscribers still receiveSubscriptionIDin each event unchanged. No exported API changes.rpc/client/http/ws_reconnect_test.go: regression test against the node's real subscribe handler (realEventBusandrpccoreroutes behindrpcserver.NewWebsocketManager, with the node'sOnDisconnectcleanup). Across three sessions with two forced reconnects it requires that each session still delivers events and that noWS erroris logged.Testing
dev(loggedfailed to parse query: offset 0: EOF, or no resubscription on the new session) and passes repeatedly with the fix under-race, including a separate check against an upstream that assigns a new ID per session../rpc/client/http/,./rpc/client/,./light/proxy/and./light/rpc/pass with-race;gofmt,go vetandgolangci-lint --new-from-rev=origin/devare clean.Breaking changes
None.
Prior work
rpc/client/http/ws.goand still contains the alias logic. The two will conflict; whichever merges second must drop the aliases rather than restore them. The new test lives in its own file and will catch a regression.🤖 Co-authored by Claudius the Magnificent AI Agent
🤖 Generated with Claude Code