Repository navigation
Conversation
Once shutdown has begun, new client connections completed the handshake and were closed with 1001 straight away. A client sees a successful dial, then a connection that drops before or during its first request; clients that verify a connection before subscribing can report it as healthy and then wait for a subscribe timeout. The docs already describe new WebSocket work as refused from the start of shutdown. Refuse the upgrade with 503 before the handshake instead. The check after the handshake stays for a shutdown that begins mid-handshake.
snowkide
force-pushed
the
fix/ws-refuse-upgrade-while-draining
branch
from
November 4, 2026 14:03
8c122c7 to
4736c3b
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Once shutdown has begun (
draining), a new WebSocket upgrade still completed the handshake and was then closed with1001straight away (ws_server.go, check afteractiveWsConns.Store). To a client that is a successful dial followed by a connection that drops before or during its first request.That is worse than a refusal for clients that verify a connection before using it: the dial and check pass, the client reports the endpoint as healthy, and its next call (for example
eth_subscribe newHeads) fails or waits for its own timeout before the client reconnects. A load balancer can keep sending new connections to a shutting-down instance for a short while (its endpoint update has to propagate), so this happens on ordinary rolling restarts, not only in races.The docs already describe new WebSocket work as refused from the start of shutdown. This makes the upgrade path match:
handleWebSocketrefuses the upgrade with503before the handshake whiledrainingis set, the same flag that makes the healthcheck return503.1001when shutdown begins, in-flight requests still get up towaitBeforeShutdown.docs/pages/operation/websocket.mdx: one sentence on the503.No config change, no change to shutdown timing.
Test plan
TestWebSocket_ShutdownClosesPromptlyAfterInflightRequests: its last step now expects the late dial to fail the handshake (ErrBadHandshake) with HTTP503. It fails without the fix (handshake completes with101) and passes with it (also with-race, 3 runs).go test ./erpc/ -run 'WebSocket|Ws|HealthCheck|Shutdown|Drain'passes (2 runs).TestWebSocket_UnsubscribeIsScopedToConnectionis flaky onmainas well (it counts process-wide connection gauges left by earlier tests); it failed once in one of 3 batch runs with this change and once when run alone on unchangedmain.go vet ./erpc/,gofmt.