Repository navigation
Keep reconnecting after stalls and unexpected errors - #2
Open
jaltmayerpizzorno wants to merge 1 commit into
Open
jaltmayerpizzorno wants to merge 1 commit into
jaltmayerpizzorno wants to merge 1 commit into
Conversation
Two ways the client could end up disconnected for good, with every entity unavailable and nothing in the log: - The websocket connect, the Noise handshake and the initial Device.getStatus were unbounded. A controller that accepts the socket but never answers (e.g. while it is restarting) stalled the attempt, and with it the reconnect loop, forever. Bound each by PING_TIMEOUT; an unanswered getStatus is logged and the ping loop then catches a dead connection. - The reconnect loop only caught PoolsideConnectionError and PoolsideAuthError. Any other exception from a reconnect attempt (e.g. a malformed handshake message) or from the receive loop ended the loop silently. Log it and keep retrying with backoff, closing the old socket first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
After running fine for days, the Home Assistant integration lost its connection to my Attendant at 3:40 AM and never got it back. Every entity went unavailable, but the config entry still showed as loaded, and nothing was logged at warning level or above. Reloading the integration reconnected immediately, so the controller itself was reachable.
The connection was lost without a log entry, so I can't tell which path it took. Reading
client.py, though, there are two ways the client can end up permanently disconnected without logging anything:ws_connect, the Noise handshake (_read_binarywaits onws.receive()), and the initialDevice.getStatusinasync_refresh_statushave no timeout. A controller that accepts the socket but doesn't answer (for example while it restarts) stalls_async_connect_onceforever. Because the reconnect loop awaits that call, the loop stalls with it. The ping loop can't help, because it only starts after a successful connect._async_reconnect_looponly catchesPoolsideConnectionErrorandPoolsideAuthError. Any other exception ends the loop, and nothing ever retrieves the error because the task is kept on the client. Examples are a Noise error fromread_messageduring the handshake, ajson.loadsfailure in the receive loop, or a status listener raising. The client is left disconnected for good.Fix
PING_TIMEOUT, and raisePoolsideConnectionErroron timeout so the loop retries with backoff.Device.getStatusinasync_refresh_statusthe same way. A timeout is logged like the other errors there, and the ping loop then detects the dead connection._async_reconnect_loop, log and survive exceptions from the receive task and from reconnect attempts, and close the old socket before retrying.Tests
Four new tests. Each fails on
mainand passes with this change:test_async_connect_times_out_on_silent_controller: a real websocket server that accepts the connection but never answers the handshake.test_refresh_status_times_outtest_reconnect_loop_survives_unexpected_errortest_reconnect_loop_survives_failed_receive_loopThe full suite passes (36 tests), and
ruff check/ruff format --checkare clean.🤖 Generated with Claude Code