Report Studio's terminal websocket errors instead of reconnecting silently - #1959
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Improves Studio job-log error reporting and terminal cleanup.
Changes:
- Surfaces terminal WebSocket handshake and server-frame errors.
- Extracts reconnect backoff and clears stale banners.
- Adds regression tests for error and interruption paths.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/datachain/studio.py |
Handles terminal frames and reconnect cleanup. |
src/datachain/remote/studio.py |
Converts 401/403 handshakes into actionable errors. |
tests/test_cli_studio.py |
Tests handshake, frame, retry, and interruption behavior. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
shcheklein
reviewed
Aug 28, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
dreadatour
commented
Aug 28, 2026
| return received, last_log_id | ||
|
|
||
|
|
||
| async def _wait_before_reconnect(retry_count: int) -> str: |
Contributor
Author
There was a problem hiding this comment.
Note: move out from show_logs_from_client to avoid # noqa: PLR0915.
Deploying datachain with
|
| Latest commit: |
014b3a2
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://c3a97072.datachain-2g6.pages.dev |
| Branch Preview URL: | https://fix-job-logs-terminal-errors.datachain-2g6.pages.dev |
amritghimire
approved these changes
Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
datachain job logs <id>against a team name the server does not accept printed nothing but a reconnect countdown and never said why — 15 attempts with backoff capped at 60s, so roughly ten minutes of silence before "Failed to reconnect". In practice you interrupt it first:Studio does report the problem over the log-streaming websocket. The client dropped it, in two independent places.
Refused handshake. For an unknown team, a team you cannot access, or an invalid token, Studio refuses the websocket handshake with HTTP 403.
websocketsraises a handshake exception, whichtail_job_logscaught in a blanketexcept (WebSocketException, OSError)and turned into alogger.debugplus a barereturn— indistinguishable from a normal end-of-stream.Error frame. For a malformed job id, or a job you cannot access, Studio accepts the connection, sends
{"message": "Job ID is incorrect or not found"}, and closes with code 4404. That frame was yielded into a branch chain inshow_logs_from_clientthat only handleslog_blobs/logs/job, so it fell through every branch and was discarded.What changed
tail_job_logsraisesDataChainErrorwhen a refused handshake carries 401 or 403, naming the team. Every other failure — a dropped connection, a 5xx from a gateway mid-deploy — still returns and lets the caller reconnect, sojob runkeeps its REST-polling fallback.show_logs_from_clientraises the server's own message when it sees the terminal{"message": ...}frame.reconnecting in Ns...countdown._wait_before_reconnect, next to the existing_print_reconnect_msgand_clear_linehelpers. Behavior is identical — the sleep still uses the pre-increment retry count and the debug line still reports the same attempt number.The error message names the team and the token as possible causes, because Studio deliberately answers the same way whether a team does not exist or you simply cannot reach it. The client cannot tell those apart and should not pretend to.
🤖 Co-authored by Claude Code and ChatGPT