Repository navigation
fix: recover a stale active job on every queue poll, not just reconnect - #16739
christian-byrne wants to merge 2 commits into
Conversation
executionStore.clearActiveJobIfStale() was only wired into useReconnectQueueRefresh (the WS-reconnect path). Routine queueStore.update() polling never called it, so a dropped terminal WebSocket message (execution_success/error/interrupted) left activeJobId, nodeProgressStates, and previews pinned to a job that has already left the Running/Pending queue snapshot, until the next reconnect. Call clearActiveJobIfStale() from queueStore.update() itself whenever the queue fetch succeeds, gated on the fresh Running/Pending snapshot alone — independent of whether the job has shown up in the concurrently-fetched history yet. getQueue and getHistory are fetched with Promise.allSettled and can disagree in the narrow window right after a job finishes; gating on history reintroduces the exact race this fixes. Follow-up to #11866 (comment) (jaeone94, review thread on src/stores/queueStore.ts). That PR was closed (superseded by #11951 for its executionStore.ts gating half); this is the queueStore.ts race jaeone94 flagged and glary-bot agreed to address in a follow-up, which never happened. <!-- authored-by:agent lane-act-fe-11866-followup-ea648b2b -->
|
Important Approval pendingCodeRabbit has no unresolved comments, but it has not reviewed the latest commit. Use the checkbox below to review the latest commit. CodeRabbit will approve the changes if it finds no blocking issues.
Comment |
🎨 Storybook: ✅ Built — View Storybook🎭 Playwright: ✅ 1912 passed, 0 failed · 2 flaky📊 Browser Reports
📦 Bundle: 9.68 MB gzip 🔴 +316 BDetailsSummary
Category Glance App Entry Points — 3.74 kB (baseline 3.74 kB) • ⚪ 0 BMain entry bundles and manifests
Status: 1 added / 1 removed Graph Workspace — 1.42 MB (baseline 1.42 MB) • ⚪ 0 BGraph editor runtime, canvas, workflow orchestration
Status: 2 added / 2 removed / 1 unchanged Views & Navigation — 124 kB (baseline 124 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces
Status: 13 added / 13 removed / 4 unchanged Panels & Settings — 621 kB (baseline 621 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens
Status: 10 added / 10 removed / 17 unchanged User & Accounts — 27.8 kB (baseline 27.8 kB) • ⚪ 0 BAuthentication, profile, and account management bundles
Status: 6 added / 6 removed / 5 unchanged Editors & Dialogs — 126 kB (baseline 126 kB) • ⚪ 0 BModals, dialogs, drawers, and in-app editors
Status: 7 added / 7 removed / 1 unchanged UI Components — 117 kB (baseline 117 kB) • ⚪ 0 BReusable component library chunks
Status: 7 added / 7 removed / 8 unchanged Data & Services — 3.59 MB (baseline 3.59 MB) • 🔴 +563 BStores, services, APIs, and repositories
Status: 13 added / 13 removed / 3 unchanged Utilities & Hooks — 560 kB (baseline 560 kB) • ⚪ 0 BHelpers, composables, and utility bundles
Status: 18 added / 18 removed / 18 unchanged Vendor & Third-Party — 18.1 MB (baseline 18.1 MB) • ⚪ 0 BExternal libraries and shared vendor chunks Status: 18 unchanged Other — 16.4 MB (baseline 16.4 MB) • ⚪ 0 BBundles that do not match a named category
Status: 72 added / 72 removed / 224 unchanged ⚡ Performance
|
christian-byrne
left a comment
There was a problem hiding this comment.
Moving the stale-active-job reconciliation from reconnect-only onto every queue poll is the right direction, but as written it widens a start race that previously only existed during reconnect. Details inline. The reconnect composable's explicit call still earns its place because update() skips reconciliation when the queue fetch fails, so I would keep it.
Full context for agent readers
Verified against the head commit c623990 with unit tests on src/stores/queueStore.test.ts, src/composables/useReconnectQueueRefresh.test.ts and src/stores/executionStore.test.ts (189 passed, 1 skipped). Code paths checked: executionStore.clearActiveJobIfStale calls resetExecutionState when the active id is absent from the snapshot; activeJobId is only set in handleExecutionStart; handleExecuting, handleExecuted and handleExecutionCached early-return when there is no active job, so a wrongly cleared running job disappears from the UI until execution_success. GraphView.vue's onStatus handler calls queuePendingTaskCountStore.update then queueStore.update; autoQueueService subscribes to the count store and calls app.queuePrompt when the pending count reaches zero in instant mode, so on every auto-queue iteration a GET /queue and a POST /prompt are issued from the same event. useQueuePolling (8 to 32 s), useJobMenu, ui.ts and app.ts also call update().
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #16739 +/- ##
==========================================
+ Coverage 80.97% 82.53% +1.56%
==========================================
Files 2451 2033 -418
Lines 125533 118065 -7468
Branches 31580 31551 -29
==========================================
- Hits 101648 97447 -4201
+ Misses 23373 20441 -2932
+ Partials 512 177 -335
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 422 files with indirect coverage changes 🚀 New features to boost your workflow:
|
|
Heads-up: this conflicts with #21282.
Full context for agent readers#21282 reads the queue through |
Follow-up to #11866, which was closed (superseded by #11951 for its
executionStore.tsactive-workflow-gating half). This PR is the other half: thequeueStore.tsrace jaeone94 flagged in review, which glary-bot agreed to fix in a follow-up that never landed before the PR was closed.Problem
getQueue()andgetHistory()are fetched concurrently viaPromise.allSettledinqueueStore.update(). In the narrow window right after a job finishes, a job can be absent from the freshRunning/Pendingsnapshot while not yet present in history. If the terminal WebSocket message (execution_success/execution_error/execution_interrupted) is also dropped, nothing clearsexecutionStore.activeJobId,nodeProgressStates, or pending previews — they stay pinned to a job that has already left the queue, until the next WS reconnect.executionStore.clearActiveJobIfStale()already exists and already solves exactly this (it clears the active job when the server's queue snapshot doesn't list it), but it was only wired intouseReconnectQueueRefresh— the reconnect path, not routine polling.Fix
Call
clearActiveJobIfStale(activeJobIds)from insidequeueStore.update()itself, right after the existingreconcileInitializingJobscall, gated only on the queue fetch succeeding — independent of history. This runs on every poll and on everystatus/execution_successWS-triggeredqueueStore.update()call (seeGraphView.vue'sonStatus/onExecutionSuccess), not just reconnect.Evidence
Added 3 new regression tests in
queueStore.test.ts(update() - stale active job recovery) covering: reconciliation fires with an empty snapshot even when history is also empty (the race itself), the active job is preserved when still present in the fresh snapshot, and reconciliation is skipped when the queue fetch itself fails. UpdateduseReconnectQueueRefresh.test.ts's call-count assertion sinceclearActiveJobIfStaleis now (harmlessly, idempotently) called from both the routine path and the existing explicit reconnect-path call.