Skip to content

test: fix WorkflowTab hover-delay flake from wall-time fake timers - #22038

Merged
christian-byrne merged 1 commit into
mainfrom
fix/workflowtab-hover-flake
Oct 27, 2026
Merged

christian-byrne merged 1 commit into
mainfrom
fix/workflowtab-hover-flake

Conversation

@christian-byrne

Copy link
Copy Markdown
Contributor
  • WorkflowTab hover-delay tests flaked under CI load
  • Cause: global fake timers use shouldAdvanceTime: true, so wall time leaks into the 200ms/100ms delays
  • Fix: those tests pin shouldAdvanceTime: false
Full context for agent readers

Failure

src/components/topbar/WorkflowTab.test.ts > WorkflowTab - workflow status indicator > delays hover preview without moving focus and dismisses after leaving failed on #21341 (job https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/40394181583/job/121824507992):

  • first attempt: Unable to find an accessible element with the role "dialog" at the "still open after 99ms" check (the hide timer had already fired)
  • both retries: expect(element).not.toBeInTheDocument() at the "not open after 199ms" check (the show timer had already fired)

Root cause

vite.config.mts sets fakeTimers: { now: TEST_SYSTEM_TIME, shouldAdvanceTime: true }, and vitest.timer.setup.ts calls vi.useFakeTimers() before every test. With shouldAdvanceTime, the fake clock also moves forward with real elapsed time. The test steps the popover's useTimeoutFn delays (200ms show, 100ms hide) one millisecond at a time with vi.advanceTimersByTimeAsync(199) + (1) and (99) + (1). Once a loaded runner spends more than ~1ms of wall time between those steps, a timer fires a step early. Either assertion can then fail, which matches both CI errors. On an idle machine the drift stays under 1ms, so it passed locally.

Fix

The workflow status indicator describe's beforeEach calls vi.useFakeTimers({ shouldAdvanceTime: false }). Vitest merges that with the project fakeTimers config, so now is kept. Time moves only when the test advances it. This is the same pattern other suites already use (e.g. src/stores/assetDownloadStore.test.ts, src/components/queue/job/JobAssetsList.test.ts). Only the two hover/click tests in that describe depend on timers. The other tests in it do not care.

Alternatives not taken:

  • loosening the assertions (e.g. advanceTimersByTimeAsync(150)): hides the boundary the test exists to check
  • flipping shouldAdvanceTime globally: much wider blast radius, since other suites rely on it

Evidence

Both hover tests temporarily run with { repeats: 50 }, -t "hover|click-dismissed", on a shared 8-core host at load average 26–36. "Under load" adds 10 extra node -e 'for(;;){}' busy loops. Each run is 100 executions.

variant condition result
before idle-ish 1 failed (:153, :165), same lines as CI
before under load, 2 runs 2/2 failed (:153, :165)
after idle-ish 2 passed (100/100)
after under load, 3 runs 3/3 passed (300/300)
after idle, { repeats: 200 } on the hover test passed (200/200)

The full file passes (24/24). oxfmt, oxlint and pnpm typecheck are clean.

The frontend fake timers run with shouldAdvanceTime, so real elapsed
time also advances the clock. The hover tests step the 200ms show and
100ms hide delays to the millisecond; on a loaded runner a timer fired
one step early. Pin shouldAdvanceTime: false for that describe.
@christian-byrne
christian-byrne requested a review from a team October 27, 2026 02:32
@mrseek-reviews
mrseek-reviews Bot requested a review from DrJKL October 27, 2026 02:32
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 10/27/2026, 02:34:15 AM UTC

Links

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

No verdict is available. Open the workflow run to confirm the check was intentionally skipped, or fix the collection failure and re-run it.

Findings and check details

The matrix runs on same-repository version-bump PRs with relevant changes, main-branch pushes, and manual dispatches. This candidate does not qualify, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

Findings and check details

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since b0d816d3216e08ead715c73607f8b40f7388d566

Full docs: ADR-DEVEX-LINT-0015

🏛️ Architecture: ✅ Passed

Findings and check details

Warnings (210)

  • workspace path matches no source files: src/platform/workspace/fixtures/**. Remove or update it in docs/architecture/domains/records/workspace.domain.json.
  • src/components/graph/selectionToolbox/LinkSelectionActions.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/searchbox/v2/filterChip.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/Popover.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/card/Card.stories.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/card/Card.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/card/CardFooter.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/card/CardHeader.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/card/card.variants.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/widgetHistory/WidgetHistoryButtons.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/widgetHistory/WidgetHistoryButtons.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/graph/menuOption.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/useLiveGraphNode.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/useLiveGraphNode.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/useWidgetValueHistory.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/useWidgetValueHistory.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/widgetHistoryScope.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/widgetHistoryScope.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/litegraphServiceAccess.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/nodeDef/ComfyNodeDefImpl.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/widgets/dynamicGroupDiagnostics.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/widgets/dynamicGroupDiagnostics.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/widgets/dynamicGroupEnforcementBoundary.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/legacyUI/attachLegacyUI.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/selection/selectionIntent.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/agentPanel.historyIdentity.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/cloudBadges.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/dynamicPrompts.serializeValue.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/electronAdapter.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/index.cloud.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/index.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/cameraInfo.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/findNumericWidget.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/load3dHost.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/load3dHost.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/load3dInstances.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/load3dLazy.retry.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/load3dSettings.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/load3dWidgets.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/load3dWidgets.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/reportLoadModelOutcome.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/reportLoadModelOutcome.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/slotDefaults.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/touchCanvasAccess.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/touchCanvasAccess.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/touchPointerGuards.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/i18n.literalMessages.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/ContextMenu.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/LGraphCanvas.startRendering.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/LiteGraphGlobal.cloneObject.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • …and 160 more
Reproduce locally
pnpm architecture:check

Full docs: Frontend domain catalog

@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between b0d816d and 34c0149.

Files selected for processing: 1
  • src/components/topbar/WorkflowTab.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

Walkthrough

The workflow status test setup now enables Vitest fake timers with automatic time advancement disabled before resetting the mocked workflow status.

Changes: Workflow status tests

Layer / File(s) Summary
Configure fake timers
src/components/topbar/WorkflowTab.test.ts
The beforeEach setup enables fake timers with automatic time advancement disabled before resetting the mocked workflow status.

Priority: Low

Estimated code review effort: 1 (Trivial) | ~3 minutes

Change: Bug fix

Possibly related issues

  • Comfy-Org/ComfyUI_frontend#15782: This change also configures Vitest fake timers and disables automatic time advancement, which may relate to the issue’s reported timing measurements.

Suggested reviewers: drjkl

Merge Risk: Minimal · up to 34c01

The test timer setup is appropriate for deterministic hover-delay checks and is ready to merge.

Pre-merge checks: Passed 6
Check name Status Explanation
Title check Passed The title clearly identifies the fix for the WorkflowTab hover-delay test flake caused by wall-time fake timers.
Description check Passed The description explains the failure, root cause, fix, alternatives, and verification results. It does not use the template headings exactly, but it includes the required summary and verification info…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
End-To-End Test Suggestion For User-Facing Changes Passed The pull request changes only src/components/topbar/WorkflowTab.test.ts and updates test timer setup. It does not change application behavior or add a user-facing feature. This check encourages, but…
Adr Compliance For Entity/Litegraph Changes Passed The changed-file list contains only src/components/topbar/WorkflowTab.test.ts. It does not modify src/lib/litegraph/, src/ecs/, or graph-entity-related files covered by this check.
Full details: Description check

Explanation

The description explains the failure, root cause, fix, alternatives, and verification results. It does not use the template headings exactly, but it includes the required summary and verification information.

Full details: End-To-End Test Suggestion For User-Facing Changes

Explanation

The pull request changes only src/components/topbar/WorkflowTab.test.ts and updates test timer setup. It does not change application behavior or add a user-facing feature. This check encourages, but does not require, a Playwright test.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Oct 27, 2026
@codecov

codecov Bot commented Oct 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main   #22038      +/-   ##
==========================================
- Coverage   87.71%   86.66%   -1.06%     
==========================================
  Files        2894     3004     +110     
  Lines      158374   157718     -656     
  Branches    43310    40391    -2919     
==========================================
- Hits       138922   136681    -2241     
- Misses      18612    20317    +1705     
+ Partials      840      720     -120     
Flag Coverage Δ
billing-web-unit ?
unit 85.70% <ø> (-0.70%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 576 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@christian-byrne
christian-byrne added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit 0aa64be Oct 27, 2026
90 checks passed
@christian-byrne
christian-byrne deleted the fix/workflowtab-hover-flake branch October 27, 2026 04:17
@christian-byrne christian-byrne added core/1.59 Backport PRs for core 1.59 cloud/1.59 Backport PRs for cloud 1.59 needs-backport Fix/change that needs to be cherry-picked to the current feature freeze branch labels Oct 29, 2026
@comfy-pr-bot

Copy link
Copy Markdown
Member

@christian-byrne Successfully backported to #22257

@comfy-pr-bot

Copy link
Copy Markdown
Member

@christian-byrne Successfully backported to #22258

comfy-pr-bot added a commit that referenced this pull request Oct 29, 2026
…l-time fake timers (#22257)

Backport of #22038 to `core/1.59`

Automatically created by backport workflow.

Co-authored-by: Christian Byrne <cbyrne@comfy.org>
comfy-pr-bot added a commit that referenced this pull request Oct 29, 2026
…ll-time fake timers (#22258)

Backport of #22038 to `cloud/1.59`

Automatically created by backport workflow.

Co-authored-by: Christian Byrne <cbyrne@comfy.org>
@comfy-pr-bot comfy-pr-bot added released:cloud PR has been released to cloud and removed needs-backport Fix/change that needs to be cherry-picked to the current feature freeze branch labels Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cloud/1.59 Backport PRs for cloud 1.59 core/1.59 Backport PRs for core 1.59 released:cloud PR has been released to cloud risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants