Skip to content

test: pin the fake clock in the useSubscription deadline-report test - #22209

Merged
christian-byrne merged 1 commit into
mainfrom
matt/be-24342-pin-fake-clock-deadline-test
Oct 30, 2026
Merged

christian-byrne merged 1 commit into
mainfrom
matt/be-24342-pin-fake-clock-deadline-test

Conversation

@mattmillerai

Copy link
Copy Markdown
Member

ELI5

One billing test sometimes failed the first time it ran even though the code it checks works fine. Its fake clock kept moving forward on its own, so a deadline check fired one step early. This change stops that clock from auto-advancing for that one test, so it gives the same result every run.

Summary

The shared Vitest setup installs fake timers with shouldAdvanceTime: true. sinon's auto-advance interval calls tick(20) every 20 real ms. When that tick lands during a yield inside vi.advanceTimersByTimeAsync, the advance target moves 20 ms further, so 43_000 + 556_999 puts the clock at 600,019. The 600,000 ms deadline wake-up then fires one assertion early. A cold worker running this test alone always takes longer than 20 real ms, so the failure showed up deterministically on the first run.

The fix is test-only: re-install vi.useFakeTimers({ shouldAdvanceTime: false }) as the first statement of reports at the deadline without another lifecycle event. This is the repo's established opt-out (see useWorkspaceAuth.test.ts, JobAssetsList.test.ts, ModelLibrarySidebarTab.test.ts). The global afterEach in vitest.timer.setup.ts already restores real timers. The opt-out is scoped to this one test on purpose: four other tests in the file rely on auto-advance for setTimeout flushes and vi.waitFor. Production useSubscription.ts and vite.config.mts are unchanged.

Verification

  • Single test, cold worker, --retry=0: 10/10 pass
  • Full file, CI=true --retry=0: 96/96 pass
  • Full file, --retry=0: 96/96 pass
  • Revert check: without the line, the single test failed 2 of 3 cold runs
  • oxfmt, oxlint, pre-commit typecheck:app: clean
  • Doc-drift sweep: 6 references checked, 0 updated (all generic useFakeTimers / useSubscription mentions; none describe this test's timer setup)

Provenance

  • Authored by: agent-work loop
  • Verified:
    • Tests: 10/10 isolated deadline-test runs, then 96/96 for the full file (CI and non-CI), all with --retry=0
    • Mutation check: removing the opt-out brings back the first-run failure (2 of 3 runs)
    • Format/lint/typecheck: clean (oxfmt, oxlint, pre-commit typecheck:app)
    • Not run locally: full pnpm test:unit, pnpm knip, full CI matrix
  • Deviations: the proposed two-line explanatory comment above the opt-out was left out. This follows the repo's no-gratuitous-comments rule and matches the comment-free precedent tests listed above.

Residual

None. This only changes the one test it targets. The underlying shouldAdvanceTime interaction may still make other tests flaky when they assert on exact fake-clock boundaries; this PR did not sweep the corpus for them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattmillerai mattmillerai added agent-coded Opened by the agent-work code loop cursor-review Trigger multi-model Cursor agent review on this PR labels Oct 29, 2026
@mattmillerai
mattmillerai requested a review from a team October 29, 2026 04:20
@mattmillerai mattmillerai added agent-coded Opened by the agent-work code loop cursor-review Trigger multi-model Cursor agent review on this PR labels Oct 29, 2026
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 10/29/2026, 04:22:02 AM UTC

Links

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

No verdict is available. Open the workflow run to confirm the check was intentionally skipped, or fix the collection failure and re-run it.

Findings and check details

The matrix runs on same-repository version-bump PRs with relevant changes, main-branch pushes, and manual dispatches. This candidate does not qualify, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

Findings and check details

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since ba3a8928e7a41b10cbea4f227c6ef476821c9462

Full docs: ADR-DEVEX-LINT-0015

🏛️ Architecture: ✅ Passed

Findings and check details

Warnings (273)

  • billing path matches no source files: src/platform/billing/composables/activateBillingScope.testUtils.ts. Remove or update it in docs/architecture/domains/records/billing.domain.json.
  • error-handling path matches no source files: src/utils/errorUtil.ts. Remove or update it in docs/architecture/domains/records/error-handling.domain.json.
  • graph-model path matches no source files: src/types/linkTopology.ts. Remove or update it in docs/architecture/domains/records/graph-model.domain.json.
  • graph-model path matches no source files: src/types/nodeState.ts. Remove or update it in docs/architecture/domains/records/graph-model.domain.json.
  • linear-mode path matches no source files: src/utils/appMode.ts. Remove or update it in docs/architecture/domains/records/linear-mode.domain.json.
  • link-connection path matches no source files: src/lib/litegraph/src/infrastructure/LinkConnectorEventMap.ts. Remove or update it in docs/architecture/domains/records/link-connection.domain.json.
  • node-widgets path matches no source files: src/types/widgetState.ts. Remove or update it in docs/architecture/domains/records/node-widgets.domain.json.
  • subgraphs path matches no source files: src/lib/litegraph/src/infrastructure/SubgraphEventMap.ts. Remove or update it in docs/architecture/domains/records/subgraphs.domain.json.
  • vue-node-widgets path matches no source files: src/types/positionConfig.ts. Remove or update it in docs/architecture/domains/records/vue-node-widgets.domain.json.
  • workspace path matches no source files: src/platform/workspace/fixtures/**. Remove or update it in docs/architecture/domains/records/workspace.domain.json.
  • src/components/TopbarActions.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/searchbox/v2/filterChip.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/videoConcat/VideoConcatClipStrip.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/videoConcat/VideoConcatClipTile.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/videoConcat/VideoConcatPanel.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/videoConcat/VideoConcatPanel.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/videoConcat/VideoConcatPlaybackBar.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/videoConcat/WidgetVideoConcat.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/videoConcat/WidgetVideoConcat.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/videoConcat/__fixtures__/clipTiles.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/widgetHistory/WidgetHistoryButtons.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/widgetHistory/WidgetHistoryButtons.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/graph/menuOption.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/useLiveGraphNode.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/useLiveGraphNode.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/useWidgetValueHistory.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/useWidgetValueHistory.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/concatSegmentPreviews.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/concatSegmentPreviews.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/linkedConcatSlots.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/syncConcatOrderWithLinks.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/syncConcatOrderWithLinks.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/useClipStripReorder.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/useSequencePlayback.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/useSequencePlayback.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/useVideoConcatClips.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/useVideoConcatClips.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/useVideoConcatOrder.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/useVideoConcatOrder.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/videoConcatOrder.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/videoConcatOrder.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/videoSequence.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/videoConcat/videoSequence.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/widgetHistoryScope.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/widgetHistoryScope.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/litegraphServiceAccess.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/nodeDef/ComfyNodeDefImpl.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/widgets/autogrowConvertToSubgraph.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/widgets/dynamicGroupDiagnostics.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/widgets/dynamicGroupDiagnostics.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • …and 223 more
Reproduce locally
pnpm architecture:check

Full docs: Frontend domain catalog

@github-actions github-actions Bot added the risk:xhigh PR risk grade (advisory shadow check; grader-owned) label Oct 29, 2026
@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between ba3a892 and 66a6620.

Files selected for processing: 1
  • src/platform/billing/subscription/composables/useSubscription.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

Walkthrough

The checkout deadline-reporting test now disables automatic fake-timer advancement before exercising the deadline.

Changes: Deadline test

Layer / File(s) Summary
Deadline test timer setup
src/platform/billing/subscription/composables/useSubscription.test.ts
The test uses fake timers with automatic time advancement disabled.

Priority: Low

Estimated code review effort: 1 (Trivial) | ~3 minutes

Change: Bug fix

Possibly related issues

  • Issue 15782 — Both concern the repository’s fake-timer setup. This change disables automatic advancement for the deadline test, but does not address the issue’s documentation work.

Possibly related PRs

Suggested reviewers: drjkl

Merge Risk: Minimal · up to 66a66

This scoped test-only timer change is ready to merge.

Pre-merge checks: Passed 6
Check name Status Explanation
Title check Passed The title names the test file and the specific change (pinning the fake clock in the deadline-report test). It matches the single-line change that sets fake timers with shouldAdvanceTime disabled, so …
Description check Passed The description follows the template. It has an ELI5 section and a Summary that explains the cause (shared Vitest setup uses shouldAdvanceTime: true, which moves the clock during an async advance) and…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
End-To-End Test Suggestion For User-Facing Changes Passed The pull request changes only src/platform/billing/subscription/composables/useSubscription.test.ts to configure fake timers for a test. It does not fix or add a user-facing frontend feature, and it…
Adr Compliance For Entity/Litegraph Changes Passed The changed file is src/platform/billing/subscription/composables/useSubscription.test.ts. It is not under src/lib/litegraph/, src/ecs/, or a graph-entity-related path. The custom check does not…
Full details: Title check

Explanation

The title names the test file and the specific change (pinning the fake clock in the deadline-report test). It matches the single-line change that sets fake timers with shouldAdvanceTime disabled, so it is accurate and specific. The 68-character length is acceptable, and the conventional 'test:' prefix fits the change.

Full details: Description check

Explanation

The description follows the template. It has an ELI5 section and a Summary that explains the cause (shared Vitest setup uses shouldAdvanceTime: true, which moves the clock during an async advance) and the fix. The Verification section gives concrete test commands, run counts, a revert check, and lint/format/typecheck results, and it lists what was not run. The template's Summary asks for 1-2 sentences, and this Summary is longer. The optional issue-link line is absent, which is acceptable because the template marks it as optional. The Residual section notes a remaining risk for other tests.

Full details: End-To-End Test Suggestion For User-Facing Changes

Explanation

The pull request changes only src/platform/billing/subscription/composables/useSubscription.test.ts to configure fake timers for a test. It does not fix or add a user-facing frontend feature, and it does not require an end-to-end test.

Full details: Adr Compliance For Entity/Litegraph Changes

Explanation

The changed file is src/platform/billing/subscription/composables/useSubscription.test.ts. It is not under src/lib/litegraph/, src/ecs/, or a graph-entity-related path. The custom check does not apply.

Finishing Touches: 💡 1
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

✅ No high-signal findings.

Panel: 6/6 reviewers contributed findings.

@codecov

codecov Bot commented Oct 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main   #22209      +/-   ##
==========================================
- Coverage   87.46%   87.36%   -0.11%     
==========================================
  Files        2923     2815     -108     
  Lines      158649   155262    -3387     
  Branches    43467    42295    -1172     
==========================================
- Hits       138760   135639    -3121     
+ Misses      18968    18868     -100     
+ Partials      921      755     -166     
Flag Coverage Δ
billing-web-unit ?
unit 86.15% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 108 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dante01yoon dante01yoon left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The root cause holds up: the global vi.useFakeTimers() in vitest.timer.setup.ts auto-advances the clock, which can push an exact-boundary assertion (556_999 then +1) past the 600,000 ms deadline. Scoping shouldAdvanceTime: false to this one test matches the existing opt-outs (JobAssetsList.test.ts, GraphCanvas.test.ts), and the global afterEach already restores real timers. The revert check in the description (2 of 3 cold runs fail without the line) is enough evidence that the flake is real.

@christian-byrne

Copy link
Copy Markdown
Contributor

Merging at the approved head 66a66209. main is 48 commits ahead; one of them (#21701) touches this same test file, but only adds ended_at: null to the shared buildStatus fixture near the top. This PR adds one vi.useFakeTimers line inside the deadline test, ~1,100 lines away, and does not read ended_at. The rest of the delta touches no file here. Test-only change.

@christian-byrne
christian-byrne merged commit af5a3a8 into main Oct 30, 2026
151 of 166 checks passed
@christian-byrne
christian-byrne deleted the matt/be-24342-pin-fake-clock-deadline-test branch October 30, 2026 00:44
xmarre pushed a commit to xmarre/ComfyUI_frontend that referenced this pull request Oct 31, 2026
…NT-3485) (Comfy-Org#22481)

## Summary

Fixes the `useAgentSession` g6 and g18 tests, which failed when run
alone and flaked in the agent suite.

The suite's fake clock has `shouldAdvanceTime: true`, so it also follows
real time. Both tests assert a timer boundary, and a cold start spends
enough real time to cross it. g6 failed every time alone: `getMessages`
had already run twice after advancing 999 ms. g18 failed 1 of 5 runs
alone on main.

Both tests now call `vi.useFakeTimers({ shouldAdvanceTime: false })`,
the same fix as Comfy-Org#22209. Test-only: production recovery timing is
unchanged.

## Verification

- g6 alone: failed 3/3 before, passed 13/13 after.
- g18 alone: 4/5 before, 8/8 after.
- `useAgentSession.test.ts` in full: 313 passed.

Fixes FRONT-3485
nomadoor pushed a commit to nomadoor/ComfyUI_frontend that referenced this pull request Nov 2, 2026
…omfy-Org#22666)

## What

Pins the fake clock in the AgentPanel test "locks immediately but only
shows the conversation veil for a slow opening": `vi.useFakeTimers({
shouldAdvanceTime: false })`.

## Why

Fixes FRONT-3700. The test asserts the veil is still hidden at 119 ms of
its 120 ms delay. The suite default (`fakeTimers: { shouldAdvanceTime:
true }` in `vite.config.mts`) lets real elapsed time move the fake clock
too, so on a loaded machine the clock can pass 120 ms before that check
and the veil already shows. It failed once that way in a full
`src/workbench/extensions/agent` run and passed alone. Same cause and
fix as the useSubscription deadline test
(Comfy-Org#22209); the next
test in this file that asserts an exact boundary already pins it the
same way.

## Proof

- Red: with the default clock and a temporary 60 ms real sleep inserted
before the 119 ms advance (simulated load), the test failed with
`expected <div role="status" …> to be null`, the reported failure.
- Green: with the clock pinned and the same 60 ms sleep still in place,
it passes. The sleep is not committed.
- Whole file: 31/31 pass.

Test-only, one file, inside `src/workbench/extensions/agent/**`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-coded Opened by the agent-work code loop cursor-review Trigger multi-model Cursor agent review on this PR risk:xhigh PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants