Skip to content

test: localize workbench automock state - #18546

Merged
DrJKL merged 13 commits into
mainfrom
test/automock-07-stacked-cleanup
Sep 24, 2026
Merged

DrJKL merged 13 commits into
mainfrom
test/automock-07-stacked-cleanup

Conversation

@DrJKL

@DrJKL DrJKL commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Stacked on #18501. Remove migration-introduced module-scoped aliases so automocked services are referenced at their lifecycle owner.

Changes

  • What: Remove 18 module-scoped service/fixture bindings across 14 tests in favor of direct module/composable references or test-local telemetry values.
  • What: Remove 7 redundant expect(vi.mocked(fn)).toHaveBeenCalled* wrappers across 3 tests; retain vi.mocked only for configuration or typed .mock access.

Review Focus

Hoisted functions required by vi.mock collection timing and pre-existing lifecycle-owned SUT/store fixtures remain unchanged. The stacked diff contains 15 owned test files and no shared mock changes.

Verification:

  • targeted 15-file union: 564 passed
  • complete 44-file owned union: 1,004 passed
  • shuffled 44-file union, seed 707: 1,004 passed
  • real mock-owner suites: 305 passed, 1 skipped
  • typecheck, lint, format, Knip, Fallow, diff and ownership checks passed

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: ee798315-b594-42da-8968-b1c7bc90d668

📥 Commits

Reviewing files that changed from the base of the PR and between fb8e58f and 9569fdb.

📒 Files selected for processing (2)
  • src/workbench/extensions/agent/AgentPanelRoot.test.ts
  • src/workbench/extensions/agent/stores/agent/agentPanelStore.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 4 remain after this review.


📝 Walkthrough

Walkthrough

The test suite now configures and inspects mocked APIs, services, dialogs, and telemetry through imported module methods or composables. Existing test scenarios and expectations remain generally unchanged.

Changes

Test Mock Refactor

Layer / File(s) Summary
Platform test mock access
src/platform/keybindings/presetService.test.ts, src/platform/secrets/api/secretsApi.test.ts, src/platform/settings/*, src/platform/updates/components/ReleaseNotificationToast.test.ts
Platform tests use dialog composables and mocked API or error-handler methods directly for mock setup and assertions.
Agent API test mock access
src/workbench/extensions/agent/components/agent/Composer.test.ts, src/workbench/extensions/agent/components/agent/message/{MessageFeedback,ReplyAudioCard}.test.ts, src/workbench/extensions/agent/composables/agent/useAgentConsent.test.ts, src/workbench/extensions/agent/services/agent/agentRestClient.test.ts, src/workbench/extensions/agent/stores/agent/agentRunModeStore.test.ts
Agent tests configure and inspect api.fetchApi directly instead of using local mock aliases.
Agent workflow and telemetry test mock access
src/workbench/extensions/agent/AgentPanelRoot.test.ts, src/workbench/extensions/agent/components/agent/message/TabLinkCard.test.ts, src/workbench/extensions/agent/feedback.test.ts, src/workbench/extensions/agent/stores/agent/agentPanelStore.test.ts
Agent tests obtain workflow-service and telemetry mocks through their composables instead of cached module-scope references.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Refactor

Merge Risk: ⚪ Minimal · up to 9569f

This is a test-side mock-access refactor; no concrete merge-blocking risk remains in the inspected changes.

🚥 Pre-merge checks | ✅ 6 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
End-To-End Regression Coverage For Fixes ⚠️ Warning The commit subject test: fix direct telemetry mock access uses bug-fix language. The changed files are under src/, and the supplied file list contains no browser_tests/ file. The PR description … Add or update a Playwright regression test under browser_tests/, or add a concrete explanation in the PR description for why an end-to-end regression test is not practical.
✅ Passed checks (6 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Website End-To-End Regression Coverage ✅ Passed No changed file is under apps/website/src/ or apps/website/public/. The listed changes are test files under src/, so this website-specific check does not apply.
Adr Compliance For Entity/Litegraph Changes ✅ Passed The changed-file list contains only test files under platform and workbench paths. It contains no files under src/lib/litegraph/, src/ecs/, or graph-entity-related paths. The ADR compliance check …
Title check ✅ Passed The title clearly summarizes the main change: localizing automock state in tests.
Description check ✅ Passed The description includes the required summary, changes, and review focus sections. It explains the test refactoring and records verification results. Optional dependency, breaking-change, and screensh…
Full details: End-To-End Regression Coverage For Fixes

Explanation

The commit subject test: fix direct telemetry mock access uses bug-fix language. The changed files are under src/, and the supplied file list contains no browser_tests/ file. The PR description explains the automock refactoring but does not give a concrete reason why an end-to-end regression test was not added.

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since 6e891abd1806da620be79102a9c08ad75af1ee5f

Full docs: ADR-DEVEX-LINT-0015

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 09/24/2026, 08:36:54 PM UTC

Links

🎭 Playwright: ✅ 2124 passed, 0 failed · 1 flaky

📊 Browser Reports
  • chromium: View Report (✅ 2103 / ❌ 0 / ⚠️ 1 / ⏭️ 13)
  • chromium-2x: View Report (✅ 2 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • chromium-0.5x: View Report (✅ 1 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • mobile-chrome: View Report (✅ 18 / ❌ 0 / ⚠️ 0 / ⏭️ 0)

📦 Bundle Size

⚠️ Size data collection failed. Check the CI workflow logs.

🔬 E2E Coverage

⚠️ Coverage collection failed. Check the CI workflow logs.

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

Base automatically changed from test/automock-07-workbench-platform to main September 23, 2026 05:26
@DrJKL
DrJKL requested review from a team, christian-byrne and ltdrdata as code owners September 23, 2026 05:26
@github-actions github-actions Bot added the risk:xhigh PR risk grade (advisory shadow check; grader-owned) label Sep 23, 2026
coderabbitai[bot]
coderabbitai Bot previously approved these changes Sep 23, 2026
@codecov

codecov Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main   #18546      +/-   ##
==========================================
+ Coverage   83.79%   84.65%   +0.86%     
==========================================
  Files        3026     2302     -724     
  Lines      149499   132287   -17212     
  Branches    38645    35707    -2938     
==========================================
- Hits       125276   111993   -13283     
+ Misses      22918    19992    -2926     
+ Partials     1305      302    -1003     
Flag Coverage Δ
billing-web-unit ?
unit 80.82% <ø> (+0.07%) ⬆️
website-unit ?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 739 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…ked-cleanup

Amp-Thread-ID: https://ampcode.com/threads/T-01a0cb8f-0bf7-76fa-8e50-2a14431319be

# Conflicts:
#	src/platform/keybindings/presetService.test.ts
#	src/platform/secrets/api/secretsApi.test.ts
#	src/platform/settings/composables/useSettingsUrlLoader.integration.test.ts
#	src/platform/settings/globalSettingsApi.test.ts
#	src/platform/updates/components/ReleaseNotificationToast.test.ts
#	src/workbench/extensions/agent/AgentPanelRoot.test.ts
#	src/workbench/extensions/agent/components/agent/Composer.test.ts
#	src/workbench/extensions/agent/components/agent/message/MessageFeedback.test.ts
#	src/workbench/extensions/agent/components/agent/message/ReplyAudioCard.test.ts
#	src/workbench/extensions/agent/components/agent/message/TabLinkCard.test.ts
#	src/workbench/extensions/agent/composables/agent/useAgentConsent.test.ts
#	src/workbench/extensions/agent/feedback.test.ts
#	src/workbench/extensions/agent/services/agent/agentRestClient.test.ts
#	src/workbench/extensions/agent/stores/agent/agentPanelStore.test.ts
#	src/workbench/extensions/agent/stores/agent/agentRunModeStore.test.ts
coderabbitai[bot]
coderabbitai Bot previously approved these changes Sep 23, 2026
@DrJKL DrJKL added agent Comfy Agent product, integration, tests or product documentation agent-onboarding Agent consent, first-use tour or onboarding gates agent-session Agent sessions, history, turn lifecycle or workflow binding agent-test-only Agent tests, fixtures and test support only; no production changes agent-test-unit Touches Agent-related unit/component tests or their support agent-ui Agent panel, composer, transcript or other user-interface components labels Sep 24, 2026
@DrJKL
DrJKL requested a review from benjcooley September 24, 2026 07:34
@datadog-official

datadog-official Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Pipelines

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 5 Pipeline jobs failed

CI: Billing Web E2E | billing-web-e2e-gate

View in Datadog · View in GitHub Actions

CI: Billing Web Unit | billing-web-unit-gate

View in Datadog · View in GitHub Actions

CI: Lint Format | lint-and-format

View in Datadog · View in GitHub Actions

View all 5 failed jobs.

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9569fdb | Docs | View more details | Give us feedback!

@benjcooley benjcooley left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BLOCKER] Requesting changes: the alias removal is incomplete in two touched files, so the current head does not typecheck and 13 unit tests fail.

Severity Count
BLOCKER 2
SHOULD FIX 0
TRIVIAL 0

The lint-and-format aggregate failure is downstream of the typecheck failure; the standalone lint job passes.

assert.exists(telemetryProvider)
const telemetry = vi.mocked(telemetryProvider)

function getTelemetryMock() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BLOCKER] The old telemetry binding was removed, but this file still uses it at lines 3023, 3028, 3046, 3055, 3089, 3097, 3120, and 3128. The current head therefore fails typecheck and five tests throw ReferenceError: telemetry is not defined. Replace those remaining references with lifecycle-local access too. Per the unit-test review rule, please inline that access rather than keeping this getTelemetryMock() alias-by-renaming helper.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9569fdb. Removed the getTelemetryMock alias, replaced all remaining stale telemetry references with direct useTelemetry() access, and kept vi.mocked only for typed .mockClear/.mock.calls access. AgentPanelRoot plus agentPanelStore now pass 287/287, and the full typecheck passes.

return store
}

function getTelemetryMock() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BLOCKER] The old telemetry binding was removed, but the pagehide tests still reference it throughout lines 195–311. The current head has eleven resulting TypeScript errors and eight tests throw ReferenceError: telemetry is not defined. Migrate those references as well, and inline the singleton lookup rather than retaining this getTelemetryMock() alias-by-renaming helper.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9569fdb. Removed the getTelemetryMock alias and converted every pagehide assertion to direct useTelemetry() access; the one .mock.calls inspection uses vi.mocked for typing. AgentPanelRoot plus agentPanelStore pass 287/287, and the full typecheck passes.

@DrJKL
DrJKL requested a review from benjcooley September 24, 2026 20:53
@DrJKL
DrJKL dismissed benjcooley’s stale review September 24, 2026 20:54

Blockers fixed

@DrJKL
DrJKL added this pull request to the merge queue Sep 24, 2026
Merged via the queue into main with commit 84fc839 Sep 24, 2026
105 of 155 checks passed
@DrJKL
DrJKL deleted the test/automock-07-stacked-cleanup branch September 24, 2026 21:23
huntcsg added a commit that referenced this pull request Sep 24, 2026
`test: localize workbench automock state` (#18546) removed this file's
module-level `telemetry` binding and inlined every read as `useTelemetry()!`,
because the automock state is now per-test rather than hoisted. That landed on
main while this branch was open.

Git merged the two textually with no conflict — main's deletion of the binding
won, and the paywall-telemetry block added here kept referencing it — so the
merge result failed to compile with 22 `TS2304: Cannot find name 'telemetry'`.
The branch and main each typecheck alone, which is why only the merge queue
saw it.

This block now uses main's idiom: `useTelemetry()!` for assertion targets and
`vi.mocked(useTelemetry())!` where the mock object itself is needed. Verified
by merging current main locally: typecheck clean and the suite green at 276.
@DrJKL DrJKL removed agent Comfy Agent product, integration, tests or product documentation agent-test-only Agent tests, fixtures and test support only; no production changes agent-test-unit Touches Agent-related unit/component tests or their support agent-session Agent sessions, history, turn lifecycle or workflow binding agent-ui Agent panel, composer, transcript or other user-interface components agent-onboarding Agent consent, first-use tour or onboarding gates labels Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:xhigh PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants