Skip to content

[backport core/1.55] perf: skip document-wide modal scans while typing - #19196

Closed
christian-byrne wants to merge 1 commit into
core/1.55from
backport-19057-to-core-1.55-manual
Closed

christian-byrne wants to merge 1 commit into
core/1.55from
backport-19057-to-core-1.55-manual

Conversation

@christian-byrne

Copy link
Copy Markdown
Contributor

Backports #19057 to core/1.55 after the automated cherry-pick conflicted.

This preserves the release branch behavior while adding the editor hot-path guard, store-backed auto-queue handling, performance harness, and regression coverage. Release-specific compatibility uses the existing Comfy.Agent.ThreadId storage key.

Validation:

  • focused ChangeTracker + PerformanceHelper suites pass
  • app typecheck passes
  • browser typecheck passes
  • pre-push Knip passes
  • git diff --check passes

- skip document-wide modal scans for native text-editing events while
preserving modal handling for other inputs
- honor store-backed auto-queue mode and cover native editor fast paths
- measure composer typing during the workload, with two unmeasured
type-and-clear warm-ups and named Playwright phases
- close both frame-measurement boundaries and recover cleanly from page
closure or bounded rAF timeouts

This is the measured optimization following the original harness in
[#19051](#19051) and
the protocol-compliant warmed baseline in
[#19159](#19159).

- [Warmed baseline exact-head
run](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/36371643330):
46/46 localhost and 4/4 cloud performance tests passed; retained
`perf-metrics` contains 49 measurements, including two samples for each
agent-composer arm.
- The current exact-head run compares the same 73-character workload
after rebasing onto that warmed baseline.
- The durable regression assertion verifies native contenteditable,
input, and textarea events do not invoke the document-wide modal scan.

- `pnpm vitest run src/scripts/changeTracker.test.ts` — 80 passed
- `pnpm vitest run
browser_tests/fixtures/helpers/PerformanceHelper.test.ts` — 2 passed
- real Chromium immediate-block frame falsifier — 1 passed
- `pnpm typecheck:browser`
- focused formatting and lint checks
- pre-push Knip and changed-file Fallow audit passed

Backport targets: core/1.54, cloud/1.54, core/1.55, cloud/1.55.

---------

Co-authored-by: Alexander Brown <drjkl@comfy.org>
@coderabbitai

coderabbitai Bot commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Ignore keyword(s) in the title.

⛔ Ignored keywords (2)
  • [release]
  • [backport

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: 9bdde499-136e-402c-9cbc-e0d243722d82

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@christian-byrne christian-byrne added backport Backporting a PR onto a release candidate greenlight cursor-review Trigger multi-model Cursor agent review on this PR labels Sep 28, 2026
@christian-byrne christian-byrne self-assigned this Sep 28, 2026
@christian-byrne christian-byrne added agent Comfy Agent product, integration, tests or product documentation agent-test-unit Touches Agent-related unit/component tests or their support agent-test-playwright Touches Agent-related Playwright tests, fixtures or configuration labels Sep 28, 2026
@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 09/28/2026, 05:19:26 AM UTC

Links

🎭 Playwright: ✅ 2123 passed, 0 failed · 1 flaky

📊 Browser Reports
  • chromium: View Report (✅ 2102 / ❌ 0 / ⚠️ 1 / ⏭️ 13)
  • chromium-2x: View Report (✅ 2 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • chromium-0.5x: View Report (✅ 1 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • mobile-chrome: View Report (✅ 18 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • New-test walkthrough (chromium, recorded video): View Report

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since fa2f9a51849b1873e1257db1d4c42a741d8a2a0e

Full docs: ADR-DEVEX-LINT-0015

🧩 Custom Nodes Ecosystem Matrix: ✅ Passed

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

Stage Result What a failure here means
pin-status success Informational. It never fails the build; it warns when the corpus snapshot is over 30 days old.
corpus success Infrastructure: the pack mirror could not be fetched, validated or cached. Not your diff.
shards success Harness integrity, not pack breakage. A shard only checks that every built spec wrote its row; vitest's own exit code is deliberately ignored because pack code leaks unhandled rejections.
verdicts success The population verdict per renderer. This is the stage that judges packs. Exit 1 is FAIL; exit 2 is withheld — rows were missing, so nothing was measured, which is not the same as your change breaking packs.
detection-proofs success The instrument self-test: the matrix must FAIL on a deliberately poisoned corpus. A failure here means the gate cannot be trusted this run, in either direction.

On a pin-bump PR, a red gate means the ecosystem moved — not that your diff broke something. That is what the bump PR is for. Outside a pin bump or a cold rebuild of the unpinned tail, a red means the diff.

Note the corpus size and the executed count are different numbers, and the verdict is against the executed count: roughly 3,100 of ~5,100 packs ship no extension-shaped JS and are skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

@github-actions
github-actions Bot requested a review from huang47 September 28, 2026 05:18
@github-actions github-actions Bot added the risk:xhigh PR risk grade (advisory shadow check; grader-owned) label Sep 28, 2026
@christian-byrne
christian-byrne removed the request for review from huang47 September 28, 2026 05:19
@christian-byrne

Copy link
Copy Markdown
Contributor Author

Superseded by automated backport #19190, which landed after the prerequisite scrolling backport reached this release branch.

@christian-byrne
christian-byrne deleted the backport-19057-to-core-1.55-manual branch September 28, 2026 05:22

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @christian-byrne.

Found 10 finding(s).

Severity Count
🟠 High 2
🟡 Medium 5
🟢 Low 3

Panel: 8/8 reviewers contributed findings.


const activeEl = document.activeElement
if (
!isAutoQueueOnChange() &&

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High — Replacing !app.ui.autoQueueEnabled || app.ui.autoQueueMode === 'instant' with !isAutoQueueOnChange() makes the text-editor bail-out depend on the queue store, so with useQueueSettingsStore().mode === 'change' (the "Run on change" option in ComfyQueueButton.vue) a keydown inside an <input>/<textarea> now falls through to the rAF body and reaches changeTracker.undoRedo() — previously that config (autoQueueEnabled === false) hit the input check and returned. Pressing Ctrl/Cmd+Z while typing in a prompt widget therefore undoes the workflow on top of the control's native text undo, leaving editor and graph history inconsistent; the new test only covers a plain a keydown, so keep the undo/redo path excluded regardless of queue mode. Raised by 2 of 8 reviewers (claude-opus-5-thinking-max edge-case, gpt-5.6-sol-max edge-case).


- name: Run cloud performance tests
id: perf_cloud
continue-on-error: true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High — id: perf_cloud is assigned and the step is continue-on-error: true, but nothing ever reads steps.perf_cloud.outcome — "Fail if performance tests failed" and "Warn if performance tests failed on a PR" still gate only on steps.perf.outcome. A failing or erroring cloud leg (i.e. the entire new agentComposer.spec.ts suite) is therefore invisible: green on main, no PR warning, and a baseline committed without the cloud measurements. Raised by 5 of 8 reviewers (claude-opus-5-thinking-max adversarial, gpt-5.6-sol-max adversarial, claude-opus-5-thinking-max edge-case, gpt-5.6-sol-max edge-case, kimi-k3-max edge-case).

state.frameRequestId = requestAnimationFrame(tick)
}

window.__perfFrameState = state

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — window.__perfFrameState = state clobbers any existing state without cancelling its in-flight rAF, and the old tick closure keeps rescheduling itself for the life of the page, appending to an array nobody can reach. With two helpers on one page (the per-test comfyPage.perf plus the new PerformanceHelper(page) the new specs construct), the second helper's stopFrameMeasurement also deletes the shared key, so the first helper's stopMeasuring silently returns [] and records frameDurationMs: 0 as if it were a real measurement. Cancel window.__perfFrameState?.frameRequestId (or refuse re-entry) before installing new state. Raised by 4 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k3-max adversarial, claude-opus-5-thinking-max edge-case, gpt-5.6-sol-max edge-case).

# of timing out without failing the PR job against a UI that cannot
# expose the workload.
- name: Build cloud frontend for cloud performance tests
run: pnpm build:cloud

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — Neither the new pnpm build:cloud step nor the "Merge distribution performance metrics" step is continue-on-error, unlike both perf-run steps. A cloud build break — or a JSON.parse throw / [...localhost.measurements, ...cloud.measurements] TypeError on a truncated or shape-mismatched report at line 100 — fails the job and skips "Save perf baseline to perf-data branch" (its if has no status function, so success() is implied), discarding already-collected localhost metrics. That is exactly the regression the comment on that step says was fixed for issue #15545. Raised by 4 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k3-max adversarial, claude-opus-5-thinking-max edge-case, kimi-k3-max edge-case).

})
this.snapshot = await this.getSnapshot()
try {
await this.startFrameMeasurement()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — this.snapshot = await this.getSnapshot() runs before startFrameMeasurement(), so the measured window includes the frame-loop setup: a CDP round trip plus a full wait for the next animation frame. Every snapshot-derived metric (durationMs, taskDurationMs, styleRecalcs, layouts) and the TBT accumulator now carry that overhead, shifting all baselines stored in perf-data; install the frame loop first and snapshot afterwards. Raised by 2 of 8 reviewers (claude-opus-5-thinking-max edge-case, gpt-5.6-sol-max edge-case).


test.describe(
'Agent composer performance',
{ tag: ['@perf', '@cloud'] },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — Tagging this describe @​cloud in addition to @​perf makes it match the cloud Playwright project (grep: /@​cloud/, grepInvert: /@​oss|@​mobile-ios/), which ci-tests-e2e.yaml runs on every PR via --project=cloud. Unlike chromium, the cloud project does not grep-invert @​perf, so these tests would also execute there under that project's much shorter timeout and with CI retries, writing perf temp files during a normal e2e run — add @​perf to the cloud project's grepInvert. Raised by 1 of 8 reviewers (claude-opus-5-thinking-max adversarial).

const timeoutId = window.setTimeout(() => {
cancelAnimationFrame(state.frameRequestId)
delete window.__perfFrameState
reject(new Error('Timed out waiting for the initial animation frame'))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — startFrameMeasurement rejects if the first animation frame does not arrive within 1s and startMeasuring rethrows, whereas the replaced measureFrameDurations allowed 5s and degraded to []. This turns a soft degrade into a hard failure for every existing @​perf spec — extensionLifecycle.spec.ts calls startMeasuring() right after loading large-graph-workflow, precisely when a CPU-starved CI container can block the renderer past 1s. Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case).

this.cdp = null
try {
if (measurementInProgress && !this.page.isClosed()) {
await this.stopFrameMeasurement()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — dispose() can now reject: stopFrameMeasurement only swallows errors when page.isClosed(), so a destroyed execution context propagates out. Both new specs call dispose() from finally, so the teardown error replaces the real failure — and because stopMeasuring's Promise.all leaves this.snapshot set when it rejects, dispose() retries the same failing evaluate. Make the stop best-effort inside dispose(). Raised by 3 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k3-max adversarial, gemini-3.1-pro edge-case).

!isAutoQueueOnChange() &&
(activeEl?.tagName === 'INPUT' ||
activeEl?.tagName === 'TEXTAREA' ||
(activeEl instanceof HTMLElement && activeEl.isContentEditable))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — Adding isContentEditable widens the bail-out beyond the previous INPUT/TEXTAREA check, so keydowns in any contenteditable no longer reach changeTracker.undoRedo() — including litegraph's property value spans, which LGraphCanvas.ts marks contenteditable and which commit a graph mutation on blur with no native history of their own. If dropping graph undo there is intended it deserves a note; otherwise exclude those spans the way keybindingService.ts special-cases SPAN.property_value. Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case).

const boundaryText = (response: Element | undefined) => {
if (response === undefined) return null
return (
response.textContent.match(/Turn \d+ is complete/)?.[0] ??

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — Element.textContent is typed string | null, so response.textContent.match(...) is an unguarded possibly-null dereference that should fail pnpm typecheck:browser under the strict config browser_tests/tsconfig.json inherits. Use response.textContent?.match(...). Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case).

@codecov

codecov Bot commented Sep 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##           core/1.55   #19196   +/-   ##
==========================================
  Coverage      84.66%   84.66%           
==========================================
  Files           2301     2301           
  Lines         131962   132034   +72     
  Branches       33747    33770   +23     
==========================================
+ Hits          111724   111788   +64     
- Misses         19919    19927    +8     
  Partials         319      319           
Flag Coverage Δ
unit 80.59% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/scripts/changeTracker.ts 95.61% <100.00%> (+0.31%) ⬆️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Comfy Agent product, integration, tests or product documentation agent-test-playwright Touches Agent-related Playwright tests, fixtures or configuration agent-test-unit Touches Agent-related unit/component tests or their support backport Backporting a PR onto a release candidate cursor-review Trigger multi-model Cursor agent review on this PR greenlight risk:xhigh PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants