Skip to content

[backport cloud/1.55] perf: skip document-wide modal scans while typing - #19194

Closed
christian-byrne wants to merge 1 commit into
cloud/1.55from
backport-19057-to-cloud-1.55-manual
Closed

christian-byrne wants to merge 1 commit into
cloud/1.55from
backport-19057-to-cloud-1.55-manual

Conversation

@christian-byrne

Copy link
Copy Markdown
Contributor

Backports #19057 to cloud/1.55 after the automated cherry-pick conflicted.

This preserves the release branch behavior while adding the editor hot-path guard, store-backed auto-queue handling, performance harness, and regression coverage. Release-specific compatibility uses the existing Comfy.Agent.ThreadId storage key.

Validation:

  • focused ChangeTracker + PerformanceHelper suites pass
  • app typecheck passes
  • browser typecheck passes
  • pre-push Knip passes
  • git diff --check passes

- skip document-wide modal scans for native text-editing events while
preserving modal handling for other inputs
- honor store-backed auto-queue mode and cover native editor fast paths
- measure composer typing during the workload, with two unmeasured
type-and-clear warm-ups and named Playwright phases
- close both frame-measurement boundaries and recover cleanly from page
closure or bounded rAF timeouts

This is the measured optimization following the original harness in
[#19051](#19051) and
the protocol-compliant warmed baseline in
[#19159](#19159).

- [Warmed baseline exact-head
run](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/36371643330):
46/46 localhost and 4/4 cloud performance tests passed; retained
`perf-metrics` contains 49 measurements, including two samples for each
agent-composer arm.
- The current exact-head run compares the same 73-character workload
after rebasing onto that warmed baseline.
- The durable regression assertion verifies native contenteditable,
input, and textarea events do not invoke the document-wide modal scan.

- `pnpm vitest run src/scripts/changeTracker.test.ts` — 80 passed
- `pnpm vitest run
browser_tests/fixtures/helpers/PerformanceHelper.test.ts` — 2 passed
- real Chromium immediate-block frame falsifier — 1 passed
- `pnpm typecheck:browser`
- focused formatting and lint checks
- pre-push Knip and changed-file Fallow audit passed

Backport targets: core/1.54, cloud/1.54, core/1.55, cloud/1.55.

---------

Co-authored-by: Alexander Brown <drjkl@comfy.org>
@coderabbitai

coderabbitai Bot commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Ignore keyword(s) in the title.

⛔ Ignored keywords (2)
  • [release]
  • [backport

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: c1d56084-d378-46b4-9c52-0173f0ad7bfc

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@christian-byrne christian-byrne added backport Backporting a PR onto a release candidate greenlight cursor-review Trigger multi-model Cursor agent review on this PR agent Comfy Agent product, integration, tests or product documentation agent-test-unit Touches Agent-related unit/component tests or their support agent-test-playwright Touches Agent-related Playwright tests, fixtures or configuration labels Sep 28, 2026
@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

🎭 Playwright: ✅ 2121 passed, 0 failed · 2 flaky

📊 Browser Reports
  • chromium: View Report (✅ 2100 / ❌ 0 / ⚠️ 2 / ⏭️ 13)
  • chromium-2x: View Report (✅ 2 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • chromium-0.5x: View Report (✅ 1 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • mobile-chrome: View Report (✅ 18 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • New-test walkthrough (chromium, recorded video): View Report

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 09/28/2026, 05:19:48 AM UTC

Links

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since 2d74852442d4cfae400df354925c09cb1f935d9a

Full docs: ADR-DEVEX-LINT-0015

🧩 Custom Nodes Ecosystem Matrix: ✅ Passed

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

Stage Result What a failure here means
pin-status success Informational. It never fails the build; it warns when the corpus snapshot is over 30 days old.
corpus success Infrastructure: the pack mirror could not be fetched, validated or cached. Not your diff.
shards success Harness integrity, not pack breakage. A shard only checks that every built spec wrote its row; vitest's own exit code is deliberately ignored because pack code leaks unhandled rejections.
verdicts success The population verdict per renderer. This is the stage that judges packs. Exit 1 is FAIL; exit 2 is withheld — rows were missing, so nothing was measured, which is not the same as your change breaking packs.
detection-proofs success The instrument self-test: the matrix must FAIL on a deliberately poisoned corpus. A failure here means the gate cannot be trusted this run, in either direction.

On a pin-bump PR, a red gate means the ecosystem moved — not that your diff broke something. That is what the bump PR is for. Outside a pin bump or a cold rebuild of the unpinned tail, a red means the diff.

Note the corpus size and the executed count are different numbers, and the verdict is against the executed count: roughly 3,100 of ~5,100 packs ship no extension-shaped JS and are skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

@github-actions github-actions Bot added the risk:xhigh PR risk grade (advisory shadow check; grader-owned) label Sep 28, 2026
@github-actions
github-actions Bot requested a review from huang47 September 28, 2026 05:18
@christian-byrne
christian-byrne removed the request for review from huang47 September 28, 2026 05:19
@christian-byrne

Copy link
Copy Markdown
Contributor Author

Superseded by automated backport #19191, which landed after the prerequisite scrolling backport reached this release branch.

@christian-byrne
christian-byrne deleted the backport-19057-to-cloud-1.55-manual branch September 28, 2026 05:22

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @christian-byrne.

Found 10 finding(s).

Severity Count
🟠 High 1
🟡 Medium 6
🟢 Low 3

Panel: 8/8 reviewers contributed findings.


- name: Run cloud performance tests
id: perf_cloud
continue-on-error: true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High — This step gets id: perf_cloud and continue-on-error: true, but the later "Fail if performance tests failed" and "Warn if performance tests failed on a PR" steps only read steps.perf.outcome, so a cloud perf failure or timeout leaves the job green with no alert on main or on PRs. Add steps.perf_cloud.outcome to both gates. Raised by 3 of 8 reviewers (gpt-5.6-sol-max adversarial, gpt-5.6-sol-max edge-case, claude-opus-5-thinking-max edge-case).

!isAutoQueueOnChange() &&
(activeEl?.tagName === 'INPUT' ||
activeEl?.tagName === 'TEXTAREA' ||
(activeEl instanceof HTMLElement && activeEl.isContentEditable))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — Adding isContentEditable to the bail-out means a focused contenteditable no longer reaches bindInputEl = activeEl, so the one-shot change/input/blur listener bindInput used to register (which captured the edit into undo history) never attaches, and undoRedo is never reached for Ctrl+Z/Ctrl+Y. Confirm no in-canvas contenteditable editor (e.g. litegraph property/title editing, DOM widgets) relied on that capture, or narrow the bail-out to the composer element. Raised by 1 of 8 reviewers (claude-opus-5-thinking-max edge-case).


test.describe(
'Agent composer performance',
{ tag: ['@perf', '@cloud'] },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — The @​cloud tag also makes this spec eligible for the regular cloud project in playwright.config.ts, whose grepInvert does not exclude @​perf the way chromium's does, so the --project=cloud job in ci-tests-e2e.yaml will collect it under normal timeouts and retries — where the 100-turn variant (200 rendered messages plus three pressSequentially passes) will almost certainly time out. Add @​perf to the cloud project's grepInvert. Raised by 1 of 8 reviewers (claude-opus-5-thinking-max adversarial).

const boundaryText = (response: Element | undefined) => {
if (response === undefined) return null
return (
response.textContent.match(/Turn \d+ is complete/)?.[0] ??

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — Element.textContent is typed string | null, so response.textContent.match(...) is an unguarded dereference and fails strict-null typechecking of browser_tests. Use response.textContent?.match(...) or read into a local with a ?? '' default. Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case).

# of timing out without failing the PR job against a UI that cannot
# expose the workload.
- name: Build cloud frontend for cloud performance tests
run: pnpm build:cloud

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — This build step has neither continue-on-error nor if: always(), so any cloud-build breakage fails the job here; the later baseline-save step's if: has no status function, which GitHub implicitly ANDs with success(), so it is skipped — silently stopping the main-branch perf baseline until the 7-day freshness alarm fires, the same regression the comment further down says was fixed for #15545. Either make the baseline save always()-guarded, or mark this step continue-on-error and skip the cloud run when it failed so it never measures the localhost artifact. Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case).

this.cdp = null
try {
if (measurementInProgress && !this.page.isClosed()) {
await this.stopFrameMeasurement()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — stopFrameMeasurement only swallows evaluate errors when page.isClosed(), so a destroyed execution context (navigation or renderer crash mid-measurement — a common perf-test failure mode) makes dispose() reject. Both new specs call dispose() from a finally block and the comfyPage fixture calls it in teardown, so the cleanup error replaces the real test failure; make dispose() best-effort and swallow or log errors from the stop path. Raised by 3 of 8 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case, kimi-k3-max edge-case).

try {
return await this.page.evaluate(async () => {
const state = window.__perfFrameState
if (!state) return []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — When window.__perfFrameState is missing at stop time — any navigation between startMeasuring() and stopMeasuring() wipes it, since it is installed via page.evaluate rather than an init script — this returns [] and stopMeasuring reports frameDurationMs: 0 and p95FrameDurationMs: 0. A lost measurement is then indistinguishable from a perfect one, gets through recordMeasurement, and can be committed as the main baseline; fail or omit the frame fields instead of reporting zeros. Raised by 1 of 8 reviewers (claude-opus-5-thinking-max edge-case).

state.frameRequestId = requestAnimationFrame(tick)
}

window.__perfFrameState = state

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — This overwrites any existing __perfFrameState without cancelling its frameRequestId, orphaning a self-rescheduling tick loop that then runs for the life of the page with an unbounded durationsMs array — perturbing the new measurement while the first helper's stop returns empty data. Cancel window.__perfFrameState?.frameRequestId before installing the new state. Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k3-max adversarial).

this.getSnapshot(),
this.stopFrameMeasurement()
])
const before = this.snapshot

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — this.snapshot = null runs only after this Promise.all resolves, so a rejection from getSnapshot() or stopFrameMeasurement() leaves snapshot set: the next startMeasuring() throws the already-in-progress error and dispose() retries the same failing stop path. Clear snapshot before awaiting, or in a finally. Raised by 2 of 8 reviewers (kimi-k3-max adversarial, claude-opus-5-thinking-max edge-case).

const cloud = JSON.parse(fs.readFileSync(process.argv[2], "utf8"));
process.stdout.write(JSON.stringify({
...cloud,
measurements: [...localhost.measurements, ...cloud.measurements]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — The merge script parses both files and spreads .measurements with no validation, so a truncated or field-less metrics file (interrupted run, partial write) throws under bash -e. This step has no continue-on-error, so the whole job fails even though both test steps are soft-failing, and because mv never runs the uploaded artifact and committed baseline keep cloud measurements only. Guard the parse and default each measurements to []. Raised by 3 of 8 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case, kimi-k3-max adversarial).

@codecov

codecov Bot commented Sep 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@              Coverage Diff              @@
##             cloud/1.55   #19194   +/-   ##
=============================================
  Coverage              ?   80.59%           
=============================================
  Files                 ?     2198           
  Lines                 ?   117577           
  Branches              ?    35694           
=============================================
  Hits                  ?    94763           
  Misses                ?    22468           
  Partials              ?      346           
Flag Coverage Δ
unit 80.59% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/scripts/changeTracker.ts 84.36% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Comfy Agent product, integration, tests or product documentation agent-test-playwright Touches Agent-related Playwright tests, fixtures or configuration agent-test-unit Touches Agent-related unit/component tests or their support backport Backporting a PR onto a release candidate cursor-review Trigger multi-model Cursor agent review on this PR greenlight risk:xhigh PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants