Skip to content

test(website): drop whole-route JS ceiling from models catalogue budget - #19669

Merged
DrJKL merged 1 commit into
mainfrom
test/website-drop-models-total-js-budget
Sep 30, 2026
Merged

DrJKL merged 1 commit into
mainfrom
test/website-drop-models-total-js-budget

Conversation

@DrJKL

@DrJKL DrJKL commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Drop the whole-route JS ceiling from the models catalogue budget test. Every page loads the same shared layout, and that shared code is almost the entire total, so the ceiling failed on unrelated site changes (raised 4× in 2 weeks, e.g. #17971, and failing again after #19494).

Changes

  • What: Remove totalBytes < 1_825_000 from apps/website/e2e/models-performance.spec.ts. Keep the catalogue-only delta budget (< 900_000).

Review Focus

Measured on a production build of main:

Bytes Limit
Route total 1,824,947 1,825,000
Shared layout (loaded on /) 1,658,812 none
Catalogue delta 166,135 900,000

The regression this test was written for, server catalogue data in the client bundle (#17604), already fails the build via src/integrations/workshop-client-boundary.ts. Verified locally: importing content/workshop-router-contracts.json into ModelsCatalogue.vue fails the build with "contains server catalogue data". The delta budget has never been raised and still catches a heavy dependency added to the catalogue island.

@DrJKL
DrJKL requested a review from a team September 30, 2026 21:33
@vercel

vercel Bot commented Sep 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Actions Updated
billing-web Ignored Ignored Sep 30, 2026 9:33pm UTC
billing-web-test Ignored Ignored Sep 30, 2026 9:33pm UTC

Request Review

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

🌐 Website E2E

Tip

All tests passed.

Status ✅ Passed
Report View Report

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since b01e5bf488e4873fc891229b3ba5a0da4102d06c

Full docs: ADR-DEVEX-LINT-0015

🔗 Website Preview

Website Preview: https://comfy-website-preview-pr-19669.vercel.app

Models is built and gated by PostHog (workshop-enabled), using staging Cloud.

This commit: https://website-frontend-2qc95spaz-comfyui.vercel.app

Last updated: 2026-09-30T21:35:40Z for 9df0600

@coderabbitai

coderabbitai Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Repository guideline files applied to this review (4)
docs/guidance/testing-principles.md — configured
apps/website/AGENTS.md — auto-discovered
docs/guidance/typescript.md — configured
src/AGENTS.md — auto-discovered

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: 1abb4e11-8952-4d59-bb0b-d917d7efa4a1

📥 Commits

Reviewing files that changed from the base of the PR and between b01e5bf and 9df0600.

📒 Files selected for processing (1)
  • apps/website/e2e/models-performance.spec.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review. Your free on-demand review promotion remains active until October 9, 2026 at 6:00 PM UTC.


📝 Walkthrough

Walkthrough

The catalogue performance test still checks scripts loaded on /hub/models/ that were not requested on /. It keeps the positive-byte and 900,000-byte checks and removes the whole-route 1,825,000-byte ceiling.

Changes

Catalogue JavaScript budget

Layer / File(s) Summary
Added scripts budget
apps/website/e2e/models-performance.spec.ts
The test checks the bytes for scripts on the catalogue page that were not requested on /. It retains the greater-than-zero and 900,000-byte checks and removes the total-route byte check.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Suggested reviewers: maanilverma

Merge Risk: ⚪ Minimal · up to 9df06

The change removes a brittle shared-layout limit while retaining the catalogue-specific performance budget, with no concrete merge-blocking risk identified.

🚥 Pre-merge checks | ✅ 7
✅ Passed checks (7 passed)
Check name Status Explanation
Description check ✅ Passed The description is complete and follows the repository template. It explains the motivation, identifies the exact code change, documents the retained catalogue delta budget, and provides relevant revi…
Title check ✅ Passed The title clearly and concisely describes the main change: removing the whole-route JavaScript ceiling from the models catalogue budget test.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
End-To-End Regression Coverage For Fixes ✅ Passed The PR title and commit subject use test language, not bug-fix language. The change is exclusively in apps/website, which this check excludes because website Playwright tests use apps/website/e2e/. Th…
Website End-To-End Regression Coverage ✅ Passed The PR changes only apps/website/e2e/models-performance.spec.ts. It does not change files under apps/website/src/ or apps/website/public/, so this website runtime coverage check is not applicabl…
Adr Compliance For Entity/Litegraph Changes ✅ Passed The changed file is apps/website/e2e/models-performance.spec.ts. It is not under src/lib/litegraph/, src/ecs/, and it is not related to graph entities. The ADR compliance check does not apply.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Sep 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main   #19669      +/-   ##
==========================================
- Coverage   84.54%   84.43%   -0.11%     
==========================================
  Files        3357     3284      -73     
  Lines      161826   159607    -2219     
  Branches    44034    43297     -737     
==========================================
- Hits       136812   134769    -2043     
+ Misses      23104    23038      -66     
+ Partials     1910     1800     -110     
Flag Coverage Δ
billing-web-unit ?
website-unit 79.65% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 73 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Sep 30, 2026
@DrJKL
DrJKL enabled auto-merge September 30, 2026 21:52
@DrJKL
DrJKL requested a review from benjcooley September 30, 2026 21:52
@DrJKL
DrJKL requested review from a team and huang47 September 30, 2026 21:52
@DrJKL
DrJKL added this pull request to the merge queue Sep 30, 2026
Merged via the queue into main with commit 21cca02 Sep 30, 2026
94 checks passed
@DrJKL
DrJKL deleted the test/website-drop-models-total-js-budget branch September 30, 2026 22:00
nomadoor pushed a commit to nomadoor/ComfyUI_frontend that referenced this pull request Oct 1, 2026
…rg#19364)

## Summary

Attach an installer dropdown to the download hero's primary button so
visitors can choose a different desktop installer.

## Changes

- **Dependencies**: Targets main. The unrelated Router fixture and
checkout PRs have been removed from this stack.
- Offer Windows x64/ARM64, macOS Apple Silicon, and Linux x64 AppImage
from one installer table; share selected installer records and platform
icons between the button and menu.
- Share the primary button's device-detection gate; retain phone email
capture and the existing plain ecosystem download button.
- Translate the menu into English and Chinese, label Windows x64
consistently, and identify the NVIDIA-only ARM64 build. Retain
platform-level download analytics.

## Review Focus

- Unknown desktops retain both Windows and macOS fallback buttons, with
one dropdown attached to the first.
- The final two-PR stack passes 36 focused installer/composable unit
tests. The dropdown fix also passed 43 focused unit tests and 25
download browser tests before this final restack. Current main includes
Comfy-Org#19669, which retains the catalogue-only JavaScript budget and removes
the unrelated whole-route ceiling. No checkout prerequisite remains. CI
has not been checked after this final restack.

## Screenshots (if applicable)

[Preview](https://comfy-website-preview-pr-19364.vercel.app/download/)

Before

<img width="1425" height="891" alt="Download hero before the dropdown"
src="https://github.com/user-attachments/assets/00c4ebc5-0c11-4525-adf0-5f2ed7c1f472"
/>

Existing after capture, before the concise x64 copy update: desktop

<img width="1680" height="1003" alt="Installer dropdown with hardware
requirements"
src="https://github.com/user-attachments/assets/115f5288-0ba2-48ad-b4b7-133d5e983842"
/>

Existing after capture, before the concise x64 copy update: narrow 320px
viewport

<img width="320" height="844" alt="Installer dropdown at 320 pixels"
src="https://github.com/user-attachments/assets/59c9ad4d-dbd0-4e6c-b1c9-2b92ac953c49"
/>

Existing after capture, before the concise x64 copy update: Chinese

<img width="1680" height="1003" alt="Chinese installer dropdown"
src="https://github.com/user-attachments/assets/ab1c8f4f-4e2d-4806-ac5e-fbc268779c97"
/>
christian-byrne added a commit that referenced this pull request Oct 3, 2026
… models catalogue budget (#19968)

## Summary

Backport of #19669 to `cloud/1.55`: drop the whole-route JavaScript
ceiling from the models catalogue budget test, which on this release
line is a coin flip that blocks website- and `packages/`-touching
carriers.

## Changes

- **What**: Remove `expect(totalBytes).toBeLessThan(1_815_000)` from
`apps/website/e2e/models-performance.spec.ts`. Keep the catalogue-only
delta budget (`< 900_000`). The resulting test body is **identical to
`main`'s**.

## Review Focus

**The ceiling is non-deterministic on this branch, not merely
outgrown.** On one fixed build of `cloud/1.55` (`a9968daf95`) the
assertion reports two discrete values with the ceiling sitting between
them:

| mode | scripts | route total | vs 1,815,000 |
|---|---|---|---|
| low | 85 | 1,803,417 | passes |
| high | 93 | 1,835,228 | fails |

Ten consecutive focused runs against that single build: **5 failed, 5
passed**.

The 31,811-byte swing is eight homepage/brand chunks and **no catalogue
code**: `ProductShowcaseSection` (10,626), `GalleryDetailModal` (9,623),
`AIModelsSection` (4,412), `ModelCreationsSection` (3,836),
`BrandButton` (1,177), `brandButton.variants` (1,039), `CardArrow`
(1,004), `cta` (94). They race the `shared` baseline captured on `/`:
when they have not arrived before the baseline is sampled they are
absent from `shared`, then charged in full against `/models/`.

That explains a result that otherwise looks like drift: the gate passed
on #19670 (152/152, 2026-09-30) and failed 3/3 on #19953 (1,835,341,
2026-10-02) with no catalogue change in between. Hosted runners are
slower, which biases them toward the failing mode.

The ceiling had also been ratcheted by unrelated billing work, exactly
as #19669 describes upstream: #17971 set 1,810,000, and the #19120
billing backport (#19207) raised it to 1,815,000 on this line while
measuring over it.

**What still guards the route:**

- The catalogue-only delta budget is kept and binds: measured
507,500–526,375 against 900,000. Tightening it to 500,000 **fails** at
507,500, so it is not vacuous.
- The regression this test was written for — server catalogue data in
the client bundle (#17604) — is caught at build time by
`apps/website/src/integrations/workshop-client-boundary.ts`, which is
present on this branch.

**Validation** (`cloud/1.55` @ `a9968daf95`, local production build,
port 6255):

- Before: 5/10 focused runs failed at 1,835,228.
- After: `models-performance.spec.ts` passes **10/10** (2 tests each).
- `pnpm --filter @comfyorg/website typecheck` — 0 errors; `oxfmt
--check`, `oxlint --type-aware`, `eslint` clean.

I am not merging this; it needs a frontend reviewer's approval.
Alexander Brown (@DrJKL) authored #19669 and is the natural reviewer.

## Screenshots (if applicable)

Not applicable; this removes a test assertion and changes no product
code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants