Skip to content

[backport cloud/1.55] test(website): drop whole-route JS ceiling from models catalogue budget - #19968

Merged
christian-byrne merged 1 commit into
cloud/1.55from
op324/website-models-budget-1.55
Oct 3, 2026
Merged

christian-byrne merged 1 commit into
cloud/1.55from
op324/website-models-budget-1.55

Conversation

@christian-byrne

Copy link
Copy Markdown
Contributor

Summary

Backport of #19669 to cloud/1.55: drop the whole-route JavaScript ceiling from the models catalogue budget test, which on this release line is a coin flip that blocks website- and packages/-touching carriers.

Changes

  • What: Remove expect(totalBytes).toBeLessThan(1_815_000) from apps/website/e2e/models-performance.spec.ts. Keep the catalogue-only delta budget (< 900_000). The resulting test body is identical to main's.

Review Focus

The ceiling is non-deterministic on this branch, not merely outgrown. On one fixed build of cloud/1.55 (a9968daf95) the assertion reports two discrete values with the ceiling sitting between them:

mode scripts route total vs 1,815,000
low 85 1,803,417 passes
high 93 1,835,228 fails

Ten consecutive focused runs against that single build: 5 failed, 5 passed.

The 31,811-byte swing is eight homepage/brand chunks and no catalogue code: ProductShowcaseSection (10,626), GalleryDetailModal (9,623), AIModelsSection (4,412), ModelCreationsSection (3,836), BrandButton (1,177), brandButton.variants (1,039), CardArrow (1,004), cta (94). They race the shared baseline captured on /: when they have not arrived before the baseline is sampled they are absent from shared, then charged in full against /models/.

That explains a result that otherwise looks like drift: the gate passed on #19670 (152/152, 2026-09-30) and failed 3/3 on #19953 (1,835,341, 2026-10-02) with no catalogue change in between. Hosted runners are slower, which biases them toward the failing mode.

The ceiling had also been ratcheted by unrelated billing work, exactly as #19669 describes upstream: #17971 set 1,810,000, and the #19120 billing backport (#19207) raised it to 1,815,000 on this line while measuring over it.

What still guards the route:

  • The catalogue-only delta budget is kept and binds: measured 507,500–526,375 against 900,000. Tightening it to 500,000 fails at 507,500, so it is not vacuous.
  • The regression this test was written for — server catalogue data in the client bundle (fix(website): improve Models performance, input recovery and SDK examples #17604) — is caught at build time by apps/website/src/integrations/workshop-client-boundary.ts, which is present on this branch.

Validation (cloud/1.55 @ a9968daf95, local production build, port 6255):

  • Before: 5/10 focused runs failed at 1,835,228.
  • After: models-performance.spec.ts passes 10/10 (2 tests each).
  • pnpm --filter @comfyorg/website typecheck — 0 errors; oxfmt --check, oxlint --type-aware, eslint clean.

I am not merging this; it needs a frontend reviewer's approval. Alexander Brown (@DrJKL) authored #19669 and is the natural reviewer.

Screenshots (if applicable)

Not applicable; this removes a test assertion and changes no product code.

… models catalogue budget

Backport of #19669 to `cloud/1.55`.

The whole-route ceiling on this release line is flaky by construction, not
merely ratcheted. On one fixed build of cloud/1.55 (a9968da) the assertion
reports two discrete values and the ceiling sits between them:

| mode | scripts | total | vs 1,815,000 |
|---|---|---|---|
| low  | 85 | 1,803,417 | passes |
| high | 93 | 1,835,228 | fails |

Ten consecutive focused runs on that single build: 5 failed, 5 passed.

The 31,811-byte swing is eight homepage/brand chunks and no catalogue code:
ProductShowcaseSection (10,626), GalleryDetailModal (9,623), AIModelsSection
(4,412), ModelCreationsSection (3,836), BrandButton (1,177),
brandButton.variants (1,039), CardArrow (1,004), cta (94). They race the
`shared` baseline captured on `/`: when they have not arrived by the time the
baseline is sampled they are missing from `shared`, then charged in full
against `/models/`.

That is why the gate passed on #19670 (152/152, 2026-09-30) and failed 3/3 on
#19953 (1,835,341, 2026-10-02) with no catalogue change between them.

The ceiling had also been ratcheted by unrelated billing work, as #19669
describes upstream: #17971 set 1,810,000, and the #19120 billing backport
(#19207) raised it to 1,815,000 on this line.

The catalogue-only delta budget is kept and still binds: it measures 507,500
to 526,375 against its 900,000 ceiling, and tightening it to 500,000 fails at
507,500, so it is not vacuous. The regression this test was written for,
server catalogue data in the client bundle (#17604), is caught at build time
by apps/website/src/integrations/workshop-client-boundary.ts, which is present
on this branch.

After this change the file's two tests pass 10/10 on the same build.

This leaves the resulting test body identical to `main`'s.
@vercel

vercel Bot commented Oct 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Actions Updated
billing-web Ignored Ignored Oct 3, 2026 12:28am UTC
billing-web-test Ignored Ignored Oct 3, 2026 12:28am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Ignore keyword(s) in the title.

⛔ Ignored keywords (2)
  • [release]
  • [backport

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Team
  • Run ID: 1f7130ea-9b9c-4136-91cb-7d9f3c0c5dcc

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Comment @coderabbitai help to get the list of available commands.

@christian-byrne christian-byrne added backport Backporting a PR onto a release candidate cloud/1.55 Backport PRs for cloud 1.55 labels Oct 3, 2026
@github-actions
github-actions Bot requested a review from ryan-di October 3, 2026 00:29
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

🌐 Website E2E

Tip

All tests passed.

Status ✅ Passed
Report View Report

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

📦 Bundle Size

⏳ Size data collection in progress…

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since a9968daf959f722c7461c3a48efee1d8b5404d91

Full docs: ADR-DEVEX-LINT-0015

@codecov

codecov Bot commented Oct 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@              Coverage Diff              @@
##             cloud/1.55   #19968   +/-   ##
=============================================
  Coverage              ?   76.24%           
=============================================
  Files                 ?      691           
  Lines                 ?    16917           
  Branches              ?     4750           
=============================================
  Hits                  ?    12899           
  Misses                ?     3063           
  Partials              ?      955           
Flag Coverage Δ
website-unit 76.24% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Oct 3, 2026

@christian-byrne christian-byrne left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head release review at 0d2403ba6b1f9b8c54e21394a7dabd5b284723bd: no blocking findings. This is a one-file semantic backport of merged #19669 to cloud/1.55; it removes only the nondeterministic whole-route ceiling and preserves the catalogue-only < 900_000 budget.

Acceptance: production build succeeded; focused desktop Playwright passed with the retained budget at 526,375 bytes, and a temporary mutation to 500,000 failed at the assertion (then restored, clean tree). Hosted run 37082308244 has all four website-e2e shards and aggregate green. scripts/check-merge-ready.sh ... --target-branch cloud/1.55 reports MERGE READY against target a9968daf959f722c7461c3a48efee1d8b5404d91, with 104 terminal non-adverse checks and no unresolved threads/objections.

Authority: program/release-backport-authority.md; GitHub cannot accept author self-approval, so this is the required honest COMMENT review. The grant explicitly permits an in-scope release carrier merge after gates and says REVIEW_REQUIRED alone is not a new approval requirement.

@christian-byrne
christian-byrne merged commit ecf0803 into cloud/1.55 Oct 3, 2026
108 checks passed
@christian-byrne
christian-byrne deleted the op324/website-models-budget-1.55 branch October 3, 2026 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport Backporting a PR onto a release candidate cloud/1.55 Backport PRs for cloud 1.55 risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants