Repository navigation
[backport cloud/1.55] test(website): drop whole-route JS ceiling from models catalogue budget - #19968
Conversation
… models catalogue budget Backport of #19669 to `cloud/1.55`. The whole-route ceiling on this release line is flaky by construction, not merely ratcheted. On one fixed build of cloud/1.55 (a9968da) the assertion reports two discrete values and the ceiling sits between them: | mode | scripts | total | vs 1,815,000 | |---|---|---|---| | low | 85 | 1,803,417 | passes | | high | 93 | 1,835,228 | fails | Ten consecutive focused runs on that single build: 5 failed, 5 passed. The 31,811-byte swing is eight homepage/brand chunks and no catalogue code: ProductShowcaseSection (10,626), GalleryDetailModal (9,623), AIModelsSection (4,412), ModelCreationsSection (3,836), BrandButton (1,177), brandButton.variants (1,039), CardArrow (1,004), cta (94). They race the `shared` baseline captured on `/`: when they have not arrived by the time the baseline is sampled they are missing from `shared`, then charged in full against `/models/`. That is why the gate passed on #19670 (152/152, 2026-09-30) and failed 3/3 on #19953 (1,835,341, 2026-10-02) with no catalogue change between them. The ceiling had also been ratcheted by unrelated billing work, as #19669 describes upstream: #17971 set 1,810,000, and the #19120 billing backport (#19207) raised it to 1,815,000 on this line. The catalogue-only delta budget is kept and still binds: it measures 507,500 to 526,375 against its 900,000 ceiling, and tightening it to 500,000 fails at 507,500, so it is not vacuous. The regression this test was written for, server catalogue data in the client bundle (#17604), is caught at build time by apps/website/src/integrations/workshop-client-boundary.ts, which is present on this branch. After this change the file's two tests pass 10/10 on the same build. This leaves the resulting test body identical to `main`'s.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
|
Important Review skippedIgnore keyword(s) in the title. ⛔ Ignored keywords (2)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
🌐 Website E2ETip All tests passed.
🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measureExecutes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight This candidate changes nothing the matrix measures, so every expensive stage was skipped. Reproduce locally# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
npx vitest run --config vitest.matrix.config.mts
Full docs: scripts/registry-census/README.md 📦 Bundle Size
🌾 Fallow: ✅ PassedFallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened. No new findings in the changed files. Reproduce locally# the base the gate compared against
npx fallow audit --root . --quiet --format json \
--changed-since a9968daf959f722c7461c3a48efee1d8b5404d91
Full docs: ADR-DEVEX-LINT-0015 |
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## cloud/1.55 #19968 +/- ##
=============================================
Coverage ? 76.24%
=============================================
Files ? 691
Lines ? 16917
Branches ? 4750
=============================================
Hits ? 12899
Misses ? 3063
Partials ? 955
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
christian-byrne
left a comment
There was a problem hiding this comment.
Exact-head release review at 0d2403ba6b1f9b8c54e21394a7dabd5b284723bd: no blocking findings. This is a one-file semantic backport of merged #19669 to cloud/1.55; it removes only the nondeterministic whole-route ceiling and preserves the catalogue-only < 900_000 budget.
Acceptance: production build succeeded; focused desktop Playwright passed with the retained budget at 526,375 bytes, and a temporary mutation to 500,000 failed at the assertion (then restored, clean tree). Hosted run 37082308244 has all four website-e2e shards and aggregate green. scripts/check-merge-ready.sh ... --target-branch cloud/1.55 reports MERGE READY against target a9968daf959f722c7461c3a48efee1d8b5404d91, with 104 terminal non-adverse checks and no unresolved threads/objections.
Authority: program/release-backport-authority.md; GitHub cannot accept author self-approval, so this is the required honest COMMENT review. The grant explicitly permits an in-scope release carrier merge after gates and says REVIEW_REQUIRED alone is not a new approval requirement.
Summary
Backport of #19669 to
cloud/1.55: drop the whole-route JavaScript ceiling from the models catalogue budget test, which on this release line is a coin flip that blocks website- andpackages/-touching carriers.Changes
expect(totalBytes).toBeLessThan(1_815_000)fromapps/website/e2e/models-performance.spec.ts. Keep the catalogue-only delta budget (< 900_000). The resulting test body is identical tomain's.Review Focus
The ceiling is non-deterministic on this branch, not merely outgrown. On one fixed build of
cloud/1.55(a9968daf95) the assertion reports two discrete values with the ceiling sitting between them:Ten consecutive focused runs against that single build: 5 failed, 5 passed.
The 31,811-byte swing is eight homepage/brand chunks and no catalogue code:
ProductShowcaseSection(10,626),GalleryDetailModal(9,623),AIModelsSection(4,412),ModelCreationsSection(3,836),BrandButton(1,177),brandButton.variants(1,039),CardArrow(1,004),cta(94). They race thesharedbaseline captured on/: when they have not arrived before the baseline is sampled they are absent fromshared, then charged in full against/models/.That explains a result that otherwise looks like drift: the gate passed on #19670 (152/152, 2026-09-30) and failed 3/3 on #19953 (1,835,341, 2026-10-02) with no catalogue change in between. Hosted runners are slower, which biases them toward the failing mode.
The ceiling had also been ratcheted by unrelated billing work, exactly as #19669 describes upstream: #17971 set 1,810,000, and the #19120 billing backport (#19207) raised it to 1,815,000 on this line while measuring over it.
What still guards the route:
apps/website/src/integrations/workshop-client-boundary.ts, which is present on this branch.Validation (
cloud/1.55@a9968daf95, local production build, port 6255):models-performance.spec.tspasses 10/10 (2 tests each).pnpm --filter @comfyorg/website typecheck— 0 errors;oxfmt --check,oxlint --type-aware,eslintclean.I am not merging this; it needs a frontend reviewer's approval. Alexander Brown (@DrJKL) authored #19669 and is the natural reviewer.
Screenshots (if applicable)
Not applicable; this removes a test assertion and changes no product code.