Skip to content

fix(website): hide the workflows that do not run yet - #19288

Closed
benjcooley wants to merge 1 commit into
mainfrom
benjcooley/workshop-disable-broken-workflows
Closed

benjcooley wants to merge 1 commit into
mainfrom
benjcooley/workshop-disable-broken-workflows

Conversation

@benjcooley

Copy link
Copy Markdown
Collaborator

Summary

Ben asked that every workflow that doesn't currently run be hidden, with the reason recorded in the availability manifest. The 2026-09-28 live sweep of all 30 published workflows on production left three failing, and each is now disabled until its fix merges.

Changes

Review Focus

The 2026-09-28 live sweep left three workflows failing on production:
product-photo-to-video (worker lost past the 10 GiB CPU-job limit while
building its combined video, BE-6855), and remove-object and
virtual-try-on (their default examples fail their own validation). Each
is disabled in the availability manifest with its reason until the PR
that fixes it re-enables it (#19284, #19272).
@benjcooley
benjcooley requested review from a team September 28, 2026 20:27
@vercel

vercel Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Actions Updated
billing-web Ignored Ignored Sep 28, 2026 8:27pm UTC
billing-web-test Skipped Skipped Sep 28, 2026 8:27pm UTC

Request Review

@vercel
vercel Bot temporarily deployed to Preview – billing-web-test September 28, 2026 20:27 Inactive
@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: dd166a98-1fb1-4092-bc74-0a3e214436c5

📥 Commits

Reviewing files that changed from the base of the PR and between e422943 and bec0b2f.

📒 Files selected for processing (3)
  • apps/website/e2e/workshop-workflows.spec.ts
  • apps/website/src/config/workshop-workflow-content.test.ts
  • apps/website/src/data/workshop-model-availability.json

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

Three workshop workflows are marked disabled in the availability data. Tests update validation and catalog expectations to reflect 27 published workflows.

Changes

Workshop workflow availability

Layer / File(s) Summary
Disable unavailable workflows
apps/website/src/data/workshop-model-availability.json, apps/website/src/config/workshop-workflow-content.test.ts
The availability data disables product-photo-to-video, remove-object, and virtual-try-on. Tests check that these workflows are disabled and absent from detail lookup and page paths.
Update published catalog expectations
apps/website/src/config/workshop-workflow-content.test.ts, apps/website/e2e/workshop-workflows.spec.ts
Tests require valid initial examples for enabled workflows, exclude disabled workflows from the catalog comparison, and expect 27 published workflows across catalog scenarios.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: anupreet-comfy

Merge Risk: ⚪ Minimal · up to bec0b

The three unavailable workflows will be omitted from the website’s normal catalog and detail routes. No actionable merge-blocking risk is evident in this change.

Architecture Summary

Architecture risk: 🔵 Low · up to bec0b

The change affects 1 system.

Changed systems: apps/website

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — apps/website (service) was modified; 3 changed files map to changed impact.

Before / after behavior

  • observed — Modified behavior in apps/website/e2e/workshop-workflows.spec.ts: The featured workflow catalogue card-count expectation changes from 30 to 27.
  • observed — Modified behavior in apps/website/e2e/workshop-workflows.spec.ts: The full-list heading and card-count expectations change from 30 workflows to 27.
  • observed — Modified behavior in apps/website/e2e/workshop-workflows.spec.ts: The unfiltered workflow catalogue card-count expectation changes from 30 to 27 before model filtering.
  • observed — Modified behavior in apps/website/e2e/workshop-workflows.spec.ts: After clearing model filters, the expected workflow catalogue count changes from 30 to 27.
🚥 Pre-merge checks | ✅ 7
✅ Passed checks (7 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: hiding workflows that currently fail or do not run.
Description check ✅ Passed The description is complete and relevant. It explains the reason for the change, lists the affected workflows and re-enabling PRs, summarizes test updates, and identifies merge-order and validation co…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
End-To-End Regression Coverage For Fixes ✅ Passed The PR title and commit subject contain the bug-fix signal fix. However, all listed changes are under apps/website, which the check explicitly excludes because that package uses its own Playwright…
Website End-To-End Regression Coverage ✅ Passed The PR changes website runtime behavior by disabling three workflows in apps/website/src/data/workshop-model-availability.json. It also updates apps/website/e2e/workshop-workflows.spec.ts assertio…
Adr Compliance For Entity/Litegraph Changes ✅ Passed The changed files are limited to website workflow tests and the workshop model availability manifest. None are under src/lib/litegraph/, src/ecs/, or graph-entity-related files. The ADR-LAYOUT and…
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@benjcooley benjcooley left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👨‍💻 BILL — review of #19288 at bec0b2f70a.

Verdict: the code is accepted. One SHOULD FIX in the merge coordination, because #19256 will break whichever of the two lands second.

Severity Count
BLOCKER 0
SHOULD FIX 1 (cross-PR)
TRIVIAL 0

The PR itself is correct

  • The three manifest entries carry specific, dated reasons and name the PR that re-enables each.
  • The counts are right: product loses product-photo-to-video and virtual-try-on (6→4), cleanup loses remove-object (6→5), for 27 in total.
  • The route-parity filter matches the manifest.
  • Dropping the partialExamples table is safe: both partial examples are now unpublished, so "every published default validates" holds and guards against new partial examples.
  • The new table test asserts each of the three is disabled, has no page and has no route.

SHOULD FIX (cross-PR): #19256 × this PR. Git merges them cleanly, but the website e2e will fail after both land. #19256's test (e2e/workshop-workflows.spec.ts, "workflow launch groups lead to the existing shared form") opens "Change a material", follows its shelf link to ?type=workflows&category=product and asserts toHaveCount(6). With this PR, that shelf holds 4 cards. It becomes 5 after #19284 and 6 after #19272. Fix: whichever merges second updates that assertion. Better still, make it count-independent (for example, assert every card on the filtered shelf is a product workflow and the count is above zero), so the re-enable PRs don't have to chase it.

What each re-enable PR must do after this lands (all three edits, or its tests go red):

  • #19284: remove the workflows/product-photo-to-video manifest entry, remove its row from the "withholds %s" table, and set product 4→5 and total 27→28 in the unit test and the e2e counts.
  • #19272: this one already conflicts textually in workshop-workflow-content.test.ts (both PRs drop partialExamples). Remove both entries and both table rows, set cleanup 5→6 and product +1 (to 5 or 6, depending on order), and raise the total and e2e counts to match.
  • #19282: already closed, so nothing to do.

Verification: 218 tests pass locally on this head (workshop-workflow-content, src/content). CI was 5 passed, 28 pending when I posted.

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since e4229433c30c410a096594c064ba2298c7013398

Full docs: ADR-DEVEX-LINT-0015

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

🌐 Website E2E

Caution

Some tests failed.

Status ❌ Failed
Report View Report
Screenshot diffs 0
Other failures 1

Warning

1 non-screenshot failure — these require manual review.

🔗 Website Preview

Website Preview: https://comfy-website-preview-pr-19288.vercel.app

Models is built and gated by PostHog (workshop-enabled), using staging Cloud.

This commit: https://website-frontend-3yyqhe83m-comfyui.vercel.app

Last updated: 2026-09-28T20:31:38Z for bec0b2f

@codecov

codecov Bot commented Sep 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main   #19288      +/-   ##
==========================================
- Coverage   84.21%   84.18%   -0.04%     
==========================================
  Files        3209     3172      -37     
  Lines      156522   155731     -791     
  Branches    40640    40444     -196     
==========================================
- Hits       131813   131099     -714     
+ Misses      23016    22985      -31     
+ Partials     1693     1647      -46     
Flag Coverage Δ
billing-web-unit ?
website-unit 78.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 37 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-official

Copy link
Copy Markdown

Pipelines

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 2 Pipeline jobs failed

CI: Website E2E | website-e2e — 🔧 Needs a code fix, caused by this PR

View in Datadog · View in GitHub Actions

CI: Website E2E | website-e2e-shard (3, 4) — 🔧 Needs a code fix, caused by this PR

View in Datadog · View in GitHub Actions

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: bec0b2f | Docs | View more details | Give us feedback!

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Sep 28, 2026
@benjcooley

Copy link
Copy Markdown
Collaborator Author

Superseded by #19291. All three workflows this PR would hide are fixed there (product-photo-to-video via ConcatenateVideo; remove-object and virtual-try-on via complete default examples), so hiding and then re-enabling them nets to nothing.

@benjcooley benjcooley closed this Sep 28, 2026
benjcooley added a commit that referenced this pull request Sep 28, 2026
## Summary

Makes every published Cloud workflow run. Combines the remaining
workflow fixes into one PR: #19284 (product-photo-to-video) and #19272
(remove-object, virtual-try-on). #19288's temporary disables are no
longer needed because all three are fixed here. Together with #19251
(already merged), all 30 published workflows pass live on production.

## Changes

- **product-photo-to-video** (Bob's commits from #19284): Cloud lost the
worker at ~700 s decoding the three MiniMax videos to stitch them. The
combined video now uses core `ConcatenateVideo` (joins VIDEO objects
without decoding); the downloadable graph and SVG match.
- **remove-object**: the example now includes a website-owned 1024×1024
mask over the apple
(`public/workflow-inputs/remove-object-apple-mask.png`).
- **virtual-try-on**: the example now includes the template's own
subject (pinned workflow_templates revision).
- **Tests**: every published workflow's example must validate in the
shared form; a browser test opens remove-object and virtual-try-on and
asserts their inputs are pre-filled.

## Review Focus

Live evidence on production (`test:workflow-pages --execute`):
- product-photo-to-video at `7c3f7304e4`: 3/3 passed, all 4 outputs
incl. the combined video — runs `56635afd…` (191 s), `daff6b57…` (184
s), `98babfa5…` (213 s).
- remove-object with this mask: run
`1de167fa-7317-4963-b3aa-24e3535a016b`; virtual-try-on with this
subject: run `33160fe7-ea1e-46a4-b4cf-e8531cc335f9`.
- Every other published workflow passed in the full sweep after #19251.

Post-deploy: the mask is served from comfy.org once this ships; run
remove-object once through the site.

Previously accepted by Bill as #19272 and #19284 (same commits). CI's
`/models/` JavaScript-budget failure is fixed by #19256, which merges
first.

Validation: 7,377 website unit tests, website typecheck, 15 workflow
browser tests on the CI-equivalent build.


## Why product-photo-to-video has no new browser test

The change is inside the Cloud graph (`ConcatenateVideo` replacing the
decode-and-stitch subgraph) and only has an effect when Cloud executes
it. The website's Playwright suites stub Cloud's `/api/prompt` and
`/api/jobs` (`e2e/workshop-workflow-runs.spec.ts`), so no browser test
can observe whether Cloud keeps the worker alive or produces the
combined video. The page, its form and its submission path are unchanged
and remain covered by the existing workflow browser tests; the graph
change is verified where it runs — three live production runs above (all
four outputs, combined video included) — and by the scheduled live sweep
in #19274.

This branch was previously deployed

1 inactive deployment
Preview – billing-web-test — bec0b2f7 Deployed Sep 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant