Skip to content

feat(website): Cinematic Studio opens on a playground-style side panel - #19316

Merged
benjcooley merged 1 commit into
mainfrom
mariana/cinematic-studio-panel
Sep 29, 2026
Merged

benjcooley merged 1 commit into
mainfrom
mariana/cinematic-studio-panel

Conversation

@mariana953

@mariana953 mariana953 commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Replaces #19227 with the same tree as one commit on main. #19227's history still carried the pre-squash PP Formula commits from the old Hub stack (authored as an agent identity), which fails the co-author check now that it targets main. Nothing in the diff changed. #19227 has its review history.

Cinematic Studio opens on the side panel layout (D) by default. The panel follows the compact list design: Model, Scene (AI prompt switch in the box), one Shot card listing Camera and every direction part, References, and Format as three menus above the run button. The bottom composer (?ux=e) uses segmented controls whose parts each open their own picker or menu, a References menu behind the +, and shared tooltips. Camera options are drawn as shaded clay objects; shot, light, film and look options as neutral line icons. Credit amounts stay off both layouts for now.

Changes

  • What:
    • CinematicStudioPage.vue: default layout d (also in production; ?ux=e still opens the composer on previews); passes :show-credits="false" to both layouts.
    • showCredits prop (default true) threaded down to CinematicGenerateAction; when false the estimate, shortfall note and reduce button are hidden and a no-credits gate keeps only the buy button (note as tooltip from lg; other notes stay inline).
    • Side panel
      • CinematicPanel.vue: Model, Scene, Shot, References, Format with small sentence-case labels; the model trigger drops the provider name (cinematic.model.heading is "Model"); footer not sticky.
      • CinematicShotList.vue (was CinematicDirectionGrid.vue): one card, a row per Camera / Shot / Light / Film / Look / Grade (icon or grade swatch, part, choice, chevron), each opening its picker.
      • CinematicStudioPanel.vue: the picker sits in the grid, measures itself and centres on the opening row, clamped to the grid; bottom sheet on mobile. Same 2fr/3fr split as reshoot/ReshootStudio.vue.
      • CinematicSceneField.vue: "View full prompt" and promptSegments removed (also from useCinematicShot); the AI prompt switch sits at the foot of the scene box.
      • CinematicReferenceSlot.vue: empty slots are dashed "+ Character" / "+ Palette" buttons; filled slots show the image.
      • CinematicOutputControls.vue: aspect, resolution and takes as three CinematicMenus side by side.
    • Bottom composer
      • CinematicDirectionSegments.vue (was CinematicDirectionChips.vue): Shot, Light, Film, Look, Grade as one segmented control of icons (Grade: its palette swatch); each opens its single-group picker, which closes on a pick. The tabbed direction picker, its Done button and cinematic.picker.done are removed.
      • CinematicFormatSegments.vue: aspect, resolution and takes (×1–×4) as segmented menus. Options shared with the side panel through useFormatMenus.ts.
      • CinematicReferenceMenu.vue: the + opens a References menu (Character, Palette); each item opens its file picker, shows its file once attached, and attached references can be removed there. The references popover and the references/format/direction popover keys are gone (PickerKey only).
      • CinematicComposer.vue: the scene grows to four lines then scrolls; the AI prompt switch sits top right (own line on phones); the camera chip shows the body icon and focal length and is named "Camera: ", with the full summary in its tooltip.
    • Shared
      • direction-icons.ts + CinematicDirectionIcon.vue (new): line drawings for every shot, light, film and look option, in currentColor and site tokens (no hex). Used in the composer segments and shot-list rows instead of preview images; CinematicOptionGrid keeps the preview and adds the icon in a corner. New direction-icons.test.ts.
      • CinematicTooltip.vue: one styled reka tooltip for the model menu, camera chip, direction and format segments, aspect toggles and the AI prompt switch. CinematicMenu takes tooltip and wraps the whole menu, so the dropdown keeps its own popper anchor.
      • CinematicCheckBadge.vue: a yellow check on the chosen starter shot and the chosen picker tile.
      • camera-icons.ts + CinematicOptionIcon.vue: body, lens, focal and aperture icons are shaded shapes in an oblique view. Colours come from site tokens via color-mix; unchosen options are dimmed. New camera-icons.test.ts.
      • CinematicMenu.vue: chosen row filled instead of checked; side prop.
      • catalog.ts: distinct preview frames for the default tiles; gradeGroup typed DirectionGroup<'grade'>.
      • copy.ts: cinematic.section.shot; zh-CN "Optional" fixed (was "竖屏").
    • Also: empty stage frame without fill or outline; Re-shoot drop-zone hint shortened; Hub card links to the plain studio URL.

Review Focus

  • Same content as feat(website): Cinematic Studio opens on a playground-style side panel #19227. git diff between feat(website): Cinematic Studio opens on a playground-style side panel #19227's head (cebc807) and this PR's head is empty for every changed file. CodeRabbit's findings there were fixed in b60b0eb, which this includes.
  • Credits are hidden, not removed. One prop on the page.
  • Tooltip + dropdown nesting. A reka tooltip nested inside a dropdown root captured the dropdown trigger's popper anchor and parked the menu off screen; CinematicMenu and CinematicReferenceMenu now put the tooltip outside DropdownMenuRoot. happy-dom does not lay out poppers, so this was checked in Chromium.
  • Demo runs. ?demo=1 (existing useCinematicDemoRun) renders sample frames without credits, for design review on previews.
  • Overlaps. Rob's feat(website): run the Re-shoot app on the CrossView deployment #18910 also edits reshoot/. feat(website): link each workshop app to its GitHub repository #19313 (GitHub link beside the title) also touches CinematicStudioPanel.vue, but not the same lines.
  • Tests. Side panel headings and shot-list row picking; direction segments open only their own options; composer aspect and AI prompt reach the render request; tooltips name composer controls; camera chip name for a set and an auto focal length; picker centred on its row; references menu attach/remove; camera and direction icons are distinct per option.
  • Checks run locally: typecheck:website, oxfmt, oxlint, eslint, fallow, the co-author check, and the workshop unit tests (1013).

Screenshots (if applicable)

Preview (needs a signed-in staff account), once deploy-preview passes:

The side panel becomes the default layout: Model, Scene, one Shot card
listing the camera and every direction part, References and Format. The
bottom composer uses segmented controls whose parts open their own pickers
or menus, a References menu behind the +, and shared tooltips. Camera
options are drawn as shaded objects and direction options as neutral line
icons. Credit amounts stay hidden on both layouts for now.

Same tree as #19227, rebuilt as one commit on main.
@mariana953
mariana953 requested review from a team and benjcooley September 28, 2026 23:28
@vercel

vercel Bot commented Sep 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Actions Updated
billing-web Ignored Ignored Sep 28, 2026 11:28pm UTC
billing-web-test Ignored Ignored Sep 28, 2026 11:28pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 8672860e-e4e4-4c49-acec-e5f221ffa1ce

📥 Commits

Reviewing files that changed from the base of the PR and between 8d3a084 and b5cfc84.

📒 Files selected for processing (40)
  • apps/website/src/components/workshop/cinematic-studio/CinematicCheckBadge.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicComposer.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicDirectionChips.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicDirectionGrid.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicDirectionIcon.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicDirectionSegments.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicEnhanceSwitch.test.ts
  • apps/website/src/components/workshop/cinematic-studio/CinematicEnhanceSwitch.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicFirstRun.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicFormatSegments.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicGenerateAction.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicMenu.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicOptionGrid.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicOptionIcon.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicOptionList.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicOutputControls.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicPanel.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicPicker.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicReferenceMenu.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicReferenceSlot.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicSceneField.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicShotList.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicStageCard.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicStudio.test.ts
  • apps/website/src/components/workshop/cinematic-studio/CinematicStudio.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicStudioPage.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicStudioPanel.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicTooltip.vue
  • apps/website/src/components/workshop/cinematic-studio/picker-key.ts
  • apps/website/src/components/workshop/cinematic-studio/reshoot/ReshootStudio.vue
  • apps/website/src/components/workshop/cinematic-studio/reshoot/ReshootUpload.vue
  • apps/website/src/components/workshop/cinematic-studio/useFormatMenus.ts
  • apps/website/src/composables/useCinematicShot.ts
  • apps/website/src/lib/workshop/cinematic-studio/camera-icons.test.ts
  • apps/website/src/lib/workshop/cinematic-studio/camera-icons.ts
  • apps/website/src/lib/workshop/cinematic-studio/catalog.ts
  • apps/website/src/lib/workshop/cinematic-studio/copy.ts
  • apps/website/src/lib/workshop/cinematic-studio/direction-icons.test.ts
  • apps/website/src/lib/workshop/cinematic-studio/direction-icons.ts
  • apps/website/src/lib/workshop/cinematic-studio/reshoot-copy.ts
💤 Files with no reviewable changes (2)
  • apps/website/src/components/workshop/cinematic-studio/CinematicDirectionGrid.vue
  • apps/website/src/components/workshop/cinematic-studio/CinematicDirectionChips.vue

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 3 remain after this review.


📝 Walkthrough

Walkthrough

Cinematic Studio updates its shot, format, reference, and enhancement controls. It adds camera and direction illustrations, changes picker behavior, and adjusts studio layouts and credit display. Tests cover the updated controls and interactions.

Changes

Cinematic Studio refresh

Layer / File(s) Summary
Option illustrations and selection states
apps/website/src/components/workshop/cinematic-studio/CinematicCheckBadge.vue, CinematicFirstRun.vue, CinematicDirectionIcon.vue, CinematicOptionGrid.vue, CinematicOptionIcon.vue, CinematicOptionList.vue, CinematicStageCard.vue, apps/website/src/lib/workshop/cinematic-studio/camera-icons.ts, camera-icons.test.ts, direction-icons.ts, direction-icons.test.ts, catalog.ts
Camera and direction options use layered SVG drawings. Option grids and starter-shot cards show selection badges. Catalog preview images and related styles change.
Shared format, reference, and scene controls
apps/website/src/components/workshop/cinematic-studio/CinematicFormatSegments.vue, useFormatMenus.ts, CinematicOutputControls.vue, CinematicTooltip.vue, CinematicMenu.vue, CinematicReferenceMenu.vue, CinematicReferenceSlot.vue, CinematicEnhanceSwitch.vue, CinematicEnhanceSwitch.test.ts, CinematicSceneField.vue, apps/website/src/lib/workshop/cinematic-studio/copy.ts
Format controls use menus for aspect ratio, resolution, and take count. The reference menu attaches and removes cast and palette images. The scene field uses a dedicated enhancement switch and no longer displays segmented prompt text.
Shot settings and picker flow
apps/website/src/components/workshop/cinematic-studio/CinematicComposer.vue, CinematicDirectionChips.vue, CinematicDirectionGrid.vue, CinematicDirectionSegments.vue, CinematicShotList.vue, CinematicPicker.vue, picker-key.ts, CinematicPanel.vue, CinematicStudio.vue, CinematicStudioPanel.vue
The composer and side panel use separate shot-setting pickers and a shot list. Picker keys and selection behavior now target individual settings. The composer binds format, enhancement, and reference state through models.
Studio layout, credits, and validation
apps/website/src/components/workshop/cinematic-studio/CinematicGenerateAction.vue, CinematicStudioPage.vue, CinematicStudio.test.ts, apps/website/src/components/workshop/cinematic-studio/reshoot/ReshootStudio.vue, ReshootUpload.vue, apps/website/src/lib/workshop/cinematic-studio/reshoot-copy.ts, apps/website/src/composables/useCinematicShot.ts
The side panel becomes the default studio layout. Credit estimates and cost display can be hidden. Tests cover updated controls, picker positioning, layout switching, and generation behavior. The Re-shoot layout and upload text also change.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  actor User
  participant CinematicShotList
  participant CinematicStudioPanel
  participant CinematicPicker
  User->>CinematicShotList: Open a shot-setting row
  CinematicShotList->>CinematicStudioPanel: Emit the picker key
  CinematicStudioPanel->>CinematicPicker: Open and position the picker
  User->>CinematicPicker: Select an option
  CinematicPicker->>CinematicStudioPanel: Pass the selection to choose
Loading

Possibly related PRs

  • Comfy-Org/ComfyUI_frontend#18695: Introduces the Cinematic Studio structures that this pull request revises, including its components, picker, prompt, and Re-shoot flows.

Merge Risk: ⚪ Minimal · up to b5cfc

The updated controls keep take selection within the supported range and hide credit amounts in both layouts. No identified issue needs to be resolved before merging.

Security Architecture Review

Security architecture risk: 🔵 Low · up to b5cfc

The new layout and hidden credit amounts do not appear to change who can submit a generation request or how an in-progress request is handled. Some downstream enforcement was outside the reviewed change, so the assessment retains limited uncertainty.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The effective changed exposure is the Cinematic Studio browser experience: its default panel and both layouts’ credit presentation. The inspected request continues through the existing workshop session and router path.

Trust Boundaries and Controls

  • observed — User-selected settings and references reach generation only after the shared shot readiness check; the run composable rechecks its gate and session and verifies refreshed credentials against the initiating user and workspace before rendering or upload.

Resilience and Maintainability Implications

  • observed — Cancellation and interruption remain centralized in the run composable, which ignores aborted results, releases outputs, and limits retries to settled failures or cancellations.

Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Website End-To-End Regression Coverage ❓ Inconclusive The PR changes observable website runtime behavior in apps/website/src/, including the default Cinematic Studio layout and interactive picker, reference, format, and navigation flows. The available … Provide the full PR description or confirm that its omitted section contains no concrete explanation for not adding a Playwright regression test. If no explanation exists, add or update a relevant Playwright test under apps/website/e2e/, …
✅ Passed checks (6 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
End-To-End Regression Coverage For Fixes ✅ Passed The PR title and commit subject use feat, not bug-fix language. The changed files are exclusively under apps/website, which this check excludes from failure. The check passes.
Adr Compliance For Entity/Litegraph Changes ✅ Passed The changed-file list contains only website Cinematic Studio, composable, and cinematic-studio library files. It contains no files under src/lib/litegraph/ or src/ecs/, and no files related to gra…
Title check ✅ Passed The title clearly identifies the primary change: making the Cinematic Studio side-panel layout the default.
Description check ✅ Passed The description follows the template and provides a detailed summary, changes, review focus, testing information, and preview links. It also explains the rebuild from #19227 and the credit-display beh…
Full details: Website End-To-End Regression Coverage

Explanation

The PR changes observable website runtime behavior in apps/website/src/, including the default Cinematic Studio layout and interactive picker, reference, format, and navigation flows. The available changed-file summaries show no apps/website/e2e/**/*.spec.ts assertion update. However, the PR description is explicitly truncated, so the required check cannot determine whether the omitted text gives a concrete reason why no end-to-end regression test was added.

Resolution

Provide the full PR description or confirm that its omitted section contains no concrete explanation for not adding a Playwright regression test. If no explanation exists, add or update a relevant Playwright test under apps/website/e2e/, or document why such a test is not practical.

✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@benjcooley benjcooley left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified replacement for #19227. The clean PR has the same 40-file change set and stable patch ID as the fully reviewed predecessor, now as one Mariana-authored commit on main with no AI co-author trailer. No design or code changes were introduced by the replacement.

@benjcooley benjcooley left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👨‍💻 BILL — exact-head review of #19316 at b5cfc841ba: READY.

Severity Count
BLOCKER 0
SHOULD FIX 0
TRIVIAL 0

Same change as #19227, with clean history

  • Authorship: one commit, authored and committed by Mariana Martinho, with no Co-authored-by or AI trailer.
  • Identical change: the stable patch-id is a0bd7cf433c42a318431b19703cd312d27b7f948, identical to #19227's final head cebc807ab1, each measured from its own merge base. Both change the same 40 files (+2269/−967).
  • Merges cleanly with current main (dab0dbee1c).

One commit was new to me. My last #19227 verdict (0/0/0) was at b60b0ebf3f. The final #19227 head adds one more commit, cebc807ab1 "draw direction options as neutral line icons", so #19316 contains it. I've reviewed it now for correctness and accessibility; the drawings are Mar's:

  • CinematicDirectionIcon: the SVG is aria-hidden, and its gradient, pattern and filter ids come from useId(), so they're unique per instance and SSR-stable. The segment buttons keep their aria-label (title: label).
  • No empty icons: the segments bar and the Shot list render icons only for [...lookGroups, gradeGroup]. direction-icons.test.ts asserts every look option has a unique drawing. Every grade option carries a palette, so grade always takes the swatch branch. Camera parts keep their own previews and never go through this path.
  • Option grid: the corner icon shows only when an option has a preview and a drawing, and never for auto.

Verification: all 19 Cinematic Studio test files (171 tests) pass locally on this head. CI when I posted: 5 pass, 27 pending. Merge on green.

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since 8d3a084a07a62dfb7d84ae747db6f1e25d4349b5

Full docs: ADR-DEVEX-LINT-0015

🌐 Website E2E

Tip

All tests passed.

Status ✅ Passed
Report View Report

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

🔗 Website Preview

Website Preview: https://comfy-website-preview-pr-19316.vercel.app

Models is built and gated by PostHog (workshop-enabled), using staging Cloud.

This commit: https://website-frontend-he7xsy4ds-comfyui.vercel.app

Last updated: 2026-09-28T23:40:18Z for b5cfc84

@codecov

codecov Bot commented Sep 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.41379% with 9 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...rkshop/cinematic-studio/CinematicReferenceMenu.vue 93.47% 0 Missing and 3 partials ⚠️
...rkshop/cinematic-studio/CinematicReferenceSlot.vue 71.42% 1 Missing and 1 partial ⚠️
...onents/workshop/cinematic-studio/useFormatMenus.ts 87.50% 0 Missing and 2 partials ⚠️
...ts/workshop/cinematic-studio/CinematicShotList.vue 96.29% 0 Missing and 1 partial ⚠️
.../src/lib/workshop/cinematic-studio/camera-icons.ts 98.21% 0 Missing and 1 partial ⚠️
@@            Coverage Diff             @@
##             main   #19316      +/-   ##
==========================================
- Coverage   85.07%   84.18%   -0.90%     
==========================================
  Files        2338     3180     +842     
  Lines      134365   155661   +21296     
  Branches    34338    40306    +5968     
==========================================
+ Hits       114310   131042   +16732     
- Misses      19702    22990    +3288     
- Partials      353     1629    +1276     
Flag Coverage Δ
website-unit 78.56% <97.41%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
.../workshop/cinematic-studio/CinematicCheckBadge.vue 100.00% <100.00%> (ø)
...ts/workshop/cinematic-studio/CinematicComposer.vue 94.33% <100.00%> (ø)
...rkshop/cinematic-studio/CinematicDirectionIcon.vue 100.00% <100.00%> (ø)
...op/cinematic-studio/CinematicDirectionSegments.vue 100.00% <100.00%> (ø)
...rkshop/cinematic-studio/CinematicEnhanceSwitch.vue 100.00% <100.00%> (ø)
...ts/workshop/cinematic-studio/CinematicFirstRun.vue 100.00% <100.00%> (ø)
...kshop/cinematic-studio/CinematicFormatSegments.vue 100.00% <100.00%> (ø)
...kshop/cinematic-studio/CinematicGenerateAction.vue 95.34% <100.00%> (ø)
...onents/workshop/cinematic-studio/CinematicMenu.vue 100.00% <100.00%> (ø)
.../workshop/cinematic-studio/CinematicOptionGrid.vue 84.61% <100.00%> (ø)
... and 24 more

... and 808 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Sep 28, 2026
@benjcooley
benjcooley added this pull request to the merge queue Sep 29, 2026
@benjcooley
benjcooley removed this pull request from the merge queue due to a manual request Sep 29, 2026
@benjcooley
benjcooley merged commit 1d21581 into main Sep 29, 2026
95 checks passed
@benjcooley
benjcooley deleted the mariana/cinematic-studio-panel branch September 29, 2026 01:21
benjcooley added a commit that referenced this pull request Sep 29, 2026
… models, animate a still (#19370)

## Summary

Stacked on #19326 (base branch `rob/cinematic-studio-luis-port`); it
retargets to `main` once #19326 merges. Both are rebased onto Mariana's
side-panel redesign (#19316): her layout, menus and clay camera icons
are the base, and video fits into her components. It adds Luis's
Image/Video mode to Cinematic Studio on main's Router path. The video
models go through the same `router_render`, credit gate, parallel takes
and retries as image shots, and their requests are built with
`studioRouterForm`.

- **Image / Video switch** above the side panel (and above the composer
in the composer layout). Each mode keeps its own model.
- **Ten video models, in picker order:** Seedance 2.5, Seedance 2.5
Edit, Seedance 2.0, Seedance 2.0 Fast, Seedance 2.0 Mini, Wan 3.0,
FLUX.3 Video, Gemini Omni Flash 1.1, Grok Imagine 1.5, Kling 3.0. Each
has picker tags for its sharpest resolution, length range and sound.
- Each text-to-video model is paired with its image-to-video operation.
Adding a starting frame moves the shot there, the same way references
move an image shot to its edit operation.
- Kling 3.0 is text-only until its image-to-video is runnable on the
Router.
  - Veo 3 is left out on purpose.
- **What a shot can choose comes from each contract.** Lengths,
resolutions, frames, sound, starting/ending frame and source video are
all read through the same shared Router mapping that sends the request,
so the picker only offers what reaches the model. That also covers
models whose fields are named differently (Veo, Wan, Gemini) where
Luis's parser missed them.
- **Video controls in Mariana's components:**
- Output (side panel) and the composer's Format segments show frame,
resolution, length and "Generate audio" (off by default), sharing one
`useVideoMenus` helper the way her image menus share `useFormatMenus`.
- References (side panel) and the composer's "+" menu hold the starting
and ending frame, or the video to edit (Seedance 2.5 Edit), next to
Colors. One `referenceSlots` helper decides which slots a model gets in
both layouts.
  - Her AI prompt switch shows a motion hint in video mode.
- **Playback:** video takes play in the frame and show as a muted
thumbnail. "Use as reference" is offered on stills in image mode only.
"Animate image" starts a video from a still.
- **One clip per video shot, and no price shown.** The pricing rules
have nothing that matches a studio clip (only Grok image-to-video, at a
fixed 6 s / 480p), so the credit gate only asks for a positive balance.

## Studio polish

- **A finished take keeps its frame and column width until its image or
video has loaded.** Before, the stage collapsed or shrank to a small
box, and the filmstrip jumped.
- **Filmstrip thumbnails share one 3:2 shape**, cropped to fit, whatever
the take's frame.
- **The progress label reads "Generating image" or "Generating video"**
instead of "Rendering take A".
- **On a video take, the take details and actions sit at the top of the
frame**, so they don't cover the browser's play bar.

## Reusing a still ("Animate image", "Use as reference")

Both buttons used to download the take in the browser. Most providers
leave the image on their own storage without a CORS header, so the
download failed and the buttons silently did nothing. We checked this
from a localhost page with `fetch` and with header checks:

| Where the still lives | Models | Page can read it? |
|---|---|---|
| Inside the reply | GPT Image 2 / 2.5, Nano Banana Pro / 2 | yes |
| Comfy storage (Router re-host) | FLUX 2, Grok | comfy.org only (not
localhost or previews) |
| BytePlus | Seedream 5.0 Pro | no |
| DashScope Singapore / Beijing | Qwen | no / yes |
| Recraft | Recraft V4.1 | yes |

A reused take is now kept as a reference to the output: its link, plus
the picture when the page could read it.
- **Link:** sent to models whose contract takes URLs, since their server
fetches it and nothing is uploaded from the page. That covers Seedance
2.5 / 2.0 / Fast, Wan, Gemini and Grok image-to-video, and Qwen
references.
- **Picture:** sent to models that need the bytes. When the page
couldn't read the picture, Generate is held back with a note instead of
failing silently.

**Known limitation: animating a GPT Image or Nano Banana still.** These
models return the picture inside the reply, so there's no link to pass
on.
- **Seedance / Wan / Gemini / Grok** take the starting frame as a link,
so the still is uploaded to Comfy storage first. Comfy storage only
allows uploads from comfy.org, so this fails on localhost and previews
("Your file could not be uploaded"). It works on comfy.org.
- **FLUX.3 Video** takes the picture inside the request. A large still
(a 3840×2160 GPT PNG) can go over the site's 10 MB request limit.

Seedream's Router contract doesn't expose `response_format`, so asking
it for `b64_json` isn't possible from the studio.

### Long term (not a blocker, discussion in Slack)

The durable fix is for Comfy to keep every Router output and let the
Router take it back as an input:
1. Store every output in Comfy storage or saved assets
(`comfy_save_asset`), as FLUX / Grok / Seedance video already are.
2. Accept an asset id or Comfy-signed URL for `reference_images` /
`first_frame`, resolved server-side.

Then no image passes through the browser at all. `TakeImage` is shaped
to become that asset id. Smaller asks: localhost and preview origins on
`api-nodes-prod` CORS, and `response_format` on the BytePlus image
contracts.

## Testing on localhost

- **Fully testable:** all text-to-video; FLUX.3 and Seedance 2.0 Mini
image-to-video (the frame is sent inline); and animating a still into
the link-taking models.
- **Needs comfy.org:** uploading your own file as a frame for Seedance /
Wan / Gemini / Grok, and the Seedance Edit source video. These go
through Comfy storage, whose CORS is comfy.org-only.

## Test plan

- [x] `video.test.ts`: capabilities read from real contracts (lengths,
resolutions, frames, audio, first/last frame, source video, frame
choice) and request parameters for each kind.
- [x] `take-image.test.ts`: keeping a take (link, picture or both), what
each model receives, and which contracts accept links.
- [x] `CinematicStudio.test.ts`: a clip in video mode, animating a
still, animating a provider-hosted still by link, the "needs the
picture" note, and reference hidden in video mode.
`CinematicTakeMedia.test.ts`: video playback.
- [x] Offline request preparation for all 18 video operations (network
blocked): each prepares a valid request with the watermark off.
- [x] `CinematicTakeFrame.test.ts`: the frame and column hold until
media loads, the Generating image/video label, even thumbnails, and the
take details at the top of a clip.
- [x] `models.test.ts`: `videoShotBlock` (source video needed, no
starting frame, unreadable frame).
- [x] Rebased on #19326 + current main. Website suite: 514 files, 9,775
tests passing. Typecheck, knip and the fallow audit pass.
- [x] Browser, side panel (default): mode switch; the ten video models
in order with tags; video Output shows 16:9 · 720p · 5s · Generate
audio; References shows Starting frame + Colors; Seedance 2.5 Edit shows
"Video to edit" and holds Generate with "Add the video to edit first.";
image mode unchanged.
- [x] Browser, composer layout (`?ux=e`): video Format segments; the "+"
menu lists Starting frame + Colors in video mode and Character, Palette,
Colors in image mode.
- [x] Earlier, before the rebase: Generating image/video label, 84×56
thumbnails, Animate / Use as reference with demo stills, take details at
the top of a clip.
- [ ] Live signed-in video runs (spend credits).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: luisvitela-hub <luisvitela-hub@users.noreply.github.com>
Co-authored-by: benjcooley <benjcooley@gmail.com>
pull Bot pushed a commit to Penguinjanator/ComfyUI_frontend that referenced this pull request Sep 29, 2026
…, a Colors palette, and request fixes (Comfy-Org#19326)

## Summary

Ports self-contained pieces of Luis's `cinema-studio-integration` work
onto main, fitted into Mariana's side-panel redesign (Comfy-Org#19316). Her
layout, Format menus, References menu and clay camera icons stay as they
are, and main's Router path and credit estimates are untouched. Six
commits: image models, dropping Seedream 4.5, the Colors palette, two
request fixes (no watermark; no catalogue examples in studio requests),
and tests for sampling colours.

### 1. Image models and frames

- Cinematic Studio now offers **13 image models** (was 5). It adds
Seedream 5.0 Pro, FLUX 2 Max, Nano Banana 2, Qwen Image 3.0, GPT Image
2, GPT Image 2.5 Flare and Sunburst, Grok Imagine Image 2.0 and Recraft
V4.1, and **drops Seedream 4.5**. Seedream 5.0 Pro is the new default.
It has no 21:9 size, so the studio opens on 16:9, and starter shots
asking for 21:9 move to it. Video stays out of the studio for now.
- Adds **3:2 and 2:3** frames.
- **Each model only offers the frames it can make.** The frame menus in
Mariana's Format controls (side panel and composer segments) share one
filter in `useFormatMenus`.
- **Frames per model:** The frames are read from the model's Router
contract (its size menu). Switching to a model that can't make the
chosen frame moves it to the nearest one. With references attached, the
frames follow the edit operation that will actually run.
- **The studio asks for the exact size** of that frame
(`model_specific`), and the credit estimate prices that same size.
- **Reference support is read from each contract**: which operation
keeps references and how many images it takes. The existing "{model}
can't use references" block now also covers attaching more images than a
model accepts.
- **No Router or Models-tab changes.** Everything goes through the
existing `router_render` path, credentials and credit gate. The
contract-reading logic is ported from Luis's `cinema-studio-integration`
branch.


### 2. Colors palette (sent as words)

- A **Colors** control holds up to eight swatches: add, change, remove,
or mark one as the main color. In the side panel it sits in References,
below Character and Palette. In the bottom composer it's a **Colors**
item in the References (+) menu that opens a small Colors panel.
- **From an image** fills the swatches with a picture's dominant colors.
The picture is read in the browser and never uploaded. It's resized
without smoothing, so only colors actually in the picture count.
- The colors reach the prompt as words: `Color grade: dominant colors
#204060, #e06060, #ffa060. Keep skin tones believable.`, plus a `Palette
priority: … is the main color` line when one is marked. So they add no
reference image, cost nothing extra and work with every model, including
those that can't take references.
- The existing Palette image slot is unchanged. Both can be used
together.
- The prompt wording and color counting are ported from Luis's branch.

### 3. Two request fixes

- **No watermark.** Seedream 5.0 Pro, both generate and edit, stamps a
visible watermark unless asked not to, because its Router contract
defaults `watermark` to true. Studio requests now turn off every
watermark switch a model's contract has, found by name.
- **No catalogue examples in studio requests.** `router_render` starts a
request from the model page's first example unless it's given a `form`.
The studio didn't intend that, but it happened:
  - FLUX 2 Max attached its example sofa photo to every take.
  - GPT Image asked for `quality: high`.
  - Krea asked for `creativity: low`.
- The edit operations carried an example input until a reference
replaced it.

Studio takes and their credit estimates now pass their own form: the
model's contract defaults with the shot on top. `request.test.ts` checks
every studio model and edit operation for catalogue links, stray files
and watermarks left on. It fails on FLUX 2 Max if the old behaviour is
put back.

## Why (image models)

Before this change the studio sent `{aspect_ratio, resolution}` and let
the shared Router parameter mapping pick the model's nearest size **by
pixel count, not by shape**. From main's own mapping:

Seedream 4.5 is no longer in the studio, but it shows the problem most
clearly:

| Model | Chosen | Sent before | Sent now |
|---|---|---|---|
| Seedream 4.5 | 1:1 · 1K | 2304×1728 (4:3) | 2048×2048 |
| Seedream 4.5 | 16:9 · 2K | 2496×1664 (3:2) | 2560×1440 |
| Seedream 5.0 Pro | 16:9 · 2K | 2048×2048 (square) | 1920×1080 |
| GPT Image 2 | 4:3 · 2K | 2048×2048 (square) | 4:3 not offered |

## Per-model frames

| Model | Frames | References |
|---|---|---|
| Seedream 5.0 Pro | 16:9, 3:2, 2:3, 1:1, 9:16 | via edit operation |
| Nano Banana Pro / 2 | all 7 | via edit operation |
| FLUX 2 Pro / Max | all 7 (free width/height) | itself |
| Qwen Image 3.0 Pro / 3.0 | 3:2, 2:3, 1:1 | itself |
| Krea 2 Large | all 7 (21:9 sent as its 2.35:1) | none |
| GPT Image 2 / 2.5 Flare / 2.5 Sunburst | 16:9, 3:2, 2:3, 1:1, 9:16 |
none: the Router has no image input for them yet (see below) |
| Grok Imagine Image 2.0 | all but 21:9 | none: the Router has no image
input for it yet (see below) |
| Recraft V4.1 | 16:9, 3:2, 2:3, 1:1, 9:16 | none |

A listed size counts as a frame when it's within 3% of it. For example,
Recraft's 1344×768 is 16:9 to 1.6%, and Krea's 2.35:1 is 21:9 to 0.7%.
Frames that are genuinely different shapes stay hidden: 4:3 and 3:2 are
12% apart.

## Open discussion (not a blocker): how Comfy Router maps aspect ratios

The shared Router mapping (`closestOption` in
`config/router-parameter-options.ts`) resolves a requested
`aspect_ratio`/`resolution` to the nearest listed size **by pixel
count**. For a single model page that's a reasonable fallback. For an
app like Cinematic Studio, which promises a frame across many models, it
silently changes the shape of the picture (see the table above). The
user asked for 1:1 and got 4:3, and nothing told them. Model-specific
size menus also differ a lot, e.g. Recraft's 1344×768 vs a true 16:9, or
Krea's 2.35:1 vs 21:9.

This PR works around it inside the studio by reading each contract's
size menu and sending an explicit size. It might be worth discussing
whether the Router (or the shared mapping) should:

1. match by shape first and then by size, or
2. report back when it had to change the shape, or
3. expose each model's supported aspect ratios as data, so apps don't
each have to parse size menus.

Happy to follow whatever direction the Router owners prefer. None of
this blocks the PR.

### Related: GPT Image and Grok image edits are not on the Router yet

GPT Image 2, 2.5 Flare and 2.5 Sunburst can edit images and take
reference images (OpenAI's edits endpoint and Comfy's own GPT Image
nodes support this). But the Router operation
`/v2/models/openai/gpt-image-2` (and the 2.5 variants) only accepts
generation fields (prompt, size, quality, background, output format).
That's why their edit pages were turned off in Comfy-Org#17951. Grok Imagine
Image 2.0's edit operation is in the same position.

Because the studio reads reference support from each Router contract,
these models show "can't use references" when a Cast or Palette image is
attached. When the Router adds image inputs for them, a contract refresh
should turn references on in the studio automatically, plus a one-line
`REFERENCE_OPERATIONS` entry if the edit ends up as a separate
operation.

## Also worth knowing

- **Reference uploads can't be tested on localhost.** Nano Banana Pro/2
and Qwen 3.0 / 3.0 Pro upload references to Comfy storage
(`api-nodes-prod`) before running. That bucket's CORS only allows
`https://comfy.org` and `https://platform.comfy.org`, so the browser
blocks the upload from any `localhost` port. This is the same for the
Models tab on main. Seedream and FLUX send references inline, so they
work locally. On comfy.org (and www, which redirects there) uploads
work. It might be worth adding localhost and preview origins to that
bucket's CORS so reviewers can test uploads; that's a non-blocking infra
question.

- **Credit estimates:** 3 of the 13 models have one (Seedream 5.0 Pro,
FLUX 2 Pro, FLUX 2 Max). The rest show "Cost varies", the same as Nano
Banana Pro, Krea and Qwen Pro already do on main, because the estimate
only covers models with verified pricing rules. The credit gate still
requires a positive balance.
- **Links from model pages:** the 9 new model pages now link to the
studio (via `cinematicStudioHref`). The Seedream 4.5 page no longer
does.

## Test plan

- [x] `colors.test.ts`, `CinematicColors.test.ts` and `prompt.test.ts`
cover the prompt words, the main-color line, the 8-color cap, near-shade
counting, and add/remove/main/clear. They also check that sampling from
an image fills the swatches and that an unreadable image shows an error.
- [x] New `frames.test.ts` runs against real Router contracts. It covers
the frames per model, the size sent (Seedream 1:1 → 2048×2048, Recraft
16:9 → 1344×768 through the real mapper), free-size models, reference
capacity, nearest-frame fallback, and the reference-count rule.
- [x] Rebased on current main (with Comfy-Org#19316). Full website suite: 510
files, 9,746 tests passing. Typecheck, knip and the fallow audit pass.
- [x] Browser: the side panel (default) lists 13 models; the Format menu
shows Recraft's 5 frames only; Colors appears in References (panel) and
in the + menu (composer), where it opens the Colors panel.
- [x] Checked in the browser, Colors: sampling a three-color picture
gives exactly those three colors, the star marks the main color, and the
credit estimate is unchanged. The control shows in both the popover
layout and the side-panel layout.
- [x] Checked in the browser, image models:
  - The model menu lists 13 models.
  - Recraft's Format menu lists only its 5 frames.
  - Switching from 21:9 to Recraft or GPT Image moves the frame to 16:9.
- With a Cast image attached, GPT Image 2 shows "can't use references"
and Generate stays blocked.
- [ ] Live signed-in run on a few new models (spends credits; not done
yet).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: luisvitela-hub <luisvitela-hub@users.noreply.github.com>
Co-authored-by: benjcooley <benjcooley@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants