Skip to content

feat(website): Hand product swap app (mocked) - #20865

Open
mariana953 wants to merge 12 commits into
mariana/editor-kitfrom
mariana/app-hand-product-swap
Open

mariana953 wants to merge 12 commits into
mariana/editor-kitfrom
mariana/app-hand-product-swap

Conversation

@mariana953

@mariana953 mariana953 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

New Workshop app at /hub/apps/hand-product-swap/: drop a photo of a hand holding something, pick a product, and get the same hand and grip holding the new product. It works like the reference Hand product swap app (inputs Hand + Product, Resolution 1K/2K/4K with 2K as default, seed, Result/Compare views, run label "Swap product"), inside the shared full-screen editor. The backend is mocked. Built on the shared editor kit (#20630), with no dependency on Relight or Move anything. Replaces #19820. The image loader is now lib/workshop/load-image.ts and the box type lives in placement.ts, so nothing is borrowed from another app.

Changes

  • What:
    • Hand photo large in the centre, with the product as a reference card on it (click it to change the product, or drop an image on it). The tagline "Same hand & grip, new product" shows on the photo (desktop) and in the empty state.
    • Placement box removed. The backend takes no placement field: the model finds what the hand holds and keeps the same hand and grip. The box (drag, resize, arrow keys, Redraw box, Reset box) mapped to nothing, so it is gone. Undo and redo now cover the product and resolution.
    • Floating left panel: hand photo row, Product (example tiles can, serum, cream, plus upload), then Resolution (shared output menu, each size with its output pixels) and Seed (shared seed field with a new-seed button) as direct rows. There is no Advanced section. In layout E the bottom composer has Product and Seed chips that open trays, Resolution as a pill, and the run button. Phones get the bottom sheet with a summary row and the run button.
    • Result opens on Result. The kit's EditorCompareToggle in the pill turns on Compare (the before/after split line through EditorResult/EditorCompare). The pill also has Edit and Try again. Download is in the header, right of GitHub.
    • Wiring (unchanged): catalogue jsonl, display json, zod enums, hub-app-names.json, routes.ts, apps.ts, CinematicStudioPage, availability, vercel.json, en + zh-CN copy in lib/workshop/hand-product-swap/copy.ts. Staff flag: workshop-hand-product-swap-app-enabled.
    • Example photos (unchanged): hand.jpg, product-{can,serum,tube}.jpg, result-{can,serum,tube}.jpg.
    • Shared kit adopted (from feat(website): shared full-screen editor pieces for the Workshop apps #20630): the product grid is the kit's EditorTiles (4 square columns, custom white-backed tiles, trailing upload tile); the hand row's Change, the product card and the upload tile use EditorUploadSlot; the stage drop uses EditorDropZone (now with the hover highlight); paste uses useImagePaste; the run button shows the job's progress (Queued, then the percent) through EditorRun's progress. Removed the local HandSwapUpload, HandSwapProductTile and files.ts (firstImage).
  • Additions (small, from the parity brief):
    • Drag and drop an image onto the stage (sets the hand photo), onto the product card, the product upload tile or the hand row's Change button.
    • Paste an image from the clipboard. It sets the hand photo until there is one, then the product.
    • Run progress: the wait shows "Queued", then the percent, plus the product, resolution and elapsed time, on the photo and on the run button.
    • Download name: <hand photo name>-swapped-<seed>.<result type>, for example hand-holding-can-swapped-42.jpg.
    • Not added: a "what's missing" run label. A product is always selected and the run button only shows once there is a hand photo, so nothing can be missing when the button is visible.

Backend contract

HandSwapRequest in apps/website/src/lib/workshop/hand-product-swap/contract.ts maps 1:1 to the multipart fields of POST /api/run/hand-product-swap:

{
  hand: string                     // photo of a hand holding something
  product: string                  // product image (cut-out PNG or shot on white)
  resolution: '1K' | '2K' | '4K'   // default '2K'
  seed: number                     // default 42
}
// progress → { kind: 'queued' } | { kind: 'running', percent: number }
// result   → { url: string, seed: number }

No extension fields. Credits shown: 14 / 14 / 25.

Mock: runHandSwap (mock-run.ts) reports queued for 0.4 s, then a climbing percent for 2 s. For the example hand with one of the three example products it answers with that product's example photo. Anything else is composited on a canvas: white backdrops keyed out, then drawn at the example's known grip or a guessed centred grip. That guess is internal to the mock and is not sent.

Review Focus

  • Removing the placement box (see above). placement.ts (gripRect) is now used only by the mock's canvas drawing.
  • useHandProductSwap.ts: one phase union (editing | running{progress} | done | failed). A dropped or pasted image is ignored while a run is going. A new product arriving over a result goes back to editing.
  • Kit pieces used: EditorCompareToggle, EditorResult, EditorOutput, EditorSeedField, EditorPanelRow, EditorTiles, EditorUploadSlot, EditorDropZone, useImagePaste and EditorRun progress. No kit files change in this PR.
  • Tests: contract, mock (examples table, canvas fallback, progress, cancel), download name, composable (exact request, undo/redo, paste, progress), component (Result then Compare, no box, drop and paste, pill order in layouts D/E) and e2e.
  • Shared with the other app PRs. The app registers itself in the kit's EDITOR_APPS list on the studio page (one entry), and its copy goes through lib/workshop/app-copy.ts. Shared helpers (app-copy.ts, and where used load-image.ts, nudge.ts, useSnapshotHistory.ts) are identical in every app PR, so they merge cleanly whichever lands first.

Screenshots (if applicable)

Desktop 1440 with the example, running at 40%, Result and Compare, layout E, and phone 390 (sheet, result, layout E) were reviewed locally. The phone has no horizontal overflow.


Generated by Claude Code

@mariana953
mariana953 requested review from a team and dante01yoon as code owners October 12, 2026 20:06
@coderabbitai

coderabbitai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: ccda6147-ee74-4ab8-b4de-197ca25763d2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

📦 Bundle Size

⏳ Size data collection in progress…

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since fa6d451f5d507b9e31956425e939e0ee8d12cd1d

Full docs: ADR-DEVEX-LINT-0015

🌐 Website E2E

Tip

All tests passed.

Status ✅ Passed
Report View Report

🔗 Website Preview

Website Preview: https://comfy-website-preview-pr-20865.vercel.app

Models is built and gated by PostHog (workshop-enabled), using staging Cloud.

This commit: https://website-frontend-f8ry2ghl9-comfyui.vercel.app

Last updated: 2026-10-14T01:10:07Z for 36d6c3a

@codecov

codecov Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.88076% with 89 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
.../src/lib/workshop/hand-product-swap/render-swap.ts 2.22% 44 Missing ⚠️
apps/website/src/composables/useHandProductSwap.ts 82.85% 4 Missing and 14 partials ⚠️
apps/website/src/lib/workshop/load-image.ts 0.00% 6 Missing ⚠️
...nents/workshop/hand-product-swap/HandSwapTrays.vue 42.85% 4 Missing ⚠️
...onents/workshop/hand-product-swap/HandSwapSeed.vue 0.00% 3 Missing ⚠️
...onents/workshop/hand-product-swap/HandSwapMain.vue 75.00% 1 Missing and 1 partial ⚠️
...ents/workshop/hand-product-swap/HandSwapStudio.vue 91.66% 2 Missing ⚠️
...ts/workshop/hand-product-swap/HandSwapComposer.vue 75.00% 1 Missing ⚠️
...nents/workshop/hand-product-swap/HandSwapPanel.vue 93.33% 0 Missing and 1 partial ⚠️
.../workshop/hand-product-swap/HandSwapResolution.vue 85.71% 0 Missing and 1 partial ⚠️
... and 7 more

❌ Your patch check has failed because the patch coverage (75.88%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

@@                  Coverage Diff                  @@
##             mariana/editor-kit   #20865   +/-   ##
=====================================================
  Coverage                      ?   85.78%           
=====================================================
  Files                         ?     3580           
  Lines                         ?   169411           
  Branches                      ?    46413           
=====================================================
  Hits                          ?   145333           
  Misses                        ?    22160           
  Partials                      ?     1918           
Flag Coverage Δ
website-unit 80.57% <75.88%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...ebsite/src/components/workshop/ModelsCatalogue.vue 100.00% <100.00%> (ø)
.../workshop/cinematic-studio/CinematicStudioPage.vue 91.46% <100.00%> (ø)
...nents/workshop/hand-product-swap/HandSwapDocks.vue 100.00% <100.00%> (ø)
...nts/workshop/hand-product-swap/HandSwapHistory.vue 100.00% <100.00%> (ø)
...workshop/hand-product-swap/HandSwapProductCard.vue 100.00% <100.00%> (ø)
...ts/workshop/hand-product-swap/HandSwapProducts.vue 100.00% <100.00%> (ø)
.../workshop/hand-product-swap/HandSwapResultDock.vue 100.00% <100.00%> (ø)
...ponents/workshop/hand-product-swap/HandSwapRun.vue 100.00% <100.00%> (ø)
...nts/workshop/hand-product-swap/HandSwapShimmer.vue 100.00% <100.00%> (ø)
...nents/workshop/hand-product-swap/HandSwapStage.vue 100.00% <100.00%> (ø)
... and 28 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Oct 12, 2026

@dante01yoon dante01yoon left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Removing the placement box that mapped to nothing was the right call, and the request shape is simple. Requesting changes for the copy approach (shared with the sibling app PRs) and internal comments.

Size. This PR is about 2,650 lines (excluding binaries). That is far past the ~300-line guideline and hard to review well in one pass. Could you split it, e.g. (1) wiring: catalogue, routes, flag, Hub card; (2) pure lib/ logic with its tests; (3) components and e2e?

Stacking. Base is mariana/editor-kit (#20630), which is still open with changes requested. Every app PR edits the same shared lists (routes.ts, apps.ts, catalogue-apps.ts, CinematicStudioPage.vue, schema enums, locales, vercel.json), so they will conflict with each other as they land. The identical helpers (app-copy.ts, load-image.ts, useSnapshotHistory.ts, nudge.ts) would be better landed once in #20630 than duplicated in every app branch.

Checks: CI is green; I didn't run tests locally.

named: CopyValues = {}
): string {
const [one, many = one] = (entry[locale] ?? entry.en).split(' | ')
return (named.n === 1 ? one : many).replace(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: appCopy fills placeholders with .replace and parses | plurals by hand, and the app's strings live in a TS copy.ts instead of src/locales/<locale>/main.json. apps/website/AGENTS.md says copy goes in main.json and is read with translationsFor(locale).t, and to "never fill a placeholder with .replace". Doing it this way also skips the catalog test that compiles each message and checks zh-CN coverage. It also treats a literal {word} the visitor types (a star name, a light name) as a placeholder whenever named has that key.

Could the app's strings move under a namespaced key in main.json (moveAnything.*, relight.*, …) and be read through t(key, named, { locale })? If keeping them out of the site-wide bundle is the real goal (as for catalogue-apps.ts), please get that exception agreed on #20630 first, where all seven apps can share it, without hand-rolled interpolation.

})
}

/** A pasted image is the hand photo until there is one, then the product. */

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: this PR adds comments to internal, non-exported code, like this one and the others on inner functions of this composable. The repo rule only allows a short JSDoc on exported API. Please drop the internal ones and let the names carry the meaning.

}

/** Swaps the present without a new step, for the moves of one drag. */
export function replace<T>(history: History<T>, present: T): History<T> {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: checkpoint/replace only existed for the placement-box drag, which this PR removes. They're now exercised only by history.test.ts. This file is also a third undo implementation next to useSnapshotHistory (in the sibling PRs). Could the composable use useSnapshotHistory and drop history.ts and its test?

reset({ url, name, width, height })
}

async function decode(file: File): Promise<SwapImage | undefined> {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: decode shares one pendingUrl between hand and product uploads. Dropping a hand photo and then quickly a product (or pasting twice) makes the first decode fail the pendingUrl !== url check, so that upload is silently dropped. Per-slot pending tokens would avoid it. (PLAUSIBLE)

* the hand holds by itself and keeps the same hand and grip, so there is no
* placement to send.
*/
export interface HandSwapRequest {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: HandSwapRequest is a hand-written request/response contract for a backend that doesn't exist yet. Is there a tracking issue for the endpoint? Once it ships in the OpenAPI spec, these types should be imported from @comfyorg/ingest-types rather than kept by hand. Could you link the issue here (or in the PR body) so the swap isn't forgotten?

This branch was previously deployed

1 inactive deployment
website-approval — 36d6c3af Deployed Oct 14, 2026 by mariana953 via fast-lane #1918
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants