Skip to content

Run Live Painting on SD 1.5 even when Text to Image is on Klein - #172

Merged
MehranMarxian merged 1 commit into
mainfrom
fix/live-painting-model
Nov 5, 2026
Merged

MehranMarxian merged 1 commit into
mainfrom
fix/live-painting-model

Conversation

@MehranMarxian

@MehranMarxian MehranMarxian commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

What changed and why

Beta gate G4, finding 1. Live Painting's live tier is SD 1.5 plus the LCM LoRA, loaded through CheckpointLoaderSimple, but it took whatever the Text to Image Model dropdown held. Since v0.39 a starter install opens Text to Image on FLUX.2 Klein, a diffusion model in another folder. So on a fresh install every live cycle failed with "Live cycle failed (1/3 consecutive): The model flux-2-k…", and every new user would hit it.

The fix: chooseLivePaintingCheckpoint, a pure function in src/ui/tools/livePainting.ts with tests, picks the live checkpoint:

  1. Text to Image's model, when ComfyUI lists it as a checkpoint and its name reads SD 1.x or gives no family. An SD 1.5 fine-tune with an unhelpful file name is the common case, and it worked before, so it's kept.
  2. Otherwise the recommended epicrealism_naturalSinRC1VAE.safetensors, if installed.
  3. Otherwise the first checkpoint whose name says SD 1.x.
  4. Otherwise it doesn't start. It says: "Live Painting runs on an SD 1.5 checkpoint, and ComfyUI has none in models/checkpoints. Install one, for example epicrealism_naturalSinRC1VAE.safetensors, then start the session again."

When it uses another checkpoint, the status reads, for example: "Live Painting runs on SD 1.5, so it uses epicrealism_naturalSinRC1VAE.safetensors instead of flux-2-klein-4b-fp8.safetensors." Text to Image's own selection is never changed.

Panel text: the Live Painting hint (appMarkup.ts, text only, no CSS) now says this, instead of telling people to pick a model on Text to Image. The G4 row in docs/beta-gate.md notes the fix.

Verified

npm run typecheck   exit 0
npm run lint        exit 0
npm test            Test Files 141 passed (141), Tests 1387 passed (1387)
npm run build       "Copied UXP assets to dist."

tests/ui/livePaintingCheckpoint.test.ts (7 tests) covers:

  • Klein falls back to epicRealism, with the message;
  • an SD 1.5 model and an unknown-family model are kept;
  • SDXL is not run live;
  • with epicRealism missing, it uses the first SD 1.x checkpoint;
  • with no SD 1.5 at all it refuses, in plain text;
  • with no model selected it picks one.

Checked in Photoshop 2025 by Claude, after a reload, with Text to Image left on txt2img-flux2-klein (the exact failing case from the G4 run):

  • Start Live Session, then paint a stroke. Cycles 1 to 3 succeeded, and the preview followed the stroke.
  • ComfyUI's history shows all three cycles ran epicrealism_naturalSinRC1VAE.safetensors with the SD 1.5 LCM LoRA (lcm\SD1.5\pytorch_lora_weights.safetensors), status success.
  • Text to Image was still on Klein afterwards.

Could NOT be verified

  • The substitution note was barely visible. It shows only until the first cycle replaces the status line, which took about a second here.
  • The "no SD 1.5 checkpoint" message wasn't seen in Photoshop: this machine has four SD 1.x checkpoints. The unit test covers it.

What Mehran should check in Photoshop

Smoke checklist blocks (docs/uxp-smoke-checklist.md):

  • Block 0 (the panel starts)
  • Block 5 (Live Painting)

Load: UXP Developer Tool › OpenLayer › Reload.

  • 1. Text to Image › Workflow txt2img-flux2-klein. Then Home › Live Painting: read the new hint.
  • 2. Start Live Session, paint two strokes on an empty layer. See: the preview updates; no "Live cycle failed".
  • 3. Stop Live Session, then go back to Text to Image. See: still on Klein.

🤖 Generated with Claude Code

…on Klein

The live tier is SD 1.5 plus the LCM LoRA through CheckpointLoaderSimple,
but it took whatever the Text to Image Model dropdown held. Since v0.39 a
starter install opens Text to Image on FLUX.2 Klein, a diffusion model in
another folder, so every live cycle failed with "The model flux-2-k..."
for every new user. Beta gate G4, finding 1.

chooseLivePaintingCheckpoint keeps the Text to Image model when ComfyUI
lists it as a checkpoint and it is SD 1.5 or of unknown family (an SD 1.5
fine-tune with an unhelpful name, which ran before), and otherwise uses
the recommended epicRealism or the first SD 1.x checkpoint, saying which.
With none installed it says what to install instead of starting.
Text to Image's own selection is not changed. The Live Painting hint now
says this instead of telling people to pick a model on Text to Image.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MehranMarxian
MehranMarxian merged commit 51aa56c into main Nov 5, 2026
1 check passed
MehranMarxian added a commit that referenced this pull request Nov 5, 2026
CHANGELOG section checked against git log v0.40.0-alpha..main (#167 to
#172 plus the form fix on this branch): the Live Painting, Workflow
Health and CPU-VRAM fixes, port 8000, the CPU-only message, the
object_info snapshot test, and the beta gate and smoke run.

README: the "New in" item describes v0.41 as the beta candidate, the
v0.40 item stays as "In v0.40", and the verified line names ComfyUI
0.44.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MehranMarxian MehranMarxian mentioned this pull request Nov 5, 2026
2 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant