Run Live Painting on SD 1.5 even when Text to Image is on Klein

docs/beta-gate.md

@@ -18,7 +18,7 @@ the same change.
 | G1 | **Runs on a current ComfyUI.** Every runnable preset's nodes and inputs match a recorded `/object_info` from the newest ComfyUI release, and a test fails when they drift. | Roadmap U2: a snapshot in the repo, a test of every preset's `requiredNodes` against it, and a script that diffs a live ComfyUI against the snapshot. Its output against a current ComfyUI is pasted in the PR. | **Done in v0.41.** `tests/fixtures/comfyui-object-info.json` is recorded from ComfyUI 0.44.0 (80 node classes) by `npm run object-info`, and `tests/comfy/objectInfoSnapshot.test.ts` checks every node, input, nested input and fixed choice of all 39 presets against it. All match. Running Workflow Health against 0.44 found a real fault: it read only required inputs and marked 13 working presets "Missing ComfyUI node"; fixed, it now reports 28 ready, 10 experimental and 1 model in the wrong folder on the test machine. `npm run object-info -- --check` against a fresh Comfy Desktop 0.38.1 lists only the custom node packs it has not installed. |
 | G2 | **Finds Comfy Desktop.** The port scan includes Comfy Desktop's default port. | A test on `COMFY_PORT_CANDIDATES`, plus Settings › Find ComfyUI Active Port against a ComfyUI started on that port. | **Done in v0.41.** 8000 is scanned second, right after 8188, with a test. Not yet run against a real Comfy Desktop: on the test machine a ComfyUI on 8188 always answers first. A clean-install tester (G6) using Comfy Desktop checks it for real. |
 | G3 | **Says so when there is no GPU.** When ComfyUI reports only a CPU device, the panel says which tools will be too slow to use, rather than failing or hanging. | A unit test with a CPU-only `/system_stats` fixture, plus one run against ComfyUI started with `--cpu`. | **Done in v0.41.** Check ComfyUI says "ComfyUI is online, but on CPU only." with the reason underneath, and Detect Hardware reports "Processor only, no graphics card", VRAM none, tier No GPU. A CPU device no longer counts its system RAM as VRAM. Checked in Photoshop 2025 on 2026-11-04 against a real Comfy Desktop instance started with `--cpu` (ComfyUI 0.38.1, port 8189), and against a GPU ComfyUI 0.44.0 on 8188, which still reports normally. |
-| G4 | **Passes the full smoke checklist on the release build.** All blocks of [the UXP smoke checklist](uxp-smoke-checklist.md) on the build that will be tagged, with a row in its Results table. | The Results table. | **Passed on 2026-11-05 on main at fd679fc** (footer 0.40.0, ComfyUI 0.44.0, Photoshop 2025): blocks 0 to 5, every step, with five findings recorded in the Results table. Run again on the exact build that is tagged if code changes before then. The first finding (Live Painting fails while Text to Image is on Klein, the default since v0.39) hits every new user, so fix it before the beta. |
+| G4 | **Passes the full smoke checklist on the release build.** All blocks of [the UXP smoke checklist](uxp-smoke-checklist.md) on the build that will be tagged, with a row in its Results table. | The Results table. | **Passed on 2026-11-05 on main at fd679fc** (footer 0.40.0, ComfyUI 0.44.0, Photoshop 2025): blocks 0 to 5, every step, with five findings recorded in the Results table. Run again on the exact build that is tagged if code changes before then. The first finding (Live Painting fails while Text to Image is on Klein, the default since v0.39) hits every new user, so fix it before the beta. Fixed in v0.41: Live Painting now picks an SD 1.5 checkpoint itself and leaves Text to Image alone. |
 | G5 | **Works on Photoshop 2024.** Someone other than the author runs blocks 0 to 2 of the smoke checklist on Photoshop 2024 with the beta candidate. | Their report, linked here. | **Open.** One tester confirmed the panel works on Photoshop 2024 with v0.37.1 ([issue 127](https://github.com/MehranMarxian/OpenLayer/issues/127)). Nothing newer has been run on 2024. |
 | G6 | **A first image on a clean install.** From an empty ComfyUI to a first layer by following [Getting started](getting-started.md), on a machine that is not the author's. | The tester's step count and any place they got stuck, linked here. | **Open.** v0.39 timed it at seven clicks on the author's machine only. |
 

src/ui/App.ts

@@ -302,6 +302,8 @@ import {
 } from "./historyMetadata";
 import {
   getLivePaintingStateBadgeLabel,
+  chooseLivePaintingCheckpoint,
+  type LivePaintingCheckpointChoice,
   LivePaintingSessionV2,
   type LivePaintingState
 } from "./tools/livePainting";
@@ -7845,10 +7847,26 @@ export function renderApp(rootElement: HTMLElement) {
       return;
     }
 
-    const checkpointName = readSelectValue(elements.checkpoint);
+    // The live graph is SD 1.5 only, so the Text to Image model is used only
+    // when it is an SD checkpoint; see chooseLivePaintingCheckpoint.
+    setLiveStatus("Finding an SD 1.5 checkpoint for the live tier...");
+    let checkpointChoice: LivePaintingCheckpointChoice;
+
+    try {
+      checkpointChoice = chooseLivePaintingCheckpoint(
+        readSelectValue(elements.checkpoint),
+        withoutSegmentationModels(await new ComfyClient(elements.serverUrl.value).getCheckpointNames()),
+        RECOMMENDED_SKETCH_CHECKPOINT
+      );
+    } catch (caughtError) {
+      setLiveStatus(getErrorMessage(caughtError));
+      return;
+    }
+
+    const checkpointName = checkpointChoice.checkpointName;
 
     if (!checkpointName) {
-      setLiveStatus("Choose a checkpoint on the Text to Image screen first.");
+      setLiveStatus(checkpointChoice.message);
       return;
     }
 
@@ -7892,6 +7910,11 @@ export function renderApp(rootElement: HTMLElement) {
 
     try {
       await session.start();
+
+      if (checkpointChoice.substituted && livePaintingSession === session && session.isRunning()) {
+        setLiveStatus(checkpointChoice.message);
+      }
+
       void session.checkRefineAvailability()
         .then((gapMessage) => {
           if (gapMessage && livePaintingSession === session && session.isRunning()) {

src/ui/appMarkup.ts

@@ -642,11 +642,11 @@ export function createAppMarkup() {
             <span class="muted-label">Two-tier session</span>
           </div>
           <div class="diagnostics-line live-hint">
-            The live tier runs the model picked in the Model dropdown on the Text to Image screen,
-            paired with your local SD 1.5 LCM LoRA. Refine uses Krea-2 Turbo.
-            If no model is selected there yet, open Text to Image, choose a workflow and a model,
-            then come back here. Once the session is running, paint in the document and the preview
-            follows your strokes.
+            The live tier runs an SD 1.5 checkpoint with your local LCM LoRA: the Text to Image
+            model when that is SD 1.5, otherwise the first SD 1.5 checkpoint ComfyUI has. FLUX.2
+            Klein and the other large models are too slow to follow strokes live. Refine uses
+            Krea-2 Turbo. Once the session is running, paint in the document and the preview follows
+            your strokes.
           </div>
           <label class="field">
             <span class="label">Prompt${createPromptWalletControlsMarkup("live-prompt")}</span>

src/ui/tools/livePainting.ts

@@ -18,6 +18,7 @@ import {
   type LiveCycleSample
 } from "../livePaintingTimings";
 import { createOpenLayerError, getErrorMessage } from "../../utils/errors";
+import { detectCheckpointFamily } from "../../comfy/modelCompatibility";
 
 export type LivePaintingState =
   | "idle"
@@ -43,6 +44,59 @@ export function getLivePaintingStateBadgeLabel(state: LivePaintingState): LivePa
   }
 }
 
+/**
+ * The checkpoint the live tier runs. Its graph is SD 1.5 plus the LCM LoRA
+ * through CheckpointLoaderSimple, so it cannot take whatever the Text to Image
+ * Model dropdown holds: since v0.39 a starter install opens Text to Image on
+ * FLUX.2 Klein, a diffusion model in another folder, and every live cycle
+ * failed with "The model flux-2-klein... ". Beta gate G4, finding 1.
+ *
+ * - The Text to Image model, when ComfyUI lists it as a checkpoint and it is
+ *   not a known non-SD-1.5 family. "unknown" is kept: an SD 1.5 fine-tune with
+ *   an unhelpful file name is the common case, and it ran before.
+ * - Otherwise the recommended SD 1.5 checkpoint, if installed.
+ * - Otherwise the first checkpoint whose name says SD 1.x.
+ * - Otherwise nothing, with a message that names what to install.
+ *
+ * Text to Image's own selection is never changed.
+ */
+export type LivePaintingCheckpointChoice =
+  | Readonly<{ checkpointName: string; substituted: boolean; message: string }>
+  | Readonly<{ checkpointName: null; substituted: false; message: string }>;
+
+export function chooseLivePaintingCheckpoint(
+  textToImageModel: string,
+  installedCheckpoints: readonly string[],
+  preferredCheckpoint: string
+): LivePaintingCheckpointChoice {
+  const family = textToImageModel ? detectCheckpointFamily(textToImageModel) : "unknown";
+
+  if (textToImageModel && installedCheckpoints.includes(textToImageModel) && (family === "sd1" || family === "unknown")) {
+    return { checkpointName: textToImageModel, substituted: false, message: "" };
+  }
+
+  const fallback = installedCheckpoints.includes(preferredCheckpoint)
+    ? preferredCheckpoint
+    : installedCheckpoints.find((name) => detectCheckpointFamily(name) === "sd1");
+
+  if (!fallback) {
+    return {
+      checkpointName: null,
+      substituted: false,
+      message:
+        "Live Painting runs on an SD 1.5 checkpoint, and ComfyUI has none in models/checkpoints. Install one, for example epicrealism_naturalSinRC1VAE.safetensors, then start the session again."
+    };
+  }
+
+  return {
+    checkpointName: fallback,
+    substituted: true,
+    message: textToImageModel
+      ? `Live Painting runs on SD 1.5, so it uses ${fallback} instead of ${textToImageModel}.`
+      : `Live Painting runs on SD 1.5 and uses ${fallback}.`
+  };
+}
+
 export type LivePaintingV2Callbacks = {
   onStatus: (message: string) => void;
   onTimings: (message: string) => void;

tests/ui/livePaintingCheckpoint.test.ts

@@ -0,0 +1,57 @@
+import { describe, expect, it } from "vitest";
+import { chooseLivePaintingCheckpoint } from "../../src/ui/tools/livePainting";
+
+const EPIC = "epicrealism_naturalSinRC1VAE.safetensors";
+const KLEIN = "flux-2-klein-4b-fp8.safetensors";
+
+/**
+ * Live Painting's graph is SD 1.5 + the LCM LoRA through CheckpointLoaderSimple.
+ * A starter install opens Text to Image on FLUX.2 Klein, and Live Painting used
+ * to pass that file straight in: every cycle failed. Beta gate G4, finding 1.
+ */
+describe("chooseLivePaintingCheckpoint", () => {
+  it("uses an SD 1.5 checkpoint instead of FLUX.2 Klein, and says so", () => {
+    const choice = chooseLivePaintingCheckpoint(KLEIN, ["sd_xl_base_1.0.safetensors", EPIC], EPIC);
+
+    expect(choice.checkpointName).toBe(EPIC);
+    expect(choice.substituted).toBe(true);
+    expect(choice.message).toBe(`Live Painting runs on SD 1.5, so it uses ${EPIC} instead of ${KLEIN}.`);
+  });
+
+  it("keeps the Text to Image model when it is an SD 1.5 checkpoint", () => {
+    const choice = chooseLivePaintingCheckpoint("dreamshaper_v1.5.safetensors", ["dreamshaper_v1.5.safetensors", EPIC], EPIC);
+
+    expect(choice).toEqual({ checkpointName: "dreamshaper_v1.5.safetensors", substituted: false, message: "" });
+  });
+
+  it("keeps a checkpoint whose family the name does not reveal, as before", () => {
+    // An SD 1.5 fine-tune with an unhelpful name is the common case.
+    expect(chooseLivePaintingCheckpoint("myStyle_final.safetensors", ["myStyle_final.safetensors", EPIC], EPIC).checkpointName)
+      .toBe("myStyle_final.safetensors");
+  });
+
+  it("does not run SDXL live, which the LCM SD 1.5 LoRA cannot drive", () => {
+    expect(chooseLivePaintingCheckpoint("sd_xl_base_1.0.safetensors", ["sd_xl_base_1.0.safetensors", EPIC], EPIC).checkpointName)
+      .toBe(EPIC);
+  });
+
+  it("falls back to the first SD 1.x checkpoint when the recommended one is missing", () => {
+    expect(chooseLivePaintingCheckpoint(KLEIN, ["sd_xl_base_1.0.safetensors", "v1-5-pruned-emaonly.safetensors"], EPIC).checkpointName)
+      .toBe("v1-5-pruned-emaonly.safetensors");
+  });
+
+  it("refuses with a plain message when there is no SD 1.5 checkpoint at all", () => {
+    const choice = chooseLivePaintingCheckpoint(KLEIN, ["sd_xl_base_1.0.safetensors"], EPIC);
+
+    expect(choice.checkpointName).toBeNull();
+    expect(choice.message).toMatch(/SD 1\.5 checkpoint/);
+    expect(choice.message).not.toMatch(/`|--|\*\*/);
+  });
+
+  it("picks one when Text to Image has no model selected yet", () => {
+    const choice = chooseLivePaintingCheckpoint("", [EPIC], EPIC);
+
+    expect(choice.checkpointName).toBe(EPIC);
+    expect(choice.message).toBe(`Live Painting runs on SD 1.5 and uses ${EPIC}.`);
+  });
+});