diff --git a/.agents/skills/converting-custom-nodes/SKILL.md b/.agents/skills/converting-custom-nodes/SKILL.md new file mode 100644 index 00000000000..7e8c0fbca2a --- /dev/null +++ b/.agents/skills/converting-custom-nodes/SKILL.md @@ -0,0 +1,11 @@ +--- +name: converting-custom-nodes +description: 'Converts third-party custom-node JS off deprecated/unpublished ComfyUI APIs onto the published node API. Use for Magic Patch conversion work, migrating a pack, or reviewing a generated patch. Triggers on: convert custom node, magic patch, migrate pack, port to node API, output.links, input.link, widgets.splice, converted-widget.' +--- + +# Converting Custom Nodes + +The canonical cross-agent instructions live at +`../../../.claude/skills/converting-custom-nodes/SKILL.md`. Read that file +completely before converting or reviewing a pack, and resolve its pattern +references from the canonical skill directory. diff --git a/.claude/skills/converting-custom-nodes/SKILL.md b/.claude/skills/converting-custom-nodes/SKILL.md new file mode 100644 index 00000000000..ec949344e6b --- /dev/null +++ b/.claude/skills/converting-custom-nodes/SKILL.md @@ -0,0 +1,556 @@ +--- +name: converting-custom-nodes +description: 'Converts third-party custom-node JS off deprecated/unpublished ComfyUI APIs onto the published node API. Use for Magic Patch conversion work, migrating a pack, or reviewing a generated patch. Triggers on: convert custom node, magic patch, migrate pack, port to node API, output.links, input.link, widgets.splice, converted-widget.' +--- + +# Converting Custom Nodes + +Converts third-party pack JS from the old, unpublished ComfyUI internals onto +the published node API (`src/platform/nodeApi/`, documented in `docs/node-api/`). +Use `docs/node-api/comfy-api.d.ts` as the exact machine-readable contract. If a +member is absent there, it is not published. + +This is **migration, not compatibility work**. The goal is that the old surface +can be _deleted_, so never add a shim — rewrite the call site. + +## When to Use + +- Converting a pack that Magic Patch escalated (`convert()` reported it in + `escalated`, meaning the mechanical rules deliberately refused). +- Hand-writing a conversion for an upstream PR. +- Reviewing a generated patch before it ships. + +Do **not** use it to make broken code work again by any means available. A +conversion that reintroduces the old coupling is worse than no conversion. + +## The two invariants + +Everything below serves these. If a conversion violates either, it is wrong even +if the pack appears to work. + +**1. The wire format must be byte-identical.** +`graphToPrompt` output and the serialized workflow must not change. This is the +frontend's contract with the backend, and it holds for 1,801/1,801 packs today — +which makes it an extremely sharp detector. The most common way to break it is +disconnect-and-reconnect, which allocates **new link ids**. Use +`output.moveLinksTo()` when re-homing links. + +**2. Behaviour must be equivalent, except where the old code threw.** +The generated test has two halves: `equivalence` (must pass on the original +_and_ the converted source) and `fix` (must fail before, pass after). If you +cannot state an equivalence claim, you do not yet understand the code well +enough to convert it. + +## Steps + +### 1. Read what the code is actually doing + +Do not pattern-match on the API name. The census surfaces are misleading: + +- A `widgets.splice(i, 1)` immediately followed by `splice(i, 0, w)` is **not a + reorder** — it is a cache-invalidation hack. Replace it with + `widget.setOption(key, value)`, which invalidates properly. +- `onDrawForeground` is often **not drawing**. 47% of draw-callback bodies never + touch a drawing primitive; they enforce size, poll for changes, or sync DOM + visibility. Those become `setSizeConstraints`, `widget.on('change')`, and the + widget mount lifecycle respectively. + +### 2. Check whether the object is live or serialized + +**The single most dangerous confusion in this work.** These look identical: + +```js +node.inputs[0].link = null // live slot — convert +p.workflow.nodes[i].inputs[0].link = null // serialized JSON — LEAVE ALONE +``` + +The second is correct as written; rewriting it corrupts a working pack. Trace +the variable to its origin before touching anything named `link`, `links`, +`inputs`, `outputs` or `widgets`. If it came from `graphToPrompt`, a fetch, a +`JSON.parse`, or a `.workflow` property, it is data — not a graph. + +### 3. Apply the mapping + +| Old | New | Notes | +| ---------------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------------------- | +| `output.links.push(id)` | `output.connectTo(nodeId, inputRef)` | creating a connection | +| moving links between own outputs | `output.moveLinksTo(ref)` | **preserves link ids** — required for the wire gate | +| `output.links` (read) | `output.links()` | frozen snapshot, safe to iterate while disconnecting | +| `input.link = null` | `input.disconnect()` | check step 2 first | +| `input.link` (read) | `input.source()` / `input.isConnected` | | +| `node.type = x` | delete the line | usually a defensive no-op; true type replacement is a **gap** — punt it | +| `slot.type = x`, `slot.name = x` | `slot.modify({ type, name })` | atomic, one undo step; keeps existing links | +| `widget.type = 'converted-widget'` | `widget.setHidden(true)` | ⚠️ old hack also suppressed serialization — see `references/widgets.md` | +| `widgets.splice` to reorder | `widgets.reorder(names)` | throws on a partial list instead of dropping widgets | +| `widgets.splice(i,1)` + `splice(i,0,w)` | `widget.setOption(key, value)` | same index in/out = cache invalidation, **not** a reorder | +| `widgets.push(w)` | `widgets.add(def)` | | +| `widgets = [...]` / `widgets.length = n` | `widgets.remove(name)` | assignment drops renderer tracking; length skips teardown | +| `getCustomWidgets` POJO | `defs.defineWidgetType(type, { render })` | preserves the backend-declared widget's wire cell | +| `{...input}` / `{...node}` | `input.snapshot()` / `node.snapshot()` | accessors moved to the prototype, so spread yields nothing | +| `nodeType.prototype.onNodeCreated = ...` | `defs.extend(sel, (b) => b.onCreated((node) => { ... }))` | selector = the hook's existing guard clause | +| `nodeType.prototype.onExecuted = ...` | `b.onExecuted((node, result) => { ... })` | see `references/node-definitions.md` | +| `nodeType.prototype.onConfigure = ...` | `b.onConfigured((node, data) => { ... })` | | +| `nodeType.prototype.onRemoved = ...` | `b.onRemoved((node) => { ... })` | | +| `widget.inputEl` | `widgets.mount({ name, render })` | the element arrives as `render`'s argument; there is no `inputEl` to read | +| `this.widgets.length = n` | remove by name | assigning length skips widget teardown | +| `+!!this.inputs[0].widget` | `input.isWidgetInput` | converted-widget sniffing | +| `onDrawForeground` (drawing) | `widgets.canvas({ name, draw })` | renders in canvas _and_ Nodes 2.0 | +| `onDrawForeground` (sizing) | `node.setSizeConstraints({ autoHeight })` | see `references/draw-callbacks.md` | +| `onDrawForeground` (polling) | `widget.on('change', (value) => { ... })` | pick the narrowest event | +| `extends LGraphNode` | `defs.define({ type, ... })` | virtual nodes use `resolve(ctx)` | +| `onConnectInput` returning `false` | `b.onBeforeConnect((node, e) => false)` | any listener refusing is enough | +| `getExtraMenuOptions` (node menu) | `b.addMenuItem({ label, run })` | entries accumulate; canvas/slot menus are still a **gap** | + +Prefer `comfy.supports('...')` over version comparisons — see +`docs/node-api/registration.md`. + +### 4. Watch for the frame-to-event shift + +A draw callback recomputed from current state every repaint, so nothing ever +needed to announce a change. Canvas surfaces must be **redrawn when the value +changes**. A port that calls `redraw()` from inside the old callback body will +appear to work and quietly run on every repaint forever. + +### 5. Write the test before claiming success + +State an equivalence claim and a fix claim. Run the file against both sources: +equivalence green on both, fix red on the original and green on the converted. +A fix-only test proves the conversion did _something_, not that it was safe. + +### 6. Refuse when you should + +Escalate or decline rather than guess when: + +- You cannot tell whether the object is live or serialized (step 2). +- The replacement depends on intent you cannot recover — e.g. re-homing links + versus rebuilding a mirror have different correct answers. +- The published API has no destination. A missing API is a **core gap to file**, + not something to work around in a pack. `docs/node-api/README.md` defines the + contract boundary. + +A refusal costs a round trip. A wrong rewrite of working code is invisible until +a user hits it. + +A refusal must answer **why**. Keep one adjacent comment block that records: + +1. the user-visible behavior; +2. the exact old mechanism; +3. the ownership, determinism, wire-format, renderer, scope or lifecycle + guarantee that makes the mechanism unacceptable; +4. the supported remainder and exact user loss (`INOPERABLE: nothing` when + there is none); and +5. the concrete API capability or policy change that would reverse the + decision, or why the behavior must remain host-owned. + +“Not supported”, “no API”, “renderer internals are unavailable”, and a list of +removed property names are not reasons. They describe absence. If the behavior +is acceptable and only a public destination is absent, that is `API-GAP`, not +`REFUSED`. If the behavior survives through another published mechanism, name +that destination and refuse only the old technique. + +**A partial conversion is worse than a punt.** Two checks enforce this and both +fail the whole file: + +- `retires-the-old-surface` — no `X.prototype.foo = ...` may remain. Converting + a function body while leaving the prototype assignment that reaches it moves + nothing; the old surface still cannot be deleted, which is the only reason + this programme exists. +- `no-unknown-api-members` — every member you introduce must exist in the + published API. Do not invent a plausible-sounding method. If the capability + you need is absent, that is `api-gap`, and naming it precisely is the most + useful thing you can do. + +Both of these caught real conversions that every other check passed. + +## Keep the diff small — it is the message + +These diffs are read by the pack's author, often as a pull request. A patch that +touches ten lines says _we moved you off two deprecated calls_. A patch that +rewrites the file says _we rewrote your code_, and it will be rejected on sight +even when it is correct. Restructuring you did not need is not neutral. + +**Change the registration. Leave everything else where it is.** + +```js +// before +app.registerExtension({ + name: 'x.ShowText', + async beforeRegisterNodeDef(nodeType, nodeData, app) { + if (nodeData.name === 'ShowText') { + function populate(text) { + /* 30 lines */ + } + nodeType.prototype.onExecuted = function (message) { + populate.call(this, message.text) + } + } + } +}) +``` + +Two conversions of that, both correct: + +```js +// ✗ restructured — populate hoisted, dedented, renamed, signature changed. +// Every line of a 30-line helper shows as changed. +function populateText(node, lines) { + /* 30 lines, reflowed */ +} +comfy.defs.extend('ShowText', (b) => b.onExecuted(populateText)) + +// ✓ minimal — the helper stays exactly where and as it was. +comfy.defs.extend('ShowText', (b) => { + function populate(text) { + /* 30 lines, byte-identical */ + } + b.onExecuted((node, result) => populate.call(node, result.text)) +}) +``` + +Both work. The second is the one an author merges. + +Rules that follow from this: + +- **Do not hoist, reorder or rename anything** the conversion does not require. + Keep helper names, parameter names and their order. +- **Keep helpers nested where they were nested.** Pulling one to module scope + changes every line of it. +- **Do not normalise style.** Quotes, semicolons, spacing and line breaks are + the author's, not yours. +- **Do not "improve" adjacent code.** A bug next to your change is not your + change. + +### Indentation: match the original where you can, fix it where you must + +The patch gets applied to the author's working tree, so a diff that minimises +its own size by leaving the body at its old depth ships them badly indented +code. That is a worse outcome than a larger diff. + +The rule, in order: + +1. **Prefer the original indentation.** If a line's nesting depth has not + actually changed, do not touch its leading whitespace. +2. **Re-indent when the nesting genuinely changed.** Removing a + `registerExtension` wrapper takes two levels off everything inside it, and + the result has to be correctly indented. Do it. +3. **Never re-indent anything whose nesting did not change.** That is the + churn worth eliminating, and it is entirely elective. + +So the thing to avoid is not re-indentation, it is _gratuitous restructuring_ — +hoisting a helper to module scope, renaming its parameters, reflowing a function +the conversion never touched. Those change every line of code that did not need +to change. Correct indentation is not negotiable; unnecessary movement is. + +Across the current database the unavoidable dedent accounts for 17–39% of added +lines. `run_checks` reports the proportion so you can see whether yours is in +that range or well past it. + +## Recognise the intent, not the call + +Most old-API code is a workaround for something missing. Port the call and you +carry the workaround across; recognise what it was _for_ and it usually +collapses. Check these before converting anything mechanically. + +**If you are converting `api.addEventListener('b_preview')` or +`'b_preview_with_metadata'` or `'executing'`** — check whether the pack is +correlating frames to one node (a module-level `execId`, a +`displayNodeId === this.id` test, a `serverSupportsFeature` probe). That whole +apparatus answers "is this frame mine?". **Use `b.onPreview((node, frame) => +…)`**, which answers it for you. Delete the global; it also mis-attributes +frames when two nodes preview at once. + +**If you are converting `onDrawForeground` / `onDrawBackground` / anything +using `ctx`** — check what it actually draws. Rectangles, images, text and +lines are all a canvas can give you and all a DOM can. **Use +`node.widgets.canvas({ name, height, draw(ctx, [w, h]) })`** and keep the +drawing code as it is. It renders under both the old graph renderer and Nodes +2.0. Do not reach for the graph's shared context — that is the thing that ties +a pack to the old renderer. + +**If you are converting hand-rolled hit testing** — bounding-box maths against +`node.pos`/`node.size`, pointer capture on `document`, hit tests against link +curves. Check whether it exists only because canvas has nothing to attach a +listener to. **Mount the control with `node.widgets.mount(...)` and use +ordinary DOM events**; most of the geometry disappears rather than being +ported. + +**If you are converting `node.addDOMWidget(...)`** — **use +`node.widgets.mount({ name, render(container), destroy() })`**. Put the teardown +in `destroy`: a mounted element owns listeners, timers and observers that node +removal would otherwise leave running. + +If that DOM widget's `getValue` / `setValue` held an editor document, give the +mount a `defaultValue` and synchronize through the `MountedValue` passed to +`render`. Set `serialize: true` and, for frontend-only state, `sendToPrompt: +false`; use the widget's `beforeSerialize` event when the live editor must be +sampled immediately before writing. Once the document is a real widget value, +normal workflow serialization also carries it through paste and duplicate — a +`clone()` override and side cache are not another requirement. + +Count those cells per widget before consolidating an interface. Two legacy DOM +rows with workflow serialization enabled produce two positional +`widgets_values` entries even when both values are empty and neither reaches +the prompt. Replacing them with one larger mount changes the wire format; keep +two mounts, or prove from the widget flags that one row was never serialized. + +If the old editor patched `ChangeTracker.undoRedo` to keep Ctrl+Z local, do not +port the patch. Focused `input`, `textarea`, and `contenteditable` elements own +their undo history, including while auto-queue-on-change is watching edits. +Mount the editor and keep its own undo/redo handlers on that element. + +**If you are converting `node.addInput` / `removeInput` / `addOutput`** — the +pack is almost certainly growing slots as the last one fills (the "Multi" +combiner shape). **Use `node.inputs.add(name, type)` / `node.inputs.remove(ref)`** +and the matching `node.outputs` calls, driven from `b.onConnectionsChanged`. + +**If you are converting `canvas.selected_nodes` / `selectedItems`** — the pack +wants the user's current selection, and is reaching into the canvas for it. +**Use `comfy.graph.selection()`**, which returns node handles. + +**If you are writing to a node or widget handle** — every read and write is a +**method**, never a property: `setTitle`, `setColor`, `setBgColor`, `setMode`, +`setCollapsed`, `setProperty`, `getSize`/`setSize({ width, height })` on nodes; +`getValue`/`setValue`, `isHidden`/`setHidden`, `setOption` on widgets. Property +syntax compiles and silently does nothing. See the table in +`references/node-definitions.md`. + +**If the pack registered a custom renderer for a backend-declared widget +type** — **use `comfy.defs.defineWidgetType(type, { render })`**, not an extra +mounted widget. The renderer receives the declared widget's value, so it keeps +the same positional `widgets_values` cell. Use `context.onNodeReady` when its +DOM needs the owning node; constructors run before a node has an id or graph. + +**If you are converting `registerCustomNodes` / `extends LGraphNode` / +`isVirtualNode` / `applyToGraph`** — identify which of four intents the node +serves (annotation, wire, value, or acting on other nodes) and use **`comfy.defs.define`** +with `execution: 'frontend'` and, for wires and values, a pure `resolve`. See +`references/node-definitions.md` — nodes that act on others mutate neighbours through +handles in widget callbacks, never in `resolve`. + +**If the pack deletes and recreates a node to repair it after definitions +change** — **use `comfy.graph.replace(node.id, node.type)`**. Same-type +replacement intentionally creates a fresh registered instance while retaining +the user's values, properties and compatible links in one undo step. + +**If you are converting `onResize` / `computeSize` / a per-frame `setSize`** — +check whether the pack is enforcing a minimum, or growing to fit something it +mounted. **Use `node.setSizeConstraints({ minWidth, minHeight, maxWidth, +maxHeight, autoHeight })`** once, rather than re-asserting size on every frame. +`autoHeight` is usually the real intent. + +**If you are converting `onSerialize` / `chainCallback(node, 'onSerialize')`** — +the pack is saving its own state into the node. **Use `b.onSerialize((node) => +({ myKey: … }))`**; it comes back through `b.onConfigured`. Core fields — +`type`, `widgets_values`, `inputs`, `pos` and the rest — are ignored if you +return them, because changing those changes what the workflow means. + +**If you are converting `onConnectInput` / `onConnectOutput`** — check what the +pack does with the return value. If it returns `false` to refuse a wire, **use +`b.onBeforeConnect((node, e) => …)`** and return `false` to refuse; `e` carries +`side`, `index`, `peerNodeId` and `peerType`. Any listener refusing is enough — +one pack cannot be overruled by another's silence. If it does not return +`false`, it is only observing, so **use `b.onConnectionsChanged`** instead: a +veto that never vetoes is a listener wearing the wrong hat. + +**If you are converting `getExtraMenuOptions` or a `ContextMenu` the pack builds +itself** — check whether it is a menu _on a node_. If so, **use +`b.addMenuItem({ label, run(node) })`**; entries from every pack accumulate +rather than overwrite. A `ContextMenu` constructed to build a canvas-wide or +slot menu is not this, and is still a gap — punt it and name it. + +**If you are converting a pack that writes its own name-keyed shape into +`widgets_values`** (a dict keyed by widget name, rather than the positional +array) — the pack is reaching for something the new model already gives it. +**Widget values are keyed by name at runtime**: `widgetValueStore` is keyed by +`graphId:nodeId:name` (`src/types/widgetId.ts`), with no index in the identity. +The positional array is only the legacy _serialized_ form, which is why +`widgets_values` is reserved. + +Replacing the dict-shaped `widgets_values` with core's positional array changes +the saved format. It cannot meet the byte-identical conversion requirement. +Escalate this case for an explicit format-migration decision before converting it. + +If that migration is approved, keep the reading of old workflows. Core assigns positionally +and _then_ calls `onConfigure`, so `b.onConfigured` receives the saved node with +its legacy shape intact. Port the pack's rename maps, retired-widget handling +and positional-to-named fallbacks into that hook: those are the conversion, and +dropping them silently loses user data. + +**If you are converting reads of `canvas.connecting_links`, `resizing_node` or +`node_widget`** — the pack is asking one question, not three: _is the editor +already mid-gesture?_ **Use `comfy.isInteracting()`** and stand down while it is +true. Do not reach for the individual fields; which gestures exist is the +editor's business and will change. + +**If you are converting a pack that listens on `document` for pointer moves to +build an editing gesture** (drag one node onto another, shake to disconnect, +drop onto a link) — **use `comfy.onNodeMoved`**, which reports node movement +under both renderers from one subscription. Two cautions: it does not say +whether a _person_ moved the node, so guard your own writes against re-entry; +and for a gesture that commits on release, pair it with +`comfy.onNodeDragEnd(nodes => …)`, which reports every node the drag moved. + +`onNodeDragEnd` is **Nodes 2.0 only** — the legacy canvas renderer publishes no +drag lifecycle, so it never fires there. Say so in your summary rather than +pretending the conversion is renderer-neutral. + +**If you are converting `canvas.setDirty(...)` / `setDirtyCanvas(...)`** — +**delete it.** There is no published repaint request, deliberately. Handle +writes invalidate on their own, and a `widgets.canvas` surface has `redraw()`. +If something genuinely fails to repaint after a handle write, that is a bug in +the API — report it rather than working around it. + +**If you are converting `addWidget('button', name, null, callback)` or any +widget whose callback is an action rather than a value change** — **use +`widget.on('activate', fn)`**. A button's value never moves, so `on('change')` +can never fire for one. Keep the widget itself (`widgets.add({ type: 'button', +name, value: null })`) — its `widgets_values` entry is positional, and dropping +it shifts every widget after it. + +**If you are converting `registerExtension({ commands, keybindings })` or +`app.extensionManager.toast`** — **use `comfy.commands`**: +`register({ id, label, run, keybinding })` declares the command and binds its +key together, and `notify({ severity, summary, detail })` raises a toast. Ids +must be namespaced. The key is registered as a _default_, so a user's own +binding survives the pack re-registering on every load. + +**If you are converting `api.apiURL(...)` or `api.addEventListener(...)`** — +**use `comfy.backend`**: `url('/view?…')` builds a route that honours how the +host is served, and `on(event, detail => …)` subscribes to any backend message, +including one your own Python side emits. For the built-in preview channel +prefer `b.onPreview`, which answers "is this frame for my node?" — `backend.on` +hands you the raw payload and leaves correlation to you. + +**If the pack adds its own file input beside a backend COMBO** — inspect the +Python declaration before porting it. `image_upload`, `animated_image_upload` +and `video_upload` in that input's options tell the host to supply the chooser, +upload and preview. Delete the duplicate UI; adding another widget changes the +positional wire format while recreating behavior core already owns. + +If a pack rewrites `graphToPrompt` only to name a cache or sidecar file, inspect +its Python before declaring the feature impossible. A pack-owned preview route +may already serve file inputs without a graph run; connected tensors can use +`queue.run({ nodes: [node] })`, then refresh the sidecar from `b.onExecuted` and +`execution_cached`. A hidden id declared as `UNIQUE_ID` is supplied by the host. +A hidden string default such as `"0"` is instead a pack-backend identity bug: +record the simultaneous-node limitation precisely, but do not mislabel the +published queue, execution and backend mechanisms as missing. + +**If you are converting `app.ui.settings.getSettingValue` / `addSetting`, or a +`settings: [...]` array in `registerExtension`** — **use `comfy.settings`**: +`declare({ id, name, type, defaultValue })` once at load, then `get(id)` and +`await set(id, value)`. Ids must be namespaced (`MyPack.thing`) — one flat space +is shared with core and every other pack, and the id is where the value lives +permanently. Re-declaring does not reset a stored value, so declaring on every +load is correct. + +**If the pack replaces the canvas background draw method only to show an +image** — **write `Comfy.Canvas.BackgroundImage` through `comfy.settings`**. +The host loads and redraws it under both renderers. Preserve and restore the +previous setting when the pack's temporary background mode stops. + +**If you are converting a read of `LiteGraph.NODE_SLOT_HEIGHT`, +`NODE_TITLE_HEIGHT`, `ROUND_RADIUS` or `vueNodesMode`** — **use +the published answer to the operation, not another numeric renderer constant**. +`getBounds`, `getSlotPosition`, `getScreenRect`, `widgets.mount` and +`widgets.canvas` already account for layout and renderer choice. There is no +published `comfy.constants`; if behavior truly requires reproducing the +renderer geometry after those mechanisms are considered, name that specific +gap rather than inventing one. + +**If you are converting `this._somethingPrivate = x` on a node** — handles hold +no arbitrary properties. **Keep a `Map` keyed by +`${node.graphId}:${node.id}`** and clear the entry in `b.onRemoved`. This is +supported, not a workaround; the old property was collected with the node and +a Map is not. + +**If you are converting `node.imgs = [img]` + `setDirtyCanvas`** — the pack is +showing an image on the node. **Use `widgets.canvas` and `drawImage` in +`draw`**, then `redraw()` when the image changes. + +**If you are converting a captured-and-chained `widget.callback`**, check +whether the listener only observes value changes. **Use `widget.on('change', (v, +old) => …)`**, which is additive: no other pack can drop your listener by +forgetting to call through. + +**If you are converting `widget.serializeValue`** — classify its return. A +constant `undefined` keeps the value out of the prompt while still saving it: +that is `options: { serialize: false }` on a widget the pack adds, or +`widget.setOption('serialize', false)` on an existing one. It is not +`serialize: false`, which stops the value being saved and still sends it. A +synchronous substitute uses +`widget.on('beforeSerialize', event => event.setSerializedValue(value))`. If it +awaits derived state, an async `comfy.queue.guard` may commit that state before +the prompt is built, but guards time out after five seconds. Anything that can +legitimately take longer remains an API gap rather than a safe conversion. +Note the two flags are distinct: `options.serialize` gates the API prompt, +`widget.serialize` gates workflow persistence. + +**If you are converting `widget.type = 'converted-widget'`** — the pack is +hiding a widget, not changing its kind. **Use `widget.setHidden(true)`.** The +`origType`/`origComputeSize`/`origSerializeValue` bookkeeping around it existed +only to undo the hack; it has no readers and goes away. + +## Pattern references + +Deep dives, loaded only when relevant. `SKILL.md` stays short on purpose; detail +lives here. + +| Reference | Covers | +| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `references/nodegraph-101.md` | **Read first if you have not converted a pack before.** What a node graph is, the definition/class/instance distinction, the lifecycle, workflow vs prompt, and why packs patch prototypes at all. | +| `references/node-definitions.md` | `beforeRegisterNodeDef` + prototype patching — **1,265 packs, 47.4% of installs, the largest surface**. The selector is already written as the hook's guard clause. | +| `references/widgets.md` | Widget-array mutation and the converted-widget protocol — 286 packs / 21.6%, overlapping cohorts totalling more. Where naive conversions are most often silently wrong. | +| `references/draw-callbacks.md` | `onDraw*` — 420 packs, 32.2% of installs. Measured breakdown showing 47% never draw at all, a decision tree, and the canvas→CSS mapping. | + +### Adding a capability + +If a conversion is blocked and the fix is a **new API capability**, the addition +is a design decision, not a detail. Record the tradeoff and rejected alternatives +in an ADR under `docs/adr/`, using the repository's topic/number/slug naming +convention, then update `docs/node-api/`. An addition made under conversion +pressure with its rationale only in a commit +message is one nobody can argue with later. + +### Adding a pattern + +Add a reference when a pattern is (a) seen in real pack code, not anticipated, +and (b) large or subtle enough that the mapping table row is insufficient. + +Each reference should carry: + +1. **How common it is**, measured — not estimated. Counts come from the current + registry corpus, should record the dataset pin, and should be labelled + grep-derived where they are. +2. **What the code is actually doing**, which is often not what the API name + suggests. Classify before mapping. +3. **Real before/after**, quoted from a named pack and file. +4. **The trap** — the way a plausible conversion silently goes wrong. + +If a pattern turns out to be mechanically safe, add it to Magic Patch's rule +catalog instead: a rule with golden cases beats prose, because CI runs it. +Prose is for the judgement calls. + +## Where things live + +| | | +| ------------------------- | -------------------------------------- | +| Published API guides | `docs/node-api/` | +| Machine-readable contract | `docs/node-api/comfy-api.d.ts` | +| Published API code | `src/platform/nodeApi/` | +| Declaration generator | `scripts/node-api/gen_api_dts.mjs` | +| Capability-list generator | `scripts/node-api/gen_api_surface.mjs` | + +When Magic Patch tooling is present, keep mechanically safe rewrites and their +golden cases in its rule catalog. This skill is for the intent and judgement +that cannot be encoded as a mechanical rule. + +## Checklist + +- [ ] Traced every `link`/`links`/`widgets` reference to live-vs-serialized +- [ ] No shim, no compatibility wrapper, no reintroduced old API +- [ ] Link ids preserved where links were moved +- [ ] Equivalence claim stated and passing on both sources +- [ ] Fix claim red on the original, green on the converted source +- [ ] `graphToPrompt` and serialized workflow byte-identical +- [ ] Anything uncertain escalated rather than guessed diff --git a/.claude/skills/converting-custom-nodes/references/draw-callbacks.md b/.claude/skills/converting-custom-nodes/references/draw-callbacks.md new file mode 100644 index 00000000000..19a5d467f15 --- /dev/null +++ b/.claude/skills/converting-custom-nodes/references/draw-callbacks.md @@ -0,0 +1,223 @@ +# Converting draw callbacks + +`onDrawForeground`, `onDrawBackground`, `onDrawTitle*`, `onDrawCollapsed`. + +**420 packs / 34.3M downloads / 32.2% of registry installs** touch these. They +are canvas-only, so they silently do nothing in Nodes 2.0 — no warning, no +error, no visible failure. That silence is why this cohort needs care. + +## The hook name is misleading — most of it is not drawing + +Measured over all **2,787 draw-callback bodies** in the corpus: + +| What the body actually does | Bodies | % | Packs | +| ---------------------------------------- | ------ | ----- | ----- | +| Draws something | 1,468 | 52.7% | 289 | +| Prototype-patch plumbing | 901 | 32.3% | 198 | +| Layout / size enforcement — _no drawing_ | 178 | 6.4% | 39 | +| State sync / polling — _no drawing_ | 169 | 6.1% | 46 | +| DOM element sync — _no drawing_ | 71 | 2.5% | 37 | + +**47.3% never touch a drawing primitive. 127 packs hook a draw callback and +never draw at all.** They are using it as the only recurring callback available. + +## Decide which of four things it is — before writing anything + +``` +Does the body call ctx.fillText/fillRect/drawImage/arc/... ? +├─ no +│ ├─ only calls the original (`orig?.apply(this, arguments)`) → DELETE (plumbing) +│ ├─ assigns this.size / ensureMinimumSize / computeSize → setSizeConstraints +│ ├─ compares state and rebuilds on difference → widget.on('change') / onConnectionsChanged +│ └─ toggles .hidden / .style / wrapperEl → widget.setHidden() / mount's own element +└─ yes + ├─ small status label or badge → node.addBadge() + ├─ custom text, bar, border, tint, image, or icon → widgets.canvas() + ├─ interactive (hit-testing, dragging, mouse handling) → widgets.mount() + own DOM/canvas + └─ arbitrary composition → widgets.mount() +``` + +## 1. Prototype plumbing — 32.3%, just delete it + +```js +// before +const onDrawForeground = nodeType.prototype.onDrawForeground +nodeType.prototype.onDrawForeground = function (ctx) { + const r = onDrawForeground?.apply?.(this, arguments) + /* ...actual work... */ + return r +} +``` + +The capture-and-chain wrapper exists only because prototype patching has no +composition. Registered callbacks compose by construction, so the wrapper +disappears entirely — keep only the body. + +This is the highest-volume, lowest-risk transform in the whole programme. + +## 2. Size enforcement — 39 packs + +```js +// before — re-asserted on every repaint +if (Number.isFinite(desired) && Math.abs(this.size?.[1] - desired) > 1) { + this.size[1] = desired +} + +// after — declared once +node.setSizeConstraints({ minHeight: desired }) +``` + +`autoHeight: true` is usually what the pack actually wants: it added a DOM +widget of unknown height and hand-computed the node size to fit. In a +DOM-rendered node that is just layout, and needs no pack code at all. + +## 3. Polling — 46 packs + +```js +// before — string-joins every group title on every repaint, and mouse +// movement marks the canvas dirty, so this ran constantly +const titles = (app.graph._groups?.map((g) => g.title) || []).join() +if (this.lastKnownGroupTitles !== titles) { + this.lastKnownGroupTitles = titles + rebuildUI(this) +} + +// after — for the cases the API reaches +node.widgets.get('mode').on('change', () => rebuildUI(node)) +``` + +Pick the **narrowest** event that exists. `widget.on('change')` covers polling +for a widget's own value, and `b.onConnectionsChanged` covers polling for +wiring. + +**Polling for graph structure — group titles, node counts, anything outside the +node — is a gap.** There is no `graph.onChange`; do not emit one. Punt the file +and name the event you needed. + +## 4. DOM visibility sync — 37 packs + +```js +// before +node.painter.canvas.wrapperEl.hidden = this.flags.collapsed +``` + +A mounted DOM widget's lifecycle handles this. If you find yourself syncing an +element's visibility to node state, the element should be a widget. + +## 5. Actual decoration — the informational half + +For a small status label that belongs in node chrome, use `node.addBadge()`: + +```js +const removeBadge = node.addBadge(() => ({ + text: currentStatus(node) +})) +``` + +Keep the returned removal function when the badge has a shorter lifetime than +the node. The callback runs during drawing, so it must remain fast. + +Every canvas primitive in use has an exact DOM equivalent, so this is a +translation rather than a redesign: + +| Canvas (packs using) | DOM/CSS | +| --------------------------------------------------- | --------------------------------------- | +| `fillText` 326, `measureText` 206 | a text node — measurement becomes free | +| `fillRect` 234 / `roundRect` 193 / `strokeRect` 159 | `background`, `border-radius`, `border` | +| `drawImage` 203 | `` | +| `arc` 181 / `ellipse` 58 | `border-radius: 50%` | +| `translate` 170 / `rotate` 136 | `transform` | +| `globalAlpha` 150 | `opacity` | +| `clip` 138 | `overflow: hidden` | +| `setLineDash` 115 | `border-style: dashed` | +| `shadowBlur` 95 | `box-shadow` | +| `createLinearGradient` 79 | `linear-gradient()` | + +For custom drawing, the shipped destination is **`node.widgets.canvas()`** — a +per-node drawing surface that works under both renderers, because the canvas is +a DOM element the legacy renderer positions over the graph and Nodes 2.0 +renders natively: + +```js +// before — ComfyUI-Custom-Scripts mathExpression.js, ran every repaint +nodeType.prototype.onDrawForeground = function (ctx) { + const v = app.nodeOutputs?.[this.id] + if (!this.flags.collapsed && v) { + ctx.save() + ctx.font = 'bold 12px sans-serif' + ctx.fillText(v.value[0], x, y) + ctx.restore() + } +} + +// after — same drawing code, but event-driven +b.onCreated((node) => { + const surface = node.widgets.canvas({ + name: 'result', + height: 22, + draw(ctx) { + ctx.font = 'bold 12px sans-serif' + ctx.fillText(stateFor(node).value ?? '', 4, 15) + } + }) + stateFor(node).surface = surface +}) +b.onExecuted((node, result) => { + const values = result.raw.value + stateFor(node).value = Array.isArray(values) ? String(values[0] ?? '') : '' + stateFor(node).surface?.redraw() +}) +``` + +`draw` runs on mount, on resize, and on `redraw()` — never per frame. Keep the +pack's `ctx` code as close to verbatim as you can; the conversion is _when_ it +runs, not _what_ it draws. The collapsed check disappears (a hidden widget is +not drawn), and pixel positions are relative to the surface, not the node. + +`node.addBadge()` is the declarative destination for simple labels; +`widgets.canvas()` remains the destination for arbitrary pack drawing. + +## 6. Interactive controls — becomes a widget + +```js +// mxtoolkit Slider2D.js — a full draggable 2D slider painted by hand +this.node.onDrawForeground = function (ctx) { + ctx.fillStyle = 'rgba(20,20,20,0.8)' + ctx.beginPath() + ctx.roundRect(shiftLeft - 4, shiftLeft - 4, ...) + ctx.fill() + // dots, handles, hit-testing... +} +``` + +This is not decoration. It was painted by hand because no custom-widget API +existed. It becomes **`node.widgets.mount({ name, render, destroy })`** — the +pack appends its own `` (or any DOM) to the container and keeps its +drawing code, but pointer events now land on a real element, so hand-rolled +hit-testing against bounding boxes mostly disappears. kjnodes' `editor_base.js` +already works exactly this way and needs only the mount call swapped. + +## The trap: frame to event + +A draw callback recomputes from current state on every repaint, so **nothing +ever needs to announce a change**. After conversion, the drawing must be +refreshed _when the value changes_. + +A naive port that calls `redraw()` from inside a per-frame path will appear to +work — and quietly run on every repaint forever. Redraw from the event that +changes the data: `onExecuted`, `widget.on('change')`, `onConnectionsChanged`. + +## What you can stop doing + +- **LOD checks.** rgthree hand-rolls `canvas.ds.scale < 0.6` to skip drawing + when zoomed out. Handled centrally now. +- **Collapsed checks.** `if (this.flags.collapsed) return` — layout's problem. +- **Defensive try/catch around drawing.** Several packs wrap draw calls to avoid + breaking node rendering; a `canvas()` surface draws into its own element, so + a throw cannot take node rendering down with it. + +## Source data + +Measured over the 4,969-pack registry corpus used when this reference was +authored. Counts are grep-derived and have a known false-positive rate — +sample-verify before citing any individual number. diff --git a/.claude/skills/converting-custom-nodes/references/node-definitions.md b/.claude/skills/converting-custom-nodes/references/node-definitions.md new file mode 100644 index 00000000000..6c8cd6d086e --- /dev/null +++ b/.claude/skills/converting-custom-nodes/references/node-definitions.md @@ -0,0 +1,335 @@ +# Converting `beforeRegisterNodeDef` and prototype patching + +**1,265 packs / 50.4M downloads / 47.4% of registry installs** register a +`beforeRegisterNodeDef` hook, and **1,191 of them use it to patch the generated +class's prototype**. This is the largest surface in the ecosystem — bigger than +the widget and painting cohorts combined — and the one that couples packs to our +internals most tightly. + +## What they patch — measured + +Prototype assignments across those 1,265 packs, litegraph methods only +(bundled-library noise like `_next`, `dispose`, `toString` filtered out): + +| Patched method | Sites | Packs | Destination | +| --------------------- | ----- | ------- | ------------------------------ | +| `onNodeCreated` | 3,161 | **943** | `b.onCreated(cb)` | +| `onExecuted` | 964 | **497** | `b.onExecuted(cb)` | +| `onConfigure` | 1,272 | **429** | `b.onConfigured(cb)` | +| `onConnectionsChange` | 454 | 223 | `b.onConnectionsChanged(cb)` | +| `onDrawForeground` | 446 | 199 | `references/draw-callbacks.md` | +| `onRemoved` | 353 | 158 | `b.onRemoved(cb)` | +| `constructor` | 473 | 49 | `b.onCreated(cb)` | +| `getDefaultShape` | 335 | 3 | no replacement — escalate | + +## The selector is already written — it's the guard clause + +Every one of these hooks runs for **every registered node type**, so essentially +all of them open with a filter and return. That filter _is_ the selector: + +```js +// before — ComfyUI-KJNodes jsnodes.js:37 +async beforeRegisterNodeDef(nodeType, nodeData, app) { + if (!nodeData?.category?.startsWith('KJNodes')) return + switch (nodeData.name) { + case 'ImageBatchMulti': + case 'ImageAddMulti': + nodeType.prototype.onNodeCreated = function () { + setupDynamicInputs(this, { type: 'IMAGE', prefix: 'image_' }) + } + break + ... + } +} + +// after +comfy.defs.extend(['ImageBatchMulti', 'ImageAddMulti'], (b) => { + b.onCreated((node) => setupDynamicInputs(node, { type: 'IMAGE', prefix: 'image_' })) +}) +``` + +This is mechanical: lift the `if`/`switch` condition into the selector, one +`extend` call per case group. + +It is also a real performance fix, not just tidiness. With 1,265 packs hooking +and a few thousand node types, boot currently runs **millions of callbacks that +immediately return**. A declarative predicate can be indexed. + +Selector forms: exact type, array of types, `RegExp` over the type, or +`{ category }` where category is a string or a `RegExp`. The regex form covers +the prefix filter 53 packs open with — +`nodeData.category.startsWith('KJNodes')` becomes `{ category: /^KJNodes/ }`. + +`onCreated` fires when the node **joins a graph**, not inside the constructor. +Litegraph's `onNodeCreated` runs before the node has an id, a graph, or store +registration, so widget writes made there are lost on insert. If a pack relied +on running before insertion, escalate — that ordering is not reproducible. + +## Chaining boilerplate disappears + +```js +// before — capture-and-chain, because prototype patching has no composition +const onExecuted = nodeType.prototype.onExecuted +nodeType.prototype.onExecuted = function (message) { + onExecuted?.apply(this, arguments) + populate.call(this, message.text) +} + +// after — registered callbacks compose by construction +b.onExecuted((node, result) => populate(node, result.text)) +``` + +Whether the old form worked at all depended on load order and on every pack +remembering to call through. Two packs patching the same method with one +forgetting silently broke the other. + +## `onExecuted` — the second-largest, and easy to get wrong + +497 packs. The result shape is now explicit rather than a raw backend payload: + +```js +// before +nodeType.prototype.onExecuted = function (message) { + populate.call(this, message.text) +} + +// after +b.onExecuted((node, result) => populate(node, result.text)) +``` + +`ExecutionResult` exposes `images`, `text`, and `raw` for everything else. +Custom output keys survive in `raw` — ADR-NODE-OUTPUTS-0007's passthrough +schema guarantees it — so a pack reading a bespoke key keeps working. + +## A worked example touching four surfaces at once + +ComfyUI-Custom-Scripts `showText.js:10` is representative of the harder cases: + +```js +// before +async beforeRegisterNodeDef(nodeType, nodeData, app) { + if (nodeData.name === 'ShowText|pysssss') { + function populate(text) { + if (this.widgets) { + // On older frontend versions there is a hidden converted-widget + const isConvertedWidget = +!!this.inputs?.[0].widget + for (let i = isConvertedWidget; i < this.widgets.length; i++) { + this.widgets[i].onRemove?.() + } + this.widgets.length = isConvertedWidget + } + for (const l of text) { + const w = ComfyWidgets.STRING(this, 'text_' + this.widgets?.length, ...).widget + w.inputEl.readOnly = true + w.inputEl.style.opacity = 0.6 + } + } + ... + } +} +``` + +Four separate conversions: + +| Old | New | +| ------------------------------------------------------ | --------------------------- | +| `nodeData.name === 'ShowText\|pysssss'` guard | the `defs.extend` selector | +| `+!!this.inputs?.[0].widget` converted-widget sniffing | `input.isWidgetInput` | +| `this.widgets.length = n` truncation | remove by name (below) | +| `w.inputEl` DOM access | `widgets.mount({ render })` | + +Truncation has no single-call replacement, deliberately — assigning `length` +skips each widget's teardown, which is why the pack has to call `onRemove()` by +hand first: + +```js +// after — removal runs teardown for you +for (const name of node.widgets.names().slice(keep)) { + node.widgets.remove(name) +} +``` + +Note `isConvertedWidget` exists only to skip a widget the _old frontend_ hid via +the converted-widget protocol. With `setHidden()` as a real property, that +whole line of reasoning goes away — check `input.isWidgetInput` if you actually +care whether an input is a widget's socket form. + +## Pack-owned node types — `registerCustomNodes` / `extends LGraphNode` + +86 packs / 18.2% of installs define their own types by subclassing. The +replacement is `comfy.defs.define(...)`: the definition is plain data, and no +class is ever yours. + +**First identify the intent.** "Virtual node" is four different things, and the +conversion differs for each: + +| The node exists to… | Examples | Convert to | +| --------------------------------------- | ---------------------------- | ----------------------------------------------------------- | +| annotate — never executes | Note nodes | `execution: 'frontend'`, no `resolve` | +| be a wire — indirection | Reroute, `SetNode`/`GetNode` | `resolve` returning `forwardTo` | +| be a value — UI-held literal | Primitive, constants | `resolve` returning `literal` | +| act on _other_ nodes — a remote control | rgthree Fast Muter | **not `resolve` at all** — handle calls in widget callbacks | + +```js +// before +class GetNode extends LGraphNode { + constructor() { + super() + this.addOutput('value', '*') + this.isVirtualNode = true + } + applyToGraph() { + /* rewrites links on the LIVE graph mid-serialize */ + } +} +LiteGraph.registerNodeType('GetNode', GetNode) + +// after — declared, and resolution is a pure answer over a read-only view +comfy.defs.define({ + type: 'GetNode', + execution: 'frontend', + outputs: [{ name: 'value', type: '*' }], + widgets: [{ type: 'string', name: 'key', value: '' }], + resolve: ({ self, nodesOfType }) => { + const setter = nodesOfType('SetNode').find( + (n) => n.widgetValue('key') === self.widgetValue('key') + ) + return { + value: setter ? { forwardTo: setter.input('value') } : { omit: true } + } + } +}) +``` + +`resolve` answers, per output: `{ forwardTo: inputRef }` ("whatever feeds that +input"), `{ literal: value }`, or `{ omit: true }`. Our pass follows chains +(Get → Set → Reroute → …) with cycle detection. You never see or touch the +prompt being built — a resolver that throws poisons one prompt build and the +graph is untouched, which is exactly what `applyToGraph` could not guarantee. +A simple reroute is one line: `resolve: ({ self }) => ({ out: { forwardTo: +self.input('in') } })`. + +**Nodes that act on other nodes are ordinary nodes.** A Fast Muter is a remote control: a row of toggles, one per wired-in node. A Fast Muter is `defs.define` with +`execution: 'frontend'`, buttons via `widgets`, and callbacks that call +`comfy.graph.nodesOfType(...)` + `node.setMode('bypass')` — edit-time changes +the user can undo, not serialization behaviour. Do not put neighbour mutation +in `resolve`; `resolve` cannot write anything, by design. + +**Do not carry over** `isVirtualNode`, `applyToGraph`, or the subclass itself. +If the node's `applyToGraph` does something none of the three resolution shapes +can express, that is an `api-gap` punt — name what it rewrites. + +### Resolution runs once per graph, and does not cross a subgraph + +Both passes — `resolve` and `setSupply` — are invoked separately for every +graph the prompt draws from: the root, and each subgraph interior. So a +resolver or supplier inside a subgraph works exactly as it does at the top +level, and you do not need to do anything to opt in. + +What you cannot do is reach across the boundary. `nodesOfType` and +`unconnectedInputs()` see the scope the node lives in and nothing outside it, +so a broadcaster in the root does not feed an input inside a subgraph, and one +inside a subgraph does not feed the root. That is deliberate: a subgraph whose +interior depended on invisible outside state would resolve differently +depending on where it was placed, and the workflow would not show why. + +If the original walked every graph in the document and wired across the cut, +convert the within-scope half — it is the part that still means something — +and refuse the crossing half by naming the mechanism, not the absence. + +## Node handles are accessors, not properties + +Every read and write on a `NodeHandle` is a method — the contract generated by +`scripts/node-api/gen_api_dts.mjs`, so a read can be a store query and a write +can become a command. Property syntax silently does nothing or throws. + +| Old (on the raw node) | Published API | +| ------------------------ | ------------------------------------------- | +| `node.title = t` | `node.setTitle(t)` / `getTitle()` | +| `node.color = c` | `node.setColor(c)` / `getColor()` | +| `node.bgcolor = c` | `node.setBgColor(c)` / `getBgColor()` | +| `node.mode = 4` | `node.setMode('bypass')` / `getMode()` | +| `node.flags.collapsed` | `node.isCollapsed()` / `setCollapsed(b)` | +| `node.flags.pinned` | `node.isPinned()` / `setPinned(b)` | +| `node.shape = s` | `node.setShape(s)` / `getShape()` | +| `node.properties[k] = v` | `node.setProperty(k, v)` / `getProperty(k)` | +| `node.size` / `node.pos` | `node.getSize()` / `getPosition()` | + +**Both setters take one tuple**, not two numbers — the legacy `setSize(w, h)` +and `setPos(x, y)` arities are gone: + +```js +// before +node.setSize([200, 58]) // LiteGraph took an array here +node.color = '#1b4669' +node.bgcolor = '#29699c' + +// after +node.setSize({ width: 200, height: 58 }) +node.setColor('#1b4669') +node.setBgColor('#29699c') +``` + +Widget handles follow the same rule: `getValue()`/`setValue()`, +`isHidden()`/`setHidden()`, `getOptions()`/`setOption()`, and `widgetType` +rather than `type`. See `widgets.md`. + +## Per-instance state + +Handles hold no arbitrary properties, so `node._myState = x` has no target. Keep +the state yourself, keyed by graph scope and node id: + +```js +// before — a property stashed on the node instance +nodeType.prototype.onExecuted = function (output) { + if (this._lastHash === hash) return + this._lastHash = hash +} + +// after — the pack owns its own state +const stateByNode = new Map() +const stateKey = (node) => `${node.graphId}:${node.id}` +const stateFor = (node) => { + const key = stateKey(node) + let s = stateByNode.get(key) + if (!s) stateByNode.set(key, (s = {})) + return s +} + +comfy.defs.extend('MyNode', (b) => { + b.onExecuted((node, result) => { + const state = stateFor(node) + if (state.lastHash === hash) return + state.lastHash = hash + }) + b.onRemoved((node) => stateByNode.delete(stateKey(node))) +}) +``` + +This is a supported conversion, not a workaround. Clean up in `onRemoved` — the +old form was collected with the node, and a Map is not. + +## Traps + +**The constructor self-assignment.** `this.type = this.type ?? undefined` is a +defensive no-op that now throws, killing construction entirely. Handled +mechanically by the `type-write-noop` rule — 8 packs, 3.86M downloads, including +rgthree's whole 12-type virtual-node family. + +**`nodeData` is a definition, not a node.** Reading `nodeData.input.required.x` +is fine; it becomes `b.def.inputs`. But the shape differs — do not assume a +field-by-field rename. + +**Async hooks.** `async beforeRegisterNodeDef` is common and usually gratuitous. +`defs.extend` is synchronous; if a pack genuinely needs async setup, do it in +`onCreated` and guard for the node being removed before it resolves. + +**Order-dependent patches.** A pack that reads `nodeType.prototype.onNodeCreated` +expecting another pack's patch to already be installed has no equivalent, by +design. Escalate — it needs the author. + +## Source data + +Prototype-assignment counts derive from the 4,969-pack registry corpus used +when this reference was authored (2,562 files scanned). Grep-derived; +sample-verify before citing an individual number. diff --git a/.claude/skills/converting-custom-nodes/references/nodegraph-101.md b/.claude/skills/converting-custom-nodes/references/nodegraph-101.md new file mode 100644 index 00000000000..3855565c11f --- /dev/null +++ b/.claude/skills/converting-custom-nodes/references/nodegraph-101.md @@ -0,0 +1,183 @@ +# ComfyUI node graphs in ten minutes + +Read this first if you have not converted a pack before. It exists because the +alternative is inferring the model from the pack's source, which is slow and +tends to produce a plausible but wrong mental picture. + +## What the thing is + +ComfyUI is a **node graph editor whose graph is a program**. The user wires +nodes together; the frontend compiles that wiring into a JSON payload; a Python +backend executes it and streams results back. + +``` +user edits graph → graphToPrompt() → POST /prompt → backend executes + ▲ │ + └────────────── node.onExecuted(message) ◀───────────────┘ +``` + +Two artifacts come out of the same graph, and confusing them is the single most +common conversion error: + +| | **Workflow** | **Prompt** | +| ------------------- | ---------------------------------------------------------- | -------------------------- | +| Purpose | what the user saves and reloads | what the backend runs | +| Contains | positions, colours, titles, collapsed state, widget values | node inputs and links only | +| Produced by | `graph.serialize()` | `graphToPrompt()` | +| Frontend-only nodes | present | must be resolved away | + +**Both must come out byte-identical after a conversion.** That is the hard +constraint: a user's saved file and the job they queue cannot change. + +## What a node actually is + +A node exists in two halves. + +**The backend half** is a Python class. It declares its inputs, outputs and +category, and the server sends that declaration to the frontend as a **node +definition** — `nodeData` in the old hook. This is a _description_, not a node. + +**The frontend half** is a JavaScript class generated _from_ that definition, +registered by type name. Every node the user drops on the canvas is an instance +of it. + +``` +node definition (from backend) → generated class → instances on the canvas + "KSampler", inputs, outputs KSampler node #7, node #12 +``` + +So there are three distinct things, and packs act on all three: + +- **the definition** — before any class exists +- **the class** — affects every instance of that type +- **the instance** — one node on one canvas + +A conversion that moves code between these levels changes behaviour. Watch for +it: `nodeData.name` is a _definition_; `this` inside `onNodeCreated` is an +_instance_. + +## The lifecycle + +Everything a pack hooks hangs off this sequence: + +1. **Definitions arrive** from the backend. +2. **`beforeRegisterNodeDef`** — for each definition, every extension gets a + chance to modify the definition and patch the about-to-be-registered class. + _This is where nearly half of all packs do their work._ +3. **`registerNodeType`** puts the class in the registry under its type name. +4. **Instance created** — user drops a node, or a workflow loads one. + `onNodeCreated` fires (before the node has an id or a graph). +5. **`onAdded`** — the node joins the graph. Now it has an id and is + addressable. _This is where the published `onCreated` fires, deliberately._ +6. **`onConfigure`** — only for nodes loaded from a saved workflow; restores + widget values and any pack-specific state. +7. **`onExecuted(message)`** — backend produced output for this node. +8. **`onRemoved`** — node deleted. + +## Inputs, outputs, widgets + +- **Inputs / outputs** are sockets. Links connect an output to an input. +- **Widgets** are the controls drawn _on_ the node — a seed number, a sampler + dropdown, a text box. A widget holds a value that becomes an input at + execution time. + +The wrinkle: a widget can be **promoted to a socket** so another node can drive +it. Historically the frontend faked this by setting `widget.type = +'converted-widget'` and stashing the old type — a hack that packs learned to +detect and imitate. It is now a real property. When you see +`'converted-widget'`, the pack is almost always trying to _hide_ a widget, not +change its kind. + +`widgets_values` in a saved workflow is a **positional array** — index matters, +names are not stored. This is why widget order and count are part of the wire +format, and why removing a widget is not a cosmetic change. + +## Why packs customise the frontend at all + +Packs are not being gratuitous. There are a handful of recurring motives, and +recognising which one you are looking at usually tells you the replacement. +Counts are sites across the ~5,000-pack registry corpus; a pack often has +several motives at once. + +| Motive | What it looks like | Scale | +| --------------------------------------------------------------- | ---------------------------------------------------- | ------------------- | +| **Show backend output on the node** — text, previews, progress | patch `onExecuted`, create a display widget | 497 packs | +| **Set up per-instance state** — dynamic inputs, defaults, DOM | patch `onNodeCreated` | 943 packs | +| **Restore that state on load** | patch `onConfigure` | 429 packs | +| **React to wiring** — add a slot when the last one fills | patch `onConnectionsChange` | 223 packs | +| **Draw on the node** — badges, overlays, custom controls | patch `onDrawForeground` | 199 packs | +| **Define frontend-only nodes** — reroutes, switches, note nodes | `registerCustomNodes`, `isVirtualNode` | 86 packs | +| **Change what gets saved or queued** | patch `serialize`, `serializeValue`, `graphToPrompt` | fewer, highest risk | + +The last row is where conversions do damage, because it is the row that touches +the wire format. + +## Why any of this needs converting + +Almost all of the above is done by **monkey-patching the generated class's +prototype**: + +```js +const original = nodeType.prototype.onExecuted +nodeType.prototype.onExecuted = function (message) { + original?.apply(this, arguments) // ← if you forget this, you break other packs + myBehaviour.call(this, message) +} +``` + +Three things are wrong with this, and they motivate the whole published API: + +1. **It reaches into internals.** `nodeType.prototype`, `node.widgets`, + `link.origin_id` are implementation, and they are being reshaped. +2. **It does not compose.** Whether your handler survives depends on every + other pack remembering to call through. One that forgets silently disables + yours, and load order decides who wins. +3. **It cannot be undone.** There is no unpatch, so nothing can be torn down. + +The published API replaces each of these with something registered rather than +patched: handlers are additive, ordered, and individually removable, and the +entity classes stay closed. + +## The mental model to convert with + +> A pack **declares** what it wants for **which node types**, and **reacts** to +> a small set of lifecycle events using **handles** that expose behaviour but +> not internals. + +Concretely, the shape of nearly every conversion is: + +```js +// before: run for every type, filter, patch the prototype +app.registerExtension({ + name: 'x', + beforeRegisterNodeDef(nodeType, nodeData) { + if (nodeData.name !== 'MyNode') return // ← the selector + const orig = nodeType.prototype.onExecuted // ← the chaining + nodeType.prototype.onExecuted = function (m) { + orig?.apply(this, arguments) + populate.call(this, m.text) // ← the actual behaviour + } + } +}) + +// after: the selector is declared, the chaining disappears, behaviour is unchanged +comfy.defs.extend('MyNode', (b) => { + b.onExecuted((node, result) => populate(node, result.text)) +}) +``` + +The guard clause becomes the selector. The capture-and-chain boilerplate goes +away entirely. What is left is the behaviour, which you should be changing as +little as possible. + +## Things that will mislead you + +- **`this` is not always a node.** Inside a patched prototype method it is an + instance; inside `beforeRegisterNodeDef` it is not. +- **`nodeData` is a definition, not a node.** It has no widgets and no id. +- **A widget named the same is not the same widget.** Packs remove and recreate + readout widgets on every execution. +- **`app` is the whole application.** A pack reaching for `app.graph` usually + wants its own node's graph, and often only needs the node. +- **Absence of a hook means nothing.** Plenty of packs put their logic in a + module-level side effect that runs at import. diff --git a/.claude/skills/converting-custom-nodes/references/widgets.md b/.claude/skills/converting-custom-nodes/references/widgets.md new file mode 100644 index 00000000000..e8d3cc0d0a5 --- /dev/null +++ b/.claude/skills/converting-custom-nodes/references/widgets.md @@ -0,0 +1,335 @@ +# Converting widget-array mutation and the converted-widget protocol + +> **Accessor style.** The published handles follow the generated declaration +> from `scripts/node-api/gen_api_dts.mjs`: reads and writes are methods, not properties — +> `widget.getValue()` / `setValue(v)`, `isHidden()` / `setHidden(b)`, +> `getOptions()` / `setOption(key, value)`, `setLabel(s)`, and `widgetType` for +> the type. Node handles keep `getTitle()`/`setTitle()` style likewise. + +The largest cohort by pack count, and the one where naive conversions are most +likely to be silently wrong. + +| Surface | Packs | Installs | +| ------------------------- | ----- | -------- | +| `widgets.splice` | 286 | 21.6% | +| `widget.type` overwrite | 270 | 18.6% | +| converted-widget protocol | 238 | 25.1% | +| `widgets = [...]` | 226 | 10.9% | +| `widgets.push` | 142 | — | +| `getCustomWidgets` POJO | 91 | 16.8% | +| `widgets.length = 0` | 31 | — | + +These overlap heavily; the same pack usually hits several. + +## Classify before converting — `splice` is often not a reorder + +```js +// ComfyUI-KJNodes setgetnodes.js:988 +// Fresh options object (live getter preserved) + remove/re-add to force +// Vue re-extraction. +w.options = newOpts +const idx = this.widgets.indexOf(w) +if (idx >= 0) { + this.widgets.splice(idx, 1) + this.widgets.splice(idx, 0, w) +} +``` + +Removed and reinserted **at the same index** — the array is unchanged. This is +cache invalidation, not reordering. It converts to: + +```js +node.widgets.get(name).setOption('values', newValues) +``` + +The hack disappears: invalidation is the API's problem now. A converter that +assumes "splice means reorder" produces nonsense here. + +**Read the indices before choosing a rule.** Same index in and out → invalidation. +Different index → a real move (`widgets.move`). A full rewrite of the array → +`widgets.reorder`. + +## The converted-widget protocol — ~20 lines become one property + +The full pattern, from ComfyUI-Easy-Use `easyExtraMenu.js:339`: + +```js +const CONVERTED_TYPE = 'converted-widget' + +function hideWidget(node, widget, suffix = '') { + widget.origType = widget.type + widget.origComputeSize = widget.computeSize + widget.origSerializeValue = widget.serializeValue + widget.computeSize = () => [0, -4] // -4 offsets litegraph's inter-widget gap + widget.type = CONVERTED_TYPE + suffix + widget.serializeValue = () => { + if (!node.inputs) return undefined + const input = node.inputs.find((i) => i.widget?.name === widget.name) + if (!input || !input.link) return undefined // unlinked → do not serialize + return widget.origSerializeValue + ? widget.origSerializeValue() + : widget.value + } + if (widget.linkedWidgets) { + for (const w of widget.linkedWidgets) hideWidget(node, w, ':' + widget.name) + } +} +``` + +Everything above exists to emulate a property that did not exist: + +```js +node.widgets.get(name).setHidden(true) +``` + +Gone with it: the `origType`/`origComputeSize`/`origSerializeValue` save-and- +restore dance, the `[0, -4]` magic number, and the type-string mangling. + +### ⚠️ The trap: `hidden` does not imply "do not serialize" + +The old hack **coupled** two things. Hiding a widget also installed a +`serializeValue` that returned `undefined` unless the matching input was linked. + +In the published API those are orthogonal — `hidden` is presentation, +`serialize` is persistence. That is the better design, but it means a literal +one-line conversion **changes behaviour**: a hidden widget now serializes where +it previously did not. + +So check what the pack relied on: + +- Hiding purely for presentation → `setHidden(true)` is complete. +- Hiding _and_ suppressing serialization → call `setHidden(true)`, and handle + serialization explicitly. If the value should persist only when the socket is + connected, that condition belongs in the pack's own `onSerialize`. + +This is a wire-format change if you get it wrong, so it will be caught by the +gate — but understand it rather than letting the gate find it. + +### Linked widgets + +The recursion over `widget.linkedWidgets` (seed + seed-control being the classic +pair) becomes an explicit relationship. `widget.setLinked(names)` records it, +`widget.linked()` reads it, and `setHidden()` cascades to linked widgets. The +recursive hide bookkeeping goes away. + +## `widget.type` overwrite — 270 packs + +Almost always the converted-widget hack above. If it is genuinely trying to +change a widget's _kind_, there is no replacement: type is identity. Remove the +widget and add the intended one. + +```js +// before +widget.type = 'converted-widget' // → widget.setHidden(true) + +// before — genuinely changing kind +widget.type = 'combo' // → remove + add, or escalate +``` + +## Array assignment and truncation + +```js +// before +this.widgets = this.widgets.filter((w) => w.name !== 'seed') +this.widgets.length = 0 +this.widgets.push(w) + +// after +node.widgets.remove('seed') +for (const name of node.widgets.names()) node.widgets.remove(name) +node.widgets.add(def) +``` + +Two reasons not to translate these literally: + +- **Assigning a new array drops the renderer's tracking.** The array identity is + what the renderer watches; `widgets.reorder` splices in place for exactly this + reason. +- **Assigning `length` skips teardown.** Packs that do it correctly call + `widget.onRemove?.()` first — see Custom-Scripts `showText.js`. `remove()` + runs teardown for you, so the manual loop goes away. + +## Creating a widget + +`ComfyWidgets.*` is an unpublished internal. `node.widgets.add(def)` replaces it, +and the `def` is plain data — no `node`, no `app`, no return-value unwrapping: + +```js +// before +const w = ComfyWidgets.STRING( + this, + 'text', + ['STRING', { multiline: true }], + app +).widget +w.inputEl.readOnly = true +w.inputEl.style.opacity = 0.6 + +// after +const widget = node.widgets.add({ + type: 'textarea', + name: 'text', + value: '', + disabled: true +}) +``` + +`disabled: true` replaces the `readOnly` + `opacity` pair — the two lines packs +use to fake a read-only widget. It is a real property, so the styling stays +consistent with every other disabled widget instead of being hand-rolled. + +| Old factory | `type` | +| ----------------------------------------------- | ------------------------------------------- | +| `ComfyWidgets.STRING(..., { multiline: true })` | `'textarea'` | +| `ComfyWidgets.STRING(...)` | `'string'` | +| `ComfyWidgets.INT` / `FLOAT` | `'number'` (or `'slider'` with `min`/`max`) | +| `ComfyWidgets.BOOLEAN` | `'toggle'` | +| `ComfyWidgets.COMBO` | `'combo'`, values via `options.values` | +| `ComfyWidgets.MARKDOWN` | `'markdown'` | +| `ComfyWidgets.COLOR` | `'color'` | + +### Keeping a widget out of the prompt, or out of the saved workflow + +These are two destinations with two switches. Convert each to its own. + +```js +// before: the value is saved in the workflow and not sent in the prompt +widget.serializeValue = async () => {} + +// after +node.widgets.add({ + type: 'textarea', + name: 'text', + options: { serialize: false } +}) +// or, for a widget that already exists +widget.setOption('serialize', false) +``` + +```js +// before: none of the node's widget values are saved in the workflow +this.serialize_widgets = false + +// after +node.setSerializeWidgets(false) +``` + +`serialize: false` on a widget definition is a third thing: that one widget is +not saved in the workflow, and is still sent in the prompt. It does not replace +a `serializeValue` override. + +All three are wire-format switches, so a conversion that drops or swaps one +changes what the saved workflow or the prompt contains. `serialize` is orthogonal to `hidden` and `disabled`: +a widget can be visible and unsaved, or hidden and saved. + +`add` throws if the name is already taken — rebuild is remove-then-add, and the +throw catches the common bug of appending a duplicate every execution. + +**Do not convert a remove-and-recreate into `remove` plus a bare +`ComfyWidgets.*` call.** That leaves the pack on the unpublished surface, which +is the entire thing the conversion exists to end. If the widget it needs has no +`add` equivalent, that is an `api-gap` punt, not a partial conversion. + +## Never let a write vanish + +A handle lookup can fail. Reached through `?.`, a _write_ behind it does +nothing at all and the pack cannot tell: + +```js +// ✗ silently does nothing when the node has not joined a graph yet — which is +// exactly when a helper like this is usually called +comfy.graph.node(String(node.id))?.widgets.get(name)?.setHidden(true) + +// ✓ use the handle you were given +function hideForGood(node, name) { + node.widgets.get(name)?.setHidden(true) // reads may be optional +} +``` + +`comfy.graph.node(id)` resolves through the graph, so it returns nothing for a +node that has not been added yet. If you find yourself looking a node up by id +inside a helper, the helper should be taking a `NodeHandle` instead — its +caller has one. + +Optional chaining on a read returns `undefined` for a missing widget. On a +write, it silently skips the mutation. + +## Reordering + +```js +node.widgets.reorder(['prompt', 'seed', 'steps']) // full permutation +node.widgets.move('prompt', 0) // single move +``` + +`reorder` **throws on a partial list** rather than dropping the widgets you +omitted — which is precisely how the splice idiom lost them. The error names +what is missing. + +## Updating options with live combo values + +```js +const comboOptions = { + get values() { + return currentModels + } +} +``` + +To keep reading that incoming getter as its backing list changes, pass a +function: + +```js +widget.setOption('values', () => comboOptions.values) +``` + +`widget.setOption('values', comboOptions.values)` reads the getter immediately +and installs that one list. It does not copy the incoming property descriptor. +`setOption` preserves getters only on existing options that it does not replace; +changing `tooltip`, for example, leaves an existing `values` getter live. + +Spreading `widget.getOptions()` also reads getters and keeps their current +values. Use a function for incoming combo values that must stay live. + +`getOptions()` returns a frozen _snapshot_ by design; use `setOption` to write. + +## `getCustomWidgets` — 91 packs + +Register one renderer for the backend-declared input type: + +```js +comfy.defs.defineWidgetType('MY_PACK_SLIDER', { + defaultValue: 0, + height: 40, + render(container, value) { + const input = document.createElement('input') + input.type = 'range' + input.value = String(value.get()) + const onInput = () => value.set(Number(input.value)) + input.addEventListener('input', onInput) + container.append(input) + return () => input.removeEventListener('input', onInput) + } +}) +``` + +`render` receives a host-owned container and a value handle. Packs may use +their own framework or plain DOM, and must return teardown for listeners, +timers, and observers. Defining the type preserves the backend input's original +positional `widgets_values` cell; adding a second mounted widget does not. + +## Traps summary + +| Trap | Consequence | +| ------------------------------------------------------------------------- | ------------------------------------------------ | +| Treating same-index splice as a reorder | Nonsense conversion of a cache-invalidation hack | +| `setHidden(true)` alone, where the old code also suppressed serialization | Wire-format change | +| Hand-merging options | Live getters flattened; dynamic combos freeze | +| Assigning a new `widgets` array | Renderer stops tracking | +| Assigning `widgets.length` | Widget teardown skipped | +| Partial `reorder` list | Throws — by design; supply every name | + +## Source data + +Counts from the 4,969-pack registry census used when this reference was +authored. Grep-derived with a known false-positive rate — sample-verify before +citing an individual number. diff --git a/.fallowrc.jsonc b/.fallowrc.jsonc index 4a1c05c0ca5..f65ce42719d 100644 --- a/.fallowrc.jsonc +++ b/.fallowrc.jsonc @@ -31,7 +31,7 @@ }, // Runtime entry points fallow cannot infer. - "entry": ["index.html", "src/main.ts"], + "entry": ["examples/node-api/**/web/*.js", "index.html", "src/main.ts"], // Keep tooling and test roots reachable without treating their imports as // production runtime dependencies. @@ -43,7 +43,7 @@ "**/.storybook/{main.ts,preview.ts,style.css}", "packages/comfyui-api-types/openapi-ts.config.ts", "packages/ingest-types/openapi-ts.config.ts", - "scripts/**/*.{js,ts}" + "scripts/**/*.{js,mjs,ts}" ], "entryPointRole": "support" }, @@ -101,7 +101,8 @@ // Static stylesheet referenced from index.html, resolved by the bundler. "./materialdesignicons.min.css", // Generated sibling JS for the desktop-bridge type stubs. - "./comfyDesktopBridge.js" + "./comfyDesktopBridge.js", + "/comfy/api/v2.js" ], "ignoreExports": [ diff --git a/.github/actions/start-comfyui-server/action.yaml b/.github/actions/start-comfyui-server/action.yaml index 2af727508d6..c9b030c0350 100644 --- a/.github/actions/start-comfyui-server/action.yaml +++ b/.github/actions/start-comfyui-server/action.yaml @@ -19,5 +19,10 @@ runs: run: | set -euo pipefail cp -r ./tools/devtools/* /ComfyUI/custom_nodes/ComfyUI_devtools/ + for pack in ./examples/node-api/how_to_*; do + target="/ComfyUI/custom_nodes/$(basename "$pack")" + mkdir -p "$target" + cp -r "$pack"/* "$target"/ + done cd /ComfyUI && python3 main.py --cpu --multi-user --front-end-root "${{ inputs.front_end_root }}" & wait-for-it --service 127.0.0.1:8188 -t ${{ inputs.timeout }} diff --git a/.oxlintrc.json b/.oxlintrc.json index 18b148b3299..7b765f202f3 100644 --- a/.oxlintrc.json +++ b/.oxlintrc.json @@ -11,6 +11,7 @@ "components.d.ts", "coverage/*", "dist/*", + "docs/node-api/comfy-api.d.ts", "packages/design-system/src/css/lucideStrokePlugin.js", "packages/ingest-types/src/*.gen.ts", "packages/registry-types/src/comfyRegistryTypes.ts", @@ -516,7 +517,11 @@ } }, { - "files": ["packages/design-system/src/css/**", "tools/devtools/web/**"], + "files": [ + "packages/design-system/src/css/**", + "tools/devtools/web/**", + "examples/node-api/**/web/*.js" + ], "rules": { "comfy/no-js-extension-imports": "off" } @@ -639,6 +644,9 @@ "files": ["src/**", "packages/*/src/**"], "excludeFiles": [ "src/config/viteSeo.test.ts", + "src/platform/nodeApi/apiDts.test.ts", + "src/platform/nodeApi/apiSurface.test.ts", + "src/platform/nodeApi/capabilityDoc.test.ts", "src/stores/nodeDefStore.frequency.test.ts" ], "rules": { diff --git a/browser_tests/README.md b/browser_tests/README.md index 08b1ce24c2d..e615896d78b 100644 --- a/browser_tests/README.md +++ b/browser_tests/README.md @@ -58,15 +58,35 @@ install, install and start Docker, then run: pnpm container:start ``` -The command mounts `tools/devtools` and starts ComfyUI at `localhost:8188`. +The command mounts `tools/devtools`, the published node API example packs under +`examples/node-api`, and starts ComfyUI at `localhost:8188`. Leave it running. Use another terminal for `pnpm dev:test` and a third for `pnpm test:browser:local`. Run browser tests against port 5173 when using this container. Port 8188 serves the frontend bundled into the container, not the current frontend checkout. -The CI image is private. If it is not cached, the launcher pulls it from GHCR -with `GH_TOKEN` and gets the matching username from `gh api user`. +### Testing custom-node packs + +The dev server answers `/api/extensions` with an empty list, because pack +scripts are served by the backend and not by Vite. Port 5173 therefore loads no +pack at all, and a test that drives one passes vacuously or fails for the wrong +reason. To exercise a pack, serve your own build from the container instead: + +```bash +pnpm build +COMFYUI_FRONTEND_ROOT="$PWD/dist" pnpm container:start +PLAYWRIGHT_TEST_URL=http://localhost:8188 \ + PLAYWRIGHT_SETUP_API_URL=http://localhost:8188 \ + pnpm exec playwright test --project=chromium +``` + +Both URLs matter: `.env` pins `PLAYWRIGHT_SETUP_API_URL` to another port, and +without overriding it every test dies in fixture setup with a 405 that looks +like a product failure. + +If the image is not cached, the launcher tries to pull it from GHCR. It uses +`GH_TOKEN` and gets the matching username from `gh api user`. `COMFY_CI_CONTAINER_TOKEN` and `COMFY_CI_CONTAINER_USER` take precedence when set. The token must be a personal access token (classic) with the `read:packages` scope, owned by an account that can read diff --git a/browser_tests/assets/nodeApi/how-to-examples.json b/browser_tests/assets/nodeApi/how-to-examples.json new file mode 100644 index 00000000000..d9922afcb5f --- /dev/null +++ b/browser_tests/assets/nodeApi/how-to-examples.json @@ -0,0 +1,211 @@ +{ + "last_node_id": 14, + "last_link_id": 2, + "nodes": [ + { + "id": 1, + "type": "HowTo/ConstantText", + "pos": [80, 80], + "size": [280, 80], + "flags": {}, + "order": 0, + "mode": 0, + "inputs": [], + "outputs": [ + { "name": "text", "type": "STRING", "links": [1], "slot_index": 0 } + ], + "properties": {}, + "widgets_values": ["Hello from the How-To packs"] + }, + { + "id": 2, + "type": "HowTo/Reroute", + "pos": [420, 80], + "size": [180, 60], + "flags": {}, + "order": 1, + "mode": 0, + "inputs": [{ "name": "in", "type": "*", "link": 1 }], + "outputs": [ + { "name": "out", "type": "*", "links": [2], "slot_index": 0 } + ], + "properties": {} + }, + { + "id": 3, + "type": "HowToTextOutput", + "pos": [680, 80], + "size": [280, 120], + "flags": {}, + "order": 2, + "mode": 0, + "inputs": [{ "name": "text", "type": "STRING", "link": 2 }], + "outputs": [], + "properties": { "Node name for S&R": "HowToTextOutput" }, + "widgets_values": ["Run this node"] + }, + { + "id": 4, + "type": "HowTo/WidgetEvents", + "pos": [80, 280], + "size": [260, 120], + "flags": {}, + "order": 3, + "mode": 0, + "inputs": [], + "outputs": [ + { "name": "count", "type": "INT", "links": null, "slot_index": 0 } + ], + "properties": {}, + "widgets_values": [0, null] + }, + { + "id": 5, + "type": "HowTo/CanvasMeter", + "pos": [400, 280], + "size": [260, 120], + "flags": {}, + "order": 4, + "mode": 0, + "inputs": [], + "outputs": [ + { "name": "amount", "type": "FLOAT", "links": null, "slot_index": 0 } + ], + "properties": {}, + "widgets_values": [0.5] + }, + { + "id": 6, + "type": "HowTo/MountedSlider", + "pos": [720, 280], + "size": [260, 120], + "flags": {}, + "order": 5, + "mode": 0, + "inputs": [], + "outputs": [ + { "name": "amount", "type": "FLOAT", "links": null, "slot_index": 0 } + ], + "properties": {}, + "widgets_values": [0.5] + }, + { + "id": 7, + "type": "HowTo/LifecycleBadge", + "pos": [1040, 280], + "size": [240, 100], + "flags": {}, + "order": 6, + "mode": 0, + "inputs": [], + "outputs": [], + "properties": {}, + "howToState": "paused" + }, + { + "id": 8, + "type": "HowTo/DropTextFile", + "pos": [80, 500], + "size": [280, 160], + "flags": {}, + "order": 7, + "mode": 0, + "inputs": [], + "outputs": [ + { "name": "text", "type": "STRING", "links": null, "slot_index": 0 } + ], + "properties": {}, + "widgets_values": ["Drop a .txt file on this node"] + }, + { + "id": 9, + "type": "HowTo/GraphBuilder", + "pos": [420, 500], + "size": [260, 160], + "flags": {}, + "order": 8, + "mode": 0, + "inputs": [], + "outputs": [], + "properties": {}, + "widgets_values": [null, null, null] + }, + { + "id": 10, + "type": "HowTo/BackendPing", + "pos": [740, 500], + "size": [260, 140], + "flags": {}, + "order": 9, + "mode": 0, + "inputs": [], + "outputs": [], + "properties": {}, + "widgets_values": [null] + }, + { + "id": 11, + "type": "HowTo/BackendEvent", + "pos": [1060, 500], + "size": [260, 140], + "flags": {}, + "order": 10, + "mode": 0, + "inputs": [], + "outputs": [], + "properties": {}, + "widgets_values": [null] + }, + { + "id": 12, + "type": "HowTo/SettingsStorage", + "pos": [80, 740], + "size": [280, 180], + "flags": {}, + "order": 11, + "mode": 0, + "inputs": [], + "outputs": [], + "properties": {}, + "widgets_values": ["Hello from storage", null, null, null] + }, + { + "id": 13, + "type": "HowToRating", + "pos": [420, 740], + "size": [280, 120], + "flags": {}, + "order": 12, + "mode": 0, + "inputs": [], + "outputs": [ + { "name": "rating", "type": "INT", "links": null, "slot_index": 0 } + ], + "properties": { "Node name for S&R": "HowToRating" }, + "widgets_values": [3] + }, + { + "id": 14, + "type": "HowToTemplateText", + "pos": [760, 740], + "size": [280, 160], + "flags": {}, + "order": 13, + "mode": 0, + "inputs": [], + "outputs": [ + { "name": "text", "type": "STRING", "links": null, "slot_index": 0 } + ], + "properties": { "Node name for S&R": "HowToTemplateText" }, + "widgets_values": ["rendered-%date:yyyy-MM-dd%"] + } + ], + "links": [ + [1, 1, 0, 2, 0, "*"], + [2, 2, 0, 3, 0, "STRING"] + ], + "groups": [], + "config": {}, + "extra": { "ds": { "offset": [0, 0], "scale": 1 } }, + "version": 0.4 +} diff --git a/browser_tests/assets/nodeApi/stable-api-demo.json b/browser_tests/assets/nodeApi/stable-api-demo.json new file mode 100644 index 00000000000..347b85e518d --- /dev/null +++ b/browser_tests/assets/nodeApi/stable-api-demo.json @@ -0,0 +1,68 @@ +{ + "last_node_id": 3, + "last_link_id": 2, + "nodes": [ + { + "id": 1, + "type": "DEMO/ConstantText", + "pos": [120, 180], + "size": [280, 80], + "flags": {}, + "order": 0, + "mode": 0, + "inputs": [], + "outputs": [ + { + "name": "text", + "type": "STRING", + "links": [1], + "slot_index": 0 + } + ], + "properties": {}, + "widgets_values": ["Hello from the published node API"] + }, + { + "id": 2, + "type": "DEMO/Reroute", + "pos": [500, 180], + "size": [180, 60], + "flags": {}, + "order": 1, + "mode": 0, + "inputs": [{ "name": "in", "type": "*", "link": 1 }], + "outputs": [ + { + "name": "out", + "type": "*", + "links": [2], + "slot_index": 0 + } + ], + "properties": {} + }, + { + "id": 3, + "type": "PreviewAny", + "pos": [780, 180], + "size": [260, 100], + "flags": {}, + "order": 2, + "mode": 0, + "inputs": [{ "name": "source", "type": "*", "link": 2 }], + "outputs": [], + "properties": { "Node name for S&R": "PreviewAny" }, + "widgets_values": [] + } + ], + "links": [ + [1, 1, 0, 2, 0, "*"], + [2, 2, 0, 3, 0, "*"] + ], + "groups": [], + "config": {}, + "extra": { + "ds": { "offset": [0, 0], "scale": 1 } + }, + "version": 0.4 +} diff --git a/browser_tests/fixtures/utils/litegraphUtils.ts b/browser_tests/fixtures/utils/litegraphUtils.ts index 1f495459f9a..319e8eac07c 100644 --- a/browser_tests/fixtures/utils/litegraphUtils.ts +++ b/browser_tests/fixtures/utils/litegraphUtils.ts @@ -188,6 +188,30 @@ class NodeWidgetReference { * @returns The position of the widget's center */ async getPosition(): Promise { + if ( + await this.node.comfyPage.settings.getSetting( + 'Comfy.VueNodes.Enabled' + ) + ) { + const selector = await this.node.comfyPage.page.evaluate( + ([id, index]) => { + const widget = window + .app!.canvas.graph!.getNodeById(id) + ?.widgets?.at(index) + if (!widget) + throw new Error(`Widget ${index} on node ${id} not found.`) + return `[data-widget-name="${CSS.escape(widget.name)}"]` + }, + [this.node.id, this.index] as const + ) + const widget = this.node.comfyPage.vueNodes + .getNodeLocator(String(this.node.id)) + .locator(selector) + await expect(widget).toBeVisible() + const bounds = await widget.boundingBox() + if (!bounds) throw new Error('Widget position is unavailable') + return { x: bounds.x + bounds.width / 2, y: bounds.y + bounds.height / 2 } + } const position = await this.node.comfyPage.page.waitForFunction( ([id, index]) => { const node = window.app!.canvas.graph!.getNodeById(id) @@ -529,8 +553,9 @@ export class NodeReference { } async getContextMenuOptionNames() { await this.click('title', { button: 'right' }) - const ctx = this.comfyPage.page.locator('.litecontextmenu') - return await ctx.locator('.litemenu-entry').allInnerTexts() + const items = this.comfyPage.contextMenu.menuItems + await expect(items.first()).toBeVisible() + return await items.allInnerTexts() } async clickContextMenuOption(optionText: string) { await this.click('title', { button: 'right' }) diff --git a/browser_tests/tests/nodeApi/howToExamples.spec.ts b/browser_tests/tests/nodeApi/howToExamples.spec.ts new file mode 100644 index 00000000000..aeac5bc1f4a --- /dev/null +++ b/browser_tests/tests/nodeApi/howToExamples.spec.ts @@ -0,0 +1,271 @@ +import { + comfyExpect as expect, + comfyPageFixture as test +} from '@e2e/fixtures/ComfyPage' + +const HOW_TO_TEXT = 'Hello from the How-To packs' + +for (const vueNodes of [false, true]) { + test.describe( + `Published API How-To packs (${vueNodes ? 'Vue' : 'canvas'})`, + { tag: ['@node', '@widget', ...(vueNodes ? ['@vue-nodes'] : [])] }, + () => { + test('register and exercise representative examples', async ({ + comfyPage + }) => { + await comfyPage.workflow.loadWorkflow('nodeApi/how-to-examples') + + const workflow = await comfyPage.workflow.getExportedWorkflow() + expect(workflow.nodes).toHaveLength(14) + expect(workflow.nodes.map(({ type }) => type)).toEqual( + expect.arrayContaining([ + 'HowTo/ConstantText', + 'HowTo/Reroute', + 'HowToTextOutput', + 'HowTo/WidgetEvents', + 'HowTo/CanvasMeter', + 'HowTo/MountedSlider', + 'HowTo/LifecycleBadge', + 'HowTo/DropTextFile', + 'HowTo/GraphBuilder', + 'HowTo/BackendPing', + 'HowTo/BackendEvent', + 'HowTo/SettingsStorage', + 'HowToRating', + 'HowToTemplateText' + ]) + ) + expect( + workflow.nodes.find(({ id }) => String(id) === '7')?.howToState + ).toBe('paused') + + const lifecycleBadge = await comfyPage.nodeOps.getNodeRefById(7) + await lifecycleBadge.centerOnNode() + expect(await lifecycleBadge.getContextMenuOptionNames()).toContain( + 'Resume' + ) + await comfyPage.contextMenu.clickMenuItem('Resume') + await comfyPage.contextMenu.waitForHidden() + const resumedWorkflow = await comfyPage.workflow.getExportedWorkflow() + expect( + resumedWorkflow.nodes.find(({ id }) => String(id) === '7')?.howToState + ).toBe('ready') + + const prompt = await comfyPage.workflow.getExportedWorkflow({ + api: true + }) + expect(Object.keys(prompt)).toHaveLength(3) + expect(Object.keys(prompt)).toEqual( + expect.arrayContaining(['13', '14', '3']) + ) + expect(prompt['3']).toEqual({ + inputs: { text: HOW_TO_TEXT }, + class_type: 'HowToTextOutput', + _meta: { title: 'How-To: Text Output' } + }) + expect(prompt['13']).toEqual({ + inputs: { rating: 3 }, + class_type: 'HowToRating', + _meta: { title: 'How-To: Custom Rating Widget' } + }) + expect(prompt['14'].inputs.text).toMatch(/^rendered-\d{4}-\d{2}-\d{2}$/) + + await comfyPage.command.executeCommand('Comfy.QueuePrompt') + + // The backend node returns its text as ui output, which lands in the + // host's per-node output record when the `executed` message arrives. + // Until it does, the run has only been queued, not executed. Keyed by + // node locator, which is the bare id or `:` for the root. + await expect + .poll( + () => + comfyPage.page.evaluate(() => { + const outputs = window.app!.nodeOutputs + const entry = Object.entries(outputs).find( + ([key]) => key === '3' || key.endsWith(':3') + ) + const output = entry?.[1] + const text = output?.text + return Array.isArray(text) ? text[0] : text + }), + { timeout: 10_000 } + ) + .toBe(HOW_TO_TEXT) + + const widgetEvents = await comfyPage.nodeOps.getNodeRefById(4) + await widgetEvents.centerOnNode() + await (await widgetEvents.getWidget(1)).click() + await expect + .poll(() => widgetEvents.getWidget(0).then((w) => w.getValue())) + .toBe(1) + + const backendPing = await comfyPage.nodeOps.getNodeRefById(10) + await backendPing.centerOnNode() + await (await backendPing.getWidget(0)).click() + await expect + .poll(() => backendPing.getWidget(1).then((w) => w.getValue())) + .toBe('Hello from Python') + + const backendEvent = await comfyPage.nodeOps.getNodeRefById(11) + await backendEvent.centerOnNode() + await (await backendEvent.getWidget(0)).click() + await expect + .poll(() => backendEvent.getWidget(1).then((w) => w.getValue())) + .toBe('Event for node 11') + + const graphBuilder = await comfyPage.nodeOps.getNodeRefById(9) + await graphBuilder.centerOnNode() + await (await graphBuilder.getWidget(0)).click() + await expect.poll(() => comfyPage.nodeOps.getNodeCount()).toBe(16) + + await comfyPage.command.executeCommand('Comfy.Undo') + await expect.poll(() => comfyPage.nodeOps.getNodeCount()).toBe(14) + }) + + test('reports rejected asynchronous example actions', async ({ + comfyPage + }) => { + await comfyPage.workflow.loadWorkflow('nodeApi/how-to-examples') + + const invalidEventResponse = await comfyPage.page.request.post( + `${comfyPage.url}/how-to-api/event`, + { data: [] } + ) + expect(invalidEventResponse.status()).toBe(400) + + const backendPing = await comfyPage.nodeOps.getNodeRefById(10) + await backendPing.centerOnNode() + await comfyPage.page.route('**/how-to-api/ping', (route) => + route.fulfill({ status: 503, body: 'Unavailable' }) + ) + await (await backendPing.getWidget(0)).click() + await expect + .poll(() => + backendPing.getWidget(1).then((widget) => widget.getValue()) + ) + .toContain('Backend ping failed: HTTP 503') + const previousPingError = await backendPing + .getWidget(1) + .then((widget) => widget.getValue()) + + await comfyPage.page.unroute('**/how-to-api/ping') + await comfyPage.page.route('**/how-to-api/ping', (route) => + route.fulfill({ + status: 200, + contentType: 'application/json', + body: 'not json' + }) + ) + await (await backendPing.getWidget(0)).click() + await expect + .poll(() => + backendPing.getWidget(1).then((widget) => widget.getValue()) + ) + .not.toBe(previousPingError) + await expect + .poll(() => + backendPing.getWidget(1).then((widget) => widget.getValue()) + ) + .toContain('Backend ping failed:') + + const backendEvent = await comfyPage.nodeOps.getNodeRefById(11) + await backendEvent.centerOnNode() + await comfyPage.page.route('**/how-to-api/event', (route) => + route.fulfill({ status: 500, body: 'Event rejected' }) + ) + await (await backendEvent.getWidget(0)).click() + await expect + .poll(() => + backendEvent.getWidget(1).then((widget) => widget.getValue()) + ) + .toContain('Backend event failed: HTTP 500') + + const settingsStorage = await comfyPage.nodeOps.getNodeRefById(12) + await settingsStorage.centerOnNode() + await comfyPage.page.route('**/api/userdata/**', (route) => + route.fulfill({ status: 500, body: 'Storage rejected' }) + ) + await (await settingsStorage.getWidget(1)).click() + await expect + .poll(() => + settingsStorage.getWidget(4).then((widget) => widget.getValue()) + ) + .toContain('Save greeting failed:') + }) + + test('reports a rejected command action', async ({ comfyPage }) => { + await comfyPage.page.route( + '**/extensions/how_to_execution/execution.js', + async (route) => { + const response = await route.fetch() + const source = await response.text() + const rejectedSource = source.replace( + "api.commands.run('HowTo.ApiExamples.showGreeting')", + "api.commands.run('HowTo.ApiExamples.missing')" + ) + if (rejectedSource === source) { + throw new Error('Expected to replace the example command') + } + await route.fulfill({ response, body: rejectedSource }) + } + ) + await comfyPage.page.reload() + await comfyPage.waitForAppReady() + await comfyPage.workflow.loadWorkflow('nodeApi/how-to-examples') + + const settingsStorage = await comfyPage.nodeOps.getNodeRefById(12) + await settingsStorage.centerOnNode() + await (await settingsStorage.getWidget(3)).click() + await expect + .poll(() => + settingsStorage.getWidget(4).then((widget) => widget.getValue()) + ) + .toContain('Run greeting command failed: Command') + }) + } + ) +} + +test( + 'persists public storage across a page reload', + { tag: ['@node'] }, + async ({ comfyPage }) => { + const name = 'HowTo.ApiExamples/storage-round-trip' + const value = 'A saved greeting\n日本語 👋' + await comfyPage.page.evaluate( + async ({ name, value }) => { + if (!window.comfy) throw new Error('Published API is unavailable') + await window.comfy.storage.set(name, value) + }, + { name, value } + ) + + try { + await comfyPage.workflow.reloadAndWaitForApp() + + expect( + await comfyPage.page.evaluate(async (name) => { + if (!window.comfy) throw new Error('Published API is unavailable') + return window.comfy.storage.get(name) + }, name) + ).toBe(value) + expect( + await comfyPage.page.evaluate(async () => { + if (!window.comfy) throw new Error('Published API is unavailable') + return window.comfy.storage.list('HowTo.ApiExamples') + }) + ).toContain(name) + } finally { + await comfyPage.page.evaluate(async (name) => { + if (!window.comfy) throw new Error('Published API is unavailable') + await window.comfy.storage.remove(name) + }, name) + } + expect( + await comfyPage.page.evaluate(async (name) => { + if (!window.comfy) throw new Error('Published API is unavailable') + return window.comfy.storage.get(name) + }, name) + ).toBeUndefined() + } +) diff --git a/browser_tests/tests/nodeApi/stableApiDemo.spec.ts b/browser_tests/tests/nodeApi/stableApiDemo.spec.ts new file mode 100644 index 00000000000..52de7697dd6 --- /dev/null +++ b/browser_tests/tests/nodeApi/stableApiDemo.spec.ts @@ -0,0 +1,40 @@ +import { + comfyExpect as expect, + comfyPageFixture as test +} from '@e2e/fixtures/ComfyPage' + +const DEMO_TEXT = 'Hello from the published node API' + +test.describe('Published node API demos', { tag: ['@node'] }, () => { + test('resolve into and execute a backend prompt', async ({ comfyPage }) => { + await comfyPage.workflow.loadWorkflow('nodeApi/stable-api-demo') + + const workflow = await comfyPage.workflow.getExportedWorkflow() + expect(workflow.nodes).toHaveLength(3) + expect(workflow.nodes.map(({ type }) => type)).toEqual( + expect.arrayContaining([ + 'DEMO/ConstantText', + 'DEMO/Reroute', + 'PreviewAny' + ]) + ) + + const prompt = await comfyPage.workflow.getExportedWorkflow({ api: true }) + expect(prompt).toEqual({ + '3': { + inputs: { source: DEMO_TEXT }, + class_type: 'PreviewAny', + _meta: { title: 'Preview as Text' } + } + }) + + await comfyPage.command.executeCommand('Comfy.QueuePrompt') + + const output = await comfyPage.nodeOps.getNodeRefById(3) + await expect + .poll(async () => (await output.getWidget(0)).getValue(), { + timeout: 10_000 + }) + .toBe(DEMO_TEXT) + }) +}) diff --git a/browser_tests/tests/settings/agentExtensionProvenance.spec.ts b/browser_tests/tests/settings/agentExtensionProvenance.spec.ts index f1f5231811d..713a090f614 100644 --- a/browser_tests/tests/settings/agentExtensionProvenance.spec.ts +++ b/browser_tests/tests/settings/agentExtensionProvenance.spec.ts @@ -9,6 +9,16 @@ test.describe( 'Agent-created extension provenance', { tag: '@settings' }, () => { + test.beforeEach(async ({ page }) => { + await page.route('**/extensions', (route) => + route.fulfill({ + json: [ + '/extensions/ComfyUI_devtools/legacyCompatibility.js' + ] satisfies GetExtensionsResponse + }) + ) + }) + test( 'shows agent attribution and filters the extension list', { diff --git a/docs/node-api/README.md b/docs/node-api/README.md new file mode 100644 index 00000000000..8e84dadc125 --- /dev/null +++ b/docs/node-api/README.md @@ -0,0 +1,142 @@ +# Published custom-node API + +The published custom-node API is the supported JavaScript surface for extending +ComfyUI nodes and the editor without depending on LiteGraph, renderer, store, +DOM, or application internals. + +```js +import { comfy } from '/comfy/api/v2.js' + +comfy.defs.extend('KSampler', (definition) => { + definition.addMenuItem({ + label: 'Reset seed', + run(node) { + node.widgets.get('seed')?.setValue(0) + } + }) +}) +``` + +The module path is `/comfy/api/v2.js`; the API contract currently reports +`comfy.version === '2.0'`. The path identifies the published module generation, +while `comfy.version` versions the contract exposed by that module. + +## Why this API exists + +Legacy extensions receive live frontend objects and commonly patch prototypes, +mutate arrays, paint into the shared canvas, inspect application globals, and +reach into Pinia stores. Those techniques bind a pack to one renderer and one +internal data model. They also make ordinary frontend refactors ecosystem-wide +breaking changes. + +The published API exposes intent instead: + +- definition hooks instead of constructor and prototype patches; +- ID-backed handles instead of live entity objects; +- collections and frozen snapshots instead of mutable arrays; +- graph and slot operations instead of editing link records; +- mounted or host-rendered UI instead of host DOM selectors; +- capability probing instead of frontend-version guesses. + +The goal is functional migration, not a spelling change. If a pack used an +internal mechanism for a feature the API supports another way, use the supported +mechanism and preserve the feature. + +## Start here + +1. [Key concepts](./concepts.md) explains handles, snapshots, graph scopes, + lifecycle, mutation, and serialization. +2. [Tutorial](./tutorial.md) builds a small extension around an existing backend + node. +3. [Migration how-to](./how-to.md) maps common LiteGraph and legacy `app` + mechanisms to behavior-preserving published API replacements. +4. [Registration and lifecycle](./registration.md) covers module loading, + capability probing, definition registration, globals, and teardown. + +## Guides + +| Guide | Covers | +| ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------- | +| [Migration how-to](./how-to.md) | task-oriented LiteGraph-to-API recipes, intent forks, before/after examples, and migration traps | +| [Nodes and definitions](./nodes.md) | `NodeDef`, `NodeDefBuilder`, `NodeHandle`, lifecycle hooks, properties, menus, badges, and frontend node definitions | +| [Graphs and groups](./graph.md) | visible/root/subgraph scopes, selection, geometry, editing, undo batches, groups, and graph events | +| [Slots and links](./slots.md) | slot identity, lookup, dynamic slots, connections, link-preserving operations, and resolved sources | +| [Widgets](./widgets.md) | widget values, events, ordering, serialization, DOM mounts, canvas widgets, and custom widget types | +| [Execution and resolution](./execution.md) | queue lifecycle, backend execution, frontend-only nodes, suppliers, preview frames, and prompt-time serialization | +| [Application services](./services.md) | settings, commands, notifications, UI contributions, backend requests, workflow loading, and persistent storage | +| [API reference](./reference.md) | root members, domain interfaces, capability names, errors, and the generated declaration contract | + +## The root object + +`comfy` is installed before custom-node modules are evaluated. Its main domains +are: + +| Member | Responsibility | +| ---------------- | ---------------------------------------------------------------- | +| `comfy.defs` | Read, extend, and define node and widget types. | +| `comfy.graph` | Work with the graph currently shown by the editor. | +| `comfy.queue` | Submit, guard, observe, and interrupt runs. | +| `comfy.settings` | Declare and observe preferences. | +| `comfy.storage` | Store per-user pack documents and presets. | +| `comfy.ui` | Contribute tabs, badges, buttons, dialogs, menus, and prompts. | +| `comfy.commands` | Register and run commands, keybindings, and notifications. | +| `comfy.backend` | Call this pack's declared relative endpoints and receive events. | +| `comfy.models` | List logical names from a host-selected model catalogue. | +| `comfy.workflow` | Open workflow data and apply host text replacements. | + +Root lifecycle and observation methods cover application readiness, workflow +loads, viewport changes, node changes and movement, editor interaction state, +and the currently executing node. + +## Stability and capabilities + +The contract uses `major.minor` versioning: + +- a major change removes or changes existing behavior; +- a minor change only adds behavior; +- every supported major remains available through `comfy.forMajor(major)`. + +Branch on capabilities, not version strings: + +```js +if (comfy.supports('widgets.canvas')) { + installCanvasWidget() +} + +comfy.require('slots.connectedType') +``` + +`supports()` is cheap and never throws. `require()` throws an actionable +`ComfyUnsupportedError`. `comfy.capabilities()` returns the complete capability +set provided by the current host. + +## Contract boundary + +The following are deliberately not part of the published surface: + +- `app`, `ComfyApp`, `window.comfyAPI`, and `/scripts/app.js`; +- `LiteGraph`, `LGraph`, `LGraphNode`, `LGraphCanvas`, and their prototypes; +- Pinia stores and renderer-owned DOM elements; +- mutable link objects and internal node, widget, slot, or group arrays; +- constructors or class inheritance for entity modeling; +- global CSS selectors that restyle host-owned markup. + +If the API lacks a user capability, document the capability rather than +reintroducing an internal escape hatch. If the capability already exists through +a different domain, use that domain. + +## Authoritative contract + +The exact TypeScript surface is checked in as +[comfy-api.d.ts](./comfy-api.d.ts). It is generated from +`src/platform/nodeApi/`: + +```sh +node scripts/node-api/gen_api_dts.mjs > docs/node-api/comfy-api.d.ts +``` + +The checked-in declaration file is the machine-readable compiler contract, and +its freshness test compares it byte-for-byte with the implementation. These +guides explain how and why to use it; [the reference](./reference.md) provides a +navigable map. If prose and generated types ever disagree, treat the generated +types and their implementation tests as authoritative and fix the prose. diff --git a/docs/node-api/comfy-api.d.ts b/docs/node-api/comfy-api.d.ts new file mode 100644 index 00000000000..d82a8365233 --- /dev/null +++ b/docs/node-api/comfy-api.d.ts @@ -0,0 +1,3132 @@ +/** Generated from src/platform/nodeApi; import from '/comfy/api/v2.js'. */ +declare module '*/comfy/api/v2.js' { + export interface BackendHandle { + /** + * @deprecated Trusted host adapter only; unavailable to Secure Nodes packs. + * Use `fetch()` with a pack-relative path, or a named `comfy.*` capability. + */ + url(route: string): string + /** + * @deprecated Trusted host adapter only; unavailable to Secure Nodes packs. + * Use module-relative URLs for bundled files and named APIs for host assets. + */ + assetUrl(route: string): string + /** + * Calls this pack's declared endpoint using the same relative-path contract + * as `fetch()`. It cannot access core routes or another pack's endpoints. + */ + ownFetch(route: string, init?: RequestInit): Promise + /** + * Identifies this frontend connection to a pack's own backend route. + * Undefined until the backend establishes the connection; do not persist it. + */ + sessionId(): string | undefined + /** + * Subscribes to a backend message. The name is whatever the backend emits; + * `detail` is its payload, unparsed. + */ + on(event: string, listener: (detail: unknown) => void): Unsubscribe + /** + * In a Secure Nodes worker, calls only this pack's declared endpoints. + * Use a relative path such as `presets`, with no leading slash, dot segments, + * percent-encoded path or scheme. The host re-encodes query values, assigns + * the namespace from trusted pack identity and owns authentication. + * Shared/core operations use named async `comfy.*` methods with validation + * and capability checks. The trusted in-process adapter retains API-root + * paths for host code; it does not provide sandbox authorization. + */ + fetch(route: string, init?: RequestInit): Promise + } + + export interface BadgeContribution { + /** Namespaced, e.g. `Crystools.monitor`. Registering the same id twice throws. */ + readonly id: string + readonly text: string + readonly label?: string + readonly variant?: 'info' | 'warning' | 'error' + /** An iconify or PrimeIcons class, e.g. `pi-chart-bar`. */ + readonly icon?: string + readonly tooltip?: string + } + + /** What a pack keeps after contributing something to the chrome. */ + export interface ChromeItemHandle { + /** Changes what is shown. Only the fields given are replaced. */ + update(changes: Partial>): void + remove(): void + } + + export interface ButtonContribution { + readonly id: string + readonly icon: string + readonly label?: string + readonly tooltip?: string + /** + * The click. The event is passed because packs branch on modifiers — one + * opens its panel in a sized window on shift-click — and without it that + * behaviour has nothing to read. It is absent when something invokes the + * button directly rather than by click. + */ + run(event?: MouseEvent): void + } + + /** Present on every handle. Never throws, even when the entity is gone. */ + export interface HandleCommon { + readonly isDeleted: boolean + } + + export interface Comfy { + /** + * `major.minor`. Prefer `supports()` over comparing this — a capability + * survives being backported or reordered across minors; a version comparison + * does not. + */ + readonly version: string + /** Breaking-change generation. Incremented only when something is removed. */ + readonly major: number + /** `[major, minor, patch]` of the API this pack was handed. */ + getApiVersion(): readonly [major: number, minor: number, patch: number] + /** + * True if the API is `version` or newer. `version` is written `v2.0.0`; + * throws a `TypeError` for anything else. + */ + isApiVersionAtLeast(version: string): boolean + /** Cheap, never throws. The supported way to branch. */ + supports(capability: string): boolean + /** Asserts a capability, with an actionable error naming it. */ + require(capability: string): void + /** Every capability this host provides. */ + capabilities(): readonly string[] + /** + * Pins to a specific major, so a pack written against one is not moved onto + * the next by a host upgrade. + * + * A pinned major stays available for as long as it is supported — see + * {@link SUPPORTED_MAJORS} — not indefinitely. Withdrawal follows ComfyUI's + * phased deprecation, so a pin buys a stable contract across releases rather + * than a permanent one. + */ + forMajor(major: number): Comfy + + /** + * True when two handles refer to the same entity, whatever major, API + * instance or graph scope produced them. + * + * `===` is only reliable for handles from the same instance, the same major + * AND the same scope. Scope is the one most likely to catch a pack out: a + * node reached through `comfy.graph` while it is on screen and the same node + * reached through `graph.subgraphs()` or through a document-scoped + * `onNodeChanged` come from different handle caches, so they are equal here + * and not equal under `===`. Use this whenever a handle may have come from + * another pack, from an event, or from a graph other than the visible one. + */ + sameEntity(a: unknown, b: unknown): boolean + + /** + * Re-resolves a handle from any major or instance into one of this instance's + * own. Returns `undefined` if it is not a handle, or its entity is gone. + */ + adopt(handle: unknown): NodeHandle | undefined + + readonly graph: GraphHandle + /** Node definitions, and the replacement for `beforeRegisterNodeDef`. */ + readonly defs: DefRegistry + /** Declaring, reading and writing pack settings. */ + readonly settings: SettingsHandle + /** + * Per-user persistent storage for documents the pack's users author — + * templates, presets, saved prompts. Server-side, so it follows the user + * between machines. + */ + readonly storage: StorageHandle + /** Bounded, host-sampled hardware metrics. */ + readonly system: SystemHandle + readonly models: ModelsHandle + /** The sanctioned slice of app chrome — sidebar tabs. */ + readonly ui: UiHandle + /** Commands, their keybindings, and notifications. */ + readonly commands: CommandsHandle + /** Backend URLs and messages, including a pack's own events. */ + readonly backend: BackendHandle + /** Loading a parsed workflow into a new active document. */ + readonly workflow: WorkflowHandle + /** + * The editor is already mid-gesture — dragging a link, resizing a node, + * dragging a widget. A pack running its own pointer gesture must stand down + * while this is true. + */ + isInteracting(): boolean + /** + * Observes nodes being moved, under either renderer. + * + * For building an editing gesture — swap, insert-on-link, shake-to-detach. + * A pack that moves nodes itself will see its own writes, so guard re-entry. + */ + onNodeMoved(listener: (event: NodeMoveEvent) => void): Unsubscribe + /** + * A drag finished; every node it moved. + * + * Where an editing gesture commits — swap the pair, insert into the link + * under the cursor. **Nodes 2.0 only**: the legacy canvas renderer publishes + * no drag lifecycle, so this never fires under it. + */ + onNodeDragEnd(listener: (nodes: readonly NodeHandle[]) => void): Unsubscribe + /** + * The view panned, zoomed or was resized. + * + * For keeping something anchored to a node in sync — ask + * `node.getScreenRect()` again when this fires. Carries no payload: where a + * node is belongs to the node, and the transform belongs to the renderer. + */ + onViewportChanged(listener: () => void): Unsubscribe + /** + * The selected nodes changed. Passes the complete node selection in the + * graph on screen; groups and reroutes are left out. + */ + onSelectionChanged( + listener: (nodes: readonly NodeHandle[]) => void + ): Unsubscribe + /** + * A node changed — its mode, title, colour or shape. + * + * For observing nodes the pack does not own. rgthree's relay polls every + * 500ms and installs a `defineProperty` trap on `mode` because nothing + * reports it; this is that signal. + * + * One stream rather than a subscription per node, deliberately: node + * identity does not survive undo, reload or re-entering a subgraph, so + * anything keyed by the object stops firing silently, and keying by id + * instead never gets collected. Filter by `event.node.id`. + * + * Only fields the host tracks are reported. Position is not among them — it + * changes per frame during a drag and is served by {@link onNodeMoved}. + * + * Reports the graph on screen unless `scope: 'document'` asks for the root + * graph and every subgraph definition as well. A pack that computes from + * other nodes wants `'document'`: a relay in a subgraph the user has + * navigated away from otherwise stops recomputing while still asserting its + * last answer. Each event names the graph it came from, and resolves its node + * there — ids repeat across definitions, so `event.node.id` alone is not a + * key. + */ + onNodeChanged( + listener: (event: NodeChangeEvent) => void, + options?: NodeChangeOptions + ): Unsubscribe + /** + * The application has finished starting: canvas, settings and graph all + * exist, and node definitions are registered. + * + * This is `registerExtension({ setup })`. A pack's module body is the `init` + * half — it runs before definitions register — so anything that needs the + * running app belongs here. Registering after the app has already started is + * fine; the listener is called on the next microtask rather than dropped, + * which is what makes this safe for a pack loaded lazily. + * + * Do not poll for the DOM instead. Several packs shipped a `waitForElements` + * loop to paper over the missing hook, and a poll that outlives its target + * is a leak that only shows up on someone else's machine. + */ + onReady(listener: () => void): Unsubscribe + /** Starting a run, and knowing when one starts. */ + readonly queue: QueueHandle + /** + * The node the backend is executing, or `undefined` between runs. + * + * Packs tracked this from the raw `executing` message to badge the running + * node or follow it with the view. + */ + executingNode(): NodeHandle | undefined + /** Resolves a backend execution id, including a nested subgraph path. */ + executionNode(id: string): NodeHandle | undefined + /** Fires when {@link executingNode} changes, including to nothing. */ + onExecutingNodeChanged( + listener: (node: NodeHandle | undefined) => void + ): Unsubscribe + /** + * A workflow finished loading, and the graph is the new one. Also fires + * for undo, redo and a reload; the document's `id` is unchanged when the + * same document was rebuilt. + */ + onWorkflowLoaded( + listener: (document: DocumentHandle | undefined) => void + ): Unsubscribe + /** A document was opened, in the foreground or in a background tab. */ + onDocumentOpened(listener: (document: DocumentHandle) => void): Unsubscribe + /** A document became the one on screen. */ + onDocumentActivated( + listener: (document: DocumentHandle) => void + ): Unsubscribe + /** A document stopped being the one on screen. It may still be open. */ + onDocumentDeactivated( + listener: (document: DocumentHandle) => void + ): Unsubscribe + /** A document's session ended. The handle now reports `isDeleted`. */ + onDocumentClosed(listener: (document: DocumentHandle) => void): Unsubscribe + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface KeyCombo { + readonly key: string + readonly ctrl?: boolean + readonly alt?: boolean + readonly shift?: boolean + readonly meta?: boolean + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface CommandDef { + /** Namespaced, e.g. `MyPack.doTheThing`. Shared with core and every pack. */ + readonly id: string + /** + * A function when the label depends on state — a toggle that reads "Follow + * execution" and then "Stop following execution". It is read each time the + * label is shown, so it must return quickly. + */ + readonly label: string | (() => string) + readonly run: () => void | Promise + /** + * Bound as a default, so a user's own binding still wins. A key another + * command already holds by default stays with that command. + */ + readonly keybinding?: KeyCombo + /** + * Where the keybinding applies. Defaults to anywhere in the application. + * + * `'canvas'` limits it to the graph, so it will not fire while the user is + * typing in a node's text widget or any other field. The host already + * withholds combos a text input owns — every bare arrow, Ctrl+Left/Right, + * Ctrl+A/C/V/X/Z — but a pack binding something it does not, say Ctrl+Up, + * would otherwise fire mid-sentence. + */ + readonly scope?: 'canvas' + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface NotifyDef { + readonly severity?: 'success' | 'info' | 'warn' | 'error' + readonly summary: string + readonly detail?: string + /** Milliseconds. Omit for the host's default. */ + readonly life?: number + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface PlaySoundDef { + /** + * A browser-readable audio asset. Sandboxed hosts restrict this to pack files. + */ + readonly src: string + /** Range 0–1. Defaults to 1. */ + readonly volume?: number + } + + export interface CommandsHandle { + register(def: CommandDef): void + notify(def: NotifyDef): void + playSound(def: PlaySoundDef): Promise + /** + * Runs a command the host or another pack registered, by id. + * + * Packs reached into internals to do what a command already does — opening + * the mask editor was `ComfyApp.copyToClipspace` plus `clipspace_return_node` + * plus invoking `Comfy.MaskEditor.OpenMaskEditor` by hand. Commands are the + * sanctioned action layer, so a pack can ask for the behaviour without the + * host having to publish the machinery behind it. + * + * Rejects if no such command is registered — a pack naming a command that + * has been renamed should hear about it rather than silently do nothing. + */ + run(id: string): Promise + /** Whether a command exists, for a pack that offers an entry conditionally. */ + has(id: string): boolean + } + + /** + * The read view of a node definition. Frozen and inert, like every read here. + */ + export interface NodeDef { + readonly type: string + readonly title: string + readonly category: string + readonly description: string + readonly inputs: readonly Readonly<{ + name: string + type: string + /** The translated caption core renders for this input, when it differs. */ + localizedName?: string + /** The declared choices for a COMBO input, in backend order. */ + values?: readonly (string | number)[] + /** + * The input's declaration dict, verbatim from the backend. + * + * Same passthrough reasoning as `ExecutionResult.raw`: a pack declares its + * own keys on its own Python input spec and reads them back here to drive + * frontend behaviour, so discarding unrecognised keys breaks the pack + * against its own data. Carries `default`, `min`, `max` and the like too. + */ + options: Readonly> + }>[] + readonly outputs: readonly Readonly<{ name: string; type: string }>[] + readonly isOutputNode: boolean + /** + * The node's `hidden` input declarations, verbatim. + * + * Deliberately not merged into {@link inputs}: a hidden input is not a slot, + * and listing it as one would put a connectable input on the node for + * something the server fills in. + * + * Packs ship their own data here and read it back — easy-use and + * tinyterraNodes both carry an XY-plot axis catalogue as + * `input.hidden.plot_dict[0]`, on their own key, from their own Python spec. + * That is the same passthrough reasoning `inputs[].options` already rests on, + * and dropping it broke both packs against their own data. + * + * These are declarations, not values. `PROMPT`, `UNIQUE_ID` and + * `EXTRA_PNGINFO` appear here as the type markers the node asked for; the + * server substitutes the real thing at execution time and it never passes + * through here. + */ + readonly hidden: Readonly> + /** Which pack supplied it, when the backend reports one. */ + readonly source: string | undefined + } + + /** + * Node output as it arrives from the backend. + * + * `raw` carries everything else verbatim — ADR-NODE-OUTPUTS-0007's passthrough schema + * guarantees custom output keys survive, so a pack reading a bespoke key keeps + * working. + */ + export interface ExecutionResult { + readonly images: readonly Readonly>[] + readonly text: readonly string[] + readonly raw: Readonly> + } + + /** + * A preview frame the backend produced while this node was running. + * + * Per node rather than per channel, deliberately. Packs currently subscribe to + * `b_preview_with_metadata` *and* `b_preview`, track the executing node id in a + * module global to correlate the second one, and probe + * `serverSupportsFeature('supports_preview_metadata')` to decide which to + * trust — all to answer "is this frame mine?". Answering it once here removes + * the global, and with it the mis-attribution when two nodes preview at once. + */ + export interface PreviewFrame { + readonly blob: Blob + /** Object URL for the blob, revoked when the next frame arrives. */ + readonly url: string + } + + export interface ConnectionChangeEvent { + readonly side: 'input' | 'output' + readonly index: number + /** The changed slot's name, or `undefined` if the slot is already gone. */ + readonly name?: string + readonly connected: boolean + /** + * The node at the other end, or `undefined` on a disconnect. + * + * Packs read `link_info.origin_id` to decide what the new neighbour means — + * retype a slot to match it, adopt its label. Knowing only that *something* + * connected forced a re-walk of the whole graph to find out what. + */ + readonly peerNodeId?: string + /** The slot index at the other end, or `undefined` on a disconnect. */ + readonly peerIndex?: number + } + + /** + * The only change a node extension may make to one queued API prompt. + * + * Inputs are named from that node type's own backend declaration. The saved + * workflow is untouched; the prompt builder removes these names only from the + * executable payload it is assembling now. + */ + export interface PromptInputProjection { + readonly omitInputs: readonly string[] + } + + export type PromptInputProjector = ( + node: NodeHandle + ) => PromptInputProjection | Promise + + export interface NodeDefBuilder { + /** Current state of the definition, after any earlier extensions ran. */ + readonly def: NodeDef + + setTitle(title: string): void + setCategory(category: string): void + /** + * Declares that this node type never reaches the backend. + * + * `defs.define` takes `execution: 'frontend'` for a type the pack owns, but + * packs also mark *backend-registered* types frontend-only — a tools or + * control node that exists to drive other nodes and must not appear in the + * prompt. Without this they reach for `node.isVirtualNode`, and dropping that + * line puts a new node into `graphToPrompt`, which is a wire-format break. + * + * Supply `resolve` when the node carries a value through to something else; + * omit it and the node is simply left out. See `resolution.ts` — `resolve` is + * pure over a read-only view and must not mutate the graph. + */ + setExecution(execution: 'backend' | 'frontend', resolve?: Resolver): void + /** + * Declares what this node feeds into *other* nodes' unconnected inputs. + * + * The counterpart of `setExecution`'s `resolve`, which answers only "what + * feeds my own outputs" and is never called for a node with none. Broadcast + * packs are the reverse: they name inputs on nodes that are not themselves, + * and discover those edges rather than declaring them. + * + * Available here and not only on `defs.define` because the types that + * broadcast are registered by the pack's Python, and `defs.define` refuses a + * type that already exists — which left `supply` unreachable for every pack + * that actually needed it. + * + * Not gated on `setExecution('frontend')`: feeding somebody else and being + * skipped by the prompt builder are separate questions, and a node may + * legitimately both execute and broadcast. + */ + setSupply(supply: Supplier): void + addWidget(def: WidgetDef): void + hideWidget(name: string): void + + // Behaviour hooks, ordered by measured usage across the 1,265 packs. + /** + * Fires once the node exists *and is addressable* — after it joins a graph. + * + * Deliberately not litegraph's `onNodeCreated`, which runs inside + * `createNode()` before the node has an id, a graph, or store registration. + * A handle is id-backed, so at that moment there is nothing to hand back, and + * widget writes would land on an unregistered node and be lost on insert. + */ + onCreated( + callback: (node: NodeHandle, event: NodeCreatedEvent) => void + ): void // 943 packs + onExecuted( + callback: (node: NodeHandle, result: ExecutionResult) => void + ): void // 497 packs + onConfigured( + callback: (node: NodeHandle, data: Record) => void + ): void // 429 packs + onConnectionsChanged( + callback: (node: NodeHandle, event: ConnectionChangeEvent) => void + ): void // 223 packs + onRemoved(callback: (node: NodeHandle) => void): void // 158 packs + /** + * The node was resized, by the user or by a layout pass. + * + * Packs hung a `ResizeObserver` on their mounted element to notice this, + * which fires for the element rather than the node and misses a resize that + * does not change the element. + */ + onResized(callback: (node: NodeHandle, size: Size) => void): void + /** + * The pointer entered or left the node. + * + * Packs read `canvas.node_over` or set `node.mouseOver` to rebuild a list + * the moment the pointer arrives, or to decide which node a tooltip belongs + * to. Both are canvas internals, and the canvas is what Nodes 2.0 replaces. + */ + onHover(callback: (node: NodeHandle, hovering: boolean) => void): void + /** + * The node was double-clicked. + * + * Deliberately carries no coordinates. Hit-testing a pointer against + * node-local geometry is a pack drawing its own front end; the published + * answer is `widgets.mount` and ordinary DOM events on the element you own. + */ + onDoubleClick(callback: (node: NodeHandle) => void): void + /** + * Whether this node can accept the current browser drag. + * + * The event is the browser's data-transfer surface, not a renderer object. + * Returning `true` makes both node renderers present and route the drop. + */ + onDragOver( + callback: (node: NodeHandle, event: DragEvent) => boolean | void + ): void + /** Handles a drop the node accepted. Returning `true` claims it. */ + onDrop( + callback: ( + node: NodeHandle, + event: DragEvent + ) => boolean | void | Promise + ): void + /** + * A property the user edited in the node's properties panel. + * + * Packs used `onPropertyChanged` to keep a hand-entered value sane — rgthree + * clamps a seed's `randomMax` as it is typed. litegraph's own callback can + * only veto, reverting to the previous value, which throws the user's input + * away rather than correcting it. `setValue` replaces it instead, and writes + * without going back through `setProperty`, so a clamp cannot recurse. + */ + onPropertyChanged( + callback: (node: NodeHandle, event: PropertyChangeEvent) => void + ): void + /** Preview frames for this node, already correlated. */ + onPreview(callback: (node: NodeHandle, frame: PreviewFrame) => void): void + /** + * Contributes the pack's own state to the saved node. + * + * The returned object is merged into the serialized node, and comes back + * through `onConfigured`. Only keys the pack owns: core fields are not + * writable from here, because a pack must not be able to change what the + * workflow means. + */ + onSerialize(callback: (node: NodeHandle) => Record): void + /** + * Omits declared inputs from this node in the API prompt being built. + * + * This is not a prompt rewrite: the callback receives no prompt or input + * values, may not name another node, and cannot inject replacements. It is + * awaited on the prompt path so a sandboxed extension answers from its + * current read-only node snapshot rather than a stale cached value. + */ + onPromptSerialize(callback: PromptInputProjector): void + /** + * Vetoes or permits an incoming connection *before* it is wired. + * + * Distinct from `onConnectionsChanged`, which fires after the fact — packs + * use the pre-hook to refuse an incompatible link or relabel a slot while + * the type is still known. Returning `false` refuses. + */ + onBeforeConnect( + callback: (node: NodeHandle, event: BeforeConnectEvent) => boolean | void + ): void + /** + * The user dropped a link on a node's body and the host found no single slot + * that fits. Wire it yourself and return `true`; return nothing to let the + * host report the drop unplaceable. + * + * For a node whose one slot carries a bundle of values — a context, a pipe — + * and which wants to unpack it into several of the peer's slots at once. Both + * ends of the drag are asked, the one the user aimed at first, because the + * node with the knowledge is the drop target in one direction and the drag's + * origin in the other. + * + * The published alternative to replacing `connectByType` on the prototype, + * which is how packs did this: that changes link routing for every node in + * the document, so one pack's convenience became every other pack's + * behaviour. + */ + onUnplacedLink( + callback: (node: NodeHandle, event: UnplacedLinkEvent) => boolean | void + ): void + /** Adds an entry to this node type's context menu. */ + addMenuItem(item: NodeMenuItem): void + } + + export interface NodeCreatedEvent { + /** + * The node arrived carrying saved state — pasted, duplicated, or loaded from + * a workflow — rather than being made fresh. + * + * Read as "was `configure` called on it before it joined the graph", which is + * what actually distinguishes the cases. Packs overrode `clone()` to reset + * state a copy should not inherit — a duplicated node keeping the dynamic + * slots that were fed by the original's upstream, a duplicated reroute born + * hard-typed and refusing every other type — and `clone()` runs before the + * node has an id, so there is nothing to hand a pack there. + */ + readonly restored: boolean + /** + * The whole graph was being loaded, so {@link restored} means "came from the + * saved file" rather than "came from the clipboard". + * + * The distinction is the point: a pasted node should drop slots it cannot + * still be fed through, and a loaded one must keep every one of them or the + * workflow opens wrong. + */ + readonly loading: boolean + } + + export interface UnplacedLinkEvent { + /** Which of this node's slots the link would land on. */ + readonly side: 'input' | 'output' + /** The node at the other end of the drag. */ + readonly peerNodeId: string + /** The slot on the peer the drag started from. */ + readonly peerIndex: number + readonly type: string + /** + * The user held the modifier that means "overwrite what is already wired". + * + * Published because packs read a global keyboard service of their own to get + * it, and which modifier means this is the host's to decide. + */ + readonly replaceExisting: boolean + } + + export interface BeforeConnectEvent { + readonly side: 'input' | 'output' + readonly index: number + /** The node at the other end, when one is known. */ + readonly peerNodeId: string | undefined + /** The slot at the other end, when one is known. */ + readonly peerIndex: number | undefined + readonly peerType: string | undefined + } + + /** One entry inside a menu item's submenu. */ + export interface NodeSubMenuItem { + readonly label: string + run(node: NodeHandle): void + } + + /** + * One entry of ComfyUI's node palette: the title bar, the body, and the shade + * a group of that colour is filled with. + */ + export interface NodeColor { + readonly color: string + readonly bgColor: string + readonly groupColor: string + } + + export interface NodeMenuItem { + /** + * A function when the text depends on the node — packs label entries with + * the current state ("Unmute 3 nodes"), which a string fixed at + * registration cannot express. + */ + readonly label: string | ((node: NodeHandle) => string) + /** + * Shown only when this returns true. Without it a pack that wants an entry + * to appear conditionally has to either show it always or not at all — + * efficiency-nodes hides its seed submenu when the feature is off, and + * flattening that to a permanent entry is a worse lie than omitting it. + */ + when?(node: NodeHandle): boolean + /** Omit when the item only opens a submenu. */ + run?(node: NodeHandle): void + /** + * Turns the entry into a submenu. One level deep, deliberately: every + * measured pack uses exactly one, and nesting further is a menu design + * problem rather than an API one. + * + * A function when the children depend on the node's current state, which is + * the common case rather than the exotic one: efficiency-nodes' LoRA Stacker + * declares fifty `lora_name_N` widgets and lists only the two or three a + * user has filled. A fixed array would put fifty rows in that menu, which is + * a different menu, so the alternative to this was omitting the feature. + */ + readonly items?: + | readonly NodeSubMenuItem[] + | ((node: NodeHandle) => readonly NodeSubMenuItem[]) + /** + * Sort position among this node's pack-added entries. Lower first; entries + * without one keep registration order, which is module-load order and so + * depends on import sequence rather than intent. + */ + readonly order?: number + } + + /** + * Which definitions an extension applies to. + * + * Indexed rather than run-and-return: this predicate is almost always the guard + * clause the pack already had at the top of its hook. + */ + export type DefSelector = + | string + | readonly string[] + | RegExp + /** + * A predicate over the definition, for a guard the other forms cannot + * express — "any node taking a VAE input", which is a shape rather than a + * name. + * + * Deliberately last, and deliberately discouraged. The declarative forms + * exist because a name check can be indexed, while a predicate has to run for + * every registered type; with thousands of types that is the boot cost this + * API set out to remove. Use it only when the guard genuinely reads a def's + * inputs or outputs. + */ + | ((def: NodeDef) => boolean) + /** + * A `RegExp` category covers the prefix filter 53 packs open their hook with + * (`nodeData.category.startsWith('KJNodes')` → `{ category: /^KJNodes/ }`). + */ + | { readonly category: string | RegExp } + + /** + * A node type the pack owns, declared rather than subclassed. + * + * 86 packs (18.2% of installs) do this today with `extends LGraphNode` + + * `LiteGraph.registerNodeType`, which is OOP entity modelling — the thing ADR + * 0008 rules out. Here the definition is plain data; the class behind it is an + * internal detail of this layer, never the pack's. + */ + export interface NodeDefinition { + readonly type: string + readonly title?: string + readonly category?: string + readonly description?: string + readonly inputs?: readonly { name: string; type: string }[] + readonly outputs?: readonly { + name: string + type: string + shape?: SlotShape + }[] + readonly widgets?: readonly WidgetDef[] + /** + * `'frontend'` nodes never reach the backend: they are resolved away at + * prompt time by the resolution system, or simply omitted. + */ + readonly execution?: 'backend' | 'frontend' + /** + * Answers what each output resolves to, purely, over a read-only view. + * See `resolution.ts` — this replaces `applyToGraph`, which mutated the + * live graph mid-serialize. + */ + readonly resolve?: Resolver + /** + * What this node feeds into *other* nodes' unconnected inputs. + * + * The broadcast direction: `resolve` cannot express it, because the nodes + * being fed are not this one and the edges are discovered rather than + * declared. + */ + readonly supply?: Supplier + + onCreated?(node: NodeHandle, event: NodeCreatedEvent): void + onExecuted?(node: NodeHandle, result: ExecutionResult): void + onConfigured?(node: NodeHandle, data: Record): void + onConnectionsChanged?(node: NodeHandle, event: ConnectionChangeEvent): void + onPropertyChanged?(node: NodeHandle, event: PropertyChangeEvent): void + onDragOver?(node: NodeHandle, event: DragEvent): boolean | void + onDrop?( + node: NodeHandle, + event: DragEvent + ): boolean | void | Promise + onRemoved?(node: NodeHandle): void + onSerialize?(node: NodeHandle): Record + onPromptSerialize?: PromptInputProjector + } + + export interface DefRegistry { + /** + * Declares how an input *type* is presented — the replacement for + * `getCustomWidgets`. + * + * Not decoration: the host decides widget-vs-socket purely by whether a type + * is registered, so an unregistered one turns the input into a socket and + * drops its value from `widgets_values`. See `widgetTypes.ts`. + */ + defineWidgetType(type: string, def: WidgetTypeDef): Unsubscribe + /** + * Registers a node type the pack owns. Returns a handle that unregisters + * it — which `LiteGraph.registerNodeType` never offered. + */ + define(definition: NodeDefinition): Unsubscribe + get(type: string): NodeDef | undefined + all(): readonly NodeDef[] + has(type: string): boolean + extend( + selector: DefSelector, + apply: (builder: NodeDefBuilder) => void + ): Unsubscribe + /** + * Asks the host to reload node definitions from the backend. + * + * Combo inputs whose values the backend supplies — model lists, LoRA names, + * sampler names — are captured when definitions load, so a pack that adds a + * file server-side leaves every open picker showing the old list. This is + * `app.refreshComboInNodes()`, which packs called after saving a model + * preview or writing a new file. + * + * Refreshing is not free: it refetches every definition. Call it after a + * change the user made, not on a timer. + */ + /** + * The colour links and slots of a type are drawn in. + * + * A pack matching the theme in its own DOM — a legend, a chip, a preview — + * read `LGraphCanvas.link_type_colors` for this. Reading a design token to + * match is the opposite of drawing your own front end, so it is published; + * the table itself is not. + */ + typeColor(type: string): string + /** + * The colours behind a name in ComfyUI's node palette — `red`, `pale_blue` — + * or `undefined` for a name it does not define. + * + * Same reasoning as {@link typeColor}, and the same limit: the resolver is + * published, the table is not. What makes this a design token rather than a + * renderer internal is that the names are the user's own vocabulary. They + * pick "green" from a menu; nothing records the word, only the hex it stood + * for. So a pack offering "mute every red group" cannot match what the user + * chose without being told which hex "red" meant, and two packs did it by + * reading `LGraphCanvas.node_colors` directly. + * + * Colours move with the palette, names do not. Resolve on use; do not cache + * the result and do not persist it in a workflow. + */ + nodeColor(name: string): NodeColor | undefined + /** + * Tests an output type against an input type using the host's connection + * rules, including wildcards and comma-delimited unions. + */ + isTypeCompatible(outputType: string, inputType: string): boolean + /** + * Declares the colour for a data type this pack introduces. + * + * Packs shipping their own types — `PIPE_LINE`, `LORA_STACK`, `XYPLOT` — + * wrote straight into `LGraphCanvas.link_type_colors` so their links were + * not all grey. + * + * Refuses a type the host already colours. That write is global: one pack + * recolouring `IMAGE` restyles every graph for every other pack and the + * user has no way to see who did it. Colouring a type you brought is + * additive; colouring one you did not is not yours to decide. + */ + setTypeColor(type: string, color: string): Unsubscribe + refresh(): Promise + /** + * Node definitions were reloaded — by this pack, another pack, or the user. + * + * The listening half of `refresh()`, and what the `refreshComboInNodes` + * extension hook gave packs. A pack holding its own cached copy of a combo's + * values — a model list it filters, a picker it built — needs to rebuild it + * when the list changes underneath, and the pack that caused the change is + * usually not this one. + */ + onRefreshed(listener: () => void): Unsubscribe + } + + export interface PropertyChangeEvent { + readonly name: string + readonly value: unknown + readonly previous: unknown + /** Replaces what is stored. Last writer wins if several packs respond. */ + setValue(value: unknown): void + /** Discards the edit, restoring `previous`. */ + reject(): void + } + + /** One open workflow: an editing session, not the file behind it. */ + export interface DocumentHandle extends HandleCommon { + /** + * Identity of this editing session. The same for as long as the document + * is open, through undo, redo and tab switches, and never reused. It means + * nothing after a page reload, so do not save it. + */ + readonly id: string + /** Display name, without directory or extension. */ + readonly name: string | undefined + /** Storage path. Changes on rename; undefined once the session has ended. */ + readonly path: string | undefined + /** Whether there are edits the user has not saved. */ + readonly isModified: boolean + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface NodeInit { + title?: string + position?: { x: number; y: number } + } + + /** + * `'visible'` is the graph on screen. `'root-and-subgraphs'` is the root graph + * and every subgraph definition, each counted once however often it is placed. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export type NodeQueryScope = 'visible' | 'root-and-subgraphs' + + /** + * Every field narrows, and they combine as AND. Omitting all returns the scope. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export interface NodeQuery { + /** Defaults to `'visible'`. */ + readonly scope?: NodeQueryScope + /** An exact type, any of several, or a pattern. */ + readonly type?: string | RegExp | readonly string[] + /** Keep only nodes in the graph on screen, wherever they are scrolled to. */ + readonly rendered?: boolean + /** Keep only nodes the group currently contains. */ + readonly group?: GroupHandle + } + + export interface GraphHandle { + readonly id: string + node(id: string): NodeHandle | undefined + nodes(): readonly NodeHandle[] + nodesOfType(type: string): readonly NodeHandle[] + /** + * One flat query over the nodes in a scope. Under `'root-and-subgraphs'` + * each handle comes from its owning graph, so compare with `sameEntity()`. + */ + queryNodes(query?: NodeQuery): readonly NodeHandle[] + add(type: string, init?: NodeInit): NodeHandle + remove(id: string): boolean + links(): readonly LinkInfo[] + /** + * The supply edges prompt execution would use in this graph right now. + * + * Re-runs the registered pure suppliers and the host's priority arbitration, + * returning graph-local ids suitable for {@link OutputSlotHandle.connectTo}. + * Exact priority ties are absent, just as they are from the prompt. The + * frozen snapshot never mutates the graph. + */ + resolvedSupplies(): readonly ResolvedSupply[] + /** + * The nodes the user currently has selected. + * + * 15 packs read `canvas.selected_nodes` or `selectedItems` for this — a + * canvas internal, and the canvas is exactly what Nodes 2.0 replaces. + * Selection is a property of the document, so it is asked of the graph. + */ + selection(): readonly NodeHandle[] + /** + * Replaces the selection with these nodes. An empty list clears it. + * + * A node a pack just created is the usual case — `LGraphCanvas.add`'s + * `options.select` put it straight under the user's cursor, and without this + * the node appears but the user has to find and click it. + * + * `add: true` extends the selection instead of replacing it. + */ + select(nodes: readonly NodeHandle[], options?: { add?: boolean }): void + /** + * Pans the view so a node sits in the middle of it. + * + * Packs wrote `canvas.ds.offset` themselves to do this, which bakes in the + * renderer's transform and the device pixel ratio. Does not change zoom. + */ + centerOn(node: NodeHandle): void + /** + * The groups on the canvas, in draw order. + * + * Packs read `graph._groups` to build a group muter, a group runner, or a + * navigator. A group is a rectangle plus a title: which nodes it holds is + * derived from what it overlaps, which is why `nodes()` is a method and not + * a stored list. + */ + groups(): readonly GroupHandle[] + /** + * Scales the view. 1 is unzoomed. + * + * Packs saved a zoom level alongside a node to restore a view; without this + * a bookmark could pan but the number it stored was inert. Clamped to what + * the canvas allows, so a stored extreme cannot strand the user. + */ + setZoom(scale: number): void + /** + * Pans the view by a distance in CSS pixels, the same at every zoom. + * Positive x and y move the graph content right and down. + */ + panBy(delta: Point): void + /** + * Where the pointer is, in graph space — the coordinates {@link nodeAt} and + * {@link NodeHandle.setPosition} use. + * + * A pack adding a node from a menu put it under the cursor. Without this the + * node lands at the graph origin, which on any panned view is off screen. + * + * `undefined` when there is no canvas to measure against. + */ + pointerPosition(): Point | undefined + /** + * The document's root graph, even while the user is viewing a subgraph. + * Undefined before a document exists. + */ + root(): GraphScopeHandle | undefined + /** + * The subgraph definitions in the document, each scoped to its own nodes. + * + * `nodes()` and `node()` address the graph on screen only, so a pack that + * must reach every node — refreshing its own nodes after a run, walking a + * chain — misses anything nested. + * + * Access is *through* the subgraph rather than a flattened list. Ids are + * allocated from the root graph's counter, so they do not collide among + * nodes created in one session — but a subgraph loaded from a file brings + * its authored ids, and `configure` raises that counter without renumbering + * anything. Two independently authored subgraphs can therefore carry the + * same id. Resolving inside the owning graph is correct either way, and does + * not rest on an invariant litegraph does not promise. + * + * These are definitions, not instances. A subgraph placed three times has + * one entry, and its nodes appear once — which is what a pack acting on + * "each of my nodes" wants. + */ + subgraphs(): readonly GraphScopeHandle[] + /** + * Runs several mutations as one undo step. + * + * Without it, a pack that adds three nodes and wires them leaves the user + * pressing undo four times to get back. `graph.beforeChange()` / + * `afterChange()` did this by counting nesting depth. + * + * A scope rather than a pair of calls: the counter only captures when it + * returns to zero, so one throw between a manual `before` and `after` stops + * undo capturing anything at all, for the rest of the session, with nothing + * to show why. The scope closes on the way out either way. + * + * Synchronous on purpose. Holding the group open across an `await` would + * fold whatever the user did while waiting into the pack's undo step. + */ + batch(mutations: () => T): T + /** + * The topmost node at a point in graph space, if any. + * + * Packs building a gesture were walking every node and re-deriving its + * rectangle from renderer constants. The graph already knows, and its answer + * respects z-order, collapsed nodes and the active renderer's layout. + * + * Answers against the *rendered* layout, which is the only sensible reading + * of "what is under this point" — and is why it is not refreshed per call: a + * gesture asks this on every pointer move, and remeasuring every node each + * time would be the expensive mistake. Before the first frame it finds + * nothing. + */ + nodeAt(point: { x: number; y: number }): NodeHandle | undefined + /** + * A copy of a node, carrying its widget values and properties, added to the + * graph without links. + * + * `add(type)` only makes a fresh node of a type, so a pack duplicating a + * configured node — a prompt box the user has filled in — had no way to keep + * what it contained. Links are deliberately not copied: a duplicate wired + * into the same places is a different operation, and the caller can connect + * it themselves. + * + * `undefined` if the node is gone, or if its type is not registered — the + * copy is built through the registry, so there is nothing to build from. + * Widget values carry over only for a type that serializes them, which every + * backend-registered type does. + */ + duplicate( + id: string, + position?: { x: number; y: number } + ): NodeHandle | undefined + /** + * Rebuilds a node, optionally as another type, keeping what the user set and + * every link that still fits. Replacing with the same type repairs a node + * whose registered definition changed without discarding its state. + * `undefined` if the node is gone; throws if the type is not registered. + * + * This is a real feature four packs ship — "Convert to Context Big", "Swap to + * KSampler (Efficient)" — and all four hand-rolled it out of `graph.links`, + * `getNodeById` and `LiteGraph.createNode`, which is most of what this + * migration exists to delete. All four also got it wrong: one drops every + * widget value and hardcodes "slot 0 only", the other recurses through + * requestAnimationFrame forever on an inverted comparison and leaves a + * separate undo step for the add, each connection, and the remove. + * + * Position, custom title, colour, mode, declared properties and widget values + * carry over by name. Size is the larger of what the user set and what the new + * type needs, so a node that grew more slots is not clipped. Links are re-made + * by slot name, falling back to the same index; type checking is the ordinary + * connection rule, so a link that no longer fits is dropped and warned about + * rather than forced. The whole swap is one undo step. + */ + replace(id: string, type: string): NodeHandle | undefined + /** + * Changes when the graph does: nodes added, removed or reconfigured, links + * connected or disconnected, slots and subgraph inputs/outputs altered, and + * the node flags a reader can see — collapsed, pinned, advanced. + * + * Hold one and compare it later to learn whether anything moved since. That + * is the whole contract: an opaque token, not a count. Do not subtract two + * of them, do not expect it to start anywhere in particular, and do not + * expect consecutive changes to differ by one. Coalesced edits are free to + * advance it once, and `batch()` exists precisely so they can. + * + * A widget value committed by the user or through + * `WidgetHandle.setValue()` advances it through the same host protocol. Data + * a pack keeps outside graph and widget state does not; a canvas widget + * holding such data has `redraw()`. + */ + readonly version: number + /** Diagnostics: live handle-cache slots across all kinds. */ + } + + /** + * A subgraph definition, scoped to its own contents. + * + * Deliberately narrower than {@link GraphHandle}: adding, selecting, centring + * and zooming all address what the user is looking at, and a subgraph + * definition is not that. This is for reading and reaching nodes. + */ + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface GraphScopeHandle { + /** Stable across every instance of this subgraph. */ + readonly id: string + readonly name: string | undefined + nodes(): readonly NodeHandle[] + node(nodeId: string): NodeHandle | undefined + /** + * The groups drawn inside this subgraph. + * + * A group muter or runner that skipped these reported nothing for a + * subgraph's contents while appearing to work. + */ + groups(): readonly GroupHandle[] + /** The supply edges prompt execution would use inside this graph. */ + resolvedSupplies(): readonly ResolvedSupply[] + } + + export interface GroupHandle extends HandleCommon { + readonly id: string + getTitle(): string + setTitle(title: string): void + /** Colour as the renderer holds it, or undefined for the default. */ + getColor(): string | undefined + setColor(color: string): void + /** + * The nodes the group currently contains, recomputed on each call. + * + * Packs muted or queued "the group", which always meant its nodes. Do not + * cache the result: a drag changes it with no event. + */ + nodes(): readonly NodeHandle[] + /** The group's rectangle in graph space, title bar included. */ + getBounds(): Bounds + /** Pans the view so this group is in the middle of it. Zoom is unchanged. */ + centerOn(): void + } + + export interface NodeMoveEvent { + readonly node: NodeHandle + readonly position: { readonly x: number; readonly y: number } + } + + export interface ModelsHandle { + /** + * Lists at most 50,000 distinct logical model names from a known folder. + * The host chooses the route; filesystem locations and path indexes are + * excluded. Unknown folders, invalid replies and HTTP failures reject. + */ + list(folder: ModelFolder): Promise + } + + export type ModelFolder = + | 'checkpoints' + | 'clip' + | 'clip_vision' + | 'controlnet' + | 'diffusion_models' + | 'embeddings' + | 'loras' + | 'text_encoders' + | 'unet' + | 'upscale_models' + | 'vae' + | 'ipadapter' + | 'vae_approx' + + export type TrackedProperty = + | 'title' + | 'mode' + | 'color' + | 'bgcolor' + | 'shape' + | 'showAdvanced' + + /** + * Which graphs a listener hears from. + * + * `'visible'` is the default and the graph on screen, following the user into + * and out of subgraphs — what a pack decorating what the user is looking at + * wants. + * + * `'document'` is the root graph and every subgraph definition. A pack that + * *computes* from other nodes needs it: rgthree's relay derives a group's mute + * state from its inputs, and inside a subgraph the user had navigated away from + * it stopped recomputing while still asserting its last answer — so a group + * stayed muted against its inputs, intermittently, and healed on navigation. + */ + export type NodeChangeScope = 'visible' | 'document' + + export interface NodeChangeOptions { + scope?: NodeChangeScope + } + + export interface NodeChangeEvent { + /** The node that changed. It may belong to another pack, or to none. */ + readonly node: NodeHandle + /** + * The graph the change happened in — the root graph's id, or a subgraph + * definition's. Node ids are unique only within a graph, so a pack keeping + * its own records under `'document'` must key on both. + */ + readonly graphId: string + readonly property: TrackedProperty + readonly from: unknown + readonly to: unknown + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type NodeMode = + | 'always' + | 'never' + | 'bypass' + | 'on-event' + | 'on-trigger' + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type NodeShape = 'default' | 'box' | 'round' | 'circle' | 'card' + + export interface BadgeDef { + readonly text: string + /** Text colour. Defaults to core's badge foreground. */ + readonly color?: string + /** Background colour. Defaults to core's badge background. */ + readonly bgColor?: string + /** + * Makes the badge clickable. + * + * Two conversions declined to turn a button into a badge because a badge + * that looks pressable and does nothing is worse than the thing it replaced. + */ + onClick?(): void + } + + export interface Point { + readonly x: number + readonly y: number + } + + export interface Size { + readonly width: number + readonly height: number + } + + /** A rectangle in graph space. */ + export interface Bounds { + readonly x: number + readonly y: number + readonly width: number + readonly height: number + } + + export interface NodeSnapshot { + readonly id: string + readonly type: string + readonly title: string + readonly mode: NodeMode + readonly collapsed: boolean + readonly pinned: boolean + readonly color: string | undefined + readonly bgColor: string | undefined + readonly shape: NodeShape + readonly position: Point + readonly size: Size + } + + /** + * Shapes follow `docs/node-api/reference.md`, the published contract: + * accessor methods rather than properties, so a read can be a store query and + * a write can dispatch a command. + */ + export interface SizeConstraints { + minWidth?: number + minHeight?: number + maxWidth?: number + maxHeight?: number + /** Grow to fit content rather than holding a fixed height. */ + autoHeight?: boolean + } + + export interface NodeHandle extends HandleCommon { + readonly id: string + readonly type: string + readonly comfyClass: string + + getTitle(): string + setTitle(title: string): void + getMode(): NodeMode + setMode(mode: NodeMode): void + isCollapsed(): boolean + setCollapsed(collapsed: boolean): void + isPinned(): boolean + setPinned(pinned: boolean): void + /** + * Whether the user can resize the node. Not saved, and its size is + * unchanged. A pinned node stays fixed. + */ + setResizable(resizable: boolean): void + getColor(): string | undefined + setColor(color: string | undefined): void + getBgColor(): string | undefined + setBgColor(color: string | undefined): void + getShape(): NodeShape + setShape(shape: NodeShape): void + getProperty(key: string): unknown + getProperties(): Readonly> + setProperty(key: string, value: WidgetValue): void + /** + * Whether this node emits `widgets_values` when the workflow is serialized. + * + * Writable because packs vary it per node type, and the value is part of the + * wire format — a conversion that could not set it would change what the + * saved workflow contains. + */ + isSerializingWidgets(): boolean + setSerializeWidgets(serialize: boolean): void + + getPosition(): Point + setPosition(pos: Point): void + getSize(): Size + /** Changes size through the host's resize protocol, including `onResized`. */ + setSize(size: Size): void + /** + * The node's rectangle in graph space, title bar included. + * + * `getPosition()` is the body's top-left, so packs building a gesture were + * reconstructing this by subtracting a title height read off the renderer — + * which is only right for the default layout, and wrong for a collapsed node + * or under a different renderer. Ask the renderer instead of re-deriving it. + */ + getBounds(): Bounds + /** + * Where a slot sits, in graph space. + * + * The renderer's own answer, so it stays correct for collapsed nodes, + * widget-backed inputs and layouts that are not the default vertical stack — + * all cases the `(index + 0.7) * slotHeight` reconstruction gets wrong. + * + * `undefined` if there is no slot at that index. + */ + getSlotPosition(side: 'input' | 'output', index: number): Point | undefined + /** + * Where the node currently sits on screen, in client coordinates. + * + * For anchoring a floating panel to a node. Packs did this by reading the + * viewport's pan and zoom and doing the arithmetic themselves, which is both + * the renderer's business and wrong the moment the transform changes shape. + * + * The answer already accounts for zoom, so a pack needing to convert a pixel + * drag into graph units can divide by `width / getBounds().width` rather than + * asking for the scale factor. + * + * `undefined` when nothing is on screen to measure against. + */ + getScreenRect(): Bounds | undefined + /** + * URLs of the images this node produced when it last executed. + * + * Packs read `node.imgs` — the loaded `HTMLImageElement`s core hangs on the + * node — to walk upstream for the nearest ancestor holding a composite, or + * to scan the selection for something to feed an editor. `onExecuted` does + * not answer that: it is per node type, so it never sees another pack's + * outputs, and it only fires at the moment of execution. + * + * URLs rather than elements, deliberately. The loaded element is the + * renderer's, and its lifetime is the renderer's; a pack that wants pixels + * can load the URL itself and own the result. This also covers previews, + * which are what the node is showing when a run is still in flight. + * + * Empty when the node has not produced images. + */ + getOutputImages(): readonly string[] + /** + * Which of {@link getOutputImages} the user is looking at, or `undefined` + * when they have neither selected nor hovered one. + * + * A pack copying "the image" or saving one as a model's preview meant the + * one under the cursor, not the first of the batch. `undefined` is why this + * is not simply `0`: an entry that acts on a guess writes the wrong file to + * the server, silently. + */ + getDisplayedImageIndex(): number | undefined + /** + * The id of the graph holding this node — the root graph's id, or a + * subgraph's. + * + * A pack keeping its own records against nodes needs it: node ids are unique + * per graph, so a key built from the id alone collides once subgraphs are + * involved. Pair it with `comfy.graph.subgraphs()` to get back to the node. + */ + readonly graphId: string | undefined + /** + * Puts a small label on the node's title bar. Returns a handle that removes + * it again. + * + * Packs draw a status, a count, a cost, a model name. They did it by + * overriding `onDrawForeground` and painting into the canvas context, which + * only works under the legacy renderer and puts the pack in the business of + * laying out text. `badges` is core's own extension point and both renderers + * draw it. + * + * Pass a function for a label that changes: it is called each time the node + * is drawn, so return quickly and do not build strings you could cache. + */ + addBadge(badge: BadgeDef | (() => BadgeDef)): Unsubscribe + /** + * Declares how the node may be sized, instead of re-asserting it per frame. + * + * 39 packs recompute size inside a draw or resize callback, which is both a + * per-frame cost and a fight with the layout. `autoHeight` is usually the + * real intent: the pack mounted something of unknown height and wants the + * node to fit it. + */ + setSizeConstraints(constraints: SizeConstraints): void + getSizeConstraints(): Readonly + + readonly inputs: SlotCollection + readonly outputs: SlotCollection + readonly widgets: WidgetCollection + snapshot(): Readonly | undefined + remove(): void + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface RunOptions { + /** + * Run only these nodes and whatever feeds them, instead of the whole + * workflow. Empty is rejected rather than treated as "everything": a filter + * that matched nothing must not silently run the entire graph. + */ + nodes?: readonly NodeHandle[] + /** How many times to run. Defaults to 1. */ + batch?: number + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface RunSubmittedEvent { + /** Ids the backend accepted, in submission order. */ + readonly promptIds: readonly string[] + /** The accepted prompts and how many backend nodes each will execute. */ + readonly submissions?: readonly RunSubmission[] + /** How many submissions the backend refused. */ + readonly rejected: number + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface RunSubmission { + readonly promptId: string + readonly nodeCount: number + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface RunRejectionError { + readonly type: string + readonly message: string + readonly details: string + readonly inputName?: string + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface RunRejectedNode { + readonly nodeId: string + readonly nodeType: string + readonly errors: readonly RunRejectionError[] + } + + export interface RunRejectedEvent { + readonly status?: number + readonly error: RunRejectionError + readonly nodeErrors: readonly RunRejectedNode[] + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type AutoQueueMode = 'disabled' | 'change' | 'instant' + + export interface QueueHandle { + /** + * Queues the current workflow, exactly as pressing Run does. + * + * Resolves once the prompt has been submitted — not when it finishes + * executing. `false` means another queue call was already in flight and this + * one was folded into it. + */ + run(options?: RunOptions): Promise + /** Synchronous notification; returned cleanup runs once on attempt completion or unsubscribe. */ + onBeforeRun(listener: () => (() => void) | void): Unsubscribe + /** + * A run was submitted. This is `afterQueued` — for advancing state that + * should differ on the next run. + * + * The event names what the backend accepted, so a pack can tie its own + * progress tracking to the run it started rather than guessing that the next + * execution message belongs to it. Each submission includes the exact count + * of executable backend nodes without exposing the built prompt. `rejected` + * is how many submissions the backend refused: `onBeforeRun` fires either + * way, so without this a pack cannot tell a run that started from one that + * never did. + */ + onAfterRun(listener: (event: RunSubmittedEvent) => void): Unsubscribe + /** + * The backend refused a submitted prompt before execution began. + * + * This exposes prompt and per-node validation details without coupling a + * pack to host notifications. It does not fire for transport failures or an + * error raised after execution starts. + */ + onRejected(listener: (event: RunRejectedEvent) => void): Unsubscribe + /** + * How many runs are waiting, including the one executing. + * + * Packs tracked this from the backend's own `status` message to re-implement + * `app.ui.lastQueueSize` — deciding whether a button says Run or Cancel, + * whether an auto-runner should submit again. + */ + pending(): number + /** Fires whenever {@link pending} changes, with the new count. */ + onPendingChanged(listener: (pending: number) => void): Unsubscribe + /** + * Cancels the run in progress. The rest of the queue is untouched. + * + * Packs wrapped `api.interrupt` both to call it and to notice one — a node + * waiting on the user needs to stop waiting when the run is cancelled. + * {@link onInterrupted} is that second half. + */ + interrupt(): Promise + /** Execution was interrupted, by this pack, another, or the user. */ + onInterrupted(listener: () => void): Unsubscribe + /** The user-facing automatic queue mode. Both internal instant states read as `instant`. */ + autoQueueMode(): AutoQueueMode + /** Changes automatic queuing. `instant` arms continuous execution. */ + setAutoQueueMode(mode: AutoQueueMode): void + /** The batch count the host's own Run action will use. */ + batchCount(): number + /** Changes the host Run action's batch count. */ + setBatchCount(count: number): void + /** + * Turns off automatic queuing without cancelling the current run. + * + * A conditional workflow can use this before interrupting itself so the + * stopped iteration does not immediately start again. + */ + disableAutoQueue(): void + /** + * Holds a run until a check finishes, and can cancel it. + * + * {@link onBeforeRun} only observes: it is a notification, and the prompt + * build does not wait. Packs that needed to *stop* a run — confirm an + * incoming prompt, validate a field, warn about a cost — wrapped + * `app.queuePrompt` to do it, which is the surface being retired. + * + * Return `false` to cancel. Every guard runs, and any one `false` cancels; + * the user is not asked twice. + * + * A guard that never settles would make the application unrunnable, so one + * that takes longer than a few seconds is abandoned and the run proceeds. Do + * not put a dialog with no timeout behind this. + */ + guard(check: () => boolean | Promise): Unsubscribe + } + + /** + * "Whatever feeds this input." The only way one resolution names another. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export interface InputRef { + readonly nodeId: string + readonly input: number + } + + export type OutputResolution = + | { readonly omit: true } + | { readonly forwardTo: InputRef } + | { readonly literal: WidgetValue } + + /** + * What a resolver may see. Reads only — there is nothing here that writes. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export interface ResolvedNodeView { + readonly id: string + readonly type: string + /** + * The node's own properties, frozen. + * + * A broadcaster keeps its per-node opt-in here — cg-use-everywhere reads + * `properties.ue_properties` to decide what it may feed. Candidate inputs + * already carry `nodeProperties`, so without this a supplier could read + * every node's configuration except its own. + */ + readonly properties: Readonly> + /** The groups this node sits inside — the other half of "my group". */ + readonly groups: readonly GroupMembership[] + /** Muted, bypassed or normal, as `LGraphEventMode`. */ + readonly mode: number + readonly color: string | undefined + /** + * This node's own inputs. + * + * `unconnectedInputs()` already describes every *other* node's slots, and a + * supplier needs the same of its own: "send whatever is plugged into me to + * every unconnected input of the same type" cannot be written without + * knowing what type is plugged in. Without it a supplier is type-blind and + * would feed a CLIP into a MODEL slot in silence. + * + * `type` is the slot's declared type; `connectedType` is what actually + * arrives, resolved through reroutes, and is undefined when nothing is + * connected. + */ + readonly inputs: readonly OwnInput[] + /** This node's own outputs, in slot order. */ + readonly outputs: readonly OwnOutput[] + widgetValue(name: string): WidgetValue | undefined + input(ref: string | number): InputRef | undefined + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface ResolveView { + readonly self: ResolvedNodeView + readonly signal: AbortSignal + nodesOfType(type: string): readonly ResolvedNodeView[] + } + + /** + * May answer asynchronously: a sandboxed pack's resolver runs in a worker, so + * its answer can only arrive as a promise. The prompt path awaits it; the + * synchronous entry points (`input.resolvedSource()`, `resolvedSupplies()`) + * treat a promise as unresolved and say so — see `resolution.async.test.ts`. + */ + export type Resolver = ( + view: ResolveView + ) => + | Record + | Promise> + + /** Where an output ends up after every frontend node in the chain resolves. */ + export type ResolvedSource = + | { + readonly kind: 'output' + readonly nodeId: string + readonly output: number + } + | { readonly kind: 'literal'; readonly value: WidgetValue } + | { readonly kind: 'omitted'; readonly reason: string } + + /** An input in the graph that no link feeds. */ + /** One of a node's own inputs, as its supplier sees it. */ + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface OwnInput { + readonly index: number + readonly name: string + /** What the user sees — `label`, else `localized_name`, else `name`. */ + readonly label: string + readonly type: string + readonly connected: boolean + /** The type actually arriving, or undefined when nothing is connected. */ + readonly connectedType: string | undefined + /** The node feeding this input, if any. */ + readonly sourceNodeId: string | undefined + } + + /** One of a node's own outputs, as its supplier sees it. */ + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface OwnOutput { + readonly index: number + readonly name: string + /** What the user sees — `label`, else `localized_name`, else `name`. */ + readonly label: string + readonly type: string + } + + /** A group a node sits inside. */ + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface GroupMembership { + readonly id: string + readonly title: string + } + + export interface UnconnectedInput { + readonly nodeId: string + readonly nodeType: string + readonly input: number + readonly name: string + readonly type: string + /** + * What the user actually sees on the slot — `label`, else `localized_name`, + * else `name`. Broadcast packs match against this, not `name`, and the two + * differ in every non-English locale. + */ + readonly label: string + /** The socket form of a widget rather than a plain input. */ + readonly isWidgetInput: boolean + /** The owning node, for matching by title, mode, colour, or opt-in flags. */ + readonly nodeTitle: string + readonly nodeMode: number + readonly nodeColor: string | undefined + /** + * The groups the owning node sits inside, innermost first. + * + * Broadcast packs restrict by group — "only nodes in my group", "only nodes + * outside it", "only groups whose title matches this regex". Membership is + * geometric and recomputed here, so it matches what the user sees rather + * than anything stored. + */ + readonly nodeGroups: readonly GroupMembership[] + /** + * The owning node's properties, frozen. + * + * Broadcast packs keep their per-node opt-in here — which inputs a user has + * allowed to be fed. Without it a supplier can only match by type and would + * feed every unconnected input of that type, which is the silent + * wrong-broadcast failure this view exists to prevent. + */ + readonly nodeProperties: Readonly> + } + + /** + * An edge a node supplies into somebody else's unconnected input. + * + * `from` is the supplier's own output index, or a literal. It is deliberately + * not an arbitrary node reference: a node may only offer what it itself has, + * so one pack cannot rewire two other nodes to each other. + */ + export interface SuppliedEdge { + readonly to: InputRef + /** + * Which claim wins when several suppliers name the same input. Higher wins; + * defaults to 0. + * + * **Equal claims feed nothing.** Two suppliers that both say "highest + * priority" for one input have no correct answer, and picking either makes + * the prompt depend on node order — so the input is left unfed and the + * conflict logged. That is what the broadcast pack this exists for does, and + * it is the only choice that cannot silently produce a different image. + */ + readonly priority?: number + readonly from: + | { readonly output: number } + | { readonly literal: WidgetValue } + /** + * Whatever feeds this node's own input `k` — for a node that rebroadcasts + * its upstream rather than producing a value. + * + * The broadcast nodes this exists for have inputs and **no outputs**, so + * `{ output: n }` cannot describe them: it would name a slot the backend + * never declared and force it to execute a node that produces nothing. + * Resolved exactly as `Resolver`'s `forwardTo`, so it chains through + * reroutes for free. + */ + | { readonly forwardInput: number } + } + + export interface SupplyView { + readonly self: ResolvedNodeView + readonly signal: AbortSignal + nodesOfType(type: string): readonly ResolvedNodeView[] + /** + * Every unfed input in the graph — what a broadcaster matches against by + * type, by name, or by its own regex. + */ + unconnectedInputs(): readonly UnconnectedInput[] + } + + /** + * Answers "what do I feed", the mirror of `Resolver`'s "what feeds me". + * + * `Resolver` is demand-side: it is asked about the resolver's own outputs, and + * is never called for a node with none. cg-use-everywhere broadcasts a value + * into every matching unconnected input in the graph, which that shape cannot + * express at all — the nodes being fed are not the resolver, and the edges are + * discovered rather than declared. Hence a second, supply-side pass. + * + */ + /** May answer asynchronously, under the same rules as {@link Resolver}. */ + export type Supplier = ( + view: SupplyView + ) => readonly SuppliedEdge[] | Promise + + /** + * One winning supply after priority arbitration and source resolution. + */ + export interface ResolvedSupply { + /** The node whose supplier offered this edge. */ + readonly supplierNodeId: string + /** The unconnected input the supplier won. */ + readonly to: InputRef + /** The final source the prompt builder will use. */ + readonly from: ResolvedSource + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type SettingValue = string | number | boolean | readonly string[] + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SettingDef { + /** + * Namespaced, by convention `.` — it shares one space with core + * and every other pack, and it is what the value is stored under forever. + */ + readonly id: string + readonly name: string + /** + * Which control the panel shows. Every one of these is declarative — the + * host renders it. + * + * A pack-supplied renderer is deliberately absent. Core's own setting type + * accepts a function that is handed the value and a setter and returns an + * element; publishing that would put packs in charge of the settings + * panel's markup, which is the thing that cannot then be restyled. Packs + * that needed a colour or a file were falling back to a text field the user + * pasted into, so the gap was the missing *types*, not a missing slot. + */ + readonly type: + | 'boolean' + | 'number' + | 'slider' + | 'knob' + | 'combo' + | 'radio' + | 'text' + | 'password' + | 'color' + | 'image' + | 'url' + readonly defaultValue: SettingValue + readonly tooltip?: string + /** Panel grouping. Defaults to the id split on dots. */ + readonly category?: readonly string[] + /** + * Choices for `combo` and `radio`. + * + * A bare string is both the stored value and the label. Use the pair form + * when they differ — several packs store a semantic number and show words + * for it (`0` = off, `1` = selected, `2` = all), and comparing those + * numerically is the whole point. Flattening them to strings silently + * re-types every user's saved choice. + */ + readonly options?: readonly SettingOption[] + /** + * Bounds for `number` and `slider`. Without these a slider has no range to + * draw and packs fall back to a plain text box. + */ + readonly attrs?: SettingAttrs + readonly onChange?: (value: SettingValue, previous?: SettingValue) => void + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type SettingOption = + | string + | { readonly value: string | number; readonly label: string } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SettingAttrs { + readonly min?: number + readonly max?: number + readonly step?: number + } + + export interface SettingsHandle { + /** + * Registers a setting. Call once, at extension load: a value already stored + * for this id survives, so re-declaring cannot reset a user's choice. + */ + declare(def: SettingDef): void + get(id: string): SettingValue | undefined + set(id: string, value: SettingValue): Promise + /** + * Watches a setting, including one the pack did not declare. + * + * `declare`'s own `onChange` only fires for settings the pack owns, so a + * pack that needs to react to a *core* preference — colour palette, link + * render mode, locale — had nothing to observe and polled or ignored it. + * + * Fires on change only, not on registration. Returns a function that stops + * watching; call it from wherever the pack tears down. + */ + onChange( + id: string, + listener: (value: T | undefined, previous: T | undefined) => void + ): Unsubscribe + } + + /** + * Inside a subgraph, a link to the subgraph's own input or output has that + * boundary's id as its node id, and no node handle exists for it. + */ + export interface LinkInfo { + readonly id: string + readonly sourceNodeId: string + readonly sourceSlotId: SlotId + readonly targetNodeId: string + readonly targetSlotId: SlotId + readonly type: string + /** Position at snapshot time. Do not store across mutations. */ + readonly sourceIndex: number + readonly targetIndex: number + } + + /** + * Fields a pack may change on an existing slot. + * + * Applied atomically as one command, so a retype-plus-rename is a single undo + * step rather than two. Retyping deliberately **keeps existing links**: dynamic + * retyping (`*` -> `MODEL`) is the whole point for `SetNode`-style packs, and + * silently dropping connections is the failure mode this API exists to end. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + /** + * A slot's type, which may be a union. + * + * An array spells "this slot accepts any of these" — rgthree's + * `addInput('input', ['IMAGE', 'LATENT', 'MASK'])` is the shipped example, so + * packs do write it even though litegraph's own `ISlotType` says + * `number | string`. + * + * Both forms are accepted and stored as the comma string, because that is what + * litegraph compares against: it normalises with `String(type).split(',')`, so + * `['IMAGE','LATENT','MASK']` and `'IMAGE,LATENT,MASK'` are the same slot to + * every connection check. The saved workflow therefore holds the string where + * the original held an array — a byte difference with no behavioural one, and + * the same call already taken for slot `shape`. + * + * Reads stay `string` for the same reason. + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export type SlotType = string | string[] + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type SlotDirection = + | 'none' + | 'up' + | 'down' + | 'left' + | 'right' + | 'center' + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SlotPosition { + readonly x: number + readonly y: number + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SlotPatch { + name?: string + label?: string | undefined + /** The backend-provided translated caption. Null clears it. */ + localizedName?: string | null + type?: SlotType + /** Slot centre relative to the node body. Null restores automatic layout. */ + position?: SlotPosition | null + /** Direction in which links leave the slot. Null restores the default. */ + direction?: SlotDirection | null + /** + * The dot's colour when connected and when not. + * + * Not decoration, despite appearances: both sit on `INodeSlot` and + * `ISerialisableNodeInput` omits only `boundingRect`, `widget` and `link`, + * so they are written into the saved workflow. A pack that coloured its + * slots and then stopped saves different bytes than it used to. + * + * `null` clears one back to the renderer's default. + */ + color?: string | null + colorWhenUnconnected?: string | null + /** + * Sits on the same `INodeSlot` as the colours above and is omitted by the + * same `Omit`, so the argument made for them holds verbatim: a pack that + * shaped its slots and then stopped saves different bytes than it used to. + * + * `'default'` clears it back to the renderer's own choice. + */ + shape?: SlotShape + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface InputSlotPatch extends SlotPatch { + /** Retargets the widget this input is the socket form of. Null clears it. */ + widget?: string | null + /** Replaces the input declaration used by connected Primitive nodes. */ + widgetConfig?: InputWidgetConfig + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface InputWidgetConfig { + /** Backend input type, or the choices for a COMBO input. */ + readonly type: string | readonly (string | number)[] + readonly options?: Readonly> + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SlotSnapshot { + readonly id: SlotId + readonly index: number + readonly name: string + readonly type: string + readonly label: string | undefined + readonly localizedName: string | undefined + readonly position: SlotPosition | undefined + readonly direction: SlotDirection | undefined + readonly shape: SlotShape + readonly isConnected: boolean + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type ResolvedInputSource = + | { + readonly kind: 'output' + readonly graphId: string + readonly nodeId: string + readonly outputIndex: number + } + | { readonly kind: 'literal'; readonly value: WidgetValue } + | { readonly kind: 'omitted'; readonly reason: string } + + export interface InputSlotHandle { + readonly id: SlotId + /** Volatile — shifts when other slots are added or removed. */ + readonly index: number + readonly name: string + readonly type: string + readonly label: string | undefined + readonly isConnected: boolean + /** The type arriving through the link, including across a subgraph input. */ + readonly connectedType: string | undefined + /** Whether this input is the socket form of a widget. */ + readonly isWidgetInput: boolean + /** The declaration a connected Primitive node renders. */ + widgetConfig(): Readonly | undefined + /** Intersects this input's declaration with another compatible one. */ + mergeWidgetConfig( + config: InputWidgetConfig + ): Readonly | undefined + link(): LinkInfo | undefined + source(): { nodeId: string; outputIndex: number } | undefined + /** + * What ultimately feeds this input after frontend nodes resolve. + * + * `source()` reports the physical link, which is right for editing topology. + * This reports the executable source through reroutes, Get/Set nodes and any + * other frontend node declared with `defs.define({ resolve })`. Resolution is + * read-only and leaves the graph untouched. + */ + resolvedSource(): ResolvedInputSource | undefined + disconnect(): boolean + modify(patch: InputSlotPatch): void + /** Replaces `{...input}`, which now yields nothing useful. */ + snapshot(): Readonly + } + + export interface OutputSlotHandle { + readonly id: SlotId + readonly index: number + readonly name: string + readonly type: string + readonly label: string | undefined + readonly isConnected: boolean + /** Frozen snapshot — safe to iterate while disconnecting. */ + links(): readonly LinkInfo[] + targets(): readonly { nodeId: string; inputIndex: number }[] + connectTo(targetNodeId: string, input: SlotRef): LinkInfo | undefined + disconnect(targetNodeId?: string): boolean + modify(patch: SlotPatch): void + /** + * Moves every link on this output to another output of the same node, + * **preserving link ids**. + * + * Disconnect-and-reconnect is not equivalent: it allocates new ids, so the + * serialized workflow changes. Packs that re-home their own outputs during a + * migration depend on identity being kept. + * + * Slot types are **not** re-validated. The real-world sequence moves links + * off an output and then retypes it, so enforcing compatibility mid-move + * would reject exactly the case this exists for. + */ + moveLinksTo(target: SlotRef): readonly LinkInfo[] + snapshot(): Readonly + } + + export interface SlotCollection { + readonly length: number + get(ref: SlotRef): THandle | undefined + byId(id: SlotId): THandle | undefined + byName(name: string): THandle | undefined + /** Explicit positional access. */ + at(index: number): THandle | undefined + all(): readonly THandle[] + ids(): readonly SlotId[] + names(): readonly string[] + /** + * Adds a slot. 18 packs grow their inputs as the last one fills — the + * "Multi" combiner pattern — which needed `node.addInput` until now. + * + * `shape` is not decoration: it is written into the saved workflow, so a + * slot added without the one its pack used to set serialises differently + * from one the pack itself wrote. `'optional'` is the hollow circle + * ComfyUI draws for an input that need not be connected. + */ + add(name: string, type: SlotType, options?: SlotOptions): THandle + /** + * Removes a slot by reference. Any link into it is dropped, as it would be + * on the legacy path. + */ + remove(ref: SlotRef): boolean + /** + * Puts the slots in the given order. `names` must be a permutation of the + * current ones. + * + * Every link into or out of this node is re-pointed as part of the move, in + * one batch, so link ids — and therefore the saved workflow's `links` array + * — are unchanged. That is the whole reason this exists rather than being + * left to packs: a link stores its endpoint as a slot *index*, so a pack + * permuting the array itself silently re-points every connection, and the + * damage only shows when the workflow is next run. + * + * The slot *order* is serialized, so this changes the saved file by design — + * it is how a pack keeps its dynamic inputs matching what the backend + * declares. + */ + reorder(names: readonly string[]): void + [Symbol.iterator](): Iterator + } + + /** + * How a slot is drawn, which ComfyUI overloads to mean how it behaves. + * + * Named rather than numbered: packs wrote `{ shape: 7 }`, and 7 is meaningless + * without litegraph's RenderShape enum in front of you. + */ + export type SlotShape = 'default' | 'optional' | 'list' | 'directional' + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SlotOptions { + /** + * `'optional'` is the hollow circle for an input that need not be connected, + * `'list'` the grid ComfyUI draws for an output that yields many values, and + * `'directional'` the arrow a pack uses for a slot that only ever feeds one + * particular kind of node. + */ + shape?: SlotShape + localizedName?: string + position?: SlotPosition + direction?: SlotDirection + /** + * Names the widget this slot is the socket form of — the "convert widget to + * input" shape. + * + * Not decoration either: a slot carrying it serialises as + * `{ widget: { name } }` where a plain socket serialises as `{ pos }`, and + * the widget keeps its place in `widgets_values`. A dynamic input added + * without it changes the saved file. + */ + widget?: string + /** The declaration a connected Primitive node should render. */ + widgetConfig?: InputWidgetConfig + } + + export type SlotId = string & { readonly __brand: 'SlotId' } + + /** + * A slot reference: a string (id or name), or an explicit `{ index }`. + * + * A bare `number` is deliberately not accepted so positional access is visible + * at the call site and greppable: + * + * output.connectTo(node, 'image') // by name — preferred + * output.connectTo(node, { index: 0 }) // by position — explicit + */ + export type SlotRef = SlotId | string | { readonly index: number } + + /** + * Logical text documents scoped to the current user. Native hosts use dotted + * pack prefixes by convention; isolated hosts can enforce pack namespaces. + */ + export interface StorageHandle { + /** + * Lists logical keys including `namespace`, ready for `get()` or `remove()`. + * Physical user prefixes stay hidden. Empty folders return an empty array. + */ + list(namespace: string): Promise + /** The stored text, or `undefined` if there is none. */ + get(name: string): Promise + set(name: string, value: string): Promise + remove(name: string): Promise + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SystemMonitorCpu { + readonly utilization_percent: number | null + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SystemMonitorMemory { + readonly total: number + readonly available: number + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SystemMonitorVolume extends SystemMonitorMemory { + readonly id: string + readonly label: string + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SystemMonitorAccelerator { + readonly id: string + readonly name: string + readonly memory_total: number + readonly memory_available: number + readonly utilization_percent: number | null + readonly temperature_c: number | null + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SystemMonitorSnapshot { + readonly cpu: SystemMonitorCpu + readonly memory: SystemMonitorMemory + readonly volumes: readonly SystemMonitorVolume[] + readonly accelerators: readonly SystemMonitorAccelerator[] + } + + export interface SystemHandle { + /** + * Returns one host-sampled hardware snapshot. Volume ids are opaque and + * unsupported utilization or temperature sensors are null. + */ + monitor(): Promise + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface SidebarTabBase { + /** + * Unique across every pack, so namespace it — `'mtb.assets'`, not + * `'assets'`. Registering an id twice throws rather than silently replacing + * the other pack's tab. + */ + readonly id: string + readonly title: string + /** + * An iconify class, e.g. `'icon-[lucide--activity]'`. Omit for no icon. + */ + readonly icon?: string + readonly tooltip?: string + } + + /** + * A tab the pack draws into a container itself. + * + * Framework-agnostic, and the only form available to a pack that ships + * hand-written ES modules with no build step — which is most of them. + */ + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface MountedSidebarTab extends SidebarTabBase { + /** + * Fills the tab's panel. Called each time the tab becomes visible, so treat + * it as mount rather than as one-time setup, and put teardown in `destroy`. + */ + render(container: HTMLElement): void + /** Releases what `render` retained — listeners, timers, observers. */ + destroy?(): void + } + + /** + * A tab that is a Vue component, mounted and torn down by the host. + * + * The preferred form where a pack can build. It keeps reactivity, scoped + * styles and `onUnmounted`, and the host mounts and unmounts it. + * + * Per ADR-EXTENSIONS-PUBLIC-API-0005 the pack bundles its own Vue (~30KB gzipped) — there is no + * import map, so `import { defineComponent } from 'vue'` resolves at the + * pack's build time, not ours. That is a second Vue instance on the page, + * which the ADR weighed and accepted; nothing is shared across the boundary, + * so the two runtimes never touch. + */ + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface VueSidebarTab extends SidebarTabBase { + readonly component: VueComponent + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type SidebarTabDef = MountedSidebarTab | VueSidebarTab + + /** A Vue component bundled by the pack. */ + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type VueComponent = object + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface DialogBase { + /** + * Unique across every pack, so namespace it. The host prefixes it with + * `extension-`, which keeps packs out of the internal dialog keyspace. + */ + readonly key: string + readonly title?: string + } + + /** A dialog the pack draws into a container itself. */ + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface MountedDialog extends DialogBase { + render(container: HTMLElement): void + destroy?(): void + } + + /** A dialog that is a Vue component, mounted and torn down by the host. */ + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface VueDialog extends DialogBase { + readonly component: VueComponent + readonly props?: Readonly> + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type DialogDef = MountedDialog | VueDialog + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface DialogHandle { + close(): void + } + + export interface UiHandle { + /** + * Adds a tab to the sidebar. Returns a function that removes it again. + */ + addSidebarTab(def: SidebarTabDef): Unsubscribe + /** + * Shows a small readout in the top bar — a status, a count, a live metric. + * + * Replaces `app.menu.settingsGroup` and inserting an element next to + * `.comfy-settings-btn`. Declarative on purpose: the pack says what to show + * and the host renders it, in house style and at whatever size the viewport + * allows. Nothing here takes an element, a class or a style, which is what + * keeps the chrome ours to restyle. + * + * Returns a handle rather than an unsubscribe: for a value that changes, + * call `update({ text })`. A closure would not work — the host renders when + * reactive state changes and cannot see a plain function, so the readout + * would show its first value forever. + */ + addTopBarBadge( + badge: BadgeContribution + ): ChromeItemHandle + /** + * Adds a button to the action bar. `run` is called on click. + * + * For a pack that also wants a keyboard shortcut or a palette entry, + * register a command and call it from `run`, rather than duplicating the + * behaviour in both places. + */ + addActionBarButton( + button: ButtonContribution + ): ChromeItemHandle + /** + * Opens a modal dialog. Returns a handle that closes it again. + * + * Replaces `app.ui.dialog` and the `new app.ui.dialog.constructor()` idiom. + * Several conversions hand-rolled a native `` or borrowed core's + * `.comfy-modal` class names instead — the latter couples a pack to markup + * we rename freely, so both are worth retiring. + */ + showDialog(def: DialogDef): DialogHandle + /** + * Shows a menu where the user clicked. + * + * `b.addMenuItem` is the node's own context menu — a different menu, on a + * different target, opened by the host. This is for a menu a pack raises + * itself: a lora row's Move Up / Remove, a chip that picks an output type. + * Four files hand-rolled it by constructing the renderer's menu class + * directly, which pins them to a renderer we intend to replace. + * + * Positioned from the event so the menu lands under the pointer, which is the + * only placement that reads as a context menu. Arrow keys traverse nested + * items, Enter or Tab selects one, and Escape closes the menu. + */ + showMenu(def: MenuDef): MenuHandle + /** + * Asks the user for a value. Resolves `undefined` if they cancel. + * + * Packs called `canvas.prompt(...)`, which draws a small field at the cursor + * — clicking a lora's strength to type a new one. That field belongs to the + * legacy canvas and the host itself no longer uses it; this is the prompt the + * host does use, so a pack keeps the capability and loses only the placement. + */ + prompt(def: PromptDef): Promise + /** + * Mounts a panel over the graph view, placed by the host against an edge or + * corner. It stays put while the graph pans and zooms. + */ + mountViewportPanel(def: ViewportPanelDef): ViewportPanelHandle + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface PromptDef { + /** What is being asked for — "Strength", "Label". */ + readonly label: string + readonly value?: string + readonly placeholder?: string + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface MenuItemDef { + readonly label: string + /** Shown but not selectable. */ + readonly disabled?: boolean + /** A nested menu. Mutually exclusive with {@link run}. */ + readonly submenu?: readonly MenuItemDef[] + run?(): void + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface MenuDef { + readonly items: readonly MenuItemDef[] + /** Shown above the items. */ + readonly title?: string + /** The event that asked for the menu; it decides where the menu appears. */ + readonly event: MouseEvent + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface MenuHandle { + close(): void + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export type ViewportPanelAnchor = + | 'top-left' + | 'top-center' + | 'top-right' + | 'bottom-left' + | 'bottom-center' + | 'bottom-right' + + /** A panel the pack renders over the graph view. */ + export interface ViewportPanelDef { + /** Namespaced to the pack. */ + readonly id: string + /** The edge or corner the panel sits against. Defaults to `'top-left'`. */ + readonly anchor?: ViewportPanelAnchor + /** Pixels in from the anchored side, or along from the centre. */ + readonly offsetX?: number + /** Pixels in from the anchored top or bottom. */ + readonly offsetY?: number + readonly width?: number + readonly maxHeight?: number + readonly ariaLabel?: string + /** Fills the panel. Called once, with an element already in the document. */ + render(container: HTMLElement): void + destroy?(): void + } + + export interface ViewportPanelHandle { + remove(): void + } + + // `null` is included because core's own `WidgetValue` has it and + // `addWidget('button', name, null, cb)` produced exactly that. Omitting it made + // a null value inexpressible through the published API, so a converted button's + // `widgets_values` entry changed and the saved workflow differed. + export type WidgetValue = + | string + | number + | boolean + | object + | undefined + | null + + /** Options understood by core or by a widget type declared by the pack. */ + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface WidgetOptions { + readonly [key: string]: unknown + readonly on?: string + readonly off?: string + readonly max?: number + readonly min?: number + readonly precision?: number + readonly read_only?: boolean + readonly step?: number + readonly step2?: number + readonly multiline?: boolean + readonly property?: string + readonly socketless?: boolean + readonly canvasOnly?: boolean + readonly hideInPanel?: boolean + readonly nodeType?: string + readonly serialize?: boolean + readonly values?: unknown + readonly iconClass?: string + readonly disabled?: boolean + readonly useGrouping?: boolean + readonly placeholder?: string + readonly showThumbnails?: boolean + readonly showItemNavigators?: boolean + readonly hidden?: boolean + } + + /** + * Shapes follow `docs/node-api/reference.md`, the published contract. + * + * Accessor methods rather than properties, so a read can be a store query and + * a write can dispatch a command. + */ + export interface WidgetHandle extends HandleCommon { + readonly name: string + readonly widgetType: string + + getValue(): WidgetValue + /** + * Commits a value exactly as a user edit does: the value is written, a + * widget bound to a node property syncs it, the widget's callback chain and + * the node's `onWidgetChanged` run, and `graph.version` advances. This + * replaces the manual pair `widget.value = x; widget.callback?.(x)` — and + * the bare write too, because a write the rest of the system cannot see was + * never a feature, it was litegraph defaulting to inconsistency. + * + * Writing the current value again is a no-op, which is also what ends a + * cycle of handlers writing to each other. `on('change')` fires once per + * commit; `on('activate')` does not fire, because activate reports a user's + * act. + */ + setValue(value: WidgetValue): void + + /** + * The widgets core attached to this one — a seed's `control_after_generate`, + * a bounding box's components. + * + * `setHidden` already cascades through these, so hiding needs no call here. + * What does is reading one: a pack asks a seed's control widget whether it + * says `fixed` or `randomize` to know what the node will do next. + */ + linked(): readonly WidgetHandle[] + /** + * Replaces the controls attached to this widget. + * + * Core uses this relationship for compound inputs: hiding a seed also hides + * its `control_after_generate` picker. Packs build the same compound control + * when they add a random-seed button or an index policy, and assigning + * `linkedWidgets` directly was the only way to make conversion-to-input hide + * the whole unit. + * + * Every name must identify another widget on this node. Pass an empty array + * to clear the relationship. + */ + setLinked(names: readonly string[]): void + + isHidden(): boolean + /** + * Replaces the `type = 'converted-widget'` hack. Value is retained. + * + * Cascades to the widgets core attached to this one — a seed's + * `control_after_generate`, a bounding box's components. The legacy + * `hideWidget` helper this replaces recursed through `linkedWidgets`, and + * packs that lost the cascade were left with an orphaned control widget + * floating where its owner used to be. + */ + setHidden(hidden: boolean): void + getOptions(): Readonly | undefined + setOption(key: string, value: unknown): void + setLabel(label: string): void + + isDisabled(): boolean + setDisabled(disabled: boolean): void + isSerialized(): boolean + /** The height the host most recently allocated, or undefined before layout. */ + getHeight(): number | undefined + /** + * Pins the widget's height in graph units, instead of letting it share + * whatever space the node has spare. + * + * The node divides free height between every widget that does not state one, + * so a node carrying two mounted strips gave each half the node however + * small they were meant to be. `MountDef.height` does not do this — it sets + * the container's CSS height *inside* an allocation the renderer already + * chose, which is why a fixed strip still drifted. + * + * Replaces re-assigning `node.computeSize`, which is what packs did and + * which is not published. Omit it for a panel meant to fill the node: the + * growable path is the one that fills. + */ + setHeight(px: number): void + + /** + * Replaces capture-and-chain on `widget.callback`, which 1,000+ sites do and + * which silently drops an earlier pack's listener whenever one forgets to + * call through. Listeners here are additive and independent. + */ + on( + event: 'change', + listener: (value: WidgetValue, oldValue: WidgetValue) => void + ): Unsubscribe + on(event: 'removed', listener: () => void): Unsubscribe + /** + * The widget was activated — a button click, or a value committed. + * + * Buttons carry no value, so `change` can never fire for one and a button + * created through this API would otherwise be inert. Prefer `change` when you + * care about the value; use this when you care that the user acted — a + * programmatic `setValue` never fires it. + */ + on(event: 'activate', listener: (value: WidgetValue) => void): Unsubscribe + /** + * Contributes behavior to a host-owned multiline text editor without exposing + * its DOM. The event reports the live value and caret on each input, + * selection change, or wheel gesture; its write method preserves both the + * widget commit protocol and the requested selection. + */ + on( + event: 'textInteraction', + listener: (event: WidgetTextInteractionEvent) => void + ): Unsubscribe + /** + * The value is about to be written out, and may be replaced for this + * destination only. + * + * This is what `widget.serializeValue` did, and the reason it is back: a + * static `serialize` flag can only *suppress* a value, and a whole class of + * packs needs to *supply* a different one. rgthree's Seed keeps the sentinel + * `-1` in the saved workflow and sends the rolled seed; pysssss' PresetText + * expands `@name` into the queued prompt while the user keeps seeing the + * reference; Impact Pack embeds image data the canvas never shows. + * + * `context` says which destination is being built, because those packs want + * to change one and not the other: + * + * - `'workflow'` — the file the user saves. + * - `'prompt'` — the queued API payload the backend executes. + * - `'embedded'` — the copy of the workflow that travels with that prompt + * and is written into the output image. Distinct from `'workflow'` + * because a pack may want the image to reproduce the run while the saved + * file keeps its sentinel: rgthree's Seed saves `-1` but embeds the seed + * it actually rolled, so dragging the PNG back in reproduces it. + * + * A handler that ignores `context` changes all three. + * + * Calling `setSerializedValue` replaces the value for this write only; the + * widget itself is untouched, so the user still sees what they typed. Last + * handler to call it wins. + */ + on( + event: 'beforeSerialize', + listener: (event: WidgetSerializeEvent) => void + ): Unsubscribe + } + + /** + * Where a value is being written, and the chance to change it. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export interface WidgetSerializeEvent { + readonly context: 'workflow' | 'prompt' | 'embedded' + /** What would be written if no handler intervened. */ + readonly value: WidgetValue + setSerializedValue(value: WidgetValue): void + } + + export type Unsubscribe = () => void + + /** + * A widget whose body the pack renders itself. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export interface MountDef { + readonly name: string + /** + * Fills the mounted container. Called once, with an element already attached + * to the node. + * + * `value` holds meaningful serialized state only when `defaultValue` was + * given. A decorative mount receives the same accessor for one render shape, + * but should not use it as storage. + */ + render(container: HTMLElement, value: MountedValue): void + /** Releases anything `render` retained — listeners, timers, observers. */ + destroy?(): void + /** Reserved height in graph units. Omit to size to content. */ + readonly height?: number + /** Set false to keep the element rendered at low zoom. Defaults to true. */ + readonly hideOnZoom?: boolean + readonly hidden?: boolean + /** + * Whether the value is written into the saved workflow. + * + * Defaults to `true` when `defaultValue` makes this a value-holding control, + * and to `false` for a decorative mount. + */ + readonly serialize?: boolean + /** + * Whether the value is sent in the API prompt. Defaults to `serialize`. + * + * These are two different flags in litegraph — `widget.serialize` gates the + * saved workflow, `options.serialize` gates the prompt — and collapsing them + * into one boolean made two states unsayable. "Saved but not sent" is the + * one packs need: it is exactly what the legacy + * `addDOMWidget(…, { serialize: false })` did, and a readout that a node + * fills in from its own execution result belongs in the workflow but has no + * business appearing as an input on the next queue. + * + * Set it apart from `serialize` only when the two genuinely differ. + */ + readonly sendToPrompt?: boolean + /** + * Makes this a value-holding widget rather than decoration. + * + * Without it a mount is a drawing: it can occupy a `widgets_values` slot but + * has nothing to put in it, so a colour picker or a text box converted onto + * `mount` kept its position and silently lost what the user typed. Supplying + * a default gives the widget a real cell, reachable through `render`'s second + * argument. + */ + readonly defaultValue?: MountedData + } + + /** What a mounted control can hold. @knipIgnoreUnusedButUsedByCustomNodes */ + export type MountedData = string | number | boolean | object | null + + /** + * Reading and writing a mounted widget's value. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export interface MountedValue { + get(): MountedData + set(value: MountedData): void + /** Notified when the value changed elsewhere — a workflow load. */ + onChange(listener: (value: MountedData) => void): Unsubscribe + } + + /** + * A pointer event on the widget's own canvas, in the same units `draw` uses. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export interface CanvasPointerEvent { + /** Distance from the canvas's left edge, in CSS pixels. */ + readonly x: number + /** Distance from its top edge, in CSS pixels. */ + readonly y: number + /** The DOM event, for modifier keys, `button`, and `preventDefault()`. */ + readonly event: PointerEvent + } + + /** + * The colours a pack should draw its own controls in. + * + * Published because we told packs to draw. A widget that hardcodes its palette + * looks wrong the moment the user switches theme, and the alternative — reading + * `LiteGraph.WIDGET_BGCOLOR` and friends — is a renderer constant we intend to + * delete. These are the design system's own tokens, resolved from the widget's + * computed style, so they follow the theme without the pack knowing which one + * is active. + * + * Named by intent rather than by token, because the token names will churn and + * a pack should not have to follow. Re-read on every draw, so a theme switch + * needs nothing from the pack. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export interface CanvasTheme { + /** A control's background. */ + readonly surface: string + /** The same under the pointer. */ + readonly surfaceHovered: string + /** A control's outline. */ + readonly border: string + /** A label. */ + readonly text: string + /** A value, a unit, anything the label outranks. */ + readonly textSecondary: string + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface CanvasDef { + readonly name: string + /** Reserved height in pixels. Omit to size to the node's width. */ + readonly height?: number + draw( + context: CanvasRenderingContext2D, + size: readonly [number, number], + theme: CanvasTheme, + value: MountedValue | undefined + ): void + /** + * The pointer went down on this widget. + * + * Coordinates are relative to the canvas and in the same units `draw` + * receives, so a hit test written against the drawing works unchanged — + * which is the point. A pack that drew its own controls keeps both the + * drawing and the hit testing; only the surface changes, from the host's + * canvas to its own. + * + * The primary button is taken: it stops here rather than also reaching the + * node, or adjusting a slider would drag the node underneath it. Middle and + * right are left alone, so panning and the context menu still work over the + * widget. + * + * The pointer is captured for the gesture, so a drag that leaves the widget + * still reports moves and the release. + */ + onPointerDown?(event: CanvasPointerEvent): void + /** Moves during a drag, and hover when no button is down. */ + onPointerMove?(event: CanvasPointerEvent): void + onPointerUp?(event: CanvasPointerEvent): void + /** + * The secondary button went down on this widget. + * + * Right-click is left alone by {@link onPointerDown} so the node's own + * context menu keeps working over a widget, which is right by default and + * wrong for a widget that has its own menu — a lora row wants Move Up, Move + * Down, Remove. Declaring this claims the gesture: the browser menu is + * suppressed and the node's does not open. + */ + onContextMenu?(event: CanvasPointerEvent): void + /** + * Makes the surface hold a value rather than only draw one. + * + * Without it a drawn control that stores something has to be two widgets — a + * hidden value widget and a surface — and two widgets cannot occupy the one + * position the original had. That is not a tidiness point: `serialize` writes + * at each widget's own index and leaves a hole where a non-serializing widget + * sits, so the pair has to be ordered value-first to keep the saved array + * intact, and a pack that gets that wrong writes a null into every workflow + * the node has ever appeared in. It moved rgthree's Power Puter chip row + * below its code box. + * + * `draw` receives the current value as its fourth argument. + */ + readonly defaultValue?: MountedData + /** Whether the value reaches the saved workflow. See {@link MountDef.serialize}. */ + readonly serialize?: boolean + /** Whether the value reaches the API prompt. See {@link MountDef.sendToPrompt}. */ + readonly sendToPrompt?: boolean + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface CanvasHandle { + readonly widget: WidgetHandle + /** Redraws now. Call when the data behind the drawing changed. */ + redraw(): void + } + + /** Everything needed to create a widget. */ + export interface WidgetDef { + readonly type: string + readonly name: string + readonly value?: WidgetValue + readonly options?: WidgetOptions + /** Display-only widgets — replaces the readOnly/opacity DOM fiddling. */ + readonly disabled?: boolean + readonly hidden?: boolean + /** Whether the value is saved in the workflow; prompt inclusion uses options.serialize. */ + readonly serialize?: boolean + } + + export interface WidgetCollection { + readonly length: number + get(name: string): WidgetHandle | undefined + at(index: number): WidgetHandle | undefined + all(): readonly WidgetHandle[] + names(): readonly string[] + /** + * Replaces splice/assign reordering. `names` must be a permutation of the + * current names — a partial list throws rather than silently dropping + * widgets, which is how the array-splice idiom lost them. + */ + reorder(names: readonly string[]): void + move(name: string, toIndex: number): void + /** + * Creates a widget on this node. + * + * The counterpart to `remove` — packs that rebuild a readout widget do + * remove-then-create, and without this only half the operation has a + * destination, which makes the conversion cosmetic. + */ + add(def: WidgetDef): WidgetHandle + /** + * Mounts an element on the node and hands it to the pack to fill. + * + * The replacement for `addDOMWidget`, and the destination for hand-painted + * canvas controls. Across kjnodes' canvas editors the drawing is rectangles, + * images, straight lines and text — all DOM primitives — but a pack that + * wants to keep its existing `ctx` code can append a `` to the + * container and carry it over unchanged. + * + * The gain is not the drawing, it is the input: these editors hand-roll + * hit-testing against bounding boxes because canvas gives them nothing to + * attach a listener to. Mounted in the DOM, pointer events land on the + * element and most of that code goes away. + */ + mount(def: MountDef): WidgetHandle + /** + * A per-node drawing surface, and the destination for `onDrawForeground`. + * + * Works under both renderers without the pack knowing which it is on: the + * canvas is a DOM element, which the legacy renderer positions over the + * graph canvas and Nodes 2.0 renders directly. That is the whole reason it + * is a mounted element rather than a hook into the graph's own context — + * drawing into the shared context is what ties a pack to the old renderer. + * + * `draw` is called on mount, on resize, and whenever `redraw()` is called. + */ + canvas(def: CanvasDef): CanvasHandle + remove(name: string): boolean + [Symbol.iterator](): Iterator + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface WidgetTextSelection { + readonly start: number + readonly end: number + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface WidgetTextEventBase { + readonly value: string + readonly selection: WidgetTextSelection + /** Positions a host menu at the text editor without exposing its element. */ + readonly menuEvent: MouseEvent + /** Commits through the widget protocol and optionally restores the caret. */ + setValue(value: string, selection?: WidgetTextSelection): void + focus(): void + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface WidgetTextInputEvent extends WidgetTextEventBase { + readonly kind: 'input' | 'selection' + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface WidgetTextWheelEvent extends WidgetTextEventBase { + readonly kind: 'wheel' + readonly deltaY: number + readonly ctrlKey: boolean + /** Claims the wheel gesture so the canvas does not pan or zoom. */ + preventDefault(): void + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface WidgetTextKeyEvent extends WidgetTextEventBase { + readonly kind: 'keydown' + readonly key: string + readonly ctrlKey: boolean + readonly altKey: boolean + readonly shiftKey: boolean + readonly metaKey: boolean + readonly repeat: boolean + preventDefault(): void + stopPropagation(): void + } + + /** + * An interaction with a host-owned multiline text editor. + * + * This is the renderer-independent replacement for reaching through + * `widget.inputEl`: packs can inspect the live caret, offer a menu through + * `menuEvent`, replace text, and implement selection-based wheel edits without + * receiving the host's element or markup. + */ + export type WidgetTextInteractionEvent = + | WidgetTextInputEvent + | WidgetTextWheelEvent + | WidgetTextKeyEvent + + /** What a pack-declared widget can hold. */ + export type WidgetTypeData = string | number | boolean | object | null + + /** + * Reading and writing the widget's value, for the renderer to bind to. + * + * @knipIgnoreUnusedButUsedByCustomNodes + */ + export interface WidgetTypeValue { + get(): WidgetTypeData + set(value: WidgetTypeData): void + /** Notified when the value changes for any other reason — a workflow load. */ + onChange(listener: (value: WidgetTypeData) => void): Unsubscribe + } + + export interface WidgetTypeContext { + /** A frozen snapshot of the input declaration's current options. */ + getOptions(): Readonly> + /** + * Runs while the widget's owning node belongs to a graph. + * + * Widget constructors run before a node has an id or graph, so a node handle + * cannot be supplied directly to `render`. The listener runs after the node + * joins a graph and tears down when it leaves. + */ + onNodeReady(listener: (node: NodeHandle) => Unsubscribe | void): Unsubscribe + } + + export interface WidgetTypeDef { + /** Used when the definition supplies none. */ + readonly defaultValue?: WidgetTypeData + /** Height in pixels. Omit to size to content. */ + readonly height?: number + /** Smallest width the control needs, in pixels. */ + readonly minWidth?: number + /** Smallest height the control needs, in pixels. */ + readonly minHeight?: number + /** + * Whether the value is saved and sent. Defaults to `true`: this widget holds + * a real input value, unlike a mounted decoration. + */ + readonly serialize?: boolean + /** + * Fills the container. Return a teardown if the control owns listeners, + * timers or observers. + * + * `name` is the input being rendered — controls commonly label themselves + * with it, which a type-level renderer otherwise has no way to know. + */ + render( + container: HTMLElement, + value: WidgetTypeValue, + name: string, + context: WidgetTypeContext + ): Unsubscribe | void + } + + /** Parsed ComfyUI workflow JSON. */ + export type WorkflowData = Readonly> + + export interface WorkflowHandle { + /** Replaces the active document with parsed ComfyUI workflow JSON. */ + open(data: WorkflowData): Promise + /** Expands the active document's `%date:...%` and `%Node.widget%` tokens. */ + applyTextReplacements(value: string): string + /** The document on screen, or `undefined` before one is open. */ + current(): DocumentHandle | undefined + /** `current()?.id`: the identity of the open document's editing session. */ + documentId(): string | undefined + /** + * Adds a reader for files the host cannot load a workflow from. It is asked + * when such a file is opened or dropped, before the host gives up on it. + */ + registerImporter(importer: WorkflowImporter): Unsubscribe + } + + /** @knipIgnoreUnusedButUsedByCustomNodes */ + export interface WorkflowImportContext { + readonly name: string + readonly type: string + } + + export type WorkflowImportResult = + | { readonly workflow: Readonly> | string } + | { readonly prompt: Readonly> | string } + + /** Reads a workflow or prompt out of a file the host cannot read itself. */ + export interface WorkflowImporter { + /** Namespaced; names the importer in diagnostics. */ + readonly id: string + /** Offered a file matching a type or an extension; every file if neither is given. */ + readonly mimeTypes?: readonly string[] + readonly extensions?: readonly string[] + /** Larger files are not offered. The host never offers more than 16 MiB. */ + readonly maxBytes: number + enabled?(): boolean | Promise + /** Returns nothing to decline the file. */ + parse( + bytes: Uint8Array, + context: WorkflowImportContext + ): + | WorkflowImportResult + | null + | undefined + | Promise + } + + export const comfy: Comfy +} diff --git a/docs/node-api/concepts.md b/docs/node-api/concepts.md new file mode 100644 index 00000000000..2cefe16befd --- /dev/null +++ b/docs/node-api/concepts.md @@ -0,0 +1,223 @@ +# Key concepts + +The published API is organized around definitions, handles, snapshots, scopes, +and host-owned behavior. Understanding those five ideas prevents most migration +mistakes. + +## Definitions and instances are different surfaces + +A node definition describes a type. A node handle addresses one node in one +graph. + +| Surface | Purpose | Typical entry point | +| ---------------- | ----------------------------------------------------------- | ---------------------------------------------- | +| `NodeDef` | Frozen metadata read from a backend or frontend definition. | `comfy.defs.get(type)` | +| `NodeDefBuilder` | Register behavior for every matching instance. | `comfy.defs.extend(selector, apply)` | +| `NodeDefinition` | Declare a frontend-owned node type as plain data. | `comfy.defs.define(definition)` | +| `NodeHandle` | Read or edit one live node. | A lifecycle callback or `comfy.graph.node(id)` | + +Register type behavior at module load. Use an instance handle only after the +node has joined a graph. This is why `onCreated` runs later than a legacy +constructor hook: an ID-backed handle cannot address a node that has no graph or +ID yet. + +## Handles are closed, ID-backed capabilities + +`NodeHandle` and `WidgetHandle` are closed proxy objects. A handle stores public +identity and resolves the current entity on each access; it does not expose the +live internal object. + +Consequences: + +- unknown members are not an escape hatch to internal data; +- a handle does not keep a deleted entity alive; +- mutations pass through host-owned behavior; +- no handle exposes a constructor, prototype, store, renderer, or Vue proxy; +- `isDeleted` is available on every entity handle. + +Keep handles when convenient, but check `isDeleted` before writing through a +long-lived one. A handle belongs to the workflow it came from: after another +workflow loads, including by switching tabs, it reads as deleted. Undo and redo +keep it. Identity reads remain useful after deletion. Other reads return +no value, while a write to a deleted entity throws `ComfyDeletedError` instead +of being silently discarded. + +### Handle identity + +Do not assume object identity across API instances, majors, events, or graph +scopes: + +```js +first === second // only reliable inside one handle cache +comfy.sameEntity(first, second) // supported identity comparison +``` + +`comfy.adopt(handle)` re-resolves a foreign node handle into the current API +instance. It returns `undefined` for a non-handle, a deleted node, or a handle +kind that cannot be adopted independently. + +## Collection reads are snapshots + +List-shaped reads return frozen array snapshots: + +```js +const nodes = comfy.graph.nodes() +const links = node.outputs.get('IMAGE')?.links() ?? [] + +for (const link of links) { + // Safe even though disconnecting changes the live graph. + node.outputs.get('IMAGE')?.disconnect(link.targetNodeId) +} +``` + +The array does not update after it is returned. Ask again when you need current +state. Objects such as `NodeSnapshot`, `SlotSnapshot`, `LinkInfo`, definition +metadata, execution results, and resolver views are also inert, read-only data. + +Use collection operations to mutate live state; never mutate an array returned +by `all()`, `nodes()`, `links()`, `names()`, or similar methods. + +## Graph scope is part of identity + +`comfy.graph` means the graph currently shown by the editor. It can be the root +graph or a subgraph the user entered. + +Use: + +- `comfy.graph.root()` for the document root even while another graph is shown; +- `comfy.graph.subgraphs()` for subgraph definitions; +- `GraphScopeHandle.node(id)` to resolve an ID inside its owning graph; +- `node.graphId` when recording a node outside the callback that supplied it. + +Do not flatten a document into a map keyed only by node ID. Independently +authored subgraph definitions may contain the same IDs. A durable in-memory key +must include graph and node identity. + +Supply and frontend-node resolution also run per graph scope. A supplier inside +a subgraph may feed unconnected inputs in that subgraph, but resolution does not +cross the boundary and create invisible dependencies on outside state. + +## Mutations use host behavior + +Set values through methods such as `setTitle`, `setValue`, `modify`, +`connectTo`, and `replace`. These methods preserve the host behavior associated +with the edit: callbacks, property synchronization, link identity, layout, +redraw, serialization, and undo boundaries as applicable. + +`WidgetHandle.setValue()` is a full programmatic commit. It behaves like a user +edit for the value protocol: + +- writes the value; +- synchronizes a property-backed widget; +- runs the widget callback chain and node widget-change behavior; +- notifies `change` listeners; +- advances graph change state. + +It does not fire `activate`, because activation reports a user act. Writing the +current value again is a no-op. + +Use `graph.batch()` for a synchronous compound edit that should be one undo +step: + +```js +comfy.graph.batch(() => { + const first = comfy.graph.add('CheckpointLoaderSimple') + const second = comfy.graph.add('KSampler') + first.outputs.get('MODEL')?.connectTo(second.id, 'model') +}) +``` + +Do not hold a batch across `await`; unrelated user actions during the wait would +be folded into the pack's edit. + +## Lifecycle is explicit + +An extension module can register definitions, settings, commands, widget types, +and listeners immediately. Use lifecycle signals for state that is not ready at +module evaluation: + +| Signal | Meaning | +| ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | +| `comfy.onReady` | The application, definitions, settings, and graph have finished initial setup. Fires on the next microtask if already ready. | +| `comfy.onWorkflowLoaded` | A workflow has finished loading. Fires for every load, and for undo and redo, which rebuild the same document. | +| `comfy.onDocumentOpened` | A workflow was opened, on screen or in a background tab. | +| `comfy.onDocumentActivated` | A workflow became the one on screen. | +| `comfy.onDocumentDeactivated` | A workflow stopped being the one on screen. It may still be open. | +| `comfy.onDocumentClosed` | A workflow's editing session ended. Release anything keyed to its id. | +| `NodeDefBuilder.onCreated` | One matching node has joined a graph and can be addressed. | +| `NodeDefBuilder.onConfigured` | Saved node data was applied. | +| `NodeDefBuilder.onRemoved` | One matching node left its graph. | + +Most registrations and subscriptions return `Unsubscribe`. Retain it when the +registration has a shorter lifetime than the module or owning node. + +## Observe the narrowest behavior + +Prefer a semantic event to polling, repaint hooks, or broad document scans: + +1. a widget's `change`, `activate`, `textInteraction`, or `beforeSerialize`; +2. a definition lifecycle hook such as `onConnectionsChanged` or `onResized`; +3. root observers such as `onNodeChanged`, `onNodeMoved`, or + `onViewportChanged`; +4. `graph.version` only as an opaque structural-change token. + +`graph.version` includes widget values committed through the host protocol, but +it is still an opaque change token rather than a universal edit log. Never +subtract versions or assume consecutive increments. Data a pack keeps outside +the graph and widget model does not affect it. + +## Saved workflow and queued prompt are separate destinations + +The API distinguishes: + +- graph state in the saved workflow; +- input values sent in the API prompt; +- the workflow embedded into an output generated by that prompt. + +Mounted widgets expose separate `serialize` and `sendToPrompt` flags. Existing +widgets can replace their value for one serialization destination with a +synchronous `beforeSerialize` listener: + +```js +widget.on('beforeSerialize', (event) => { + if (event.context === 'prompt') { + event.setSerializedValue(expandReferences(String(event.value))) + } +}) +``` + +The live widget value is unchanged. Ignoring `event.context` changes all three +destinations. Serialization handlers are synchronous; starting asynchronous +work inside one does not delay the prompt or workflow write. + +## Frontend execution is pure resolution + +A frontend-only node is ordinary editor state that does not reach the backend. +At prompt time it either: + +- is omitted; +- forwards an output to one of its inputs; +- supplies a literal value. + +Resolvers and suppliers receive frozen read views and return data. They do not +mutate the graph or a prompt draft. This keeps prompt construction deterministic +and prevents a failed resolver from leaving the document half-edited. + +Use ordinary node and graph methods for permanent editor actions. Use resolution +only to describe what execution means. + +## Errors are part of the contract + +API failures are plain `Error` subclasses with no internal object attached: + +| Error | Cause | +| ------------------------- | ----------------------------------------------------- | +| `ComfyApiError` | Base class for API failures. | +| `ComfyDeletedError` | A mutation targeted a deleted handle. | +| `ComfyReadonlyError` | Code attempted to assign a read-only public property. | +| `ComfyAmbiguousSlotError` | A name matched more than one slot. | +| `ComfyUnsupportedError` | `require()` named a capability the host lacks. | + +The constructors are not currently exported from `/comfy/api/v2.js`. Prefer +capability checks, `isDeleted`, and `undefined` handling for expected absence; +use ordinary `Error` fields when reporting an unexpected failure. diff --git a/docs/node-api/demo-nodes.md b/docs/node-api/demo-nodes.md new file mode 100644 index 00000000000..d8fd9d558fb --- /dev/null +++ b/docs/node-api/demo-nodes.md @@ -0,0 +1,74 @@ +# DEMO nodes + +The devtools custom-node pack includes two small frontend-only nodes built only +with the published API. Together they form an executable example of definition +registration and prompt-time resolution. + +| Node | Behavior | +| -------------------- | ------------------------------------------------------------------------- | +| `DEMO Constant Text` | Stores a string in the workflow and supplies it as a literal at run time. | +| `DEMO Reroute` | Forwards its input source and does not appear in the backend prompt. | + +Both nodes are ordinary workflow entities: they can be connected, duplicated, +saved, and restored. They use `execution: 'frontend'`, so the backend never +needs a Python class named `DEMO/ConstantText` or `DEMO/Reroute`. + +## Install the demos + +The definitions live in +[`tools/devtools/web/stableApiDemo.js`](../../tools/devtools/web/stableApiDemo.js). +Install the repository's devtools pack into a local ComfyUI checkout: + +```sh +mkdir -p /path/to/ComfyUI/custom_nodes/ComfyUI_devtools +cp -r tools/devtools/* /path/to/ComfyUI/custom_nodes/ComfyUI_devtools/ +``` + +Restart ComfyUI, then search the node library for `DEMO`. The demo module pins +itself to API major 2 with `comfy.forMajor(2)` and requires the capabilities it +uses before registering either type. + +## Run the example + +1. Add `DEMO Constant Text`, `DEMO Reroute`, and `Preview as Text`. +2. Connect constant `text` to reroute `in`. +3. Connect reroute `out` to preview `source`. +4. Change the constant's `value`, then run the workflow. + +`Preview as Text` displays the value. The API prompt contains only the backend +preview node, with the resolved string as its `source` input. The two DEMO nodes +remain in the saved workflow but are intentionally absent from the prompt. + +The checked-in workflow at +[`browser_tests/assets/nodeApi/stable-api-demo.json`](../../browser_tests/assets/nodeApi/stable-api-demo.json) +is the same graph used by the browser test. + +For broader, installable examples of widgets, graph interaction, execution, +backend services, and settings, see the +[How-To node packs](../../examples/node-api/README.md). + +## How the definitions work + +`DEMO Constant Text` resolves its output to a literal: + +```js +resolve: ({ self }) => ({ + text: { literal: String(self.widgetValue('value') ?? '') } +}) +``` + +`DEMO Reroute` resolves its output to its input, or omits the output when it is +not connected: + +```js +resolve: ({ self }) => { + const input = self.input('in') + return { out: input ? { forwardTo: input } : { omit: true } } +} +``` + +Resolvers are pure. They may return directly or return a promise; prompt +execution awaits async resolvers concurrently with a deadline. Synchronous +editor inspection cannot await a promise and treats it as unresolved. Resolvers +do not edit links, remove nodes, or mutate a prompt draft. See [Execution and +resolution](./execution.md) for the complete contract. diff --git a/docs/node-api/execution.md b/docs/node-api/execution.md new file mode 100644 index 00000000000..46170991e8f --- /dev/null +++ b/docs/node-api/execution.md @@ -0,0 +1,295 @@ +# Execution, queueing, and frontend resolution + +The execution API separates permanent graph edits, prompt-time resolution, +queue submission, and backend results. Keeping those stages distinct prevents a +pack from mutating the live document merely to construct a prompt. + +## Queueing a run + +```js +const submitted = await comfy.queue.run() +``` + +`run()` behaves like the host Run action. It resolves when submission finishes, +not when backend execution completes. `false` means another queue call was +already in flight and this call was folded into it. + +Run part of the graph with explicit output nodes: + +```js +await comfy.queue.run({ + nodes: comfy.graph.selection(), + batch: 2 +}) +``` + +The host includes the dependencies that feed those nodes. An empty `nodes` +array is rejected rather than interpreted as “run everything.” + +## Queue lifecycle + +### Before submission + +```js +const stop = comfy.queue.onBeforeRun(() => { + const restore = preparePromptState() + return () => restore() +}) +``` + +The listener runs before the prompt is built. It must be synchronous; the +prompt builder does not await work started here. + +A returned cleanup runs when the attempt ends whether it was submitted, +rejected, canceled, or threw. This pairing exists for legacy behavior that must +temporarily change graph state and restore it, but prefer `beforeSerialize`, a +frontend resolver, or a supplier when those express the intent without graph +mutation. + +### After submission + +```js +const stop = comfy.queue.onAfterRun((event) => { + console.info(event.promptIds) + console.info(event.submissions) + console.info(event.rejected) +}) +``` + +`onAfterRun` means the submission attempt finished. It is not an execution- +complete event. Accepted submissions include prompt IDs and backend node counts; +`rejected` reports how many submissions the backend refused. + +### Validation rejection + +```js +const stop = comfy.queue.onRejected(({ status, error, nodeErrors }) => { + reportValidationFailure(status, error, nodeErrors) +}) +``` + +This event covers a prompt the backend rejects before execution begins. It +includes the top-level error and per-node input validation details. It does not +represent a transport failure or an exception raised after execution starts. + +## Guarding a run + +Use a guard when the decision may be asynchronous and must be made before the +prompt is built: + +```js +const answer = await comfy.ui.prompt({ + label: 'Estimated cost is high. Type RUN to continue.' +}) +const stop = comfy.queue.guard(() => answer === 'RUN') +try { + await comfy.queue.run() +} finally { + stop() +} +``` + +Every registered guard runs; any `false` cancels the attempt. A guard that +throws is treated as allowing the run. All guards share a short host timeout, +after which the run proceeds so one extension cannot make ComfyUI permanently +unrunnable. Do not place an indefinitely blocking dialog behind a guard. + +`onBeforeRun` observes and prepares. `guard` can delay and cancel. Do not use one +as an approximation of the other. + +## Queue state and interruption + +```js +const count = comfy.queue.pending() +const stopPending = comfy.queue.onPendingChanged((next) => updateCount(next)) + +await comfy.queue.interrupt() +const stopInterrupted = comfy.queue.onInterrupted(() => releaseWaiters()) +``` + +`pending()` includes the currently executing run. `interrupt()` stops that run; +it does not clear the remainder of the queue. + +The API also exposes the host's user-facing queue settings: + +```js +comfy.queue.autoQueueMode() // 'disabled' | 'change' | 'instant' +comfy.queue.setAutoQueueMode('change') +comfy.queue.batchCount() +comfy.queue.setBatchCount(4) +comfy.queue.disableAutoQueue() +``` + +Use `disableAutoQueue()` before a self-interrupting conditional workflow so the +automatic runner does not immediately submit it again. + +## Observing backend execution + +The root API resolves backend execution IDs, including nested subgraph paths: + +```js +const current = comfy.executingNode() +const node = comfy.executionNode(executionId) + +const stop = comfy.onExecutingNodeChanged((next) => { + updateRunningBadge(next) +}) +``` + +`executingNode()` is `undefined` between nodes and runs. Use +`executionNode(id)` instead of parsing nested execution IDs or looking up the +visible graph by the final numeric segment. + +For results, register behavior on the node definition: + +```js +comfy.defs.extend('MyPack/Analyzer', (definition) => { + definition.onExecuted((node, result) => { + console.info(result.images, result.text, result.raw) + }) + + definition.onPreview((node, frame) => { + showPreview(node, frame.url) + }) +}) +``` + +`ExecutionResult.raw` preserves custom output keys from the pack's own backend. +`PreviewFrame.url` is an object URL revoked when the next frame arrives; copy or +consume it before retaining a preview beyond that lifetime. + +## Frontend-only nodes + +Frontend nodes remain ordinary editor entities but do not execute on the +backend. Define one with `execution: 'frontend'`, or mark a backend-defined type +with `NodeDefBuilder.setExecution('frontend', resolver?)`. + +A resolver answers what each of its own outputs means: + +```js +comfy.defs.define({ + type: 'MyPack/Reroute', + inputs: [{ name: 'in', type: '*' }], + outputs: [{ name: 'out', type: '*' }], + execution: 'frontend', + resolve({ self }) { + const input = self.input('in') + return { + out: input ? { forwardTo: input } : { omit: true } + } + } +}) +``` + +Each output name maps to one `OutputResolution`: + +```js +{ + omit: true +} +{ + forwardTo: inputRef +} +{ + literal: value +} +``` + +The resolver receives a frozen `ResolveView`: + +- `self` gives the resolver's ID, type, properties, groups, mode, color, own + inputs and outputs, and widget values; +- `nodesOfType(type)` returns other frozen views in the same graph scope; +- `self.input(nameOrIndex)` creates the only reference a resolver may forward. + +Resolution follows chains to a physical backend output, literal, or omission, +with cycle detection. A resolver must be pure. It may return its result directly or as a promise; async resolvers run concurrently, receive an `AbortSignal` on the view, and are abandoned at the resolution deadline, so an awaited call that never settles cannot stall the prompt. It cannot edit +the graph or a prompt draft. + +`InputSlotHandle.resolvedSource()` performs synchronous editor inspection without +changing topology. It cannot await resolver promises; the prompt path awaits +them instead. + +## Suppliers and broadcast behavior + +A supplier is the supply-side counterpart to a resolver. It answers which +unconnected inputs elsewhere in the same graph this node offers to feed: + +```js +comfy.defs.extend('MyPack/BroadcastModel', (definition) => { + definition.setSupply(({ self, unconnectedInputs }) => { + const output = self.outputs.find(({ name }) => name === 'MODEL') + if (!output) return [] + + return unconnectedInputs() + .filter((input) => input.type === 'MODEL') + .map((input) => ({ + to: { nodeId: input.nodeId, input: input.input }, + from: { output: output.index }, + priority: 10 + })) + }) +}) +``` + +A supplied source can be: + +- one of the supplier's own outputs; +- a literal; +- whatever feeds one of the supplier's own inputs (`forwardInput`). + +It cannot name an arbitrary third-party node as a source. A supplier may offer +what it owns, not rewire two bystanders. + +`unconnectedInputs()` exposes matching data needed by real broadcast packs: +slot name, translated label, type, widget-input status, owner title/mode/color, +groups, and frozen owner properties. + +When multiple suppliers claim one input, higher priority wins. Exact priority +ties feed nothing instead of making execution depend on graph order. + +Resolution runs independently in each graph scope. It never crosses a subgraph +boundary. + +## Inspecting winning supplies + +```js +for (const edge of comfy.graph.resolvedSupplies()) { + if (edge.supplierNodeId !== broadcaster.id) continue + if (edge.from.kind !== 'output') continue + + comfy.graph + .node(edge.from.nodeId) + ?.outputs.at(edge.from.output) + ?.connectTo(edge.to.nodeId, { index: edge.to.input }) +} +``` + +This recomputes synchronous resolver and supplier results with the same priority +arbitration as prompt execution. Promise-returning resolvers are unresolved here, +and async suppliers contribute no edge. Use it to convert synchronous virtual +broadcasts to real links. The prompt path also awaits async results, so this +editor inspection may show fewer edges. + +The returned IDs are local to that graph scope. `GraphScopeHandle` offers the +same read for root or subgraph definitions. + +## Prompt-time widget serialization + +Frontend resolution changes topology. A widget's `beforeSerialize` changes one +value for one destination: + +```js +widget.on('beforeSerialize', (event) => { + if (event.context === 'prompt') { + event.setSerializedValue(resolveTemplate(String(event.value))) + } +}) +``` + +Use this for sentinel expansion, prompt templates, rolled seeds, or embedded +reproduction data. It is synchronous and does not mutate the live widget. + +Do not edit the built prompt or workflow snapshot. Use partial queue execution, +frontend resolution, supply, widget serialization, and ordinary graph commands +for the supported intents those internal edits previously combined. diff --git a/docs/node-api/graph.md b/docs/node-api/graph.md new file mode 100644 index 00000000000..4b1dca2c595 --- /dev/null +++ b/docs/node-api/graph.md @@ -0,0 +1,265 @@ +# Graphs and groups + +`comfy.graph` is the editing surface for the graph currently shown to the user. +It provides graph-safe node, selection, viewport, group, link, and mutation +operations without exposing `LGraph` or `LGraphCanvas`. + +## Visible graph, root graph, and subgraphs + +The active view is not always the document root: + +```js +const visible = comfy.graph +const root = comfy.graph.root() +const definitions = comfy.graph.subgraphs() +``` + +| Handle | Scope | +| ---------------------------------------------- | ---------------------------------------------------------------------------------- | +| `GraphHandle` (`comfy.graph`) | The graph currently shown in the editor. Includes editing and viewport operations. | +| `GraphScopeHandle` (`root()` or `subgraphs()`) | One graph definition. Read-oriented: nodes, groups, and resolved supplies. | + +A subgraph entry represents a definition, not each placed instance. If the same +definition is placed three times, it appears once and its internal nodes appear +once. + +Node IDs must be resolved inside their owning graph. Do not collect all document +nodes into a map keyed only by ID: + +```js +function documentNodes() { + const scopes = [comfy.graph.root(), ...comfy.graph.subgraphs()].filter( + Boolean + ) + + return scopes.flatMap((scope) => + scope.nodes().map((node) => ({ graphId: scope.id, node })) + ) +} +``` + +Use `node.graphId` when persisting pack-owned in-memory state about a node. + +## Looking up nodes and links + +```js +const node = comfy.graph.node('42') +const samplers = comfy.graph.nodesOfType('KSampler') +const allNodes = comfy.graph.nodes() +const links = comfy.graph.links() +const mine = comfy.graph.queryNodes({ + scope: 'root-and-subgraphs', + type: /^MyPack\// +}) +``` + +`queryNodes()` is one flat query. `scope` is `'visible'` by default or +`'root-and-subgraphs'` for the root graph and every subgraph definition. +`type`, `rendered` and `group` each narrow the result, and they combine. Under +`'root-and-subgraphs'` each handle comes from its owning graph, so compare with +`comfy.sameEntity()`. + +Every list is a frozen snapshot. `LinkInfo` is inert data with source and target +node IDs, slot IDs, types, and endpoint indexes at snapshot time. Indexes are +volatile; do not store them across slot mutations. Inside a subgraph, links to +the subgraph's own inputs and outputs are listed too; their node ID is the +boundary's, and no node handle exists for it. + +Use `node.inputs` and `node.outputs` to edit connectivity. `graph.links()` is for +inspection, not mutation. + +`graph.nodeAt({ x, y })` returns the topmost node at a graph-space point using +the rendered layout and z-order. It can find nothing before the first render. + +## Creating and removing nodes + +```js +const node = comfy.graph.add('KSampler', { + title: 'Preview sampler', + position: { x: 360, y: 220 } +}) + +const copy = comfy.graph.duplicate(node.id, { x: 680, y: 220 }) +comfy.graph.remove(node.id) +``` + +`add()` constructs through the registered definition. It throws when the type +does not exist. `duplicate()` carries serializable widgets and properties but +does not copy links. It returns `undefined` when the source is gone or cannot be +constructed. + +`NodeHandle.remove()` removes its own node. `graph.remove(id)` is the equivalent +when only an ID is available. + +## Replacing a node + +Node type is identity and is read-only. Rebuild with `replace()`: + +```js +const replacement = comfy.graph.replace(node.id, 'KSamplerAdvanced') +``` + +The operation carries position, a user-customized title, colors, mode, +compatible properties, widget values by name, and every link that still fits. +It matches slots by name first and by index as a fallback. Incompatible links +are dropped with a warning rather than forced into the wrong slot. The complete +replacement is one undo step. + +Replacing with the same type is useful when a refreshed definition changed and +an existing node must be rebuilt without discarding its state. + +## Compound edits and undo + +Use `batch()` for one synchronous user operation: + +```js +comfy.graph.batch(() => { + const loader = comfy.graph.add('CheckpointLoaderSimple', { + position: { x: 100, y: 100 } + }) + const sampler = comfy.graph.add('KSampler', { + position: { x: 500, y: 100 } + }) + loader.outputs.get('MODEL')?.connectTo(sampler.id, 'model') + comfy.graph.select([loader, sampler]) +}) +``` + +The scope closes even when the callback throws. It is synchronous by design; +never include an `await` inside it. + +## Selection and viewport + +```js +const selected = comfy.graph.selection() +comfy.graph.select([node]) +comfy.graph.select([another], { add: true }) +comfy.graph.select([]) // clear + +comfy.graph.centerOn(node) +comfy.graph.setZoom(1.25) +comfy.graph.panBy({ x: 120, y: 0 }) +const pointer = comfy.graph.pointerPosition() + +const stop = comfy.onSelectionChanged((nodes) => showDetails(nodes)) +``` + +Selection and viewport methods address the visible graph and active editor. +`select()` and `centerOn()` skip a handle from another graph, such as a +subgraph that is not open. +`pointerPosition()` is in graph coordinates and can return `undefined` when no +canvas is available. `centerOn()` does not change zoom. `panBy()` moves the view +by a distance in CSS pixels, the same at every zoom. `onSelectionChanged()` +passes the complete node selection in the visible graph each time it changes; +selecting only a group or reroute does not notify. + +For a panel anchored to a node, read `node.getScreenRect()` and update it when +the viewport changes: + +```js +const stop = comfy.onViewportChanged(() => { + const rectangle = node.getScreenRect() + if (rectangle) positionPanel(rectangle) +}) +``` + +Do not read or reconstruct canvas pan, scale, offsets, title height, or device +pixel ratio. + +## Groups + +```js +for (const group of comfy.graph.groups()) { + console.info(group.id, group.getTitle(), group.nodes().length) +} +``` + +`GroupHandle` provides: + +- `isDeleted`; +- `getTitle()` / `setTitle()`; +- `getColor()` / `setColor()`; +- `nodes()` for the nodes geometrically contained by the group; +- `getBounds()` in graph space; +- `centerOn()`, which pans only while the group's graph is on screen. + +Groups are derived rectangles, not parents that own a stored child list. Ask +`nodes()` again after layout changes. A kept group handle follows its group +through undo and redo, and reads as deleted once the group is removed or +another workflow loads. A subgraph scope's `groups()` reads groups +inside that definition. + +## Structural version token + +`graph.version` changes when graph-visible state changes, including node or slot +structure, connections, node flags, and widget values committed through the +host protocol. + +Treat it as an opaque token: + +```js +const before = comfy.graph.version +await refreshExternalData() +if (comfy.graph.version !== before) rebuildIndex() +``` + +Do not subtract versions, assume increments of one, or use it as a replayable +event log. Pack state held outside the graph and widget model does not affect +it. A custom canvas widget whose external drawing data changed should call its +own `redraw()`. + +## Node and graph observation + +There is no per-frame tick. Observe the semantic operation: + +### Node property changes + +```js +const stop = comfy.onNodeChanged( + ({ graphId, node, property, from, to }) => { + updateIndex(graphId, node.id, property, from, to) + }, + { scope: 'document' } +) +``` + +The default scope is `visible`. `scope: 'document'` includes the root and every +subgraph definition. Each event names its graph because node ID alone is not a +document-wide key. + +The tracked fields are title, mode, foreground color, background color, shape, +and advanced-widget visibility. Position uses the movement stream instead +because it changes continuously during a drag. + +### Movement and drag completion + +```js +const stopMove = comfy.onNodeMoved(({ node, position }) => { + updateGuide(node, position) +}) + +const stopEnd = comfy.onNodeDragEnd((nodes) => { + commitDropBehavior(nodes) +}) +``` + +A pack that moves nodes from its own movement handler must guard against +re-entry. `onNodeDragEnd` is available under Nodes 2.0; the legacy renderer does +not publish a drag lifecycle. + +### Editor interaction state + +`comfy.isInteracting()` reports whether the editor is already handling a link, +node, or widget gesture. A pack starting its own pointer gesture should stand +down while it is true. + +## Resolved supply inspection + +`graph.resolvedSupplies()` synchronously resolves suppliers with the priority +arbitration used by prompt construction. It returns winning graph-local edges +without mutating the graph. Promise-returning resolvers are unresolved here, and +async suppliers contribute no edge; prompt construction awaits those results. +Use this inspection to materialize synchronous broadcasts as real links. +Exact-priority ties are absent, matching execution. + +See [Execution and resolution](./execution.md) for supplier semantics. diff --git a/docs/node-api/how-to.md b/docs/node-api/how-to.md new file mode 100644 index 00000000000..6f585c54bd7 --- /dev/null +++ b/docs/node-api/how-to.md @@ -0,0 +1,1260 @@ +# Migration how-to: LiteGraph intent to the published API + +This guide answers “I used to do X through LiteGraph or `app`; how do I preserve +that behavior through the published API?” It is based on the custom-node +conversion corpus, including cases where the correct replacement was not the +API with the most similar name. + +The published entry point is always: + +```js +import { comfy } from '/comfy/api/v2.js' +``` + +Use this guide for live frontend behavior. If an object came from `JSON.parse`, +a workflow file, `graphToPrompt`, or a backend response, it is data rather than +a live node. Do not replace fields inside serialized data with handle methods. + +## Start with the behavior + +Before choosing a replacement, state what the old code accomplishes for the +user. One old mechanism often served several unrelated purposes: + +| Old mechanism | Possible intent | Published destination | +| ----------------------------------- | ------------------------------ | ------------------------------------------------------ | +| `onDrawForeground` | Show a small status label | `node.addBadge()` | +| `onDrawForeground` | Draw a control | `node.widgets.canvas()` | +| `onDrawForeground` | Enforce node size | `node.setSizeConstraints()` | +| `onDrawForeground` | Poll a value | A widget, node, graph, or viewport event | +| `onDrawForeground` | Keep DOM aligned | `node.widgets.mount()` and `comfy.onViewportChanged()` | +| `getCustomWidgets` | Render one backend input type | `comfy.defs.defineWidgetType()` | +| `getCustomWidgets` | Add a surface to one live node | `node.widgets.mount()` | +| `widgets.splice()` | Reorder widgets | `widgets.reorder()` or `widgets.move()` | +| Remove and reinsert the same widget | Invalidate changed options | `widget.setOption()` | +| `node.type = value` | Defensive self-assignment | Delete the write | +| `node.type = newType` | Replace the node type | `comfy.graph.replace()` | +| `widget.callback(value)` | Commit a new value | `widget.setValue(value)` | +| Capture and chain `widget.callback` | Observe changes | `widget.on('change', listener)` | +| Button `widget.callback()` | Run an action | `widget.on('activate', listener)` | + +Do not migrate by spelling alone. Trace what the callback reads, what it writes, +when it runs, and which saved or queued bytes it affects. + +## Quick lookup by legacy surface + +This index gives the usual destination. Follow the detailed recipe whenever the +row names more than one destination or changes saved workflow topology. + +| Legacy surface | Published destination | +| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | +| `app.registerExtension(...)` | Split registration across `comfy.defs`, `settings`, `commands`, UI, and lifecycle APIs | +| `beforeRegisterNodeDef` | `comfy.defs.extend(selector, builder => ...)` | +| `nodeType.prototype.onNodeCreated` | `builder.onCreated()` | +| `nodeType.prototype.onConfigure` | `builder.onConfigured()` | +| `nodeType.prototype.onExecuted` | `builder.onExecuted()` | +| `nodeType.prototype.onRemoved` | `builder.onRemoved()` | +| `nodeType.prototype.onConnectionsChange` | `builder.onConnectionsChanged()` | +| `nodeType.prototype.onDragOver` / `onDragDrop` | `builder.onDragOver()` / `builder.onDrop()` | +| `nodeType.prototype.onSerialize` | `builder.onSerialize()` for pack-owned fields | +| `extends LGraphNode` / `registerNodeType` | `comfy.defs.define()` | +| `app.graph._nodes` | `comfy.graph.nodes()` | +| `graph.getNodeById(id)` | `comfy.graph.node(id)` or a scoped `graph.node(id)` | +| `graph._groups` | `comfy.graph.groups()` or a scoped `graph.groups()` | +| `canvas.selected_nodes` | `comfy.graph.selection()` | +| `canvas.selectNode(...)` | `comfy.graph.select(...)` | +| `canvas.centerOnNode(node)` | `comfy.graph.centerOn(node)` | +| `LiteGraph.createNode()` plus `graph.add()` | `comfy.graph.add()` | +| `node.clone()` plus `graph.add()` | `comfy.graph.duplicate()` | +| `graph.remove(node)` | `node.remove()` or `comfy.graph.remove()` | +| Replace or retype a live node | `comfy.graph.replace()` | +| `graph.beforeChange()` / `afterChange()` | `comfy.graph.batch()` | +| `graph._version++` | Delete the write; published mutations update `graph.version` | +| Document pointer listeners for a node gesture | `comfy.onNodeMoved()` and, in Nodes 2.0, `comfy.onNodeDragEnd()` | +| `canvas.connecting_links` / `resizing_node` | `comfy.isInteracting()` | +| `node.pos` / `node.size` | `getPosition()` / `setPosition()` and `getSize()` / `setSize()` | +| `node.getBounding()` | `node.getBounds()` | +| `node.getConnectionPos()` | `node.getSlotPosition()` | +| Canvas transform and `graph_mouse` | `node.getScreenRect()` and `comfy.graph.pointerPosition()` | +| `input.link` | `input.isConnected`, `input.link()`, `input.source()`, or `input.disconnect()` | +| `output.links` | `output.links()`, `output.connectTo()`, or `output.disconnect()` | +| Disconnect then reconnect to another output | `output.moveLinksTo()` | +| `node.addInput()` / `addOutput()` | `node.inputs.add()` / `node.outputs.add()` | +| `node.removeInput()` / `removeOutput()` | `node.inputs.remove()` / `node.outputs.remove()` | +| Direct slot name, type, label, or shape writes | `slot.modify()` | +| Mutate slot arrays to reorder | `node.inputs.reorder()` / `node.outputs.reorder()` | +| `LiteGraph.isValidConnection()` | `comfy.defs.isTypeCompatible()` | +| Read or merge a connected Primitive's config | `input.widgetConfig()` / `input.mergeWidgetConfig()` | +| Patch `connectByType` for a link dropped on a node | `builder.onUnplacedLink()` | +| `widget.value = value; widget.callback(value)` | `widget.setValue(value)` | +| Replace or chain `widget.callback` | `widget.on('change')` or `widget.on('activate')` | +| `widget.type = 'converted-widget'` | `widget.setHidden(true)` | +| `widget.options[key] = value` | `widget.setOption(key, value)` | +| `widget.disabled = value` | `widget.setDisabled(value)` | +| `widget.linkedWidgets` | `widget.linked()` / `widget.setLinked()` | +| `widget.computeSize` | `widget.setHeight()` or node size constraints | +| `widgets.push()` / `removeWidget()` | `widgets.add()` / `widgets.remove()` | +| `widgets.splice()` / replace widget array | `widgets.move()` / `widgets.reorder()` after classifying intent | +| `node.addDOMWidget()` | `node.widgets.mount()` | +| `widget.inputEl` | `textInteraction` for a host editor, or `widgets.mount()` for a pack control | +| `getCustomWidgets` for an input type | `comfy.defs.defineWidgetType()` | +| `widget.serializeValue` | Widget serialization flags or `beforeSerialize` | +| Name-keyed object written into `widgets_values` | Delete the write; use named handles and migrate old data in `onConfigured()` | +| Custom upload UI beside an upload-declared input | Delete it; use the host's built-in upload control | +| `node.imgs = ...` for pack drawing | `node.widgets.canvas()` and `CanvasHandle.redraw()` | +| Draw status text in node chrome | `node.addBadge()` | +| `onDrawForeground` / `onDrawBackground` | Canvas widget, semantic event, mount lifecycle, or size constraint based on intent | +| `canvas.setDirty()` / `setDirtyCanvas()` | Usually delete; use `CanvasHandle.redraw()` for external canvas-widget data | +| Read `node.imgs` / `imageIndex` from another node | `getOutputImages()` / `getDisplayedImageIndex()` | +| `getExtraMenuOptions` | `builder.addMenuItem()` | +| `new LiteGraph.ContextMenu(...)` | `comfy.ui.showMenu()` when the pack owns the triggering gesture | +| Inject sidebar, top-bar, action-bar, or modal DOM | `comfy.ui` contributions | +| `app.ui.settings` / extension `settings` | `comfy.settings` | +| Replace the canvas background draw hook | Write the core `Comfy.Canvas.BackgroundImage` setting | +| Extension `commands`, keybindings, and toast | `comfy.commands` | +| `api.fetchApi()` / `api.apiURL()` | `comfy.backend.fetch(relativePath)` for own declared endpoints; named `comfy.*` methods for shared operations | +| `api.addEventListener()` for pack messages | `comfy.backend.on()` | +| Global preview and executing listeners | Definition `onPreview()` or root execution observers | +| `app.queuePrompt()` | `comfy.queue` plus serialization, resolution, or supply for the actual intent | +| Read or mutate queue setting stores | `queue.autoQueueMode()`, `setAutoQueueMode()`, `batchCount()`, or `setBatchCount()` | +| `graphToPrompt` wrapper used only for a sidecar | Pack backend route, partial queueing, and correlated execution events | +| `app.loadGraphData()` | `comfy.workflow.open()` | +| `app.applyTextReplacements()` | `comfy.workflow.applyTextReplacements()` | +| `localStorage` or direct user-data APIs | `comfy.storage` for user-authored pack documents | +| `LGraphCanvas.node_colors[name]` | `comfy.defs.nodeColor(name)` | +| Read or write pack link-type colors | `comfy.defs.typeColor()` / `setTypeColor()` | + +## Registration and lifecycle + +### How do I replace `app.registerExtension()`? + +Split the registration by behavior. The old extension object combined node +definition hooks, commands, settings, custom widgets, and application +lifecycle. The published API gives each one an explicit owner. + +| Legacy extension field | Published API | +| -------------------------------------- | ---------------------------------------------- | +| `beforeRegisterNodeDef` | `comfy.defs.extend()` | +| `registerCustomNodes` | `comfy.defs.define()` | +| `getCustomWidgets` | `comfy.defs.defineWidgetType()` | +| `settings` | `comfy.settings.declare()` | +| `commands` and keybindings | `comfy.commands.register()` | +| `setup` for registration | Run at module scope | +| `setup` that needs the initialized app | `comfy.onReady()` | +| Workflow setup | Definition hooks or `comfy.onWorkflowLoaded()` | + +Old: + +```js +app.registerExtension({ + name: 'MyPack.SamplerTools', + beforeRegisterNodeDef(nodeType, nodeData) { + if (nodeData.name !== 'KSampler') return + + const previous = nodeType.prototype.onNodeCreated + nodeType.prototype.onNodeCreated = function () { + previous?.apply(this, arguments) + installSamplerTools(this) + } + } +}) +``` + +Published API: + +```js +comfy.defs.extend('KSampler', (definition) => { + definition.onCreated((node, event) => { + installSamplerTools(node, event) + }) +}) +``` + +Definition callbacks compose. Do not capture and invoke a previous prototype +method. + +### How do I run code after a node is restored? + +Use `onCreated` for a live, addressable node and inspect its event: + +```js +comfy.defs.extend('MyPack/Node', (definition) => { + definition.onCreated((node, event) => { + if (!event.restored) initializeFreshNode(node) + if (event.loading) restoreDocumentResources(node) + }) + + definition.onConfigured((node, savedData) => { + migratePackOwnedState(node, savedData) + }) +}) +``` + +`restored` covers nodes carrying saved state, including duplicate and paste. +`loading` distinguishes a workflow load. Use `onConfigured` when the behavior +needs the saved node record. + +### How do I keep private state on a node? + +Do not assign custom fields to `LGraphNode` or `NodeHandle`. Keep pack state in +a map and include graph scope in its key: + +```js +const state = new Map() +const keyOf = (node) => `${node.graphId}:${node.id}` + +comfy.defs.extend('MyPack/Node', (definition) => { + definition.onCreated((node) => { + state.set(keyOf(node), { expanded: false }) + }) + + definition.onRemoved((node) => { + state.delete(keyOf(node)) + }) +}) +``` + +## Nodes and graphs + +### How do I read or change node state? + +Handles use methods so reads can resolve current store state and writes can use +host mutation behavior. + +| Legacy live-node access | Published API | +| ----------------------- | ---------------------------------------------------------- | +| `node.title` | `node.getTitle()` / `node.setTitle(title)` | +| `node.mode` | `node.getMode()` / `node.setMode(mode)` | +| `node.flags.collapsed` | `node.isCollapsed()` / `node.setCollapsed(value)` | +| `node.flags.pinned` | `node.isPinned()` / `node.setPinned(value)` | +| `node.color` | `node.getColor()` / `node.setColor(color)` | +| `node.bgcolor` | `node.getBgColor()` / `node.setBgColor(color)` | +| `node.shape` | `node.getShape()` / `node.setShape(shape)` | +| `node.properties[name]` | `node.getProperty(name)` / `node.setProperty(name, value)` | +| `node.pos` | `node.getPosition()` / `node.setPosition(point)` | +| `node.size` | `node.getSize()` / `node.setSize(size)` | +| `{ ...node }` | `node.snapshot()` | + +Assigning a new property to a closed handle is not a mutation API. Use the +setter, or a pack-owned map when the value is not entity state. + +### How do I enumerate or find nodes? + +```js +const visibleNodes = comfy.graph.nodes() +const sampler = comfy.graph.node(nodeId) +const samplers = comfy.graph.nodesOfType('KSampler') +const selected = comfy.graph.selection() +``` + +These replace `app.graph._nodes`, `app.graph.getNodeById()`, +`canvas.selected_nodes`, and renderer selection stores for node selection. +Returned arrays are frozen snapshots. + +`comfy.graph` is the graph currently visible. For document-wide work, preserve +scope rather than flattening IDs: + +```js +const scopes = [comfy.graph.root(), ...comfy.graph.subgraphs()].filter(Boolean) + +for (const scope of scopes) { + for (const node of scope.nodes()) indexNode(scope.id, node) +} +``` + +### How do I add, clone, remove, or replace a node? + +| Legacy operation | Published API | +| --------------------------------------------------- | ------------------------------------------------ | +| `LiteGraph.createNode(type)` plus `graph.add(node)` | `comfy.graph.add(type, init)` | +| `graph.add(node.clone())` | `comfy.graph.duplicate(node.id, position)` | +| `graph.remove(node)` | `node.remove()` or `comfy.graph.remove(node.id)` | +| Delete and recreate to refresh a definition | `comfy.graph.replace(node.id, node.type)` | +| Change `node.type` | `comfy.graph.replace(node.id, newType)` | + +```js +const replacement = comfy.graph.replace(node.id, 'KSamplerAdvanced') +``` + +`replace()` carries compatible state and links and groups the operation into one +undo step on the visible graph. A defensive write such as +`node.type = node.type ?? undefined` has no user behavior and should be removed +instead of converted. + +### How do I make several edits one undo step? + +Replace `graph.beforeChange()` / `graph.afterChange()` pairs with a synchronous +scope: + +```js +comfy.graph.batch(() => { + const first = comfy.graph.add('CheckpointLoaderSimple') + const second = comfy.graph.add('KSampler') + first.outputs.get('MODEL')?.connectTo(second.id, 'model') + comfy.graph.select([first, second]) +}) +``` + +Do not hold a batch across `await`. + +### How do I respond to graph changes without polling? + +Choose the narrowest semantic signal: + +| Intent | Published signal | +| -------------------------------------- | ----------------------------------- | +| A widget value changed | `widget.on('change', listener)` | +| A connection changed | `definition.onConnectionsChanged()` | +| A property changed | `definition.onPropertyChanged()` | +| Any visible or document node changed | `comfy.onNodeChanged()` | +| A node moved | `comfy.onNodeMoved()` | +| A Nodes 2.0 drag ended | `comfy.onNodeDragEnd()` | +| Pan, zoom, or viewport resize | `comfy.onViewportChanged()` | +| Coarse “did graph state change?” check | Compare `comfy.graph.version` | + +Do not increment `graph._version` yourself. Published mutations and committed +widget values advance host change state. Treat `graph.version` as an opaque +token, not a counter or event log. + +### How do I accept a file dropped from the browser? + +Register the behavior on the node definition instead of replacing renderer +drop methods: + +```js +comfy.defs.extend('MyPack/ImageNode', (definition) => { + definition.onDragOver((_node, event) => + Array.from(event.dataTransfer?.types ?? []).includes('Files') + ) + + definition.onDrop(async (node, event) => { + const file = event.dataTransfer?.files?.[0] + if (!file) return false + + const uploadedName = await uploadFile(file) + node.widgets.get('image')?.setValue(uploadedName) + return true + }) +}) +``` + +Returning `true` from `onDragOver` asks both renderers to present and route the +drop. Returning `true` from `onDrop` claims it; handlers after the claimant do +not run. The host gets the first opportunity, so this extends file handling +rather than replacing behavior another pack or core already owns. + +## Slots and links + +### How do I inspect an input connection? + +| Legacy input access | Published API | +| ---------------------------------------------- | ------------------------ | +| `input.link != null` | `input.isConnected` | +| `graph.links[input.link]` | `input.link()` | +| Find the immediate source | `input.source()` | +| Follow frontend reroutes and virtual values | `input.resolvedSource()` | +| Determine the arriving type through a boundary | `input.connectedType` | + +```js +const input = node.inputs.get('model') +if (input?.isConnected) { + console.info(input.source()) + console.info(input.resolvedSource()) +} +``` + +Use the physical source for graph editing and the resolved source for reasoning +about execution. + +### How do I connect or disconnect slots? + +```js +const output = source.outputs.get('IMAGE') + +output?.connectTo(target.id, 'image') +target.inputs.get('image')?.disconnect() +output?.disconnect(target.id) +output?.disconnect() +``` + +These replace editing `input.link`, pushing into `output.links`, and mutating +the graph's link map. Normal compatibility checks and connection-veto hooks +still run. + +### How do I add or remove dynamic inputs and outputs? + +```js +comfy.defs.extend('MyPack/MultiImage', (definition) => { + definition.onConnectionsChanged((node) => { + const last = node.inputs.at(node.inputs.length - 1) + if (last?.isConnected) { + node.inputs.add(`image_${node.inputs.length + 1}`, 'IMAGE') + } + }) +}) +``` + +Use `node.inputs.add/remove` and `node.outputs.add/remove` instead of +`addInput`, `removeInput`, `addOutput`, or array mutation. Removing a slot +disconnects its links through the host. + +### How do I rename or retype a slot? + +Use one atomic patch: + +```js +node.outputs.get('value')?.modify({ + name: 'model', + label: 'MODEL', + type: 'MODEL', + shape: 'directional' +}) +``` + +This replaces direct writes to `slot.name`, `slot.type`, `slot.label`, and +renderer shape fields. Existing links remain attached. + +### How do I reorder slots safely? + +```js +node.inputs.reorder(['model', 'positive', 'negative', 'latent_image']) +``` + +The list must be a complete permutation. The host updates serialized endpoint +indexes while preserving which logical slots the links reach. + +### How do I move links to another output? + +```js +node.outputs.get('old_output')?.moveLinksTo('new_output') +``` + +Use this instead of disconnecting and reconnecting. It preserves link IDs, +which is required for workflow wire compatibility. + +### How do I veto or observe a proposed connection? + +Use `onBeforeConnect` only when the old hook can refuse: + +```js +comfy.defs.extend('MyPack/StrictInput', (definition) => { + definition.onBeforeConnect((_node, event) => { + if ( + event.side === 'input' && + event.peerType && + !comfy.defs.isTypeCompatible(event.peerType, 'MODEL') + ) { + return false + } + }) +}) +``` + +If the old `onConnectInput` or `onConnectOutput` never returned `false`, it was +an observer. Use `onConnectionsChanged` instead. + +### How do I preserve widget-backed input constraints? + +A socket converted from a widget carries the declaration a connected +Primitive node should render. Read or intersect that declaration through the +input handle: + +```js +const input = node.inputs.get('seed') +const current = input?.widgetConfig() + +const merged = input?.mergeWidgetConfig({ + type: 'INT', + options: { min: 0, max: 0xffffffffffff, step: 1 } +}) + +if (!merged) reportIncompatibleDeclarations(current) +``` + +`mergeWidgetConfig()` updates the declaration only when the two widget types +are compatible. It returns `undefined` and leaves the input unchanged when +they are not. For a dynamic socket that represents one of the node's widgets, +pass both `widget` and `widgetConfig` to `inputs.add()`, or apply them together +with `input.modify()`. The widget name must already exist on that node. + +### How do I handle a link dropped on a node when no one slot fits? + +Use `onUnplacedLink` when a node supports expanding a bundle or pipe into +multiple connections: + +```js +comfy.defs.extend('MyPack/ContextPipe', (definition) => { + definition.onUnplacedLink((node, event) => { + if (event.type !== 'CONTEXT' || event.side !== 'output') return false + + return connectContextToPeer(node, { + nodeId: event.peerNodeId, + outputIndex: event.peerIndex, + replaceExisting: event.replaceExisting + }) + }) +}) +``` + +The callback must make the connections through slot handles and return `true` +only when it placed the link. Both ends can be offered the gesture and the +first claimant wins. This replaces a global `connectByType` patch without +changing link-routing behavior for every other node in the document. + +## Widgets + +### How do I read and write a widget value? + +```js +const widget = node.widgets.get('seed') +const previous = widget?.getValue() +widget?.setValue(42) +``` + +`setValue()` replaces both a bare `widget.value = value` and the common pair +`widget.value = value; widget.callback?.(value)`. It uses the host's commit +protocol: property synchronization, host callback behavior, node change +behavior, one `change` event, and graph change tracking. + +An equal value is a no-op. A programmatic write does not fire `activate`. + +### How do I replace a chained `widget.callback`? + +Observe values additively: + +```js +const stop = widget.on('change', (value, previous) => { + refreshPreview(value, previous) +}) +``` + +For an action widget such as a button: + +```js +node.widgets.get('refresh')?.on('activate', () => { + refreshModels() +}) +``` + +Do not capture, replace, or call through another callback. If old code invoked +its own handler once to initialize state, call that handler directly after +subscribing. + +### How do I hide a converted widget? + +```js +node.widgets.get('seed')?.setHidden(true) +``` + +This replaces `widget.type = 'converted-widget'` and the bookkeeping that +restored the old type and size function. Hiding and serialization are separate: +`setHidden()` retains the widget's existing saved and prompt behavior. Audit any +old `serializeValue` override separately. + +### How do I add, remove, or reorder widgets? + +| Legacy mutation | Published API | +| --------------------------------------------------- | ---------------------------- | +| `widgets.push(widget)` | `widgets.add(definition)` | +| `widgets.splice(index, 1)` | `widgets.remove(name)` | +| Move with `splice()` | `widgets.move(name, index)` | +| Replace `node.widgets` with a sorted array | `widgets.reorder(names)` | +| Remove and insert the same widget at the same index | Usually `widget.setOption()` | + +```js +node.widgets.add({ type: 'button', name: 'refresh', value: null }) +node.widgets.move('refresh', 0) +node.widgets.remove('refresh') +``` + +`reorder()` requires every current widget name exactly once. It cannot silently +drop widgets or bypass teardown. + +### How do I change widget options? + +```js +widget.setOption('values', refreshedModels) +widget.setOption('min', 0) +widget.setOption('max', 100) +widget.setLabel('Model') +widget.setDisabled(true) +``` + +`getOptions()` returns a frozen view. Do not mutate it or remove and reinsert a +widget to make the renderer notice changes. + +### How do I replace `addDOMWidget()` or `widget.inputEl`? + +For a pack-owned control on one node, mount into a host-owned container: + +```js +function addNotes(node) { + let release + + return node.widgets.mount({ + name: 'notes', + defaultValue: '', + serialize: true, + sendToPrompt: false, + render(container, value) { + const textarea = document.createElement('textarea') + textarea.value = String(value.get()) + + const onInput = () => value.set(textarea.value) + textarea.addEventListener('input', onInput) + const stopValue = value.onChange((next) => { + textarea.value = String(next) + }) + container.append(textarea) + + release = () => { + textarea.removeEventListener('input', onInput) + stopValue() + } + }, + destroy() { + release?.() + } + }) +} +``` + +The host controls mounting, zoom visibility, and removal. Clean up listeners, +timers, and observers in `destroy()`. Keep one mount per serialized legacy +widget cell unless you have proved that consolidating them preserves +`widgets_values`. + +For behavior on a host-owned multiline editor, do not mount a duplicate merely +to access its element. Subscribe to `textInteraction`: + +```js +widget.on('textInteraction', (event) => { + if (event.kind === 'keydown' && event.key === 'Enter' && event.ctrlKey) { + event.preventDefault() + runText(event.value) + } +}) +``` + +### How do I replace `getCustomWidgets` for a backend input type? + +Register the renderer for that type before nodes are constructed: + +```js +comfy.defs.defineWidgetType('MY_PACK_COLOR', { + defaultValue: '#ffffff', + render(container, value, name, context) { + const input = document.createElement('input') + input.type = 'color' + input.value = String(value.get()) + input.ariaLabel = name + + const onInput = () => value.set(input.value) + const stopValue = value.onChange((next) => { + input.value = String(next) + }) + input.addEventListener('input', onInput) + container.append(input) + + const stopReady = context.onNodeReady((node) => + installNodeBehavior(node, input) + ) + return () => { + stopReady() + stopValue() + input.removeEventListener('input', onInput) + input.remove() + } + } +}) +``` + +Do not add a second mounted widget for the same input. A type renderer preserves +the original positional `widgets_values` cell and keeps the input a widget +instead of turning it into a socket. + +### How do I replace `widget.serializeValue`? + +Classify the behavior: + +| Old behavior | Published mechanism | +| ------------------------------------------- | ------------------------------------------------------------------- | +| Returns `undefined`: saved but not sent | `options: { serialize: false }`, or `setOption('serialize', false)` | +| Pack-owned widget is not saved | Create it with `serialize: false`; it is still sent | +| Mounted value saves but is not sent | `serialize: true`, `sendToPrompt: false` | +| Substitute a value for one destination | `beforeSerialize` | +| Asynchronously prepare data before queueing | `comfy.queue.guard()` followed by a normal committed value | + +```js +widget.on('beforeSerialize', (event) => { + if (event.context === 'prompt') { + event.setSerializedValue(expandReferences(String(event.value))) + } +}) +``` + +The three contexts are `workflow`, `prompt`, and `embedded`. The handler is +synchronous and changes only that write, not the live value. + +### How do I migrate a name-keyed `widgets_values` override? + +Delete the live override. Runtime widget identity is already the widget name: + +```js +node.widgets.get('seed')?.setValue(nextSeed) +``` + +The positional array remains the workflow wire format and is host-owned. Do +not replace it with a record, rewrite it during serialization, or mirror live +values into it. + +Keep compatibility with workflows the old pack already saved. The host applies +normal positional values before `onConfigured`, and that hook also receives +the original saved node data. Read the pack's old record there, translate +renamed or retired keys, and commit the recovered values through named widget +handles. That migration is user-data compatibility; the live serialization +override is not. + +```js +comfy.defs.extend('MyPack/LegacyNode', (definition) => { + definition.onConfigured((node, saved) => { + const legacy = saved.widgets_values + if (!legacy || Array.isArray(legacy) || typeof legacy !== 'object') return + + for (const [oldName, value] of Object.entries(legacy)) { + const currentName = renamedWidgets[oldName] ?? oldName + node.widgets.get(currentName)?.setValue(value) + } + }) +}) +``` + +### How do I migrate a custom upload widget? + +Inspect the Python input declaration first. If its options already include +`image_upload`, `animated_image_upload`, `video_upload`, or `audio_upload`, the +host supplies the chooser, upload, value commit, and preview. Remove the +duplicate frontend widget. + +Do not replace it with `widgets.mount()` merely because the original used DOM. +An extra widget adds a positional `widgets_values` cell and can shift every +later value. Mount only when the pack provides behavior beyond the declared +upload contract, and preserve the original serialized-cell count. + +## Drawing, geometry, and editor interaction + +### How do I replace an `onDraw*` callback? + +Inspect its body first. + +#### It draws a small status label + +Use a badge when the content belongs in node chrome rather than in an +interactive widget: + +```js +const removeBadge = node.addBadge(() => ({ + text: currentStatus(node) +})) +``` + +The function is evaluated while drawing, so it must return quickly. Keep the +returned removal function when the badge has a shorter lifetime than the node. + +#### It actually draws + +Move the drawing and hit testing to a pack-owned canvas widget: + +```js +const surface = node.widgets.canvas({ + name: 'status', + height: 32, + draw(context, [width, height], theme) { + context.fillStyle = theme.surface + context.fillRect(0, 0, width, height) + context.fillStyle = theme.text + context.fillText(currentStatus(node), 8, height / 2) + }, + onPointerDown({ event }) { + if (event.button === 0) openStatus(node) + } +}) + +surface.redraw() +``` + +This renders under both the legacy renderer and Nodes 2.0. Use the supplied +theme instead of LiteGraph color constants. + +#### It enforces size + +Declare the constraint once: + +```js +node.setSizeConstraints({ + minWidth: 280, + minHeight: 180, + autoHeight: true +}) +``` + +#### It polls for state + +Move the body to the event that changes the state: `widget.on('change')`, +`onPropertyChanged`, `onConnectionsChanged`, `onNodeChanged`, or +`onWorkflowLoaded`. + +#### It positions DOM over the graph + +Prefer a mounted widget. If the UI must remain outside the node, read +`node.getScreenRect()` and update it from `comfy.onViewportChanged()`. + +### How do I replace renderer geometry constants? + +| Legacy renderer data | Published answer | +| ----------------------------------- | -------------------------------------------------------------------------------- | +| `LiteGraph.NODE_TITLE_HEIGHT` | Difference between `node.getBounds()` and `node.getSize()` when genuinely needed | +| Slot spacing and node position math | `node.getSlotPosition(side, index)` | +| Canvas pan and scale | `node.getScreenRect()` or `comfy.graph.pointerPosition()` | +| Hit test against nodes | `comfy.graph.nodeAt(point)` | +| Theme widget colors | `CanvasTheme` passed to `widgets.canvas()` | +| Detect a concurrent editor gesture | `comfy.isInteracting()` | + +Use the answer to the operation instead of publishing another renderer +constant. + +### How do I request a repaint? + +Do not port `canvas.setDirty()` or `node.setDirtyCanvas()`. Handle mutations +invalidate their own host views. For external data behind a canvas widget, call +that surface's `redraw()`. + +### How do I rebuild a node-drag editing gesture? + +Observe semantic movement rather than document pointer events and canvas drag +state: + +```js +let gesture + +const stopMove = comfy.onNodeMoved(({ node, position }) => { + gesture = { + dragged: node, + target: findDropCandidate(node, position) + } +}) + +const stopEnd = comfy.onNodeDragEnd((nodes) => { + if ( + gesture?.target && + nodes.some((node) => comfy.sameEntity(node, gesture.dragged)) + ) { + commitGesture(gesture.dragged, gesture.target) + } + gesture = undefined +}) +``` + +`onNodeMoved` works under both renderers. It reports movement, not proof that a +person caused it, so guard mutations made by the gesture against re-entry. +`onNodeDragEnd` is Nodes 2.0 only because the legacy renderer has no published +drag-completion lifecycle. If the action must work under both renderers, design +an explicit command or button rather than pretending release is observable. + +Use `comfy.isInteracting()` when the old code read several renderer flags only +to ask whether the editor was already in the middle of any gesture. + +## Menus and application UI + +### How do I add a node context-menu item? + +```js +comfy.defs.extend('KSampler', (definition) => { + definition.addMenuItem({ + label: (node) => (node.isPinned() ? 'Unpin' : 'Pin'), + run(node) { + node.setPinned(!node.isPinned()) + } + }) +}) +``` + +This replaces `getExtraMenuOptions` and node-menu prototype patches. Entries +from multiple packs compose. Use `when`, a dynamic label, or `items` for a +submenu rather than constructing `LiteGraph.ContextMenu`. + +### How do I open a menu from my own button or surface? + +```js +comfy.ui.showMenu({ + event: mouseEvent, + title: 'Output type', + items: [ + { label: 'Image', run: () => choose('IMAGE') }, + { + label: 'Latent', + submenu: [{ label: 'Samples', run: () => choose('LATENT') }] + } + ] +}) +``` + +The triggering `MouseEvent` gives the host an anchor. This does not add a new +hook to a host-owned canvas or slot menu; use it only when the pack owns the +gesture that asks for the menu. + +### How do I replace DOM insertion into ComfyUI chrome? + +| Old DOM target | Published contribution | +| ---------------------------- | ------------------------------- | +| Sidebar markup | `comfy.ui.addSidebarTab()` | +| Top-bar status text | `comfy.ui.addTopBarBadge()` | +| Action button | `comfy.ui.addActionBarButton()` | +| Modal markup | `comfy.ui.showDialog()` | +| Toast | `comfy.commands.notify()` | +| Command palette and shortcut | `comfy.commands.register()` | + +Contributions are declarative so host layout, theme, accessibility, and +lifecycle remain host-owned. + +### How do I replace extension settings? + +```js +comfy.settings.declare({ + id: 'MyPack.previewQuality', + name: 'Preview quality', + type: 'slider', + defaultValue: 80, + attrs: { min: 1, max: 100, step: 1 } +}) + +const quality = comfy.settings.get('MyPack.previewQuality') +await comfy.settings.set('MyPack.previewQuality', 90) +``` + +Use `settings.onChange()` instead of polling, including for a core setting the +pack does not own. IDs must be namespaced. + +### How do I set a temporary graph background? + +If the old code replaced the canvas background renderer only to show an image, +write the core setting instead: + +```js +const setting = 'Comfy.Canvas.BackgroundImage' +const previous = String(comfy.settings.get(setting) ?? '') + +await comfy.settings.set(setting, imageUrl) + +async function stopBackgroundMode() { + await comfy.settings.set(setting, previous) +} +``` + +The host owns loading and drawing the image under both renderers. Preserve and +restore the user's previous value when the pack's temporary mode ends. + +### How do I use host palette colors? + +Resolve design tokens at the point of use: + +```js +const modelLink = comfy.defs.typeColor('MODEL') +const red = comfy.defs.nodeColor('red') + +if (red) { + node.setColor(red.color) + node.setBgColor(red.bgColor) +} +``` + +Use `setTypeColor(type, color)` only for a link type the pack owns; it refuses +core-owned types and returns an unsubscribe that restores the prior mapping. +`nodeColor()` returns the title, body, and group fill colors behind a palette +name. Do not cache or copy the host's internal color tables. + +## Execution, previews, and backend services + +### How do I replace `api.queuePrompt()` wrappers? + +| Intent | Published API | +| ------------------------------------------ | ------------------------------ | +| Queue like the Run button | `comfy.queue.run()` | +| Run selected output nodes and dependencies | `comfy.queue.run({ nodes })` | +| Synchronous final preparation | `comfy.queue.onBeforeRun()` | +| Asynchronous validation or confirmation | `comfy.queue.guard()` | +| Advance state after submission | `comfy.queue.onAfterRun()` | +| Observe backend validation refusal | `comfy.queue.onRejected()` | +| Replace one serialized widget value | `widget.on('beforeSerialize')` | +| Change virtual prompt topology | Frontend resolution or supply | + +Do not rebuild a prompt wrapper when only one stage is needed. + +### How do I read or change auto-queue and batch settings? + +Use the queue service rather than queue stores or settings IDs: + +```js +const mode = comfy.queue.autoQueueMode() +const batch = comfy.queue.batchCount() + +comfy.queue.setAutoQueueMode('change') +comfy.queue.setBatchCount(4) +``` + +The modes are `disabled`, `change`, and `instant`. Call +`disableAutoQueue()` before a self-interrupting conditional workflow so the +automatic runner does not immediately submit it again. It does not cancel the +run already in progress. + +### How do I define a frontend-only or virtual node? + +Replace `extends LGraphNode`, `isVirtualNode`, and prompt mutation with plain +data and pure resolution: + +```js +comfy.defs.define({ + type: 'MyPack/Reroute', + inputs: [{ name: 'in', type: '*' }], + outputs: [{ name: 'out', type: '*' }], + execution: 'frontend', + resolve({ self }) { + const input = self.input('in') + return { out: input ? { forwardTo: input } : { omit: true } } + } +}) +``` + +A resolver may omit, forward one of its own inputs, or return a literal. It may +return directly or as a promise, and cannot mutate the graph or prompt. Prompt +execution awaits async resolvers concurrently with a deadline; synchronous +editor inspection treats a pending promise as unresolved. + +### How do I implement “use this value everywhere” behavior? + +Use a supplier rather than scanning and editing the built prompt: + +```js +comfy.defs.extend('MyPack/BroadcastModel', (definition) => { + definition.setSupply(({ self, unconnectedInputs }) => { + const output = self.outputs.find(({ name }) => name === 'MODEL') + if (!output) return [] + + return unconnectedInputs() + .filter(({ type }) => type === 'MODEL') + .map((input) => ({ + from: { output: output.index }, + to: { nodeId: input.nodeId, input: input.input }, + priority: 10 + })) + }) +}) +``` + +Supply resolution is graph-local and does not cross subgraph boundaries. +`graph.resolvedSupplies()` exposes the winning edges when a command needs to +materialize the same choices as real links. + +### How do I correlate execution or preview events with a node? + +```js +comfy.defs.extend('MyPack/PreviewNode', (definition) => { + definition.onExecuted((node, result) => { + updateResult(node, result) + }) + + definition.onPreview((node, frame) => { + updatePreview(node, frame.url) + }) +}) +``` + +This replaces global `b_preview`, `b_preview_with_metadata`, and module-level +“currently executing ID” correlation for a node's own frames. For global +execution UI, use `comfy.executingNode()`, `comfy.executionNode(id)`, and +`comfy.onExecutingNodeChanged()`. + +### How do I read images produced by another node? + +Definition callbacks are intentionally correlated only with the node type they +extend. For a command, panel, or overlay that inspects an arbitrary node, read +that node's output state directly: + +```js +const images = producer.getOutputImages() +const displayed = producer.getDisplayedImageIndex() +const selectedUrl = displayed === undefined ? undefined : images[displayed] +``` + +The image list is a frozen URL snapshot. The displayed index is the image the +user selected or is hovering, and can be `undefined` when no image is singled +out. + +### How do I replace a `graphToPrompt` wrapper used for a sidecar cache? + +Trace the Python first. A wrapper that only chose a cache filename does not +require access to the built prompt: + +1. Use the pack's authenticated route through `comfy.backend.fetch()` to read + or refresh the sidecar. +2. If the backend needs connected tensors, run that node and its dependencies + with `comfy.queue.run({ nodes: [node] })`. +3. Refresh correlated state from the definition's `onExecuted` callback and + `comfy.backend.on('execution_cached', ...)` when the cached-result path also + matters. +4. Use a backend hidden `UNIQUE_ID` input when simultaneous node instances need + distinct identity. + +A hidden string whose default is the same for every node is not a frontend API +gap; it is insufficient backend identity. State the simultaneous-node +limitation instead of restoring prompt mutation. + +### How do I call my Python routes or receive custom messages? + +The relative-route example requires the optional Secure Nodes pack-route broker. +It is not provided by the ordinary `/comfy/api/v2.js` instance in this stack. + +```js +const response = await comfy.backend.fetch('presets') +const stop = comfy.backend.on('my-pack-progress', (detail) => { + updateProgress(detail) +}) +``` + +With that broker, declare `presets` as a relative route in this pack's `secure-nodes.json`. +`backend.fetch('presets')` and `backend.ownFetch('presets')` call only that pack's +endpoint. Paths have no leading slash, traversal, percent-encoded path, or scheme. +The host alone assigns the base and builds the authenticated request. + +The shared-page API instead accepts same-origin paths beginning with `/` in +`backend.fetch`; `backend.ownFetch` rejects requests without the broker. Its +`backend.url` and `backend.assetUrl` helpers build shared-page URLs. The broker's +pack-relative restrictions do not apply to that ordinary host. See the +[routing contract](./services.md#pack-relative-backend-requests-and-events). +Use `new URL('./asset.css', import.meta.url)` for an admitted bundled file. + +### How do I load a workflow or store pack documents? + +```js +await comfy.workflow.open(parsedWorkflow) + +await comfy.storage.set('MyPack.presets/portrait', JSON.stringify(preset)) +const saved = await comfy.storage.get('MyPack.presets/portrait') +``` + +`workflow.open()` replaces `app.loadGraphData()` for an explicit user action. +`comfy.storage` replaces direct user-data APIs or `localStorage` for named +server-side presets, prompts, and templates. + +### How do I apply ComfyUI filename and workflow tokens? + +```js +const filename = comfy.workflow.applyTextReplacements( + '%date:yyyy-MM-dd%_%KSampler.seed%' +) +``` + +This uses the active root graph and the same token language as core. It throws +when no graph is active. Do not copy the token parser or read workflow widgets +through renderer objects. + +## Things not to translate + +Some old code should disappear rather than acquire a new spelling: + +| Legacy code | What to do | +| --------------------------------------------------------------------- | --------------------------------------------------- | +| `node.type = node.type ?? undefined` | Delete the defensive no-op. | +| `canvas.setDirty()` after a handle write | Delete it; the mutation invalidates its view. | +| Restore fields saved only to undo `converted-widget` | Delete the workaround; use `setHidden()`. | +| Duplicate upload widget for an input already declaring upload options | Delete it; let the host render the upload behavior. | +| Rewrite named widget values into `widgets_values` | Delete it; address live widgets by name. | +| Poll inside a draw callback | Subscribe to the state-changing event. | + +Other mechanisms have no sanctioned equivalent and must not be recreated with +an internal escape hatch: + +- patching `LGraph`, `LGraphNode`, `LGraphCanvas`, renderer, or widget + prototypes; +- mutating the built prompt or core workflow snapshot; +- selecting and restyling host-owned DOM with global selectors; +- publishing or consuming live internal stores and mutable link records; +- making a supplier or resolver cross a subgraph boundary; +- using a per-frame callback as a general tick. + +If a user capability remains after checking the supported alternatives, name +that capability precisely as an API gap. Do not invent a plausible member or +reintroduce the old object. + +## How do I document a refusal? + +A refusal is an architectural decision, not shorthand for “the conversion was +hard” or “I did not find a method.” It must answer **why** the old mechanism or +requested capability is outside the published contract. + +Every refusal record must include: + +1. **User behavior:** what the feature does from the user's point of view. +2. **Mechanism:** the exact live object, prototype, store, DOM, prompt, or + renderer operation the original used. +3. **Reason:** which ownership, determinism, wire-format, renderer-independence, + scope, or lifecycle guarantee that mechanism violates. +4. **Outcome:** the supported replacement and remaining loss. State explicitly + when the loss is nothing. +5. **Boundary:** the precise published capability or policy change that would + make the refused remainder supportable, or why it must remain host-owned. + +Keep those facts in one adjacent comment block. Use the existing terminal +markers for the outcome rather than inventing another marker: + +```js +// REFUSED: replacing LGraphCanvas.prototype.prompt to change the host's +// numeric-entry behavior for every node and every pack. +// Reason: a pack would own host-global editor behavior, and callback order +// would decide which pack's replacement wins. +// RESTORED: this pack's numeric editor uses comfy.ui.prompt from its own menu. +// INOPERABLE: nothing. +// Reconsider if the host publishes a scoped numeric-editor contribution. +``` + +When behavior is genuinely lost, name the loss rather than softening it: + +```js +// REFUSED: wrapping graphToPrompt to replace the built prompt with an implicit +// cross-product of widget values. +// Reason: execution would depend on hidden frontend code rather than the saved +// graph, and multiple wrappers would compose in load order. +// DROPPED: one queue action no longer expands into undeclared executions. +// This remains host-owned unless the graph gains a serializable fan-out node. +``` + +These are not valid reasons: + +- “not supported”; +- “no API”; +- “renderer internals are unavailable”; +- “cannot be converted”; +- a list of removed property names without the behavior they implemented. + +Those statements describe absence or effort, not a decision. If the behavior +is acceptable but the public surface is merely missing, classify it as an API +gap. If another published mechanism preserves it, convert through that +mechanism and record the refused technique only as localized rationale. + +## Migration safety checklist + +- Trace every `inputs`, `outputs`, `links`, `widgets`, and `properties` value to + determine whether it is live state or serialized data. +- State the user-visible behavior before selecting a replacement. +- Preserve workflow and prompt wire format, including positional widget cells + and link IDs. +- Use the narrowest semantic event instead of polling or draw-time work. +- Keep frontend resolvers and suppliers pure and graph-scoped. +- Use methods for handle reads and writes; do not attach arbitrary properties. +- Probe optional behavior with `comfy.supports()`. +- Verify both behavioral equivalence and the specific failure the migration is + intended to fix. + +For the detailed contracts behind these recipes, continue with +[Nodes](./nodes.md), [Graphs](./graph.md), [Slots](./slots.md), +[Widgets](./widgets.md), [Execution](./execution.md), and +[Application services](./services.md). diff --git a/docs/node-api/nodes.md b/docs/node-api/nodes.md new file mode 100644 index 00000000000..b983e1f57f5 --- /dev/null +++ b/docs/node-api/nodes.md @@ -0,0 +1,264 @@ +# Nodes and definitions + +Node type registration and live node editing are separate APIs. Use the +definition registry to install behavior by type, and a `NodeHandle` to work with +one node that already belongs to a graph. + +## Reading definitions + +```js +const definition = comfy.defs.get('KSampler') + +if (definition) { + console.info(definition.title, definition.category) + console.info(definition.inputs) + console.info(definition.outputs) +} +``` + +`NodeDef` is frozen metadata: + +| Field | Meaning | +| ------------------------------------------ | -------------------------------------------------------------------------------------------------------- | +| `type`, `title`, `category`, `description` | Stable definition identity and presentation. | +| `inputs` | Declared inputs, including name, type, localized name, combo values, and a frozen `options` passthrough. | +| `outputs` | Declared output names and types. | +| `hidden` | Backend hidden-input declarations. These are not connectable slots. | +| `isOutputNode` | Whether the backend marks the node as an output. | +| `source` | The backend-reported pack that supplied the type, if known. | + +`inputs[].options` and `hidden` deliberately preserve pack-owned backend data. +They are declarations, not live widget values or hidden-input execution values. + +Use `defs.all()` to take a snapshot of every definition and `defs.has(type)` for +an existence check. `defs.refresh()` asks the backend to reload definitions; +`defs.onRefreshed()` observes completion. + +## Extending definitions + +```js +const stop = comfy.defs.extend('KSampler', (definition) => { + definition.setTitle('KSampler with tools') + definition.addWidget({ + type: 'button', + name: 'reset_seed' + }) + + definition.onCreated((node, event) => { + if (!event.restored) node.setColor('#334155') + node.widgets.get('reset_seed')?.on('activate', () => { + node.widgets.get('seed')?.setValue(0) + }) + }) +}) +``` + +Builder registrations compose. The host applies every matching extension rather +than making each pack capture and call a previous prototype method. An +extension may register before or after the type it matches: one registered +later applies to nodes created from then on, and callbacks always run in +registration order. + +### Builder configuration + +| Method | Purpose | +| ------------------------------------------------- | ----------------------------------------------------------------------- | +| `setTitle(title)` | Change the displayed type title. | +| `setCategory(category)` | Change where the type appears in node search. | +| `setExecution('backend' \| 'frontend', resolve?)` | Mark a backend-defined type frontend-only or restore backend execution. | +| `setSupply(supplier)` | Declare broadcast-style edges into other nodes' unconnected inputs. | +| `addWidget(def)` | Add a declared widget to every instance. | +| `hideWidget(name)` | Hide a declared widget while retaining its value. | +| `addMenuItem(item)` | Add a host-rendered context-menu entry. | + +Structural changes to a live node—dynamic slots, values, ordering, or +connections—belong on the instance handles supplied by lifecycle callbacks. + +### Lifecycle and behavior hooks + +| Hook | When it runs | +| ----------------------------------- | ---------------------------------------------------------- | +| `onCreated(node, event)` | The node joined a graph and is addressable. | +| `onConfigured(node, data)` | Saved node data was applied. | +| `onRemoved(node)` | The node left its graph. | +| `onExecuted(node, result)` | The backend returned the node's execution result. | +| `onPreview(node, frame)` | A preview frame was correlated with this executing node. | +| `onConnectionsChanged(node, event)` | A slot connected or disconnected. | +| `onBeforeConnect(node, event)` | A proposed connection may be accepted or vetoed. | +| `onUnplacedLink(node, event)` | A link dropped on the node had no unique destination slot. | +| `onResized(node, size)` | A user or layout operation resized the node. | +| `onHover(node, hovering)` | The pointer entered or left the node. | +| `onDoubleClick(node)` | The node was double-clicked. | +| `onPropertyChanged(node, event)` | A user edited a node property. | +| `onDragOver(node, event)` | Decide whether the node accepts a browser drag. | +| `onDrop(node, event)` | Handle a browser drop the node accepted. | +| `onSerialize(node)` | Return pack-owned fields to merge into the saved node. | + +`onBeforeConnect` returns `false` to veto. `onUnplacedLink` returns `true` after +the callback wires the link itself. `onDragOver` returns `true` to route the +drop; `onDrop` returns `true` to claim it. + +`onSerialize` must return deterministic, synchronous, pack-owned data. Its data +comes back through `onConfigured`. Do not return core workflow fields or mutate +a serialization object supplied by the host. + +## Defining a frontend-owned node + +```js +const unregister = comfy.defs.define({ + type: 'MyPack/ConstantText', + title: 'Constant Text', + category: 'My Pack', + outputs: [{ name: 'text', type: 'STRING' }], + widgets: [{ type: 'text', name: 'value', value: '', serialize: true }], + execution: 'frontend', + resolve: ({ self }) => ({ + text: { literal: self.widgetValue('value') ?? '' } + }), + onCreated(node) { + node.setSizeConstraints({ minWidth: 220 }) + } +}) +``` + +`NodeDefinition` is plain data, not a class. It supports inputs, outputs, +widgets, frontend execution, supply, and the common lifecycle callbacks. + +For builder-only behavior such as menu entries, preview frames, resize, hover, +double-click, connect veto, or unplaced links, define the type and extend the +same type separately. + +## Working with a live node + +Node handles can come from lifecycle callbacks, graph lookups, selection, +groups, execution events, or subgraph scopes. + +```js +const node = comfy.graph.node(nodeId) +if (node) { + node.setTitle('Primary sampler') + node.setMode('always') + node.setPosition({ x: 320, y: 180 }) + node.setSize({ width: 280, height: 420 }) + node.setProperty('role', 'primary') +} +``` + +### Identity and presentation + +- `id` is identity inside the owning graph. +- `graphId` identifies that graph. +- `type` is immutable definition identity. +- `comfyClass` is the backend class identifier when it differs from `type`. +- title, mode, collapsed state, pinned state, colors, and shape use explicit + getter/setter pairs. + +Do not assign `type`. Use `comfy.graph.replace(node.id, newType)` to rebuild the +node and preserve compatible state and links. + +### Properties + +Use `getProperty`, `getProperties`, and `setProperty`. `getProperties()` returns +an inert object rather than a mutable reference. + +`onPropertyChanged` can normalize or reject a user edit: + +```js +definition.onPropertyChanged((_node, event) => { + if (event.name !== 'strength') return + const value = Number(event.value) + if (!Number.isFinite(value)) event.reject() + else event.setValue(Math.min(1, Math.max(0, value))) +}) +``` + +`setValue()` replaces the pending property value without re-entering the +property callback. `reject()` restores the previous value. + +### Geometry + +| Method | Coordinate space or behavior | +| --------------------------------- | -------------------------------------------------------- | +| `getPosition()` / `setPosition()` | Node body position in graph space. | +| `getSize()` / `setSize()` | Size through the host resize protocol. | +| `getBounds()` | Full node bounds in graph space, including title layout. | +| `getSlotPosition(side, index)` | Renderer-computed slot center in graph space. | +| `getScreenRect()` | Client-coordinate screen rectangle, if rendered. | +| `setSizeConstraints()` | Declarative min/max dimensions and auto-height. | + +Do not reconstruct title height, slot spacing, pan, zoom, or device-pixel-ratio +math from renderer constants. + +### Output images + +`getOutputImages()` returns URLs for the images or previews the node currently +exposes. It never returns renderer-owned `HTMLImageElement` objects. +`getDisplayedImageIndex()` identifies the image selected or hovered by the user, +or returns `undefined` when there is no such choice. + +### Collections and snapshots + +Each node exposes: + +```js +node.inputs +node.outputs +node.widgets +``` + +These are operation-oriented collections, not arrays. See +[Slots and links](./slots.md) and [Widgets](./widgets.md). + +`node.snapshot()` returns a frozen `NodeSnapshot` with identity, presentation, +position, and size. It can return `undefined` when the entity is gone. + +## Menus and badges + +Node menu entries are declarative and host-rendered: + +```js +definition.addMenuItem({ + label: (node) => + node.getMode() === 'never' ? 'Enable node' : 'Disable node', + when: (node) => !node.isDeleted, + run(node) { + node.setMode(node.getMode() === 'never' ? 'always' : 'never') + }, + order: 20 +}) +``` + +An item may instead provide one level of `items`, either a fixed array or a +function of the current node. A submenu parent omits `run`. + +Badges are small labels rendered in node chrome under both renderers: + +```js +const removeBadge = node.addBadge({ + text: 'cached', + onClick() { + clearCache(node) + } +}) +``` + +Pass a function for a dynamic badge. It runs during drawing, so it must be fast. +The return value removes the badge. + +## Pack-owned instance state + +Do not add fields to a node or handle. Keep state outside the entity and include +graph scope in the key: + +```js +const state = new Map() +const keyOf = (node) => `${node.graphId}:${node.id}` + +comfy.defs.extend('MyPack/Node', (definition) => { + definition.onCreated((node) => state.set(keyOf(node), { open: false })) + definition.onRemoved((node) => state.delete(keyOf(node))) +}) +``` + +When handles may come from different API instances or scopes, compare them with +`comfy.sameEntity()` rather than `===`. diff --git a/docs/node-api/reference.md b/docs/node-api/reference.md new file mode 100644 index 00000000000..3e9cbc0b7dc --- /dev/null +++ b/docs/node-api/reference.md @@ -0,0 +1,769 @@ +# API reference + +This page is a navigable map of the published surface. The exact, exhaustive, +machine-readable TypeScript contract is checked in as +[comfy-api.d.ts](./comfy-api.d.ts) and generated from the implementation: + +```sh +node scripts/node-api/gen_api_dts.mjs > docs/node-api/comfy-api.d.ts +``` + +If a member is absent from that declaration file, it is not published. The +topic guides describe lifecycle, invariants, and recommended use; this page +collects signatures and points to those guides. + +Only `comfy` is a runtime entry point. The named interfaces and type aliases +below describe arguments, return values, callbacks, and snapshots reached from +that object; they are not additional global services. + +## Entry point + +```js +import { comfy } from '/comfy/api/v2.js' +``` + +See [Registration, lifecycle, and globals](./registration.md). + +## `Comfy` + +### Contract and identity + +| Member | Returns | Purpose | +| ------------------------------ | ----------------------------------- | -------------------------------------------------------- | +| `version` | `string` | Current API `major.minor`. | +| `major` | `number` | Breaking-change generation. | +| `getApiVersion()` | `readonly [number, number, number]` | `[major, minor, patch]`. | +| `isApiVersionAtLeast(version)` | `boolean` | Compare with a version written `v2.0.0`. | +| `supports(capability)` | `boolean` | Test a named capability. | +| `require(capability)` | `void` | Throw an actionable error if unavailable. | +| `capabilities()` | `readonly string[]` | All capabilities supplied by the host. | +| `forMajor(major)` | `Comfy` | Pin to a supported public major. | +| `sameEntity(a, b)` | `boolean` | Compare handles across scopes, majors, or API instances. | +| `adopt(handle)` | `NodeHandle \| undefined` | Re-resolve a foreign node handle into this instance. | + +### Domain roots + +| Member | Type | Guide | +| ---------- | ---------------- | ------------------------------------------------------------------- | +| `graph` | `GraphHandle` | [Graphs](./graph.md) | +| `defs` | `DefRegistry` | [Nodes and definitions](./nodes.md) | +| `queue` | `QueueHandle` | [Execution](./execution.md) | +| `settings` | `SettingsHandle` | [Services](./services.md#settings) | +| `storage` | `StorageHandle` | [Services](./services.md#per-user-storage) | +| `ui` | `UiHandle` | [Services](./services.md#sidebar-tabs) | +| `commands` | `CommandsHandle` | [Services](./services.md#commands-keybindings-and-notifications) | +| `backend` | `BackendHandle` | [Services](./services.md#pack-relative-backend-requests-and-events) | +| `workflow` | `WorkflowHandle` | [Services](./services.md#workflow-service) | +| `system` | `SystemHandle` | [Services](./services.md#system-monitoring) | +| `models` | `ModelsHandle` | [Services](./services.md#model-catalogues) | + +### Lifecycle and observation + +| Member | Signature | +| ------------------------ | ---------------------------------------------------------------------------- | +| `isInteracting` | `() => boolean` | +| `onNodeMoved` | `(listener: (event: NodeMoveEvent) => void) => Unsubscribe` | +| `onNodeDragEnd` | `(listener: (nodes: readonly NodeHandle[]) => void) => Unsubscribe` | +| `onViewportChanged` | `(listener: () => void) => Unsubscribe` | +| `onSelectionChanged` | `(listener: (nodes: readonly NodeHandle[]) => void) => Unsubscribe` | +| `onNodeChanged` | `(listener, options?: NodeChangeOptions) => Unsubscribe` | +| `onReady` | `(listener: () => void) => Unsubscribe` | +| `onWorkflowLoaded` | `(listener: (document: DocumentHandle \| undefined) => void) => Unsubscribe` | +| `onDocumentOpened` | `(listener: (document: DocumentHandle) => void) => Unsubscribe` | +| `onDocumentActivated` | `(listener: (document: DocumentHandle) => void) => Unsubscribe` | +| `onDocumentDeactivated` | `(listener: (document: DocumentHandle) => void) => Unsubscribe` | +| `onDocumentClosed` | `(listener: (document: DocumentHandle) => void) => Unsubscribe` | +| `executingNode` | `() => NodeHandle \| undefined` | +| `executionNode` | `(executionId: string) => NodeHandle \| undefined` | +| `onExecutingNodeChanged` | `(listener: (node: NodeHandle \| undefined) => void) => Unsubscribe` | + +`onNodeDragEnd` is Nodes 2.0 only. See +[Graphs and groups](./graph.md#node-and-graph-observation). + +### Observation payloads + +| Type | Fields or values | +| ------------------- | --------------------------------------------------------------- | +| `NodeMoveEvent` | `node`, `position: { x, y }` | +| `NodeChangeOptions` | `scope?: 'visible' \| 'document'` | +| `NodeChangeScope` | `'visible' \| 'document'` | +| `TrackedProperty` | `title`, `mode`, `color`, `bgcolor`, `shape`, or `showAdvanced` | +| `NodeChangeEvent` | `node`, `graphId`, `property`, `from`, `to` | +| `Unsubscribe` | `() => void` | + +The document scope includes the root and all subgraph definitions. Since node +IDs are graph-local, key records from `NodeChangeEvent` by both `graphId` and +`node.id`. + +## `DefRegistry` + +| Member | Signature or result | +| ------------------ | ---------------------------------------------------------------------------------- | +| `defineWidgetType` | `(type: string, def: WidgetTypeDef) => Unsubscribe` | +| `define` | `(definition: NodeDefinition) => Unsubscribe` | +| `extend` | `(selector: DefSelector, apply: (builder: NodeDefBuilder) => void) => Unsubscribe` | +| `get` | `(type: string) => NodeDef \| undefined` | +| `all` | `() => readonly NodeDef[]` | +| `has` | `(type: string) => boolean` | +| `typeColor` | `(type: string) => string` | +| `nodeColor` | `(name: string) => NodeColor \| undefined` | +| `isTypeCompatible` | `(outputType: string, inputType: string) => boolean` | +| `setTypeColor` | `(type: string, color: string) => Unsubscribe` | +| `refresh` | `() => Promise` | +| `onRefreshed` | `(listener: () => void) => Unsubscribe` | + +`setTypeColor` accepts a type introduced by the pack and refuses a core-owned +type. Resolve palette colors when used rather than persisting or caching them. + +### `NodeDefBuilder` + +Configuration methods: + +- `setTitle(title)` +- `setCategory(category)` +- `setExecution(execution, resolve?)` +- `setSupply(supplier)` +- `addWidget(def)` +- `hideWidget(name)` +- `addMenuItem(item)` + +Behavior registration methods: + +- `onCreated` +- `onExecuted` +- `onConfigured` +- `onConnectionsChanged` +- `onRemoved` +- `onResized` +- `onHover` +- `onDoubleClick` +- `onDragOver` +- `onDrop` +- `onPropertyChanged` +- `onPreview` +- `onSerialize` +- `onPromptSerialize` +- `onBeforeConnect` +- `onUnplacedLink` + +The current frozen definition is available as `builder.def`. + +Related types: `NodeDef`, `NodeDefinition`, `DefSelector`, `NodeCreatedEvent`, +`ExecutionResult`, `PreviewFrame`, `ConnectionChangeEvent`, +`PropertyChangeEvent`, `BeforeConnectEvent`, `UnplacedLinkEvent`, +`PromptInputProjection`, `PromptInputProjector`, `NodeMenuItem`, +`NodeSubMenuItem`, and `NodeColor`. + +### Definition data + +`NodeDef` is the frozen read view returned by `get()`, `all()`, and +`builder.def`: + +| Field | Type or contents | +| -------------- | -------------------------------------------------------------------------------- | +| `type` | Registered type name | +| `title` | Display title | +| `category` | Palette category | +| `description` | Backend description | +| `inputs` | `name`, `type`, optional `localizedName`, optional combo `values`, and `options` | +| `outputs` | `name` and `type` | +| `isOutputNode` | Whether it is an execution output | +| `hidden` | Hidden backend declarations; these are not connectable slots | +| `source` | Supplying pack, when reported by the backend | + +`NodeDefinition` declares a frontend-owned type. It requires `type` and may +set `title`, `category`, `description`, `inputs`, `outputs`, `widgets`, +`execution`, `resolve`, and `supply`. It can also declare `onCreated`, +`onExecuted`, `onConfigured`, `onConnectionsChanged`, `onPropertyChanged`, +`onDragOver`, `onDrop`, `onRemoved`, and `onSerialize` callbacks directly. +It may also declare `onPromptSerialize`, which returns a +`PromptInputProjection` directly or in a promise. A projection names inputs to +omit from that prompt build. + +`DefSelector` is a type string, string array, regular expression, predicate +over `NodeDef`, or `{ category: string | RegExp }`. + +### Definition callback payloads + +| Type | Fields and behavior | +| ----------------------- | ------------------------------------------------------------------------------------------ | +| `NodeCreatedEvent` | `restored`, `loading` | +| `ExecutionResult` | `images`, `text`, and passthrough `raw` | +| `PreviewFrame` | `blob`, temporary object `url` | +| `ConnectionChangeEvent` | `side`, `index`, optional `name`, `connected`, optional `peerNodeId`, optional `peerIndex` | +| `PropertyChangeEvent` | `name`, `value`, `previous`, `setValue(value)`, `reject()` | +| `BeforeConnectEvent` | `side`, `index`, optional `peerNodeId`, `peerIndex`, and `peerType` | +| `UnplacedLinkEvent` | `side`, `peerNodeId`, `peerIndex`, `type`, `replaceExisting` | + +`NodeMenuItem` has a string or node-dependent `label`, optional `when`, +optional `run`, optional one-level `items`, and optional numeric `order`. +`NodeSubMenuItem` contains `label` and `run(node)`. `NodeColor` contains +`color`, `bgColor`, and `groupColor`. + +See [Nodes and definitions](./nodes.md). + +## `GraphHandle` + +### Lookup and inspection + +| Member | Result | +| -------------------- | ------------------------------- | +| `id` | `string` | +| `node(id)` | `NodeHandle \| undefined` | +| `nodes()` | `readonly NodeHandle[]` | +| `nodesOfType(type)` | `readonly NodeHandle[]` | +| `queryNodes(query?)` | `readonly NodeHandle[]` | +| `links()` | `readonly LinkInfo[]` | +| `groups()` | `readonly GroupHandle[]` | +| `selection()` | `readonly NodeHandle[]` | +| `root()` | `GraphScopeHandle \| undefined` | +| `subgraphs()` | `readonly GraphScopeHandle[]` | +| `resolvedSupplies()` | `readonly ResolvedSupply[]` | +| `nodeAt(point)` | `NodeHandle \| undefined` | +| `pointerPosition()` | `Point \| undefined` | +| `version` | opaque `number` change token | + +A `NodeQuery` has optional `scope`, `type`, `rendered` and `group`; every field +narrows the result. `scope` is a `NodeQueryScope`: `visible`, the default, or +`root-and-subgraphs`. + +### Mutation and view + +| Member | Signature or result | +| ----------- | --------------------------------------------------------------------- | +| `add` | `(type: string, init?: NodeInit) => NodeHandle` | +| `duplicate` | `(id: string, position?: Point) => NodeHandle \| undefined` | +| `replace` | `(id: string, type: string) => NodeHandle \| undefined` | +| `remove` | `(id: string) => boolean` | +| `select` | `(nodes: readonly NodeHandle[], options?: { add?: boolean }) => void` | +| `centerOn` | `(node: NodeHandle) => void` | +| `setZoom` | `(scale: number) => void` | +| `panBy` | `(delta: Point) => void` | +| `batch` | `(mutations: () => T) => T` | + +### `GraphScopeHandle` + +Read-only graph scope with `id`, optional `name`, `nodes()`, `node(id)`, +`groups()`, and `resolvedSupplies()`. + +### `GroupHandle` + +Provides `id`, `isDeleted`, title and color getter/setters, `nodes()`, +`getBounds()`, and `centerOn()`. Writing to a deleted group throws +`ComfyDeletedError`. + +`NodeInit` accepts optional `title` and `position`. `Point` is `{ x, y }`, +`Size` is `{ width, height }`, and `Bounds` is `{ x, y, width, height }`. + +See [Graphs and groups](./graph.md). + +## `NodeHandle` + +Every node handle has `isDeleted`, `id`, `graphId`, `type`, and `comfyClass`. + +### State + +| Read | Write | +| -------------------------------------- | --------------------------------- | +| `getTitle()` | `setTitle(title)` | +| `getMode()` | `setMode(mode)` | +| `isCollapsed()` | `setCollapsed(collapsed)` | +| `isPinned()` | `setPinned(pinned)` | +| | `setResizable(resizable)` | +| `getColor()` | `setColor(color)` | +| `getBgColor()` | `setBgColor(color)` | +| `getShape()` | `setShape(shape)` | +| `getProperty(key)` / `getProperties()` | `setProperty(key, value)` | +| `isSerializingWidgets()` | `setSerializeWidgets(serialize)` | +| `getPosition()` | `setPosition(point)` | +| `getSize()` | `setSize(size)` | +| `getSizeConstraints()` | `setSizeConstraints(constraints)` | + +### Other members + +- `getBounds()` +- `getSlotPosition(side, index)` +- `getScreenRect()` +- `getOutputImages()` +- `getDisplayedImageIndex()` +- `addBadge(badge)` +- `snapshot()` +- `remove()` +- `inputs`, `outputs`, and `widgets` + +Related types: `NodeMode`, `NodeShape`, `NodeSnapshot`, `BadgeDef`, `Point`, +`Size`, `Bounds`, and `SizeConstraints`. + +### Node data types + +| Type | Shape or values | +| ----------------- | --------------------------------------------------------------------------- | +| `NodeMode` | `always`, `never`, `bypass`, `on-event`, or `on-trigger` | +| `NodeShape` | `default`, `box`, `round`, `circle`, or `card` | +| `BadgeDef` | `text`, optional `color`, `bgColor`, and `onClick()` | +| `SizeConstraints` | Optional `minWidth`, `minHeight`, `maxWidth`, `maxHeight`, and `autoHeight` | +| `NodeSnapshot` | Identity, title, mode/flags/colors/shape, `position`, and `size` | + +Every live handle implements `HandleCommon`, whose `isDeleted` flag remains +safe to read after removal. Other reads return inert absence and mutating a +deleted handle throws unless the requested end state is already achieved by an +idempotent removal or disconnect. + +See [Nodes and definitions](./nodes.md). + +## Slots and links + +### `SlotCollection` + +The generic `SlotCollection` interface backs both input and output collections. + +| Member | Result | +| --------------------------- | -------------------------------------------------- | +| `length` | `number` | +| `get(ref)` | handle or `undefined`; throws on an ambiguous name | +| `byId(id)` | handle or `undefined` | +| `byName(name)` | handle or `undefined` | +| `at(index)` | handle or `undefined` | +| `all()` | frozen handle array | +| `ids()` | frozen `SlotId` array | +| `names()` | frozen string array | +| `add(name, type, options?)` | new handle | +| `remove(ref)` | `boolean` | +| `reorder(names)` | `void` | + +Collections are iterable. + +### `InputSlotHandle` + +Identity and state: `id`, `index`, `name`, `type`, `label`, `isConnected`, +`connectedType`, and `isWidgetInput`. + +Methods: `widgetConfig()`, `mergeWidgetConfig(config)`, `link()`, `source()`, +`resolvedSource()`, `disconnect()`, `modify(patch)`, and `snapshot()`. + +### `OutputSlotHandle` + +Identity and state: `id`, `index`, `name`, `type`, `label`, and `isConnected`. + +Methods: `links()`, `targets()`, `connectTo(targetNodeId, input)`, +`disconnect(targetNodeId?)`, `modify(patch)`, `moveLinksTo(target)`, and +`snapshot()`. + +Related types: `SlotId`, `SlotRef`, `SlotType`, `SlotShape`, `SlotOptions`, +`SlotPatch`, `InputSlotPatch`, `SlotSnapshot`, `LinkInfo`, `InputWidgetConfig`, +and `ResolvedInputSource`. + +### Slot and link data + +| Type | Shape or values | +| ------------------- | ---------------------------------------------------------------------------------------------------------- | +| `SlotId` | Stable branded string | +| `SlotRef` | Slot ID, exact name, or explicit `{ index }` | +| `SlotType` | String or string array; arrays normalize to the comma-separated wire type | +| `SlotShape` | `default`, `optional`, `list`, or `directional` | +| `SlotDirection` | `none`, `up`, `down`, `left`, `right`, or `center` | +| `SlotPosition` | `{ x, y }` relative to the node body | +| `InputWidgetConfig` | String or combo-array `type`, plus optional frozen `options` | +| `LinkInfo` | `id`, `sourceNodeId`, `sourceSlotId`, `targetNodeId`, `targetSlotId`, `type`, `sourceIndex`, `targetIndex` | + +`SlotOptions` configures a newly added slot with `shape`, `localizedName`, +`position`, `direction`, `widget`, and `widgetConfig`. `SlotPatch` can change +`name`, `label`, `localizedName`, `type`, `position`, `direction`, `color`, +`colorWhenUnconnected`, and `shape`; `InputSlotPatch` adds `widget` and +`widgetConfig`. Use `null` where the declaration says it restores the host +default. + +`SlotSnapshot` contains stable identity plus current index, name, type, label, +localized name, position, direction, shape, and connection state. +`ResolvedInputSource` is an output (`graphId`, `nodeId`, `outputIndex`), a +literal value, or an omission with a reason. + +See [Slots and links](./slots.md). + +## Widgets + +### `WidgetCollection` + +| Member | Result | +| -------------------------------------- | --------------------------- | +| `length` | `number` | +| `get(name)` / `at(index)` | `WidgetHandle \| undefined` | +| `all()` / `names()` | frozen snapshots | +| `reorder(names)` / `move(name, index)` | `void` | +| `add(def)` | `WidgetHandle` | +| `mount(def)` | `WidgetHandle` | +| `canvas(def)` | `CanvasHandle` | +| `remove(name)` | `boolean` | + +The collection is iterable. + +### `WidgetHandle` + +Every widget handle has `isDeleted`, `name`, and `widgetType`. + +| Area | Members | +| ------------- | --------------------------------------------------------------------------------------------- | +| Value | `getValue()`, `setValue(value)` | +| Relationships | `linked()`, `setLinked(names)` | +| Visibility | `isHidden()`, `setHidden(hidden)` | +| Options | `getOptions()`, `setOption(key, value)`, `setLabel(label)` | +| State | `isDisabled()`, `setDisabled(disabled)`, `isSerialized()` | +| Layout | `getHeight()`, `setHeight(px)` | +| Events | `on('change' \| 'removed' \| 'activate' \| 'textInteraction' \| 'beforeSerialize', listener)` | + +Related types: `WidgetValue`, `WidgetOptions`, `WidgetDef`, +`WidgetSerializeEvent`, `MountDef`, `MountedValue`, `CanvasDef`, `CanvasHandle`, +`CanvasPointerEvent`, `CanvasTheme`, `WidgetTypeDef`, `WidgetTypeValue`, +`WidgetTypeContext`, and the `WidgetText*` event types. + +### Widget declarations and mounted values + +| Type | Important fields or methods | +| -------------- | ---------------------------------------------------------------------------------- | +| `WidgetDef` | `type`, `name`, optional value/options/disabled/hidden/serialize fields | +| `MountDef` | `name`, `render`, optional teardown/layout/visibility/serialization/default fields | +| `MountedData` | String, number, boolean, object, or `null` | +| `MountedValue` | `get()`, `set(value)`, `onChange(listener)` | +| `CanvasHandle` | `widget`, `redraw()` | + +`WidgetValue` also allows `undefined`. `WidgetOptions` is an open, frozen +record. Its named common fields are `on`, `off`, `max`, `min`, `precision`, +`read_only`, `step`, `step2`, `multiline`, `property`, `socketless`, +`canvasOnly`, `hideInPanel`, `nodeType`, `serialize`, `values`, `iconClass`, +`disabled`, `useGrouping`, `placeholder`, `showThumbnails`, +`showItemNavigators`, and `hidden`. Read with `getOptions()` and update through +`setOption()`. + +`WidgetSerializeEvent` contains `context`, the unmodified `value`, and +`setSerializedValue(value)`. Context is `workflow`, `prompt`, or `embedded`. + +### Canvas widget types + +`CanvasDef` requires `name` and `draw(context, size, theme, value)`. It may set +`height`, value/serialization fields, and `onPointerDown`, `onPointerMove`, +`onPointerUp`, and `onContextMenu` handlers. `CanvasPointerEvent` supplies +widget-relative `x`, `y`, and the original `PointerEvent`. + +`CanvasTheme` supplies `surface`, `surfaceHovered`, `border`, `text`, and +`textSecondary`. It is resolved afresh for each draw, so do not cache it. + +### Widget type registration + +`WidgetTypeData` is string, number, boolean, object, or `null`. +`WidgetTypeValue` provides `get`, `set`, and `onChange`. `WidgetTypeContext` +provides a frozen `getOptions()` snapshot and `onNodeReady(listener)`. + +`WidgetTypeDef` may declare `defaultValue`, `height`, `minWidth`, `minHeight`, +and `serialize`, and must implement: + +```ts +interface WidgetTypeDef { + render( + container: HTMLElement, + value: WidgetTypeValue, + name: string, + context: WidgetTypeContext + ): Unsubscribe | void +} +``` + +### Host text-editor events + +All `WidgetTextInteractionEvent` variants contain the current `value`, a +`WidgetTextSelection` (`start`, `end`), `menuEvent`, `setValue()`, and +`focus()`: + +| Variant | Additional fields | +| ---------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| `WidgetTextInputEvent` | `kind: 'input' \| 'selection'` | +| `WidgetTextWheelEvent` | `kind: 'wheel'`, `deltaY`, `ctrlKey`, `preventDefault()` | +| `WidgetTextKeyEvent` | `kind: 'keydown'`, `key`, `ctrlKey`, `altKey`, `shiftKey`, `metaKey`, `repeat`, `preventDefault()`, `stopPropagation()` | + +`WidgetTextEventBase` is the shared portion of those variants. + +See [Widgets](./widgets.md). + +## `QueueHandle` + +| Member | Signature or result | +| ------------------------------ | --------------------------------------------------------------- | +| `run` | `(options?: RunOptions) => Promise` | +| `onBeforeRun` | `(listener: () => (() => void) \| void) => Unsubscribe` | +| `onAfterRun` | `(listener: (event: RunSubmittedEvent) => void) => Unsubscribe` | +| `onRejected` | `(listener: (event: RunRejectedEvent) => void) => Unsubscribe` | +| `pending` | `() => number` | +| `onPendingChanged` | `(listener: (pending: number) => void) => Unsubscribe` | +| `interrupt` | `() => Promise` | +| `onInterrupted` | `(listener: () => void) => Unsubscribe` | +| `autoQueueMode` | `() => AutoQueueMode` | +| `setAutoQueueMode` | `(mode: AutoQueueMode) => void` | +| `batchCount` / `setBatchCount` | read or change the host batch count | +| `disableAutoQueue` | `() => void` | +| `guard` | `(check: () => boolean \| Promise) => Unsubscribe` | + +Related types: `RunOptions`, `RunSubmittedEvent`, `RunSubmission`, +`RunRejectedEvent`, `RunRejectedNode`, `RunRejectionError`, and `AutoQueueMode`. + +### Queue payloads + +| Type | Fields or values | +| ------------------- | ----------------------------------------------------------------------- | +| `RunOptions` | Optional `nodes` and `batch` | +| `RunSubmittedEvent` | Accepted `promptIds`, optional `submissions`, rejected submission count | +| `RunSubmission` | `promptId`, executable `nodeCount` | +| `RunRejectionError` | `type`, `message`, `details`, optional `inputName` | +| `RunRejectedNode` | `nodeId`, `nodeType`, `errors` | +| `RunRejectedEvent` | Optional HTTP `status`, top-level `error`, and per-node `nodeErrors` | +| `AutoQueueMode` | `disabled`, `change`, or `instant` | + +See [Execution](./execution.md). + +## Resolution and supply + +| Type | Purpose | +| ---------------------------------- | -------------------------------------------------------------- | +| `Resolver` | Maps a frontend node's own output names to `OutputResolution`. | +| `OutputResolution` | Omit, forward to an input, or emit a literal. | +| `ResolveView` / `ResolvedNodeView` | Frozen data visible to a resolver. | +| `Supplier` | Offers edges into unconnected inputs in the same graph. | +| `SupplyView` / `UnconnectedInput` | Frozen matching data visible to a supplier. | +| `SuppliedEdge` | One offered edge and optional priority. | +| `ResolvedSupply` | One winning, fully resolved edge. | +| `ResolvedSource` | Final output, literal, or omission. | + +### Resolver view + +`InputRef` is `{ nodeId, input }`. `OutputResolution` is one of +`{ omit: true }`, `{ forwardTo: InputRef }`, or `{ literal: WidgetValue }`. +`ResolvedSource` is the corresponding final output, literal, or omission with +a reason. + +`ResolvedNodeView` contains `id`, `type`, frozen `properties`, `groups`, raw +numeric `mode`, `color`, `inputs`, `outputs`, `widgetValue(name)`, and +`input(ref)`. Its `OwnInput` entries contain index/name/label/type, connection +state, resolved `connectedType`, and optional `sourceNodeId`; `OwnOutput` +contains index/name/label/type. `GroupMembership` contains `id` and `title`. + +`ResolveView` adds `self` and `nodesOfType(type)`. A `Resolver` returns a record +keyed by its own output names. + +### Supplier view + +`UnconnectedInput` contains `nodeId`, `nodeType`, `input`, `name`, `type`, +display `label`, `isWidgetInput`, `nodeTitle`, `nodeMode`, `nodeColor`, +`nodeGroups`, and frozen `nodeProperties`. `SupplyView` exposes `self`, +`nodesOfType()`, and `unconnectedInputs()`. + +Each `SuppliedEdge` contains `to: InputRef`, an optional `priority`, and `from` +as the supplier's output, a literal, or one of the supplier's forwarded inputs. +`ResolvedSupply` contains `supplierNodeId`, `to`, and the final `from` source. + +See [Execution and resolution](./execution.md#frontend-only-nodes). + +## Application service handles + +### `SettingsHandle` + +- `declare(def: SettingDef): void` +- `get(id): SettingValue | undefined` +- `set(id, value): Promise` +- `onChange(id, listener): Unsubscribe` + +`SettingValue` is string, number, boolean, or a readonly string array. +`SettingDef` contains `id`, `name`, declarative `type`, `defaultValue`, and +optional `tooltip`, `category`, `options`, `attrs`, and `onChange`. +`SettingOption` is a string or `{ value, label }`; `SettingAttrs` contains +optional `min`, `max`, and `step`. + +### `StorageHandle` + +- `list(namespace): Promise` +- `get(name): Promise` +- `set(name, value): Promise` +- `remove(name): Promise` + +Listed keys include the requested logical namespace and can be passed directly +to `get()` or `remove()`. Records belong to the current user. In ordinary local +ComfyUI, dotted pack prefixes avoid collisions by convention; they do not +isolate extensions from one another. An isolated host can enforce additional +pack scoping. Physical user prefixes are not returned. + +### `CommandsHandle` + +- `register(def: CommandDef): void` +- `notify(def: NotifyDef): void` +- `playSound(def: PlaySoundDef): Promise` +- `run(id): Promise` +- `has(id): boolean` + +`CommandDef` contains namespaced `id`, static or dynamic `label`, `run`, an +optional `KeyCombo`, and optional canvas scope. `KeyCombo` contains `key` plus +optional Ctrl/Alt/Shift/Meta flags. `NotifyDef` contains `summary` and optional +`severity`, `detail`, and lifetime in milliseconds. `PlaySoundDef` contains a +browser-readable `src` and an optional volume from 0 to 1. + +### `UiHandle` + +- `addSidebarTab(def): Unsubscribe` +- `addTopBarBadge(def): ChromeItemHandle` +- `addActionBarButton(def): ChromeItemHandle` +- `showDialog(def): DialogHandle` +- `showMenu(def): MenuHandle` +- `prompt(def): Promise` +- `mountViewportPanel(def): ViewportPanelHandle` + +`mountViewportPanel` places a pack-rendered panel against an edge or corner of +the graph view. A `ViewportPanelDef` has `id`, `render(container)`, and optional +`anchor`, `offsetX`, `offsetY`, `width`, `maxHeight`, `ariaLabel` and +`destroy()`. `anchor` is a `ViewportPanelAnchor` and defaults to `top-left`; +the offsets are pixels in from the anchored sides. `remove()` on the returned +`ViewportPanelHandle` takes the panel down and calls `destroy()`. + +Sidebar definitions share `SidebarTabBase` (`id`, `title`, optional `icon` and +`tooltip`). `MountedSidebarTab` adds `render(container)` and optional +`destroy()`; `VueSidebarTab` adds a bundled `VueComponent`. `SidebarTabDef` is +the union. + +Dialogs share `DialogBase` (`key`, optional `title`). `MountedDialog` adds +`render(container)` and optional `destroy()`; `VueDialog` adds `component` and +optional `props`. `DialogDef` is the union. `DialogHandle.close()` closes it. +The returned handle type is `DialogHandle`. + +`PromptDef` contains `label`, optional initial `value`, and optional +`placeholder`. `MenuDef` contains an anchoring `MouseEvent`, optional `title`, +and `MenuItemDef` entries. A menu item has `label`, optional `disabled`, and +either `submenu` or `run()`. `MenuHandle.close()` closes it. +The returned handle type is `MenuHandle`. + +`BadgeContribution` contains namespaced `id`, `text`, and optional label, +variant, icon, and tooltip. `ButtonContribution` contains namespaced `id`, +icon, optional label/tooltip, and `run(event)`. Their `ChromeItemHandle` can +`update()` all fields except identity or `remove()` the contribution. + +### `BackendHandle` + +- `ownFetch(route, init?): Promise` +- `sessionId(): string | undefined` +- `on(event, listener): Unsubscribe` +- `fetch(route, init?): Promise` + +Both fetch methods accept relative paths to the calling pack's own declared +endpoints, such as `presets`. Absolute paths, traversal, percent-encoded paths, +schemes, and caller-selected prefixes are rejected. The host assigns and retains +the base; shared/core operations use named async `comfy.*` capabilities. +See the [routing contract](./services.md#pack-relative-backend-requests-and-events). +URL builders retained for trusted host adapters are deprecated for pack use. + +### `ModelsHandle` + +- `list(folder: ModelFolder): Promise` + +`ModelFolder` defines the closed set of catalogue folders. `list()` returns +distinct logical model names. The host selects the route and validates +responses; unknown folders, malformed +replies and HTTP failures reject. It does not expose filesystem locations, +library path indexes or arbitrary file reads. See +[Model catalogues](./services.md#model-catalogues) for folders and bounds. + +### `WorkflowHandle` + +- `open(data: WorkflowData): Promise` +- `applyTextReplacements(value: string): string` +- `current(): DocumentHandle | undefined` +- `documentId(): string | undefined` +- `registerImporter(importer: WorkflowImporter): Unsubscribe` + +`WorkflowData` is parsed workflow JSON represented as a readonly record. + +`current()` is the workflow on screen, as a `DocumentHandle`: `id`, `name`, +`path`, `isModified` and `isDeleted`. Its `id` identifies the editing session. +It stays the same for as long as the workflow is open, through undo, redo, tab +switches and renames, and is never reused: closing a workflow and opening it +again starts a new session with a new id. It means nothing after a page reload, +so do not save it. `documentId()` is `current()?.id`, and is absent before a +workflow is open. + +Key per-document pack state on that id. Allocate it in `onDocumentOpened` or +on first use, and release it in `onDocumentClosed`, whose handle reports +`isDeleted`. Node, widget and slot handles have a shorter life than the +session: they stop resolving when another workflow loads into the graph, +including on a tab switch. + +`registerImporter` adds a reader for files the host cannot load a workflow +from, such as a JPEG that carries one in its metadata. A `WorkflowImporter` has +`id`, `maxBytes`, `parse(bytes, context)`, and optional `mimeTypes`, +`extensions` and `enabled()`. When such a file is opened or dropped, importers +whose types or extensions match are asked in registration order, with the +file's bytes, up to their `maxBytes` and never more than 16 MiB. `parse` +receives a `WorkflowImportContext` with the file's `name` and `type`, and +returns a `WorkflowImportResult`, `{ workflow }` or `{ prompt }`, or nothing to +decline. A file the host can read itself is never offered. + +### `SystemHandle` + +- `monitor(): Promise` + +`SystemMonitorSnapshot` contains CPU utilization, total and available memory, +volumes, and accelerators. Unsupported utilization and temperature sensors are +`null`. Volume and accelerator IDs are opaque. Related types are +`SystemMonitorCpu`, `SystemMonitorMemory`, `SystemMonitorVolume`, and +`SystemMonitorAccelerator`. + +See [Application services](./services.md). + +## Implemented capabilities + +This block is generated from `CAPABILITIES` in `comfyApi.ts`. + + + +`backend`, `commands`, `commands.playSound`, `defs.define`, `defs.extend`, +`defs.inputValues`, `defs.localizedInputNames`, `defs.typeCompatibility`, +`execution.node`, `graph.nodes`, `graph.selection`, +`interaction.nodeDragEnd`, `interaction.nodeMoved`, `interaction.state`, +`models.list`, `node.changeScope`, `node.connectVeto`, `node.fileDrop`, +`node.geometry`, `node.menu`, `node.onPreview`, `node.onSerialize`, +`node.resolve`, `node.sizeConstraints`, `queue.disableAutoQueue`, +`queue.settings`, `serialization.control`, `settings`, `slots.connect`, +`slots.connectedType`, `slots.dynamic`, `slots.identity`, `slots.layout`, +`slots.localizedName`, `slots.moveLinks`, `slots.resolvedSource`, +`slots.retype`, `slots.widgetConfig`, `storage`, `supply.outputs`, +`supply.resolved`, `system.monitor`, `ui.sidebarTab`, `viewport.changed`, +`widgets.canvas`, `widgets.create`, `widgets.height`, `widgets.hidden`, +`widgets.linked`, `widgets.mount`, `widgets.reorder`, +`widgets.textInteraction`, `widgets.typeContext`, `workflow.open`, +`workflow.textReplacements`. + + +Use `comfy.capabilities()` at runtime. The generated block exists for discovery +and is checked against the implementation. + +## Errors and absence + +Expected lookup absence returns `undefined` or `false` where the signature says +so. Invalid operations throw named `Error` subclasses, including: + +- `ComfyApiError` for general contract failures; +- `ComfyDeletedError` for a write through a deleted handle; +- `ComfyReadonlyError` for assigning a read-only handle member; +- `ComfyAmbiguousSlotError` for ambiguous explicit slot lookup; +- `ComfyUnsupportedError` for a failed capability requirement. + +The error constructors are not currently exported by `/comfy/api/v2.js`; code +can inspect ordinary `Error` fields but should primarily avoid expected failures +with capability probes, lookup checks, and `isDeleted`. + +## Source files + +| Domain | Implementation | +| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | +| Root and versioning | `src/platform/nodeApi/comfyApi.ts` | +| Definitions | `src/platform/nodeApi/defsRegistry.ts` | +| Graph and groups | `src/platform/nodeApi/graphHandle.ts`, `groupHandle.ts` | +| Nodes | `src/platform/nodeApi/nodeHandle.ts` | +| Slots | `src/platform/nodeApi/slotHandle.ts`, `slotRef.ts` | +| Widgets | `src/platform/nodeApi/widgetHandle.ts`, `widgetTypes.ts`, `widgetTextInteraction.ts` | +| Queue and resolution | `src/platform/nodeApi/queueHandle.ts`, `resolution.ts` | +| Services | `settingsHandle.ts`, `storageHandle.ts`, `commandsHandle.ts`, `uiHandle.ts`, `backendHandle.ts`, `workflowHandle.ts`, `systemHandle.ts` | + +Every domain has focused unit tests beside its implementation. diff --git a/docs/node-api/registration.md b/docs/node-api/registration.md new file mode 100644 index 00000000000..237305ec74d --- /dev/null +++ b/docs/node-api/registration.md @@ -0,0 +1,280 @@ +# Registration, lifecycle, and globals + +Custom-node JavaScript is loaded as an ES module. Register published behavior +from the module body, then use explicit lifecycle signals for work that needs a +running graph or a loaded workflow. + +## Importing the API + +```js +import { comfy } from '/comfy/api/v2.js' +``` + +The frontend installs one API instance immediately before it loads custom-node +modules. `/comfy/api/v2.js` re-exports that instance rather than constructing a +new registry. + +The same object is present as `globalThis.comfy` and `window.comfy`. The global +is useful in the browser console and for diagnosing a host. Extension modules +should import `comfy` so the dependency is explicit and tooling can understand +it. + +Ordinary local ComfyUI loads extension modules into the main page with full +page access. This API provides compatibility and lifecycle contracts; it does +not isolate those modules. An optional extension host can enforce additional +permissions and pack namespaces. Check that host's documented support before +depending on an isolated runtime. + +These legacy globals and modules are not part of this contract: + +- `window.comfyAPI`; +- `window.app`; +- `/scripts/app.js`, `/scripts/api.js`, `/scripts/widgets.js`; +- `LiteGraph` and renderer globals. + +## Versioning and capability probes + +```js +console.info(comfy.version) // API version, currently '2.0' +console.info(comfy.major) // currently 2 +console.info(comfy.getApiVersion()) // currently [2, 0, 0] + +if (comfy.supports('slots.dynamic')) { + installDynamicInputs() +} + +comfy.require('workflow.open') +``` + +Use `supports()` for optional behavior and `require()` for a feature without +which the extension cannot work. Do not compare application versions, parse +`comfy.version`, or probe an internal member. To test the API version itself, +call `comfy.isApiVersionAtLeast('v2.0.0')`: the array `getApiVersion()` +returns cannot be compared with `>=`. + +`comfy.capabilities()` returns a frozen list of everything the host provides. +`comfy.forMajor(major)` pins a public major when a pack deliberately maintains +more than one implementation. + +## Registration at module load + +The API is ready for declarations before the graph has completed setup. These +operations normally belong at module scope: + +```js +comfy.settings.declare({ + id: 'SeedTools.showBadge', + name: 'Show seed badge', + type: 'boolean', + defaultValue: true +}) + +comfy.commands.register({ + id: 'SeedTools.resetSelected', + label: 'Reset selected sampler seeds', + scope: 'canvas', + run() { + for (const node of comfy.graph.selection()) { + node.widgets.get('seed')?.setValue(0) + } + } +}) + +comfy.defs.extend('KSampler', (definition) => { + definition.onCreated((node) => installSamplerBehavior(node)) +}) +``` + +Registration IDs share host-wide namespaces. Prefix setting, command, tab, +dialog, top-bar badge, and action-button IDs with a stable pack name. + +## Extending backend node definitions + +`defs.extend(selector, apply)` is the replacement for +`beforeRegisterNodeDef` and prototype patching: + +```js +const stop = comfy.defs.extend( + { category: /^image\/postprocessing/ }, + (definition) => { + definition.onExecuted((node, result) => { + rememberImages(node, result.images) + }) + } +) +``` + +A selector can be: + +- an exact type string; +- an array of type strings; +- a type-name regular expression; +- `{ category: string | RegExp }`; +- a predicate over `NodeDef` when the other forms cannot express the match. + +Prefer an indexable selector. A predicate must inspect every registered +definition and should be reserved for structural questions such as “any node +with a VAE input.” + +The builder's `def` is the frozen definition after earlier extensions have run. +Registered callbacks compose; there is no previous prototype callback to +capture or invoke. + +## Defining a frontend-owned node type + +Use plain data rather than subclassing `LGraphNode`: + +```js +const unregister = comfy.defs.define({ + type: 'SeedTools/Reroute', + title: 'Seed Tools Reroute', + category: 'Seed Tools', + inputs: [{ name: 'in', type: '*' }], + outputs: [{ name: 'out', type: '*' }], + execution: 'frontend', + resolve({ self }) { + const input = self.input('in') + return { out: input ? { forwardTo: input } : { omit: true } } + } +}) +``` + +The `type` must be globally unique. `define()` returns an unregister function. +`execution: 'frontend'` keeps the node out of the backend prompt. A resolver is +optional: without one, the node is simply omitted. See +[Execution and resolution](./execution.md) before defining execution behavior. + +## Defining an input widget type + +`defineWidgetType()` replaces `getCustomWidgets` for a Python input type: + +```js +const unregister = comfy.defs.defineWidgetType('SEED_TOOLS_COLOR', { + defaultValue: '#ffffff', + minWidth: 120, + render(container, value, name, context) { + const input = document.createElement('input') + input.type = 'color' + input.value = String(value.get()) + input.ariaLabel = name + input.addEventListener('input', () => value.set(input.value)) + container.append(input) + + const stopValue = value.onChange((next) => { + input.value = String(next) + }) + return () => { + stopValue() + input.remove() + } + } +}) +``` + +Type-level widget construction happens before the owner has joined a graph. The +render callback therefore receives a value accessor and a `WidgetTypeContext`, +not a node handle. Use `context.onNodeReady()` when behavior genuinely needs the +owning `NodeHandle`. + +## Lifecycle signals + +### Application ready + +```js +const stop = comfy.onReady(() => { + rebuildIndex(comfy.defs.all()) +}) +``` + +At this point the canvas, settings, graph, and node definitions exist. A listener +registered after readiness still runs on the next microtask. + +### Workflow loaded + +```js +const stop = comfy.onWorkflowLoaded(() => { + restorePackStateFor(comfy.graph.root()) +}) +``` + +This fires after every workflow load, and after an undo or redo, which rebuilds +the same document. It is the replacement for a one-time setup hook when behavior +belongs to each document. The listener receives the open document; its `id` is +unchanged when the same document was rebuilt. + +### Document sessions + +```js +const notes = new Map() + +comfy.onDocumentClosed((document) => notes.delete(document.id)) + +function notesForOpenDocument() { + const id = comfy.workflow.documentId() + if (id === undefined) return undefined + if (!notes.has(id)) notes.set(id, []) + return notes.get(id) +} +``` + +A document's `id` identifies its editing session: the same through undo, redo +and tab switches while the workflow stays open, and never reused. Use +`onDocumentActivated` and `onDocumentDeactivated` for anything tied to being on +screen, and `onDocumentOpened` and `onDocumentClosed` for state that lives as +long as the workflow is open. + +### Definition lifecycle + +Use `NodeDefBuilder` or `NodeDefinition` callbacks for individual instances: + +- `onCreated` after the node joins a graph; +- `onConfigured` after saved data is applied; +- `onRemoved` when it leaves; +- `onExecuted` and `onPreview` for backend results; +- `onConnectionsChanged`, `onResized`, `onHover`, `onDoubleClick`, + `onPropertyChanged`, `onDragOver`, and `onDrop` for semantic editor behavior. + +`NodeCreatedEvent.restored` distinguishes fresh nodes from nodes carrying saved +state. `NodeCreatedEvent.loading` distinguishes workflow load from paste or +duplication. + +## Cleanup and ownership + +Registrations and subscriptions commonly return `Unsubscribe`: + +```js +const cleanup = [ + comfy.defs.extend('KSampler', installDefinition), + comfy.onWorkflowLoaded(rebuild), + comfy.ui.addSidebarTab(tabDefinition) +] + +function disposePack() { + for (const stop of cleanup.splice(0)) stop() +} +``` + +Match cleanup to ownership: + +- module registrations may live for the page; +- a sidebar tab or dialog releases listeners, observers, and timers from its + `destroy` callback; +- a mounted widget releases them from `MountDef.destroy` or the function + returned by a widget type's `render`; +- per-node state should be removed from `onRemoved`. + +Do not attach private fields to a node handle. Keep pack-owned state in a map +keyed by graph ID and node ID, and release it with the node lifecycle. + +## Type contract + +Use the complete declarations checked in as +[comfy-api.d.ts](./comfy-api.d.ts). To regenerate them from the matching +frontend revision: + +```sh +node scripts/node-api/gen_api_dts.mjs > docs/node-api/comfy-api.d.ts +``` + +If a type or member is absent from that file, it is not published. The runtime +global is not a substitute for missing declarations. diff --git a/docs/node-api/services.md b/docs/node-api/services.md new file mode 100644 index 00000000000..baa7ed49f18 --- /dev/null +++ b/docs/node-api/services.md @@ -0,0 +1,422 @@ +# Application services + +The root API exposes narrow services for behavior that legitimately belongs +outside one node: settings, commands, host UI contributions, backend calls, +workflow loading, and per-user storage. + +## Settings + +Declare a setting once at module load: + +```js +comfy.settings.declare({ + id: 'MyPack.previewQuality', + name: 'Preview quality', + type: 'slider', + defaultValue: 80, + category: ['My Pack', 'Preview'], + attrs: { min: 1, max: 100, step: 1 }, + onChange(value, previous) { + rebuildPreview(value, previous) + } +}) +``` + +Setting IDs share one namespace with core and every pack. Include a stable pack +prefix. Redeclaring an ID does not reset an existing user's value. + +Supported controls are: + +- `boolean`, `number`, `slider`, and `knob`; +- `combo` and `radio`; +- `text`, `password`, `color`, `image`, and `url`. + +For combo and radio choices, a string is both stored value and label. Use +`{ value, label }` when they differ so numeric values remain numeric. + +Read, write, and observe settings: + +```js +const quality = comfy.settings.get('MyPack.previewQuality') +await comfy.settings.set('MyPack.previewQuality', 90) + +const stop = comfy.settings.onChange( + 'Comfy.LinkRelease.Action', + (value, previous) => respondToCoreSetting(value, previous) +) +``` + +`onChange()` can observe a setting the pack did not declare. It fires on change, +not on registration. + +Settings are small preferences. Use `comfy.storage` for named content the user +authors. + +## Commands, keybindings, and notifications + +```js +comfy.commands.register({ + id: 'MyPack.centerPrimary', + label: () => `Center ${primaryLabel()}`, + keybinding: { key: 'm', ctrl: true, shift: true }, + scope: 'canvas', + run() { + const node = findPrimaryNode() + if (node) comfy.graph.centerOn(node) + } +}) +``` + +Command IDs must be namespaced. A label may be dynamic and should return +quickly. A keybinding is a default so a user's custom binding wins. + +`scope: 'canvas'` prevents the keybinding from firing while the user is typing +in a node widget or another field. Omit it for an application-wide command. + +Run a host or pack command without reaching into its implementation: + +```js +if (comfy.commands.has('Comfy.MaskEditor.OpenMaskEditor')) { + await comfy.commands.run('Comfy.MaskEditor.OpenMaskEditor') +} +``` + +`run()` rejects when the command does not exist. Use `has()` for an optional +entry. + +Show a notification: + +```js +comfy.commands.notify({ + severity: 'warn', + summary: 'Model unavailable', + detail: 'Refresh node definitions after installing it.', + life: 5000 +}) +``` + +Severity is `success`, `info`, `warn`, or `error` and defaults to `info`. + +## Sidebar tabs + +Hand-written modules can render into a host container: + +```js +const removeTab = comfy.ui.addSidebarTab({ + id: 'MyPack.assets', + title: 'My assets', + icon: 'icon-[lucide--folder]', + render(container) { + mountAssetBrowser(container) + }, + destroy() { + unmountAssetBrowser() + } +}) +``` + +`render()` can run each time the tab becomes visible. Treat it as a mount and +release retained resources from `destroy()`. + +A built pack can instead provide a bundled Vue component: + +```js +comfy.ui.addSidebarTab({ + id: 'MyPack.monitor', + title: 'Monitor', + component: MonitorTab +}) +``` + +Per ADR-EXTENSIONS-PUBLIC-API-0005, a pack bundles its own Vue. Do not import +the host's internal Vue runtime or pass host reactive objects across the +boundary. + +## Top-bar badges and action buttons + +These contributions are declarative so the host retains control of layout and +style: + +```js +const badge = comfy.ui.addTopBarBadge({ + id: 'MyPack.queueState', + text: 'Idle', + variant: 'info', + tooltip: 'My Pack queue state' +}) + +badge.update({ text: 'Running', variant: 'warning' }) +badge.remove() +``` + +```js +const button = comfy.ui.addActionBarButton({ + id: 'MyPack.openPanel', + icon: 'icon-[lucide--panel-right-open]', + label: 'Open My Pack', + run(event) { + openPanel({ detached: event.shiftKey }) + } +}) +``` + +IDs must be namespaced and unique. An update changes only supplied fields. A +removed contribution cannot be updated again. + +If an action also needs a palette entry or shortcut, put behavior in a command +and have the button call `comfy.commands.run()`. + +## Dialogs, menus, and prompts + +### Dialog + +```js +const dialog = comfy.ui.showDialog({ + key: 'MyPack.modelInfo', + title: 'Model information', + render(container) { + renderModelInfo(container) + }, + destroy() { + releaseModelInfo() + } +}) + +dialog.close() +``` + +Dialogs also accept a bundled Vue `component` and optional frozen `props`. +Dialog keys must be namespaced because the host maps them into one dialog +keyspace. + +### Context menu raised by a pack + +```js +const menu = comfy.ui.showMenu({ + title: 'Output type', + event: mouseEvent, + items: [ + { label: 'Image', run: () => choose('IMAGE') }, + { + label: 'Latent', + submenu: [ + { label: 'Samples', run: () => choose('LATENT') }, + { label: 'Noise', disabled: true } + ] + } + ] +}) +``` + +The `MouseEvent` positions the menu. A submenu item is mutually exclusive with +`run`. Use `NodeDefBuilder.addMenuItem()` instead when the host is opening a +node's own context menu. + +### Prompt + +```js +const label = await comfy.ui.prompt({ + label: 'Group name', + value: group.getTitle(), + placeholder: 'Name' +}) +``` + +The result is `undefined` when the user cancels. + +## Pack-relative backend requests and events + +### Calling this pack's declared endpoints + +This section requires the optional Secure Nodes pack-route broker. The ordinary +`/comfy/api/v2.js` instance in this stack does not install that broker: +`backend.fetch` requires a same-origin path beginning with `/`, and +`backend.ownFetch` rejects requests. `backend.url` and `backend.assetUrl` are +available for shared-page URLs. The restrictions below describe the broker's +pack-scoped API. + +```js +const response = await comfy.backend.fetch('presets', { + method: 'GET' +}) +``` + +With the broker, `backend.fetch()` and `backend.ownFetch()` call only endpoints declared by the +calling Secure Nodes pack. New code uses relative endpoint paths, with no +leading slash, empty or dot segments, percent-encoded path, scheme, authority, +backslash, fragment, or control character. Invalid paths are rejected rather +than decoded or normalized. Query values are re-encoded by the host. + +The host assigns the route base from trusted pack identity and constructs +`/ext//` internally. The namespace is stable across +pack revisions and restarts; it is not an authorization secret. Pack code must +not choose a base. The host attaches authentication only after +validating the pack-relative request; a pack cannot select core routes, +another pack's endpoints, or an external server. + +Backend declarations use the same relative path in `secure-nodes.json`: + +```json +{ + "routes": [ + { + "method": "get", + "path": "presets", + "module": "_secure_routes", + "handler": "list_presets", + "permissions": ["storage"] + } + ] +} +``` + +The runtime mounts declarations beneath the system-assigned base and checks +the declared method and endpoint. A pack with no declared routes has no endpoint +to call. Bodies remain bounded data; credentials and server request objects stay +in the host. + +Legacy URL builders (`backend.url()`, `ownUrl()`, and `assetUrl()`) are not a +supported pack API. Trusted-host adapters retained in declarations are +deprecated for pack use. Do not construct `/ext/` URLs or use a plain fetch to +bypass the host broker. + +The interim runtime accepts an older `ownFetch()` request carrying the exact +calling pack's `/ext//` prefix, then strips and rebuilds it under +the host's grant. It also normalizes one legacy leading slash in backend route +declarations with a deprecation warning. These compatibility forms cannot +address another namespace; new pack code uses relative paths. + +### Shared capabilities = named async API methods + +Shared/core operations use named async methods on `comfy.*`, with typed inputs, +bounded results, host validation, and a declared capability where required. +For example, `await comfy.system.monitor()` reads a validated snapshot and +`await comfy.queue.interrupt()` interrupts execution. Queue status has the +local `comfy.queue.pending()` snapshot and `onPendingChanged()` subscription. + +Model listing uses `comfy.models.list(folder)`. Image view/upload and other +shared operations require their +published named capabilities. If the current declaration lacks the needed +method, report an API gap with the intended behavior and bounds. Do not fetch +`/userdata`, `/settings`, `/internal`, `/queue`, `/view`, `/upload`, or another +pack's routes. Proposed methods are unavailable until implemented and published. + +### Model catalogues + +```js +const names = await comfy.models.list('loras') +``` + +`models.list(folder)` returns distinct logical model names, including logical +subfolder names. It exposes no filesystem locations or library path indexes. +`ModelFolder` is closed: `checkpoints`, `clip`, `clip_vision`, `controlnet`, +`diffusion_models`, `embeddings`, `loras`, `text_encoders`, `unet`, +`upscale_models`, `vae`, `ipadapter`, and `vae_approx`. Unknown folders reject; +a pack cannot supply a URL or extend that list by passing a string. + +The host selects the model-library route and falls back to the plain-ComfyUI +catalogue only on HTTP 404. Other HTTP failures and malformed replies reject. +At most 50,000 names of 1,024 characters are admitted, without control characters, +absolute/drive paths or traversal. The secure broker shares concurrent requests, +reuses results for two seconds and allows at most 20 listing starts per ten +seconds per calling pack. These broker checks also apply when a frontend model +service handles the request. Model listing does not grant model loading or other +proposed model operations. + +### Bundled files + +For an admitted file next to the pack module, use a module-relative URL: + +```js +const stylesheet = new URL('./panel.css', import.meta.url) +``` + +The host serves admitted bundled assets. This does not grant a backend route; +do not guess the pack's install directory or route base. + +### Backend messages and session identity + +```js +const stop = comfy.backend.on('my-pack-progress', (detail) => { + updateProgress(detail) +}) + +const session = comfy.backend.sessionId() +``` + +Event payloads are `unknown` because a pack owns its own event schema. Validate +before use. The session ID can be `undefined` until the backend connection is +established and must not be persisted. + +## System monitoring + +Read one validated hardware snapshot from the host: + +```js +const snapshot = await comfy.system.monitor() +console.log(snapshot.memory.available) +``` + +The snapshot contains CPU utilization, total and available memory, mounted +volumes, and accelerators. Unsupported utilization and temperature sensors are +`null`. Treat volume and accelerator IDs as opaque; they identify entries only +within the returned host data. + +## Workflow service + +Open parsed ComfyUI workflow JSON as the active document: + +```js +const data = JSON.parse(text) +await comfy.workflow.open(data) +``` + +This replaces the current document and is therefore an explicit user-facing +action. Validate or confirm untrusted input before calling it. + +Expand the host's workflow text tokens against the active root graph: + +```js +const filename = comfy.workflow.applyTextReplacements( + '%date:yyyy-MM-dd%_%KSampler.seed%' +) +``` + +It throws when no graph is active. + +## Per-user storage + +Storage is for named text documents such as presets, templates, and saved +prompts: + +```js +await comfy.storage.set('MyPack.presets/portrait', JSON.stringify(preset)) + +const text = await comfy.storage.get('MyPack.presets/portrait') +const names = await comfy.storage.list('MyPack.presets') + +await comfy.storage.remove('MyPack.presets/portrait') +``` + +Listed keys include the requested logical folder: `list('MyPack.presets')` +returns keys such as `MyPack.presets/portrait`. Pass each returned key unchanged +to `get()` or `remove()`; do not add the folder again. + +Ordinary local ComfyUI stores these documents for the selected user. Keep a +stable dotted pack prefix to avoid collisions and do not include `..`. These +prefixes are a naming convention, not an isolation boundary between extensions +running in the same page. + +An isolated host can additionally bind the storage namespace to trusted pack +identity. In that mode, choosing another pack's name does not select its data; +the host adds the physical namespace and returns logical names only. + +Secure Nodes hosts that have not implemented this isolation reject pack storage +RPCs. A dotted key alone does not make shared user data safe to expose. + +`get()` returns `undefined` for a missing item; `list()` returns an empty frozen +array when the namespace has no entries. + +The configured host storage backend owns persistence and cross-machine +availability. Use settings for small preferences, and storage for content the +user expects to retain and manage by name. diff --git a/docs/node-api/slots.md b/docs/node-api/slots.md new file mode 100644 index 00000000000..8e618701a2f --- /dev/null +++ b/docs/node-api/slots.md @@ -0,0 +1,231 @@ +# Slots and links + +Slots and links are exposed through stable handles and frozen snapshots. Use +them instead of `node.inputs`, `node.outputs`, `input.link`, `output.links`, or +mutable link records from LiteGraph. + +## Slot identity and references + +A slot index is a position and changes when the slot list changes. A `SlotId` is +stable for the lifetime of the slot. + +Methods accepting `SlotRef` support: + +```js +node.inputs.get('model') // exact name, preferred +node.inputs.get(slotId) // stable runtime identity +node.inputs.get({ index: 0 }) // explicit positional access +``` + +A bare number is deliberately not accepted. Use `{ index }` so a volatile +positional dependency is visible in code review and search. + +String resolution is: + +1. exact `SlotId`; +2. exact slot name; +3. while named slots are unavailable, a canonical integer string such as `'0'` + resolves positionally; +4. no match. + +`get(name)` throws `ComfyAmbiguousSlotError` when more than one slot has that +name. `byName(name)` returns `undefined` on ambiguity. Output names may +legitimately repeat, so use a slot ID to select a particular output. + +Slot IDs are runtime identity. Workflows still serialize link endpoints by +index, so do not persist a `SlotId` across save and reload. + +## Collections + +Every node has input and output collections: + +```js +const inputs = node.inputs +const outputs = node.outputs + +inputs.length +inputs.get('model') +inputs.byId(slotId) +inputs.byName('model') +inputs.at(0) +inputs.all() +inputs.ids() +inputs.names() +``` + +The returned arrays are frozen snapshots. Collections are iterable and their +handles remain operation-oriented. + +## Inspecting an input + +```js +const input = node.inputs.get('model') +if (input) { + console.info(input.id, input.index, input.name, input.type) + console.info(input.label, input.connectedType, input.isConnected) +} +``` + +Important input reads: + +| Method or field | Meaning | +| --------------------------- | ------------------------------------------------------------------------------- | +| `link()` | Frozen `LinkInfo` for the physical incoming link. | +| `source()` | Physical source node ID and output index. Stops at a frontend reroute. | +| `resolvedSource()` | Executable source after frontend-node resolution: output, literal, or omission. | +| `connectedType` | Type arriving through the connection, including a subgraph boundary. | +| `isWidgetInput` | Whether the slot is the socket form of a widget. | +| `widgetConfig()` | Input declaration used by a connected Primitive node. | +| `mergeWidgetConfig(config)` | Compatible intersection of this and another declaration. | +| `snapshot()` | Frozen `SlotSnapshot`. | + +Use `source()` to edit physical topology. Use `resolvedSource()` for synchronous +inspection through reroutes, Get/Set nodes, and other frontend resolvers. +Promise-returning resolvers are unresolved in this view; prompt construction +awaits them. + +## Inspecting an output + +```js +const output = node.outputs.get('IMAGE') +if (output) { + const links = output.links() + const targets = output.targets() +} +``` + +`links()` and `targets()` are snapshots, so it is safe to iterate while +disconnecting. + +## Connecting and disconnecting + +```js +const made = source.outputs.get('IMAGE')?.connectTo(target.id, 'image') + +target.inputs.get('image')?.disconnect() +source.outputs.get('IMAGE')?.disconnect(target.id) +source.outputs.get('IMAGE')?.disconnect() // all targets +``` + +`connectTo()` returns the new `LinkInfo`, or `undefined` when the endpoint does +not exist or the host rejects the connection. Normal compatibility and +definition hooks still apply. + +Do not update `LinkInfo`; it is a record of one observation. Perform the edit +through the slot handles. + +## Adding and removing dynamic slots + +```js +const input = node.inputs.add('image_3', 'IMAGE', { + shape: 'optional', + localizedName: 'Third image' +}) + +node.outputs.add('batch', 'IMAGE', { shape: 'list' }) +node.inputs.remove(input.id) +``` + +`SlotOptions` supports: + +- `shape`: `'default'`, `'optional'`, `'list'`, or `'directional'`; +- `localizedName`; +- a custom graph-space `position` and link `direction`; +- `widget`, naming the widget whose socket form this input represents; +- `widgetConfig`, the declaration a connected Primitive should render. + +Shapes, localized names, positions, directions, colors, and widget-input +metadata can affect saved workflow bytes. Reproduce the original declaration +when wire compatibility matters. + +Removing a slot disconnects its links. + +## Reordering slots safely + +```js +node.inputs.reorder(['model', 'positive', 'negative', 'latent_image']) +``` + +The names must be a complete permutation of the current slots. The host updates +every affected link endpoint as part of the operation and preserves link IDs. +Permuting an internal slot array directly would silently retarget connections, +because serialized links store endpoint indexes. + +The slot order itself is serialized, so reordering changes the workflow by +design while preserving which logical slots its links reach. + +## Modifying a slot + +Apply related changes atomically: + +```js +node.outputs.get('value')?.modify({ + name: 'model', + label: 'MODEL', + type: 'MODEL', + shape: 'directional' +}) +``` + +`SlotPatch` supports name, label, localized name, type, position, direction, +connected and unconnected colors, and shape. `InputSlotPatch` additionally +supports widget identity and widget configuration. + +A type may be a string or an array of accepted types. The host normalizes an +array to the comma-separated form used by its compatibility checks. + +Retyping keeps existing links. Dynamic wildcard-to-concrete nodes depend on +that behavior; explicitly disconnect a link when the feature requires it. + +## Moving output links without replacing them + +```js +const moved = node.outputs.get('old_output')?.moveLinksTo('new_output') +``` + +`moveLinksTo()` moves every link to another output on the same node and +preserves link IDs. Disconnecting and reconnecting would allocate new IDs and +change the serialized workflow. + +The move deliberately does not revalidate types. The observed migration pattern +moves links away and then retypes a slot; checking compatibility halfway through +would reject that valid sequence. + +## Widget-backed inputs + +A dynamic input that is the socket form of a widget must carry the relationship: + +```js +node.inputs.add('strength', 'FLOAT', { + widget: 'strength', + widgetConfig: { + type: 'FLOAT', + options: { default: 1, min: 0, max: 2, step: 0.05 } + } +}) +``` + +This is not cosmetic. Widget-backed inputs serialize differently from ordinary +sockets, and the widget keeps its position in `widgets_values`. + +`InputSlotHandle.modify({ widget, widgetConfig })` updates an existing input. +Use `null` for `widget` to clear the relationship. + +## Physical and resolved topology + +Frontend-only nodes can forward, replace, or omit execution values without +changing physical links. Keep the distinction explicit: + +```js +const physical = input.source() +const executable = input.resolvedSource() +``` + +An executable source is one of: + +- `{ kind: 'output', graphId, nodeId, outputIndex }`; +- `{ kind: 'literal', value }`; +- `{ kind: 'omitted', reason }`. + +Resolution is read-only and scoped to the input's graph. See +[Execution and resolution](./execution.md) for defining resolvers and suppliers. diff --git a/docs/node-api/tutorial.md b/docs/node-api/tutorial.md new file mode 100644 index 00000000000..fdbfbedeb00 --- /dev/null +++ b/docs/node-api/tutorial.md @@ -0,0 +1,166 @@ +# Tutorial: extend an existing node + +This tutorial adds a live seed badge and a “Reset seed” menu item to the built-in +`KSampler` node. It demonstrates module registration, definition hooks, node and +widget handles, a user-visible mutation, and lifecycle timing without touching a +frontend global or prototype. + +## 1. Create the pack layout + +```text +seed_tools/ +├── __init__.py +└── web/ + └── seed-tools.js +``` + +Expose the web directory from `__init__.py`: + +```py +NODE_CLASS_MAPPINGS = {} +WEB_DIRECTORY = "./web" + +__all__ = ["NODE_CLASS_MAPPINGS", "WEB_DIRECTORY"] +``` + +ComfyUI discovers JavaScript modules in that directory when it loads the custom +node package. + +## 2. Import the API + +Create `web/seed-tools.js`: + +```js +import { comfy } from '/comfy/api/v2.js' + +comfy.require('defs.extend') +``` + +The published module is installed before custom-node modules evaluate. Import +it directly; do not import `/scripts/app.js`, read `window.comfyAPI`, or wait for +a DOM element. + +`require()` makes the dependency explicit. If the host is too old, the error +names the missing capability and the API version. For an optional enhancement, +use `supports()` and skip only that enhancement instead. + +## 3. Extend `KSampler` + +Register the definition behavior at module scope: + +```js +comfy.defs.extend('KSampler', (definition) => { + definition.onCreated((node) => { + node.addBadge(() => { + const seed = node.widgets.get('seed')?.getValue() + return { + text: `Seed ${String(seed ?? 'unset')}`, + bgColor: '#334155' + } + }) + }) + + definition.addMenuItem({ + label: 'Reset seed', + run(node) { + node.widgets.get('seed')?.setValue(0) + } + }) +}) +``` + +There is no constructor and no prototype patch: + +- `defs.extend()` selects a registered node type; +- `onCreated()` receives an ID-backed `NodeHandle` after the node joins a graph; +- `node.widgets.get()` finds the seed by stable name rather than array position; +- `setValue()` commits through the same value protocol as a user edit; +- `addBadge()` asks the host to render node chrome under either renderer. + +The badge callback is evaluated when the node is drawn, so it reads the latest +seed. Keep a dynamic badge callback fast. For expensive work, update cached pack +state from a widget `change` listener and have the badge read that state. + +## 4. Add feedback after execution + +Extend a `PreviewImage` node downstream of your sampler to observe its delivered +UI result. `onExecuted` observes these results, not every node completion: +`KSampler` returns latent data without a UI result. + +```js +comfy.defs.extend('PreviewImage', (definition) => { + definition.onExecuted((node, result) => { + comfy.commands.notify({ + severity: 'success', + summary: 'Preview ready', + detail: `${node.getTitle()} produced ${result.images.length} image(s)`, + life: 2500 + }) + }) +}) +``` + +Multiple extensions of the same type compose. They do not need to capture and +call a previous callback. In a real pack, keep related hooks in one +`defs.extend()` call when that makes the behavior easier to read. + +## 5. Use application readiness only when needed + +Definition registration belongs at module scope. Work that needs the initialized +graph belongs behind `onReady`: + +```js +const stopReady = comfy.onReady(() => { + const count = comfy.graph.nodesOfType('KSampler').length + console.info(`[seed-tools] ${count} sampler(s) in the visible graph`) +}) +``` + +`onReady` fires once. If the extension must repeat work after every workflow +open, use `onWorkflowLoaded` instead. + +The returned function unsubscribes. A module-lifetime listener normally lives +as long as the page; a listener owned by a tab, dialog, widget, or node should be +released when that owner is destroyed. + +## 6. Make an optional feature degrade cleanly + +Suppose a later version anchors a panel to the node's screen rectangle: + +```js +if (comfy.supports('viewport.changed')) { + const stop = comfy.onViewportChanged(() => { + for (const node of comfy.graph.nodesOfType('KSampler')) { + const rectangle = node.getScreenRect() + if (rectangle) updatePanel(node.id, rectangle) + } + }) +} +``` + +The capability check describes the needed behavior. Do not compare frontend +application versions or test whether an internal property happens to exist. + +## 7. Verify behavior + +Exercise at least these cases in a real frontend: + +1. create a fresh KSampler and confirm the badge appears; +2. change the seed manually and confirm the badge follows it; +3. choose “Reset seed” and confirm linked seed controls and serialization react + as they would to a user edit; +4. duplicate, save, reload, and delete the node; +5. connect the sampler through VAE Decode to a PreviewImage node, run a fresh + workflow execution, and confirm the notification names that preview node; +6. repeat under every renderer the pack claims to support. + +Static type and conformance checks cannot prove those interactions. They are the +final behavior contract a user experiences. + +## Next steps + +- [Migration how-to](./how-to.md) +- [Registration and lifecycle](./registration.md) +- [Nodes and definitions](./nodes.md) +- [Widgets](./widgets.md) +- [API reference](./reference.md) diff --git a/docs/node-api/widgets.md b/docs/node-api/widgets.md new file mode 100644 index 00000000000..a5913051327 --- /dev/null +++ b/docs/node-api/widgets.md @@ -0,0 +1,342 @@ +# Widgets + +Widgets are addressed by name through `node.widgets`. The API separates +ordinary value widgets, per-node mounted surfaces, per-node canvas surfaces, +and type-level widget renderers. + +## Widget collections + +```js +const widgets = node.widgets + +widgets.length +widgets.get('seed') +widgets.at(0) +widgets.all() +widgets.names() +``` + +Use names whenever possible. `at(index)` is explicitly positional and becomes +stale when widgets are inserted, removed, or reordered. + +All list reads are frozen snapshots. The collection itself provides mutation +operations: + +```js +widgets.add({ + type: 'button', + name: 'refresh' +}) + +widgets.move('refresh', 0) +widgets.reorder(['refresh', 'model', 'seed']) +widgets.remove('refresh') +``` + +`reorder()` requires every current name exactly once. It updates both legacy +and Nodes 2.0 render order. `add()` rejects duplicate names. + +For behavior that belongs to every node of a type, prefer +`NodeDefBuilder.addWidget()` and install listeners from `onCreated`. + +## Values and commits + +```js +const seed = node.widgets.get('seed') +if (seed) { + const current = seed.getValue() + seed.setValue(Number(current) + 1) +} +``` + +`setValue()` is not a bare assignment. It commits exactly as the host's user +edit protocol does: + +- writes the value; +- synchronizes a property-backed widget; +- runs the host and pack callback chain; +- calls node widget-change behavior; +- fires one `change` event; +- advances graph change state. + +Writing an equal value is a no-op. A programmatic commit never fires +`activate`; that event represents a user act. + +## Widget events + +```js +const stopChange = widget.on('change', (value, previous) => { + updatePreview(value, previous) +}) + +const stopActivate = widget.on('activate', () => { + performAction() +}) + +const stopRemoved = widget.on('removed', () => { + releaseWidgetState() +}) +``` + +| Event | Use it for | +| ----------------- | -------------------------------------------------------------------------------------------- | +| `change` | A committed value changed. Receives new and old values. | +| `activate` | The user acted: clicked a button or committed a control. Programmatic writes do not fire it. | +| `removed` | The widget left the node. | +| `textInteraction` | Caret, input, keyboard, selection, and wheel behavior for a host-owned multiline editor. | +| `beforeSerialize` | Synchronously replace the value for workflow, prompt, or embedded serialization. | + +Listeners are additive. Do not capture or replace `widget.callback`. + +## Visibility, disabled state, labels, and options + +```js +widget.setHidden(true) +widget.setDisabled(true) +widget.setLabel('Resolved model') +widget.setOption('min', 0) +widget.setOption('max', 1) +``` + +`setHidden()` is the replacement for assigning the special +`'converted-widget'` type. It retains the value and cascades to linked controls. +Visibility, disabled state, and serialization are independent. + +Use: + +- `isHidden()`; +- `isDisabled()`; +- `isSerialized()`; +- `getOptions()` for a frozen options view; +- `setOption(key, value)` for one option. + +Options include common numeric bounds, combo values, read-only and multiline +flags, placeholders, presentation flags, and widget-input metadata. Do not +mutate the object returned by `getOptions()`. + +## Linked widgets + +Compound controls use an explicit relationship: + +```js +seed.setLinked(['control_after_generate']) +const controls = seed.linked() +``` + +Every name passed to `setLinked()` must identify another widget on the node. +Pass an empty array to clear the relationship. Hiding the owner automatically +hides linked controls; reading `linked()` is useful when their current values +affect behavior. + +## Height and layout + +```js +widget.setHeight(120) +const allocated = widget.getHeight() +``` + +`setHeight()` pins the widget's allocation in graph units. `getHeight()` returns +the latest host allocation, or `undefined` before layout. + +This differs from `MountDef.height`: the mount option gives the inner container +a height, while `WidgetHandle.setHeight()` changes how much node layout assigns +to the widget. Leave the height unpinned for an editor intended to fill spare +space. + +Use `node.setSizeConstraints()` for node-level minimum, maximum, or auto-height +behavior instead of reassigning a size callback. + +## Serialization + +### Declared widgets + +`WidgetDef.serialize` controls whether an added widget writes to the saved +workflow. The value is still sent in the prompt. To save a value without +sending it, which is what a legacy `serializeValue` override returning nothing +did, set `options: { serialize: false }` instead. + +```js +definition.addWidget({ + type: 'text', + name: 'status', + value: '', + disabled: true, + serialize: false +}) +``` + +### Destination-specific values + +```js +widget.on('beforeSerialize', (event) => { + if (event.context === 'prompt') { + event.setSerializedValue(expandPrompt(String(event.value))) + } +}) +``` + +Contexts are: + +- `workflow`: a workflow saved by the user; +- `prompt`: the API payload sent for execution; +- `embedded`: the workflow copy embedded into output from that prompt. + +`setSerializedValue()` changes that write only. The live widget stays unchanged. +Handlers are synchronous and the last replacement wins. + +## Mounting a DOM control on one node + +`widgets.mount()` is the replacement for `addDOMWidget`: + +```js +let stopInput + +const handle = node.widgets.mount({ + name: 'strength_editor', + defaultValue: 1, + serialize: true, + sendToPrompt: true, + height: 36, + render(container, value) { + const input = document.createElement('input') + input.type = 'range' + input.min = '0' + input.max = '2' + input.step = '0.05' + input.value = String(value.get()) + + const onInput = () => value.set(Number(input.value)) + input.addEventListener('input', onInput) + container.append(input) + + const stopValue = value.onChange((next) => { + input.value = String(next) + }) + stopInput = () => { + input.removeEventListener('input', onInput) + stopValue() + } + }, + destroy() { + stopInput?.() + } +}) +``` + +The host owns mounting and removal. Release listeners, timers, observers, and +other retained resources in `destroy()`. + +Mount behavior: + +- `defaultValue` makes the mount value-holding; omit it for decoration; +- `serialize` defaults to `true` for a value-holding control and `false` for a + decorative mount; +- `sendToPrompt` defaults to `serialize` and can differ for “saved but not sent” + readouts; +- `hideOnZoom` defaults to `true`; +- `height` reserves an inner container height; +- `hidden` controls initial visibility. + +The `MountedValue` accessor offers `get()`, `set(value)`, and `onChange()`. +Object defaults are cloned per node so instances do not share mutable data. + +## A canvas surface owned by the widget + +`widgets.canvas()` lets a pack keep canvas drawing code without drawing into the +host's shared graph canvas: + +```js +let meterWidth = 1 + +const surface = node.widgets.canvas({ + name: 'meter', + height: 48, + defaultValue: 0, + serialize: false, + draw(context, [width, height], theme, value) { + meterWidth = width + const amount = Number(value?.get() ?? 0) + context.fillStyle = theme.surface + context.fillRect(0, 0, width, height) + context.fillStyle = theme.text + context.fillRect(0, 0, width * amount, height) + }, + onPointerDown({ x, event }) { + event.preventDefault() + surface.widget.setValue(Math.max(0, Math.min(1, x / meterWidth))) + } +}) + +surface.redraw() +``` + +The surface is a pack-owned DOM canvas positioned by either renderer. The host +handles backing-store scaling and supplies design-system colors on every draw. + +Pointer coordinates are in the same CSS-pixel coordinate system as `draw()`. +The underlying `PointerEvent` carries buttons and modifiers. A context-menu +handler claims secondary-click behavior; without one, the node context menu +continues to work. + +`CanvasHandle.widget` exposes the ordinary widget handle. Call `redraw()` when +external data used by `draw()` changes. + +## Interacting with a host text editor + +Do not reach for `widget.inputEl`. Subscribe to `textInteraction`: + +```js +widget.on('textInteraction', (event) => { + if (event.kind === 'keydown' && event.key === 'Enter' && event.ctrlKey) { + event.preventDefault() + runText(event.value) + } + + if (event.kind === 'input') { + updateCompletions(event.value, event.selection, event.menuEvent) + } +}) +``` + +All variants expose the current string, selection, a `menuEvent` for positioning +a host menu, `setValue()` with optional restored selection, and `focus()`. +Keyboard and wheel variants expose their relevant modifiers and cancellation +methods. + +## Defining a widget type + +Use `defs.defineWidgetType()` when a backend input type needs a renderer on +every node: + +```js +const unregister = comfy.defs.defineWidgetType('MY_PACK_RATING', { + defaultValue: 0, + minWidth: 100, + serialize: true, + render(container, value, name, context) { + const control = buildRatingControl(container, name, value.get()) + control.onChange((rating) => value.set(rating)) + + const stopValue = value.onChange((rating) => control.set(rating)) + const stopReady = context.onNodeReady((node) => + installNodeSpecificRatingBehavior(node, control) + ) + + return () => { + stopReady() + stopValue() + control.destroy() + } + } +}) +``` + +This is different from `widgets.mount()`: + +- `defineWidgetType()` declares presentation for an input type before its nodes + join a graph; +- `mount()` adds one widget to one live node; +- a type-level renderer gets a `WidgetTypeContext`, and obtains a node later + through `onNodeReady`; +- registered input types remain widgets rather than silently becoming sockets, + which affects both workflow and prompt serialization. diff --git a/examples/node-api/README.md b/examples/node-api/README.md new file mode 100644 index 00000000000..78fa17050e8 --- /dev/null +++ b/examples/node-api/README.md @@ -0,0 +1,33 @@ +# Published node API examples + +These installable custom-node packs demonstrate one published frontend API +idea at a time. They import only `/comfy/api/v2.js`; none of their frontend +code imports ComfyUI source files, patches generated node classes, or reaches +through the legacy `app` global. + +| Pack | Focus | +| -------------------------- | ----------------------------------------------------------------- | +| `how_to_frontend_nodes` | frontend-only nodes, literals, reroutes, suppliers, dynamic slots | +| `how_to_widgets` | widget events, canvas and DOM widgets, custom widget types | +| `how_to_graph_interaction` | badges, menus, lifecycle, file drops, graph mutation | +| `how_to_execution` | queueing, results, mask editor, backend routes/events, storage | + +## Install + +Copy each pack you want into the backend's `custom_nodes` directory: + +```sh +cp -R examples/node-api/how_to_* /path/to/ComfyUI/custom_nodes/ +``` + +Restart ComfyUI and search the node library for `API Examples`. Each pack has +its own README with a short exercise and the API features it demonstrates. + +The repository's `pnpm container:start` command mounts all four packs for local +browser testing. `browser_tests/tests/nodeApi/howToExamples.spec.ts` exercises +registration, prompt resolution, widget events, backend calls/events, graph +batching, undo, and a real backend run. + +The examples target API major 2 explicitly. They call `require()` for every +capability central to the example so an older frontend fails with an actionable +message instead of silently losing behavior. diff --git a/examples/node-api/how_to_execution/README.md b/examples/node-api/how_to_execution/README.md new file mode 100644 index 00000000000..87e6a9922d8 --- /dev/null +++ b/examples/node-api/how_to_execution/README.md @@ -0,0 +1,18 @@ +# How-To: execution and services + +This pack combines small backend nodes/routes with the published frontend API: + +- **Mask Editor Image** uses the backend's ordinary upload image node and adds + an Open Mask Editor action through the host command service. Load and run an + image before opening the editor. +- **Text Output** queues itself from a context-menu action and consumes its + correlated `onExecuted` text result in a dynamic badge. +- **Backend Ping** calls a pack-owned authenticated route, validates the JSON, + updates a widget, and raises a host notification. +- **Backend Event** posts to a pack route and consumes the validated custom + websocket event through `backend.on()`. +- **Settings and Storage** declares a preference and command, then saves and + restores a named per-user greeting. + +The Python is deliberately small. It exists only where the demonstrated +frontend behavior needs a real executable node, upload widget, route, or event. diff --git a/examples/node-api/how_to_execution/__init__.py b/examples/node-api/how_to_execution/__init__.py new file mode 100644 index 00000000000..ba51731cd2e --- /dev/null +++ b/examples/node-api/how_to_execution/__init__.py @@ -0,0 +1,5 @@ +from .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS + +WEB_DIRECTORY = "./web" + +__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"] diff --git a/examples/node-api/how_to_execution/nodes.py b/examples/node-api/how_to_execution/nodes.py new file mode 100644 index 00000000000..7b30e4fc842 --- /dev/null +++ b/examples/node-api/how_to_execution/nodes.py @@ -0,0 +1,60 @@ +import json + +from aiohttp import web +from comfy.comfy_types import InputTypeDict +from nodes import LoadImage +from server import PromptServer + + +class HowToMaskEditor(LoadImage): + CATEGORY = "API Examples/Execution" + + +class HowToTextOutput: + @classmethod + def INPUT_TYPES(cls) -> InputTypeDict: + return { + "required": { + "text": ( + "STRING", + {"default": "Run this node", "multiline": True}, + ) + } + } + + RETURN_TYPES = () + FUNCTION = "run" + OUTPUT_NODE = True + CATEGORY = "API Examples/Execution" + + def run(self, text): + return {"ui": {"text": [text]}, "result": ()} + + +@PromptServer.instance.routes.get("/how-to-api/ping") +async def ping(_request): + return web.json_response({"ok": True, "message": "Hello from Python"}) + + +@PromptServer.instance.routes.post("/how-to-api/event") +async def emit_event(request): + try: + body = await request.json() + except (json.JSONDecodeError, UnicodeDecodeError) as error: + raise web.HTTPBadRequest(text="Expected a JSON object") from error + if not isinstance(body, dict): + raise web.HTTPBadRequest(text="Expected a JSON object") + message = str(body.get("message", "Hello from a backend event")) + PromptServer.instance.send_sync("how-to-api-event", {"message": message}) + return web.json_response({"ok": True}) + + +NODE_CLASS_MAPPINGS = { + "HowToMaskEditor": HowToMaskEditor, + "HowToTextOutput": HowToTextOutput, +} + +NODE_DISPLAY_NAME_MAPPINGS = { + "HowToMaskEditor": "How-To: Mask Editor Image", + "HowToTextOutput": "How-To: Text Output", +} diff --git a/examples/node-api/how_to_execution/web/execution.js b/examples/node-api/how_to_execution/web/execution.js new file mode 100644 index 00000000000..bb3109631f2 --- /dev/null +++ b/examples/node-api/how_to_execution/web/execution.js @@ -0,0 +1,277 @@ +import { comfy } from '/comfy/api/v2.js' + +const api = comfy.forMajor(2) + +api.require('backend') +api.require('commands') +api.require('defs.define') +api.require('graph.selection') +api.require('node.menu') +api.require('settings') +api.require('storage') + +const textResults = new Map() +const textResultCleanups = new Map() +const keyOf = (node) => `${node.graphId}:${node.id}` +const errorDetail = (error) => + error instanceof Error ? error.message : String(error) +const reportActionError = (summary, error, status) => { + const detail = errorDetail(error) + status?.setValue(`${summary} failed: ${detail}`) + api.commands.notify({ severity: 'error', summary, detail }) +} +const runAction = (action, onError) => { + void action().catch(onError) +} +const isRecord = (value) => value !== null && typeof value === 'object' +const backendMessage = (body) => { + if (!isRecord(body) || body.ok !== true) { + throw new Error('Invalid backend response') + } + if (typeof body.message !== 'string') { + throw new Error('Invalid backend response') + } + return body.message +} +const eventMessage = (detail) => { + if (!isRecord(detail)) return undefined + return typeof detail.message === 'string' ? detail.message : undefined +} + +api.defs.extend('HowToTextOutput', (definition) => { + definition.onCreated((node) => { + const key = keyOf(node) + textResults.set(key, 'not run') + textResultCleanups.set( + key, + node.addBadge(() => ({ text: textResults.get(key) ?? 'not run' })) + ) + }) + definition.onExecuted((node, result) => { + textResults.set(keyOf(node), result.text[0] ?? 'complete') + }) + definition.onRemoved((node) => { + const key = keyOf(node) + textResultCleanups.get(key)?.() + textResultCleanups.delete(key) + textResults.delete(key) + }) + definition.addMenuItem({ + label: 'Run This Node', + run: (node) => { + runAction( + () => api.queue.run({ nodes: [node] }), + (error) => reportActionError('Run node', error) + ) + } + }) +}) + +const openMaskEditor = async (node) => { + if (!api.commands.has('Comfy.MaskEditor.OpenMaskEditor')) { + api.commands.notify({ + severity: 'warn', + summary: 'Mask editor is unavailable' + }) + return + } + api.graph.select([node]) + await api.commands.run('Comfy.MaskEditor.OpenMaskEditor') +} + +api.defs.extend('HowToMaskEditor', (definition) => { + definition.onCreated((node) => { + node.addBadge(() => ({ text: `${node.getOutputImages().length} image(s)` })) + }) + definition.addMenuItem({ + label: 'Open Mask Editor', + when: (node) => node.getOutputImages().length > 0, + run: (node) => { + runAction( + () => openMaskEditor(node), + (error) => reportActionError('Open mask editor', error) + ) + } + }) +}) + +const pingBackend = async (node) => { + const response = await api.backend.fetch('/how-to-api/ping') + if (!response.ok) { + throw new Error(`HTTP ${response.status}`) + } + const message = backendMessage(await response.json()) + node.widgets.get('status')?.setValue(message) + api.commands.notify({ + severity: 'success', + summary: 'Backend ping', + detail: message + }) +} + +api.defs.define({ + type: 'HowTo/BackendPing', + title: 'How-To: Backend Ping', + category: 'API Examples/Execution', + description: 'Calls and validates a pack-owned backend route.', + widgets: [ + { type: 'button', name: 'ping', value: null, serialize: false }, + { + type: 'text', + name: 'status', + value: 'not called', + disabled: true, + serialize: false + } + ], + execution: 'frontend', + onCreated(node) { + const status = node.widgets.get('status') + node.widgets.get('ping')?.on('activate', () => { + runAction( + () => pingBackend(node), + (error) => reportActionError('Backend ping', error, status) + ) + }) + } +}) + +const eventStatuses = new Map() + +api.backend.on('how-to-api-event', (detail) => { + const message = eventMessage(detail) + if (message === undefined) return + for (const widget of eventStatuses.values()) { + widget.setValue(message) + } +}) + +const emitBackendEvent = async (node) => { + const response = await api.backend.fetch('/how-to-api/event', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ message: `Event for node ${node.id}` }) + }) + if (!response.ok) { + throw new Error(`HTTP ${response.status}`) + } +} + +api.defs.define({ + type: 'HowTo/BackendEvent', + title: 'How-To: Backend Event', + category: 'API Examples/Execution', + description: 'Emits and validates a pack-owned backend event.', + widgets: [ + { type: 'button', name: 'emit_event', value: null, serialize: false }, + { + type: 'text', + name: 'status', + value: 'waiting', + disabled: true, + serialize: false + } + ], + execution: 'frontend', + onCreated(node) { + const status = node.widgets.get('status') + if (status) eventStatuses.set(keyOf(node), status) + node.widgets.get('emit_event')?.on('activate', () => { + runAction( + () => emitBackendEvent(node), + (error) => reportActionError('Backend event', error, status) + ) + }) + }, + onRemoved(node) { + eventStatuses.delete(keyOf(node)) + } +}) + +api.settings.declare({ + id: 'HowTo.ApiExamples.greeting', + name: 'How-To example greeting', + type: 'text', + defaultValue: 'Hello from settings', + category: ['API Examples', 'How-To'] +}) + +api.commands.register({ + id: 'HowTo.ApiExamples.showGreeting', + label: 'How-To: Show saved greeting', + keybinding: { key: 'h', ctrl: true, shift: true }, + scope: 'canvas', + run() { + api.commands.notify({ + severity: 'info', + summary: 'Saved greeting', + detail: String(api.settings.get('HowTo.ApiExamples.greeting')) + }) + } +}) + +const saveGreeting = async (greeting, status) => { + const value = String(greeting?.getValue() ?? '') + await api.settings.set('HowTo.ApiExamples.greeting', value) + await api.storage.set('HowTo.ApiExamples/greeting', value) + status?.setValue('Saved') +} + +const loadGreeting = async (greeting, status) => { + const value = await api.storage.get('HowTo.ApiExamples/greeting') + if (value !== undefined) greeting?.setValue(value) + status?.setValue(value === undefined ? 'Nothing saved' : 'Loaded') +} + +const runGreetingCommand = async (status) => { + await api.commands.run('HowTo.ApiExamples.showGreeting') + status?.setValue('Command completed') +} + +api.defs.define({ + type: 'HowTo/SettingsStorage', + title: 'How-To: Settings and Storage', + category: 'API Examples/Execution', + description: 'Uses settings, commands, and named per-user storage.', + widgets: [ + { + type: 'text', + name: 'greeting', + value: 'Hello from storage', + serialize: true + }, + { type: 'button', name: 'save', value: null, serialize: false }, + { type: 'button', name: 'load', value: null, serialize: false }, + { type: 'button', name: 'run_command', value: null, serialize: false }, + { + type: 'text', + name: 'status', + value: 'ready', + disabled: true, + serialize: false + } + ], + execution: 'frontend', + onCreated(node) { + const greeting = node.widgets.get('greeting') + const status = node.widgets.get('status') + node.widgets.get('save')?.on('activate', () => { + runAction( + () => saveGreeting(greeting, status), + (error) => reportActionError('Save greeting', error, status) + ) + }) + node.widgets.get('load')?.on('activate', () => { + runAction( + () => loadGreeting(greeting, status), + (error) => reportActionError('Load greeting', error, status) + ) + }) + node.widgets.get('run_command')?.on('activate', () => { + runAction( + () => runGreetingCommand(status), + (error) => reportActionError('Run greeting command', error, status) + ) + }) + } +}) diff --git a/examples/node-api/how_to_frontend_nodes/README.md b/examples/node-api/how_to_frontend_nodes/README.md new file mode 100644 index 00000000000..6d5ef98be4e --- /dev/null +++ b/examples/node-api/how_to_frontend_nodes/README.md @@ -0,0 +1,17 @@ +# How-To: frontend nodes + +This pack defines four nodes that remain in the saved workflow but resolve out +of the backend prompt: + +- **Constant Text** returns a widget value as a literal. +- **Reroute** forwards its input. +- **Broadcast Text** supplies its value to unconnected `STRING` inputs in the + same group. +- **First Connected** adds another input whenever its final input is connected, + then forwards the first connected value. + +Try `Constant Text -> Reroute -> Preview as Text`. To try the broadcaster, put +it and a node with an unconnected string input inside the same group. + +The relevant APIs are `defs.define`, pure `resolve`/`supply` callbacks, and +`node.inputs.add()` from a connection lifecycle event. diff --git a/examples/node-api/how_to_frontend_nodes/__init__.py b/examples/node-api/how_to_frontend_nodes/__init__.py new file mode 100644 index 00000000000..065398a0d6f --- /dev/null +++ b/examples/node-api/how_to_frontend_nodes/__init__.py @@ -0,0 +1,5 @@ +NODE_CLASS_MAPPINGS = {} +NODE_DISPLAY_NAME_MAPPINGS = {} +WEB_DIRECTORY = "./web" + +__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"] diff --git a/examples/node-api/how_to_frontend_nodes/web/frontendNodes.js b/examples/node-api/how_to_frontend_nodes/web/frontendNodes.js new file mode 100644 index 00000000000..2b4a8781c48 --- /dev/null +++ b/examples/node-api/how_to_frontend_nodes/web/frontendNodes.js @@ -0,0 +1,97 @@ +import { comfy } from '/comfy/api/v2.js' + +const api = comfy.forMajor(2) + +api.require('defs.define') +api.require('node.resolve') +api.require('supply.outputs') +api.require('slots.dynamic') + +api.defs.define({ + type: 'HowTo/ConstantText', + title: 'How-To: Constant Text', + category: 'API Examples/Frontend Nodes', + description: 'Resolves a widget value to a prompt literal.', + outputs: [{ name: 'text', type: 'STRING' }], + widgets: [ + { + type: 'text', + name: 'value', + value: 'Hello from a frontend node', + serialize: true + } + ], + execution: 'frontend', + resolve: ({ self }) => ({ + text: { literal: String(self.widgetValue('value') ?? '') } + }) +}) + +api.defs.define({ + type: 'HowTo/Reroute', + title: 'How-To: Reroute', + category: 'API Examples/Frontend Nodes', + description: 'Forwards an input without appearing in the backend prompt.', + inputs: [{ name: 'in', type: '*' }], + outputs: [{ name: 'out', type: '*' }], + execution: 'frontend', + resolve: ({ self }) => { + const input = self.input('in') + return { out: input ? { forwardTo: input } : { omit: true } } + } +}) + +api.defs.define({ + type: 'HowTo/BroadcastText', + title: 'How-To: Broadcast Text', + category: 'API Examples/Frontend Nodes', + description: 'Supplies text to unconnected string inputs in the same group.', + widgets: [ + { + type: 'text', + name: 'value', + value: 'Shared prompt text', + serialize: true + } + ], + execution: 'frontend', + supply: ({ self, unconnectedInputs }) => { + const groupIds = new Set(self.groups.map(({ id }) => id)) + if (groupIds.size === 0) return [] + + const value = String(self.widgetValue('value') ?? '') + return unconnectedInputs() + .filter( + (input) => + input.type === 'STRING' && + input.nodeGroups.some(({ id }) => groupIds.has(id)) + ) + .map((input) => ({ + to: { nodeId: input.nodeId, input: input.input }, + from: { literal: value } + })) + } +}) + +api.defs.define({ + type: 'HowTo/FirstConnected', + title: 'How-To: First Connected', + category: 'API Examples/Frontend Nodes', + description: 'Grows inputs and forwards the first connected one.', + inputs: [{ name: 'input_1', type: '*' }], + outputs: [{ name: 'out', type: '*' }], + execution: 'frontend', + resolve: ({ self }) => { + const first = self.inputs.find(({ connected }) => connected) + const input = first ? self.input(first.index) : undefined + return { out: input ? { forwardTo: input } : { omit: true } } + }, + onConnectionsChanged(node) { + const last = node.inputs.at(node.inputs.length - 1) + if (last?.isConnected) { + node.inputs.add(`input_${node.inputs.length + 1}`, '*', { + shape: 'optional' + }) + } + } +}) diff --git a/examples/node-api/how_to_graph_interaction/README.md b/examples/node-api/how_to_graph_interaction/README.md new file mode 100644 index 00000000000..82e73c1e0e7 --- /dev/null +++ b/examples/node-api/how_to_graph_interaction/README.md @@ -0,0 +1,17 @@ +# How-To: graph interaction + +This pack demonstrates editor behavior without reaching into a canvas or node +instance: + +- **Lifecycle Badge** stores pack-owned state outside the node, saves/restores + it with lifecycle hooks, and exposes host-rendered menu actions and a badge. +- **Image Gate** accepts only image connections through `onBeforeConnect` and + otherwise acts as a frontend reroute. +- **Drop Text File** accepts a browser `.txt` file and commits its contents to a + real widget. +- **Graph Builder** adds and connects a local source/target pair in one undo + step. Its other buttons demonstrate duplicate and same-type replacement. + +Open the Lifecycle Badge node's context menu to change its state. Drag a text +file onto Drop Text File. Use Graph Builder and press Undo once to remove the +entire generated pair. diff --git a/examples/node-api/how_to_graph_interaction/__init__.py b/examples/node-api/how_to_graph_interaction/__init__.py new file mode 100644 index 00000000000..065398a0d6f --- /dev/null +++ b/examples/node-api/how_to_graph_interaction/__init__.py @@ -0,0 +1,5 @@ +NODE_CLASS_MAPPINGS = {} +NODE_DISPLAY_NAME_MAPPINGS = {} +WEB_DIRECTORY = "./web" + +__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"] diff --git a/examples/node-api/how_to_graph_interaction/web/graphInteraction.js b/examples/node-api/how_to_graph_interaction/web/graphInteraction.js new file mode 100644 index 00000000000..b08e6aaf268 --- /dev/null +++ b/examples/node-api/how_to_graph_interaction/web/graphInteraction.js @@ -0,0 +1,196 @@ +import { comfy } from '/comfy/api/v2.js' + +const api = comfy.forMajor(2) + +api.require('defs.define') +api.require('defs.typeCompatibility') +api.require('graph.nodes') +api.require('node.connectVeto') +api.require('node.fileDrop') +api.require('node.menu') +api.require('node.resolve') +api.require('node.sizeConstraints') +api.require('slots.connect') + +const lifecycleState = new Map() +const lifecycleCleanup = new Map() +const stateKey = (node) => `${node.graphId}:${node.id}` + +api.defs.extend('HowTo/LifecycleBadge', (definition) => { + definition.addMenuItem({ + label: (node) => + lifecycleState.get(stateKey(node)) === 'paused' ? 'Resume' : 'Pause', + run(node) { + const key = stateKey(node) + const next = lifecycleState.get(key) === 'paused' ? 'ready' : 'paused' + lifecycleState.set(key, next) + } + }) + definition.addMenuItem({ + label: 'Set state', + items: [ + { + label: 'Ready', + run: (node) => lifecycleState.set(stateKey(node), 'ready') + }, + { + label: 'Paused', + run: (node) => lifecycleState.set(stateKey(node), 'paused') + } + ] + }) +}) + +api.defs.define({ + type: 'HowTo/LifecycleBadge', + title: 'How-To: Lifecycle Badge', + category: 'API Examples/Graph Interaction', + description: 'Persists pack-owned state and renders a dynamic badge.', + execution: 'frontend', + onConfigured(node, data) { + const saved = data.howToState + if (saved === 'ready' || saved === 'paused') { + lifecycleState.set(stateKey(node), saved) + } + }, + onCreated(node) { + const key = stateKey(node) + if (!lifecycleState.has(key)) lifecycleState.set(key, 'ready') + node.setSizeConstraints({ minWidth: 220, minHeight: 80 }) + lifecycleCleanup.set( + key, + node.addBadge(() => ({ + text: lifecycleState.get(key) ?? 'ready', + onClick: () => { + const next = lifecycleState.get(key) === 'paused' ? 'ready' : 'paused' + lifecycleState.set(key, next) + } + })) + ) + }, + onSerialize(node) { + return { howToState: lifecycleState.get(stateKey(node)) ?? 'ready' } + }, + onRemoved(node) { + const key = stateKey(node) + lifecycleCleanup.get(key)?.() + lifecycleCleanup.delete(key) + lifecycleState.delete(key) + } +}) + +api.defs.extend('HowTo/ImageGate', (definition) => { + definition.onBeforeConnect((_node, event) => { + if (event.side !== 'input' || event.peerType === undefined) return + return api.defs.isTypeCompatible(event.peerType, 'IMAGE') + }) +}) + +api.defs.define({ + type: 'HowTo/ImageGate', + title: 'How-To: Image Gate', + category: 'API Examples/Graph Interaction', + description: 'Vetoes non-image links before forwarding its input.', + inputs: [{ name: 'image', type: '*' }], + outputs: [{ name: 'image', type: 'IMAGE' }], + execution: 'frontend', + resolve: ({ self }) => { + const input = self.input('image') + return { image: input ? { forwardTo: input } : { omit: true } } + } +}) + +api.defs.define({ + type: 'HowTo/DropTextFile', + title: 'How-To: Drop Text File', + category: 'API Examples/Graph Interaction', + description: 'Accepts a text file dropped from the browser.', + outputs: [{ name: 'text', type: 'STRING' }], + widgets: [ + { + type: 'text', + name: 'text', + value: 'Drop a .txt file on this node', + options: { multiline: true }, + serialize: true + } + ], + execution: 'frontend', + resolve: ({ self }) => ({ + text: { literal: String(self.widgetValue('text') ?? '') } + }), + onDragOver(_node, event) { + return [...(event.dataTransfer?.items ?? [])].some( + (item) => item.kind === 'file' + ) + }, + async onDrop(node, event) { + const file = [...(event.dataTransfer?.files ?? [])].find((candidate) => + candidate.name.toLowerCase().endsWith('.txt') + ) + if (!file) return false + node.widgets.get('text')?.setValue(await file.text()) + node.setTitle(`How-To: ${file.name}`) + return true + } +}) + +api.defs.define({ + type: 'HowTo/GraphSource', + title: 'How-To: Generated Source', + category: 'API Examples/Graph Interaction/Generated', + outputs: [{ name: 'text', type: 'STRING' }], + widgets: [ + { type: 'text', name: 'value', value: 'Generated in one undo step' } + ], + execution: 'frontend', + resolve: ({ self }) => ({ + text: { literal: String(self.widgetValue('value') ?? '') } + }) +}) + +api.defs.define({ + type: 'HowTo/GraphTarget', + title: 'How-To: Generated Target', + category: 'API Examples/Graph Interaction/Generated', + inputs: [{ name: 'text', type: 'STRING' }], + execution: 'frontend' +}) + +api.defs.define({ + type: 'HowTo/GraphBuilder', + title: 'How-To: Graph Builder', + category: 'API Examples/Graph Interaction', + description: 'Creates a connected pair as one undoable graph edit.', + widgets: [ + { type: 'button', name: 'build_pair', value: null, serialize: false }, + { type: 'button', name: 'duplicate_self', value: null, serialize: false }, + { type: 'button', name: 'repair_self', value: null, serialize: false } + ], + execution: 'frontend', + onCreated(node) { + node.widgets.get('build_pair')?.on('activate', () => { + const { x, y } = node.getPosition() + api.graph.batch(() => { + const source = api.graph.add('HowTo/GraphSource', { + position: { x: x + 320, y } + }) + const target = api.graph.add('HowTo/GraphTarget', { + position: { x: x + 640, y } + }) + source.outputs.get('text')?.connectTo(target.id, 'text') + api.graph.select([source, target]) + api.graph.centerOn(target) + }) + }) + node.widgets.get('duplicate_self')?.on('activate', () => { + const { x, y } = node.getPosition() + const duplicate = api.graph.duplicate(node.id, { x: x + 40, y: y + 40 }) + if (duplicate) api.graph.select([duplicate]) + }) + node.widgets.get('repair_self')?.on('activate', () => { + const replacement = api.graph.replace(node.id, node.type) + if (replacement) api.graph.select([replacement]) + }) + } +}) diff --git a/examples/node-api/how_to_widgets/README.md b/examples/node-api/how_to_widgets/README.md new file mode 100644 index 00000000000..4f5fba9ce80 --- /dev/null +++ b/examples/node-api/how_to_widgets/README.md @@ -0,0 +1,20 @@ +# How-To: widgets + +This pack demonstrates the four widget ownership models: + +- **Widget Events** uses ordinary declared widgets and additive `activate` + listeners. +- **Canvas Meter** draws with host theme colors and handles pointer input in a + renderer-neutral canvas widget. +- **Mounted Slider** mounts an accessible DOM range control and tears down all + retained listeners. +- **Custom Rating Widget** uses `defs.defineWidgetType()` for a type declared by + the Python node. +- **Prompt Serialization** expands ComfyUI text tokens only in the prompt; the + saved workflow keeps the original template. +- **Text Interaction** observes a host-owned multiline editor and handles + Ctrl/Cmd+Enter without reaching for its DOM element. + +Connect an output to a compatible backend node to inspect the resolved value. +For Prompt Serialization, save the workflow before queueing and compare the +saved template with the backend input. diff --git a/examples/node-api/how_to_widgets/__init__.py b/examples/node-api/how_to_widgets/__init__.py new file mode 100644 index 00000000000..ba51731cd2e --- /dev/null +++ b/examples/node-api/how_to_widgets/__init__.py @@ -0,0 +1,5 @@ +from .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS + +WEB_DIRECTORY = "./web" + +__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"] diff --git a/examples/node-api/how_to_widgets/nodes.py b/examples/node-api/how_to_widgets/nodes.py new file mode 100644 index 00000000000..3aac6e0e362 --- /dev/null +++ b/examples/node-api/how_to_widgets/nodes.py @@ -0,0 +1,47 @@ +class HowToRating: + @classmethod + def INPUT_TYPES(cls): + return {"required": {"rating": ("HOW_TO_RATING", {"default": 3})}} + + RETURN_TYPES = ("INT",) + RETURN_NAMES = ("rating",) + FUNCTION = "run" + CATEGORY = "API Examples/Widgets" + + def run(self, rating): + return (int(rating),) + + +class HowToTemplateText: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "text": ( + "STRING", + { + "default": "rendered-%date:yyyy-MM-dd%", + "multiline": True, + }, + ) + } + } + + RETURN_TYPES = ("STRING",) + RETURN_NAMES = ("text",) + FUNCTION = "run" + CATEGORY = "API Examples/Widgets" + + def run(self, text): + return (text,) + + +NODE_CLASS_MAPPINGS = { + "HowToRating": HowToRating, + "HowToTemplateText": HowToTemplateText, +} + +NODE_DISPLAY_NAME_MAPPINGS = { + "HowToRating": "How-To: Custom Rating Widget", + "HowToTemplateText": "How-To: Prompt Serialization", +} diff --git a/examples/node-api/how_to_widgets/web/widgets.js b/examples/node-api/how_to_widgets/web/widgets.js new file mode 100644 index 00000000000..607d4b9c0f9 --- /dev/null +++ b/examples/node-api/how_to_widgets/web/widgets.js @@ -0,0 +1,235 @@ +/* global document */ +import { comfy } from '/comfy/api/v2.js' + +const api = comfy.forMajor(2) + +api.require('defs.define') +api.require('node.resolve') +api.require('widgets.canvas') +api.require('widgets.mount') +api.require('widgets.textInteraction') +api.require('widgets.typeContext') +api.require('serialization.control') + +api.defs.defineWidgetType('HOW_TO_RATING', { + defaultValue: 3, + minWidth: 160, + serialize: true, + render(container, value, name, context) { + const label = document.createElement('label') + const output = document.createElement('span') + const input = document.createElement('input') + + label.textContent = `${name}: ` + input.type = 'range' + input.min = '1' + input.max = '5' + input.step = '1' + input.value = String(value.get()) + output.textContent = input.value + label.append(input, output) + container.append(label) + + const onInput = () => value.set(Number(input.value)) + input.addEventListener('input', onInput) + const stopValue = value.onChange((next) => { + input.value = String(next) + output.textContent = input.value + }) + const stopReady = context.onNodeReady((node) => + node.addBadge(() => ({ text: `${value.get()}/5` })) + ) + + return () => { + input.removeEventListener('input', onInput) + stopValue() + stopReady() + } + } +}) + +const isSubmitKey = (event) => event.kind === 'keydown' && event.key === 'Enter' +const hasSubmitModifier = (event) => event.ctrlKey || event.metaKey +const handleTextInteraction = (event) => { + if (!isSubmitKey(event)) return + if (!hasSubmitModifier(event)) return + event.preventDefault() + api.commands.notify({ + severity: 'success', + summary: 'Text interaction received', + detail: event.value + }) +} + +api.defs.define({ + type: 'HowTo/WidgetEvents', + title: 'How-To: Widget Events', + category: 'API Examples/Widgets', + description: 'Uses additive widget activation listeners.', + outputs: [{ name: 'count', type: 'INT' }], + widgets: [ + { + type: 'number', + name: 'count', + value: 0, + disabled: true, + serialize: true + }, + { type: 'button', name: 'increment', value: null, serialize: false } + ], + execution: 'frontend', + resolve: ({ self }) => ({ + count: { literal: Number(self.widgetValue('count') ?? 0) } + }), + onCreated(node) { + const count = node.widgets.get('count') + node.widgets.get('increment')?.on('activate', () => { + count?.setValue(Number(count.getValue()) + 1) + }) + } +}) + +const canvasCleanups = new Map() + +api.defs.define({ + type: 'HowTo/CanvasMeter', + title: 'How-To: Canvas Meter', + category: 'API Examples/Widgets', + description: 'Draws and edits a value through a canvas widget.', + outputs: [{ name: 'amount', type: 'FLOAT' }], + widgets: [ + { + type: 'number', + name: 'amount', + value: 0.5, + hidden: true, + serialize: true + } + ], + execution: 'frontend', + resolve: ({ self }) => ({ + amount: { literal: Number(self.widgetValue('amount') ?? 0) } + }), + onCreated(node) { + const amount = node.widgets.get('amount') + let width = 1 + const surface = node.widgets.canvas({ + name: 'meter', + height: 48, + serialize: false, + draw(context, [nextWidth, height], theme) { + width = nextWidth + const value = Number(amount?.getValue() ?? 0) + context.fillStyle = theme.surface + context.fillRect(0, 0, width, height) + context.fillStyle = theme.border + context.fillRect(0, 0, width * value, height) + context.fillStyle = theme.text + context.fillText(`${Math.round(value * 100)}%`, 8, height / 2 + 4) + }, + onPointerDown({ x, event }) { + event.preventDefault() + amount?.setValue(Math.max(0, Math.min(1, x / width))) + } + }) + canvasCleanups.set( + `${node.graphId}:${node.id}`, + amount?.on('change', () => surface.redraw()) + ) + }, + onRemoved(node) { + const key = `${node.graphId}:${node.id}` + canvasCleanups.get(key)?.() + canvasCleanups.delete(key) + } +}) + +api.defs.define({ + type: 'HowTo/MountedSlider', + title: 'How-To: Mounted Slider', + category: 'API Examples/Widgets', + description: 'Mounts an accessible DOM control with explicit teardown.', + outputs: [{ name: 'amount', type: 'FLOAT' }], + widgets: [ + { + type: 'number', + name: 'amount', + value: 0.5, + hidden: true, + serialize: true + } + ], + execution: 'frontend', + resolve: ({ self }) => ({ + amount: { literal: Number(self.widgetValue('amount') ?? 0) } + }), + onCreated(node) { + const amount = node.widgets.get('amount') + let destroy + node.widgets.mount({ + name: 'slider', + height: 36, + render(container) { + const label = document.createElement('label') + const input = document.createElement('input') + label.textContent = 'Amount ' + input.type = 'range' + input.min = '0' + input.max = '1' + input.step = '0.01' + input.value = String(amount?.getValue() ?? 0) + label.append(input) + container.append(label) + + const onInput = () => amount?.setValue(Number(input.value)) + input.addEventListener('input', onInput) + const stopValue = amount?.on('change', (next) => { + input.value = String(next) + }) + destroy = () => { + input.removeEventListener('input', onInput) + stopValue?.() + } + }, + destroy() { + destroy?.() + } + }) + } +}) + +api.defs.extend('HowToTemplateText', (definition) => { + definition.onCreated((node) => { + node.widgets.get('text')?.on('beforeSerialize', (event) => { + if (event.context === 'prompt') { + event.setSerializedValue( + api.workflow.applyTextReplacements(String(event.value)) + ) + } + }) + }) +}) + +api.defs.define({ + type: 'HowTo/TextInteraction', + title: 'How-To: Text Interaction', + category: 'API Examples/Widgets', + description: 'Observes keyboard input in a host-owned text editor.', + outputs: [{ name: 'text', type: 'STRING' }], + widgets: [ + { + type: 'text', + name: 'text', + value: 'Press Ctrl/Cmd+Enter', + options: { multiline: true }, + serialize: true + } + ], + execution: 'frontend', + resolve: ({ self }) => ({ + text: { literal: String(self.widgetValue('text') ?? '') } + }), + onCreated(node) { + node.widgets.get('text')?.on('textInteraction', handleTextInteraction) + } +}) diff --git a/knip.config.ts b/knip.config.ts index 0b25c13d170..8b67fe711f5 100644 --- a/knip.config.ts +++ b/knip.config.ts @@ -6,7 +6,7 @@ const config: KnipConfig = { workspaces: { '.': { entry: [ - '{build,scripts}/**/*.{js,ts}', + '{build,scripts}/**/*.{js,mjs,ts}', 'vitest.matrix.config.mts', 'src/assets/css/style.css', 'src/scripts/ui/menu/index.ts', @@ -104,6 +104,9 @@ const config: KnipConfig = { // the build imports it — that is the point: it is the entry point // packs import from, so knip cannot see a consumer. 'public/comfy/api/v2.js', + // Example packs: ComfyUI loads their web/ scripts by URL at runtime, + // exactly as a real pack does, so the build has no import to find. + 'examples/node-api/**/web/*.js', // Agent review check config, not part of the build // Devtools extensions, included dynamically 'tools/devtools/web/**' diff --git a/scripts/node-api/gen_api_dts.mjs b/scripts/node-api/gen_api_dts.mjs new file mode 100644 index 00000000000..4b2393e89aa --- /dev/null +++ b/scripts/node-api/gen_api_dts.mjs @@ -0,0 +1,134 @@ +import { readFileSync, readdirSync, realpathSync } from 'node:fs' +import { join, resolve } from 'node:path' +import process from 'node:process' +import { fileURLToPath, URL } from 'node:url' +import ts from 'typescript' +import { format } from 'oxfmt' +import formatConfig from '../../.oxfmtrc.json' with { type: 'json' } + +import { reachableDeclarationNames } from './gen_api_surface.mjs' + +function sourceDirectory() { + return fileURLToPath(new URL('../../src/platform/nodeApi/', import.meta.url)) +} + +function tupleElementUnion(type, tuples) { + if ( + !ts.isIndexedAccessTypeNode(type) || + type.indexType.kind !== ts.SyntaxKind.NumberKeyword + ) + return undefined + const object = ts.isParenthesizedTypeNode(type.objectType) + ? type.objectType.type + : type.objectType + if (!ts.isTypeQueryNode(object) || !ts.isIdentifier(object.exprName)) { + return undefined + } + return tuples.get(object.exprName.text) +} + +/** Exported interfaces and type aliases, with their doc comments. */ +export function declarations(source) { + const sourceFile = ts.createSourceFile( + 'nodeApi.ts', + source, + ts.ScriptTarget.Latest, + true, + ts.ScriptKind.TS + ) + const tuples = new Map() + for (const statement of sourceFile.statements) { + if (!ts.isVariableStatement(statement)) continue + for (const declaration of statement.declarationList.declarations) { + const value = declaration.initializer + if ( + !ts.isIdentifier(declaration.name) || + !value || + !ts.isAsExpression(value) || + value.type.getText(sourceFile) !== 'const' || + !ts.isArrayLiteralExpression(value.expression) || + !value.expression.elements.length || + !value.expression.elements.every(ts.isStringLiteral) + ) + continue + tuples.set( + declaration.name.text, + value.expression.elements + .map((item) => item.getText(sourceFile)) + .join(' | ') + ) + } + } + + return sourceFile.statements.flatMap((statement) => { + const kind = ts.isInterfaceDeclaration(statement) + ? 'interface' + : ts.isTypeAliasDeclaration(statement) + ? 'type' + : undefined + const exported = statement.modifiers?.some( + (modifier) => modifier.kind === ts.SyntaxKind.ExportKeyword + ) + if (!kind || !exported) return [] + let text = statement.getFullText(sourceFile) + const union = ts.isTypeAliasDeclaration(statement) + ? tupleElementUnion(statement.type, tuples) + : undefined + if (union) { + const start = + statement.type.getStart(sourceFile) - statement.getFullStart() + const end = statement.type.end - statement.getFullStart() + text = text.slice(0, start) + union + text.slice(end) + } + + return [ + { + name: statement.name.text, + kind, + text: text.trim() + } + ] + }) +} + +export async function buildApiDts(directory = sourceDirectory()) { + const files = readdirSync(directory) + .filter((f) => f.endsWith('.ts') && !f.endsWith('.test.ts')) + .sort() + + const published = reachableDeclarationNames(directory) + const sections = [] + for (const file of files) { + const found = declarations( + readFileSync(join(directory, file), 'utf8') + ).filter((d) => published.has(d.name)) + if (!found.length) continue + sections.push(found.map((d) => d.text).join('\n\n')) + } + + const source = + `/** Generated from src/platform/nodeApi; import from '/comfy/api/v2.js'. */\n` + + `declare module '*/comfy/api/v2.js' {\n` + + sections + .join('\n\n') + .split('\n') + .map((line) => (line ? ` ${line}` : '')) + .join('\n') + + `\n\n` + + ` export const comfy: Comfy\n` + + `}\n` + const result = await format('comfy-api.d.ts', source, formatConfig) + if (result.errors.length) { + throw new Error(result.errors.map((error) => error.message).join('\n')) + } + return result.code +} + +const entryPath = process.argv[1] +if ( + entryPath && + realpathSync(fileURLToPath(import.meta.url)) === + realpathSync(resolve(entryPath)) +) { + process.stdout.write(await buildApiDts()) +} diff --git a/scripts/node-api/gen_api_surface.mjs b/scripts/node-api/gen_api_surface.mjs new file mode 100644 index 00000000000..d9b822b99c4 --- /dev/null +++ b/scripts/node-api/gen_api_surface.mjs @@ -0,0 +1,148 @@ +/** + * Regenerates `src/platform/nodeApi/apiSurface.ts` from the published API. + * + * node scripts/node-api/gen_api_surface.mjs + * + * The generated set is what the conformance harness checks converted code + * against, so it has to track the API exactly; `apiSurface.test.ts` re-derives + * it and fails if this was not re-run. + * + * Reachability from `Comfy`, not "every interface in the folder": the folder + * also holds the registry's own bookkeeping shapes, and scanning those put + * `handleFor`, `handleForNode` and `active` in the published set — so the + * harness would wave through a conversion calling an internal. + * + * Names are followed whether or not they are exported. Several published types + * are un-exported to satisfy knip, and dropping them took their members out of + * the set, which made the harness reject correct conversions. + */ +import console from 'node:console' +import { readFileSync, readdirSync, realpathSync, writeFileSync } from 'node:fs' +import { join, resolve } from 'node:path' +import process from 'node:process' +import { fileURLToPath, URL } from 'node:url' + +import ts from 'typescript' + +function sourceDirectory() { + return fileURLToPath(new URL('../../src/platform/nodeApi/', import.meta.url)) +} + +const ROOT_TYPE = 'Comfy' + +/** Declarations that carry members, by name, across the whole directory. */ +function declarationsByName(dir) { + const byName = new Map() + for (const file of readdirSync(dir).sort()) { + if (!file.endsWith('.ts') || file.endsWith('.test.ts')) continue + const source = ts.createSourceFile( + file, + readFileSync(join(dir, file), 'utf8'), + ts.ScriptTarget.ES2023, + true + ) + for (const statement of source.statements) { + if ( + ts.isInterfaceDeclaration(statement) || + ts.isTypeAliasDeclaration(statement) + ) { + const existing = byName.get(statement.name.text) ?? [] + existing.push(statement) + byName.set(statement.name.text, existing) + } + } + } + return byName +} + +function declarationMemberName(node) { + if ( + !ts.isPropertySignature(node) && + !ts.isMethodSignature(node) && + !ts.isPropertyDeclaration(node) && + !ts.isMethodDeclaration(node) + ) + return undefined + if (!ts.isIdentifier(node.name) || node.name.text.startsWith('__')) + return undefined + return node.name.text +} + +function referencedTypeName(node) { + if (!ts.isTypeReferenceNode(node)) return undefined + return ts.isIdentifier(node.typeName) + ? node.typeName.text + : node.typeName.right.text +} + +function inheritedTypeName(node) { + if ( + !ts.isExpressionWithTypeArguments(node) || + !ts.isIdentifier(node.expression) + ) { + return undefined + } + return node.expression.text +} + +function walk(node, members, referenced) { + const member = declarationMemberName(node) + if (member) members.add(member) + const reference = referencedTypeName(node) + if (reference) referenced.add(reference) + const inherited = inheritedTypeName(node) + if (inherited) referenced.add(inherited) + node.forEachChild((child) => walk(child, members, referenced)) +} + +/** Names of every declaration reachable from the `Comfy` root. */ +export function reachableDeclarationNames(dir = sourceDirectory()) { + return walkFromRoot(dir).seen +} + +export function deriveApiMembers(dir = sourceDirectory()) { + return walkFromRoot(dir).members +} + +function walkFromRoot(dir) { + const byName = declarationsByName(dir) + const members = new Set() + const seen = new Set() + const queue = [ROOT_TYPE] + + while (queue.length) { + const name = queue.pop() + if (seen.has(name)) continue + seen.add(name) + for (const declaration of byName.get(name) ?? []) { + const referenced = new Set() + walk(declaration, members, referenced) + for (const next of referenced) { + if (!seen.has(next) && byName.has(next)) queue.push(next) + } + } + } + return { members, seen } +} + +const entryPath = process.argv[1] +if ( + entryPath && + realpathSync(fileURLToPath(import.meta.url)) === + realpathSync(resolve(entryPath)) +) { + const directory = sourceDirectory() + const names = [...deriveApiMembers(directory)].sort((a, b) => + a < b ? -1 : a > b ? 1 : 0 + ) + const header = readFileSync(join(directory, 'apiSurface.ts'), 'utf8').split( + 'export const API_MEMBERS' + )[0] + writeFileSync( + join(directory, 'apiSurface.ts'), + `${header}export const API_MEMBERS: ReadonlySet = new Set([\n${names + .map((n) => ` '${n}'`) + .join(',\n')}\n])\n` + ) + console.error(`${names.length} members`) +} diff --git a/scripts/node-api/gen_capability_list.mjs b/scripts/node-api/gen_capability_list.mjs new file mode 100644 index 00000000000..2928f1c448a --- /dev/null +++ b/scripts/node-api/gen_capability_list.mjs @@ -0,0 +1,89 @@ +/** + * Rewrites the implemented-capability lists in the node API design and + * reference docs from `CAPABILITIES` in comfyApi.ts. + * + * node scripts/node-api/gen_capability_list.mjs [--check] + * + * Hand-maintaining that list cost a whole conversion run: the doc said eleven + * capabilities while the code had eighteen, so agents read `widgets.mount`, + * `widgets.canvas`, `setSizeConstraints` and `defs.define` as unimplemented and + * punted twelve files against capabilities that were sitting right there. The + * banner exists to stop conversions against unbuilt API; stale, it does the + * opposite and is far more expensive. + */ +import console from 'node:console' +import { readFileSync, writeFileSync } from 'node:fs' +import process from 'node:process' +import { fileURLToPath, URL } from 'node:url' + +const API = fileURLToPath( + new URL('../../src/platform/nodeApi/comfyApi.ts', import.meta.url) +) +const REFERENCE_DOC = fileURLToPath( + new URL('../../docs/node-api/reference.md', import.meta.url) +) +const REFERENCE_START = '' +const REFERENCE_END = '' + +export function implementedCapabilities() { + const source = readFileSync(API, 'utf8') + const block = /const CAPABILITIES[\s\S]*?\n\]\)/.exec(source)?.[0] ?? '' + return [...block.matchAll(/\[\s*'([^']+)'/g)].map((m) => m[1]).sort() +} + +function renderReference(names) { + const out = [] + let current = '' + for (const name of names) { + const part = `\`${name}\`` + if (`${current}${part}, `.length > 78) { + out.push(current.trimEnd()) + current = '' + } + current += `${part}, ` + } + out.push(current.trimEnd().replace(/,$/, '.')) + return out.join('\n') +} + +function updateReference(names) { + const doc = readFileSync(REFERENCE_DOC, 'utf8') + const from = doc.indexOf(REFERENCE_START) + const to = doc.indexOf(REFERENCE_END) + if (from === -1 || to === -1) { + throw new Error(`Could not find the capability block in ${REFERENCE_DOC}`) + } + return { + path: REFERENCE_DOC, + source: doc, + next: + doc.slice(0, from + REFERENCE_START.length) + + `\n\n${renderReference(names)}\n` + + doc.slice(to) + } +} + +const names = implementedCapabilities() +let updates +try { + updates = [updateReference(names)] +} catch (error) { + console.error(error instanceof Error ? error.message : error) + process.exit(2) +} + +if (process.argv.includes('--check')) { + const stale = updates.filter(({ source, next }) => source !== next) + if (stale.length) { + console.error( + `${stale.map(({ path }) => path).join(', ')} list different ` + + 'capabilities than CAPABILITIES.\n' + + 'Run: node scripts/node-api/gen_capability_list.mjs' + ) + process.exit(1) + } + console.error(`capability lists are current (${names.length})`) +} else { + for (const { path, next } of updates) writeFileSync(path, next) + console.error(`wrote ${names.length} capabilities to ${updates.length} docs`) +} diff --git a/scripts/start-comfyui-e2e.sh b/scripts/start-comfyui-e2e.sh index 31bffe1ddbb..3c771a9069b 100755 --- a/scripts/start-comfyui-e2e.sh +++ b/scripts/start-comfyui-e2e.sh @@ -31,22 +31,26 @@ fi runtime_root="$(mktemp -d)" docker_config="$runtime_root/docker-config" devtools_stage="$runtime_root/devtools" +examples_stage="$runtime_root/examples" cleanup() { "${docker[@]}" rm -f "$container" >/dev/null 2>&1 || true rm -rf "$runtime_root" } trap cleanup EXIT -mkdir -p "$docker_config" "$devtools_stage" +mkdir -p "$docker_config" "$devtools_stage" "$examples_stage" cp -R "$repo_root/tools/devtools/." "$devtools_stage/" -chmod -R a+rX "$devtools_stage" +for pack in how_to_frontend_nodes how_to_widgets how_to_graph_interaction how_to_execution; do + cp -R "$repo_root/examples/node-api/$pack" "$examples_stage/$pack" +done +chmod -R a+rX "$devtools_stage" "$examples_stage" unreadable="$( - find "$devtools_stage" \ + find "$devtools_stage" "$examples_stage" \ \( \( -type f ! -perm -o=r \) -o \( -type d ! -perm -o=rx \) \) \ -print -quit )" if [[ -n "$unreadable" ]]; then - echo "Staged devtools path is unreadable by the container user: $unreadable" >&2 + echo "Staged custom-node path is unreadable by the container user: $unreadable" >&2 echo 'Tests using the ComfyPage fixture would fail with an unrelated' >&2 echo 'HTTP error. Aborting.' >&2 exit 1 @@ -90,10 +94,31 @@ else echo "Using cached image $image" fi +frontend_mount=() +frontend_args=() +if [[ -n "${COMFYUI_FRONTEND_ROOT:-}" ]]; then + if [[ ! -d "$COMFYUI_FRONTEND_ROOT" ]]; then + echo "COMFYUI_FRONTEND_ROOT is not a directory: $COMFYUI_FRONTEND_ROOT" >&2 + exit 1 + fi + frontend_mount=(--mount + "type=bind,src=$COMFYUI_FRONTEND_ROOT,dst=/frontend-dist,readonly") + frontend_args=(--front-end-root /frontend-dist) +fi + "${docker[@]}" run --rm --name "$container" \ --publish "127.0.0.1:$port:8188" \ + "${frontend_mount[@]}" \ --mount \ "type=bind,src=$devtools_stage,dst=/ComfyUI/custom_nodes/ComfyUI_devtools,readonly" \ + --mount \ + "type=bind,src=$examples_stage/how_to_frontend_nodes,dst=/ComfyUI/custom_nodes/how_to_frontend_nodes,readonly" \ + --mount \ + "type=bind,src=$examples_stage/how_to_widgets,dst=/ComfyUI/custom_nodes/how_to_widgets,readonly" \ + --mount \ + "type=bind,src=$examples_stage/how_to_graph_interaction,dst=/ComfyUI/custom_nodes/how_to_graph_interaction,readonly" \ + --mount \ + "type=bind,src=$examples_stage/how_to_execution,dst=/ComfyUI/custom_nodes/how_to_execution,readonly" \ "$image" \ bash -lc \ - 'cd /ComfyUI && exec python3 main.py --cpu --multi-user --listen 0.0.0.0' + "cd /ComfyUI && exec python3 main.py --cpu --multi-user --listen 0.0.0.0 ${frontend_args[*]}" diff --git a/scripts/start-comfyui-e2e.test.sh b/scripts/start-comfyui-e2e.test.sh index 7df87163079..502de468cd5 100755 --- a/scripts/start-comfyui-e2e.test.sh +++ b/scripts/start-comfyui-e2e.test.sh @@ -17,6 +17,13 @@ chmod 750 "$fixture/tools/devtools" chmod 640 "$fixture/tools/devtools/fixture.txt" chmod 750 "$fixture/tools/devtools/nested" chmod 640 "$fixture/tools/devtools/nested/fixture.bin" +example_packs=(how_to_frontend_nodes how_to_widgets how_to_graph_interaction how_to_execution) +for pack in "${example_packs[@]}"; do + mkdir -p "$fixture/examples/node-api/$pack/web" + printf '%s\n' "$pack" > "$fixture/examples/node-api/$pack/web/pack.js" + chmod 750 "$fixture/examples/node-api/$pack" "$fixture/examples/node-api/$pack/web" + chmod 640 "$fixture/examples/node-api/$pack/web/pack.js" +done cat > "$test_root/bin/docker" <<'EOF' #!/usr/bin/env bash @@ -30,18 +37,28 @@ case "${1:-}" in ;; run) mount_source="" + example_mounts=0 while (($#)); do - if [[ "$1" == type=bind,src=*,dst=/ComfyUI/custom_nodes/ComfyUI_devtools,readonly ]]; then - mount_source="${1#type=bind,src=}" - mount_source="${mount_source%,dst=*}" + if [[ "$1" == type=bind,src=*,dst=/ComfyUI/custom_nodes/*,readonly ]]; then + source="${1#type=bind,src=}" + source="${source%,dst=*}" + name="${1##*/custom_nodes/}" + name="${name%,readonly}" + [[ "$source" != "$SOURCE_REPO"/* ]] + [[ -z "$(find "$source" \( \( -type f ! -perm -o=r \) -o \( -type d ! -perm -o=rx \) \) -print -quit)" ]] + if [[ "$name" == ComfyUI_devtools ]]; then + mount_source="$source" + else + [[ "$(cat "$source/web/pack.js")" == "$name" ]] + example_mounts=$((example_mounts + 1)) + fi fi shift done [[ -n "$mount_source" ]] - [[ "$mount_source" != "$SOURCE_DEVTOOLS" ]] - [[ -z "$(find "$mount_source" \( \( -type f ! -perm -o=r \) -o \( -type d ! -perm -o=rx \) \) -print -quit)" ]] + [[ $example_mounts == 4 ]] [[ "$(cat "$mount_source/fixture.txt")" == fixture ]] - cmp "$SOURCE_DEVTOOLS/nested/fixture.bin" "$mount_source/nested/fixture.bin" + cmp "$SOURCE_REPO/tools/devtools/nested/fixture.bin" "$mount_source/nested/fixture.bin" printf '%s\n' "$mount_source" > "$CAPTURE_MOUNT" exit "${DOCKER_RUN_STATUS:-0}" ;; @@ -58,7 +75,7 @@ run_launcher() { mkdir -p "$tmpdir" PATH="$test_root/bin:$PATH" \ TMPDIR="$tmpdir" \ - SOURCE_DEVTOOLS="$fixture/tools/devtools" \ + SOURCE_REPO="$fixture" \ CAPTURE_MOUNT="$test_root/mount" \ DOCKER_RUN_STATUS="$status" \ AMP_ORB='' \ diff --git a/src/platform/nodeApi/apiDts.test.ts b/src/platform/nodeApi/apiDts.test.ts new file mode 100644 index 00000000000..a983502959f --- /dev/null +++ b/src/platform/nodeApi/apiDts.test.ts @@ -0,0 +1,306 @@ +import { execFileSync } from 'node:child_process' +import { + cpSync, + mkdirSync, + mkdtempSync, + readFileSync, + rmSync, + symlinkSync, + writeFileSync +} from 'node:fs' +import { tmpdir } from 'node:os' +import { resolve } from 'node:path' + +import ts from 'typescript' +import { describe, expect, it, onTestFinished } from 'vitest' + +import { + buildApiDts, + declarations +} from '../../../scripts/node-api/gen_api_dts.mjs' + +function contractDiagnostics( + source: string, + pack?: string, + globals?: string +): string[] { + const contractPath = '/comfy-api.d.ts' + const packPath = '/pack.ts' + const options: ts.CompilerOptions = { + lib: ['lib.es2023.d.ts', 'lib.dom.d.ts'], + module: ts.ModuleKind.ESNext, + noEmit: true, + skipLibCheck: false, + strict: true, + target: ts.ScriptTarget.ES2023, + types: [] + } + const host = ts.createCompilerHost(options) + const getSourceFile = host.getSourceFile.bind(host) + const fileExists = host.fileExists.bind(host) + const readFile = host.readFile.bind(host) + + const virtual = new Map([[contractPath, source]]) + if (pack !== undefined) virtual.set(packPath, pack) + if (globals !== undefined) virtual.set('/consumer-globals.d.ts', globals) + + host.fileExists = (fileName) => virtual.has(fileName) || fileExists(fileName) + host.readFile = (fileName) => virtual.get(fileName) ?? readFile(fileName) + host.getSourceFile = (fileName, languageVersion, onError, shouldCreate) => { + const contents = virtual.get(fileName) + return contents === undefined + ? getSourceFile(fileName, languageVersion, onError, shouldCreate) + : ts.createSourceFile(fileName, contents, languageVersion, true) + } + + return ts + .getPreEmitDiagnostics(ts.createProgram([...virtual.keys()], options, host)) + .map((diagnostic) => + ts.flattenDiagnosticMessageText(diagnostic.messageText, '\n') + ) +} + +describe('generated custom-node API contract', () => { + it('matches the checked-in machine-readable contract', async () => { + const directory = resolve(process.cwd(), 'src/platform/nodeApi') + const checkedIn = readFileSync('docs/node-api/comfy-api.d.ts', 'utf8') + + expect(checkedIn).toBe(await buildApiDts(directory)) + }) + + it('includes only exported type declarations and their own documentation', () => { + const source = ` +/** Public options. */ +export interface Options { + enabled: boolean +} + +const privateState = { enabled: false } + +/** Public mode. */ +export type Mode = 'on' | 'off' + +export function mutate(): void {} +` + + const extracted = declarations(source) + + expect(extracted.map(({ kind, name }) => ({ kind, name }))).toEqual([ + { kind: 'interface', name: 'Options' }, + { kind: 'type', name: 'Mode' } + ]) + expect(extracted.map(({ text }) => text).join('\n')).not.toContain( + 'privateState' + ) + expect(extracted.map(({ text }) => text).join('\n')).not.toContain( + 'function mutate' + ) + expect(extracted[1].text).toContain('/** Public mode. */') + }) + + it( + 'emits a self-contained TypeScript contract', + { timeout: 15_000 }, + async () => { + const directory = resolve(process.cwd(), 'src/platform/nodeApi') + expect(contractDiagnostics(await buildApiDts(directory))).toEqual([]) + } + ) + + it( + 'keeps API types separate from consumer globals', + { timeout: 15_000 }, + async () => { + const directory = resolve(process.cwd(), 'src/platform/nodeApi') + const pack = ` +import { comfy } from '/comfy/api/v2.js' + +comfy.graph.nodes()[0]?.setPosition({ x: 1, y: 2 }) +const unrelatedPoint: Point = { z: 3 } +export const height = unrelatedPoint.z +` + + expect( + contractDiagnostics( + await buildApiDts(directory), + pack, + 'interface Point { z: number }' + ) + ).toEqual([]) + } + ) + + it( + 'exports named types from the API module', + { timeout: 15_000 }, + async () => { + const directory = resolve(process.cwd(), 'src/platform/nodeApi') + const pack = ` +import { comfy, type Point } from '/comfy/api/v2.js' + +const position: Point = { x: 1, y: 2 } +comfy.graph.nodes()[0]?.setPosition(position) +` + + expect(contractDiagnostics(await buildApiDts(directory), pack)).toEqual( + [] + ) + } + ) + + it( + 'expands private tuple-backed public types', + { timeout: 15_000 }, + async () => { + const directory = mkdtempSync(resolve(tmpdir(), 'node-api-tuples-')) + onTestFinished(() => rmSync(directory, { recursive: true, force: true })) + writeFileSync( + resolve(directory, 'api.ts'), + ` +const modes = ['disabled', 'instant'] as const +export type Mode = (typeof modes)[number] +export interface Comfy { setMode(mode: Mode): void } +` + ) + const contract = await buildApiDts(directory) + const pack = ` +import { comfy } from '/comfy/api/v2.js' +comfy.setMode('instant') +comfy.setMode('unknown') +` + + expect(contractDiagnostics(contract, pack)).toEqual([ + expect.stringContaining('"unknown"') + ]) + expect(contract).not.toMatch(/\bmodes\b/) + } + ) + + it( + 'types the lifecycle conversion recipes', + { timeout: 15_000 }, + async () => { + const directory = resolve(process.cwd(), 'src/platform/nodeApi') + const skill = readFileSync( + '.claude/skills/converting-custom-nodes/SKILL.md', + 'utf8' + ) + const recipes = skill + .split('\n') + .filter((line) => + /^\| `nodeType\.prototype\.on(?:NodeCreated|Executed|Configure|Removed) =/.test( + line + ) + ) + .map((line) => + line + .split('|')[2] + .trim() + .replace(/^`|`$/g, '') + .replace('...', 'node.getTitle()') + ) + expect(recipes).toHaveLength(4) + const pack = ` +import { comfy } from '/comfy/api/v2.js' + +const defs = comfy.defs +const sel = 'PreviewImage' +const node = comfy.graph.nodes()[0]! +const result = { text: [], images: [], raw: {} } +const data = {} +defs.extend(sel, (b) => { + ${recipes.join('\n')} +}) +` + + expect(contractDiagnostics(await buildApiDts(directory), pack)).toEqual( + [] + ) + } + ) + + it('publishes nothing a pack cannot reach from Comfy', async () => { + const directory = resolve(process.cwd(), 'src/platform/nodeApi') + const contract = await buildApiDts(directory) + + expect(contract).not.toMatch( + /\b(?:PropSpec|HandleSpec|HandleToken|ResolveOptions)\b/ + ) + }) + + it('rejects reassigning a root member', { timeout: 15_000 }, async () => { + const directory = resolve(process.cwd(), 'src/platform/nodeApi') + const pack = ` +import { comfy } from '/comfy/api/v2.js' + +comfy.queue = comfy.queue +` + + expect(contractDiagnostics(await buildApiDts(directory), pack)).toEqual([ + expect.stringContaining('read-only') + ]) + }) + + it( + 'types the import the documentation tells packs to write', + { timeout: 15_000 }, + async () => { + const directory = resolve(process.cwd(), 'src/platform/nodeApi') + const pack = ` +import { comfy } from '/comfy/api/v2.js' + +export const title: string | undefined = comfy.graph.nodes()[0]?.getTitle() +export const missing = comfy.graph.noSuchMember() +` + + expect(contractDiagnostics(await buildApiDts(directory), pack)).toEqual([ + expect.stringContaining('noSuchMember') + ]) + } + ) + + it('runs both generators from a checkout path containing spaces', () => { + const checkout = mkdtempSync(resolve(tmpdir(), 'node api generator ')) + try { + mkdirSync(resolve(checkout, 'scripts'), { recursive: true }) + mkdirSync(resolve(checkout, 'src/platform'), { recursive: true }) + cpSync('.oxfmtrc.json', resolve(checkout, '.oxfmtrc.json')) + cpSync('scripts/node-api', resolve(checkout, 'scripts/node-api'), { + recursive: true + }) + cpSync( + 'src/platform/nodeApi', + resolve(checkout, 'src/platform/nodeApi'), + { recursive: true } + ) + symlinkSync(resolve('node_modules'), resolve(checkout, 'node_modules')) + const surfacePath = resolve( + checkout, + 'src/platform/nodeApi/apiSurface.ts' + ) + writeFileSync( + surfacePath, + readFileSync(surfacePath, 'utf8').replace( + "'onPromptSerialize'", + "'staleMember'" + ) + ) + + const declaration = execFileSync( + process.execPath, + [resolve(checkout, 'scripts/node-api/gen_api_dts.mjs')], + { encoding: 'utf8' } + ) + execFileSync(process.execPath, [ + resolve(checkout, 'scripts/node-api/gen_api_surface.mjs') + ]) + + expect(declaration).toContain('interface Comfy') + const surface = readFileSync(surfacePath, 'utf8') + expect(surface).toContain("'onPromptSerialize'") + expect(surface).not.toContain("'staleMember'") + } finally { + rmSync(checkout, { recursive: true, force: true }) + } + }) +}) diff --git a/src/platform/nodeApi/apiSurface.test.ts b/src/platform/nodeApi/apiSurface.test.ts new file mode 100644 index 00000000000..3445d2bdc80 --- /dev/null +++ b/src/platform/nodeApi/apiSurface.test.ts @@ -0,0 +1,15 @@ +import { describe, expect, it } from 'vitest' + +import { deriveApiMembers } from '../../../scripts/node-api/gen_api_surface.mjs' +import { API_MEMBERS } from './apiSurface' + +describe('published API surface', () => { + it('matches the interfaces it was generated from', () => { + // The conformance harness rejects converted code that references a member + // outside this set, so a stale set would either block a valid conversion or + // wave through an invented member. + const derived = deriveApiMembers('src/platform/nodeApi') + const byName = (a: string, b: string) => a.localeCompare(b) + expect([...API_MEMBERS].sort(byName)).toEqual([...derived].sort(byName)) + }) +}) diff --git a/src/platform/nodeApi/apiSurface.ts b/src/platform/nodeApi/apiSurface.ts new file mode 100644 index 00000000000..8adc9c12cd8 --- /dev/null +++ b/src/platform/nodeApi/apiSurface.ts @@ -0,0 +1,423 @@ +/** + * Every member name the published API defines. + * + * Generated from the interfaces in this directory by + * `scripts/node-api/gen_api_surface.mjs`; `apiSurface.test.ts` re-derives it + * and fails on drift. + * + * Used by the conformance check that rejects a conversion referencing an API + * member that does not exist. Two agent conversions invented one and passed + * every other check, because nothing executes an agent-authored rewrite. + * + * The example this note used to give was `setSizeConstraints`, which has since + * been built and is now in the set below — so the warning against inventing API + * was itself teaching readers that a real member was fake. Name a wished-for + * member and the note rots the day someone grants the wish. The set is the + * example; it is generated, so it cannot say anything untrue. + */ +export const API_MEMBERS: ReadonlySet = new Set([ + 'accelerators', + 'add', + 'addActionBarButton', + 'addBadge', + 'addMenuItem', + 'addSidebarTab', + 'addTopBarBadge', + 'addWidget', + 'adopt', + 'all', + 'alt', + 'altKey', + 'anchor', + 'applyTextReplacements', + 'ariaLabel', + 'assetUrl', + 'at', + 'attrs', + 'autoHeight', + 'autoQueueMode', + 'available', + 'backend', + 'batch', + 'batchCount', + 'bgColor', + 'blob', + 'border', + 'byId', + 'byName', + 'canvas', + 'canvasOnly', + 'capabilities', + 'category', + 'centerOn', + 'close', + 'collapsed', + 'color', + 'colorWhenUnconnected', + 'comfyClass', + 'commands', + 'component', + 'connectTo', + 'connected', + 'connectedType', + 'context', + 'cpu', + 'ctrl', + 'ctrlKey', + 'current', + 'declare', + 'def', + 'defaultValue', + 'define', + 'defineWidgetType', + 'defs', + 'deltaY', + 'description', + 'destroy', + 'detail', + 'details', + 'direction', + 'disableAutoQueue', + 'disabled', + 'disconnect', + 'documentId', + 'draw', + 'duplicate', + 'enabled', + 'end', + 'error', + 'errors', + 'event', + 'executingNode', + 'execution', + 'executionNode', + 'extend', + 'extensions', + 'fetch', + 'focus', + 'forMajor', + 'forwardInput', + 'forwardTo', + 'from', + 'get', + 'getApiVersion', + 'getBgColor', + 'getBounds', + 'getColor', + 'getDisplayedImageIndex', + 'getHeight', + 'getMode', + 'getOptions', + 'getOutputImages', + 'getPosition', + 'getProperties', + 'getProperty', + 'getScreenRect', + 'getShape', + 'getSize', + 'getSizeConstraints', + 'getSlotPosition', + 'getTitle', + 'getValue', + 'graph', + 'graphId', + 'group', + 'groupColor', + 'groups', + 'guard', + 'has', + 'height', + 'hidden', + 'hideInPanel', + 'hideOnZoom', + 'hideWidget', + 'icon', + 'iconClass', + 'id', + 'ids', + 'images', + 'index', + 'input', + 'inputIndex', + 'inputName', + 'inputs', + 'interrupt', + 'isApiVersionAtLeast', + 'isCollapsed', + 'isConnected', + 'isDeleted', + 'isDisabled', + 'isHidden', + 'isInteracting', + 'isModified', + 'isOutputNode', + 'isPinned', + 'isSerialized', + 'isSerializingWidgets', + 'isTypeCompatible', + 'isWidgetInput', + 'items', + 'key', + 'keybinding', + 'kind', + 'label', + 'length', + 'life', + 'link', + 'linked', + 'links', + 'list', + 'literal', + 'loading', + 'localizedName', + 'major', + 'max', + 'maxBytes', + 'maxHeight', + 'maxWidth', + 'memory', + 'memory_available', + 'memory_total', + 'menuEvent', + 'mergeWidgetConfig', + 'message', + 'meta', + 'metaKey', + 'mimeTypes', + 'min', + 'minHeight', + 'minWidth', + 'mode', + 'models', + 'modify', + 'monitor', + 'mount', + 'mountViewportPanel', + 'move', + 'moveLinksTo', + 'multiline', + 'name', + 'names', + 'node', + 'nodeAt', + 'nodeColor', + 'nodeCount', + 'nodeErrors', + 'nodeGroups', + 'nodeId', + 'nodeMode', + 'nodeProperties', + 'nodeTitle', + 'nodeType', + 'nodes', + 'nodesOfType', + 'notify', + 'off', + 'offsetX', + 'offsetY', + 'omit', + 'omitInputs', + 'on', + 'onAfterRun', + 'onBeforeConnect', + 'onBeforeRun', + 'onChange', + 'onClick', + 'onConfigured', + 'onConnectionsChanged', + 'onContextMenu', + 'onCreated', + 'onDocumentActivated', + 'onDocumentClosed', + 'onDocumentDeactivated', + 'onDocumentOpened', + 'onDoubleClick', + 'onDragOver', + 'onDrop', + 'onExecuted', + 'onExecutingNodeChanged', + 'onHover', + 'onInterrupted', + 'onNodeChanged', + 'onNodeDragEnd', + 'onNodeMoved', + 'onNodeReady', + 'onPendingChanged', + 'onPointerDown', + 'onPointerMove', + 'onPointerUp', + 'onPreview', + 'onPromptSerialize', + 'onPropertyChanged', + 'onReady', + 'onRefreshed', + 'onRejected', + 'onRemoved', + 'onResized', + 'onSelectionChanged', + 'onSerialize', + 'onUnplacedLink', + 'onViewportChanged', + 'onWorkflowLoaded', + 'open', + 'options', + 'order', + 'output', + 'outputIndex', + 'outputs', + 'ownFetch', + 'panBy', + 'parse', + 'path', + 'peerIndex', + 'peerNodeId', + 'peerType', + 'pending', + 'pinned', + 'placeholder', + 'playSound', + 'pointerPosition', + 'position', + 'precision', + 'preventDefault', + 'previous', + 'priority', + 'prompt', + 'promptId', + 'promptIds', + 'properties', + 'property', + 'props', + 'queryNodes', + 'queue', + 'raw', + 'read_only', + 'reason', + 'redraw', + 'refresh', + 'register', + 'registerImporter', + 'reject', + 'rejected', + 'remove', + 'render', + 'rendered', + 'reorder', + 'repeat', + 'replace', + 'replaceExisting', + 'require', + 'resolve', + 'resolvedSource', + 'resolvedSupplies', + 'restored', + 'root', + 'run', + 'sameEntity', + 'scope', + 'select', + 'selection', + 'self', + 'sendToPrompt', + 'serialize', + 'sessionId', + 'set', + 'setAutoQueueMode', + 'setBatchCount', + 'setBgColor', + 'setCategory', + 'setCollapsed', + 'setColor', + 'setDisabled', + 'setExecution', + 'setHeight', + 'setHidden', + 'setLabel', + 'setLinked', + 'setMode', + 'setOption', + 'setPinned', + 'setPosition', + 'setProperty', + 'setResizable', + 'setSerializeWidgets', + 'setSerializedValue', + 'setShape', + 'setSize', + 'setSizeConstraints', + 'setSupply', + 'setTitle', + 'setTypeColor', + 'setValue', + 'setZoom', + 'settings', + 'severity', + 'shape', + 'shift', + 'shiftKey', + 'showDialog', + 'showItemNavigators', + 'showMenu', + 'showThumbnails', + 'side', + 'signal', + 'size', + 'snapshot', + 'socketless', + 'source', + 'sourceIndex', + 'sourceNodeId', + 'sourceSlotId', + 'src', + 'start', + 'status', + 'step', + 'step2', + 'stopPropagation', + 'storage', + 'subgraphs', + 'submenu', + 'submissions', + 'summary', + 'supplierNodeId', + 'supply', + 'supports', + 'surface', + 'surfaceHovered', + 'system', + 'targetIndex', + 'targetNodeId', + 'targetSlotId', + 'targets', + 'temperature_c', + 'text', + 'textSecondary', + 'title', + 'to', + 'tooltip', + 'total', + 'type', + 'typeColor', + 'ui', + 'unconnectedInputs', + 'update', + 'url', + 'useGrouping', + 'utilization_percent', + 'value', + 'values', + 'variant', + 'version', + 'volume', + 'volumes', + 'when', + 'widget', + 'widgetConfig', + 'widgetType', + 'widgetValue', + 'widgets', + 'width', + 'workflow', + 'x', + 'y' +]) diff --git a/src/platform/nodeApi/capabilityDoc.test.ts b/src/platform/nodeApi/capabilityDoc.test.ts new file mode 100644 index 00000000000..f1a55303672 --- /dev/null +++ b/src/platform/nodeApi/capabilityDoc.test.ts @@ -0,0 +1,64 @@ +import { execFileSync } from 'node:child_process' +import { readFileSync } from 'node:fs' +import { describe, expect, it } from 'vitest' + +import { reachableDeclarationNames } from '../../../scripts/node-api/gen_api_surface.mjs' + +describe('the documented capability list', () => { + it('matches CAPABILITIES', () => { + // A stale list is not a documentation nit. When the doc said eleven + // capabilities and the code had eighteen, agents read widgets.mount, + // widgets.canvas, setSizeConstraints and defs.define as unimplemented and + // punted twelve files against API that already existed. + expect(() => + execFileSync('node', [ + 'scripts/node-api/gen_capability_list.mjs', + '--check' + ]) + ).not.toThrow() + }) + + it('routes every reviewed service and extension hook', () => { + const reference = readFileSync('docs/node-api/reference.md', 'utf8') + + for (const entry of [ + '`system`', + '`SystemHandle`', + '`monitor(): Promise`', + '`playSound(def: PlaySoundDef): Promise`', + '`documentId(): string | undefined`', + '`onPromptSerialize`', + '`PromptInputProjection`', + '`PromptInputProjector`' + ]) { + expect(reference).toContain(entry) + } + + for (const privateEntry of [ + '`cacheSize`', + '## Host-plumbing declarations' + ]) { + expect(reference).not.toContain(privateEntry) + } + }) + + it('names every public type reachable from Comfy', () => { + const reference = readFileSync('docs/node-api/reference.md', 'utf8') + const missing = [ + ...reachableDeclarationNames('src/platform/nodeApi') + ].filter((name) => !reference.includes(`\`${name}\``)) + + expect(missing).toEqual([]) + }) + + it('documents async resolver behavior consistently', () => { + const guides = ['execution.md', 'how-to.md', 'demo-nodes.md'].map((name) => + readFileSync(`docs/node-api/${name}`, 'utf8') + ) + + for (const guide of guides) { + expect(guide).toMatch(/resolver[\s\S]{0,500}(promise|async)/i) + expect(guide).not.toMatch(/resolvers? (?:are|is) pure and synchronous/i) + } + }) +}) diff --git a/src/platform/nodeApi/skillRoutes.test.ts b/src/platform/nodeApi/skillRoutes.test.ts new file mode 100644 index 00000000000..6b6e4fdc7b0 --- /dev/null +++ b/src/platform/nodeApi/skillRoutes.test.ts @@ -0,0 +1,76 @@ +/** + * Every destination the conversion skill names must exist. + * + * The skill is the single artifact steering every conversion, and a destination + * that does not exist is worse than a missing capability: the agent writes + * confident code against it, every static check passes, and the failure only + * appears when a user loads the pack. This has already cost more converted files + * than absent API has — the table routed drawing at `node.decorations.set`, + * widget invalidation at `setOptions` (the member is `setOption`), and polling + * at `node.onChange`, none of which ship. + * + * Only the *destination* column is checked. The left column names the legacy + * API on purpose, and planned-but-unshipped capabilities are exactly what must + * not be recommended. + */ +import { readFileSync } from 'node:fs' +import { join } from 'node:path' +import { describe, expect, it } from 'vitest' + +import { API_MEMBERS } from './apiSurface' + +const SKILL = join( + import.meta.dirname, + '../../../.claude/skills/converting-custom-nodes/SKILL.md' +) + +/** + * Prose that reads as a call but names no member. + * + * `resolve` and `mount` are contract keys the pack implements rather than + * members it calls; the rest are English. + */ +const NOT_MEMBERS = new Set(['delete', 'punt', 'remove', 'the', 'it']) + +/** The middle column of every mapping row, which is where destinations live. */ +function destinationCells(markdown: string): string[] { + return markdown + .split('\n') + .filter((line) => line.startsWith('|') && !/^\|[\s:|-]+\|$/.test(line)) + .map((line) => line.split('|').map((cell) => cell.trim())) + .filter((cells) => cells.length >= 4 && !/^`?Old`?$/.test(cells[1])) + .map((cells) => cells[2]) +} + +/** Members named as calls (`foo(`) or accesses (`.foo`) inside code spans. */ +function membersIn(cell: string): string[] { + const code = [...cell.matchAll(/`([^`]+)`/g)] + .map((m) => m[1]) + // A file path's extension is not a member access. + .filter((span) => !span.includes('/')) + .join(' ') + return [ + ...[...code.matchAll(/\b([A-Za-z_]\w*)\s*\(/g)].map((m) => m[1]), + ...[...code.matchAll(/\.([A-Za-z_]\w*)\b/g)].map((m) => m[1]) + ].filter((name) => !NOT_MEMBERS.has(name)) +} + +describe('the skill only routes to API that exists', () => { + const markdown = readFileSync(SKILL, 'utf8') + + it('finds the mapping table, so a passing run means something', () => { + // Without this, a rename that breaks the parse would make every assertion + // below vacuously true — the check would report success having read nothing. + expect(destinationCells(markdown).length).toBeGreaterThan(15) + }) + + it('names no member the published API does not define', () => { + const unknown = destinationCells(markdown) + .flatMap((cell) => membersIn(cell).map((name) => ({ cell, name }))) + .filter(({ name }) => !API_MEMBERS.has(name)) + + expect( + unknown.map(({ name, cell }) => `${name} — routed by: ${cell}`) + ).toEqual([]) + }) +}) diff --git a/src/platform/nodeApi/widgetHandle.test.ts b/src/platform/nodeApi/widgetHandle.test.ts index a3ac9758c89..65452a52391 100644 --- a/src/platform/nodeApi/widgetHandle.test.ts +++ b/src/platform/nodeApi/widgetHandle.test.ts @@ -92,8 +92,6 @@ describe('widget surface', () => { }) describe('setOptions preserves live accessors', () => { - // kjnodes defines `values` as a getter for dynamic combos; a spread-based - // merge would pin it to a one-time snapshot. it('keeps a getter a getter', () => { let calls = 0 const w = node.widgets![0] @@ -127,6 +125,28 @@ describe('widget surface', () => { expect(opts.tooltip).toBe('hi') expect(opts.step).toBe(2) }) + + it('reads an incoming combo list again after its backing values change', () => { + let values = ['first'] + const incoming = { + get values() { + return values + } + } + const combo = node.addWidget('combo', 'model', 'first', () => undefined, { + values: [] + }) + const widget = widgets.get('model')! + widget.setOption('values', () => incoming.values) + + const readValues = () => { + const current = combo.options.values + return typeof current === 'function' ? current(combo) : current + } + expect(readValues()).toEqual(['first']) + values = ['second'] + expect(readValues()).toEqual(['second']) + }) }) describe('hidden replaces the converted-widget hack', () => { diff --git a/tools/devtools/README.md b/tools/devtools/README.md index 6d4ceae428a..5a49ed51be9 100644 --- a/tools/devtools/README.md +++ b/tools/devtools/README.md @@ -7,6 +7,8 @@ This directory contains development tools and test utilities for ComfyUI, previo - `__init__.py` - Server endpoints for development tools (`/api/devtools/*`) - `dev_nodes.py` - Development and testing nodes for ComfyUI - `fake_model.safetensors` - Test fixture for model loading tests +- `web/stableApiDemo.js` - Runnable frontend-only nodes built with the published + node API; see [`docs/node-api/demo-nodes.md`](../../docs/node-api/demo-nodes.md) ## Purpose diff --git a/tools/devtools/web/stableApiDemo.js b/tools/devtools/web/stableApiDemo.js new file mode 100644 index 00000000000..9571b909d37 --- /dev/null +++ b/tools/devtools/web/stableApiDemo.js @@ -0,0 +1,40 @@ +import { comfy } from '/comfy/api/v2.js' + +const api = comfy.forMajor(2) + +api.require('defs.define') +api.require('node.resolve') + +api.defs.define({ + type: 'DEMO/ConstantText', + title: 'DEMO Constant Text', + category: 'DEMO', + description: 'Supplies text to a backend node without executing itself.', + outputs: [{ name: 'text', type: 'STRING' }], + widgets: [ + { + type: 'text', + name: 'value', + value: 'Hello from the published node API', + serialize: true + } + ], + execution: 'frontend', + resolve: ({ self }) => ({ + text: { literal: String(self.widgetValue('value') ?? '') } + }) +}) + +api.defs.define({ + type: 'DEMO/Reroute', + title: 'DEMO Reroute', + category: 'DEMO', + description: 'Forwards its input without becoming a backend prompt node.', + inputs: [{ name: 'in', type: '*' }], + outputs: [{ name: 'out', type: '*' }], + execution: 'frontend', + resolve: ({ self }) => { + const input = self.input('in') + return { out: input ? { forwardTo: input } : { omit: true } } + } +})