diff --git a/docs/adr/ECS-WIDGETS-0023-widget-entities-with-a-legacy-layer.md b/docs/adr/ECS-WIDGETS-0023-widget-entities-with-a-legacy-layer.md index f00db0403ac..8abb878eff8 100644 --- a/docs/adr/ECS-WIDGETS-0023-widget-entities-with-a-legacy-layer.md +++ b/docs/adr/ECS-WIDGETS-0023-widget-entities-with-a-legacy-layer.md @@ -161,3 +161,175 @@ the registration boundary instead of adding it to renderer inner loops. Bevy's [ECS callback example](https://bevy.org/examples/ecs-entity-component-system/callbacks/) shows a related behavior boundary: a component holds a registered system ID, and the ECS runs that system on demand. + +--- + +## Amended (2026-10-26): a per-node widget form is emitted only where it can be _proven_ + +`widgets_values_form` is a per-node declaration of which widgets serialize and in what order +(`comfy-multi-player` schema amendment A24). Object-shaped forms are closed and catalog-free. +Array-form writes still require the pinned catalog for A25's dynamic-combo overlay; projection +uses the stored form. An incorrect form can still misassociate values: + +- A node with **no** form keeps today's catalog behaviour, which is imperfect but visible — a + name that does not resolve is refused loudly. +- A node with a **wrong** form is authoritative. Values are misplaced silently and nothing + downstream has anything left to disagree with. + +So the rule is not "derive the order from `node.widgets`". It is: + +> **Emit a form only where core can prove the final `widgets_values` is the array it itself +> produced from `node.widgets`. Where it cannot prove that, emit nothing.** + +Deriving is a guess dressed as a declaration; proving is not. `proveWidgetForm` +(`src/lib/litegraph/src/widgets/widgetFormProof.ts`) is that predicate, and it returns an order +**only** alongside `provable: true` — never a partial order beside a refusal. The result type is a +discriminated union, so using `order` as a `string[]` without checking `provable` does not compile, and an empty +serializing order is refused (`widgets_not_serialized`) because the document rejects an empty form. + +### Why deriving does not work, measured + +The obvious implementation — derive after `runExtensionSerializeHook`, since that hook receives the +final values — is wrong for two independent reasons. `replayed` is required for the same reason +as `coreIsLastWriter` below: a missing-node shell's `has_errors` lives on the node instance, not in +the serialized payload, so only the caller can say the payload was replayed. + +**`LGraphNode.serialize` is not the last writer.** `SubgraphNode.serializeFromStoreState` calls +`super.serializeFromStoreState(state)` — hook included — and _then_ replaces `widgets_values` +from `this.inputs` filtered to those carrying a `widgetId`, or deletes it. That is a different +collection from `node.widgets` (promoted widgets are linked inputs, per +[ADR-SUBGRAPH-PROMOTION-0009](SUBGRAPH-PROMOTION-0009-represent-promoted-widgets-as-linked-inputs.md)), +so a form written at the end of the base method is already stale when the subclass returns. +Identity alone does not catch this (see the call-site section below); `coreIsLastWriter` does. The +proof still checks **array identity**, deliberately not deep equality, against an instance hook +that assigns a new array: a rebuilt equal-looking array came from a source core cannot see, and a +coincidental match is not evidence. That check is only meaningful when the caller passes the array +core built before the hook ran; passing the final array makes it vacuous. + +**`node.widgets` is a public JavaScript API that extensions mutate**, as the ADR body above +already describes. Shipping packs splice it, truncate it (`this.widgets.length = n`), reorder +it, and replace it wholesale during `onNodeCreated`; `this.serialize_widgets = false` appears in +the low hundreds of repositories. None of that is a defect to fix — it is the contract this ADR +documents — so the form has to tolerate it rather than assume it away. + +### The flag trap, which is the most likely way to get this wrong + +Three predicates look interchangeable and are deliberately different: + +| predicate | effect on `widgets_values` | +| ------------------------------------ | --------------------------------------------- | +| `widget.serialize === false` | **excluded** — the only one that governs it | +| `widget.options.serialize === false` | **not excluded**; affects the API prompt only | +| either false | neither; governs promotion and preview | + +`addWidget` does not hoist the options-bag flag onto the widget, so a widget flagged only there +**does occupy a positional slot.** That is why every `KSampler` serializes 7 values +(`control_after_generate`) and every `LoadImage` serializes 2 (its upload button). A form built from `options.serialize` is one slot +short on both. + +`widgets/widgetFormProof.serialize.test.ts` pins the _rule_ against the real serializer: the flag +cases build an `LGraphNode` in a graph, add widgets through `addWidget`, and read +`node.serialize()`, so the assertion is on production's own `widgets_values` rather than on a +filter the test duplicates. A widget added with `{ serialize: false }` in its options bag still +occupies a slot. It imports `LGraphNode` through the LiteGraph barrel, which the `litegraph-api` +compatibility contract permits; `widgetFormProof.test.ts` keeps the pure-predicate cases. + +It does **not** instantiate a real `KSampler` or `LoadImage`, so the 7 and the 2 above remain +measurements of production code rather than assertions this file makes. Pinning them against the +real widget lists needs the extension constructors and so lives in the stacked test PR; until +that lands, treat those two numbers as evidence for the rule rather than as regression cover for +it. + +### Refusals, each a measured shape + +`class_may_rewrite_payload` · `replayed_serialization` · `widgets_not_serialized` · +`values_missing` · `values_replaced` · `values_not_positional` · `length_mismatch` · +`duplicate_widget_name` · `unnameable_widget` + +Four are worth calling out. **`replayed_serialization`** covers an unregistered node type, whose +payload is replayed verbatim and never reaches the hook, and whose widgets are _synthesised from +the stored values_ with names `UNKNOWN`, `UNKNOWN_1`, … A form there would be syntactically +valid, authoritative, and meaningless, so it is refused before any other check can accept it. +**`duplicate_widget_name`** conservatively refuses serializing widgets sharing a name. Core's +named projection overwrites duplicate keys, although the document's positional array-form path +can represent duplicate occurrences. Measured in the corpus: `RecordAudio` stores 2 values under +1 name. + +**`values_missing`** and **`values_replaced`** are deliberately distinct, because they send a +reader to different problems. `values_missing` is an absent `widgets_values` left by an +instance hook that deleted it. `values_replaced` is an array core did not produce, which an +`onSerialize` hook assigning a new array leaves behind. (`SubgraphNode` and class-level +`serialize` overrides never reach either: `class_may_rewrite_payload` refuses them first.) `values_not_positional` is reserved for an +object-shaped payload: the VHS family's `onSerialize` writes a dict keyed by every widget name, +including widgets flagged `serialize === false`, so its keys and any derived order disagree. +`widgets_not_serialized` covers `this.serialize_widgets = false`, which rgthree's group muters, +was-node-suite's fast groups and Crystools' debugger all set. + +### What the proof does not guarantee + +Array identity proves the payload is still core's own array. It does not prove the array's +_contents_ were untouched: a serialize hook can reorder `node.widgets`, or mutate the produced +array in place, while preserving both identity and length. `proveWidgetForm` would then report +`provable: true` with an order that does not match the values. + +That is a known limit, not an oversight, and it is why no production code calls this predicate +yet; its only consumer is the registry census, which reports refusal rates and emits nothing. A +producer must close both mutation paths before calling this predicate. A pre-hook order snapshot +handles later widget-list reordering, but the producer must also detect changes to the produced +values or otherwise prove that the final name/value association is unchanged. That belongs in +the producer change, where the pre-hook state is in scope. + +### The call site is part of the rule, not an implementation detail + +The first version of this amendment said the form is emitted "after the extension serialize hook" +and left it there. That is wrong, and driving the predicate through the real serialize path proved +it: at the end of `LGraphNode.serializeFromStoreState` a `SubgraphNode`'s `widgets_values` is +**still core's own array** — the subclass replaces it only after `super` returns. So the identity +check, which exists precisely to catch that subclass, passed. The proof **accepted**, returning an +order of two names for a payload that ended up holding one value. + +An acceptance is the one outcome this design cannot tolerate, so the fix is not a better call site — +it is to stop trusting the caller to know it stands at the last writer. `coreIsLastWriter` is a +**required** option with no default: + +```ts +coreIsLastWriter: node.serializeFromStoreState === + LGraphNode.prototype.serializeFromStoreState && + node.serialize === LGraphNode.prototype.serialize +``` + +It compares against the live base prototype, so an extension that globally wraps +`LGraphNode.prototype.serialize` still compares equal. That is a known limit for the producer to +close, for example by comparing against references captured at module load. + +Required rather than derived for two reasons. `tsc` then refuses any new call site until the +question has been answered, so the assumption cannot creep back in silently. And the comparison +needs both the instance and the base prototype in scope, which is true at the call site and not +inside the widgets layer — deriving it here would mean importing `LGraphNode` across the +`node-widgets` → `graph-model` boundary that this repo's own architecture gate enforces. + +It is checked **before** every other refusal, because a payload that is not final makes the rest +moot: reporting `duplicate_widget_name` on a payload about to be replaced sends a reader to the +wrong problem. The refusal is `class_may_rewrite_payload`, and it covers packs that override +`serialize` exactly as it covers `SubgraphNode`. + +Passing a hardcoded `true` to silence the type error reintroduces the defect. That is worth saying +out loud, because it is the cheapest wrong fix available. + +### Scope + +This amendment adds the predicate and a diagnostic (`needsFormButHasNone`, which identifies +object-shaped values with no form — the population a form would rescue and that nothing can +currently count). **Nothing emits a form through `proveWidgetForm` yet.** The producer is +deliberately a later change, so the predicate can be landed, measured against the custom-node +ecosystem, and trusted first. + +That is narrower than "no form is emitted anywhere", and the difference matters to whoever wires +this up. The Agent wire already declares forms on its own `add_node` path: +`docOpMinter.wireNodeSnapshot` attaches `widgets_values_form` from +`provenWidgetValuesForm` (`src/workbench/extensions/agent/crdt/widgetValuesForm.ts`, per +[ADR-WIDGET-SERIALIZATION-0047](WIDGET-SERIALIZATION-0047-agent-wire-widget-form-is-proven-not-derived.md)). +The Agent producer proves the filtered named payload, while this predicate checks positional-array +provenance. Their acceptance policies intentionally differ: #22505 and #22508 show that importing +the positional identity or class-override refusals into the Agent wire would reject valid +uncatalogued nodes. Any future consolidation must preserve those separate contracts. diff --git a/src/lib/litegraph/src/widgets/widgetFormProof.serialize.test.ts b/src/lib/litegraph/src/widgets/widgetFormProof.serialize.test.ts new file mode 100644 index 00000000000..c002c33d07a --- /dev/null +++ b/src/lib/litegraph/src/widgets/widgetFormProof.serialize.test.ts @@ -0,0 +1,77 @@ +import { describe, expect, it } from 'vitest' + +import { LGraph, LGraphNode } from '@/lib/litegraph/src/litegraph' +import { proveWidgetForm } from '@/lib/litegraph/src/widgets/widgetFormProof' + +function nodeInGraph(): LGraphNode { + const graph = new LGraph() + const node = new LGraphNode('test') + node.serialize_widgets = true + graph.add(node) + return node +} + +function coreIsLastWriter(node: LGraphNode): boolean { + return ( + node.serialize === LGraphNode.prototype.serialize && + node.serializeFromStoreState === + LGraphNode.prototype.serializeFromStoreState + ) +} + +function proveRealSerialization(node: LGraphNode) { + const serialised = node.serialize() + return { + serialised, + proof: proveWidgetForm( + node.widgets, + serialised, + serialised.widgets_values, + { + serializeWidgets: node.serialize_widgets, + replayed: node.last_serialization !== undefined, + coreIsLastWriter: coreIsLastWriter(node) + } + ) + } +} + +describe('proveWidgetForm, driven through LGraphNode.serialize', () => { + it('counts a widget flagged only in its options bag, because widgets_values does', () => { + const node = nodeInGraph() + node.addWidget('number', 'seed', 1, () => undefined, {}) + node.addWidget( + 'combo', + 'control_after_generate', + 'randomize', + () => undefined, + { serialize: false, values: ['randomize', 'fixed'] } + ) + + const { serialised, proof } = proveRealSerialization(node) + + expect(serialised.widgets_values).toEqual([1, 'randomize']) + expect(proof.provable).toBe(true) + expect(proof.order).toEqual(['seed', 'control_after_generate']) + expect(proof.refusal).toBeNull() + }) + + it('excludes a widget flagged on the widget itself, because widgets_values does too', () => { + const node = nodeInGraph() + node.addWidget('text', 'audio', 'clip.mp3', () => undefined, {}) + const widgetFlagged = node.addWidget( + 'button', + 'audioUI', + '', + () => undefined, + {} + ) + widgetFlagged.serialize = false + + const { serialised, proof } = proveRealSerialization(node) + + expect(serialised.widgets_values).toEqual(['clip.mp3']) + expect(proof.provable).toBe(true) + expect(proof.order).toEqual(['audio']) + }) +}) diff --git a/src/lib/litegraph/src/widgets/widgetFormProof.test.ts b/src/lib/litegraph/src/widgets/widgetFormProof.test.ts new file mode 100644 index 00000000000..b2b2f1edb3c --- /dev/null +++ b/src/lib/litegraph/src/widgets/widgetFormProof.test.ts @@ -0,0 +1,243 @@ +import { describe, expect, it } from 'vitest' + +import { + needsFormButHasNone, + proveWidgetForm +} from '@/lib/litegraph/src/widgets/widgetFormProof' + +const serializing = { + serializeWidgets: true, + replayed: false, + coreIsLastWriter: true +} + +describe('proveWidgetForm', () => { + it('proves an ordinary node whose values it produced itself', () => { + const widgets = [{ name: 'seed' }, { name: 'steps' }] + const values = [1, 20] + + const proof = proveWidgetForm( + widgets, + { widgets_values: values }, + values, + serializing + ) + + expect(proof).toEqual({ + provable: true, + refusal: null, + order: ['seed', 'steps'] + }) + }) + + describe('refuses rather than guessing', () => { + it('when a subclass or extension replaced the array with an equal-looking one', () => { + const widgets = [{ name: 'a' }, { name: 'b' }] + + const proof = proveWidgetForm( + widgets, + { widgets_values: [1, 2] }, + [1, 2], + serializing + ) + + expect(proof).toEqual({ + provable: false, + refusal: 'values_replaced', + order: null + }) + }) + + it('when a subgraph deleted the values entirely', () => { + const widgets = [{ name: 'promoted_a' }, { name: 'promoted_b' }] + + const proof = proveWidgetForm(widgets, {}, [0, 0], serializing) + + expect(proof.refusal).toBe('values_missing') + }) + + it('when an extension replaced the array with a name-keyed object', () => { + const widgets = [ + { name: 'video' }, + { name: 'vhslatentpreview', serialize: false } + ] + + const proof = proveWidgetForm( + widgets, + { + widgets_values: { video: 'clip.mp4', vhslatentpreview: {}, crf: 23 } + }, + ['clip.mp4'], + serializing + ) + + expect(proof.refusal).toBe('values_not_positional') + }) + }) + + describe('refuses a payload it did not build', () => { + it('when the payload was replayed from a previous save', () => { + const widgets = [{ name: 'UNKNOWN' }, { name: 'UNKNOWN_1' }] + const values = [1, 2] + + const proof = proveWidgetForm( + widgets, + { widgets_values: values }, + values, + { + ...serializing, + replayed: true + } + ) + + expect(proof.refusal).toBe('replayed_serialization') + }) + }) + + describe('refuses when the widget list and the values disagree', () => { + it('when serialize_widgets is falsy', () => { + const proof = proveWidgetForm( + [{ name: 'mode' }, { name: 'filter' }], + {}, + undefined, + { ...serializing, serializeWidgets: false } + ) + + expect(proof.refusal).toBe('widgets_not_serialized') + }) + + it('when every widget is excluded from serialization', () => { + const values: unknown[] = [] + + const proof = proveWidgetForm( + [{ name: 'preview', serialize: false }], + { widgets_values: values }, + values, + serializing + ) + + expect(proof).toEqual({ + provable: false, + refusal: 'widgets_not_serialized', + order: null + }) + }) + + it('when every widget is excluded but values were still written', () => { + const values = [1] + + const proof = proveWidgetForm( + [{ name: 'preview', serialize: false }], + { widgets_values: values }, + values, + serializing + ) + + expect(proof.refusal).toBe('length_mismatch') + }) + + it('when the widget list was spliced after the values were written', () => { + const values = [1, 2] + + const proof = proveWidgetForm( + [{ name: 'a' }], + { widgets_values: values }, + values, + serializing + ) + + expect(proof.refusal).toBe('length_mismatch') + }) + + it('when two serializing widgets share a name', () => { + const values = ['a', 'b'] + + const proof = proveWidgetForm( + [{ name: 'audio' }, { name: 'audio' }], + { widgets_values: values }, + values, + serializing + ) + + expect(proof.refusal).toBe('duplicate_widget_name') + }) + + it('when a widget has no usable name', () => { + const values = [1, 2] + + const proof = proveWidgetForm( + [{ name: 'ok' }, { name: '' }], + { widgets_values: values }, + values, + serializing + ) + + expect(proof.refusal).toBe('unnameable_widget') + }) + }) + + it('refuses a node whose class can still rewrite the payload', () => { + const values = [1, 2] + + const proof = proveWidgetForm( + [{ name: 'promoted_a' }, { name: 'promoted_b' }], + { widgets_values: values }, + values, + { ...serializing, coreIsLastWriter: false } + ) + + expect(proof).toEqual({ + provable: false, + refusal: 'class_may_rewrite_payload', + order: null + }) + }) + + it('checks the last writer before any other refusal that also applies', () => { + const values = [1, 2] + + const proof = proveWidgetForm( + [{ name: 'dup' }, { name: 'dup' }], + { widgets_values: values }, + values, + { serializeWidgets: true, replayed: true, coreIsLastWriter: false } + ) + + expect(proof.refusal).toBe('class_may_rewrite_payload') + }) +}) + +describe('needsFormButHasNone', () => { + it('flags an object-shaped payload with no form', () => { + expect(needsFormButHasNone({ widgets_values: { video: 'clip.mp4' } })).toBe( + true + ) + }) + + it('flags an object-shaped payload whose form is null', () => { + expect( + needsFormButHasNone({ + widgets_values: { video: 'clip.mp4' }, + widgets_values_form: null + }) + ).toBe(true) + }) + + it('does not flag one that already declares a form', () => { + expect( + needsFormButHasNone({ + widgets_values: { video: 'clip.mp4' }, + widgets_values_form: { order: ['video'] } + }) + ).toBe(false) + }) + + it.for([ + { name: 'positional values', serialised: { widgets_values: [1, 2] } }, + { name: 'absent payload', serialised: {} }, + { name: 'null payload', serialised: { widgets_values: null } }, + { name: 'scalar payload', serialised: { widgets_values: 'clip.mp4' } } + ])('does not flag $name', ({ serialised }) => { + expect(needsFormButHasNone(serialised)).toBe(false) + }) +}) diff --git a/src/lib/litegraph/src/widgets/widgetFormProof.ts b/src/lib/litegraph/src/widgets/widgetFormProof.ts new file mode 100644 index 00000000000..d1853a3defc --- /dev/null +++ b/src/lib/litegraph/src/widgets/widgetFormProof.ts @@ -0,0 +1,128 @@ +interface WidgetForFormProof { + name: string + serialize?: boolean +} + +type WidgetFormRefusal = + | 'class_may_rewrite_payload' + | 'replayed_serialization' + | 'widgets_not_serialized' + | 'values_missing' + | 'values_replaced' + | 'values_not_positional' + | 'length_mismatch' + | 'duplicate_widget_name' + | 'unnameable_widget' + +export interface WidgetFormOptions { + serializeWidgets?: boolean + /** True when the payload was replayed from a previous save rather than built here. */ + replayed: boolean + /** True when the node uses core's base serialization implementations. */ + coreIsLastWriter: boolean +} + +export type WidgetFormProof = + | { provable: true; refusal: null; order: string[] } + | { provable: false; refusal: WidgetFormRefusal; order: null } + +function refuse(refusal: WidgetFormRefusal): WidgetFormProof { + return { provable: false, refusal, order: null } +} + +function serializingWidgets( + widgets: readonly WidgetForFormProof[] +): WidgetForFormProof[] { + return widgets.filter((widget) => widget.serialize !== false) +} + +function preconditionRefusal( + options: WidgetFormOptions +): WidgetFormRefusal | null { + if (!options.coreIsLastWriter) return 'class_may_rewrite_payload' + if (options.replayed) return 'replayed_serialization' + return null +} + +function provenValues( + widgets: readonly WidgetForFormProof[] | undefined, + serialised: { widgets_values?: unknown }, + producedValues: unknown, + options: WidgetFormOptions +): { values: unknown[] } | { refusal: WidgetFormRefusal } { + if (!widgets?.length || !options.serializeWidgets) { + return { refusal: 'widgets_not_serialized' } + } + + const values = serialised.widgets_values + if (values === undefined) return { refusal: 'values_missing' } + if (producedValues === undefined || values !== producedValues) { + return { + refusal: Array.isArray(values) + ? 'values_replaced' + : 'values_not_positional' + } + } + if (!Array.isArray(values)) return { refusal: 'values_not_positional' } + return { values } +} + +function addressableOrder( + serializing: readonly WidgetForFormProof[] +): { order: string[] } | { refusal: WidgetFormRefusal } { + const order: string[] = [] + const seen = new Set() + for (const widget of serializing) { + const name = widget.name + if (typeof name !== 'string' || name.length === 0) { + return { refusal: 'unnameable_widget' } + } + if (seen.has(name)) return { refusal: 'duplicate_widget_name' } + seen.add(name) + order.push(name) + } + return { order } +} + +/** + * Checks positional-array provenance: identity, cardinality and names. The caller must establish + * that the name/value correspondence did not change after core produced the values, and must + * pass as `producedValues` the array core built before the extension hook ran; passing the final + * array makes the identity check vacuous. + */ +export function proveWidgetForm( + widgets: readonly WidgetForFormProof[] | undefined, + serialised: { widgets_values?: unknown }, + producedValues: unknown, + options: WidgetFormOptions +): WidgetFormProof { + const precondition = preconditionRefusal(options) + if (precondition !== null) return refuse(precondition) + + const proven = provenValues(widgets, serialised, producedValues, options) + if ('refusal' in proven) return refuse(proven.refusal) + + const serializing = serializingWidgets(widgets ?? []) + if (serializing.length !== proven.values.length) + return refuse('length_mismatch') + if (serializing.length === 0) return refuse('widgets_not_serialized') + + const addressed = addressableOrder(serializing) + if ('refusal' in addressed) return refuse(addressed.refusal) + + return { provable: true, refusal: null, order: addressed.order } +} + +/** True when object-shaped widget values have no form that interprets them. */ +export function needsFormButHasNone(serialised: { + widgets_values?: unknown + widgets_values_form?: unknown +}): boolean { + const values = serialised.widgets_values + return ( + values !== null && + typeof values === 'object' && + !Array.isArray(values) && + serialised.widgets_values_form == null + ) +}