Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/INVARIANTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ older reader emits the key as passthrough and mis-projects `widgets_values` —
which is the SHA-pinned-consumer consequence A1 and A2 both document, and why
the pins move in one coordinated step.

Schema v6 adds validated producer-owned DynamicGroup row IDs in node properties (schema A26). Earlier readers could resize a tagged group without updating those IDs, so v1–v5 documents are refused and re-minted from source. Reader/document-host adoption and FE sign-off remain release prerequisites.

### KA-12 — Catalog pinned at mint
**Rule:** `meta.catalog_version` cites the catalog by SHA, not branch; reject NAMED widget writes to uncatalogued classes loudly. A subgraph instance's `type` is a definition UUID that no catalog describes, and its promoted widget values are POSITIONAL by frontend contract (ADR 0009), so a promoted host write (schema Amendment A15) is not a widget-name resolution and needs no catalog entry for the instance — it still needs A CATALOG (`catalog_required` without one) to tell an instance from an unseen class, and still refuses (`uncatalogued_widget_write`) to lay a positional array over named values the catalog cannot describe. A node that DECLARES its own ordered widget identity (`widgets_values_form`, schema Amendment A24) resolves names and occurrences from that declaration without a catalog mapping. A25 makes one validation exception: supplied `dynamic_groups` metadata identifies count fields for allocation-bound checks and refuses a count change that would invalidate the declaration, using the addressed occurrence's current value. The declaration still owns ordering; the catalog does not replace it. Without that metadata, the package cannot identify a count field, so callers must enforce compatible producer, catalog, and reader adoption. An undeclared node keeps every refusal above, and `meta.catalog_version` still pins the catalog every other node projects against.
**Why:** Replay semantics must not drift with a moving vocabulary.
Expand Down
1 change: 1 addition & 0 deletions docs/decisions/EXCEPTIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ is retained as history, not authorization to accept unprojectable writes.
| 2026-10-03 (PROPOSED) | KA-5 | **`insert_workflow` derives numeric group ids by the same pure 53-bit hash as KA-5's link ids, and accepts the same residual collision risk** (PR #264). ComfyUI_frontend's workflow schema declares `groups[].id` as `z.number()` (root and `definitions.subgraphs[].groups`) and refuses the whole workflow on a string, so the string `derivedId` groups used to carry made inserted workflows unloadable. `derivedGroupId` folds `insert:<opId>:<scope>:group:<original>` into `[1, Number.MAX_SAFE_INTEGER]` from the op's own content only (no document read, no retry), so replicas converge regardless of arrival order. Residual probability is the KA-5 row's: `≈ k²/2^54` for `k` groups in one document. **Unlike links, a realized group-id collision is NOT detected**: `applier.ts` merges groups by canonical JSON, not by id, so two distinct groups sharing an id both stay in the document and project with the same id; the frontend's behaviour on duplicate group ids is unverified. `project.ts` coerces a stored string group id to a number ONLY when it matches the derived form (`DERIVED_GROUP_ID`), so a workflow's own string group id round-trips `project(mint(wf))` unchanged and still fails the frontend schema exactly as it would without this package. | `src/remap.ts` `derivedGroupId`, `numericId`; `src/project.ts` `projectGroups`, `DERIVED_GROUP_ID`; `test/insert-workflow-numeric-group-ids.regression.test.ts` | Owner and approval: maintainer's to assign (proposed for Christian Byrne, who reviewed #264). Sunset: revisit if ComfyUI_frontend widens a group's `id` to `string \| number`, at which point the string `derivedId` suffices and this exception can be retired; add id-level collision detection to the groups merge before then if duplicate ids prove harmful on the canvas. |
| 2026-10-04 (PROPOSED) | FC-10 | **`widgets_values_form` is a package-local, provisional workflow FIELD** (schema Amendment A24), because the pinned catalog is per-class while the two shapes that need describing are per-instance: a node with two same-named serializable widgets, and a node whose custom serializer emits an object whose non-widget keys must survive a widget write. Neither is derivable from the end state, so the producer declares its instance's ordered widget identity and this package decomposes, addresses and projects through that declaration with no catalog lookup. comfy-cli's frozen `op-vocabulary-v1.md` does not define the field, so the deviation is from FC-10's cite-the-pinned-vocabulary rule rather than from KA-12 (a declaration is the name-to-position mapping, supplied by the party that knows it, not a guess at one). Bounded deliberately: ONE declared key, closed to `{order}` and refused rather than ignored when malformed; every other part of the form is derived; an undeclared node keeps the catalog path and all of its existing refusals; `SCHEMA_VERSION` is unchanged on A2's precedent and under A2's exact condition. **Two residuals, both stated rather than discovered later:** `insert_workflow` strips every `__`-prefixed key at every depth (`prepareInsertedWorkflow` / `scrubPrivateKeys`, a pre-existing anti-forgery guard), so a serializer's own `__`-prefixed data key inside `widgets_values` does not survive THAT path — pinned by test rather than papered over; and the §8.3 autogrow `inputcount` bump still writes the `widgets` map directly and consults no declared order, which is A23's pre-existing gap one op over. | `src/widget-form.ts`; `src/doc.ts` `widgetStorageFor`/`widgetStorageOf`/`storeWidgetValues`; `src/project.ts` `projectWidgets`; `src/applier.ts` `validateWidgetName`/`rejectIfOpaqueWidgets`/`hostWriteStorage`/`createAddedNode`/`rejectUnprojectableWidgets`; `src/read.ts`; `fixtures/golden-vectors/wire-layout.json`; `test/node-local-widget-form.test.ts`; schema Amendment A24 | Owner and approval: maintainer's to assign (FE-3036's owner). Sunset: amend comfy-cli's frozen vocabulary to define the field and pin the adopted definition here, reconciling the name and shape if upstream differs; retire outright if the catalog acquisition ever becomes per-instance, at which point the declaration becomes redundant rather than merely unneeded. |
| 2026-10-11 (PROPOSED) | KA-12 | A25 uses supplied DynamicGroup metadata to validate count writes even on an A24 self-described node. Name/occurrence resolution and projection still use the producer declaration; metadata only checks allocation bounds and refuses changing the addressed occurrence's count without an updated form. Metadata omission cannot identify a controller and remains a caller admission concern. | `src/applier.ts` `validateDynamicGroupValue`; `test/dynamic-group.test.ts`; schema Amendments A24/A25 | Owner and approval: maintainer's to assign. Reconcile when count safety becomes part of the producer declaration or atomic row-operation protocol. |
| 2026-10-11 (PROPOSED) | FC-10 / KA-12 | A26's `properties.dynamicGroupRows` is a package-local draft field, not part of the frozen vocabulary. It preserves producer-owned IDs and requires supplied catalog group metadata for membership/allocation validation, including on self-described nodes; widget mapping still follows the declaration. Schema v6 fences older writers, and tagged scalar resize is refused until an atomic row protocol exists. | `src/dynamic-group-rows.ts`; `src/doc.ts`; `src/read.ts`; `src/applier.ts`; schema Amendment A26 | Owner and approval: maintainer's to assign. Sunset: adopt and pin the reviewed row-operation/producer contract, reconciling the field and schema before enabling writers. |
12 changes: 10 additions & 2 deletions docs/dynamic-groups.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@ Like dynamic-combo selections, count changes only change projection. Fields in h

An explicit `widgets_values_form` remains authoritative for names, occurrences, and projection order. Supplied group metadata is used for count validation only: allocation bounds are checked, and a change from the addressed occurrence's stored count that requires replacing the declaration is refused before mutation. A catalog-free declaration cannot identify group controllers, so that path does not provide count safety. This PR does not introduce an op for changing producer-owned forms.

No wire key or op kind changes. New catalog content changes the catalog hash. Consumers must adopt this reader before receiving a DynamicGroup catalog; old documents remain tied to their original catalog and must be re-minted to adopt a new layout (KA-11, KA-12, FC-10).
A25 does not change wire keys or op kinds; A26 below introduces the opt-in row identity manifest and schema v6. New catalog content changes the catalog hash. Consumers must adopt this reader before receiving a DynamicGroup catalog; old documents remain tied to their original catalog and must be re-minted to adopt a new layout (KA-11, KA-12, FC-10).

## Namespace safety

Relative group fields must be distinct. A controller cannot also be a DynamicCombo selector or another group's row field. Each source widget order may declare a controller once, and any generated names already present must be its contiguous minimum-row block. The selected layout also refuses duplicate generated fields/controllers before they can share a register. Ordinary duplicate names keep their existing occurrence identities; unrelated dotted names are not rejected merely for sharing a group prefix. Seed companions such as `weights.0.seed.0` remain legal when declared by the template metadata.

## Reader and producer adoption

The released reader `0.3.10` and this draft both use schema v5. The schema gate and a catalog hash do **not** prove that an old reader understands `dynamic_groups`. An offline probe with the CLI composite fixture demonstrated that the old reader can mint a new positional workflow, acknowledge an `after` edit, and write that value into the first row's enabled position while leaving the real trailing value unchanged. Omitting group metadata produces the same unsupported mapping even with the current reader.
The released reader `0.3.10` uses schema v5. A25 originally used v5 as well; A26 in this draft requires v6. The schema gate and a catalog hash do **not** prove that an old reader understands `dynamic_groups`. An offline probe with the CLI composite fixture demonstrated that the old reader can mint a new positional workflow, acknowledge an `after` edit, and write that value into the first row's enabled position while leaving the real trailing value unchanged. Omitting group metadata produces the same unsupported mapping even with the current reader.

| Producer declaration | Reader/catalog combination | Supported scope |
| --- | --- | --- |
Expand Down Expand Up @@ -53,3 +53,11 @@ JS
```

The released reader acknowledges the write at position 4, replacing the enabled value. This reader edits the final `tail` position. Both readers accept an existing-field edit with the fixture's explicit form; that is a separate supported path, not evidence of live row-resize support.

## Draft row identity reader

Schema A26 accepts an opt-in `properties.dynamicGroupRows` manifest, for example `{"loras": ["row-a", "row-c"]}` alongside a stored `loras=2`. IDs are opaque producer-owned strings scoped to the node lifetime and group. The reader validates distinct nonempty IDs, known catalog groups, stored counts and allocation bounds. It preserves saved rows above a changed advertised maximum and does not assign IDs to untagged workflows.

`readGraph` exposes a deeply frozen `dynamicGroupRows` field for tagged nodes. Scalar count edits that would contradict a listed group's IDs are refused, even without a catalog at apply time. Existing field edits keep their current numeric addresses; stable-ID editing is not implemented by this reader. The next stage must implement atomic row insertion/deletion, stable field targets, links/promotions and admission together.

This is a reader preparation draft, not a released row-operation capability. Schema v6 prevents v5 readers from accepting a tagged document; current readers refuse older documents, which the host must re-mint from source. Reader, document-host and producer pins must be coordinated and reviewed before adoption. No CLI, frontend or agent producer is enabled by this PR.
18 changes: 15 additions & 3 deletions docs/multiplayer-schema.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Multiplayer workflow-document schema — v5
# Multiplayer workflow-document schema — v6

`SCHEMA_VERSION = 5`
`SCHEMA_VERSION = 6`

> **State: DRAFT — awaiting FE sign-off (FE-1330).**
>
Expand Down Expand Up @@ -892,7 +892,7 @@ the epoch; cross-epoch struct updates never merge.

## 10. Versioning and `migrate()`

- `SCHEMA_VERSION = 5`, stored in `meta.schema_version` at mint.
- `SCHEMA_VERSION = 6`, stored in `meta.schema_version` at mint.
- Private-alpha policy keeps one current format: old layouts are re-minted at
their source and compatibility readers/migrations are not provided.
- `migrate(doc, fromVersion)` contract: exact no-op when
Expand Down Expand Up @@ -2503,3 +2503,15 @@ here.
An opt-in `dynamic_groups` catalog entry expands a saved row-count controller into indexed widget identities. These use the existing name-keyed registers; no op kind or on-document key changes. The new catalog hash requires coordinated reader adoption. Existing documents retain their original catalog and are re-minted to adopt the new layout.

Count writes change the projected layout, retaining hidden registers and applying fresh-row defaults only at read time. They do not delete rows or links. Explicit `widgets_values_form` declarations remain authoritative; supplied group metadata adds allocation-bound validation and rejects a count change requiring a new declaration before mutation. This is A24's one catalog-validation exception, applied to the addressed occurrence in top-level, interior, and promoted writes. Without group metadata, the package cannot identify count fields; compatible producer/catalog/reader admission remains the caller's responsibility. See [DynamicGroup catalog support](dynamic-groups.md) for the producer contract, bounds, and fixture provenance.

## Amendment A26 — 2026-10-11 — DynamicGroup row identity reader (schema v6)

An opt-in node carries `properties.dynamicGroupRows`, mapping catalog group names to ordered arrays of distinct, nonempty row ID strings. IDs are scoped to the node lifetime and group; different groups or nodes may use the same string. The producer supplies the IDs, and this reader never invents or replaces them. Each listed group requires pinned catalog metadata and an explicitly stored count equal to its ID array length. Existing allocation bounds apply; a changed advertised min/max does not discard saved rows.

Mint, definition import, `add_node` and `insert_workflow` validate before attaching node maps. Projection validates catalog membership and stored counts again. `readGraph` exposes only the validated, deeply frozen ID manifest as `NodeSnapshot.dynamicGroupRows`, without copying unrelated properties. Its catalog-free validation covers shape and count; catalog admission remains the caller's responsibility. Workflow insertion preserves this closed ID mapping while continuing to scrub unrelated private keys.

Ordinary scalar count changes on tagged groups are refused before mutation on top-level, interior and promoted paths, including when the apply caller omits the catalog. Unchanged counts and existing-field edits retain the current registers and replay behavior. Untagged nodes keep A25 behavior. No new operation kind is implemented here: atomic insertion/deletion, stable-ID field addressing, row-aware links/promotions and writer admission remain required follow-ups.

Schema v6 is an adoption boundary. A v5 reader would accept a tagged group's scalar resize and leave its row IDs attached to the wrong positions. Readers and document hosts must adopt v6 together before a producer emits the manifest. Per §10 and KA-11, v1–v5 documents are refused without relabelling; the host re-mints their source workflows. A package source change is not consumer sign-off or permission to release. Existing JSON conformance fixtures remain workflow/op inputs and are re-minted at the current schema; the wire-layout vector records v6. No corpus expectation is regenerated from this applier.

`test/dynamic-group-row-identity.test.ts` covers imports, round trips, two groups with a sibling DynamicCombo and seed companion, malformed manifests, unchanged-count/field edits in both orders, byte-identical rejection/retry/abort, frozen snapshots and old-schema refusal. The existing schema, clock and wire-layout suites retain the shared read gate.
4 changes: 2 additions & 2 deletions fixtures/golden-vectors/wire-layout.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"format_version": 1,
"comment": "Canonical WIRE layout of the multiplayer Y.Doc (schema §1). `roots` maps each logical role to the Y.Doc ROOT TYPE NAME as it is spelled inside an encoded Yjs update; `reserved_node_keys` names the per-node keys the doc owns. These strings are a cross-implementation contract, not an internal detail: two replicas that disagree about them cannot exchange a document, because a follower forking from the bootstrap snapshot (KA-10) resolves roots by name. Renaming any of them is a layout break: schema §10 requires a SCHEMA_VERSION bump, an explicit old-layout disposition, updated fixtures or a fixture-format note, an amendment section in the schema document, and FE sign-off, plus a matching edit here and in docs/INVARIANTS.md KA-3. test/wire-layout-contract.test.ts holds the four-way name agreement and the SCHEMA_VERSION triple's consistency; it does NOT enforce the bump itself, so read §10 rather than treating a green suite as permission. A second implementation should read this file rather than transcribing the names. Schema v4 adds clock_reservations; schema v5 adds occurrence-addressed widget keys. Amendment A24 adds the reserved per-node key `widget_form` WITHOUT a schema bump, on Amendment A2's precedent: it is reachable only on a node whose payload declared `widgets_values_form`, so no earlier v5 document can contain it. Private-alpha old layouts are refused, not migrated. The reservation root is created on first successful admission and empty roots are absent from snapshots.",
"schema_version": 5,
"comment": "Canonical WIRE layout of the multiplayer Y.Doc (schema \u00a71). `roots` maps each logical role to the Y.Doc ROOT TYPE NAME as it is spelled inside an encoded Yjs update; `reserved_node_keys` names the per-node keys the doc owns. These strings are a cross-implementation contract, not an internal detail: two replicas that disagree about them cannot exchange a document, because a follower forking from the bootstrap snapshot (KA-10) resolves roots by name. Renaming any of them is a layout break: schema \u00a710 requires a SCHEMA_VERSION bump, an explicit old-layout disposition, updated fixtures or a fixture-format note, an amendment section in the schema document, and FE sign-off, plus a matching edit here and in docs/INVARIANTS.md KA-3. test/wire-layout-contract.test.ts holds the four-way name agreement and the SCHEMA_VERSION triple's consistency; it does NOT enforce the bump itself, so read \u00a710 rather than treating a green suite as permission. A second implementation should read this file rather than transcribing the names. Schema v4 adds clock_reservations; schema v5 adds occurrence-addressed widget keys. Amendment A24 adds the reserved per-node key `widget_form` WITHOUT a schema bump, on Amendment A2's precedent: it is reachable only on a node whose payload declared `widgets_values_form`, so no earlier v5 document can contain it. Private-alpha old layouts are refused, not migrated. The reservation root is created on first successful admission and empty roots are absent from snapshots. Amendment A26 uses schema v6 to fence older writers from resizing groups whose producer-owned row IDs are persisted in node properties; root names and reserved node keys are unchanged.",
"schema_version": 6,
"roots": {
"nodes": "nodes",
"links": "links",
Expand Down
Loading
Loading