From e0525785d836a00cb88427208865c6b0792cf889 Mon Sep 17 00:00:00 2001 From: bymyself Date: Fri, 23 Oct 2026 00:19:07 -0700 Subject: [PATCH 1/3] docs: rule that hidden DynamicGroup rows are retained, not deleted (ADR-039) Settles the open question in the #276 review: retain hidden row keys and correct the enumerated frontend readers, rather than adding a document delete operation. No source change. The ruling records why a delete op is refused (its intent is inferred from a count change; it makes a visibility change and a value change contend on one register; it makes shrink/regrow lossy), enumerates the eleven frontend readers that see retained keys and which three must change, and names the one test #276 still owes - a byte-identical replay assertion for the row-count op itself. --- ...mic-group-rows-are-retained-not-deleted.md | 177 ++++++++++++++++++ 1 file changed, 177 insertions(+) create mode 100644 docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md diff --git a/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md b/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md new file mode 100644 index 0000000..3dd4055 --- /dev/null +++ b/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md @@ -0,0 +1,177 @@ +# ADR-039: Hidden DynamicGroup rows are retained, not deleted + +- **Status:** Accepted. No document delete operation will be added for widget registers. +- **Date:** 2026-10-23 +- **Decider:** in-app Agent program, `operator:delivery`, row `dgarch-4` +- **Supersedes nothing. Constrains:** this package's op vocabulary, and the + `ComfyUI_frontend` readers enumerated below. + +## Context + +A DynamicGroup row count is one register; each row field is its own +name-keyed register (`..`). Lowering the count +changes projection only, so the vacated rows' fields keep their registers. +[#276](https://github.com/Comfy-Org/comfy-multi-player/pull/276) does this +deliberately: a receiver-side write to clear them would have an outcome that +depends on op arrival order (KA-2, KA-4). + +Two frontend bug reports asked for the opposite. `ComfyUI_frontend` #21092 +states the expectation "after a row removal, the document holds no keys for +indexes at or above the new row count", and #21091 reports that the defaults +the frontend announces for a regrown row can overwrite a peer's concurrent +edit at the same index. Neither is expressible with existing ops: `set_widget` +with `null` stores a null *value*, and only `set_node_field` treats `null` as a +key delete. + +So the choice was: retain hidden keys and correct every reader, or add a +delete operation to the document contract. + +## Decision + +**Retain hidden keys. Do not add a widget delete operation.** Shrinking a +DynamicGroup is a visibility change, not a data deletion. Absence of a register +means "show the template default", resolved at read time by `groupLayout`, with +no writer involved. + +Three consequences are part of the decision, not implementation detail: + +1. **#21092's stated expectation is rejected.** The document legitimately holds + keys for indexes at or above the current row count. A reader that treats a + key's presence as proof the row exists is the defect. +2. **A regrown row is a restore, not a creation.** Its value is the retained + register where one exists, and the template default where none does. The + frontend must therefore stop announcing defaults for added rows and read the + retained register instead. +3. **The row-change contract needs no structural operation.** Whole-group + replacement — the producer writes the count and every visible row field in + one batch — is sufficient *because* keys are retained: it makes every + visible row's register authoritative and leaves hidden rows intact for + regrow. + +### Why not a delete operation + +A delete op can be made to converge: remove the key from `widgets`, keep its +stamp in `__stamps` so a later-arriving lower-stamped write cannot resurrect +the value. `set_node_field`'s `null` clear (ADR-032) is the in-repo precedent. +It is rejected anyway, on four grounds. + +- **Its intent is inferred, not received.** No user or producer ever expresses + "delete this field". The producer observes that a count went down and derives + a deletion from it. That is re-deriving intent from an end state, which the + in-app Agent program's invariant list forecloses by name (its FORECLOSE entry + 13; this package's register stops at FC-10 and has no equivalent id, FC-8 + being the narrower replay case). The count already carries the real intent + losslessly, so nothing has to be inferred. +- **It makes two orthogonal intents contend.** Today a peer hiding row 2 and a + peer editing `loras.2.strength` write different registers, so both survive + and regrow shows the edit. With a delete they share one register, and which + user's intent survives is decided by whichever Lamport stamp happened to be + higher. Convergent, and still data loss chosen by a timestamp race. + #276's `keeps named values coherent when a row edit races with hiding that + row` is the test that would have to be inverted to assert this. +- **It makes shrink/regrow lossy**, contradicting the dynamic-combo precedent + (`dynamic-combo-read-time-defaults`) and what a user who shrinks a group and + grows it back expects. +- **It does not pay for itself.** Of the eleven frontend readers enumerated + below, three need a change under retention. A delete op dissolves those three + and adds a new op kind, a mixed-version admission question against released + reader 0.3.10, and tombstone-stamp semantics — for a problem whose measured + symptoms are diagnostic noise and one row-ceiling comment. + +`SCHEMA_VERSION` stays 5: nothing about the document layout changes, because +nothing changes. + +### What retention requires of `ComfyUI_frontend` + +Retained keys reach the frontend through three doors. The set is bounded and +closed; this is the enumeration that made retention viable. Measured at +`ComfyUI_frontend` `479cecf9ac`. + +| Reader | Door | Verdict | +|---|---|---| +| `changed()` — `liveGraphApplier.ts:1551` | raw doc map | **Must change.** It widens a partial frame only for overflow aliases, not for a changed controller's row fields, so a count-only frame regrows rows at template defaults and never applies the retained registers. | +| `reportMissingWidgets()` — `liveGraphApplier.ts:1176` via `syncWidgets` | raw doc map | **Must change.** A hidden-row key resolves to no widget and emits `agent_graph_widget_missing` on every full or catch-up frame after a shrink. These are false positives in a signal N1 alert thresholds are being built on. | +| `announceAddedValues()` — `dynamicGroupWidget.ts:366`, `announceValue()` — `widgetValueStore.ts:358` | producer | **Must change.** This is #21091's root cause. Under retention the regrown row's value is already in the document, so announcing a default is both unnecessary and a race. The shift announce from #21057 stays — a shift is a real content change. | +| `getRestoredWidgetValueCount()` — `widgetValueStore.ts:168` | `widgets_values_named` | **Audit.** Counts leftover keys, inflating `validateSavedRowCount`'s ceiling. Low severity, same family as the row ceiling below. | +| `nodeHistoryWidgetValues.ts:86` | `widgets_values_named` | **Cosmetic.** Hidden-row keys surface in node history and diffs. | +| `documentedRowCount()` — `liveGraphApplier.ts:509` | raw doc map | Correct as written. It counts documented rows, which under retention are exactly the rows a regrow may legitimately restore (#21264's intent). Wants a comment naming the model, not a behaviour change. | +| `mountedWidgetValues()` — `liveGraphApplier.ts:549` | raw doc map | Correct as written: creation passes `reportMissing: false`, so leftovers are dropped silently. | +| `positionalWidgetValues()` — `liveGraphApplier.ts:570` | raw doc map | Safe by construction — iterates live widgets, not document keys. | +| `LGraphNode.configure` named restore — `LGraphNode.ts:280` | `widgets_values_named` | Safe by construction — indexes by live widget name. | +| `SubgraphNode.ts:497` promoted named restore | `widgets_values_named` | Safe, same shape. | +| `templateInputService.ts:39` | `widgets_values_named` | Safe, indexes by widget name. | + +**The second door is not obvious and was nearly missed.** Hidden-row keys are +absent from `widgets_values` but **present in `widgets_values_named`**, because +A23 keeps that passthrough register coherent on every `set_widget` and it is +not derived from the group layout. Measured on #276 at `8283096`: after +shrinking the populated fixture from 2 rows to 1, `widgets_values` is +`["head",1,"A.safetensors",1,true,"tail"]` while `widgets_values_named` still +carries `loras.1.lora_name`, `loras.1.strength` and `loras.1.enabled`. So +"readers that bypass `project()`" is not the boundary; readers of the named +projection see them too. + +## Consequences + +- Shrink then regrow restores the user's values, on the canvas and in the + document, without a receiver-side write. +- #21092 closes as not-a-bug on its stated expectation, with the row-ceiling + comment and the diagnostic classification as the real follow-ups. +- #21091 closes by deleting the announce, not by write-if-absent semantics. + Write-if-absent is permanently unsafe under retention, because a retained + leftover and a newer peer edit are indistinguishable at the write site — the + reason it was proposed is the reason it cannot work here. +- The row-change contract may rely on whole-group replacement. No row-level + structural operation is required by deletion semantics. +- One test is owed before #276 merges: see below. + +## Tests required before approval + +Four cases gate this ruling. Three exist and pass at #276 head `8283096` +(`npx vitest run test/dynamic-group.test.ts`, 61 passed): + +1. **shrink/regrow** — `keeps hidden row values when the count changes`. +2. **out-of-order arrival** — `converges when the row count and a row value + arrive in either order`, both orders from one seeded snapshot. +3. **defaults versus peer edit** — `keeps named values coherent when a row edit + races with hiding that row`, both orders. +4. **retry** — asserted for row-field and trailing edits by `edits %s without + shifting its siblings`, but **not for the count op itself**. The behaviour is + correct (measured: a replayed count op reports `no-op` and leaves + `encodeStateAsUpdate` byte-identical); only the assertion is missing. Add to + `test/dynamic-group.test.ts`: + +```ts +it("replays a row count op as a byte-identical no-op", () => { + const doc = mint(cases.populated, catalog); + const shrink = edit("loras", 0); + expect(applyOps(doc, [shrink], catalog).outcomes[0]?.outcome).toBe("applied"); + const grow = edit("loras", 2, 2); + expect(applyOps(doc, [grow], catalog).outcomes[0]?.outcome).toBe("applied"); + const before = Y.encodeStateAsUpdate(doc); + const projected = project(doc, catalog).nodes[0]!.widgets_values; + for (const op of [shrink, grow]) { + expect(applyOps(doc, [op], catalog).outcomes[0]?.outcome).toBe("no-op"); + } + expect(Y.encodeStateAsUpdate(doc)).toEqual(before); + expect(project(doc, catalog).nodes[0]!.widgets_values).toEqual(projected); + doc.destroy(); +}); +``` + +Case 3 exists at this package's layer only. #21091's race lives in the +frontend producer, which this package cannot see, so a frontend case covering +concurrent same-index adds plus a field edit is owed with the `announceAddedValues` +removal — not here. + +## Glossary + +- **CRDT:** conflict-free replicated data type; the shared Yjs document here. +- **LWW:** last-writer-wins ordering by the operation's immutable stamp. +- **Hidden row:** a row whose index is at or above the current count, whose + field registers remain in the document and are absent from the projection. +- **KA-2, KA-4:** the ordering-key-rides-inside-the-op and + deterministic-idempotent-applier invariants in `docs/INVARIANTS.md`. +- **Never re-derive intent from end states:** the in-app Agent program's + FORECLOSE entry 13. Not registered here; this package's foreclose ids end at + FC-10. From f8f4c4e111f89df78fb343715d5641577c0667a7 Mon Sep 17 00:00:00 2001 From: bymyself Date: Mon, 26 Oct 2026 20:50:49 +0000 Subject: [PATCH 2/3] docs: distinguish DynamicGroup restore from local add --- ...mic-group-rows-are-retained-not-deleted.md | 58 +++++++++++-------- 1 file changed, 33 insertions(+), 25 deletions(-) diff --git a/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md b/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md index 3dd4055..796e653 100644 --- a/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md +++ b/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md @@ -13,7 +13,10 @@ name-keyed register (`..`). Lowering the count changes projection only, so the vacated rows' fields keep their registers. [#276](https://github.com/Comfy-Org/comfy-multi-player/pull/276) does this deliberately: a receiver-side write to clear them would have an outcome that -depends on op arrival order (KA-2, KA-4). +depends on what that receiver has observed. A receiver that sees the count +change before a concurrent field write derives a different clear op-set from +one that sees the field write first. The clear is therefore new producer intent, +not a deterministic interpretation of the count operation. Two frontend bug reports asked for the opposite. `ComfyUI_frontend` #21092 states the expectation "after a row removal, the document holds no keys for @@ -38,10 +41,12 @@ Three consequences are part of the decision, not implementation detail: 1. **#21092's stated expectation is rejected.** The document legitimately holds keys for indexes at or above the current row count. A reader that treats a key's presence as proof the row exists is the defect. -2. **A regrown row is a restore, not a creation.** Its value is the retained - register where one exists, and the template default where none does. The - frontend must therefore stop announcing defaults for added rows and read the - retained register instead. +2. **A count-only regrow is a restore; an explicit local add is a creation.** A + remote, load, undo or redo count change restores the retained register where + one exists and the template default where none does. A user's local **Add** + action creates a default-valued row and announces those defaults as ordinary + `set_widget` intent. This distinction is carried by the producer provenance, + not inferred by a receiver from the count. 3. **The row-change contract needs no structural operation.** Whole-group replacement — the producer writes the count and every visible row field in one batch — is sufficient *because* keys are retained: it makes every @@ -91,7 +96,7 @@ closed; this is the enumeration that made retention viable. Measured at |---|---|---| | `changed()` — `liveGraphApplier.ts:1551` | raw doc map | **Must change.** It widens a partial frame only for overflow aliases, not for a changed controller's row fields, so a count-only frame regrows rows at template defaults and never applies the retained registers. | | `reportMissingWidgets()` — `liveGraphApplier.ts:1176` via `syncWidgets` | raw doc map | **Must change.** A hidden-row key resolves to no widget and emits `agent_graph_widget_missing` on every full or catch-up frame after a shrink. These are false positives in a signal N1 alert thresholds are being built on. | -| `announceAddedValues()` — `dynamicGroupWidget.ts:366`, `announceValue()` — `widgetValueStore.ts:358` | producer | **Must change.** This is #21091's root cause. Under retention the regrown row's value is already in the document, so announcing a default is both unnecessary and a race. The shift announce from #21057 stays — a shift is a real content change. | +| `announceAddedValues()` — `dynamicGroupWidget.ts:366`, `announceValue()` — `widgetValueStore.ts:358` | producer | **Keep.** A local Add is received intent to create a default-valued row. Removing the announce reintroduces #20914: remove-last/add resurrects the removed values, remove-middle/add can produce `[A, C, C]`, and bound and unbound workflows give different results. #21091's same-index add-versus-edit race is accepted as a V1 positional-model residual. The shift announce from #21057 also stays because a shift is a real content change. | | `getRestoredWidgetValueCount()` — `widgetValueStore.ts:168` | `widgets_values_named` | **Audit.** Counts leftover keys, inflating `validateSavedRowCount`'s ceiling. Low severity, same family as the row ceiling below. | | `nodeHistoryWidgetValues.ts:86` | `widgets_values_named` | **Cosmetic.** Hidden-row keys surface in node history and diffs. | | `documentedRowCount()` — `liveGraphApplier.ts:509` | raw doc map | Correct as written. It counts documented rows, which under retention are exactly the rows a regrow may legitimately restore (#21264's intent). Wants a comment naming the model, not a behaviour change. | @@ -113,19 +118,22 @@ projection see them too. ## Consequences -- Shrink then regrow restores the user's values, on the canvas and in the - document, without a receiver-side write. +- A count-only shrink then regrow restores the user's values, on the canvas and + in the document, without a receiver-side write. A local Add after a shrink + creates a default-valued row instead. - #21092 closes as not-a-bug on its stated expectation, with the row-ceiling comment and the diagnostic classification as the real follow-ups. -- #21091 closes by deleting the announce, not by write-if-absent semantics. - Write-if-absent is permanently unsafe under retention, because a retained - leftover and a newer peer edit are indistinguishable at the write site — the - reason it was proposed is the reason it cannot work here. +- #21091 remains a stated V1 conflict-policy residual. Two peers adding the + same positional index while one edits a field can lose one intent under LWW, + but every replica converges. Solving that conflict without regressing #20914 + requires stable row identity or an equivalent creation token, not + write-if-absent: a retained leftover and a newer peer edit are + indistinguishable at the write site. - The row-change contract may rely on whole-group replacement. No row-level structural operation is required by deletion semantics. -- One test is owed before #276 merges: see below. +- One package test remains as a follow-up after #276 merged: see below. -## Tests required before approval +## Tests and evidence Four cases gate this ruling. Three exist and pass at #276 head `8283096` (`npx vitest run test/dynamic-group.test.ts`, 61 passed): @@ -135,11 +143,11 @@ Four cases gate this ruling. Three exist and pass at #276 head `8283096` arrive in either order`, both orders from one seeded snapshot. 3. **defaults versus peer edit** — `keeps named values coherent when a row edit races with hiding that row`, both orders. -4. **retry** — asserted for row-field and trailing edits by `edits %s without - shifting its siblings`, but **not for the count op itself**. The behaviour is - correct (measured: a replayed count op reports `no-op` and leaves - `encodeStateAsUpdate` byte-identical); only the assertion is missing. Add to - `test/dynamic-group.test.ts`: +4. **retry follow-up** — asserted for row-field and trailing edits by `edits %s + without shifting its siblings`, but **not for the count op itself**. #276 + merged before this assertion landed. The behaviour is correct (measured: a + replayed count op reports `no-op` and leaves `encodeStateAsUpdate` + byte-identical); a follow-up must add this to `test/dynamic-group.test.ts`: ```ts it("replays a row count op as a byte-identical no-op", () => { @@ -148,21 +156,21 @@ it("replays a row count op as a byte-identical no-op", () => { expect(applyOps(doc, [shrink], catalog).outcomes[0]?.outcome).toBe("applied"); const grow = edit("loras", 2, 2); expect(applyOps(doc, [grow], catalog).outcomes[0]?.outcome).toBe("applied"); - const before = Y.encodeStateAsUpdate(doc); const projected = project(doc, catalog).nodes[0]!.widgets_values; for (const op of [shrink, grow]) { + const before = Y.encodeStateAsUpdate(doc); expect(applyOps(doc, [op], catalog).outcomes[0]?.outcome).toBe("no-op"); + expect(Y.encodeStateAsUpdate(doc)).toEqual(before); } - expect(Y.encodeStateAsUpdate(doc)).toEqual(before); expect(project(doc, catalog).nodes[0]!.widgets_values).toEqual(projected); doc.destroy(); }); ``` -Case 3 exists at this package's layer only. #21091's race lives in the -frontend producer, which this package cannot see, so a frontend case covering -concurrent same-index adds plus a field edit is owed with the `announceAddedValues` -removal — not here. +The user-visible creation semantics are held in the frontend by #20992's +remove-then-add regression: a local Add writes template defaults, while remote, +load and unbound creation mint no extra operation. #21091's race lives in that +producer and is deliberately not claimed as solved here. ## Glossary From 11921d722a83f11aa94217157297bb2b92c98445 Mon Sep 17 00:00:00 2001 From: bymyself Date: Tue, 27 Oct 2026 01:13:29 +0000 Subject: [PATCH 3/3] docs: qualify DynamicGroup undo/redo restore as snapshot restoration --- ...n-dynamic-group-rows-are-retained-not-deleted.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md b/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md index 796e653..593ee90 100644 --- a/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md +++ b/docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md @@ -42,11 +42,14 @@ Three consequences are part of the decision, not implementation detail: keys for indexes at or above the current row count. A reader that treats a key's presence as proof the row exists is the defect. 2. **A count-only regrow is a restore; an explicit local add is a creation.** A - remote, load, undo or redo count change restores the retained register where - one exists and the template default where none does. A user's local **Add** - action creates a default-valued row and announces those defaults as ordinary - `set_widget` intent. This distinction is carried by the producer provenance, - not inferred by a receiver from the count. + remote or load count change, or undo/redo of a count-only transition, + restores the retained register where one exists and the template default + where none does. A user's local **Add** action creates a default-valued row + and announces those defaults as ordinary `set_widget` intent. The frontend's + undo/redo restores serialized snapshots rather than replaying a count + operation, so undo/redo across a local Add restores the defaults that Add + wrote. This distinction is carried by the producer provenance, not inferred + by a receiver from the count. 3. **The row-change contract needs no structural operation.** Whole-group replacement — the producer writes the count and every visible row field in one batch — is sufficient *because* keys are retained: it makes every