Repository navigation
docs: ADR-039 — hidden DynamicGroup rows are retained, not deleted #292
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
e052578
docs: rule that hidden DynamicGroup rows are retained, not deleted (A…
christian-byrne f8f4c4e
docs: distinguish DynamicGroup restore from local add
christian-byrne 11921d7
docs: qualify DynamicGroup undo/redo restore as snapshot restoration
christian-byrne File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
188 changes: 188 additions & 0 deletions
188
docs/decisions/ADR-039-hidden-dynamic-group-rows-are-retained-not-deleted.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,188 @@ | ||
| # ADR-039: Hidden DynamicGroup rows are retained, not deleted | ||
|
|
||
| - **Status:** Accepted. No document delete operation will be added for widget registers. | ||
| - **Date:** 2026-10-23 | ||
| - **Decider:** in-app Agent program, `operator:delivery`, row `dgarch-4` | ||
| - **Supersedes nothing. Constrains:** this package's op vocabulary, and the | ||
| `ComfyUI_frontend` readers enumerated below. | ||
|
|
||
| ## Context | ||
|
|
||
| A DynamicGroup row count is one register; each row field is its own | ||
| name-keyed register (`<controller>.<index>.<field>`). Lowering the count | ||
| changes projection only, so the vacated rows' fields keep their registers. | ||
| [#276](https://github.com/Comfy-Org/comfy-multi-player/pull/276) does this | ||
| deliberately: a receiver-side write to clear them would have an outcome that | ||
| depends on what that receiver has observed. A receiver that sees the count | ||
| change before a concurrent field write derives a different clear op-set from | ||
| one that sees the field write first. The clear is therefore new producer intent, | ||
| not a deterministic interpretation of the count operation. | ||
|
|
||
| Two frontend bug reports asked for the opposite. `ComfyUI_frontend` #21092 | ||
| states the expectation "after a row removal, the document holds no keys for | ||
| indexes at or above the new row count", and #21091 reports that the defaults | ||
| the frontend announces for a regrown row can overwrite a peer's concurrent | ||
| edit at the same index. Neither is expressible with existing ops: `set_widget` | ||
| with `null` stores a null *value*, and only `set_node_field` treats `null` as a | ||
| key delete. | ||
|
|
||
| So the choice was: retain hidden keys and correct every reader, or add a | ||
| delete operation to the document contract. | ||
|
|
||
| ## Decision | ||
|
|
||
| **Retain hidden keys. Do not add a widget delete operation.** Shrinking a | ||
| DynamicGroup is a visibility change, not a data deletion. Absence of a register | ||
| means "show the template default", resolved at read time by `groupLayout`, with | ||
| no writer involved. | ||
|
|
||
| Three consequences are part of the decision, not implementation detail: | ||
|
|
||
| 1. **#21092's stated expectation is rejected.** The document legitimately holds | ||
| keys for indexes at or above the current row count. A reader that treats a | ||
| key's presence as proof the row exists is the defect. | ||
| 2. **A count-only regrow is a restore; an explicit local add is a creation.** A | ||
| remote or load count change, or undo/redo of a count-only transition, | ||
| restores the retained register where one exists and the template default | ||
| where none does. A user's local **Add** action creates a default-valued row | ||
| and announces those defaults as ordinary `set_widget` intent. The frontend's | ||
| undo/redo restores serialized snapshots rather than replaying a count | ||
| operation, so undo/redo across a local Add restores the defaults that Add | ||
| wrote. This distinction is carried by the producer provenance, not inferred | ||
| by a receiver from the count. | ||
| 3. **The row-change contract needs no structural operation.** Whole-group | ||
| replacement — the producer writes the count and every visible row field in | ||
| one batch — is sufficient *because* keys are retained: it makes every | ||
| visible row's register authoritative and leaves hidden rows intact for | ||
| regrow. | ||
|
|
||
| ### Why not a delete operation | ||
|
|
||
| A delete op can be made to converge: remove the key from `widgets`, keep its | ||
| stamp in `__stamps` so a later-arriving lower-stamped write cannot resurrect | ||
| the value. `set_node_field`'s `null` clear (ADR-032) is the in-repo precedent. | ||
| It is rejected anyway, on four grounds. | ||
|
|
||
| - **Its intent is inferred, not received.** No user or producer ever expresses | ||
| "delete this field". The producer observes that a count went down and derives | ||
| a deletion from it. That is re-deriving intent from an end state, which the | ||
| in-app Agent program's invariant list forecloses by name (its FORECLOSE entry | ||
| 13; this package's register stops at FC-10 and has no equivalent id, FC-8 | ||
| being the narrower replay case). The count already carries the real intent | ||
| losslessly, so nothing has to be inferred. | ||
| - **It makes two orthogonal intents contend.** Today a peer hiding row 2 and a | ||
| peer editing `loras.2.strength` write different registers, so both survive | ||
| and regrow shows the edit. With a delete they share one register, and which | ||
| user's intent survives is decided by whichever Lamport stamp happened to be | ||
| higher. Convergent, and still data loss chosen by a timestamp race. | ||
| #276's `keeps named values coherent when a row edit races with hiding that | ||
| row` is the test that would have to be inverted to assert this. | ||
| - **It makes shrink/regrow lossy**, contradicting the dynamic-combo precedent | ||
| (`dynamic-combo-read-time-defaults`) and what a user who shrinks a group and | ||
| grows it back expects. | ||
| - **It does not pay for itself.** Of the eleven frontend readers enumerated | ||
| below, three need a change under retention. A delete op dissolves those three | ||
| and adds a new op kind, a mixed-version admission question against released | ||
| reader 0.3.10, and tombstone-stamp semantics — for a problem whose measured | ||
| symptoms are diagnostic noise and one row-ceiling comment. | ||
|
|
||
| `SCHEMA_VERSION` stays 5: nothing about the document layout changes, because | ||
| nothing changes. | ||
|
|
||
| ### What retention requires of `ComfyUI_frontend` | ||
|
|
||
| Retained keys reach the frontend through three doors. The set is bounded and | ||
| closed; this is the enumeration that made retention viable. Measured at | ||
| `ComfyUI_frontend` `479cecf9ac`. | ||
|
|
||
| | Reader | Door | Verdict | | ||
| |---|---|---| | ||
| | `changed()` — `liveGraphApplier.ts:1551` | raw doc map | **Must change.** It widens a partial frame only for overflow aliases, not for a changed controller's row fields, so a count-only frame regrows rows at template defaults and never applies the retained registers. | | ||
| | `reportMissingWidgets()` — `liveGraphApplier.ts:1176` via `syncWidgets` | raw doc map | **Must change.** A hidden-row key resolves to no widget and emits `agent_graph_widget_missing` on every full or catch-up frame after a shrink. These are false positives in a signal N1 alert thresholds are being built on. | | ||
| | `announceAddedValues()` — `dynamicGroupWidget.ts:366`, `announceValue()` — `widgetValueStore.ts:358` | producer | **Keep.** A local Add is received intent to create a default-valued row. Removing the announce reintroduces #20914: remove-last/add resurrects the removed values, remove-middle/add can produce `[A, C, C]`, and bound and unbound workflows give different results. #21091's same-index add-versus-edit race is accepted as a V1 positional-model residual. The shift announce from #21057 also stays because a shift is a real content change. | | ||
| | `getRestoredWidgetValueCount()` — `widgetValueStore.ts:168` | `widgets_values_named` | **Audit.** Counts leftover keys, inflating `validateSavedRowCount`'s ceiling. Low severity, same family as the row ceiling below. | | ||
| | `nodeHistoryWidgetValues.ts:86` | `widgets_values_named` | **Cosmetic.** Hidden-row keys surface in node history and diffs. | | ||
| | `documentedRowCount()` — `liveGraphApplier.ts:509` | raw doc map | Correct as written. It counts documented rows, which under retention are exactly the rows a regrow may legitimately restore (#21264's intent). Wants a comment naming the model, not a behaviour change. | | ||
| | `mountedWidgetValues()` — `liveGraphApplier.ts:549` | raw doc map | Correct as written: creation passes `reportMissing: false`, so leftovers are dropped silently. | | ||
| | `positionalWidgetValues()` — `liveGraphApplier.ts:570` | raw doc map | Safe by construction — iterates live widgets, not document keys. | | ||
| | `LGraphNode.configure` named restore — `LGraphNode.ts:280` | `widgets_values_named` | Safe by construction — indexes by live widget name. | | ||
| | `SubgraphNode.ts:497` promoted named restore | `widgets_values_named` | Safe, same shape. | | ||
| | `templateInputService.ts:39` | `widgets_values_named` | Safe, indexes by widget name. | | ||
|
|
||
| **The second door is not obvious and was nearly missed.** Hidden-row keys are | ||
| absent from `widgets_values` but **present in `widgets_values_named`**, because | ||
| A23 keeps that passthrough register coherent on every `set_widget` and it is | ||
| not derived from the group layout. Measured on #276 at `8283096`: after | ||
| shrinking the populated fixture from 2 rows to 1, `widgets_values` is | ||
| `["head",1,"A.safetensors",1,true,"tail"]` while `widgets_values_named` still | ||
| carries `loras.1.lora_name`, `loras.1.strength` and `loras.1.enabled`. So | ||
| "readers that bypass `project()`" is not the boundary; readers of the named | ||
| projection see them too. | ||
|
|
||
| ## Consequences | ||
|
|
||
| - A count-only shrink then regrow restores the user's values, on the canvas and | ||
| in the document, without a receiver-side write. A local Add after a shrink | ||
| creates a default-valued row instead. | ||
| - #21092 closes as not-a-bug on its stated expectation, with the row-ceiling | ||
| comment and the diagnostic classification as the real follow-ups. | ||
| - #21091 remains a stated V1 conflict-policy residual. Two peers adding the | ||
| same positional index while one edits a field can lose one intent under LWW, | ||
| but every replica converges. Solving that conflict without regressing #20914 | ||
| requires stable row identity or an equivalent creation token, not | ||
| write-if-absent: a retained leftover and a newer peer edit are | ||
| indistinguishable at the write site. | ||
| - The row-change contract may rely on whole-group replacement. No row-level | ||
| structural operation is required by deletion semantics. | ||
| - One package test remains as a follow-up after #276 merged: see below. | ||
|
|
||
| ## Tests and evidence | ||
|
|
||
| Four cases gate this ruling. Three exist and pass at #276 head `8283096` | ||
| (`npx vitest run test/dynamic-group.test.ts`, 61 passed): | ||
|
|
||
| 1. **shrink/regrow** — `keeps hidden row values when the count changes`. | ||
| 2. **out-of-order arrival** — `converges when the row count and a row value | ||
| arrive in either order`, both orders from one seeded snapshot. | ||
| 3. **defaults versus peer edit** — `keeps named values coherent when a row edit | ||
| races with hiding that row`, both orders. | ||
| 4. **retry follow-up** — asserted for row-field and trailing edits by `edits %s | ||
| without shifting its siblings`, but **not for the count op itself**. #276 | ||
| merged before this assertion landed. The behaviour is correct (measured: a | ||
| replayed count op reports `no-op` and leaves `encodeStateAsUpdate` | ||
| byte-identical); a follow-up must add this to `test/dynamic-group.test.ts`: | ||
|
|
||
| ```ts | ||
| it("replays a row count op as a byte-identical no-op", () => { | ||
| const doc = mint(cases.populated, catalog); | ||
| const shrink = edit("loras", 0); | ||
| expect(applyOps(doc, [shrink], catalog).outcomes[0]?.outcome).toBe("applied"); | ||
| const grow = edit("loras", 2, 2); | ||
| expect(applyOps(doc, [grow], catalog).outcomes[0]?.outcome).toBe("applied"); | ||
| const projected = project(doc, catalog).nodes[0]!.widgets_values; | ||
| for (const op of [shrink, grow]) { | ||
| const before = Y.encodeStateAsUpdate(doc); | ||
| expect(applyOps(doc, [op], catalog).outcomes[0]?.outcome).toBe("no-op"); | ||
| expect(Y.encodeStateAsUpdate(doc)).toEqual(before); | ||
| } | ||
| expect(project(doc, catalog).nodes[0]!.widgets_values).toEqual(projected); | ||
| doc.destroy(); | ||
| }); | ||
| ``` | ||
|
|
||
| The user-visible creation semantics are held in the frontend by #20992's | ||
| remove-then-add regression: a local Add writes template defaults, while remote, | ||
| load and unbound creation mint no extra operation. #21091's race lives in that | ||
| producer and is deliberately not claimed as solved here. | ||
|
|
||
| ## Glossary | ||
|
|
||
| - **CRDT:** conflict-free replicated data type; the shared Yjs document here. | ||
| - **LWW:** last-writer-wins ordering by the operation's immutable stamp. | ||
| - **Hidden row:** a row whose index is at or above the current count, whose | ||
| field registers remain in the document and are absent from the projection. | ||
| - **KA-2, KA-4:** the ordering-key-rides-inside-the-op and | ||
| deterministic-idempotent-applier invariants in `docs/INVARIANTS.md`. | ||
| - **Never re-derive intent from end states:** the in-app Agent program's | ||
| FORECLOSE entry 13. Not registered here; this package's foreclose ids end at | ||
| FC-10. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.