Repository navigation
docs: define CRDT sequence-gap recovery semantics #16183
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| # 24. Sequence-Gap Recovery Uses State-Vector Replay | ||
|
|
||
| Date: 2026-08-28 | ||
|
|
||
| ## Status | ||
|
|
||
| Proposed | ||
|
|
||
| ## Context | ||
|
|
||
| The In-App Agent follower receives host-authored Yjs updates and projects the shared semantic | ||
| document into frontend state. Transport frames also carry a sequence number so the follower can | ||
| detect a missed frame. A sequence gap or reconnect does not imply that the document's lineage has | ||
| changed: the follower's current Y.Doc remains a valid partial replica. | ||
|
|
||
| Yjs state vectors describe the updates a replica already contains. The follower can therefore | ||
| resubscribe with its current state vector and let the host return only the missing state-vector | ||
| delta. Destroying and replacing the Y.Doc on an ordinary gap would discard that recovery property, | ||
| cause visible canvas churn, and strand projectors or other consumers that observe the existing doc | ||
| instance. | ||
|
|
||
| This decision extends the follower boundary described by the In-App Agent CRDT architecture: raw | ||
| Yjs updates remain host to follower only, while recovery keeps and advances the same follower | ||
| document. | ||
|
|
||
| ## Decision | ||
|
|
||
| On an ordinary sequence gap or reconnect, the follower keeps its existing Y.Doc and resubscribes | ||
| with that document's state vector. The host computes and returns the missing delta. The follower | ||
| withholds the out-of-sequence frame until recovery restores a continuous stream; it does not wipe | ||
| the canvas or replace the document. | ||
|
|
||
| Destroying or replacing the follower Y.Doc is reserved exclusively for an explicit `doc_reset` | ||
| frame representing a genuine document-lineage break. Before replacement, the reset must be | ||
| dispatched to every projector and consumer that observes the old document instance so each can | ||
| drop stale references and rebuild against the new lineage. | ||
|
Comment on lines
+28
to
+36
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift 🔎 Supported by static analysis🏁 Script executed: printf '%s\n' '--- applicable repository guidance ---'
find /tmp/coderabbit-repo-knowledge/comfy-org-comfyui-frontend-1c9ba904 -type f -name '*.md' -print
printf '%s\n' '--- ADR ---'
cat -n docs/adr/0024-sequence-gap-recovery-uses-state-vector-replay.mdRepository: Comfy-Org/ComfyUI_frontend Length of output: 10182 🏁 Script executed: printf '%s\n' '--- repository conventions and learnings ---'
cat /tmp/coderabbit-repo-knowledge/comfy-org-comfyui-frontend-1c9ba904/conventions/repo-wide.md
cat /tmp/coderabbit-repo-knowledge/comfy-org-comfyui-frontend-1c9ba904/learnings/repo-wide.md
printf '%s\n' '--- related protocol references ---'
rg -n -i --glob '*.md' 'doc_reset|state.?vector|sequence gap|seq-gap|lineage|In-App Agent|ADR-011' docs . 2>/dev/null | head -240Repository: Comfy-Org/ComfyUI_frontend Length of output: 25870 Define recovery precedence for a missed If 🤖 Prompt for AI Agents
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is valid and requires a cross-boundary protocol decision: Yjs state alone cannot reveal a missed lineage reset. Recommendation: make lineage ID and any pending reset part of the resubscribe response, give reset precedence over delta replay, await consumer teardown, install the new Y.Doc, then set the cursor from the reset response before draining new-lineage frames. Keep this open until the host and follower contracts agree on that metadata.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The resubscribe response must include the active lineage ID and pending reset metadata. A pending reset must take precedence over state-vector delta replay. The follower must await consumer teardown, install the new The finding remains open until both contracts define this metadata and ordering. ✏️ Learnings added
You are interacting with an AI system.
Comment on lines
+33
to
+36
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift 🔎 Supported by static analysis🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- applicable repository conventions ---'
head -5 /tmp/coderabbit-repo-knowledge/comfy-org-comfyui-frontend-1c9ba904/*/*.md 2>/dev/null || true
printf '%s\n' '--- ADR outline and target section ---'
wc -l docs/adr/0024-sequence-gap-recovery-uses-state-vector-replay.md
sed -n '1,90p' docs/adr/0024-sequence-gap-recovery-uses-state-vector-replay.mdRepository: Comfy-Org/ComfyUI_frontend Length of output: 16797 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- related ADRs ---'
find docs/adr -maxdepth 1 -type f -print | sort
printf '%s\n' '--- direct protocol references ---'
rg -n -i --glob '!docs/adr/0024-sequence-gap-recovery-uses-state-vector-replay.md' \
'doc_reset|projector|consumer|In-App Agent CRDT|workspace ADR-011' \
docs src packages apps 2>/dev/null | head -200 || true
printf '%s\n' '--- ADR tail and referenced note ---'
sed -n '88,112p' docs/adr/0024-sequence-gap-recovery-uses-state-vector-replay.md
rg -n -i 'ADR-011|lineage|state-vector replay|sequence-gap recovery' . \
--glob '!node_modules/**' --glob '!dist/**' --glob '!build/**' 2>/dev/null | head -100 || trueRepository: Comfy-Org/ComfyUI_frontend Length of output: 33390 Define a completion barrier before replacing the Y.Doc. The 🤖 Prompt for AI Agents
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is a valid lifecycle question with competing failure policies. Recommendation: require an awaited teardown barrier and fail closed: on rejection or timeout, retain the old Y.Doc, stop applying frames, emit a recoverable diagnostic, and retry or require reconnect rather than replacing under stale consumers. Keep this open until timeout ownership, duration, and retry behavior are decided.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
If a handler rejects or times out, the follower must retain the old Y.Doc, stop applying frames, emit a recoverable diagnostic, and retry recovery or require reconnect. The finding should remain open until the ADR defines timeout ownership, timeout duration, and retry behavior. ✏️ Learnings added
You are interacting with an AI system. |
||
|
|
||
| ```mermaid | ||
| flowchart TD | ||
| frame["Follower receives frame"] --> gap{"Expected sequence?"} | ||
| gap -->|Yes| apply["Apply update to existing Y.Doc"] | ||
| gap -->|No| hold["Withhold gapped frame"] | ||
| hold --> subscribe["Resubscribe with current state vector"] | ||
| subscribe --> delta["Host returns missing delta"] | ||
| delta --> apply | ||
| reset["Explicit doc_reset"] --> notify["Notify every projector and consumer"] | ||
| notify --> replace["Replace Y.Doc and establish new lineage"] | ||
| ``` | ||
|
|
||
| Implementations and reviews must treat Y.Doc replacement outside the `doc_reset` path as a | ||
| correctness defect. The recovery path must also distinguish the sequence used for transport-gap | ||
| detection from the Yjs state vector used to calculate document content that is actually missing. | ||
|
|
||
| ## Consequences | ||
|
|
||
| ### Positive | ||
|
|
||
| - Reconnect and packet-loss recovery transfer only missing CRDT state and preserve the live canvas. | ||
| - Projectors, subscriptions, and reactive bindings keep a stable Y.Doc identity during ordinary | ||
| recovery. | ||
| - Recovery remains compatible with offline and peer-style replication because it uses native Yjs | ||
| reconciliation rather than a server-only full-document replacement primitive. | ||
| - `doc_reset` becomes an explicit, auditable lineage transition instead of an implicit side effect | ||
| of transport recovery. | ||
|
|
||
| ### Negative | ||
|
|
||
| - The host must retain enough document state to encode a delta against a follower-provided state | ||
| vector. | ||
| - The follower needs a small recovery state machine to withhold out-of-sequence frames and prevent | ||
| overlapping resubscriptions. | ||
| - Every projector and consumer must implement explicit `doc_reset` handling before document | ||
| replacement can be safe. | ||
|
|
||
| ## Alternatives considered | ||
|
|
||
| ### Wipe and refetch on a gap (rejected) | ||
|
|
||
| This is operationally simple, but it throws away the CRDT's incremental recovery capability, | ||
| causes visible canvas churn, and invalidates consumers of the existing Y.Doc without an explicit | ||
| lineage event. | ||
|
|
||
| ### Buffer all frames indefinitely (rejected) | ||
|
|
||
| An unbounded buffer creates a memory risk and cannot recover frames that were lost before they | ||
| reached the client. | ||
|
|
||
| ## Notes | ||
|
|
||
| This is the governed frontend copy of the In-App Agent program's workspace ADR-011, | ||
| "Seq-gap recovery is state-vector delta replay — never wipe or replace the follower doc." It also | ||
| codifies KEEP-ALIVE invariant #13 and FORECLOSE invariant #11 from that program. The frontend-local | ||
| number is ADR-0024 because ADR-0011 already governs derived credential lifecycle and ADR-0020 | ||
| through ADR-0023 are also assigned in this repository. | ||
|
|
||
| ## Glossary | ||
|
|
||
| - **Sequence gap**: A discontinuity in transport frame numbers indicating that one or more frames | ||
| may have been missed. | ||
| - **State vector**: Compact Yjs metadata describing the document updates a replica already has. | ||
| - **State-vector delta**: The Yjs update containing document state absent from the requesting | ||
| replica. | ||
| - **Follower**: A frontend replica that receives host-authored raw Yjs updates and does not write | ||
| directly to the shared semantic document. | ||
| - **Projector**: A consumer that translates shared document state into another frontend model or | ||
| renderable representation. | ||
| - **Document lineage**: The identity and history of a shared document. A new lineage cannot be | ||
| reconciled as an ordinary continuation of the old one. | ||
| - **`doc_reset`**: An explicit protocol frame declaring a document-lineage break and authorizing | ||
| replacement of the follower Y.Doc after consumers are notified. | ||
| - **KEEP-ALIVE #13 / FORECLOSE #11**: Cross-repository guardrails requiring state-vector replay for | ||
| ordinary recovery and prohibiting silent follower-document replacement. | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🔎 Supported by static analysis
🏁 Script executed:
Repository: Comfy-Org/ComfyUI_frontend
Length of output: 8679
🏁 Script executed:
Repository: Comfy-Org/ComfyUI_frontend
Length of output: 50382
Specify how replay restores the transport cursor.
The ADR defines state-vector replay but does not define how replay advances the transport sequence. Specify recovery metadata or an acknowledgment that identifies the recovered sequence range. Define the
< expected,== expected, and> expectedtransitions, including duplicate and late frames. Define bounded buffering for frames received during recovery. Add a test that drops one frame, delivers later frames during recovery, and then delivers a duplicate.🤖 Prompt for AI Agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a valid protocol gap, but the acknowledgment schema, recovered-range semantics, and buffer bound are not determined by the existing decision. Recommendation: define a recovery response carrying lineage ID plus an inclusive transport high-water mark; discard frames below expected, apply exactly expected, buffer above expected up to an explicit count/byte limit, and resubscribe again on overflow. Keep this open as a design gate until the host contract and test location are agreed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@christian-byrne, agreed. This remains a valid design gate.The ADR needs an agreed host recovery contract before it can define the transport cursor transitions and bounded buffering precisely. The proposed recovery response should include the lineage ID and an inclusive transport high-water mark. The implementation and test should then define duplicate, late, in-order, and buffered-frame behavior.
I will keep this thread open pending agreement on the host contract and test location.
✏️ Learnings added
You are interacting with an AI system.