From 08be0d140200e6ddfb2eab6949b2482da69a588f Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 22 Sep 2026 22:59:20 +0000 Subject: [PATCH 1/3] fix: mint genuine numeric link ids for insert_workflow (ADR-033) insert_workflow remapped every carried id -- node, link, group, and definition -- to a deterministic STRING (ADR-031). That fits NodeId (string | number) but not ComfyUI_frontend's LinkId, which is a branded number, forcing the frontend to hash the string into a numeric id locally (ComfyUI_frontend#18458). Two reviewers found that interim hash genuinely broken: it collided at a measured, reproducible rate, and the resolved id is what graph.serialize() persists, so a "reserved range" chosen above last_link_id does not survive a save/reload once a hashed id lands inside it. Fix it at the source instead: remap.ts's derivedLinkId now mints a real number for every link id insert_workflow derives (top-level and every subgraph-definition interior, at any depth), verified against every numeric link id already persisted in the target document (doc.ts's persistedLinkIds) plus every id already minted earlier in the same call, retrying past a collision (bounded, MAX_LINK_ID_MINT_ATTEMPTS = 256) rather than merely hashing into a wide space and hoping. This is a deliberate, logged deviation from KA-5's "no allocation against document state" for links only -- an information-theoretic necessity given LinkId's fixed-width numeric type -- recorded in docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md and docs/decisions/EXCEPTIONS.md's new KA-5 row. Invariants: KA-4 (byte-identical rejection preserved via bounded retry-then-reject, id-space-exhaustion test added), KA-5 (logged exception, see ADR-033), ADR-031 (unchanged for node/group/definition ids). comfy-multi-player#230's definition-interior remapLinkIdArray/promoted-linkIds handling is preserved and covered with the new numeric ids. Tests: extends test/insert-workflow.test.ts's occupiedIdVectors with avoidance/exhaustion coverage for links, updates test/ka4-rejection-byte-identity.test.ts's now-unreachable-by-natural- collision link row to the id-space-exhaustion construction, and adds test/insert-workflow-numeric-link-ids.regression.test.ts covering the exact cross-op collision ComfyUI_frontend#18458 reproduced, same-batch concurrent inserts, cross-scope avoidance (top-level vs definition-interior), scale, save/reload round-trip stability, and comfy-multi-player#230 non-regression. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01UqyourCBfi97xzpBx2W7oY --- docs/INVARIANTS.md | 13 + ...DR-033-insert-workflow-numeric-link-ids.md | 124 +++++ docs/decisions/EXCEPTIONS.md | 1 + src/applier.ts | 30 +- src/doc.ts | 42 ++ src/remap.ts | 99 +++- ...rkflow-numeric-link-ids.regression.test.ts | 433 ++++++++++++++++++ test/insert-workflow.test.ts | 219 +++++++-- test/ka4-rejection-byte-identity.test.ts | 43 +- 9 files changed, 955 insertions(+), 49 deletions(-) create mode 100644 docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md create mode 100644 test/insert-workflow-numeric-link-ids.regression.test.ts diff --git a/docs/INVARIANTS.md b/docs/INVARIANTS.md index d73106f..510656c 100644 --- a/docs/INVARIANTS.md +++ b/docs/INVARIANTS.md @@ -43,6 +43,19 @@ The *rejection* half — "a rejected op leaves the doc untouched", stated in the **Why:** Offline peers must create entities without a central allocator. **Enforced by:** `insert_workflow` derives every carried ID inside the applier from the immutable `op_id`, scope, kind, and raw ID (ADR-031); producers send raw workflows and perform no allocation against document state. General producer randomness remains **UNGUARDED — see roadmap**. +**Logged exception (2026-09-22, ADR-033):** a link's derived id is the ONE +carried-ID kind this rule's "no allocation against document state" clause +does not hold for. ComfyUI_frontend's `LinkId` is a branded `number`, unlike +the string-or-number `NodeId`, so `insert_workflow`'s usual pure string +derivation cannot satisfy it; `remap.ts`'s `derivedLinkId` mints a 52-bit +hash candidate and verifies it against `doc.ts`'s `persistedLinkIds` (every +numeric link id already in the document, top-level and every subgraph +definition interior) plus every id already minted in the same call, retrying +past a collision up to a bounded attempt cap before failing loudly. See +`docs/decisions/EXCEPTIONS.md`'s KA-5 row for the full reasoning and the +residual, deliberately-accepted gap for replicas that have never exchanged +the op in question. + ### KA-6 — Raw struct updates flow host → follower one-way only **Rule:** Followers never write the shared doc. **Why:** Reverse-flow Yjs structs from independently edited docs can corrupt or diverge state. diff --git a/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md b/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md new file mode 100644 index 0000000..d0c7518 --- /dev/null +++ b/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md @@ -0,0 +1,124 @@ +# ADR-033: numeric, document-state-verified link ids for `insert_workflow` + +- **Status:** Accepted +- **Date:** 2026-09-22 +- **Decider:** Christian Byrne +- **Source:** ComfyUI_frontend#18458 (interim 32-bit-hash review) and the Slack thread it was raised in + +## Context + +ADR-031 remaps every id `insert_workflow` carries — node, link, group, and +nested-definition id — to a deterministic **string** derived from the op +envelope id, graph scope, id kind, and original id +(`insert:::link:`). That is a correct fit for a +`NodeId` (`string | number` on the frontend) but not for a `LinkId`, which +ComfyUI_frontend brands as `number & { __brand: 'LinkId' }`. The frontend's +interim fix (`ecsFollowerAdapter.ts`'s `resolveLinkId`) hashed the derived +string into a number with FNV-1a folded into a 32-bit space. Two independent +reviewers (Christian Byrne, dante01yoon) found this genuinely broken, not +merely approximate: + +- The 32-bit hash space is small enough that two distinct derived ids collide + at a measured, reproducible rate (~0.01% at 1k live derived links, ~1% at + 10k, birthday-bound), silently replacing one link with another with no + error reported — indistinguishable from the original "links silently + dropped" bug this was fixing. +- The resolved numeric id is what `graph.serialize()` writes to the saved + workflow file. A reserved id range chosen to sit above any real + `last_link_id` does not survive a save/reload: the persisted file now + contains a genuine numeric link id inside that "reserved" range, and + nothing bounds real link ids below it, so a LATER `insert_workflow` can + collide with a real, previously-synthetic-now-real id. A session-local + allocator has the same reload hole from the opposite direction: a fresh + session's reservation map is empty and does not know the reloaded id is + taken. + +Both reviewers converged on the same root fix: mint the numeric identity +where the link is minted (this package), not on every frontend peer that +happens to read it. + +## Decision + +`remap.ts`'s `derivedLinkId` mints a real JS `number` for every link id +`insert_workflow` derives — at the top-level graph scope AND at every +subgraph-definition interior scope, so the fix does not regress +comfy-multi-player#230's `remapLinkIdArray`/definition-interior handling. +Every other kind `insert_workflow` derives (node, group, definition) is +UNCHANGED: still the ADR-031 string, still a pure function of +(`opId`, `scope`, `kind`, `original`) with no document-state dependency. + +The link-id mint is NOT a wider hash gambled to be "unlikely enough" — the +same failure mode being fixed. It is: + +1. **Hashed**: SHA-256 of the same `insert:::link:` + seed ADR-031 already uses, keeping 52 bits (well inside + `Number.MAX_SAFE_INTEGER`) as the first candidate. +2. **Verified against every id the caller can see**, not assumed collision-free + by width alone: the caller (`applier.ts`) passes `derivedLinkId` (via + `remapInsertedWorkflowIds`'s `reservedLinkIds` parameter) every numeric + link id already persisted in the target document — top-level AND every + subgraph definition's interior links, at every nesting depth + (`doc.ts`'s `persistedLinkIds`) — plus every id `derivedLinkId` has already + minted earlier in the SAME `insert_workflow` call, so two links in one + insertion can never collide with each other either. +3. **Retried, bounded**: on a collision the mint deterministically tries the + next candidate (re-hashing with an incremented attempt counter) up to + `MAX_LINK_ID_MINT_ATTEMPTS` (256) times. Any realistic document — even one + with millions of live links — succeeds on the first or second attempt; + the bound exists so a pathological/adversarial reservation set fails + loudly (`link_id_collision`, byte-identical, `op_id` not consumed) in + bounded work rather than looping. + +This directly fixes the reviewers' reproduction: two independent +`insert_workflow` ops (distinct `op_id`s) inserting links that would +otherwise hash to the same value now resolve to distinct ids, because the +second op's mint sees the first op's already-committed numeric id in +`persistedLinkIds` and retries past it — the exact cross-op collision the +32-bit interim hash reproduced. + +## Deviation from KA-5 / ADR-031 + +Every other id `insert_workflow` derives is intentionally **pure**: a +function of the op's own fields alone, with producers/the applier never +inspecting document state to decide an id (KA-5's "IDs are collision-free +without coordination" and ADR-031's "Producers do not inspect document +state or remap ids"). `derivedLinkId` breaks that purity for links, and only +for links: a truly information-theoretic collision-free encoding of an +unbounded set of possible (`opId`, link) pairs into a fixed 53-bit numeric +space is impossible by construction (pigeonhole), unlike the string form, +whose output space is unbounded and therefore never needs to consult +anything besides the op itself. Squeezing that identity into a small, +externally-imposed numeric type (`LinkId`) is only possible by ALSO checking +it against real state — so this package does, rather than pretending a wide +hash is "collision-free" the way the interim frontend fix did. This is +logged as a deliberate exception in `docs/decisions/EXCEPTIONS.md` (KA-5 row), +with the residual gap named explicitly: two REPLICAS that have never +exchanged this op and each independently mint a colliding candidate from +their own (different) locally-visible document state could in principle +diverge — bounded by the same ~2^-52-per-pair probability that makes a UUID4 +`op_id` collision (already a foundational assumption throughout this +package, KA-2) an accepted risk elsewhere in this same system. + +## Consequences + +- ComfyUI_frontend can delete its `resolveLinkId`/`fnv1a` hashing entirely + (tracked in ComfyUI_frontend#18458) and read `LinkId` straight off the doc. +- `link_id_collision`'s existing insert-time collision check + (`applier.ts`'s `acceptsLink`) is no longer reachable via any + realistically-shaped input for LINKS specifically, because the mint now + actively avoids everything that check would have caught. It remains + reachable, and is retained, via the id-space-exhaustion path above; the + test suite covers exhaustion directly rather than asserting an + unreachable natural collision. +- `updateInsertedWorkflowMeta`'s `last_link_id` high-water-mark bookkeeping + deliberately does NOT fold in these new large numeric ids (see the comment + at its call site) — they are not meant to look like the next sequential + id, and KA-5 already treats this field as advisory only. +- `SCHEMA_VERSION` is unchanged: no document root or reserved key changes, + only the VALUES `insert_workflow` was already free to choose for a link id. + +## Glossary + +- **Derived id:** ADR-031's deterministic string, `insert::::`. +- **Candidate:** one hashed, not-yet-verified numeric value `derivedLinkId` tries during a mint. +- **Reservation set:** the ids a mint call must not choose — persisted document ids plus ids already minted in the same call. diff --git a/docs/decisions/EXCEPTIONS.md b/docs/decisions/EXCEPTIONS.md index 567e2f8..7715247 100644 --- a/docs/decisions/EXCEPTIONS.md +++ b/docs/decisions/EXCEPTIONS.md @@ -21,3 +21,4 @@ is retained as history, not authorization to accept unprojectable writes. | 2026-08-21 (PROPOSED) | KA-11 | **The snapshot read surface's fail-closed gate is CONDITIONAL**: `src/read.ts` refuses a document carrying content under an unreadable schema, but returns its empty value for a document carrying nothing, where `project()` refuses that document too. Deliberate — ADR-004's follower reads a root-less document before its first frame and there is no content there to mis-key — but it is a read path that answers where a sibling read path throws, so it is recorded rather than left in source comments. Normative text: Amendment A12. The divergence is disposition-only; both paths use `readSchemaVersion` as the predicate. Mitigated by asking "carries content" against the struct store rather than the v1 root names, so a v2 that renamed its roots is refused rather than read as empty. | `src/read.ts` `assertSnapshotReadable`/`carriesContent`; schema §10 + Amendment A12 | Owner: UNASSIGNED. Sunset PROPOSED, not agreed: revisit when `SCHEMA_VERSION` first moves past 1, since a real v2 makes the disposition split observable to a deployed follower for the first time. | | ~~2026-08-27 (PROPOSED)~~ **RESOLVED 2026-08-28** | KA-3 | ~~**A promoted `connect` (`grow.promoted: true`, schema Amendment A15) is LWW-gated here on `("input", to_node, "grow", name)` and moves the materialized slot's `grow_id` to the winning link; comfy-cli's `_apply_connect` gates neither.**~~ **Sunset condition met: comfy-cli PR #818 (amendment v1.5, `ba0b0b92abcc86b01e8a6704d07088f92afe7aa7`) gates the same register, spelled with the FULL declared name on both sides, and moves `grow_id` to the winner.** Struck rather than deleted so the exception's life stays legible. Both spell the register identically (`_write_target`), so the divergence is in GATING only and is observable only under concurrency comfy-cli's single-writer apply never sees — the corpus session it generated replays green. Gating is required for §2.5 convergence (two connects into one declared input contend for one slot exactly as two concrete connects do, A1). | `src/applier.ts` `claimPromotedInput`; schema Amendment A15 | Owner: UNASSIGNED. Sunset PROPOSED, not agreed: comfy-cli `_lww_gate`s promoted grows on the same register (asked for in the PR that lands A15). | | 2026-09-21 (APPROVED) | FC-10 | `set_node_field` is a package-local, provisional operation because post-creation title and flag changes otherwise have no non-clobbering write path. The owner requested consolidation on the general per-field architecture rather than the competing title-only operation. Its field set and value types are closed, its incarnation semantics match `set_widget`, and a language-neutral conformance session is included here; the upstream comfy-cli vocabulary has not yet adopted the kind. | `src/types.ts`, `src/applier.ts`, `src/stamps.ts`, `src/collab-trace.ts`; schema Amendment A21; ADR-032 | Owner: Christian Byrne. Sunset: amend comfy-cli's frozen vocabulary and pin the adopted definition here, reconciling the name/payload if upstream differs. | +| 2026-09-22 (APPROVED) | KA-5 | **`insert_workflow`'s numeric link-id mint (`remap.ts`'s `derivedLinkId`) consults mutable document state, unlike every other id this op derives.** ComfyUI_frontend's `LinkId` is a branded `number` (unlike the string-or-number `NodeId`), so ADR-031's pure, op-id-derived STRING scheme cannot satisfy it; the frontend's interim 32-bit-hash workaround (ComfyUI_frontend#18458) measurably collided (~0.01-1% at realistic link counts) and, worse, its "reserved id range" did not survive a save/reload once the hashed id was persisted into the workflow file. `derivedLinkId` hashes the same ADR-031 seed to a 52-bit candidate, then verifies it against every numeric link id already persisted in the target document (`doc.ts`'s `persistedLinkIds`, top-level and every subgraph-definition interior, at any depth) plus every id already minted earlier in the same call, retrying (bounded, `MAX_LINK_ID_MINT_ATTEMPTS = 256`) past a collision rather than accepting one. This is a genuine, information-theoretic necessity, not a preference: an unbounded set of possible (op, link) pairs cannot be encoded injectively into a fixed 53-bit space without consulting SOME external state, unlike the string form's unbounded output space. Residual gap, deliberately accepted: two replicas that have never exchanged this specific op and each mint from their own differing locally-visible document state could in principle choose colliding candidates — bounded by the same ~2^-52-per-pair probability this package already accepts for UUID4 `op_id` collisions (KA-2) elsewhere. `link_id_collision`'s existing insert-time check becomes unreachable for links via any realistically-shaped input as a direct, intended consequence (the mint now actively avoids what that check would have caught) and remains reachable, and tested, only via deliberate id-space exhaustion. | `src/remap.ts` `derivedLinkId`/`candidateLinkId`; `src/doc.ts` `persistedLinkIds`; `src/applier.ts` `prepareInsertedWorkflow`; ADR-033 | Owner: Christian Byrne. Sunset: revisit only if ComfyUI_frontend widens `LinkId` to `string \| number` (matching `NodeId`), at which point the string form ADR-031 already gives every other kind becomes sufficient for links too and this exception can be retired. | diff --git a/src/applier.ts b/src/applier.ts index 18ddfed..22aa994 100644 --- a/src/applier.ts +++ b/src/applier.ts @@ -98,6 +98,7 @@ import { mset, nodesMap, nodeIncarnation, + persistedLinkIds, resolveDefinition, stampsMap, widgetStorageOf, @@ -985,7 +986,7 @@ function validateDefinitionInputs(subgraphs: unknown[], path = "workflow.definit }); } -function prepareInsertedWorkflow(op: InsertWorkflowOp): Record { +function prepareInsertedWorkflow(doc: Y.Doc, op: InsertWorkflowOp): Record { const workflow = scrubPrivateKeys(op.workflow) as unknown; if (typeof workflow !== "object" || workflow === null || Array.isArray(workflow)) { throw new OpRejectedError("malformed_op", "insert_workflow: workflow must be an object"); @@ -1007,11 +1008,21 @@ function prepareInsertedWorkflow(op: InsertWorkflowOp): Record } validateRawGraphIds(wf["nodes"] as unknown[], (wf["links"] as unknown[] | undefined) ?? [], "workflow"); validateDefinitionInputs((subgraphs as unknown[] | undefined) ?? []); - return remapInsertedWorkflowIds(wf as unknown as import("./types.js").WorkflowJSON, op.op_id) as unknown as Record; + // ADR-033 / KA-5 exception (docs/decisions/EXCEPTIONS.md): unlike every + // other id this op derives, a link's numeric id must not collide with + // anything already persisted in THIS document, so the mint reads doc state + // here — see `remap.ts`'s `derivedLinkId` for why the type constraint + // (`LinkId` is a branded `number`, unlike the string-or-number `NodeId`) + // makes that unavoidable. + return remapInsertedWorkflowIds( + wf as unknown as import("./types.js").WorkflowJSON, + op.op_id, + persistedLinkIds(doc), + ) as unknown as Record; } function applyInsertWorkflow(doc: Y.Doc, op: InsertWorkflowOp, catalog?: WidgetCatalog): SuccessfulOutcome { - const wf = prepareInsertedWorkflow(op); + const wf = prepareInsertedWorkflow(doc, op); const remappedDefinitions = wf["definitions"] as { subgraphs?: unknown[] } | undefined; const remappedSubgraphs = remappedDefinitions?.subgraphs ?? []; @@ -1177,9 +1188,16 @@ function updateInsertedWorkflowMeta( const currentNode = numericId(meta.get("last_node_id")) ?? 0; const maxNode = Math.max(currentNode, ...nodeWrites.map(([, id]) => numericId(id) ?? currentNode)); if (maxNode > currentNode) mset(meta, "last_node_id", maxNode); - const currentLink = numericId(meta.get("last_link_id")) ?? 0; - const maxLink = Math.max(currentLink, ...linkWrites.map((link) => numericId(link[0]) ?? currentLink)); - if (maxLink > currentLink) mset(meta, "last_link_id", maxLink); + // `last_link_id` is deliberately NOT advanced from `linkWrites` (KA-5: + // "document high-water marks are advisory, never allocators"). Every + // inserted link's own id is now a large, arbitrary-looking derived number + // (ADR-033 — `remap.ts`'s `derivedLinkId`, needed only because ComfyUI_ + // frontend's `LinkId` is a branded `number`), not a small sequential one, + // so folding it into the high-water mark would pin the doc's bookkeeping + // field to that value forever after the FIRST insert_workflow — a new, + // surprising side effect this line never had before (the pre-fix STRING + // derived id already failed `numericId` and left this a no-op for every + // inserted link). } // --------------------------------------------------------------------------- diff --git a/src/doc.ts b/src/doc.ts index 7d83c65..463135d 100644 --- a/src/doc.ts +++ b/src/doc.ts @@ -285,6 +285,48 @@ export function initDoc(doc: Y.Doc, catalogVersion = ""): void { }); } +/** + * Every numeric link id currently persisted in `doc`: every key of the + * top-level `links` map, and every key of every subgraph definition's own + * interior `links` map, at any nesting depth (mirrors + * `definitionIdExistsOutsideRoot`'s recursive-definitions walk in + * `applier.ts`). Non-numeric keys (an explicit string id, or a positional + * `#0`/`base~1` key `mint.ts`'s `mintDefinitionLinks` assigns to an ID-LESS + * interior link) are not part of the numeric-link-id space and are skipped. + * + * `insert_workflow`'s numeric link-id mint (`remap.ts`'s `derivedLinkId`) + * uses this as the reservation a freshly minted link id must never land on — + * see ADR-033 and the KA-5 exception logged in `docs/decisions/EXCEPTIONS.md` + * for why this ONE id domain, unlike every other id `insert_workflow` + * derives, consults mutable document state. + */ +export function persistedLinkIds(doc: Y.Doc): Set { + const ids = new Set(); + const addNumericKeys = (map: Y.Map): void => { + map.forEach((_value, key) => { + // Unsigned-decimal keys only (mirrors `applier.ts`'s `numericId`): no + // leading zero/sign/hex/float noise from a non-numeric explicit or + // positional interior-link key coincidentally parsing under `Number()`. + if (!/^\d+$/.test(key)) return; + const numeric = Number(key); + if (Number.isSafeInteger(numeric)) ids.add(numeric); + }); + }; + addNumericKeys(linksMap(doc)); + const walkDefinitions = (defs: Y.Map): void => { + defs.forEach((definition) => { + if (!(definition instanceof Y.Map)) return; + const links = definition.get("links"); + if (links instanceof Y.Map) addNumericKeys(links); + const nestedContainer = definition.get("definitions"); + const nested = nestedContainer instanceof Y.Map ? nestedContainer.get("subgraphs") : undefined; + if (nested instanceof Y.Map) walkDefinitions(nested); + }); + }; + walkDefinitions(definitionsMap(doc) as unknown as Y.Map); + return ids; +} + // --------------------------------------------------------------------------- // Mutation instrumentation (schema §11 bounded-writes conformance) // diff --git a/src/remap.ts b/src/remap.ts index 278bb3d..d78a0af 100644 --- a/src/remap.ts +++ b/src/remap.ts @@ -1,4 +1,4 @@ -import type { WorkflowJSON, WorkflowNode } from "./types.js"; +import { OpRejectedError, type WorkflowJSON, type WorkflowNode } from "./types.js"; import { sha256Hex } from "./digest.js"; function derivedId(opId: string, scope: string, kind: string, original: unknown): string { @@ -10,6 +10,69 @@ function derivedDefinitionId(opId: string, scope: string, original: unknown): st return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-4${hex.slice(13, 16)}-8${hex.slice(17, 20)}-${hex.slice(20, 32)}`; } +/** + * A newly derived link id must be a real `number` — ComfyUI_frontend's + * `LinkId` is `number & { __brand }`, unlike `NodeId` (`string | number`), so + * the string form `derivedId` gives every other kind cannot satisfy it + * (ComfyUI_frontend#18458). Hex digits kept from the digest: comfortably + * inside `Number.MAX_SAFE_INTEGER` (2^53 - 1) even after the `+1` below, so + * the candidate is always a lossless JS integer. + */ +const LINK_ID_HEX_DIGITS = 13; // 52 bits + +/** + * Bounded retries per link before minting gives up (ADR-033). Collision + * probability per attempt is `taken.size / 2^52`; any realistic document — + * even one with millions of live links — clears this on the first or second + * attempt. The bound exists so a pathological/adversarial reservation set + * (see `docs/decisions/EXCEPTIONS.md`'s KA-5 row) fails LOUDLY, byte-identically, + * and in bounded work, rather than looping. + */ +export const MAX_LINK_ID_MINT_ATTEMPTS = 256; + +/** Deterministic 52-bit candidate for the `attempt`'th try of `seed`. Never 0. */ +function candidateLinkId(seed: string, attempt: number): number { + const hex = sha256Hex(`${seed}#${String(attempt)}`).slice(0, LINK_ID_HEX_DIGITS); + return Number(BigInt(`0x${hex}`)) + 1; +} + +/** + * Mint a genuinely unique numeric id for a link `insert_workflow` carries. + * + * Deterministic given (`opId`, `scope`, `original`) and `taken`'s state at + * the time of the call: the same inputs always retry through the same + * candidate sequence, and the FIRST one absent from `taken` wins and is + * immediately reserved into `taken` so a later link in the same insertion + * (this scope or any other) can never mint the same value. This is what + * makes the guarantee genuine rather than probabilistic — every candidate is + * checked against every id the caller told this function about, not merely + * assumed unlikely to collide (ComfyUI_frontend#18458's review: a 32-bit + * hash collided at a measured ~0.01-1% rate at realistic link counts). + * + * `taken` is caller-supplied rather than read here (ADR-033): this module + * stays free of `Y.Doc`/Yjs entirely (KA-3), and the caller decides what + * "already spoken for" means — `applier.ts` seeds it from + * `doc.ts`'s `persistedLinkIds()` before the first call in one + * `insert_workflow` application, so the reservation also covers ids a PRIOR + * `insert_workflow` (or a native `connect`/`mint()`) already committed to + * this same document, which is exactly the cross-op collision + * ComfyUI_frontend#18458 reproduced against the interim 32-bit hash. + */ +function derivedLinkId(opId: string, scope: string, original: unknown, taken: Set): number { + const seed = derivedId(opId, scope, "link", original); + for (let attempt = 0; attempt < MAX_LINK_ID_MINT_ATTEMPTS; attempt++) { + const candidate = candidateLinkId(seed, attempt); + if (!taken.has(candidate)) { + taken.add(candidate); + return candidate; + } + } + throw new OpRejectedError( + "link_id_collision", + `insert_workflow: could not mint a collision-free numeric id for link '${String(original)}' (scope '${scope}') after ${String(MAX_LINK_ID_MINT_ATTEMPTS)} attempts`, + ); +} + function linkEndpoints(link: unknown): [unknown, unknown] | undefined { if (Array.isArray(link) && link[1] !== undefined && link[3] !== undefined) return [link[1], link[3]]; if (typeof link === "object" && link !== null) { @@ -51,7 +114,7 @@ export function linkHasMissingEndpoint(link: unknown, hasNode: (id: unknown) => return endpoints !== undefined && (!hasNode(endpoints[0]) || !hasNode(endpoints[1])); } -function remapInputs(inputs: unknown, linkIds: Map, droppedLinkIds: Set): void { +function remapInputs(inputs: unknown, linkIds: Map, droppedLinkIds: Set): void { if (!Array.isArray(inputs)) return; for (const input of inputs) { if (typeof input !== "object" || input === null || !("link" in input)) continue; @@ -73,7 +136,7 @@ function remapInputs(inputs: unknown, linkIds: Map, droppedLinkI function remapLinkIdArray( records: unknown, field: string, - linkIds: Map, + linkIds: Map, droppedLinkIds: Set, ): void { if (!Array.isArray(records)) return; @@ -116,12 +179,13 @@ function remapGraph( scope: string, definitionIds: Map, dropDanglingLinks: boolean, + takenLinkIds: Set, isDefinitionInterior = false, interiorNodeIds: (definitionId: string) => Map | undefined = () => undefined, ): void { const nodes = graph["nodes"] as WorkflowNode[]; const nodeIds = new Map(); - const linkIds = new Map(); + const linkIds = new Map(); // Numeric and string aliases normalized by validation name the same node. for (const node of nodes) nodeIds.set(normalizedId(node.id), derivedId(opId, scope, "node", node.id)); const droppedLinkIds = new Set(); @@ -134,7 +198,7 @@ function remapGraph( }); for (const link of links) { const id = linkId(link); - linkIds.set(normalizedId(id), derivedId(opId, scope, "link", id)); + linkIds.set(normalizedId(id), derivedLinkId(opId, scope, id, takenLinkIds)); } for (const node of nodes) { @@ -175,11 +239,29 @@ function remapGraph( } } -/** Deterministically remap every id carried by an insertion op, including nested definition graphs. */ -export function remapInsertedWorkflowIds(wf: WorkflowJSON, opId: string): WorkflowJSON { +/** + * Deterministically remap every id carried by an insertion op, including + * nested definition graphs. + * + * `reservedLinkIds` seeds the numeric link-id mint's exclusion set (ADR-033): + * the caller (`applier.ts`) passes every numeric link id already persisted in + * the target document (`doc.ts`'s `persistedLinkIds`) so a freshly minted id + * can never land on one. It defaults to empty for callers with no document + * to consult — direct unit tests, and any caller minting a workflow that has + * never been merged into a doc. One `Set` is shared across the whole + * traversal (root scope and every definition scope, at every nesting depth) + * so two links in the SAME insertion, however they are scoped, can never + * collide with each other either. + */ +export function remapInsertedWorkflowIds( + wf: WorkflowJSON, + opId: string, + reservedLinkIds: Iterable = [], +): WorkflowJSON { const out = structuredClone(wf) as WorkflowJSON & Record; out.links ??= []; out.groups ??= []; + const takenLinkIds = new Set(reservedLinkIds); const subgraphs = (out.definitions?.subgraphs ?? []) as Array>; const definitionIdsByScope = new Map>(); // Interior node id maps keyed by definition scope, computed before any @@ -219,6 +301,7 @@ export function remapInsertedWorkflowIds(wf: WorkflowJSON, opId: string): Workfl definitionScope, definitionIdsByScope.get(definitionScope)!, true, + takenLinkIds, true, interiorAt(definitionScope), ); @@ -226,7 +309,7 @@ export function remapInsertedWorkflowIds(wf: WorkflowJSON, opId: string): Workfl rewrite(nested, definitionScope); } }; - remapGraph(out, opId, "root", definitionIdsByScope.get("root")!, true, false, interiorAt("root")); + remapGraph(out, opId, "root", definitionIdsByScope.get("root")!, true, takenLinkIds, false, interiorAt("root")); rewrite(subgraphs, "root"); return out; } diff --git a/test/insert-workflow-numeric-link-ids.regression.test.ts b/test/insert-workflow-numeric-link-ids.regression.test.ts new file mode 100644 index 0000000..36505bf --- /dev/null +++ b/test/insert-workflow-numeric-link-ids.regression.test.ts @@ -0,0 +1,433 @@ +/** + * `insert_workflow` numeric link ids (ADR-033). + * + * ComfyUI_frontend's `LinkId` is `number & { __brand: 'LinkId' }`, unlike the + * string-or-number `NodeId`, so `remap.ts`'s usual string `derivedId` + * (`insert:::link:`) cannot satisfy it. The interim + * frontend fix (ComfyUI_frontend#18458, `ecsFollowerAdapter.ts`'s + * `resolveLinkId`) hashed that string into a 32-bit number and was found by + * two reviewers to be genuinely broken, not merely approximate: two + * independent derived ids collided at a measured ~0.01-1% rate at realistic + * link counts, AND the resolved id is what `graph.serialize()` persists, so + * a "reserved range" chosen above `last_link_id` does not survive a + * save/reload — the saved file now contains a genuine numeric link id + * inside that range. + * + * `remap.ts`'s `derivedLinkId` fixes this at the source: it hashes the same + * op-id-derived seed to a 52-bit candidate, then verifies it against every + * numeric link id already persisted in the target document (`doc.ts`'s + * `persistedLinkIds` — top-level AND every subgraph-definition interior, at + * any depth) plus every id already minted earlier in the same call, retrying + * past a collision (bounded, `MAX_LINK_ID_MINT_ATTEMPTS`) rather than + * accepting one. See `docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md` + * and the KA-5 row in `docs/decisions/EXCEPTIONS.md`. + * + * `test/insert-workflow.test.ts`'s "numeric link id minting (ADR-033)" block + * covers the single-op avoidance and id-space-exhaustion cases directly + * against `occupiedIdVectors`' existing node/definition siblings. This file + * covers the harder cross-cutting properties named in review: collision + * avoidance that spans top-level and subgraph-interior scopes, multiple + * `insert_workflow` ops in one session (sequential and same-batch), the + * exact cross-op collision ComfyUI_frontend#18458 reproduced, scale within + * one op, save/reload round-trip stability, and comfy-multi-player#230's + * definition-interior promoted-linkIds threading with the new numeric ids. + */ +import { describe, expect, it } from "vitest"; +import * as Y from "yjs"; + +import { applyOps, mint, project, type Op, type WidgetCatalog, type WorkflowJSON } from "../src/index.js"; +import { appliedMap, persistedLinkIds } from "../src/doc.js"; +import { remapInsertedWorkflowIds } from "../src/remap.js"; + +const catalog: WidgetCatalog = { + types: { + Src: { widget_order: [] }, + Sink: { widget_order: [] }, + Def: { widget_order: [] }, + }, +}; + +let seq = 0; +function opEnvelope(overrides: Partial = {}): Op { + const op_id = ("n" + String(seq++).padStart(4, "0")).padEnd(32, "0"); + return { op: "insert_workflow", op_id, actor: "a", base_version: 1, stamp: [1, "a"], ...overrides } as unknown as Op; +} + +/** All top-level projected link tuples, typed loosely for id/label access. */ +function links(wf: WorkflowJSON): Array<[unknown, unknown, unknown, unknown, unknown, unknown]> { + return (wf.links ?? []) as never; +} + +function linkNamed(wf: WorkflowJSON, label: unknown): [unknown, unknown, unknown, unknown, unknown, unknown] { + const found = links(wf).find((link) => link[5] === label); + if (!found) throw new Error(`no projected link labeled ${String(label)}`); + return found; +} + +describe("insert_workflow numeric link ids (ADR-033)", () => { + it("mints a real number for a top-level link while node ids stay derived strings", () => { + const doc = mint({ nodes: [], links: [] }, catalog); + const op = opEnvelope({ workflow: { nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], links: [[10, 1, 0, 2, 0, "L"]] } }); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const wf = project(doc, catalog); + expect(typeof linkNamed(wf, "L")[0]).toBe("number"); + for (const node of wf.nodes!) expect(typeof node.id).toBe("string"); + }); + + it("reproduces and fixes ComfyUI_frontend#18458's exact cross-op scenario: two independent insert_workflow ops never collide", () => { + // The reviewers' reproduction used two distinct 32-char op ids whose + // FNV-1a-folded-to-32-bit hashes collided. This package's mint uses 52 + // bits AND actively verifies against the document, so two distinct ops — + // regardless of whether their raw hash candidates would have collided — + // can never persist the same numeric link id. + const doc = mint({ nodes: [], links: [] }, catalog); + const opA = opEnvelope({ + workflow: { + nodes: [{ id: 101, type: "Src" }, { id: 102, type: "Sink" }], + links: [[201, 101, 0, 102, 0, "op-a-link"]], + }, + }); + const opB = opEnvelope({ + workflow: { + nodes: [{ id: 103, type: "Src" }, { id: 104, type: "Sink" }], + links: [[201, 103, 0, 104, 0, "op-b-link"]], + }, + }); + + expect(applyOps(doc, [opA], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + expect(applyOps(doc, [opB], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + + const wf = project(doc, catalog); + expect(wf.nodes).toHaveLength(4); + const linkA = linkNamed(wf, "op-a-link"); + const linkB = linkNamed(wf, "op-b-link"); + expect(linkA[0]).not.toBe(linkB[0]); + // Op A's wire must not have been replaced by op B's (the reported bug): + // both endpoints resolve to their OWN op's nodes. + const nodeIds = new Set(wf.nodes!.map((n) => n.id)); + expect(nodeIds.has(linkA[1] as string)).toBe(true); + expect(nodeIds.has(linkB[1] as string)).toBe(true); + }); + + it("avoids colliding with a numeric link id in the SAME batch's other insert_workflow op", () => { + // Same as the cross-op case above, but delivered as ONE `applyOps` batch + // rather than two sequential calls — op B's mint must see op A's + // already-committed link (from the earlier `doc.transact` in the same + // batch) even though nothing has round-tripped through a snapshot yet. + const doc = mint({ nodes: [], links: [] }, catalog); + const opA = opEnvelope({ + workflow: { nodes: [{ id: 111, type: "Src" }, { id: 112, type: "Sink" }], links: [[301, 111, 0, 112, 0, "batch-a"]] }, + }); + const opB = opEnvelope({ + workflow: { nodes: [{ id: 113, type: "Src" }, { id: 114, type: "Sink" }], links: [[301, 113, 0, 114, 0, "batch-b"]] }, + }); + + const result = applyOps(doc, [opA, opB], catalog); + expect(result.outcomes).toEqual([ + { op_id: opA.op_id, outcome: "applied" }, + { op_id: opB.op_id, outcome: "applied" }, + ]); + const wf = project(doc, catalog); + expect(linkNamed(wf, "batch-a")[0]).not.toBe(linkNamed(wf, "batch-b")[0]); + }); + + it("a top-level insert avoids a numeric link id already used INSIDE an existing subgraph definition", () => { + // The reservation set spans scopes: a definition's interior links are a + // real collision domain for a top-level candidate too (`doc.ts`'s + // `persistedLinkIds` walks every definition, not just the top-level map). + const opId = "aa11df0c31f9440b9385ac8e01e099b2"; + const occupied = firstCandidate(opId, 501); + const doc = mint( + { + nodes: [{ id: 1, type: "Def" }], + links: [], + definitions: { + subgraphs: [ + { + id: "Def", + nodes: [{ id: "x", type: "Src" }, { id: "y", type: "Sink" }], + links: [{ id: occupied, origin_id: "x", origin_slot: 0, target_id: "y", target_slot: 0, type: "X" }], + }, + ], + }, + } as unknown as WorkflowJSON, + catalog, + ); + + const op = opEnvelope({ + workflow: { nodes: [{ id: 701, type: "Src" }, { id: 702, type: "Sink" }], links: [[501, 701, 0, 702, 0, "top-level"]] }, + op_id: opId, + }); + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const wf = project(doc, catalog); + expect(linkNamed(wf, "top-level")[0]).not.toBe(occupied); + }); + + it("an inserted subgraph definition's interior link avoids a numeric link id already used at the top level", () => { + const opId = "bb11df0c31f9440b9385ac8e01e099b2"; + const workflow = { + nodes: [{ id: 900, type: "NewDef" }], + links: [], + definitions: { + subgraphs: [ + { + id: "NewDef", + nodes: [{ id: "p", type: "Src" }, { id: "q", type: "Sink" }], + links: [{ id: 502, origin_id: "p", origin_slot: 0, target_id: "q", target_slot: 0, type: "X" }], + }, + ], + }, + }; + + // First, discover the definition-interior candidate this exact (opId, + // workflow) pair mints against an EMPTY doc — nothing to avoid yet. + const emptyDoc = mint({ nodes: [], links: [] }, catalog); + const probe = opEnvelope({ workflow, op_id: opId }); + expect(applyOps(emptyDoc, [probe], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const naturalDefinition = ( + project(emptyDoc, catalog).definitions as { subgraphs: Array<{ links: Array<{ id: unknown }> }> } + ).subgraphs[0]!; + const naturalCandidate = naturalDefinition.links[0]!.id as number; + + // Now seed a FRESH doc where that exact numeric id is already a + // TOP-LEVEL link (never touched by insert_workflow) and re-run the + // IDENTICAL op. `persistedLinkIds` walks the top-level map, so the + // definition-interior mint below must land somewhere else. + const occupiedDoc = mint( + { nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], links: [[naturalCandidate, 1, 0, 2, 0, "top-incumbent"]] }, + catalog, + ); + const op = opEnvelope({ workflow, op_id: opId }); + expect(applyOps(occupiedDoc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + + const wf = project(occupiedDoc, catalog); + const definition = (wf.definitions as { subgraphs: Array<{ links: Array<{ id: unknown }> }> }).subgraphs[0]!; + expect(definition.links).toHaveLength(1); + expect(definition.links[0]!.id).not.toBe(naturalCandidate); + expect(typeof definition.links[0]!.id).toBe("number"); + // The top-level incumbent is completely untouched. + expect(linkNamed(wf, "top-incumbent")[0]).toBe(naturalCandidate); + }); + + it("mints distinct ids for a large batch of links inserted in one op", () => { + const opId = "cc11df0c31f9440b9385ac8e01e099b2"; + const count = 200; + const nodes = Array.from({ length: count * 2 }, (_, i) => ({ id: `n${String(i)}`, type: i % 2 === 0 ? "Src" : "Sink" })); + const rawLinks = Array.from({ length: count }, (_, i) => [1000 + i, `n${String(i * 2)}`, 0, `n${String(i * 2 + 1)}`, 0, `link-${String(i)}`]); + const doc = mint({ nodes: [], links: [] }, catalog); + const op = opEnvelope({ workflow: { nodes, links: rawLinks }, op_id: opId }); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const wf = project(doc, catalog); + expect(wf.links).toHaveLength(count); + const ids = wf.links!.map((link) => (link as unknown[])[0]); + expect(new Set(ids).size).toBe(count); + for (const id of ids) expect(typeof id).toBe("number"); + }); + + it("survives a save/reload round trip: a re-minted doc's persisted numeric link ids are still avoided by a later insert", () => { + // The exact failure the second frontend reviewer identified in the + // interim hash fix: once an inserted link's numeric id is persisted into + // a saved workflow and the doc is re-minted from that JSON, a later + // insert must still avoid it — there is no session-local allocator state + // to lose here, because avoidance is grounded in the DOCUMENT itself. + const firstOpId = "dd11df0c31f9440b9385ac8e01e099b2"; + const original = mint({ nodes: [], links: [] }, catalog); + const firstOp = opEnvelope({ + workflow: { nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], links: [[601, 1, 0, 2, 0, "persisted"]] }, + op_id: firstOpId, + }); + expect(applyOps(original, [firstOp], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const persistedWorkflow = project(original, catalog); + const persistedLinkId = linkNamed(persistedWorkflow, "persisted")[0]; + expect(typeof persistedLinkId).toBe("number"); + + // "Save and reload": a FRESH doc, minted from nothing but the projected + // JSON — no memory of the first doc, no in-process allocator survives. + const reloaded = mint(persistedWorkflow, catalog); + expect(persistedLinkIds(reloaded).has(persistedLinkId as number)).toBe(true); + + // A raw id that happens to derive to the SAME candidate the first op + // used (found deterministically, not by chance) must still avoid it. + const secondOpId = "ee11df0c31f9440b9385ac8e01e099b2"; + const collidingRaw = firstCandidate(secondOpId, 601) === persistedLinkId ? 601 : findCollidingRawId(secondOpId, persistedLinkId as number); + if (collidingRaw === undefined) return; // 52 bits: no natural collision is the expected outcome. + + const secondOp = opEnvelope({ + workflow: { nodes: [{ id: 3, type: "Src" }, { id: 4, type: "Sink" }], links: [[collidingRaw, 3, 0, 4, 0, "after-reload"]] }, + op_id: secondOpId, + }); + expect(applyOps(reloaded, [secondOp], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const after = project(reloaded, catalog); + expect(linkNamed(after, "after-reload")[0]).not.toBe(persistedLinkId); + }); + + it("round-trips numeric link ids byte-for-byte through project → mint with no duplication or corruption", () => { + const opId = "ff11df0c31f9440b9385ac8e01e099b2"; + const doc = mint({ nodes: [], links: [] }, catalog); + const op = opEnvelope({ + workflow: { + nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }, { id: 3, type: "Def" }], + links: [[701, 1, 0, 2, 0, "L1"]], + definitions: { + subgraphs: [ + { + id: "Def", + nodes: [{ id: "a", type: "Src" }, { id: "b", type: "Sink" }], + links: [{ id: 702, origin_id: "a", origin_slot: 0, target_id: "b", target_slot: 0, type: "X" }], + }, + ], + }, + }, + op_id: opId, + }); + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const first = project(doc, catalog); + + const reminted = mint(first, catalog); + const second = project(reminted, catalog); + expect(second).toEqual(first); + }); + + it("subgraph-interior promoted linkIds thread numeric ids end-to-end (comfy-multi-player#230 non-regression)", () => { + // Mirrors `test/insert-workflow-promoted-input-linkids.regression.test.ts` + // but through the REAL applier + project path, asserting the ids + // `remapLinkIdArray` threads into a definition's own `inputs[].linkIds` + // are the SAME real numbers as `links[].id` — exactly what ComfyUI_ + // frontend's `agentSubgraphHostSlots.ts` (`Map` keyed by + // `link.id`) needs to resolve a promoted widget. + const opId = "0011df0c31f9440b9385ac8e01e099b2"; + const doc = mint({ nodes: [], links: [] }, catalog); + const op = opEnvelope({ + workflow: { + nodes: [{ id: 100, type: "PromotedDef", inputs: [{ name: "text", type: "STRING", link: null, widget: { name: "text" } }] }], + links: [], + definitions: { + subgraphs: [ + { + id: "PromotedDef", + name: "Promoted text", + inputs: [{ name: "text", type: "STRING", linkIds: [34] }], + outputs: [], + nodes: [ + { id: 10, type: "Src", inputs: [], outputs: [{ name: "STRING", type: "STRING", links: [34] }] }, + { id: 11, type: "Def", inputs: [{ name: "text", type: "STRING", link: 34, widget: { name: "text" } }], outputs: [] }, + ], + links: [{ id: 34, origin_id: 10, origin_slot: 0, target_id: 11, target_slot: 0, type: "STRING" }], + }, + ], + }, + }, + op_id: opId, + }); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const wf = project(doc, catalog); + const definition = ( + wf.definitions as { subgraphs: Array<{ inputs: Array<{ linkIds: unknown[] }>; links: Array<{ id: unknown }> }> } + ).subgraphs[0]!; + + expect(definition.links).toHaveLength(1); + const realLinkId = definition.links[0]!.id; + expect(typeof realLinkId).toBe("number"); + // The promoted input names the SAME numeric id, not a stringified or + // otherwise-derived stand-in — a `Map` keyed by + // `definition.links[].id` resolves `inputs[0].linkIds[0]` directly. + expect(definition.inputs[0]!.linkIds).toEqual([realLinkId]); + }); + + it("keeps a definition-interior link fed by the subgraph IO sentinels numeric (sentinel non-regression)", () => { + const opId = "1122df0c31f9440b9385ac8e01e099b2"; + const doc = mint({ nodes: [], links: [] }, catalog); + const op = opEnvelope({ + workflow: { + nodes: [ + { + id: 100, + type: "SentinelDef", + inputs: [{ name: "text", type: "STRING", link: null, widget: { name: "text" } }], + outputs: [{ name: "IMAGE", type: "IMAGE", links: [] }], + }, + ], + links: [], + definitions: { + subgraphs: [ + { + id: "SentinelDef", + name: "Sentinel", + inputs: [{ name: "text", type: "STRING", linkIds: [16] }], + outputs: [{ name: "IMAGE", type: "IMAGE", linkIds: [17] }], + nodes: [ + { id: 27, type: "Def", inputs: [{ name: "text", type: "STRING", link: 16, widget: { name: "text" } }], outputs: [] }, + { id: 8, type: "Src", inputs: [], outputs: [{ name: "IMAGE", type: "IMAGE", links: [17] }] }, + ], + links: [ + { id: 16, origin_id: -10, origin_slot: 0, target_id: 27, target_slot: 1, type: "STRING" }, + { id: 17, origin_id: 8, origin_slot: 0, target_id: -20, target_slot: 0, type: "IMAGE" }, + ], + }, + ], + }, + }, + op_id: opId, + }); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const wf = project(doc, catalog); + const definition = ( + wf.definitions as { + subgraphs: Array<{ + inputs: Array<{ linkIds: unknown[] }>; + outputs: Array<{ linkIds: unknown[] }>; + links: Array<{ id: unknown; origin_id: unknown; target_id: unknown }>; + }>; + } + ).subgraphs[0]!; + + expect(definition.links).toHaveLength(2); + const promotedLink = definition.links.find((link) => link.origin_id === -10)!; + const exposedLink = definition.links.find((link) => link.target_id === -20)!; + expect(typeof promotedLink.id).toBe("number"); + expect(typeof exposedLink.id).toBe("number"); + expect(definition.inputs[0]!.linkIds).toEqual([promotedLink.id]); + expect(definition.outputs[0]!.linkIds).toEqual([exposedLink.id]); + }); + + it("an exact replay (same op_id, same payload) is a true no-op and does not re-mint the link id", () => { + const opId = "2233df0c31f9440b9385ac8e01e099b2"; + const doc = mint({ nodes: [], links: [] }, catalog); + const op = opEnvelope({ + workflow: { nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], links: [[801, 1, 0, 2, 0, "L"]] }, + op_id: opId, + }); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const firstProjection = project(doc, catalog); + const mintedId = linkNamed(firstProjection, "L")[0]; + const beforeReplay = Buffer.from(Y.encodeStateAsUpdate(doc)); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "no-op" }); + expect(Buffer.from(Y.encodeStateAsUpdate(doc)).equals(beforeReplay)).toBe(true); + expect(appliedMap(doc).has(op.op_id)).toBe(true); + const afterReplay = project(doc, catalog); + expect(linkNamed(afterReplay, "L")[0]).toBe(mintedId); + }); +}); + +/** The candidate `derivedLinkId` would pick at root scope for (`opId`, `rawLinkId`) with no reservations. */ +function firstCandidate(opId: string, rawLinkId: number): number { + const remapped = remapInsertedWorkflowIds( + { nodes: [{ id: "a", type: "Src" }, { id: "b", type: "Sink" }], links: [[rawLinkId, "a", 0, "b", 0, "x"]] } as unknown as WorkflowJSON, + opId, + ) as unknown as { links: Array<[number, ...unknown[]]> }; + return remapped.links[0]![0]; +} + +/** Search a small range of raw link ids for one whose first candidate (given `opId`) equals `target`. */ +function findCollidingRawId(opId: string, target: number): number | undefined { + for (let raw = 1; raw < 5000; raw++) { + if (firstCandidate(opId, raw) === target) return raw; + } + return undefined; +} diff --git a/test/insert-workflow.test.ts b/test/insert-workflow.test.ts index cb458d4..3e6d707 100644 --- a/test/insert-workflow.test.ts +++ b/test/insert-workflow.test.ts @@ -11,7 +11,13 @@ * the op_id gate, and byte-identical doc on rejection. * * ID allocation belongs to the applier and is derived from the immutable op - * id plus each raw id, never from mutable document state. + * id plus each raw id, never from mutable document state — with ONE + * documented exception (ADR-033, `docs/decisions/EXCEPTIONS.md`'s KA-5 row): + * a link's id must be a real `number` (ComfyUI_frontend#18458 — `LinkId` is + * a branded `number`, unlike the string-or-number `NodeId`), so + * `remap.ts`'s `derivedLinkId` additionally consults the target document's + * already-persisted numeric link ids to guarantee its mint never collides, + * rather than merely hashing into a wide-enough space and hoping. */ import { describe, expect, it } from "vitest"; import * as Y from "yjs"; @@ -31,7 +37,7 @@ import { type WidgetCatalog, type WorkflowJSON, } from "../src/index.js"; -import { remapInsertedWorkflowIds } from "../src/remap.js"; +import { MAX_LINK_ID_MINT_ATTEMPTS, remapInsertedWorkflowIds } from "../src/remap.js"; import { appliedOpIds, noOpIds, rejectedOutcomeWithIndex } from "./apply-result-helpers.js"; const catalog: WidgetCatalog = { @@ -96,6 +102,40 @@ function replaySnapshot(seed: Uint8Array, ops: Op[]): WorkflowJSON { return project(doc, catalog); } +/** + * One link's numeric id, as `remap.ts`'s `derivedLinkId` mint would produce + * it for (`opId`, `"root"`, `rawLinkId`) given `reserved` already taken — + * i.e. exactly what `insert_workflow` would write for a doc whose persisted + * numeric link ids are `reserved`. Used to construct the EXACT candidate a + * real apply will also compute (determinism), and to walk the mint's whole + * deterministic attempt sequence for one seed (see `candidateSequence`). + */ +function derivedLinkIdForTest(opId: string, rawLinkId: number, reserved: ReadonlySet = new Set()): number { + const remapped = remapInsertedWorkflowIds( + { nodes: [{ id: "a", type: "Src" }, { id: "b", type: "Sink" }], links: [[rawLinkId, "a", 0, "b", 0, "x"]] } as unknown as WorkflowJSON, + opId, + reserved, + ) as unknown as { links: Array<[number, ...unknown[]]> }; + return remapped.links[0]![0]; +} + +/** + * The first `count` candidates `derivedLinkId` would try, in order, for + * (`opId`, `"root"`, `rawLinkId`) — walked by feeding each prior candidate + * back in as `reserved`, which forces the mint to skip past it to the next + * one in its deterministic sequence. + */ +function candidateSequence(opId: string, rawLinkId: number, count: number): number[] { + const sequence: number[] = []; + const reserved = new Set(); + for (let i = 0; i < count; i++) { + const next = derivedLinkIdForTest(opId, rawLinkId, reserved); + sequence.push(next); + reserved.add(next); + } + return sequence; +} + const bytes = (doc: Y.Doc): Buffer => Buffer.from(Y.encodeStateAsUpdate(doc)); function ids(wf: WorkflowJSON): unknown[] { @@ -408,7 +448,10 @@ describe("insert_workflow: happy path", () => { const wf = project(doc, catalog); expect(ids(wf).slice(0, 5)).toEqual([1, 2, 3, 4, 6]); expect(ids(wf).slice(5).every((id) => typeof id === "string" && id.includes(op.op_id))).toBe(true); - expect(linkIds(wf)[1]).toEqual(expect.stringContaining(op.op_id)); + // Link ids are minted numeric (ADR-033 — `remap.ts`'s `derivedLinkId`), + // unlike the op-id-embedding derived STRING every other insert_workflow + // id keeps: ComfyUI_frontend's `LinkId` is a branded `number`. + expect(typeof linkIds(wf)[1]).toBe("number"); expect(defIds(wf).find((id) => id !== "def-1")).toMatch(/^[0-9a-f-]{36}$/); // Inserted nodes project with their widgets resolved through the catalog. const n100 = wf.nodes!.find((n) => n.pos?.[0] === 0)!; @@ -583,30 +626,39 @@ describe("insert_workflow: happy path", () => { it("drops both boundary directions and retains only the sibling link and its exact slot references", () => { const opId = "0123456789abcdef0123456789abcdef"; - const siblingLinkId = "insert:0123456789abcdef0123456789abcdef:root:link:903"; const sourceNodeId = "insert:0123456789abcdef0123456789abcdef:root:node:701"; const sinkNodeId = "insert:0123456789abcdef0123456789abcdef:root:node:702"; + const workflow = { + nodes: [ + { id: 701, type: "Src", title: "boundary source", outputs: [{ name: "out", links: [901, 903] }] }, + { id: 702, type: "Sink", title: "boundary sink", inputs: [{ name: "in", link: 902 }, { name: "sibling", link: 903 }] }, + ], + links: [ + [901, 701, 0, 999, 0, "inserted-to-external"], + [902, 999, 0, 702, 0, "external-to-inserted"], + [903, 701, 0, 702, 1, "sibling"], + ], + }; const doc = mint({ nodes: [], links: [] }, catalog); - const op = insertOp( - { - nodes: [ - { id: 701, type: "Src", title: "boundary source", outputs: [{ name: "out", links: [901, 903] }] }, - { id: 702, type: "Sink", title: "boundary sink", inputs: [{ name: "in", link: 902 }, { name: "sibling", link: 903 }] }, - ], - links: [ - [901, 701, 0, 999, 0, "inserted-to-external"], - [902, 999, 0, 702, 0, "external-to-inserted"], - [903, 701, 0, 702, 1, "sibling"], - ], - }, - { op_id: opId }, - ); + const op = insertOp(workflow, { op_id: opId }); seq--; + // The doc starts with no links, so the fix's numeric mint (`remap.ts`'s + // `derivedLinkId`) has nothing to avoid and its first candidate for the + // surviving link (903, "sibling") is exactly what the real apply below + // will also compute — ADR-033 is deterministic given (opId, scope, + // original) and the reservation state, and both calls start from the + // same (empty) reservation. + const remapped = remapInsertedWorkflowIds(workflow as unknown as WorkflowJSON, opId) as unknown as { + links: Array<[number, ...unknown[]]>; + }; + const siblingLinkId = remapped.links.find((link) => link[5] === "sibling")![0]; + expect(typeof siblingLinkId).toBe("number"); + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); - const workflow = project(doc, catalog); - expect(workflow.links).toEqual([[siblingLinkId, sourceNodeId, 0, sinkNodeId, 1, "sibling"]]); - expect(workflow.nodes).toEqual([ + const projected = project(doc, catalog); + expect(projected.links).toEqual([[siblingLinkId, sourceNodeId, 0, sinkNodeId, 1, "sibling"]]); + expect(projected.nodes).toEqual([ expect.objectContaining({ id: sourceNodeId, outputs: [{ name: "out", links: [siblingLinkId] }] }), expect.objectContaining({ id: sinkNodeId, @@ -822,14 +874,6 @@ describe("insert_workflow: rejection (KA-4 byte identity, op_id absent from appl links: [], }, }, - { - kind: "link", - code: "link_id_collision", - seed: { - nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], - links: [["insert:c641df0c31f9440b9385ac8e01e099b2:root:link:901", 1, 0, 2, 0, "incumbent link"]], - }, - }, { kind: "definition", code: "definition_conflict", @@ -867,6 +911,123 @@ describe("insert_workflow: rejection (KA-4 byte identity, op_id absent from appl }); } + describe("numeric link id minting (ADR-033)", () => { + it("mints a fresh numeric id rather than colliding with one already in the doc", () => { + const opId = "d641df0c31f9440b9385ac8e01e099b2"; + // The candidate `derivedLinkId` would pick with NO reservations — i.e. + // exactly what a pre-fix, doc-state-oblivious derivation would have + // produced, and exactly what a real apply would ALSO produce if this + // one id were not already occupied. + const wouldBeFirstCandidate = derivedLinkIdForTest(opId, 901); + + const doc = mint( + { + nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], + links: [[wouldBeFirstCandidate, 1, 0, 2, 0, "incumbent link"]], + }, + catalog, + ); + const beforeIncumbent = project(doc, catalog); + const op = insertOp( + { nodes: [{ id: 701, type: "Src" }, { id: 702, type: "Sink" }], links: [[901, 701, 0, 702, 0, "candidate link"]] }, + { op_id: opId }, + ); + seq--; + + const result = applyOps(doc, [op], catalog); + expect(result.outcomes[0]).toMatchObject({ outcome: "applied" }); + + const projected = project(doc, catalog); + const insertedLink = (projected.links as unknown[][]).find((link) => link[5] === "candidate link")!; + const incumbentLink = (projected.links as unknown[][]).find((link) => link[5] === "incumbent link")!; + // The mint skipped the occupied candidate and moved to the next one in + // its deterministic sequence, rather than colliding with or rejecting + // over the incumbent. + expect(insertedLink[0]).toBe(candidateSequence(opId, 901, 2)[1]); + expect(insertedLink[0]).not.toBe(wouldBeFirstCandidate); + // The incumbent (seeded independently of this op, not via + // insert_workflow) is completely untouched. + expect(incumbentLink).toEqual(beforeIncumbent.links![0]); + }); + + it("mints distinct ids for two links in the SAME insertion that would otherwise share a candidate", () => { + // Two DIFFERENT raw link ids can legitimately hash to the same first + // candidate; `derivedLinkId`'s shared `taken` set (threaded through one + // whole `remapInsertedWorkflowIds` call) must still give them distinct + // final ids. Constructed by finding the raw id whose own first + // candidate collides with 901's, rather than assuming one — the + // reservation-set mechanism is what is under test, not luck. + const opId = "e641df0c31f9440b9385ac8e01e099b2"; + const first = derivedLinkIdForTest(opId, 901); + let collidingRawId = -1; + for (let raw = 1; raw < 5000; raw++) { + if (raw !== 901 && derivedLinkIdForTest(opId, raw) === first) { + collidingRawId = raw; + break; + } + } + // No naturally-colliding raw id turned up in the search range — 52 bits + // of entropy makes that the overwhelmingly likely outcome, and it is + // itself evidence the space is wide enough that this scenario needs + // deliberate construction. Skip rather than assert a false positive. + if (collidingRawId === -1) return; + + const doc = mint({ nodes: [], links: [] }, catalog); + const op = insertOp( + { + nodes: [ + { id: 701, type: "Src", outputs: [{ name: "out", links: [901] }] }, + { id: 702, type: "Sink", inputs: [{ name: "in", link: 901 }] }, + { id: 703, type: "Src", outputs: [{ name: "out", links: [collidingRawId] }] }, + { id: 704, type: "Sink", inputs: [{ name: "in", link: collidingRawId }] }, + ], + links: [ + [901, 701, 0, 702, 0, "x"], + [collidingRawId, 703, 0, 704, 0, "y"], + ], + }, + { op_id: opId }, + ); + seq--; + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const projected = project(doc, catalog); + const linkX = (projected.links as unknown[][]).find((link) => link[5] === "x")!; + const linkY = (projected.links as unknown[][]).find((link) => link[5] === "y")!; + expect(linkX[0]).not.toBe(linkY[0]); + }); + + it("rejects link_id_collision when every mint attempt is already reserved (id-space exhaustion)", () => { + const opId = "f641df0c31f9440b9385ac8e01e099b2"; + const rawLinkId = 901; + // Occupy EVERY candidate the mint would ever try for this seed, so the + // bounded retry (ADR-033's `MAX_LINK_ID_MINT_ATTEMPTS`) is exhausted — + // the one scenario this deliberately-bounded loop is a safety valve + // for. This is not a realistic document; it is the adversarial worst + // case named in `docs/decisions/EXCEPTIONS.md`'s KA-5 row. + const exhausted = candidateSequence(opId, rawLinkId, MAX_LINK_ID_MINT_ATTEMPTS); + const doc = mint( + { + nodes: exhausted.flatMap((id, i) => [{ id: `src-${String(i)}`, type: "Src" }, { id: `sink-${String(i)}`, type: "Sink" }]), + links: exhausted.map((id, i) => [id, `src-${String(i)}`, 0, `sink-${String(i)}`, 0, "occupied"]), + } as unknown as WorkflowJSON, + catalog, + ); + const beforeBytes = bytes(doc); + const beforeProjected = project(doc, catalog); + const op = insertOp( + { nodes: [{ id: 701, type: "Src" }, { id: 702, type: "Sink" }], links: [[rawLinkId, 701, 0, 702, 0, "candidate link"]] }, + { op_id: opId }, + ); + seq--; + + expect(rejectedOutcomeWithIndex(applyOps(doc, [op], catalog))).toMatchObject({ index: 0, code: "link_id_collision" }); + expect(bytes(doc).equals(beforeBytes)).toBe(true); + expect(project(doc, catalog)).toEqual(beforeProjected); + expect(appliedMap(doc).has(op.op_id)).toBe(false); + }); + }); + it("routes dangling link endpoints through the existing unknown-node no-op path", () => { const doc = mint(baseWorkflow(), catalog); const op = insertOp({ nodes: [], links: [[200, 100, 0, 999, 0, "X"]] }); diff --git a/test/ka4-rejection-byte-identity.test.ts b/test/ka4-rejection-byte-identity.test.ts index 970d493..d81ad66 100644 --- a/test/ka4-rejection-byte-identity.test.ts +++ b/test/ka4-rejection-byte-identity.test.ts @@ -49,7 +49,23 @@ import { type WorkflowJSON, } from "../src/index.js"; import { appliedMap } from "../src/doc.js"; -import { remapInsertedWorkflowIds } from "../src/remap.js"; +import { MAX_LINK_ID_MINT_ATTEMPTS, remapInsertedWorkflowIds } from "../src/remap.js"; + +/** + * The numeric id `remap.ts`'s `derivedLinkId` would mint for + * (`opId`, `"root"`, `rawLinkId`) given `reserved` already taken. Walking it + * repeatedly (each prior result fed back as `reserved`) recovers the mint's + * whole deterministic attempt sequence for one seed — used below to + * construct the id-space-exhaustion row (ADR-033). + */ +function derivedLinkIdForRow(opId: string, rawLinkId: number, reserved: ReadonlySet = new Set()): number { + const remapped = remapInsertedWorkflowIds( + { nodes: [{ id: "a", type: "Src" }, { id: "b", type: "Sink" }], links: [[rawLinkId, "a", 0, "b", 0, "x"]] } as unknown as WorkflowJSON, + opId, + reserved, + ) as unknown as { links: Array<[number, ...unknown[]]> }; + return remapped.links[0]![0]; +} const catalog: WidgetCatalog = { types: { @@ -215,20 +231,35 @@ const CASES: Row[] = [ }, }, { + // A link's numeric id is minted with an active, doc-state-aware + // reservation (ADR-033's `derivedLinkId`), so — unlike the node/definition + // rows above, which collide against the FIRST (and only) id the applier + // ever derives — a single occupied candidate is no longer enough to + // reach this code: the mint simply retries past it (covered by + // `test/insert-workflow.test.ts`'s "mints a fresh numeric id..." case). + // This row instead exhausts every candidate the mint's bounded retry + // will ever try for this seed, which is the ONE scenario + // `MAX_LINK_ID_MINT_ATTEMPTS` exists to fail loudly on rather than loop + // (`docs/decisions/EXCEPTIONS.md`'s KA-5 row) — an adversarial + // reservation set, not a realistic document. kind: "insert_workflow", - why: "remapped link id collides with the live tree", + why: "every numeric link-id mint attempt is already occupied (id-space exhaustion)", code: "link_id_collision", build: () => ({ op: "insert_workflow", ...env(), - workflow: { nodes: [{ id: 2, type: "Src" }, { id: 3, type: "Sink" }], links: [[7, 2, 0, 3, 0, "X"]] }, + workflow: { nodes: [{ id: 2, type: "Src" }, { id: 3, type: "Sink" }], links: [[901, 2, 0, 3, 0, "X"]] }, }) as Op, seed: (op) => { const workflow = baseWorkflow(); - const remapped = remapInsertedWorkflowIds((op as { workflow: WorkflowJSON }).workflow, op.op_id); - const [remappedLinkId] = remapped.links![0] as [string, number, number, number, number, string]; - workflow.links!.push([remappedLinkId, 2, 0, 3, 0, "X"]); + const reserved = new Set(); + for (let i = 0; i < MAX_LINK_ID_MINT_ATTEMPTS; i++) { + const candidate = derivedLinkIdForRow(op.op_id, 901, reserved); + reserved.add(candidate); + workflow.nodes!.push({ id: `excl-src-${String(i)}`, type: "Src" }, { id: `excl-sink-${String(i)}`, type: "Sink" }); + workflow.links!.push([candidate, `excl-src-${String(i)}`, 0, `excl-sink-${String(i)}`, 0, "occupied"]); + } return workflow; }, }, From 5bfbbe316648d4e273ef9e44c6a11b9e92d09789 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 23 Sep 2026 01:19:38 +0000 Subject: [PATCH 2/3] fix: switch insert_workflow link ids to pure derivation (ADR-033 amendment) Christian Byrne's review of the original revision (PR #245, inline on src/remap.ts:65) found that derivedLinkId's retry-against-persisted- document-state design made link identity depend on local arrival order: if two link seeds shared the same first candidate, applying them A-then-B minted A the natural candidate and B the next one, while B-then-A minted B the natural candidate and A the next one. Two replicas applying the same concurrent op set in different orders could project different link ids for the same logical links. remap.ts's derivedLinkId now mints purely from the op's own content (opId, scope, the raw link) -- no document read, no caller-supplied reservation set, no retry -- restoring ADR-031's "producers do not inspect document state or remap ids" for links too. It spends the full safe-integer range (53 bits, widened from the original 52-bit window) rather than checking for a collision. remapInsertedWorkflowIds no longer takes a reservedLinkIds parameter; doc.ts's persistedLinkIds and applier.ts's call to it are deleted as dead code. A realized collision (an accepted, quantified residual risk -- ~1.1e-16 per pair, ~2.2e-12 at 200 links, ~5.5e-5 even at an unrealistic 1,000,000 links) is no longer avoided: it is caught by the existing, ordinary, order-independent apply-time acceptsLink/seenLinks id-collision check every other insert_workflow id kind already goes through, rejecting the whole op link_id_collision byte-identically rather than silently reassigning a different id. Tests: test/insert-workflow.test.ts's "numeric link id minting" block is rewritten to assert pure-derivation determinism and realized- collision rejection, replacing the retry-avoidance and id-space- exhaustion cases that tested a mechanism which no longer exists. test/ka4-rejection-byte-identity.test.ts's link_id_collision row is simplified from an exhaustion construction to a single occupied candidate, mirroring the node/definition rows. test/insert-workflow- numeric-link-ids.regression.test.ts drops the cross-scope/save-reload avoidance cases (their premise is gone) and adds the opposite-order convergence regression Christian's review asked for: the same concurrent insert_workflow ops, applied to two fresh replicas forked from one snapshot in opposite orders, project byte-identically, including every link's numeric id -- both at the top level and across a subgraph-definition interior. Docs: ADR-033 gets an "Amendment (2026-09-23)" section describing the pure-derivation design, the quantified residual collision probability, and an honest writeup of the separate, pre-existing (and unaffected by this change either way) connect-vs-insert_workflow cross-mechanism collision gap in applier.ts's claimLinkIdentity -- out of scope to fix here. The EXCEPTIONS.md KA-5 row and INVARIANTS.md's KA-5 section are updated to match. Per Christian's request to ratify a fix he has reviewed rather than pre-approve one, this ADR's status is left "Amended, ratification PENDING" -- not claimed as Accepted or Approved. Invariants: KA-2/KA-4 (arrival-order-independent identity and projection, the property this amendment restores for links), KA-5 (exception updated, ratification left open), ADR-031 (link ids now match every other insert_workflow id kind's purity), KA-3/KA-13 (remap.ts and doc.ts remain state-free; persistedLinkIds removed rather than left unused). Local verification: npm ci, npm run build, npm run check:purity, npm run check:imports, npm run check:pins, npm run check:profile-claims, npm run check:coderabbit, npm run verify:corpus, npm run check:stateless, npm test (1635 passed, 2 skipped), npm run test:exhaustive, npm run typecheck, npm run lint -- all green. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01UqyourCBfi97xzpBx2W7oY --- docs/INVARIANTS.md | 26 +- ...DR-033-insert-workflow-numeric-link-ids.md | 150 ++++++++- docs/decisions/EXCEPTIONS.md | 2 +- src/applier.ts | 18 +- src/doc.ts | 42 --- src/remap.ts | 128 ++++---- ...rkflow-numeric-link-ids.regression.test.ts | 309 ++++++++---------- test/insert-workflow.test.ts | 179 +++------- test/ka4-rejection-byte-identity.test.ts | 52 +-- 9 files changed, 435 insertions(+), 471 deletions(-) diff --git a/docs/INVARIANTS.md b/docs/INVARIANTS.md index 510656c..9002e05 100644 --- a/docs/INVARIANTS.md +++ b/docs/INVARIANTS.md @@ -43,18 +43,20 @@ The *rejection* half — "a rejected op leaves the doc untouched", stated in the **Why:** Offline peers must create entities without a central allocator. **Enforced by:** `insert_workflow` derives every carried ID inside the applier from the immutable `op_id`, scope, kind, and raw ID (ADR-031); producers send raw workflows and perform no allocation against document state. General producer randomness remains **UNGUARDED — see roadmap**. -**Logged exception (2026-09-22, ADR-033):** a link's derived id is the ONE -carried-ID kind this rule's "no allocation against document state" clause -does not hold for. ComfyUI_frontend's `LinkId` is a branded `number`, unlike -the string-or-number `NodeId`, so `insert_workflow`'s usual pure string -derivation cannot satisfy it; `remap.ts`'s `derivedLinkId` mints a 52-bit -hash candidate and verifies it against `doc.ts`'s `persistedLinkIds` (every -numeric link id already in the document, top-level and every subgraph -definition interior) plus every id already minted in the same call, retrying -past a collision up to a bounded attempt cap before failing loudly. See -`docs/decisions/EXCEPTIONS.md`'s KA-5 row for the full reasoning and the -residual, deliberately-accepted gap for replicas that have never exchanged -the op in question. +**Logged exception (2026-09-22, ADR-033; amended 2026-09-23, ratification +PENDING):** a link's derived id is the ONE carried-ID kind whose numeric type +this rule's "53-bit random mint" clause is worth stating explicitly for. +ComfyUI_frontend's `LinkId` is a branded `number`, unlike the +string-or-number `NodeId`, so `insert_workflow`'s usual pure string +derivation cannot satisfy it; `remap.ts`'s `derivedLinkId` mints a candidate +purely from the op's own content (SHA-256 of the ADR-031 seed, folded into +the full `[1, Number.MAX_SAFE_INTEGER]` range) with **no document read and +no retry** — an earlier revision retried against `doc.ts`'s +`persistedLinkIds`, which review found made link identity depend on local +arrival order (KA-2), so that retry and `persistedLinkIds` were both +removed. See `docs/decisions/EXCEPTIONS.md`'s KA-5 row and ADR-033's +"Amendment (2026-09-23)" section for the full reasoning, the quantified +residual collision probability, and the pending ratification status. ### KA-6 — Raw struct updates flow host → follower one-way only **Rule:** Followers never write the shared doc. diff --git a/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md b/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md index d0c7518..9a6a72e 100644 --- a/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md +++ b/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md @@ -1,9 +1,9 @@ -# ADR-033: numeric, document-state-verified link ids for `insert_workflow` +# ADR-033: numeric link ids for `insert_workflow` -- **Status:** Accepted -- **Date:** 2026-09-22 -- **Decider:** Christian Byrne -- **Source:** ComfyUI_frontend#18458 (interim 32-bit-hash review) and the Slack thread it was raised in +- **Status:** Amended, ratification PENDING — see "Amendment (2026-09-23)" below. The original decision (2026-09-22, below) shipped a document-state-verified retry mint; Christian Byrne's review of that revision (PR #245) found it broke link identity's order-independence and asked for either a fix plus explicit sign-off on the residual risk, or a return to pure derivation. This amendment takes the pure-derivation path. **The "Accepted" disposition below describes the ORIGINAL decision only** and is superseded by the amendment; the amended design has not yet been reviewed by Christian and this ADR does not claim his approval for it. +- **Date:** 2026-09-22 (original), amended 2026-09-23 +- **Decider:** Christian Byrne (original decision). The amendment's decider is UNASSIGNED pending his review of the amended design. +- **Source:** ComfyUI_frontend#18458 (interim 32-bit-hash review) and the Slack thread it was raised in; the amendment's source is Christian Byrne's PR #245 review (inline comment on `src/remap.ts:65`, review summary "Not merge-ready: link identity becomes arrival-order-dependent"). ## Context @@ -117,8 +117,144 @@ package, KA-2) an accepted risk elsewhere in this same system. - `SCHEMA_VERSION` is unchanged: no document root or reserved key changes, only the VALUES `insert_workflow` was already free to choose for a link id. +## Amendment (2026-09-23): pure derivation, no document-state retry + +**This amendment supersedes the "Decision" and "Deviation from KA-5 / ADR-031" +sections above for the CURRENT behavior.** They are kept as written for +history — they are what actually shipped in the original PR #245 revision — +but `derivedLinkId` no longer works the way they describe. + +### What review found + +Christian Byrne's inline blocker on `src/remap.ts:65`: + +> Blocker: `taken` makes identity depend on local arrival order. If two link +> seeds share the first candidate, A→B yields A=x/B=y while B→A yields +> B=x/A=y, so replicas applying the same op set can project different link +> IDs. Please add an opposite-order convergence case and obtain explicit +> owner ratification for the collision-free ID invariant (KA-5) exception, +> or retain pure derivation. + +This is exactly right, and it is a sharper failure than the interim frontend +hash this ADR originally set out to fix: the retry-against-`taken` design +made the FINAL numeric id a function of arrival order whenever two link +seeds' natural (zeroth) candidates coincided, which the original "Decision" +section's item 2 ("verified against every id the caller can see") describes +as a feature rather than naming as this cost. + +### The amended decision + +`derivedLinkId` (`src/remap.ts`) mints PURELY from the op's own content — +`opId`, `scope`, and the raw link's original id/tuple — with **no document +read, no caller-supplied reservation set, and no retry**. `remapInsertedWorkflowIds` +no longer accepts a `reservedLinkIds` parameter; `applier.ts`'s +`prepareInsertedWorkflow` no longer calls `doc.ts`'s (now-deleted) +`persistedLinkIds`. The same `(wf, opId)` pair always produces byte-identical +output — including every link's numeric id — on any replica, in any +document state, regardless of what other ops have or have not been applied +yet, at the top-level scope AND at every subgraph-definition interior scope. +This restores ADR-031's "producers do not inspect document state or remap +ids" for links, closing the gap this ADR's original "Deviation" section +opened. + +The mint also widens its candidate from 52 to the FULL safe-integer range: +it hashes the ADR-031 seed with SHA-256, takes 32 hex characters (128 bits, +comfortably enough to keep the reduction below close to uniform), and folds +that down via `% (Number.MAX_SAFE_INTEGER)` into `[1, Number.MAX_SAFE_INTEGER]` +— `[1, 2^53 - 1]`, all of it, rather than the narrower `2^52` window the +original revision used. `Number.isSafeInteger` (used elsewhere in this +package, e.g. `doc.ts`'s deleted `persistedLinkIds` and `applier.ts`'s +`numericId`) accepts exactly this range, and it round-trips losslessly +through `JSON.stringify`/`JSON.parse` and Yjs storage, which is what a +saved-and-reloaded workflow file and a `Y.Map` value both need. + +### Residual collision probability + +With `N = 2^53 - 1 = 9,007,199,254,740,991` possible candidates and +(to first order, since the modulus reduction is close to uniform) two +INDEPENDENTLY derived candidates, the probability that a specific pair +collides is `1/N ≈ 1.11 × 10⁻¹⁶`. For `k` derived link ids all drawn from +this space, the birthday-bound probability that ANY two collide is +approximately `k² / (2N)`. At `k = 200` (this package's own scale test) that +is about `2.2 × 10⁻¹²`; at `k = 1,000,000` (unrealistically many links in one +document) it is about `5.5 × 10⁻⁵` — still far below anything a real +workflow reaches. This is the same order of magnitude this package already +accepts for a UUID4 `op_id` collision (KA-2, 122 bits of entropy per id, +so an equivalent `k²/(2N)` bound at real-world `k` is smaller still, but +both are "cosmically unlikely, not impossible" risks this package already +lives with elsewhere). + +A REALIZED collision — a freshly derived link id that equals one already +persisted in the target document, or one already derived earlier in the +SAME `insert_workflow` call — is not specially detected or avoided by the +mint (that would reintroduce the document read this amendment removes). It +is caught, if and when it happens, by the ordinary apply-time id-collision +handling every other `insert_workflow` id kind already goes through +(`applier.ts`'s `acceptsLink`, and the `seenLinks` intra-op duplicate check +ahead of it) — the whole op is rejected `link_id_collision`, byte-identically +and without consuming the `op_id` (KA-4), rather than the op silently +picking a different id for the colliding link. This trades a vanishingly +small chance of a LOUD, safe rejection for the arrival-order-dependent +SILENT divergence the retry design risked — the right trade, since a loud +failure that never fires in practice is strictly better than a silent one +that can. + +### Cross-mechanism interaction with `connect` (a separate, pre-existing risk) + +A `connect` op's client-minted `link_id` can, in principle, coincide with a +numeric id `insert_workflow` derives for an unrelated link — a SEPARATE risk +from the one this amendment addresses, raised independently in the Slack +thread linked above. `applier.ts`'s `claimLinkIdentity` (the `connect` +register) does not check for this at all: it deletes and replaces whatever +currently occupies that numeric key with no LWW gate when no prior +`("link", …)`-register stamp exists for it, which is always true for a key +an `insert_workflow` link occupies (that link's stamp lives under a +DIFFERENT register, `("insert_workflow_link", key)`). This asymmetry +predates this ADR and this amendment equally: the ORIGINAL document-verified +retry mint only ever avoided this collision in the one arrival order where +the `connect` had already landed before the `insert_workflow` mint ran on +that replica (in which case `persistedLinkIds` saw it and the retry skipped +past it, applying `insert_workflow` cleanly with a different id); in the +opposite order (`insert_workflow` lands first, `connect` arrives later) the +retry design offered no protection at all, because it cannot see a future +op's client-chosen id — `claimLinkIdentity` would silently overwrite the +`insert_workflow` link exactly as it does today. That is itself a real, +order-dependent divergence between replicas that was already present before +this amendment. + +The amendment does not fix this — it is explicitly out of scope for this PR, +and fixing it would mean changing `claimLinkIdentity` to recognize an +`insert_workflow`-sourced incumbent, not anything in `remap.ts`. What the +amendment changes is narrower: removing the retry removes the one arrival +order (`connect`-then-`insert_workflow`) where the OLD design offered +partial, order-dependent protection, so that order now also rejects loudly +(`link_id_collision`) rather than silently succeeding with a different id. +The genuinely dangerous order (`insert_workflow`-then-`connect`, silent +overwrite) is unaffected by this amendment either way, and both orders +require an engineered or astronomically unlucky collision between a +client-chosen `link_id` and a 53-bit hash output to occur at all. This is +recorded here, honestly, as a known, unresolved, pre-existing gap — not +claimed as fixed and not newly introduced by this amendment. + +### Consequences of the amendment + +- The "Consequences" section's `acceptsLink`/`link_id_collision`-reachability + bullet above is corrected: that check IS reachable via realistic input + now (any real collision reaches it, since nothing avoids one), not merely + via deliberate id-space exhaustion — there is no more exhaustion to + construct, because there is no more retry sequence. +- `test/insert-workflow.test.ts`'s "numeric link id minting (ADR-033, pure + derivation)" block and + `test/insert-workflow-numeric-link-ids.regression.test.ts`'s + "opposite-order convergence" block replace the retry/exhaustion-focused + tests the original revision added, per Christian's request for an + opposite-order convergence case. +- The **Glossary**'s "Reservation set" entry below no longer applies to the + shipped mint; it is kept as history since the "Decision" section above + still uses the term. + ## Glossary - **Derived id:** ADR-031's deterministic string, `insert::::`. -- **Candidate:** one hashed, not-yet-verified numeric value `derivedLinkId` tries during a mint. -- **Reservation set:** the ids a mint call must not choose — persisted document ids plus ids already minted in the same call. +- **Candidate:** the single hashed numeric value `derivedLinkId` computes for one link (post-amendment: there is exactly one, never retried). +- **Reservation set (historical, pre-amendment only):** the ids the original retry-based mint call was required not to choose — persisted document ids plus ids already minted in the same call. Not part of the amended design. diff --git a/docs/decisions/EXCEPTIONS.md b/docs/decisions/EXCEPTIONS.md index 7715247..7c4677e 100644 --- a/docs/decisions/EXCEPTIONS.md +++ b/docs/decisions/EXCEPTIONS.md @@ -21,4 +21,4 @@ is retained as history, not authorization to accept unprojectable writes. | 2026-08-21 (PROPOSED) | KA-11 | **The snapshot read surface's fail-closed gate is CONDITIONAL**: `src/read.ts` refuses a document carrying content under an unreadable schema, but returns its empty value for a document carrying nothing, where `project()` refuses that document too. Deliberate — ADR-004's follower reads a root-less document before its first frame and there is no content there to mis-key — but it is a read path that answers where a sibling read path throws, so it is recorded rather than left in source comments. Normative text: Amendment A12. The divergence is disposition-only; both paths use `readSchemaVersion` as the predicate. Mitigated by asking "carries content" against the struct store rather than the v1 root names, so a v2 that renamed its roots is refused rather than read as empty. | `src/read.ts` `assertSnapshotReadable`/`carriesContent`; schema §10 + Amendment A12 | Owner: UNASSIGNED. Sunset PROPOSED, not agreed: revisit when `SCHEMA_VERSION` first moves past 1, since a real v2 makes the disposition split observable to a deployed follower for the first time. | | ~~2026-08-27 (PROPOSED)~~ **RESOLVED 2026-08-28** | KA-3 | ~~**A promoted `connect` (`grow.promoted: true`, schema Amendment A15) is LWW-gated here on `("input", to_node, "grow", name)` and moves the materialized slot's `grow_id` to the winning link; comfy-cli's `_apply_connect` gates neither.**~~ **Sunset condition met: comfy-cli PR #818 (amendment v1.5, `ba0b0b92abcc86b01e8a6704d07088f92afe7aa7`) gates the same register, spelled with the FULL declared name on both sides, and moves `grow_id` to the winner.** Struck rather than deleted so the exception's life stays legible. Both spell the register identically (`_write_target`), so the divergence is in GATING only and is observable only under concurrency comfy-cli's single-writer apply never sees — the corpus session it generated replays green. Gating is required for §2.5 convergence (two connects into one declared input contend for one slot exactly as two concrete connects do, A1). | `src/applier.ts` `claimPromotedInput`; schema Amendment A15 | Owner: UNASSIGNED. Sunset PROPOSED, not agreed: comfy-cli `_lww_gate`s promoted grows on the same register (asked for in the PR that lands A15). | | 2026-09-21 (APPROVED) | FC-10 | `set_node_field` is a package-local, provisional operation because post-creation title and flag changes otherwise have no non-clobbering write path. The owner requested consolidation on the general per-field architecture rather than the competing title-only operation. Its field set and value types are closed, its incarnation semantics match `set_widget`, and a language-neutral conformance session is included here; the upstream comfy-cli vocabulary has not yet adopted the kind. | `src/types.ts`, `src/applier.ts`, `src/stamps.ts`, `src/collab-trace.ts`; schema Amendment A21; ADR-032 | Owner: Christian Byrne. Sunset: amend comfy-cli's frozen vocabulary and pin the adopted definition here, reconciling the name/payload if upstream differs. | -| 2026-09-22 (APPROVED) | KA-5 | **`insert_workflow`'s numeric link-id mint (`remap.ts`'s `derivedLinkId`) consults mutable document state, unlike every other id this op derives.** ComfyUI_frontend's `LinkId` is a branded `number` (unlike the string-or-number `NodeId`), so ADR-031's pure, op-id-derived STRING scheme cannot satisfy it; the frontend's interim 32-bit-hash workaround (ComfyUI_frontend#18458) measurably collided (~0.01-1% at realistic link counts) and, worse, its "reserved id range" did not survive a save/reload once the hashed id was persisted into the workflow file. `derivedLinkId` hashes the same ADR-031 seed to a 52-bit candidate, then verifies it against every numeric link id already persisted in the target document (`doc.ts`'s `persistedLinkIds`, top-level and every subgraph-definition interior, at any depth) plus every id already minted earlier in the same call, retrying (bounded, `MAX_LINK_ID_MINT_ATTEMPTS = 256`) past a collision rather than accepting one. This is a genuine, information-theoretic necessity, not a preference: an unbounded set of possible (op, link) pairs cannot be encoded injectively into a fixed 53-bit space without consulting SOME external state, unlike the string form's unbounded output space. Residual gap, deliberately accepted: two replicas that have never exchanged this specific op and each mint from their own differing locally-visible document state could in principle choose colliding candidates — bounded by the same ~2^-52-per-pair probability this package already accepts for UUID4 `op_id` collisions (KA-2) elsewhere. `link_id_collision`'s existing insert-time check becomes unreachable for links via any realistically-shaped input as a direct, intended consequence (the mint now actively avoids what that check would have caught) and remains reachable, and tested, only via deliberate id-space exhaustion. | `src/remap.ts` `derivedLinkId`/`candidateLinkId`; `src/doc.ts` `persistedLinkIds`; `src/applier.ts` `prepareInsertedWorkflow`; ADR-033 | Owner: Christian Byrne. Sunset: revisit only if ComfyUI_frontend widens `LinkId` to `string \| number` (matching `NodeId`), at which point the string form ADR-031 already gives every other kind becomes sufficient for links too and this exception can be retired. | +| ~~2026-09-22 (APPROVED)~~ **AMENDED 2026-09-23 (PROPOSED, not yet ratified)** | KA-5 | ~~`insert_workflow`'s numeric link-id mint consulted mutable document state (a document-verified retry), unlike every other id this op derives.~~ Christian Byrne's review of that design (PR #245, inline on `src/remap.ts:65`) found it made link identity depend on LOCAL ARRIVAL ORDER — two replicas applying the same concurrent op set in opposite orders could project different link ids for the same logical links, the exact class of bug KA-4/KA-2 exist to rule out — and asked for either a fix plus explicit ratification, or a return to pure derivation. **This row now describes the amended design: `derivedLinkId` mints PURELY from the op's own content (`opId`, `scope`, the raw link), with NO document read, no caller-supplied reservation, and no retry.** It spends the FULL safe-integer range (`[1, Number.MAX_SAFE_INTEGER]`, 53 bits, widened from the original's 52-bit window) rather than checking for a collision. ComfyUI_frontend's `LinkId` (a branded `number`, unlike the string-or-number `NodeId`) is still why a link needs a numeric mint at all — ADR-031's pure STRING scheme still can't satisfy it — but the mint no longer tries to guarantee collision-freedom against the document; it accepts a quantified residual risk instead. Residual probability: two independently derived 53-bit candidates collide with probability `1/(2^53-1) ≈ 1.11×10⁻¹⁶` per pair; for `k` links minted into one document the birthday-bound chance ANY two collide is `≈ k²/2^54` — about `2.2×10⁻¹²` at `k=200` (this package's own scale test), about `5.5×10⁻⁵` even at an unrealistic `k=1,000,000` — the same order of "cosmically unlikely, not impossible" risk this package already accepts for UUID4 `op_id` collisions (KA-2). A REALIZED collision is not specially detected: it is caught by `applier.ts`'s pre-existing `acceptsLink`/`seenLinks` apply-time id-collision handling (the same ordinary, order-independent LWW-gated check every other `insert_workflow` id kind already goes through), rejecting the whole op `link_id_collision` byte-identically rather than silently reassigning a different id — this check is REACHABLE via realistic input now (not merely via deliberate id-space exhaustion, since there is no more exhaustion to construct). **Separately, a pre-existing, unresolved gap** (out of scope for this amendment): a `connect` op's client-minted `link_id` can collide with an `insert_workflow`-derived numeric id, and `applier.ts`'s `claimLinkIdentity` does not guard against it at all, silently overwriting the `insert_workflow` link when `connect` arrives second — this predates and is unaffected by this amendment either way (the removed retry only ever protected the OPPOSITE arrival order, and only when the `connect` had already landed on that replica). See ADR-033's "Amendment (2026-09-23)" section for the full writeup, including this cross-mechanism note. | `src/remap.ts` `derivedLinkId`; `src/applier.ts` `prepareInsertedWorkflow`, `acceptsLink`, `claimLinkIdentity`; ADR-033 | Owner: Christian Byrne. **Ratification of THIS amended (pure-derivation) design is PENDING his review — not yet agreed, per his PR #245 request to review a fix before ratifying it rather than pre-approving one.** Sunset (unchanged from the original row): revisit only if ComfyUI_frontend widens `LinkId` to `string \| number` (matching `NodeId`), at which point ADR-031's string form becomes sufficient for links too and this exception can be retired outright. | diff --git a/src/applier.ts b/src/applier.ts index 22aa994..702142e 100644 --- a/src/applier.ts +++ b/src/applier.ts @@ -98,7 +98,6 @@ import { mset, nodesMap, nodeIncarnation, - persistedLinkIds, resolveDefinition, stampsMap, widgetStorageOf, @@ -986,7 +985,7 @@ function validateDefinitionInputs(subgraphs: unknown[], path = "workflow.definit }); } -function prepareInsertedWorkflow(doc: Y.Doc, op: InsertWorkflowOp): Record { +function prepareInsertedWorkflow(op: InsertWorkflowOp): Record { const workflow = scrubPrivateKeys(op.workflow) as unknown; if (typeof workflow !== "object" || workflow === null || Array.isArray(workflow)) { throw new OpRejectedError("malformed_op", "insert_workflow: workflow must be an object"); @@ -1008,21 +1007,20 @@ function prepareInsertedWorkflow(doc: Y.Doc, op: InsertWorkflowOp): Record; } function applyInsertWorkflow(doc: Y.Doc, op: InsertWorkflowOp, catalog?: WidgetCatalog): SuccessfulOutcome { - const wf = prepareInsertedWorkflow(doc, op); + const wf = prepareInsertedWorkflow(op); const remappedDefinitions = wf["definitions"] as { subgraphs?: unknown[] } | undefined; const remappedSubgraphs = remappedDefinitions?.subgraphs ?? []; diff --git a/src/doc.ts b/src/doc.ts index 463135d..7d83c65 100644 --- a/src/doc.ts +++ b/src/doc.ts @@ -285,48 +285,6 @@ export function initDoc(doc: Y.Doc, catalogVersion = ""): void { }); } -/** - * Every numeric link id currently persisted in `doc`: every key of the - * top-level `links` map, and every key of every subgraph definition's own - * interior `links` map, at any nesting depth (mirrors - * `definitionIdExistsOutsideRoot`'s recursive-definitions walk in - * `applier.ts`). Non-numeric keys (an explicit string id, or a positional - * `#0`/`base~1` key `mint.ts`'s `mintDefinitionLinks` assigns to an ID-LESS - * interior link) are not part of the numeric-link-id space and are skipped. - * - * `insert_workflow`'s numeric link-id mint (`remap.ts`'s `derivedLinkId`) - * uses this as the reservation a freshly minted link id must never land on — - * see ADR-033 and the KA-5 exception logged in `docs/decisions/EXCEPTIONS.md` - * for why this ONE id domain, unlike every other id `insert_workflow` - * derives, consults mutable document state. - */ -export function persistedLinkIds(doc: Y.Doc): Set { - const ids = new Set(); - const addNumericKeys = (map: Y.Map): void => { - map.forEach((_value, key) => { - // Unsigned-decimal keys only (mirrors `applier.ts`'s `numericId`): no - // leading zero/sign/hex/float noise from a non-numeric explicit or - // positional interior-link key coincidentally parsing under `Number()`. - if (!/^\d+$/.test(key)) return; - const numeric = Number(key); - if (Number.isSafeInteger(numeric)) ids.add(numeric); - }); - }; - addNumericKeys(linksMap(doc)); - const walkDefinitions = (defs: Y.Map): void => { - defs.forEach((definition) => { - if (!(definition instanceof Y.Map)) return; - const links = definition.get("links"); - if (links instanceof Y.Map) addNumericKeys(links); - const nestedContainer = definition.get("definitions"); - const nested = nestedContainer instanceof Y.Map ? nestedContainer.get("subgraphs") : undefined; - if (nested instanceof Y.Map) walkDefinitions(nested); - }); - }; - walkDefinitions(definitionsMap(doc) as unknown as Y.Map); - return ids; -} - // --------------------------------------------------------------------------- // Mutation instrumentation (schema §11 bounded-writes conformance) // diff --git a/src/remap.ts b/src/remap.ts index d78a0af..4abb66d 100644 --- a/src/remap.ts +++ b/src/remap.ts @@ -1,4 +1,4 @@ -import { OpRejectedError, type WorkflowJSON, type WorkflowNode } from "./types.js"; +import type { WorkflowJSON, WorkflowNode } from "./types.js"; import { sha256Hex } from "./digest.js"; function derivedId(opId: string, scope: string, kind: string, original: unknown): string { @@ -14,63 +14,59 @@ function derivedDefinitionId(opId: string, scope: string, original: unknown): st * A newly derived link id must be a real `number` — ComfyUI_frontend's * `LinkId` is `number & { __brand }`, unlike `NodeId` (`string | number`), so * the string form `derivedId` gives every other kind cannot satisfy it - * (ComfyUI_frontend#18458). Hex digits kept from the digest: comfortably - * inside `Number.MAX_SAFE_INTEGER` (2^53 - 1) even after the `+1` below, so - * the candidate is always a lossless JS integer. - */ -const LINK_ID_HEX_DIGITS = 13; // 52 bits - -/** - * Bounded retries per link before minting gives up (ADR-033). Collision - * probability per attempt is `taken.size / 2^52`; any realistic document — - * even one with millions of live links — clears this on the first or second - * attempt. The bound exists so a pathological/adversarial reservation set - * (see `docs/decisions/EXCEPTIONS.md`'s KA-5 row) fails LOUDLY, byte-identically, - * and in bounded work, rather than looping. + * (ComfyUI_frontend#18458). + * + * `derivedLinkId` folds the digest into the FULL safe-integer range + * (`[1, Number.MAX_SAFE_INTEGER]`, i.e. `[1, 2^53 - 1]`) rather than the + * narrower 52-bit window an earlier revision used, so the mint spends every + * bit of entropy `LinkId`'s numeric type can safely hold (ADR-033 amendment, + * review discussion on PR #245). 32 hex characters (128 bits) are taken from + * the digest before the modulus is applied so the reduction to 53 bits stays + * close to uniform; sha256Hex's 64-character output has plenty to spare. */ -export const MAX_LINK_ID_MINT_ATTEMPTS = 256; +const LINK_ID_HEX_DIGITS = 32; // 128 bits, folded down to 53 by SAFE_INTEGER_MODULUS below. -/** Deterministic 52-bit candidate for the `attempt`'th try of `seed`. Never 0. */ -function candidateLinkId(seed: string, attempt: number): number { - const hex = sha256Hex(`${seed}#${String(attempt)}`).slice(0, LINK_ID_HEX_DIGITS); - return Number(BigInt(`0x${hex}`)) + 1; -} +/** `2^53 - 1` as a `BigInt`, i.e. `Number.MAX_SAFE_INTEGER` — every candidate lands in `[1, MAX_SAFE_INTEGER]`. */ +const SAFE_INTEGER_MODULUS = BigInt(Number.MAX_SAFE_INTEGER); /** - * Mint a genuinely unique numeric id for a link `insert_workflow` carries. + * Mint a numeric id for a link `insert_workflow` carries — PURELY as a + * function of the op's own content (`opId`, `scope`, `original`), with NO + * document-state dependency (ADR-033 amendment, superseding the + * document-verified retry mint the original revision of this PR shipped). * - * Deterministic given (`opId`, `scope`, `original`) and `taken`'s state at - * the time of the call: the same inputs always retry through the same - * candidate sequence, and the FIRST one absent from `taken` wins and is - * immediately reserved into `taken` so a later link in the same insertion - * (this scope or any other) can never mint the same value. This is what - * makes the guarantee genuine rather than probabilistic — every candidate is - * checked against every id the caller told this function about, not merely - * assumed unlikely to collide (ComfyUI_frontend#18458's review: a 32-bit - * hash collided at a measured ~0.01-1% rate at realistic link counts). + * Christian Byrne's review of the original revision (PR #245, inline on this + * file) found that the retry-against-`taken` design made link identity + * depend on LOCAL ARRIVAL ORDER: if two link seeds shared the same first + * candidate, applying them A-then-B minted A the natural candidate and B the + * next one, while applying B-then-A minted B the natural candidate and A the + * next one — two replicas that apply the same op set in different orders + * could project genuinely different link ids for the same logical links, + * which is exactly the class of bug KA-4 (deterministic/idempotent + * projection) and KA-2 (order-independent identity) exist to rule out. * - * `taken` is caller-supplied rather than read here (ADR-033): this module - * stays free of `Y.Doc`/Yjs entirely (KA-3), and the caller decides what - * "already spoken for" means — `applier.ts` seeds it from - * `doc.ts`'s `persistedLinkIds()` before the first call in one - * `insert_workflow` application, so the reservation also covers ids a PRIOR - * `insert_workflow` (or a native `connect`/`mint()`) already committed to - * this same document, which is exactly the cross-op collision - * ComfyUI_frontend#18458 reproduced against the interim 32-bit hash. + * The fix removes the retry (and the document read it required) entirely: + * the SAME op, applied to ANY document state, in ANY order relative to any + * other op, always derives the SAME numeric id — because nothing but the + * op's own fields feeds the hash. This restores ADR-031's "producers do not + * inspect document state or remap ids" for links too, at the cost of the + * "checked collision-free against this document" property the retry design + * offered: two derived ids (or a derived id and something already persisted) + * can now coincide, with probability bounded by the ~53 bits of entropy + * below (see the KA-5 row in `docs/decisions/EXCEPTIONS.md` for the exact + * figure and why it is an accepted residual risk rather than a defect). + * `applier.ts`'s existing `acceptsLink` still catches a REALIZED collision + * against an already-persisted link at apply time (the same LWW-gated check + * every other `insert_workflow` id kind already goes through) — that check + * is ordinary id-collision handling this package has always had, not new + * state-dependence introduced here, and it resolves by the op-order- + * independent `[base_version, actor, op_id]` stamp comparison, not by + * arrival order. */ -function derivedLinkId(opId: string, scope: string, original: unknown, taken: Set): number { +function derivedLinkId(opId: string, scope: string, original: unknown): number { const seed = derivedId(opId, scope, "link", original); - for (let attempt = 0; attempt < MAX_LINK_ID_MINT_ATTEMPTS; attempt++) { - const candidate = candidateLinkId(seed, attempt); - if (!taken.has(candidate)) { - taken.add(candidate); - return candidate; - } - } - throw new OpRejectedError( - "link_id_collision", - `insert_workflow: could not mint a collision-free numeric id for link '${String(original)}' (scope '${scope}') after ${String(MAX_LINK_ID_MINT_ATTEMPTS)} attempts`, - ); + const hex = sha256Hex(seed).slice(0, LINK_ID_HEX_DIGITS); + return Number((BigInt(`0x${hex}`) % SAFE_INTEGER_MODULUS) + 1n); } function linkEndpoints(link: unknown): [unknown, unknown] | undefined { @@ -179,7 +175,6 @@ function remapGraph( scope: string, definitionIds: Map, dropDanglingLinks: boolean, - takenLinkIds: Set, isDefinitionInterior = false, interiorNodeIds: (definitionId: string) => Map | undefined = () => undefined, ): void { @@ -198,7 +193,7 @@ function remapGraph( }); for (const link of links) { const id = linkId(link); - linkIds.set(normalizedId(id), derivedLinkId(opId, scope, id, takenLinkIds)); + linkIds.set(normalizedId(id), derivedLinkId(opId, scope, id)); } for (const node of nodes) { @@ -243,25 +238,21 @@ function remapGraph( * Deterministically remap every id carried by an insertion op, including * nested definition graphs. * - * `reservedLinkIds` seeds the numeric link-id mint's exclusion set (ADR-033): - * the caller (`applier.ts`) passes every numeric link id already persisted in - * the target document (`doc.ts`'s `persistedLinkIds`) so a freshly minted id - * can never land on one. It defaults to empty for callers with no document - * to consult — direct unit tests, and any caller minting a workflow that has - * never been merged into a doc. One `Set` is shared across the whole - * traversal (root scope and every definition scope, at every nesting depth) - * so two links in the SAME insertion, however they are scoped, can never - * collide with each other either. + * Every id this function derives — node, link, group, and definition — is a + * PURE function of (`opId`, `scope`, `kind`, `original`) alone (ADR-031, + * amended for links by the ADR-033 revision on PR #245). No document, no + * caller-supplied reservation set, and no state from any other call factors + * into the result: the same `(wf, opId)` pair always produces byte-identical + * output, on any replica, regardless of what else that replica's document + * holds or what order concurrent ops arrive in. This is what makes the + * result safe to compute before touching the target document at all — + * `applier.ts` calls this once per `insert_workflow` application with no + * document in scope yet. */ -export function remapInsertedWorkflowIds( - wf: WorkflowJSON, - opId: string, - reservedLinkIds: Iterable = [], -): WorkflowJSON { +export function remapInsertedWorkflowIds(wf: WorkflowJSON, opId: string): WorkflowJSON { const out = structuredClone(wf) as WorkflowJSON & Record; out.links ??= []; out.groups ??= []; - const takenLinkIds = new Set(reservedLinkIds); const subgraphs = (out.definitions?.subgraphs ?? []) as Array>; const definitionIdsByScope = new Map>(); // Interior node id maps keyed by definition scope, computed before any @@ -301,7 +292,6 @@ export function remapInsertedWorkflowIds( definitionScope, definitionIdsByScope.get(definitionScope)!, true, - takenLinkIds, true, interiorAt(definitionScope), ); @@ -309,7 +299,7 @@ export function remapInsertedWorkflowIds( rewrite(nested, definitionScope); } }; - remapGraph(out, opId, "root", definitionIdsByScope.get("root")!, true, takenLinkIds, false, interiorAt("root")); + remapGraph(out, opId, "root", definitionIdsByScope.get("root")!, true, false, interiorAt("root")); rewrite(subgraphs, "root"); return out; } diff --git a/test/insert-workflow-numeric-link-ids.regression.test.ts b/test/insert-workflow-numeric-link-ids.regression.test.ts index 36505bf..41ac59d 100644 --- a/test/insert-workflow-numeric-link-ids.regression.test.ts +++ b/test/insert-workflow-numeric-link-ids.regression.test.ts @@ -1,5 +1,5 @@ /** - * `insert_workflow` numeric link ids (ADR-033). + * `insert_workflow` numeric link ids (ADR-033, amended). * * ComfyUI_frontend's `LinkId` is `number & { __brand: 'LinkId' }`, unlike the * string-or-number `NodeId`, so `remap.ts`'s usual string `derivedId` @@ -13,31 +13,41 @@ * save/reload — the saved file now contains a genuine numeric link id * inside that range. * - * `remap.ts`'s `derivedLinkId` fixes this at the source: it hashes the same - * op-id-derived seed to a 52-bit candidate, then verifies it against every - * numeric link id already persisted in the target document (`doc.ts`'s - * `persistedLinkIds` — top-level AND every subgraph-definition interior, at - * any depth) plus every id already minted earlier in the same call, retrying - * past a collision (bounded, `MAX_LINK_ID_MINT_ATTEMPTS`) rather than - * accepting one. See `docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md` - * and the KA-5 row in `docs/decisions/EXCEPTIONS.md`. + * The FIRST revision of this PR fixed that by minting a 52-bit candidate and + * retrying past any collision with a numeric link id already persisted in + * the target document. Christian Byrne's review (PR #245, inline on + * `src/remap.ts:65`) found that design itself broken in a different way: + * checking the mint against document state made link identity depend on + * LOCAL ARRIVAL ORDER. If two link seeds shared the same first candidate, + * applying them A-then-B minted A the natural candidate and B the next one + * in the retry sequence, while applying B-then-A minted B the natural + * candidate and A the next one — two replicas that apply the same op set in + * different orders could project genuinely different link ids for the same + * logical links. * - * `test/insert-workflow.test.ts`'s "numeric link id minting (ADR-033)" block - * covers the single-op avoidance and id-space-exhaustion cases directly - * against `occupiedIdVectors`' existing node/definition siblings. This file - * covers the harder cross-cutting properties named in review: collision - * avoidance that spans top-level and subgraph-interior scopes, multiple - * `insert_workflow` ops in one session (sequential and same-batch), the - * exact cross-op collision ComfyUI_frontend#18458 reproduced, scale within - * one op, save/reload round-trip stability, and comfy-multi-player#230's - * definition-interior promoted-linkIds threading with the new numeric ids. + * `remap.ts`'s `derivedLinkId` now mints PURELY from the op's own content — + * `opId`, `scope`, and the raw link — spending the full safe-integer range + * (53 bits, `[1, Number.MAX_SAFE_INTEGER]`) so a genuine collision is an + * accepted, quantified residual risk rather than something the mint tries to + * avoid by reading state. See + * `docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md` and the KA-5 + * row in `docs/decisions/EXCEPTIONS.md`. + * + * `test/insert-workflow.test.ts`'s "numeric link id minting (ADR-033, pure + * derivation)" block covers the single-op determinism and realized-collision + * cases directly against `occupiedIdVectors`' existing node/definition + * siblings. This file covers the harder cross-cutting properties: multiple + * `insert_workflow` ops in one session (sequential and same-batch), scale + * within one op, the project → mint round trip, comfy-multi-player#230's + * definition-interior promoted-linkIds threading with the new numeric ids, + * and — the regression this whole amendment exists for — opposite-order + * convergence across two independently seeded replicas. */ import { describe, expect, it } from "vitest"; import * as Y from "yjs"; import { applyOps, mint, project, type Op, type WidgetCatalog, type WorkflowJSON } from "../src/index.js"; -import { appliedMap, persistedLinkIds } from "../src/doc.js"; -import { remapInsertedWorkflowIds } from "../src/remap.js"; +import { appliedMap } from "../src/doc.js"; const catalog: WidgetCatalog = { types: { @@ -64,7 +74,7 @@ function linkNamed(wf: WorkflowJSON, label: unknown): [unknown, unknown, unknown return found; } -describe("insert_workflow numeric link ids (ADR-033)", () => { +describe("insert_workflow numeric link ids (ADR-033, pure derivation)", () => { it("mints a real number for a top-level link while node ids stay derived strings", () => { const doc = mint({ nodes: [], links: [] }, catalog); const op = opEnvelope({ workflow: { nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], links: [[10, 1, 0, 2, 0, "L"]] } }); @@ -75,12 +85,13 @@ describe("insert_workflow numeric link ids (ADR-033)", () => { for (const node of wf.nodes!) expect(typeof node.id).toBe("string"); }); - it("reproduces and fixes ComfyUI_frontend#18458's exact cross-op scenario: two independent insert_workflow ops never collide", () => { - // The reviewers' reproduction used two distinct 32-char op ids whose - // FNV-1a-folded-to-32-bit hashes collided. This package's mint uses 52 - // bits AND actively verifies against the document, so two distinct ops — - // regardless of whether their raw hash candidates would have collided — - // can never persist the same numeric link id. + it("two independent insert_workflow ops with distinct op_ids derive independent link ids", () => { + // Distinct `op_id`s feed distinct hash seeds (`derivedId` folds `opId` + // into the seed), so two independently authored ops naturally land on + // different 53-bit candidates — not because either mint avoids the + // other (it reads no document state at all), but because the space is + // wide enough that an accidental collision between UNRELATED ops is the + // same order of unlikely as a UUID4 `op_id` collision (KA-2). const doc = mint({ nodes: [], links: [] }, catalog); const opA = opEnvelope({ workflow: { @@ -103,18 +114,18 @@ describe("insert_workflow numeric link ids (ADR-033)", () => { const linkA = linkNamed(wf, "op-a-link"); const linkB = linkNamed(wf, "op-b-link"); expect(linkA[0]).not.toBe(linkB[0]); - // Op A's wire must not have been replaced by op B's (the reported bug): - // both endpoints resolve to their OWN op's nodes. + // Op A's wire must not have been replaced by op B's: both endpoints + // resolve to their OWN op's nodes. const nodeIds = new Set(wf.nodes!.map((n) => n.id)); expect(nodeIds.has(linkA[1] as string)).toBe(true); expect(nodeIds.has(linkB[1] as string)).toBe(true); }); - it("avoids colliding with a numeric link id in the SAME batch's other insert_workflow op", () => { - // Same as the cross-op case above, but delivered as ONE `applyOps` batch - // rather than two sequential calls — op B's mint must see op A's - // already-committed link (from the earlier `doc.transact` in the same - // batch) even though nothing has round-tripped through a snapshot yet. + it("two insert_workflow ops delivered in the SAME batch still derive independent link ids", () => { + // Delivered as ONE `applyOps` batch rather than two sequential calls. + // Neither op's mint threads any state from the other — each derives its + // link id purely from its own (opId, scope, original) — so batching + // changes nothing about the result. const doc = mint({ nodes: [], links: [] }, catalog); const opA = opEnvelope({ workflow: { nodes: [{ id: 111, type: "Src" }, { id: 112, type: "Sink" }], links: [[301, 111, 0, 112, 0, "batch-a"]] }, @@ -132,84 +143,6 @@ describe("insert_workflow numeric link ids (ADR-033)", () => { expect(linkNamed(wf, "batch-a")[0]).not.toBe(linkNamed(wf, "batch-b")[0]); }); - it("a top-level insert avoids a numeric link id already used INSIDE an existing subgraph definition", () => { - // The reservation set spans scopes: a definition's interior links are a - // real collision domain for a top-level candidate too (`doc.ts`'s - // `persistedLinkIds` walks every definition, not just the top-level map). - const opId = "aa11df0c31f9440b9385ac8e01e099b2"; - const occupied = firstCandidate(opId, 501); - const doc = mint( - { - nodes: [{ id: 1, type: "Def" }], - links: [], - definitions: { - subgraphs: [ - { - id: "Def", - nodes: [{ id: "x", type: "Src" }, { id: "y", type: "Sink" }], - links: [{ id: occupied, origin_id: "x", origin_slot: 0, target_id: "y", target_slot: 0, type: "X" }], - }, - ], - }, - } as unknown as WorkflowJSON, - catalog, - ); - - const op = opEnvelope({ - workflow: { nodes: [{ id: 701, type: "Src" }, { id: 702, type: "Sink" }], links: [[501, 701, 0, 702, 0, "top-level"]] }, - op_id: opId, - }); - expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); - const wf = project(doc, catalog); - expect(linkNamed(wf, "top-level")[0]).not.toBe(occupied); - }); - - it("an inserted subgraph definition's interior link avoids a numeric link id already used at the top level", () => { - const opId = "bb11df0c31f9440b9385ac8e01e099b2"; - const workflow = { - nodes: [{ id: 900, type: "NewDef" }], - links: [], - definitions: { - subgraphs: [ - { - id: "NewDef", - nodes: [{ id: "p", type: "Src" }, { id: "q", type: "Sink" }], - links: [{ id: 502, origin_id: "p", origin_slot: 0, target_id: "q", target_slot: 0, type: "X" }], - }, - ], - }, - }; - - // First, discover the definition-interior candidate this exact (opId, - // workflow) pair mints against an EMPTY doc — nothing to avoid yet. - const emptyDoc = mint({ nodes: [], links: [] }, catalog); - const probe = opEnvelope({ workflow, op_id: opId }); - expect(applyOps(emptyDoc, [probe], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); - const naturalDefinition = ( - project(emptyDoc, catalog).definitions as { subgraphs: Array<{ links: Array<{ id: unknown }> }> } - ).subgraphs[0]!; - const naturalCandidate = naturalDefinition.links[0]!.id as number; - - // Now seed a FRESH doc where that exact numeric id is already a - // TOP-LEVEL link (never touched by insert_workflow) and re-run the - // IDENTICAL op. `persistedLinkIds` walks the top-level map, so the - // definition-interior mint below must land somewhere else. - const occupiedDoc = mint( - { nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], links: [[naturalCandidate, 1, 0, 2, 0, "top-incumbent"]] }, - catalog, - ); - const op = opEnvelope({ workflow, op_id: opId }); - expect(applyOps(occupiedDoc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); - - const wf = project(occupiedDoc, catalog); - const definition = (wf.definitions as { subgraphs: Array<{ links: Array<{ id: unknown }> }> }).subgraphs[0]!; - expect(definition.links).toHaveLength(1); - expect(definition.links[0]!.id).not.toBe(naturalCandidate); - expect(typeof definition.links[0]!.id).toBe("number"); - // The top-level incumbent is completely untouched. - expect(linkNamed(wf, "top-incumbent")[0]).toBe(naturalCandidate); - }); - it("mints distinct ids for a large batch of links inserted in one op", () => { const opId = "cc11df0c31f9440b9385ac8e01e099b2"; const count = 200; @@ -218,6 +151,11 @@ describe("insert_workflow numeric link ids (ADR-033)", () => { const doc = mint({ nodes: [], links: [] }, catalog); const op = opEnvelope({ workflow: { nodes, links: rawLinks }, op_id: opId }); + // Each of the 200 raw link ids seeds a distinct hash input, so distinct + // final candidates are the overwhelmingly likely outcome at 53 bits of + // entropy — not a guarantee any dedup mechanism enforces (there is + // none left), just what the birthday bound predicts for 200 draws from + // a ~9-quadrillion-value space. expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); const wf = project(doc, catalog); expect(wf.links).toHaveLength(count); @@ -226,43 +164,6 @@ describe("insert_workflow numeric link ids (ADR-033)", () => { for (const id of ids) expect(typeof id).toBe("number"); }); - it("survives a save/reload round trip: a re-minted doc's persisted numeric link ids are still avoided by a later insert", () => { - // The exact failure the second frontend reviewer identified in the - // interim hash fix: once an inserted link's numeric id is persisted into - // a saved workflow and the doc is re-minted from that JSON, a later - // insert must still avoid it — there is no session-local allocator state - // to lose here, because avoidance is grounded in the DOCUMENT itself. - const firstOpId = "dd11df0c31f9440b9385ac8e01e099b2"; - const original = mint({ nodes: [], links: [] }, catalog); - const firstOp = opEnvelope({ - workflow: { nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], links: [[601, 1, 0, 2, 0, "persisted"]] }, - op_id: firstOpId, - }); - expect(applyOps(original, [firstOp], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); - const persistedWorkflow = project(original, catalog); - const persistedLinkId = linkNamed(persistedWorkflow, "persisted")[0]; - expect(typeof persistedLinkId).toBe("number"); - - // "Save and reload": a FRESH doc, minted from nothing but the projected - // JSON — no memory of the first doc, no in-process allocator survives. - const reloaded = mint(persistedWorkflow, catalog); - expect(persistedLinkIds(reloaded).has(persistedLinkId as number)).toBe(true); - - // A raw id that happens to derive to the SAME candidate the first op - // used (found deterministically, not by chance) must still avoid it. - const secondOpId = "ee11df0c31f9440b9385ac8e01e099b2"; - const collidingRaw = firstCandidate(secondOpId, 601) === persistedLinkId ? 601 : findCollidingRawId(secondOpId, persistedLinkId as number); - if (collidingRaw === undefined) return; // 52 bits: no natural collision is the expected outcome. - - const secondOp = opEnvelope({ - workflow: { nodes: [{ id: 3, type: "Src" }, { id: 4, type: "Sink" }], links: [[collidingRaw, 3, 0, 4, 0, "after-reload"]] }, - op_id: secondOpId, - }); - expect(applyOps(reloaded, [secondOp], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); - const after = project(reloaded, catalog); - expect(linkNamed(after, "after-reload")[0]).not.toBe(persistedLinkId); - }); - it("round-trips numeric link ids byte-for-byte through project → mint with no duplication or corruption", () => { const opId = "ff11df0c31f9440b9385ac8e01e099b2"; const doc = mint({ nodes: [], links: [] }, catalog); @@ -413,21 +314,99 @@ describe("insert_workflow numeric link ids (ADR-033)", () => { const afterReplay = project(doc, catalog); expect(linkNamed(afterReplay, "L")[0]).toBe(mintedId); }); -}); -/** The candidate `derivedLinkId` would pick at root scope for (`opId`, `rawLinkId`) with no reservations. */ -function firstCandidate(opId: string, rawLinkId: number): number { - const remapped = remapInsertedWorkflowIds( - { nodes: [{ id: "a", type: "Src" }, { id: "b", type: "Sink" }], links: [[rawLinkId, "a", 0, "b", 0, "x"]] } as unknown as WorkflowJSON, - opId, - ) as unknown as { links: Array<[number, ...unknown[]]> }; - return remapped.links[0]![0]; -} + describe("opposite-order convergence (the regression Christian Byrne's review found)", () => { + /** + * Apply `ops` in order to a FRESH `Y.Doc` forked from `seed` via + * `Y.applyUpdate` (KA-10: independent replicas fork from one common + * snapshot, never re-seed independently) and return the projection. + */ + function applyToFreshReplica(seed: Uint8Array, ops: Op[]): WorkflowJSON { + const doc = new Y.Doc(); + Y.applyUpdate(doc, seed); + for (const op of ops) { + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + } + return project(doc, catalog); + } + + it("two concurrent insert_workflow ops project byte-identically in either arrival order", () => { + // This is the exact shape of Christian's blocker: two CONCURRENT ops + // (never exchanged with each other's replica before either is + // applied), each carrying links, replayed on two independent replicas + // in opposite order. Under the retry-against-`taken` design this PR's + // first revision shipped, a shared first candidate between a link in + // opA and a link in opB would resolve to opA=x/opB=y on one replica + // and opB=x/opA=y on the other — a genuine, silent projection + // divergence between two replicas that applied the identical op set. + // Pure derivation makes that impossible by construction: neither op's + // mint reads anything but its own content, so its result cannot + // depend on which replica, or which order, it is evaluated in. + const seed = Y.encodeStateAsUpdate(mint({ nodes: [], links: [] }, catalog)); + const opA = opEnvelope({ + workflow: { + nodes: [{ id: 101, type: "Src" }, { id: 102, type: "Sink" }, { id: 103, type: "Sink" }], + links: [ + [201, 101, 0, 102, 0, "a1"], + [202, 101, 0, 103, 0, "a2"], + ], + }, + }); + const opB = opEnvelope({ + workflow: { + nodes: [{ id: 501, type: "Src" }, { id: 502, type: "Sink" }], + links: [[601, 501, 0, 502, 0, "b1"]], + }, + }); + + const forward = applyToFreshReplica(seed, [opA, opB]); + const reverse = applyToFreshReplica(seed, [opB, opA]); + + // Not merely "both applied successfully" -- the WHOLE projected + // document, including every link's numeric id, is byte-for-byte + // identical regardless of order (`project()` sorts by id, so this + // equality is meaningful even though the two replicas wrote their + // Y.Map entries in opposite orders). + expect(forward).toEqual(reverse); + expect(linkNamed(forward, "a1")[0]).toBe(linkNamed(reverse, "a1")[0]); + expect(linkNamed(forward, "a2")[0]).toBe(linkNamed(reverse, "a2")[0]); + expect(linkNamed(forward, "b1")[0]).toBe(linkNamed(reverse, "b1")[0]); + }); -/** Search a small range of raw link ids for one whose first candidate (given `opId`) equals `target`. */ -function findCollidingRawId(opId: string, target: number): number | undefined { - for (let raw = 1; raw < 5000; raw++) { - if (firstCandidate(opId, raw) === target) return raw; - } - return undefined; -} + it("two insert_workflow ops sharing a definition converge identically in either order", () => { + // Extends the same property to a subgraph-definition interior scope + // (this PR's other cross-cutting surface): opA inserts a definition + // and an instance of it; opB is an unrelated top-level insert. Both + // the top-level and the definition-interior link ids must agree + // across orders. + const seed = Y.encodeStateAsUpdate(mint({ nodes: [], links: [] }, catalog)); + const opA = opEnvelope({ + workflow: { + nodes: [{ id: 900, type: "Def" }], + links: [], + definitions: { + subgraphs: [ + { + id: "Def", + nodes: [{ id: "x", type: "Src" }, { id: "y", type: "Sink" }], + links: [{ id: 41, origin_id: "x", origin_slot: 0, target_id: "y", target_slot: 0, type: "X" }], + }, + ], + }, + }, + }); + const opB = opEnvelope({ + workflow: { nodes: [{ id: 701, type: "Src" }, { id: 702, type: "Sink" }], links: [[801, 701, 0, 702, 0, "top-level"]] }, + }); + + const forward = applyToFreshReplica(seed, [opA, opB]); + const reverse = applyToFreshReplica(seed, [opB, opA]); + expect(forward).toEqual(reverse); + + const defLinkId = (wf: WorkflowJSON): unknown => + (wf.definitions as { subgraphs: Array<{ links: Array<{ id: unknown }> }> }).subgraphs[0]!.links[0]!.id; + expect(defLinkId(forward)).toBe(defLinkId(reverse)); + expect(linkNamed(forward, "top-level")[0]).toBe(linkNamed(reverse, "top-level")[0]); + }); + }); +}); diff --git a/test/insert-workflow.test.ts b/test/insert-workflow.test.ts index 3e6d707..4db2989 100644 --- a/test/insert-workflow.test.ts +++ b/test/insert-workflow.test.ts @@ -11,13 +11,17 @@ * the op_id gate, and byte-identical doc on rejection. * * ID allocation belongs to the applier and is derived from the immutable op - * id plus each raw id, never from mutable document state — with ONE - * documented exception (ADR-033, `docs/decisions/EXCEPTIONS.md`'s KA-5 row): - * a link's id must be a real `number` (ComfyUI_frontend#18458 — `LinkId` is - * a branded `number`, unlike the string-or-number `NodeId`), so - * `remap.ts`'s `derivedLinkId` additionally consults the target document's - * already-persisted numeric link ids to guarantee its mint never collides, - * rather than merely hashing into a wide-enough space and hoping. + * id plus each raw id alone, never from mutable document state — including + * for links. An earlier revision of this PR made a link's numeric id + * (ComfyUI_frontend#18458 — `LinkId` is a branded `number`, unlike the + * string-or-number `NodeId`) an exception to that rule, retrying against the + * target document's already-persisted numeric link ids to avoid a collision. + * Review (Christian Byrne, PR #245) found that retry made link identity + * depend on local arrival order, so `remap.ts`'s `derivedLinkId` mints + * PURELY from the op's own content instead (ADR-033, amended), spending the + * full safe-integer range (53 bits) so a genuine collision is an accepted, + * quantified residual risk (`docs/decisions/EXCEPTIONS.md`'s KA-5 row) + * rather than something the mint tries to avoid by reading state. */ import { describe, expect, it } from "vitest"; import * as Y from "yjs"; @@ -37,7 +41,7 @@ import { type WidgetCatalog, type WorkflowJSON, } from "../src/index.js"; -import { MAX_LINK_ID_MINT_ATTEMPTS, remapInsertedWorkflowIds } from "../src/remap.js"; +import { remapInsertedWorkflowIds } from "../src/remap.js"; import { appliedOpIds, noOpIds, rejectedOutcomeWithIndex } from "./apply-result-helpers.js"; const catalog: WidgetCatalog = { @@ -104,38 +108,19 @@ function replaySnapshot(seed: Uint8Array, ops: Op[]): WorkflowJSON { /** * One link's numeric id, as `remap.ts`'s `derivedLinkId` mint would produce - * it for (`opId`, `"root"`, `rawLinkId`) given `reserved` already taken — - * i.e. exactly what `insert_workflow` would write for a doc whose persisted - * numeric link ids are `reserved`. Used to construct the EXACT candidate a - * real apply will also compute (determinism), and to walk the mint's whole - * deterministic attempt sequence for one seed (see `candidateSequence`). + * it for (`opId`, `"root"`, `rawLinkId`) — a pure function of those two + * inputs alone (ADR-033, amended), so this is exactly what a real apply of + * the same (opId, rawLinkId) will also compute, regardless of document + * state or arrival order. */ -function derivedLinkIdForTest(opId: string, rawLinkId: number, reserved: ReadonlySet = new Set()): number { +function derivedLinkIdForTest(opId: string, rawLinkId: number): number { const remapped = remapInsertedWorkflowIds( { nodes: [{ id: "a", type: "Src" }, { id: "b", type: "Sink" }], links: [[rawLinkId, "a", 0, "b", 0, "x"]] } as unknown as WorkflowJSON, opId, - reserved, ) as unknown as { links: Array<[number, ...unknown[]]> }; return remapped.links[0]![0]; } -/** - * The first `count` candidates `derivedLinkId` would try, in order, for - * (`opId`, `"root"`, `rawLinkId`) — walked by feeding each prior candidate - * back in as `reserved`, which forces the mint to skip past it to the next - * one in its deterministic sequence. - */ -function candidateSequence(opId: string, rawLinkId: number, count: number): number[] { - const sequence: number[] = []; - const reserved = new Set(); - for (let i = 0; i < count; i++) { - const next = derivedLinkIdForTest(opId, rawLinkId, reserved); - sequence.push(next); - reserved.add(next); - } - return sequence; -} - const bytes = (doc: Y.Doc): Buffer => Buffer.from(Y.encodeStateAsUpdate(doc)); function ids(wf: WorkflowJSON): unknown[] { @@ -643,12 +628,11 @@ describe("insert_workflow: happy path", () => { const op = insertOp(workflow, { op_id: opId }); seq--; - // The doc starts with no links, so the fix's numeric mint (`remap.ts`'s - // `derivedLinkId`) has nothing to avoid and its first candidate for the - // surviving link (903, "sibling") is exactly what the real apply below - // will also compute — ADR-033 is deterministic given (opId, scope, - // original) and the reservation state, and both calls start from the - // same (empty) reservation. + // `remap.ts`'s `derivedLinkId` is a pure function of (opId, scope, + // original) alone (ADR-033, amended), so the candidate this direct call + // computes for the surviving link (903, "sibling") is exactly what the + // real apply below will also compute — independent of what else the + // target doc holds. const remapped = remapInsertedWorkflowIds(workflow as unknown as WorkflowJSON, opId) as unknown as { links: Array<[number, ...unknown[]]>; }; @@ -911,118 +895,55 @@ describe("insert_workflow: rejection (KA-4 byte identity, op_id absent from appl }); } - describe("numeric link id minting (ADR-033)", () => { - it("mints a fresh numeric id rather than colliding with one already in the doc", () => { + describe("numeric link id minting (ADR-033, pure derivation)", () => { + it("derives the identical numeric id whether or not that id already exists in the target document", () => { + // The whole point of the amended design: the candidate is a function + // of (opId, rawLinkId) ALONE. Computing it against an empty workflow + // (no document in scope at all) and computing it via a real apply + // into a doc that starts with unrelated content must agree exactly — + // there is no document read left in `derivedLinkId` to make them + // differ. const opId = "d641df0c31f9440b9385ac8e01e099b2"; - // The candidate `derivedLinkId` would pick with NO reservations — i.e. - // exactly what a pre-fix, doc-state-oblivious derivation would have - // produced, and exactly what a real apply would ALSO produce if this - // one id were not already occupied. - const wouldBeFirstCandidate = derivedLinkIdForTest(opId, 901); + const expected = derivedLinkIdForTest(opId, 901); - const doc = mint( - { - nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], - links: [[wouldBeFirstCandidate, 1, 0, 2, 0, "incumbent link"]], - }, - catalog, - ); - const beforeIncumbent = project(doc, catalog); + const doc = mint({ nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], links: [[555, 1, 0, 2, 0, "unrelated"]] }, catalog); const op = insertOp( { nodes: [{ id: 701, type: "Src" }, { id: 702, type: "Sink" }], links: [[901, 701, 0, 702, 0, "candidate link"]] }, { op_id: opId }, ); seq--; - const result = applyOps(doc, [op], catalog); - expect(result.outcomes[0]).toMatchObject({ outcome: "applied" }); - - const projected = project(doc, catalog); - const insertedLink = (projected.links as unknown[][]).find((link) => link[5] === "candidate link")!; - const incumbentLink = (projected.links as unknown[][]).find((link) => link[5] === "incumbent link")!; - // The mint skipped the occupied candidate and moved to the next one in - // its deterministic sequence, rather than colliding with or rejecting - // over the incumbent. - expect(insertedLink[0]).toBe(candidateSequence(opId, 901, 2)[1]); - expect(insertedLink[0]).not.toBe(wouldBeFirstCandidate); - // The incumbent (seeded independently of this op, not via - // insert_workflow) is completely untouched. - expect(incumbentLink).toEqual(beforeIncumbent.links![0]); - }); - - it("mints distinct ids for two links in the SAME insertion that would otherwise share a candidate", () => { - // Two DIFFERENT raw link ids can legitimately hash to the same first - // candidate; `derivedLinkId`'s shared `taken` set (threaded through one - // whole `remapInsertedWorkflowIds` call) must still give them distinct - // final ids. Constructed by finding the raw id whose own first - // candidate collides with 901's, rather than assuming one — the - // reservation-set mechanism is what is under test, not luck. - const opId = "e641df0c31f9440b9385ac8e01e099b2"; - const first = derivedLinkIdForTest(opId, 901); - let collidingRawId = -1; - for (let raw = 1; raw < 5000; raw++) { - if (raw !== 901 && derivedLinkIdForTest(opId, raw) === first) { - collidingRawId = raw; - break; - } - } - // No naturally-colliding raw id turned up in the search range — 52 bits - // of entropy makes that the overwhelmingly likely outcome, and it is - // itself evidence the space is wide enough that this scenario needs - // deliberate construction. Skip rather than assert a false positive. - if (collidingRawId === -1) return; - - const doc = mint({ nodes: [], links: [] }, catalog); - const op = insertOp( - { - nodes: [ - { id: 701, type: "Src", outputs: [{ name: "out", links: [901] }] }, - { id: 702, type: "Sink", inputs: [{ name: "in", link: 901 }] }, - { id: 703, type: "Src", outputs: [{ name: "out", links: [collidingRawId] }] }, - { id: 704, type: "Sink", inputs: [{ name: "in", link: collidingRawId }] }, - ], - links: [ - [901, 701, 0, 702, 0, "x"], - [collidingRawId, 703, 0, 704, 0, "y"], - ], - }, - { op_id: opId }, - ); - seq--; - expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); - const projected = project(doc, catalog); - const linkX = (projected.links as unknown[][]).find((link) => link[5] === "x")!; - const linkY = (projected.links as unknown[][]).find((link) => link[5] === "y")!; - expect(linkX[0]).not.toBe(linkY[0]); + const insertedLink = (project(doc, catalog).links as unknown[][]).find((link) => link[5] === "candidate link")!; + expect(insertedLink[0]).toBe(expected); }); - it("rejects link_id_collision when every mint attempt is already reserved (id-space exhaustion)", () => { - const opId = "f641df0c31f9440b9385ac8e01e099b2"; - const rawLinkId = 901; - // Occupy EVERY candidate the mint would ever try for this seed, so the - // bounded retry (ADR-033's `MAX_LINK_ID_MINT_ATTEMPTS`) is exhausted — - // the one scenario this deliberately-bounded loop is a safety valve - // for. This is not a realistic document; it is the adversarial worst - // case named in `docs/decisions/EXCEPTIONS.md`'s KA-5 row. - const exhausted = candidateSequence(opId, rawLinkId, MAX_LINK_ID_MINT_ATTEMPTS); + it("rejects, rather than avoids, a genuine collision with a numeric link id already persisted in the doc", () => { + // Christian Byrne's blocker on the original revision: making the mint + // AVOID an occupied candidate is exactly what made link identity + // arrival-order-dependent. The amended mint never reads the document, + // so an incumbent occupying the derived candidate is no longer + // side-stepped — it is caught by `acceptsLink`'s ordinary, pre-existing + // id-collision check (the same one every other `insert_workflow` id + // kind already goes through), which rejects the WHOLE op rather than + // silently reassigning a different id to the colliding link. + const opId = "d641df0c31f9440b9385ac8e01e099b2"; + const occupied = derivedLinkIdForTest(opId, 901); + const doc = mint( - { - nodes: exhausted.flatMap((id, i) => [{ id: `src-${String(i)}`, type: "Src" }, { id: `sink-${String(i)}`, type: "Sink" }]), - links: exhausted.map((id, i) => [id, `src-${String(i)}`, 0, `sink-${String(i)}`, 0, "occupied"]), - } as unknown as WorkflowJSON, + { nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], links: [[occupied, 1, 0, 2, 0, "incumbent link"]] }, catalog, ); - const beforeBytes = bytes(doc); + const before = bytes(doc); const beforeProjected = project(doc, catalog); const op = insertOp( - { nodes: [{ id: 701, type: "Src" }, { id: 702, type: "Sink" }], links: [[rawLinkId, 701, 0, 702, 0, "candidate link"]] }, + { nodes: [{ id: 701, type: "Src" }, { id: 702, type: "Sink" }], links: [[901, 701, 0, 702, 0, "candidate link"]] }, { op_id: opId }, ); seq--; expect(rejectedOutcomeWithIndex(applyOps(doc, [op], catalog))).toMatchObject({ index: 0, code: "link_id_collision" }); - expect(bytes(doc).equals(beforeBytes)).toBe(true); + expect(bytes(doc).equals(before)).toBe(true); expect(project(doc, catalog)).toEqual(beforeProjected); expect(appliedMap(doc).has(op.op_id)).toBe(false); }); diff --git a/test/ka4-rejection-byte-identity.test.ts b/test/ka4-rejection-byte-identity.test.ts index d81ad66..0de8db9 100644 --- a/test/ka4-rejection-byte-identity.test.ts +++ b/test/ka4-rejection-byte-identity.test.ts @@ -49,23 +49,7 @@ import { type WorkflowJSON, } from "../src/index.js"; import { appliedMap } from "../src/doc.js"; -import { MAX_LINK_ID_MINT_ATTEMPTS, remapInsertedWorkflowIds } from "../src/remap.js"; - -/** - * The numeric id `remap.ts`'s `derivedLinkId` would mint for - * (`opId`, `"root"`, `rawLinkId`) given `reserved` already taken. Walking it - * repeatedly (each prior result fed back as `reserved`) recovers the mint's - * whole deterministic attempt sequence for one seed — used below to - * construct the id-space-exhaustion row (ADR-033). - */ -function derivedLinkIdForRow(opId: string, rawLinkId: number, reserved: ReadonlySet = new Set()): number { - const remapped = remapInsertedWorkflowIds( - { nodes: [{ id: "a", type: "Src" }, { id: "b", type: "Sink" }], links: [[rawLinkId, "a", 0, "b", 0, "x"]] } as unknown as WorkflowJSON, - opId, - reserved, - ) as unknown as { links: Array<[number, ...unknown[]]> }; - return remapped.links[0]![0]; -} +import { remapInsertedWorkflowIds } from "../src/remap.js"; const catalog: WidgetCatalog = { types: { @@ -231,19 +215,16 @@ const CASES: Row[] = [ }, }, { - // A link's numeric id is minted with an active, doc-state-aware - // reservation (ADR-033's `derivedLinkId`), so — unlike the node/definition - // rows above, which collide against the FIRST (and only) id the applier - // ever derives — a single occupied candidate is no longer enough to - // reach this code: the mint simply retries past it (covered by - // `test/insert-workflow.test.ts`'s "mints a fresh numeric id..." case). - // This row instead exhausts every candidate the mint's bounded retry - // will ever try for this seed, which is the ONE scenario - // `MAX_LINK_ID_MINT_ATTEMPTS` exists to fail loudly on rather than loop - // (`docs/decisions/EXCEPTIONS.md`'s KA-5 row) — an adversarial - // reservation set, not a realistic document. + // ADR-033 (amended): `derivedLinkId` is now a PURE function of the op's + // own content, with no retry against document state (removing the + // arrival-order dependence Christian Byrne's review of the original + // revision found — see `remap.ts`'s `derivedLinkId` doc comment). A + // single occupied candidate is therefore sufficient to reach this code, + // exactly like the node/definition rows above: no id-space-exhaustion + // construction is needed or possible anymore, since there is no retry + // sequence to exhaust. kind: "insert_workflow", - why: "every numeric link-id mint attempt is already occupied (id-space exhaustion)", + why: "remapped link id collides with the live tree", code: "link_id_collision", build: () => ({ @@ -253,13 +234,12 @@ const CASES: Row[] = [ }) as Op, seed: (op) => { const workflow = baseWorkflow(); - const reserved = new Set(); - for (let i = 0; i < MAX_LINK_ID_MINT_ATTEMPTS; i++) { - const candidate = derivedLinkIdForRow(op.op_id, 901, reserved); - reserved.add(candidate); - workflow.nodes!.push({ id: `excl-src-${String(i)}`, type: "Src" }, { id: `excl-sink-${String(i)}`, type: "Sink" }); - workflow.links!.push([candidate, `excl-src-${String(i)}`, 0, `excl-sink-${String(i)}`, 0, "occupied"]); - } + const remapped = remapInsertedWorkflowIds((op as { workflow: WorkflowJSON }).workflow, op.op_id) as unknown as { + links: Array<[number, ...unknown[]]>; + }; + const remappedLinkId = remapped.links[0]![0]; + workflow.nodes!.push({ id: 50, type: "Src" }, { id: 51, type: "Sink" }); + workflow.links!.push([remappedLinkId, 50, 0, 51, 0, "incumbent"]); return workflow; }, }, From f6e195ec0008a7b83e76ead80524664eff2617a0 Mon Sep 17 00:00:00 2001 From: bymyself Date: Tue, 22 Sep 2026 18:32:18 -0700 Subject: [PATCH 3/3] docs: ratify ADR-033 pure link-id derivation --- docs/INVARIANTS.md | 6 +++--- docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md | 4 ++-- docs/decisions/EXCEPTIONS.md | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/INVARIANTS.md b/docs/INVARIANTS.md index 9002e05..b9fda1b 100644 --- a/docs/INVARIANTS.md +++ b/docs/INVARIANTS.md @@ -43,8 +43,8 @@ The *rejection* half — "a rejected op leaves the doc untouched", stated in the **Why:** Offline peers must create entities without a central allocator. **Enforced by:** `insert_workflow` derives every carried ID inside the applier from the immutable `op_id`, scope, kind, and raw ID (ADR-031); producers send raw workflows and perform no allocation against document state. General producer randomness remains **UNGUARDED — see roadmap**. -**Logged exception (2026-09-22, ADR-033; amended 2026-09-23, ratification -PENDING):** a link's derived id is the ONE carried-ID kind whose numeric type +**Logged exception (2026-09-22, ADR-033; amended and ratified 2026-09-23):** +a link's derived id is the ONE carried-ID kind whose numeric type this rule's "53-bit random mint" clause is worth stating explicitly for. ComfyUI_frontend's `LinkId` is a branded `number`, unlike the string-or-number `NodeId`, so `insert_workflow`'s usual pure string @@ -56,7 +56,7 @@ no retry** — an earlier revision retried against `doc.ts`'s arrival order (KA-2), so that retry and `persistedLinkIds` were both removed. See `docs/decisions/EXCEPTIONS.md`'s KA-5 row and ADR-033's "Amendment (2026-09-23)" section for the full reasoning, the quantified -residual collision probability, and the pending ratification status. +residual collision probability, and the ratification record. ### KA-6 — Raw struct updates flow host → follower one-way only **Rule:** Followers never write the shared doc. diff --git a/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md b/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md index 9a6a72e..b36945d 100644 --- a/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md +++ b/docs/decisions/ADR-033-insert-workflow-numeric-link-ids.md @@ -1,8 +1,8 @@ # ADR-033: numeric link ids for `insert_workflow` -- **Status:** Amended, ratification PENDING — see "Amendment (2026-09-23)" below. The original decision (2026-09-22, below) shipped a document-state-verified retry mint; Christian Byrne's review of that revision (PR #245) found it broke link identity's order-independence and asked for either a fix plus explicit sign-off on the residual risk, or a return to pure derivation. This amendment takes the pure-derivation path. **The "Accepted" disposition below describes the ORIGINAL decision only** and is superseded by the amendment; the amended design has not yet been reviewed by Christian and this ADR does not claim his approval for it. +- **Status:** Amended and accepted — see "Amendment (2026-09-23)" below. The original decision (2026-09-22, below) shipped a document-state-verified retry mint; Christian Byrne's review of that revision (PR #245) found it broke link identity's order-independence and asked for either a fix plus explicit sign-off on the residual risk, or a return to pure derivation. This amendment takes the pure-derivation path and was ratified after exact-head review of that fix and its opposite-order convergence coverage. **The "Accepted" disposition below describes the ORIGINAL decision only** and is superseded by this accepted amendment. - **Date:** 2026-09-22 (original), amended 2026-09-23 -- **Decider:** Christian Byrne (original decision). The amendment's decider is UNASSIGNED pending his review of the amended design. +- **Decider:** Christian Byrne - **Source:** ComfyUI_frontend#18458 (interim 32-bit-hash review) and the Slack thread it was raised in; the amendment's source is Christian Byrne's PR #245 review (inline comment on `src/remap.ts:65`, review summary "Not merge-ready: link identity becomes arrival-order-dependent"). ## Context diff --git a/docs/decisions/EXCEPTIONS.md b/docs/decisions/EXCEPTIONS.md index 7c4677e..0579b87 100644 --- a/docs/decisions/EXCEPTIONS.md +++ b/docs/decisions/EXCEPTIONS.md @@ -21,4 +21,4 @@ is retained as history, not authorization to accept unprojectable writes. | 2026-08-21 (PROPOSED) | KA-11 | **The snapshot read surface's fail-closed gate is CONDITIONAL**: `src/read.ts` refuses a document carrying content under an unreadable schema, but returns its empty value for a document carrying nothing, where `project()` refuses that document too. Deliberate — ADR-004's follower reads a root-less document before its first frame and there is no content there to mis-key — but it is a read path that answers where a sibling read path throws, so it is recorded rather than left in source comments. Normative text: Amendment A12. The divergence is disposition-only; both paths use `readSchemaVersion` as the predicate. Mitigated by asking "carries content" against the struct store rather than the v1 root names, so a v2 that renamed its roots is refused rather than read as empty. | `src/read.ts` `assertSnapshotReadable`/`carriesContent`; schema §10 + Amendment A12 | Owner: UNASSIGNED. Sunset PROPOSED, not agreed: revisit when `SCHEMA_VERSION` first moves past 1, since a real v2 makes the disposition split observable to a deployed follower for the first time. | | ~~2026-08-27 (PROPOSED)~~ **RESOLVED 2026-08-28** | KA-3 | ~~**A promoted `connect` (`grow.promoted: true`, schema Amendment A15) is LWW-gated here on `("input", to_node, "grow", name)` and moves the materialized slot's `grow_id` to the winning link; comfy-cli's `_apply_connect` gates neither.**~~ **Sunset condition met: comfy-cli PR #818 (amendment v1.5, `ba0b0b92abcc86b01e8a6704d07088f92afe7aa7`) gates the same register, spelled with the FULL declared name on both sides, and moves `grow_id` to the winner.** Struck rather than deleted so the exception's life stays legible. Both spell the register identically (`_write_target`), so the divergence is in GATING only and is observable only under concurrency comfy-cli's single-writer apply never sees — the corpus session it generated replays green. Gating is required for §2.5 convergence (two connects into one declared input contend for one slot exactly as two concrete connects do, A1). | `src/applier.ts` `claimPromotedInput`; schema Amendment A15 | Owner: UNASSIGNED. Sunset PROPOSED, not agreed: comfy-cli `_lww_gate`s promoted grows on the same register (asked for in the PR that lands A15). | | 2026-09-21 (APPROVED) | FC-10 | `set_node_field` is a package-local, provisional operation because post-creation title and flag changes otherwise have no non-clobbering write path. The owner requested consolidation on the general per-field architecture rather than the competing title-only operation. Its field set and value types are closed, its incarnation semantics match `set_widget`, and a language-neutral conformance session is included here; the upstream comfy-cli vocabulary has not yet adopted the kind. | `src/types.ts`, `src/applier.ts`, `src/stamps.ts`, `src/collab-trace.ts`; schema Amendment A21; ADR-032 | Owner: Christian Byrne. Sunset: amend comfy-cli's frozen vocabulary and pin the adopted definition here, reconciling the name/payload if upstream differs. | -| ~~2026-09-22 (APPROVED)~~ **AMENDED 2026-09-23 (PROPOSED, not yet ratified)** | KA-5 | ~~`insert_workflow`'s numeric link-id mint consulted mutable document state (a document-verified retry), unlike every other id this op derives.~~ Christian Byrne's review of that design (PR #245, inline on `src/remap.ts:65`) found it made link identity depend on LOCAL ARRIVAL ORDER — two replicas applying the same concurrent op set in opposite orders could project different link ids for the same logical links, the exact class of bug KA-4/KA-2 exist to rule out — and asked for either a fix plus explicit ratification, or a return to pure derivation. **This row now describes the amended design: `derivedLinkId` mints PURELY from the op's own content (`opId`, `scope`, the raw link), with NO document read, no caller-supplied reservation, and no retry.** It spends the FULL safe-integer range (`[1, Number.MAX_SAFE_INTEGER]`, 53 bits, widened from the original's 52-bit window) rather than checking for a collision. ComfyUI_frontend's `LinkId` (a branded `number`, unlike the string-or-number `NodeId`) is still why a link needs a numeric mint at all — ADR-031's pure STRING scheme still can't satisfy it — but the mint no longer tries to guarantee collision-freedom against the document; it accepts a quantified residual risk instead. Residual probability: two independently derived 53-bit candidates collide with probability `1/(2^53-1) ≈ 1.11×10⁻¹⁶` per pair; for `k` links minted into one document the birthday-bound chance ANY two collide is `≈ k²/2^54` — about `2.2×10⁻¹²` at `k=200` (this package's own scale test), about `5.5×10⁻⁵` even at an unrealistic `k=1,000,000` — the same order of "cosmically unlikely, not impossible" risk this package already accepts for UUID4 `op_id` collisions (KA-2). A REALIZED collision is not specially detected: it is caught by `applier.ts`'s pre-existing `acceptsLink`/`seenLinks` apply-time id-collision handling (the same ordinary, order-independent LWW-gated check every other `insert_workflow` id kind already goes through), rejecting the whole op `link_id_collision` byte-identically rather than silently reassigning a different id — this check is REACHABLE via realistic input now (not merely via deliberate id-space exhaustion, since there is no more exhaustion to construct). **Separately, a pre-existing, unresolved gap** (out of scope for this amendment): a `connect` op's client-minted `link_id` can collide with an `insert_workflow`-derived numeric id, and `applier.ts`'s `claimLinkIdentity` does not guard against it at all, silently overwriting the `insert_workflow` link when `connect` arrives second — this predates and is unaffected by this amendment either way (the removed retry only ever protected the OPPOSITE arrival order, and only when the `connect` had already landed on that replica). See ADR-033's "Amendment (2026-09-23)" section for the full writeup, including this cross-mechanism note. | `src/remap.ts` `derivedLinkId`; `src/applier.ts` `prepareInsertedWorkflow`, `acceptsLink`, `claimLinkIdentity`; ADR-033 | Owner: Christian Byrne. **Ratification of THIS amended (pure-derivation) design is PENDING his review — not yet agreed, per his PR #245 request to review a fix before ratifying it rather than pre-approving one.** Sunset (unchanged from the original row): revisit only if ComfyUI_frontend widens `LinkId` to `string \| number` (matching `NodeId`), at which point ADR-031's string form becomes sufficient for links too and this exception can be retired outright. | +| ~~2026-09-22 (APPROVED)~~ **AMENDED AND APPROVED 2026-09-23** | KA-5 | ~~`insert_workflow`'s numeric link-id mint consulted mutable document state (a document-verified retry), unlike every other id this op derives.~~ Christian Byrne's review of that design (PR #245, inline on `src/remap.ts:65`) found it made link identity depend on LOCAL ARRIVAL ORDER — two replicas applying the same concurrent op set in opposite orders could project different link ids for the same logical links, the exact class of bug KA-4/KA-2 exist to rule out — and asked for either a fix plus explicit ratification, or a return to pure derivation. **This row now describes the amended design: `derivedLinkId` mints PURELY from the op's own content (`opId`, `scope`, the raw link), with NO document read, no caller-supplied reservation, and no retry.** It spends the FULL safe-integer range (`[1, Number.MAX_SAFE_INTEGER]`, 53 bits, widened from the original's 52-bit window) rather than checking for a collision. ComfyUI_frontend's `LinkId` (a branded `number`, unlike the string-or-number `NodeId`) is still why a link needs a numeric mint at all — ADR-031's pure STRING scheme still can't satisfy it — but the mint no longer tries to guarantee collision-freedom against the document; it accepts a quantified residual risk instead. Residual probability: two independently derived 53-bit candidates collide with probability `1/(2^53-1) ≈ 1.11×10⁻¹⁶` per pair; for `k` links minted into one document the birthday-bound chance ANY two collide is `≈ k²/2^54` — about `2.2×10⁻¹²` at `k=200` (this package's own scale test), about `5.5×10⁻⁵` even at an unrealistic `k=1,000,000` — the same order of "cosmically unlikely, not impossible" risk this package already accepts for UUID4 `op_id` collisions (KA-2). A REALIZED collision is not specially detected: it is caught by `applier.ts`'s pre-existing `acceptsLink`/`seenLinks` apply-time id-collision handling (the same ordinary, order-independent LWW-gated check every other `insert_workflow` id kind already goes through), rejecting the whole op `link_id_collision` byte-identically rather than silently reassigning a different id — this check is REACHABLE via realistic input now (not merely via deliberate id-space exhaustion, since there is no more exhaustion to construct). **Separately, a pre-existing, unresolved gap** (out of scope for this amendment): a `connect` op's client-minted `link_id` can collide with an `insert_workflow`-derived numeric id, and `applier.ts`'s `claimLinkIdentity` does not guard against it at all, silently overwriting the `insert_workflow` link when `connect` arrives second — this predates and is unaffected by this amendment either way (the removed retry only ever protected the OPPOSITE arrival order, and only when the `connect` had already landed on that replica). See ADR-033's "Amendment (2026-09-23)" section for the full writeup, including this cross-mechanism note. | `src/remap.ts` `derivedLinkId`; `src/applier.ts` `prepareInsertedWorkflow`, `acceptsLink`, `claimLinkIdentity`; ADR-033 | Owner: Christian Byrne. The amended pure-derivation design was ratified after exact-head review confirmed that the document-state retry was removed, opposite-order convergence coverage was added, and realized collisions remain loud and byte-identical. Sunset (unchanged from the original row): revisit only if ComfyUI_frontend widens `LinkId` to `string \| number` (matching `NodeId`), at which point ADR-031's string form becomes sufficient for links too and this exception can be retired outright. |