diff --git a/.agents/checks/api-contract.md b/.agents/checks/api-contract.md index f3ae2266..8abc01b9 100644 --- a/.agents/checks/api-contract.md +++ b/.agents/checks/api-contract.md @@ -14,14 +14,14 @@ Catch breaking changes to this package's public surface. Applies to `src/index.t 2. **Op vocabulary** — op kind names, required fields, and the frozen/deferred sets are a contract shared with the server. Renaming an op kind, adding a required field to an existing op, or moving an op between frozen and deferred is breaking; cite the vocabulary section. 3. **Wire envelope** — the `{ type, data }` shape and `data.v` protocol version are cross-repo. Changing an existing message type, its payload shape, or bumping `data.v` without a documented migration is breaking. 4. **Stamp/order contract** — the total order key `[base_version, actor, op_id]` and `ApplyResult` shape (`applied`, `skipped`, `failed`, `version`) are consumed downstream; narrowing or reshaping them is breaking. **That field list is incomplete and known to be wrong — see "Known defects" below ([#73](https://github.com/Comfy-Org/comfy-multi-player/issues/73)); do not rely on it as an enumeration.** - In private alpha, `base_version` is the creator-owned Lamport counter carried in the single op format; there is no schema-v3 or dual-reader surface. DQ-11 incarnation-qualified target keys must remain unchanged. -5. **Schema/catalog versioning** — bump `SCHEMA_VERSION` when an old reader would mis-project a new doc, and keep a `migrate()` path (KA-11). Silently changing on-doc shape without a version bump is a breaking, fail-open change. + In private alpha, `base_version` is the creator-owned Lamport counter carried in the single op format. DQ-11 incarnation-qualified target keys must remain unchanged. +5. **Schema/catalog versioning** — bump `SCHEMA_VERSION` when an old reader would mis-project a new doc, and document an explicit old-layout disposition (KA-11). The current schema-v3 release refuses old layouts without mutation or relabelling, provides no compatibility reader, and re-mints source workflows. A future release must explicitly choose and document refusal, migration, or another disposition rather than universally promising migration support. Silently changing on-doc shape without a version bump is a breaking, fail-open change. 6. **Type narrowing / default changes** — an exported function that used to accept a wider input now rejecting part of it, or a changed default, can break callers. ## Rules - Only flag changes that break existing consumers; do not flag additions. -- Check for a re-export alias or `migrate()` path before flagging. +- Check for a re-export alias and the documented old-layout disposition before flagging. - Do not treat `__`-prefixed internal doc keys as public API. - Critical for removed/renamed exports or op kinds; high for changed signatures or wire shapes; medium for changed defaults. Cite the affected KA-*/FC-* ID where relevant. diff --git a/CHANGELOG.md b/CHANGELOG.md index 425f5b62..0a11a110 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,15 @@ All notable changes to `@comfyorg/comfy-multi-player` are documented in this fil The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this package uses semantic versioning. +## Unreleased + +### Added + +- Added the standalone `insert_workflow` op for atomic workflow-template + insertion. The applier derives every carried node, link, group, and + definition ID from the operation ID and graph scope, avoiding allocation + against mutable document state. + ## 0.2.0 - 2026-08-30 ### Changed diff --git a/README.md b/README.md index 20d0faf7..5596c017 100644 --- a/README.md +++ b/README.md @@ -137,12 +137,12 @@ unrecognized workflow key passed through untouched. workflow content it throws `SchemaVersionError` when `meta.schema_version` is absent, is not a positive integer, or is not this package's `SCHEMA_VERSION`. A document NEWER than this package is refused rather than best-effort projected; a -document OLDER is refused too, and the message names the remedy — run -`migrate(doc, storedVersion)` on the **host** first, then read. `project()` +document OLDER is refused too. Re-mint an old document from its source workflow +with the current package; `migrate()` does not provide a compatibility reader. +`project()` never migrates, because it is a read any replica may call — a browser follower -included — and a follower must not write the shared document (KA-6/FC-5). No -follower calls it today (the frontend does not depend on this package at all), -so this is a rule about the API, not an observation about callers. +included — and a follower must not write the shared document (KA-6/FC-5). This +is an API rule independent of which consumers currently call it. This refusal is byte-exact: `encodeStateAsUpdate(doc)` and the `doc.share` key set are both unchanged. It is also strictly *less* mutating than the old @@ -170,18 +170,17 @@ the accept path. ### `migrate(doc, fromVersion): void` -Document-layout versioning, and the **migration path** a host runs before it -reads a document it did not mint. `SCHEMA_VERSION` is `1`, so today there is -nothing to step: the call validates and no-ops at v1, and throws -`SchemaVersionError` for everything else. A host never best-effort reads a -layout it does not know. +Document-layout version validation for the private-alpha current format. +`SCHEMA_VERSION` is `3`: the call validates and no-ops only at v3, and throws +`SchemaVersionError` for every older or newer layout without relabelling it. +Old layouts must be re-minted at their source; there is no compatibility reader. `migrate()` is **no longer the only fail-closed read gate** — it was, and nothing forced a caller through it, which is the fail-open gap #38 closed by putting the same check inside `project()`. Both entrypoints share one definition of the read (`readSchemaVersion`), so they agree on exactly which documents are unreadable. `migrate()` remains **host-only**: followers receive -a migrated document over the struct stream or as a new epoch (schema §10). +a current-format document over the struct stream or as a new epoch (schema §10). `fromVersion` is your *claim* about the document, and it is checked against the document's own `meta.schema_version`. It throws when: @@ -334,16 +333,18 @@ Every op carries the same envelope, minted by its creator before dispatch: } ``` -Six kinds, frozen: +Eight kinds, frozen: | Kind | Payload beyond the envelope | Batchable (authoring) | |---|---|---| | `add_node` | `node_id`, `class_type`, `pos`, `node` (the complete node object, inserted verbatim) | yes | +| `define_subgraph` | `subgraph_id`, `subgraph_definition` (the complete initial definition, inserted once) | yes | | `connect` | `link_id`, `from_node`, `from_slot`, `to_node`, `link_type`, then EITHER a numeric `to_slot` (`ConcreteConnectOp`) OR a `grow` payload with `to_slot` null/absent (`GrowConnectOp`); `grow.promoted: true` names a subgraph instance's DECLARED input, materialized on the instance and LWW-gated as one register (schema Amendment A15) | yes | | `disconnect` | `link_id`, `to_node`, `to_slot`; claims the same concrete input register as `connect` and removes the winning slot occupant | yes | | `set_widget` | `node_id`, `widget` (name, never index), `value`, optional `old`; an interior write adds `path` AND `inner_widget` together (`InteriorSetWidgetOp`); a promoted HOST write adds `promoted: {value_index, instance_path, host_widgets_values}` instead — a positional write into a subgraph instance's opaque array (schema Amendment A15) | yes | | `delete_node` | `node_id`, `removed_links` | yes | | `clear` | `removed_nodes` | no | +| `insert_workflow` | `workflow` containing required raw-ID `nodes` plus optional `links`, `groups`, and `definitions`; the applier remaps IDs deterministically from `op_id` | no | | `reset_doc` | see [open questions](docs/api-contract-proposal.md) — currently rejected `op_deferred` by this package | no | `FROZEN_OPS`, `DEFERRED_OPS`, and `BATCHABLE_OPS` are exported so you can check @@ -354,7 +355,20 @@ Compile-time assertions in `src/types.ts` pin that `FROZEN_OPS` is exactly is exactly `WireOp["op"]`, and that `BATCHABLE_OPS ⊆ FROZEN_OPS`, so the lists and the unions cannot drift apart silently. -**`Op` vs `WireOp`.** `Op` is what `applyOps` implements — the six kinds it +For `insert_workflow`, submit raw node, link, group, and definition ids without +inspecting document state. The applier owns deterministic, tree-wide remapping; +each derived id incorporates the envelope `op_id`, graph scope, id kind, and +original id. Numeric and string aliases with the same normalized id refer to +the same node, including in link endpoints. Definition ids are scoped to their +containing graph, so repeated nested ids in separate branches derive distinct +ids. A remapped definition id that collides anywhere in the stored +definition tree rejects the operation with `definition_conflict`. Duplicate or +missing raw ids reject it atomically with `malformed_op` at every definition +depth. Links with a missing origin or target node are dropped individually at +every depth, while valid sibling links remain. Private keys beginning with +`__` are recursively removed and never appear in the projected workflow. + +**`Op` vs `WireOp`.** `Op` is what `applyOps` implements — the eight kinds it can actually apply. `WireOp` is `Op` plus the deferred kinds a conforming peer may legally put on the wire, and it is what `ApplyFailure.op` and the stamp helpers take: a rejected `reset_doc` really does land in `failed.op`, so typing @@ -409,11 +423,14 @@ implementation to replay it with no failures. If you are building a submission surface in front of the applier, that admission layer is where `BATCHABLE_OPS` belongs. `test/batch-policy.test.ts` pins all of this. -The normative definition of the op envelope and the six kinds is +The historical op envelope and six-kind base vocabulary are defined by `docs/op-vocabulary-v1.md` in -[comfy-cli](https://github.com/Comfy-Org/comfy-cli), which mints these ops on -the agent side. The `Op` types here mirror those minted shapes field for field; -a divergence is a bug here. +[comfy-cli](https://github.com/Comfy-Org/comfy-cli), with the adopted connect +amendment pinned separately below. The complete current eight-kind implemented +surface is defined locally by `src/types.ts` and +[`docs/multiplayer-schema.md`](docs/multiplayer-schema.md): `define_subgraph` +and ADR-031's standalone `insert_workflow` are local extensions, not claims +about the historical comfy-cli pin. This package tracks that document at comfy-cli commit `7e732242d971daf0d2d30f22f997abfacd78986e`, plus amendment v1.2 (§11) at @@ -474,7 +491,7 @@ arrives on the update stream, not when the ack lists it as applied. **Batches abort the remainder.** If op *k* fails, ops 0..*k*-1 stay applied and ops *k*..*n* are not applied at all. Fix the failing op and resend the whole -batch with the **same** `op_id`s: the prefix comes back in `skipped`, the +batch with the **same** `op_id`s: the prefix comes back with `outcome: "no-op"`, the remainder applies. Rejected ops consume no `op_id`, so a batch is retryable. **The four `connect` rejections that used to break this are fixed (#34).** @@ -491,8 +508,8 @@ Rejection codes: `malformed_op`, `unknown_op`, `op_deferred`, `catalog_required`, `invalid_node_payload`, `unknown_widget`, `uncatalogued_widget_write`, `opaque_widgets`, `widget_out_of_range`, `input_slot_missing`, `output_slot_missing`, `not_a_subgraph`, -`interior_node_not_found`, `shared_definition_unforked`, and `apply_failed` for -anything unexpected. Match on `code`, never on `message`. +`interior_node_not_found`, `shared_definition_unforked`, `batch_aborted`, and +`apply_failed` for anything unexpected. Match on `code`, never on `message`. `uncatalogued_widget_write` means a NAME-KEYED widget write named a class the pinned catalog does not describe. `add_node` and `set_widget` (and `connect`'s @@ -630,7 +647,8 @@ Published to npm as [`@comfyorg/comfy-multi-player`](https://www.npmjs.com/packa npm install @comfyorg/comfy-multi-player ``` -Pin an **exact** version in both the frontend and the server. Conflict +The server pins an **exact** published version. The frontend consumes the +workspace source from the same commit that produces that release. Conflict resolution is a cross-process agreement about which write wins; two peers running different versions of these rules can disagree about the outcome. @@ -643,17 +661,22 @@ superseded) remains documented for historical context. ## Develop ```bash -npm install -npm run build # tsc → dist/ -npm test # vitest: schema, purity, replay, lww, convergence, roundtrip, applier -npm run check:purity # dependency-tree + bare-Node import gate -npm run check:imports # module-graph gate: no cycles, src imports yjs only, no Node builtins -npm run check:pins # cross-repo citations are pinned by SHA, not a moving ref -npm run verify:corpus # conformance fixtures match their pinned SHAs -npm run check:profile-claims # .agents/checks prose still matches the code it restates -npm run check:coderabbit # .coderabbit.yaml still matches the profiles that generate it +pnpm install +pnpm run build # tsc → dist/ +pnpm test # vitest: schema, purity, replay, lww, convergence, roundtrip, applier +pnpm run check:purity # dependency-tree + bare-Node import gate +pnpm run check:imports # module-graph gate: no cycles, src imports yjs only, no Node builtins +pnpm run check:pins # cross-repo citations are pinned by SHA, not a moving ref +pnpm run verify:corpus # conformance fixtures match their pinned SHAs +pnpm run check:profile-claims # .agents/checks prose still matches the code it restates +pnpm run check:coderabbit # .coderabbit.yaml still matches the profiles that generate it ``` +The canonical source lives at +[`packages/comfy-multi-player`](https://github.com/Comfy-Org/ComfyUI_frontend/tree/main/packages/comfy-multi-player) +in the ComfyUI frontend workspace. Run these commands from that package directory, +or use `pnpm --filter @comfyorg/comfy-multi-player ` from the workspace root. + `fixtures/` holds the replay corpus: recorded op sessions with their starting and final workflows, six conflict-resolution vectors, and the pinned catalog. Format in [`fixtures/README.md`](fixtures/README.md). Add a fixture with any diff --git a/docs/INVARIANTS.md b/docs/INVARIANTS.md index ee221807..a740f451 100644 --- a/docs/INVARIANTS.md +++ b/docs/INVARIANTS.md @@ -23,7 +23,8 @@ This is the machine-addressable review log for the package. IDs are stable; do n ### KA-3 — The op layer stays pure & portable **Rule:** Applier, projection, and mint have zero DOM/framework/LiteGraph/server-only dependencies and run identically in browser and host; assert `yjs`-only directly, not merely by denylist. **Why:** A peer, edge, browser, and Node host must execute one implementation identically. -**Enforced by:** `scripts/check-purity.mjs` (package level), `scripts/check-import-graph.mjs` + `.dependency-cruiser.cjs` rule `src-runtime-dep-is-yjs-only` (module-graph level, per source module), `test/purity.test.ts`, `test/parity.test.ts`, and [`purity`](../.agents/checks/purity.md). Cross-language implementations must pass the canonical golden vectors in `fixtures/golden-vectors/`. The schema §1 root-map NAMES (`nodes`, `links`, `definitions`, `meta`, `__applied`, `__stamps`) and the reserved per-node `__widgets_opaque` key are the shared wire layout and are pinned by name in `test/doc-mint-mutation-survivors.test.ts`; every other reader in this package goes through `nodesMap()`/`linksMap()`/… , so a consistent rename was invisible to most of the suite: measured one key at a time against the pre-existing tests, renaming `links`, `definitions`, `__applied` or `__stamps` left 277/277 green, while `nodes` reddened 2 and `meta` reddened 6. Four of the seven names had no holder at all before this file. They are additionally published as the language-agnostic vector `fixtures/golden-vectors/wire-layout.json` and held against the ENCODED bootstrap snapshot, against a replica that has only those bytes, and against this file and schema §1, by `test/wire-layout-contract.test.ts` — so a rename must move code, vector and both documents together, which is what makes a deliberate one visible in review. **Scoped, because the wording invites the wider reading:** that test does NOT force the `SCHEMA_VERSION` bump or the `migrate()` step KA-11 and schema §1/§10 require of a layout change. Its last case checks only that the three declarations of `SCHEMA_VERSION` agree with each other, and a rename applied consistently to all four places leaves them agreeing at 1 — measured, with that file fully green. The bump obligation is **UNGUARDED — see roadmap**; only its consistency once taken is held. +Schema v3 adds the first-class `__link_state` root to the shared wire layout. Private alpha has no compatibility reader: old layouts are re-minted, while `migrate()` refuses to relabel them. +**Enforced by:** `scripts/check-purity.mjs` (package level), `scripts/check-import-graph.mjs` + `.dependency-cruiser.cjs` rule `src-runtime-dep-is-yjs-only` (module-graph level, per source module), `test/purity.test.ts`, `test/parity.test.ts`, and [`purity`](../.agents/checks/purity.md). Cross-language implementations must pass the canonical golden vectors in `fixtures/golden-vectors/`. The current schema §1 root-map NAMES (`nodes`, `links`, `definitions`, `meta`, `__applied`, `__stamps`, `__link_state`) and the reserved per-node `__widgets_opaque` key are the shared wire layout. Before schema v3 added `__link_state`, the historical inventory was seven names: six roots plus that reserved widget key. Those seven were pinned by name in `test/doc-mint-mutation-survivors.test.ts`; every other reader in this package went through `nodesMap()`/`linksMap()`/… , so a consistent rename was invisible to most of the suite: measured one key at a time against the pre-existing tests, renaming `links`, `definitions`, `__applied` or `__stamps` left 277/277 green, while `nodes` reddened 2 and `meta` reddened 6. Four of those historical seven names had no holder at all before this file. The current eight names are additionally published as the language-agnostic vector `fixtures/golden-vectors/wire-layout.json` and held against the ENCODED bootstrap snapshot, against a replica that has only those bytes, and against this file and schema §1, by `test/wire-layout-contract.test.ts` — so a rename must move code, vector and both documents together, which is what makes a deliberate one visible in review. **Scoped, because the wording invites the wider reading:** that test does NOT force the `SCHEMA_VERSION` bump or the `migrate()` step KA-11 and schema §1/§10 require of a layout change. Its last case checks only that the three declarations of `SCHEMA_VERSION` agree with each other, and a rename applied consistently to all four places leaves them agreeing at 1 — measured, with that file fully green. The bump obligation is **UNGUARDED — see roadmap**; only its consistency once taken is held. ### KA-4 — The applier is deterministic and idempotent **Rule:** Same op-set + causal order gives the same projection; an identical duplicate `op_id` is a true no-op with byte-identical `encodeStateAsUpdate`, while reuse with a different canonical payload is rejected without mutation. @@ -35,12 +36,12 @@ This is the machine-addressable review log for the package. IDs are stable; do n **Amendment A9 closes three residual write-order holes:** values that clone but cannot be stored by Yjs are refused before `set_widget`, grow, and `add_node` writes; `connect.link_id` must satisfy both Y.Map-value and Y.Array-item domains; and non-iterable `delete_node.removed_links` is refused before the A7 node-presence gate or deletion. **Amendment A10 closes the reference-cycle hole and corrects the shallow write predicate from storability to encodability:** `applyOps` rejects cycles at A8's whole-op depth gate, while the write-site cycle guard independently protects `mint`; top-level `Date` values are refused at the write gate, and `mint` refuses out-of-int64 `BigInt` values because Yjs accepts but does not faithfully encode them. **Amendment A13 classifies every BigInt in an op envelope as `malformed_op` during A8's op-only canonicalization precondition, names its payload path, and therefore rejects before any document write or `op_id` record.** **Amendment A14 shape-validates `connect.link_type` as a string in the op-only precondition, before destination delete-wins or any write; arbitrary and unknown strings remain legal because this is not catalogue membership validation.** **Amendment A11 bounds the remaining breadth and size attack surface:** batches are capped before processing, while A8's whole-envelope canonicalization now caps collection entries and an approximate cost covering strings, keys, binary bytes, containers, and leaves. -The *rejection* half — "a rejected op leaves the doc untouched", stated in the `applyOps` module docstring in `src/applier.ts` and repeated as contract D4 — is swept table-driven in `test/ka4-rejection-byte-identity.test.ts`, one row per reachable rejection code across `add_node`, `set_widget`, `connect`, `connect`+grow, `delete_node`, `clear`, `reset_doc` and the envelope gate, asserting byte-identical `encodeStateAsUpdate` *and* that the `op_id` was not consumed. Completeness is enforced rather than asserted: one test compares the covered codes against `ALL_REJECTION_CODES`, and a second greps `src/applier.ts` for every `new OpRejectedError("…")` so a new code cannot be added without a row. **The four `connect` rejections this used to except are fixed (#34)** and are now ordinary rows in that sweep, asserting byte-identity and `op_id`-preservation like every other code; the block that deliberately asserted the bug is deleted and `KNOWN_KA4_VIOLATIONS` is kept empty so a regression has somewhere to be recorded. +The *rejection* half — "a rejected op leaves the doc untouched", stated in the `applyOps` module docstring in `src/applier.ts` and repeated as contract D4 — is swept table-driven in `test/ka4-rejection-byte-identity.test.ts`, one row per reachable rejection code across `add_node`, `insert_workflow`, `set_widget`, `connect`, `connect`+grow, `delete_node`, `clear`, `reset_doc` and the envelope gate, asserting byte-identical `encodeStateAsUpdate` *and* that the `op_id` was not consumed. Completeness is enforced rather than asserted: one test compares the covered codes against `ALL_REJECTION_CODES`, and a second greps `src/applier.ts` for every `new OpRejectedError("…")` so a new code cannot be added without a row. `insert_workflow` adds collision refusals for nodes and links; both validate before any write. **The four `connect` rejections this used to except are fixed (#34)** and are now ordinary rows in that sweep, asserting byte-identity and `op_id`-preservation like every other code; the block that deliberately asserted the bug is deleted and `KNOWN_KA4_VIOLATIONS` is kept empty so a regression has somewhere to be recorded. ### KA-5 — IDs are collision-free without coordination **Rule:** Use 53-bit random mint; document high-water marks are advisory, never allocators; FE stable IDs must be non-colliding strings. **Why:** Offline peers must create entities without a central allocator. -**Enforced by:** **UNGUARDED — see roadmap**. +**Enforced by:** `insert_workflow` derives every carried ID inside the applier from the immutable `op_id`, scope, kind, and raw ID (ADR-031); producers send raw workflows and perform no allocation against document state. General producer randomness remains **UNGUARDED — see roadmap**. ### KA-6 — Raw struct updates flow host → follower one-way only **Rule:** Followers never write the shared doc. @@ -72,9 +73,9 @@ Two corrections recorded by MUT-GLOB-KA4-1, both measured on this tree rather th 2. "Duplicate Yjs structs double on merge" is an ARRAY hazard. Because nodes and links are Y.Map-keyed by id, two independent `mint()`s of the same base merge to whole-value LWW at each key rather than doubling — verified, not assumed. The real re-seed hazard under this layout is silent whole-node clobber of a diverged replica's edits, which is not currently tested. **UNGUARDED — see roadmap.** `README.md` stated the doubling mechanism unscoped and has been corrected to match. ### KA-11 — Schema-version discipline is enforced on read -**Rule:** Bump `SCHEMA_VERSION` when an old reader would mis-project a new doc; fail closed on unreadable schema and provide a `migrate()` path. +**Rule:** Bump `SCHEMA_VERSION` when an old reader would mis-project a new doc and fail closed on every non-current layout. Private alpha provides no compatibility migration. **Why:** Readers must not silently mis-project an incompatible document. -**Enforced by:** `test/roundtrip.test.ts` (the `migrate()` path), `test/schema-version-on-read.test.ts` (the NORMAL read path), and `test/readonly-surface.test.ts` (the snapshot read surface, `src/read.ts`). `project()` calls `assertReadableSchema` before it reads a single key, so a doc whose `meta.schema_version` is absent, unreadable, or NEWER than this package's `SCHEMA_VERSION` is refused with `SchemaVersionError` without a separate `migrate()` call — the fail-open gap #38 closed. Both entrypoints share ONE definition of the read (`readSchemaVersion`, `src/schema-version.ts`), pinned by the `classifies meta.schema_version = … as UNREADABLE on both entrypoints` cases, so they cannot drift into two notions of unreadable. The refusal is byte-exact and materializes no root type, matching `migrate()`; the no-materialization property is asserted on `[...doc.share.keys()]`, since an empty materialized root encodes to zero bytes and the byte assertion alone cannot see it. A3's carve-out carries over unchanged: a document whose `meta` root was integrated as a different concrete Y type surfaces Yjs's own constructor-clash `Error`, not a `SchemaVersionError` — still fail-closed, still a throw, but a consumer matching on the error TYPE must expect it. **Scope, stated because the wording invites the wider reading:** an OLDER doc is refused rather than silently migrated (migration is a host-only WRITE — schema §10, KA-6/FC-5 — while `project()` is a pure read every replica runs), but at `SCHEMA_VERSION = 1` no older doc exists to construct, so that arm is exercised through the module-only `assertSchemaVersionAgainst(doc, context, expected)` and NOT through `project()`. It is guarded, not yet guarded end-to-end. Separately, the layout whose change would REQUIRE a bump — the schema §1 root-map names — is pinned by name in `test/doc-mint-mutation-survivors.test.ts` and on the encoded bytes in `test/wire-layout-contract.test.ts`; note that neither forces the bump itself (see KA-3). **A THIRD read path exists as of the snapshot surface, and it needed its own gate**: `src/read.ts` reads the same layout by the same key names, so a consumer could have routed around `project()`'s refusal by calling `readGraph` instead. It now refuses on the same `readSchemaVersion`/`assertReadableSchema` definition, with one qualification the projection does not need: it refuses only when the document CARRIES CONTENT, and returns its empty value for a document that carries none — ADR-004's follower reads a root-less document before its first frame and there is no content there to mis-key. **Read the #30 relationship precisely, because the loose version is false:** #30 made `migrate()` REFUSE that same document. That is a role split, not a precedent — migrating a document that makes no claim about its own version is an incoherent WRITE request, while reading nothing out of it is not — and it is also the one place this surface's DISPOSITION differs from `project()`'s while sharing its predicate, which Amendment A12 records against A5's consumer-impact note. **The gate's content question must not be asked in v1 vocabulary**: enumerating the §1 root names is blind to a v2 that renamed them, which is the canonical bump trigger three sentences above, so the probe reads the struct store instead (A12). Both clauses are pinned, the refusing one across five unreadable states and all seven accessors plus a renamed-root v2 document, a foreign root, and an all-tombstone document; the empty one against a bare `new Y.Doc()`, a document whose roots exist but are empty, and a document whose empty root was registered as either concrete Y type. +**Enforced by:** `test/roundtrip.test.ts`, `test/schema-version-on-read.test.ts`, `test/readonly-surface.test.ts`, and the 22 byte-identical old-layout refusal counterexamples retained by the schema-v3 integration. `project()` and `migrate()` share `readSchemaVersion`; both refuse old, absent, unreadable, or newer versions without mutation. Snapshot reads use the same predicate but return empty data for a document carrying no structs, as recorded by Amendment A12. Old private-alpha layouts are re-minted from source rather than relabelled. ### KA-12 — Catalog pinned at mint **Rule:** `meta.catalog_version` cites the catalog by SHA, not branch; reject NAMED widget writes to uncatalogued classes loudly. A subgraph instance's `type` is a definition UUID that no catalog describes, and its promoted widget values are POSITIONAL by frontend contract (ADR 0009), so a promoted host write (schema Amendment A15) is not a widget-name resolution and needs no catalog entry for the instance — it still needs A CATALOG (`catalog_required` without one) to tell an instance from an unseen class, and still refuses (`uncatalogued_widget_write`) to lay a positional array over named values the catalog cannot describe. @@ -86,25 +87,31 @@ Two corrections recorded by MUT-GLOB-KA4-1, both measured on this tree rather th **Why:** One implementation must run identically in a browser bundle, at a peer or edge, and in a bare Node doc-host serving many documents from one process, so restart and reconnect reseed from committed document state rather than package-owned persistence and two documents in one process cannot leak into each other. **This is not an alias of KA-3, and the overlap is the reason it reads like one:** KA-3 constrains the *dependency surface* (portability), KA-13 constrains *state ownership*. They coincide only on the framework-import ban, which is why `scripts/check-stateless.mjs` cites both and why the framework rules in the strict config are labelled `KA-3/KA-13`. **Enforced by:** `scripts/check-stateless.mjs` (`npm run check:stateless`), which lints every `src/**/*.ts` file against the `statelessRules` set in [`eslint.strict.config.js`](../.agents/checks/eslint.strict.config.js) under `CMP_STATELESS_ONLY=1` and then runs the colocated `test/stateless.test.ts` probe — no state shared between documents in one module instance, identical public behavior from fresh module registries, and a fresh Node process. The gate exits 2 (INCONCLUSIVE, never a pass) when a precondition or the per-file lint floor is missing, and it is merge-blocking in fact rather than by convention: [`.github/workflows/ci.yml`](../.github/workflows/ci.yml) runs it on every push and pull request. `test/invariant-ids.test.ts` holds this register itself — every `KA-*`/`FC-*` id cited anywhere in the package must resolve to a heading in this file, which is the guard that was missing while this entry was ([comfy-multi-player #152](https://github.com/Comfy-Org/comfy-multi-player/issues/152)). -### KA-11 amendment — DQ-11 incarnation namespace (schema v2) -**Rule:** A node lifetime has a durable `__incarnation` token. Imported nodes use +### KA-11 amendment — DQ-11 incarnation namespace (historical schema-v2 context) +**Historical rule:** A node lifetime has a durable `__incarnation` token. Imported nodes use `"0"`; a modern winning `add_node` carries its immutable `op_id` as the token (legacy adds without the field remain life `"0"`). Node-scoped widget ops carry that token, and `__stamps` keys include it. A write for a non-current incarnation is a consumed no-op. The v1→v2 migration translates missing tokens -and legacy widget keys to life `"0"`. +and legacy widget keys to life `"0"`. This records the schema-v2 amendment; +the current schema is v3 and does not run that migration for an old document. +Old private-alpha layouts are refused without mutation and source workflows +are re-minted, with no compatibility reader or relabelling path. **Why:** A delete followed by same-ID re-add must not let a life-1 widget stamp defeat a valid life-2 write (DQ-11, KEEP-ALIVE 4). The package regression is `test/incarnation-stamps.test.ts`; the schema and protocol implications are recorded in Amendment A16 of `docs/multiplayer-schema.md`. -### KA-2 amendment — DQ-10 direct Lamport counter semantics -**Rule:** In the single private-alpha op format, `base_version` is minted as a +### KA-2 amendment — DQ-10 direct Lamport counter semantics (historical schema-v2 context) +**Historical rule:** In the single private-alpha op format, `base_version` is minted as a creator-owned Lamport counter. The winner remains the tuple-generic `[counter, actor, op_id]`; DQ-11's A16 incarnation-qualified target keys and -the already-shipped legacy incarnation token `"0"` remain unchanged. There is -no schema-v3, migration, legacy shim, or dual-format reader. +the already-shipped legacy incarnation token `"0"` remain unchanged. At that +decision point there was no schema-v3, migration, legacy shim, or dual-format +reader. The current schema-v3 release preserves the ordering decision but +refuses old layouts; it re-mints source workflows rather than providing a +compatibility reader or relabelling an old document. **Enforced by:** `test/clock.test.ts`, the existing LWW/convergence suites, the four-family merge harness, and [`ADR-0005`](adr/0005-lamport-ordering-v1-migration.md). diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index d01e3fcc..b2503b84 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -4,6 +4,9 @@ V1 is an op-based Yjs document applier implemented once in the shared `@comfyorg/comfy-multi-player` TypeScript package. The browser and Node doc-host sidecar consume the same git-SHA-pinned package. Semantic ops are the replication unit; the widget catalog is sha256-pinned and fail-closed in the deployed architecture. +ADR-031 adds the standalone-only `insert_workflow` kind for atomic template insertion. The applier +derives tree-wide IDs from `op_id`; producers submit raw IDs and do not inspect document state. + ## Guard gaps - ~~**Positive purity assertion (#22):**~~ **CLOSED.** `scripts/check-purity.mjs` asserts directly that the declared and resolved production dependency roots are exactly `{yjs}`, and `scripts/check-import-graph.mjs` asserts the same per source module via `src-runtime-dep-is-yjs-only`. A cross-language parity guard is still only warranted if a second-language implementation is proposed; the preferred policy remains no second implementation (tracked in the separate bullet below). diff --git a/docs/api-contract-proposal.md b/docs/api-contract-proposal.md index 1921b726..0413e5e7 100644 --- a/docs/api-contract-proposal.md +++ b/docs/api-contract-proposal.md @@ -18,6 +18,14 @@ still cheap. That will not stay true. ## Part 1 — decisions to ratify +### D0. Workflow templates insert as one standalone op + +**Accepted by ADR-031.** `insert_workflow` requires `nodes`; `links`, `groups`, and `definitions` +are optional and default to empty. Producers send raw workflows. The applier remaps every carried +id and internal reference deterministically from the op envelope id plus the original id and graph +scope. Different ops therefore cannot collide or depend on arrival order, while exact replay derives +the same ids and is idempotent. The op is stamped and not batchable. + ### D1. Widgets are addressed by name, not by position **Proposal.** A node's widget values live in a map keyed by widget name. The diff --git a/docs/decisions/ADR-031-insert-workflow-op.md b/docs/decisions/ADR-031-insert-workflow-op.md new file mode 100644 index 00000000..f0dc0f6d --- /dev/null +++ b/docs/decisions/ADR-031-insert-workflow-op.md @@ -0,0 +1,42 @@ +# ADR-031: cmp-owned, op-id-derived workflow-template insertion + +- **Status:** Accepted +- **Date:** 2026-09-11 +- **Decider:** Christian +- **Source:** in-app-agent program ADR-031 and frozen FE #17501 head `8491ba33fa` + +## Context + +Agents and users need to insert a complete workflow template, including subgraph definitions, +without decomposing it into independently observable graph edits. Existing op shapes deliberately +reject definition-bearing payloads and cannot express that transaction safely. + +## Decision + +Add the non-batchable `insert_workflow` op. Its frozen envelope carries one authoritative `workflow` +with required top-level nodes and optional links, groups, and definitions. The applier remaps every +carried node, link, group, and nested-definition id to a deterministic string derived from the op +envelope id, graph scope, id kind, and original id. Producers emit the raw workflow unchanged. + +Internal node/link references and definition-instance types are rewritten to those derived ids. +Distinct insert ops therefore cannot collide, even when they carry the same raw ids; exact replay +derives the same ids and is a byte-identical no-op through the existing `op_id` gate. The operation +is one Yjs transaction and is stamped as its own register. + +## Consequences + +- Template insertion is atomic at the semantic-op boundary and deterministic on replay. +- Producers do not inspect document state or remap ids. +- Definition interiors retain scoped graph namespaces, all derived from the op id. +- Existing op schemas remain closed to definitions. + +## Invariants + +This decision touches KA-1, KA-2, KA-3, KA-4, KA-5, FC-1, FC-3, and FC-4. + +## Glossary + +- **ADR-T8:** the accepted in-app-agent program technical design for workflow insertion. +- **Canonical projection:** stable, key-sorted JSON used as the definition-content identity. +- **Envelope id:** the immutable `op_id` carried by the semantic operation. +- **Producer:** the cloud or CLI caller that emits the raw workflow payload. diff --git a/docs/multiplayer-schema.md b/docs/multiplayer-schema.md index 869a0501..4a1397b4 100644 --- a/docs/multiplayer-schema.md +++ b/docs/multiplayer-schema.md @@ -1,6 +1,6 @@ -# Multiplayer workflow-document schema — v2 +# Multiplayer workflow-document schema — v3 -`SCHEMA_VERSION = 2` +`SCHEMA_VERSION = 3` > **State: DRAFT — awaiting FE sign-off (FE-1330).** > @@ -44,6 +44,9 @@ Normative inputs, in precedence order: `fixtures/` (the evidence: three replayable sessions, six LWW vectors, the exported catalog, machine-captured findings). Every DECISION below cites the spike finding that forced it. +3. **ADR-022 / ADR-T8** — the local decision and accepted in-app-agent program + TDD add `insert_workflow` as the eighth implemented op without moving the + pinned comfy-cli vocabulary. --- @@ -58,9 +61,26 @@ Y.Doc │ last_node_id, last_link_id, │ groups, extra, config, version, … (§1.4) ├── Y.Map "__applied" op_id → 1 (§4) -└── Y.Map "__stamps" write-target key → [base_version, actor, op_id] (§4) +├── Y.Map "__stamps" write-target key → [base_version, actor, op_id] (§4) +└── Y.Map "__link_state" normalized link id → imported durable descriptor (§1.5) ``` +### 1.5 Durable link state + +Schema v3 adds `__link_state` as the accepted first-class storage boundary for +durable link intent. `mint()` seeds one versioned `authority: "imported"` +descriptor per coherent six-field link tuple. It retains the complete tuple and +the complete persisted destination-slot object, classifying it as concrete, +promoted (by full definition input name), or autogrow (by `grow_id`). Readers +fail closed on unknown descriptor versions or kinds. A successful `connect` +replaces that baseline with an operation-owned descriptor carrying its A18 +stamp and authoritative concrete, full-name promoted, or autogrow destination +metadata. Endpoint deletion strands the live tuple but retains the descriptor; +a winning endpoint re-add restores the exact tuple, slot and endpoint +references. Winning disconnect, replacement, and explicitly named +`delete_node.removed_links` retire the descriptor. Composed destinations remain +undefined until a producer and public type contract exist. + ### 1.1 Per-node Y.Map Keyed in `nodes` by `String(node.id)`. Fields: @@ -538,7 +558,8 @@ stays mandatory. unreadable (§7 rule 0) or the catalogue pin is violated (§3 pin 4), neither of which is a state a host can be compacting from, and it drops individual entries only per §7 rule 6 — - carrying forward: `__stamps` entries for still-live targets, the actor + carrying forward: `__stamps` entries for still-live targets, all + `__link_state` descriptors (including temporarily stranded endpoint intent), the actor watermarks, `catalog_version`, and the id high-water marks. The fresh doc is a new **doc epoch**: its bootstrap snapshot replaces the old one (§9), and followers resynchronize by full re-fetch (an epoch bump is a signal on @@ -581,20 +602,38 @@ node/link arrays are **not** sorted at projection — the §7 sorted-by-id rule applies to the top-level arrays only. Python's `canonical` sorts `definitions.subgraphs` by id but leaves each definition's interior arrays in authored order, and the fixtures pin that (`session-subgraph`'s def lists -node 27 before node 3). Since only `set_widget` is subgraph-scoped, interior -membership and order are static after mint; the def Y.Map therefore stores -plain `node_order`/`link_order` registers (written once at mint) and -projection emits interior arrays in that order. Definitions themselves -project sorted by definition id. +node 27 before node 3). The def Y.Map stores plain scalar +`node_order`/`link_order` registers, and projection emits interior arrays in +that order. Interior `connect` preserves surviving imported links as an +authored-order prefix and sorts op-added links after them by winning stamp. +The scalar `link_order` itself is updated to that canonical order; projection +does not depend on an object marker hidden inside the array. Definitions +themselves project sorted by definition id. + +Added-link ordering stamps use the internal `__stamps` key +`["interior_link_order", String(definition_id), String(link_id)]`. A winning +same-ID rewrite refreshes that stamp; rewriting a still-present imported link +does not reclassify it as an addition. Definition IDs identify unique +definitions. These are additional internal stamp entries, not a new root or +scalar-array layout, so the interior-order metadata did not itself trigger an +additional version bump. The combined current document schema is v3. ### 5.2 Addressing: three forms, one write target -Only `set_widget` is subgraph-scoped in the frozen vocabulary (spike Q6; -error strings captured verbatim in `fixtures/findings.json`): - -- `connect` structurally refuses interior endpoints ("a link cannot cross - the subgraph boundary") and promoted-widget targets ("promoted widget (a - value), not a link input"). +The frozen spike supported only subgraph-scoped `set_widget` (spike Q6; +historical error strings remain in `fixtures/findings.json`). The current +package also supports `connect` with a non-empty instance `path`: + +- Interior `connect` resolves both endpoints within one definition and rejects + interior autogrow and writes to shared, unforked definitions. Input stamps + include the path; normalized link identity stamps do too. This does not + enable cross-boundary wiring or an interior `disconnect` operation. Its path + is an instance route only: after the visible head is deleted, resolution + requires that instance's retained `interior_route` stamp (including its + incarnation), and a definition id is never resolved as a direct alias. + A missing instance without a retained route is an accepted, consumed no-op, + including when the path happens to name a definition. It does not edit that + definition; it is not a rejected operation with a byte-identity guarantee. - `add_node`/`delete_node` cannot address interior nodes at all. `set_widget` accepts three address forms — flat promoted (`57.text`, routed @@ -776,10 +815,12 @@ the epoch; cross-epoch struct updates never merge. ## 10. Versioning and `migrate()` -- `SCHEMA_VERSION = 2`, stored in `meta.schema_version` at mint. -- `migrate(doc, fromVersion)` contract: in-place, stepwise `vN → vN+1` - migrations composed in order; exact no-op when - `fromVersion === SCHEMA_VERSION`; host-only (followers receive the migrated +- `SCHEMA_VERSION = 3`, stored in `meta.schema_version` at mint. +- Private-alpha policy keeps one current format: old layouts are re-minted at + their source and compatibility readers/migrations are not provided. +- `migrate(doc, fromVersion)` contract: exact no-op when + `fromVersion === SCHEMA_VERSION`; fail closed without mutation for every + older or newer version; host-only (followers receive the current-format doc via the struct stream / a new epoch); a doc whose `schema_version` is **greater** than the code's `SCHEMA_VERSION` is rejected, fail-closed — never best-effort read. @@ -787,8 +828,8 @@ the epoch; cross-epoch struct updates never merge. current-version one), an unreadable `meta.schema_version` is rejected rather than assumed current, and "exact no-op" is defined at the byte level. Read A3 for the normative rule and for what this deliberately stopped checking. -- Bumping `SCHEMA_VERSION` requires: a migration step, updated fixtures or a - fixture-format note, an amendment section in this document, and FE +- Bumping `SCHEMA_VERSION` requires: an explicit old-layout disposition, + updated fixtures or a fixture-format note, an amendment section, and FE sign-off (the layout is a cross-repo contract with the FE follower). - **The version check is on the READ path, not only on `migrate()`.** `project(doc, catalog)` refuses — `SchemaVersionError`, before it reads any @@ -801,8 +842,9 @@ the epoch; cross-epoch struct updates never merge. migration is a host-only write, `project()` is a pure read available to every replica, and a follower that writes the shared doc breaks KA-6/FC-5 outright and becomes an independently edited replica, which is the FC-1 raw-struct - divergence path. The caller runs `migrate(doc, storedVersion)` first, then - reads. The refusal is byte-exact and materializes no root type, the same as + divergence path. Private-alpha callers re-mint old source data into a new + current-format document rather than relabelling its layout. The refusal is + byte-exact and materializes no root type, the same as `migrate()`'s — asserted on `[...doc.share.keys()]`, since an empty materialized root encodes to zero bytes (A3). Both entrypoints share ONE definition of the read, `readSchemaVersion` in @@ -1281,11 +1323,10 @@ materialization contrast on a snapshot-forked replica) and `test/roundtrip.test. path). Every fail-closed case runs against a real fixture workflow that projects cleanly one line earlier, so a `toThrow()` cannot pass for a reason unrelated to the schema version. -The "document is OLDER than the reader" arm has no production reachability at `SCHEMA_VERSION = 1` -— no older version exists to construct. It is exercised through -`assertSchemaVersionAgainst(doc, context, expected)`, exported from the module but deliberately NOT -from the entrypoint, since a caller free to choose `expected` could pass the document's own version -and switch the gate off. An arm no test can turn red is dead code; this one can be turned red. +At the time of this amendment, `SCHEMA_VERSION` was 1 and the older-document arm was reachable only +through the module-only `assertSchemaVersionAgainst` test seam. Schema v3 now exercises the public +read and migration entrypoints directly with preserved v1/v2 counterexamples; all are refused +byte-identically rather than translated. ### Consumer impact @@ -1304,9 +1345,9 @@ consumer repositories at their current revisions, not by analogy to A3: `clear`'s `groups`, and `clear` preserves everything else. - **The op producer never constructs a document at all.** `comfy-cli` emits ops as plain JSON; it has no Yjs dependency, no `Y.Doc`, no snapshot handling. -- **The frontend does not consume this package.** `@comfyorg/comfy-multi-player` is absent from its - `package.json` on every branch, so there is no follower call site to break, which is what ADR-004 - already records. +- **At the time of this amendment, the frontend did not consume this package.** It now consumes the + canonical workspace source from `packages/comfy-multi-player`; this historical compatibility + argument applied before that migration, not to future package changes. - **Two endpoints reach `project()`**, both in the doc-host sidecar: `/project` and `/apply` (whose response embeds a projection computed after `applyOps`). `/mint` and `/resync` do not. @@ -2043,4 +2084,29 @@ reference, then installs its own tuple and exactly its own references. The separate input register still decides whether that identity may occupy the requested destination; losing that gate leaves no tuple or dangling reference. This adds an internal `__stamps` key, not a root-layout change, so -`SCHEMA_VERSION` remains 2. +this amendment did not itself require a version bump. The combined current +document schema is v3. + +--- + +## Amendment A19 — 2026-09-11 — atomic workflow-template insertion + +ADR-031 adds the standalone-only `insert_workflow` op. Its payload is one +authoritative workflow containing required top-level `nodes` and optional +`links`, `groups`, and `definitions`. The applier deterministically remaps every +inserted id from the immutable `op_id`, graph scope, id kind, and original id, +then rewrites internal references. Producers emit raw payloads. Distinct ops +cannot collide; exact replay derives the same ids. Duplicate ids within one raw +payload remain collision errors. Malformed link tuples are `malformed_op`. + +Definition ids and every nested definition-interior graph id are remapped in +scoped namespaces. Definition-instance `type` values are rewritten with them, +so a consumer definition id is never used as a live id and no live-definition +conflict path remains. + +The whole merge occurs in one Yjs transaction and claims the +`("insert_workflow", op_id)` stamp target. Exact replay is stopped by the +existing applied-op gate and is byte-identical. This adds graph content but no +new root or node-map layout, so insertion did not itself require a version +bump. The combined current document schema is v3. Other op payloads remain +closed to definition-bearing fields. diff --git a/fixtures/golden-vectors/wire-layout.json b/fixtures/golden-vectors/wire-layout.json index c3a7747f..04d9c9e6 100644 --- a/fixtures/golden-vectors/wire-layout.json +++ b/fixtures/golden-vectors/wire-layout.json @@ -1,14 +1,15 @@ { "format_version": 1, "comment": "Canonical WIRE layout of the multiplayer Y.Doc (schema §1). `roots` maps each logical role to the Y.Doc ROOT TYPE NAME as it is spelled inside an encoded Yjs update; `reserved_node_keys` names the per-node keys the doc owns. These strings are a cross-implementation contract, not an internal detail: two replicas that disagree about them cannot exchange a document, because a follower forking from the bootstrap snapshot (KA-10) resolves roots by name. Renaming any of them is a layout break: schema §10 requires a SCHEMA_VERSION bump, a migration step, updated fixtures or a fixture-format note, an amendment section in the schema document, and FE sign-off, plus a matching edit here and in docs/INVARIANTS.md KA-3. test/wire-layout-contract.test.ts holds the four-way name agreement and the SCHEMA_VERSION triple's consistency; it does NOT enforce the bump itself, so read §10 rather than treating a green suite as permission. A second implementation should read this file rather than transcribing the names.", - "schema_version": 2, + "schema_version": 3, "roots": { "nodes": "nodes", "links": "links", "definitions": "definitions", "meta": "meta", "applied": "__applied", - "stamps": "__stamps" + "stamps": "__stamps", + "link_state": "__link_state" }, "reserved_node_keys": { "opaque_widgets": "__widgets_opaque", diff --git a/scripts/check-import-graph.mjs b/scripts/check-import-graph.mjs index 8a9652bf..5930c321 100644 --- a/scripts/check-import-graph.mjs +++ b/scripts/check-import-graph.mjs @@ -59,7 +59,7 @@ const root = process.env.IMPORT_GRAPH_ROOT || dirname(dirname(fileURLToPath(impo * not one per change. See `.agents/checks/import-graph.md` and the config's own * "NO includeOnly" comment. */ -const MIN_MODULES = Number(process.env.IMPORT_GRAPH_MIN_MODULES ?? 10); +const MIN_MODULES = Number(process.env.IMPORT_GRAPH_MIN_MODULES ?? 11); const cli = join(root, "node_modules", ".bin", "depcruise"); const run = spawnSync(cli, ["--output-type", "json", "src"], { @@ -85,6 +85,39 @@ try { process.exit(2); } +if ( + typeof report !== "object" || + report === null || + typeof report.summary !== "object" || + report.summary === null || + !Number.isInteger(report.summary.totalCruised) || + report.summary.totalCruised < 0 || + !Number.isInteger(report.summary.totalDependenciesCruised) || + report.summary.totalDependenciesCruised < 0 || + !Array.isArray(report.summary.violations) || + report.summary.violations.some( + (violation) => + typeof violation !== "object" || + violation === null || + typeof violation.from !== "string" || + (violation.to !== undefined && typeof violation.to !== "string") || + typeof violation.rule !== "object" || + violation.rule === null || + typeof violation.rule.name !== "string" || + typeof violation.rule.severity !== "string" || + (violation.cycle !== undefined && + (!Array.isArray(violation.cycle) || + violation.cycle.some( + (segment) => + typeof segment !== "string" && + (typeof segment !== "object" || segment === null || typeof segment.name !== "string"), + ))), + ) +) { + console.error("import-graph check INCONCLUSIVE: dependency-cruiser report has no valid summary"); + process.exit(2); +} + const { totalCruised, totalDependenciesCruised, violations } = report.summary; // -------------------------------------------------------------------------- diff --git a/scripts/check-stateless.mjs b/scripts/check-stateless.mjs index b43d4fb0..3bc10b9e 100644 --- a/scripts/check-stateless.mjs +++ b/scripts/check-stateless.mjs @@ -15,6 +15,8 @@ import { fileURLToPath } from "node:url"; const root = process.env.STATELESS_ROOT ?? dirname(dirname(fileURLToPath(import.meta.url))); const config = join(root, ".agents/checks/eslint.strict.config.js"); +const eslintEntrypoint = join(root, "node_modules/eslint/bin/eslint.js"); +const vitestEntrypoint = join(root, "node_modules/vitest/vitest.mjs"); const sourceFiles = readdirSync(join(root, "src"), { recursive: true }) .filter((file) => typeof file === "string" && file.endsWith(".ts")) .map((file) => join("src", file)) @@ -27,7 +29,7 @@ function inconclusive(message) { if (!existsSync(config)) inconclusive(".agents/checks/eslint.strict.config.js is missing"); if (sourceFiles.length === 0) inconclusive("no source files were found to lint"); -if (!existsSync(join(root, "node_modules/eslint"))) inconclusive("eslint is not installed; run npm ci"); +if (!existsSync(eslintEntrypoint)) inconclusive("eslint is not installed; run npm ci"); if (!existsSync(join(root, "node_modules/@typescript-eslint/parser"))) { inconclusive("@typescript-eslint/parser is not installed; run npm ci"); } @@ -36,8 +38,8 @@ if (!existsSync(join(root, "node_modules/eslint-plugin-sonarjs"))) { } const lint = spawnSync( - join(root, "node_modules/.bin/eslint"), - ["--no-config-lookup", "--config", config, "--no-warn-ignored", "--format", "json", ...sourceFiles], + process.execPath, + [eslintEntrypoint, "--no-config-lookup", "--config", config, "--no-warn-ignored", "--format", "json", ...sourceFiles], { cwd: root, encoding: "utf8", maxBuffer: 16 * 1024 * 1024, env: { ...process.env, CMP_STATELESS_ONLY: "1" } }, ); if (lint.error || lint.status === 2) { @@ -64,11 +66,18 @@ if (findings.length > 0) { process.exit(1); } +const statelessProbe = join(root, "test/stateless.test.ts"); +if (!existsSync(vitestEntrypoint)) inconclusive("vitest is not installed; run npm ci"); +if (!existsSync(statelessProbe)) inconclusive("test/stateless.test.ts is missing"); + const probe = spawnSync( - join(root, "node_modules/.bin/vitest"), - ["run", "test/stateless.test.ts", "--reporter=dot"], + process.execPath, + [vitestEntrypoint, "run", "test/stateless.test.ts", "--reporter=dot"], { cwd: root, encoding: "utf8", stdio: "inherit" }, ); +if (probe.error) inconclusive(`could not run the stateless probe: ${probe.error.message}`); +if (probe.signal) inconclusive(`stateless probe terminated by signal ${probe.signal}`); +if (probe.status === 2) inconclusive("stateless probe failed to execute"); if (probe.status !== 0) process.exit(probe.status ?? 1); const packageJson = JSON.parse(readFileSync(join(root, "package.json"), "utf8")); diff --git a/src/applier.ts b/src/applier.ts index 43aa190c..91c8dd1d 100644 --- a/src/applier.ts +++ b/src/applier.ts @@ -86,8 +86,11 @@ import { appliedMap, apush, arrayItemRefusal, + cloneForMap, countDefinitionInstances, createNodeMap, + linkStateMap, + definitionsMap, linksMap, mapValueRefusal, mdel, @@ -99,8 +102,11 @@ import { stampsMap, widgetStorageOf, } from "./doc.js"; +import { linkHasMissingEndpoint, remapInsertedWorkflowIds } from "./remap.js"; import { sha256Hex } from "./digest.js"; import { CMP_EVENT_SCHEMA_VERSION, emitCmpEvent, type CmpCallContext } from "./events.js"; +import { importedLinkState, mintDefinition } from "./mint.js"; +import { projectDefinition } from "./project.js"; import { MAX_OP_COST, MAX_OPS_PER_BATCH, @@ -118,17 +124,26 @@ import { type ApplyResult, type ConnectOp, type DeleteNodeOp, + type DefineSubgraphOp, type DisconnectOp, type GrowConnectOp, type GrowSpec, + type ImportedLinkState, type InteriorSetWidgetOp, + LINK_STATE_DESCRIPTOR_VERSION, + type LinkTuple, type CanonicalOpInspection, + type InsertWorkflowOp, type Op, + type OperationLinkDestination, + type OperationLinkState, type SetWidgetOp, type StampKey, + type SubgraphDefinition, type WidgetCatalog, type WireOp, } from "./types.js"; +import { addInteriorLinkOrder, removeInteriorLinkOrder } from "./interior-link-order.js"; import { NODE_INCARNATION_KEY } from "./types.js"; /** @@ -274,11 +289,11 @@ function opIdentity(op: unknown): string { * NOT stored — see {@link opDigest}. Exposed to tests as the definition of * what the digest is taken over. */ -export function canonicalOp(op: Op): string { +function canonicalJson(value: unknown): string { // BigInt classification takes precedence over the generic depth/cost gates. // Keep this walk iterative and bounded so even a hostile envelope cannot // turn the diagnostic into unbounded work. - const bigintStack: Array<{ value: unknown; path: string }> = [{ value: op, path: "$" }]; + const bigintStack: Array<{ value: unknown; path: string }> = [{ value, path: "$" }]; const bigintVisited = new Set(); let bigintVisits = 0; while (bigintStack.length > 0 && bigintVisits++ <= MAX_OP_COST) { @@ -313,7 +328,7 @@ export function canonicalOp(op: Op): string { // Amendment A11 extends A8's whole-envelope, pre-idempotency gate with a // breadth/size budget. Its iterative depth check keeps A8's // `payload_too_deep` vocabulary while avoiding hostile recursion. - const bounds = opBoundsRefusal(op); + const bounds = opBoundsRefusal(value); if (bounds !== null) { throw new OpRejectedError( bounds.includes("nests deeper") ? "payload_too_deep" : "malformed_op", @@ -345,7 +360,11 @@ export function canonicalOp(op: Op): string { } return value; }; - return JSON.stringify(normalize(op, 0, "$")); + return JSON.stringify(normalize(value, 0, "$")); +} + +export function canonicalOp(op: Op): string { + return canonicalJson(op); } /** @@ -461,6 +480,12 @@ function validateEnvelope(op: WireOp): void { if (!(FROZEN_OPS as readonly string[]).includes(op.op)) { throw new OpRejectedError("unknown_op", `unknown op '${op.op}'`); } + if (op.op !== "define_subgraph") { + const ordinary = op as WireOp & Record; + if ("subgraph_definition" in ordinary || "definitions" in ordinary || "subgraph_id" in ordinary) { + throw new OpRejectedError("malformed_op", `${op.op}: definition payloads and subgraph_id targets are only valid on define_subgraph`); + } + } if (typeof op.op_id !== "string" || op.op_id.length === 0) { throw new OpRejectedError("malformed_op", `${op.op}: missing op_id`); } @@ -514,9 +539,13 @@ function dispatch(doc: Y.Doc, op: Op, catalog?: WidgetCatalog): SuccessfulOutcom return applyDeleteNode(doc, op); case "clear": return applyClear(doc, op); + case "define_subgraph": + return applyDefineSubgraph(doc, op, catalog); + case "insert_workflow": + return applyInsertWorkflow(doc, op, catalog); default: // Exhaustiveness guard (issue #21): with every `Op` member cased above, - // `op` is `never` here. Add a sixth IMPLEMENTED kind to `Op` and this + // `op` is `never` here. Add a seventh IMPLEMENTED kind to `Op` and this // line stops compiling until it gets a `case`. // // Issue #17 removed the `case "reset_doc"` that used to sit here. It was @@ -532,6 +561,571 @@ function dispatch(doc: Y.Doc, op: Op, catalog?: WidgetCatalog): SuccessfulOutcom } } +function applyDefineSubgraph( + doc: Y.Doc, + op: DefineSubgraphOp, + catalog?: WidgetCatalog, +): SuccessfulOutcome { + if (!isUuid(op.subgraph_id) || !isPlainRecord(op.subgraph_definition) || op.subgraph_definition.id !== op.subgraph_id) { + throw new OpRejectedError( + "malformed_op", + "define_subgraph: subgraph_id must be a UUID matching subgraph_definition.id", + ); + } + validateSubgraphDefinition(op.subgraph_definition, "subgraph_definition"); + if (!catalog) { + throw new OpRejectedError("catalog_required", "define_subgraph: the pinned catalog is required to encode interior nodes"); + } + validateDefinitionWidgets(op.subgraph_definition, catalog); + const digest = definitionDigest(op.subgraph_definition, catalog); + const definitions = definitionsMap(doc); + const existing = definitions.get(op.subgraph_id); + if (existing !== undefined) { + const digests = definitionDigests(doc); + const existingDigest = digests[op.subgraph_id] ?? definitionDigest(existing, catalog); + if (existingDigest === digest) return "no-op"; + if (digest > existingDigest) { + assertDefinitionIdsAvailable(doc, op.subgraph_definition, op.subgraph_id); + let replacement: Y.Map; + try { + replacement = mintDefinition(op.subgraph_definition, catalog); + } catch (error) { + throw new OpRejectedError("malformed_op", `define_subgraph(${op.subgraph_id}): ${error instanceof Error ? error.message : String(error)}`); + } + const widgetEdits = definitionWidgetEdits(doc, existing); + mset(definitions, op.subgraph_id, replacement); + restoreDefinitionWidgetEdits(doc, replacement, widgetEdits); + setDefinitionDigest(doc, op.subgraph_id, digest); + return "applied"; + } + return "no-op"; + } + const existingNested = resolveDefinition(doc, op.subgraph_id); + if (existingNested !== null) { + throw new OpRejectedError( + "definition_conflict", + `define_subgraph: definition id '${op.subgraph_id}' is already registered`, + ); + } + assertDefinitionIdsAvailable(doc, op.subgraph_definition); + let definition: Y.Map; + try { + definition = mintDefinition(op.subgraph_definition, catalog); + } catch (error) { + throw new OpRejectedError( + "malformed_op", + `define_subgraph(${op.subgraph_id}): ${error instanceof Error ? error.message : String(error)}`, + ); + } + mset(definitions, op.subgraph_id, definition); + setDefinitionDigest(doc, op.subgraph_id, digest); + return "applied"; +} + +function definitionDigest(definition: SubgraphDefinition | Y.Map, catalog: WidgetCatalog): string { + return sha256Hex(canonicalJson(canonicalDefinitionValue(definition, catalog))); +} + +/** One digest representation for submitted JSON and stored definitions: their public projection. */ +function canonicalDefinitionValue( + definition: SubgraphDefinition | Y.Map, + catalog: WidgetCatalog, +): Record { + if (definition instanceof Y.Map) return projectDefinition(definition, catalog); + + // Projection reads attached Y types (not preliminary content on unattached + // maps), so integrate the temporary mint before using the authoritative + // mint/project normalization for widgets, outputs, and nested definitions. + const temporary = new Y.Doc(); + try { + const minted = mintDefinition(definition, catalog); + temporary.getMap>("definitions").set(String(definition.id), minted); + return projectDefinition(minted, catalog); + } finally { + temporary.destroy(); + } +} + +function definitionDigests(doc: Y.Doc): Record { + const value = metaMap(doc).get("__definition_digests"); + if (!isPlainRecord(value)) return {}; + return Object.fromEntries( + Object.entries(value).filter((entry): entry is [string, string] => typeof entry[1] === "string"), + ); +} + +function setDefinitionDigest(doc: Y.Doc, id: string, digest: string): void { + mset(metaMap(doc), "__definition_digests", { ...definitionDigests(doc), [id]: digest }); +} + +type DefinitionWidgetEdit = { + targetKey: string; + definitionId: string; + nodeId: string; + widget: string; + value: unknown; +}; + +function definitionWidgetEdits( + doc: Y.Doc, + existing: Y.Map, +): DefinitionWidgetEdit[] { + const edits: DefinitionWidgetEdit[] = []; + for (const targetKey of stampsMap(doc).keys()) { + let target: unknown; + try { + target = JSON.parse(targetKey); + } catch { + continue; + } + if (!Array.isArray(target) || target[0] !== "widget" || !Array.isArray(target[1])) continue; + const path = target[1].map(String); + const widget = target[3]; + if (path.length < 2 || typeof widget !== "string") continue; + const resolved = definitionNodeAtPath(doc, existing, path); + if (!resolved) continue; + const oldWidgets = resolved.node.get("widgets"); + if (oldWidgets instanceof Y.Map && oldWidgets.has(widget)) { + edits.push({ + targetKey, + definitionId: resolved.definitionId, + nodeId: path.at(-1)!, + widget, + value: structuredClone(oldWidgets.get(widget)), + }); + } + } + return edits; +} + +function definitionNodeAtPath( + doc: Y.Doc, + root: Y.Map, + path: string[], +): { definitionId: string; node: Y.Map } | null { + let definitionId = path[0]!; + let node = definitionNode(root, definitionId, path[1]!); + if (!node) { + const instance = nodesMap(doc).get(definitionId); + if (!(instance instanceof Y.Map) || String(instance.get("type")) !== String(root.get("id"))) return null; + definitionId = String(root.get("id")); + node = definitionNode(root, definitionId, path[1]!); + } + if (!node) return null; + for (const nodeId of path.slice(2)) { + definitionId = String(node.get("type")); + node = definitionNode(root, definitionId, nodeId); + if (!node) return null; + } + return { definitionId, node }; +} + +function restoreDefinitionWidgetEdits( + doc: Y.Doc, + replacement: Y.Map, + edits: DefinitionWidgetEdit[], +): void { + for (const edit of edits) { + const newNode = definitionNode(replacement, edit.definitionId, edit.nodeId); + const newWidgets = newNode?.get("widgets"); + if (!(newWidgets instanceof Y.Map)) { + mdel(stampsMap(doc), edit.targetKey); + continue; + } + mset(newWidgets, edit.widget, edit.value); + } +} + +function definitionNode( + root: Y.Map, + definitionId: string, + nodeId: string, +): Y.Map | null { + if (String(root.get("id")) === definitionId) { + const nodes = root.get("nodes"); + const node = nodes instanceof Y.Map ? nodes.get(nodeId) : undefined; + return node instanceof Y.Map ? node : null; + } + const container = root.get("definitions"); + const nested = container instanceof Y.Map ? container.get("subgraphs") : undefined; + if (!(nested instanceof Y.Map)) return null; + for (const child of nested.values()) { + if (child instanceof Y.Map) { + const node = definitionNode(child, definitionId, nodeId); + if (node) return node; + } + } + return null; +} + +function assertDefinitionIdsAvailable( + doc: Y.Doc, + definition: Record, + excludedRootId?: string, + submitted = new Set(), +): void { + const visit = (candidate: Record, path: string): void => { + const id = String(candidate.id); + if (submitted.has(id)) throw new OpRejectedError("malformed_op", `define_subgraph: duplicate definition id '${id}' at ${path}`); + submitted.add(id); + const nested = candidate.definitions; + if (isPlainRecord(nested) && Array.isArray(nested.subgraphs)) nested.subgraphs.forEach((child, index) => { + if (isPlainRecord(child)) visit(child, `${path}.definitions.subgraphs[${index}]`); + }); + }; + visit(definition, "subgraph_definition"); + for (const id of submitted) { + if (definitionIdExistsOutsideRoot(doc, id, excludedRootId)) { + throw new OpRejectedError("definition_conflict", `define_subgraph: definition id '${id}' is already registered`); + } + } +} + +function definitionIdExistsOutsideRoot( + doc: Y.Doc, + id: string, + excludedRootId?: string, +): boolean { + const containsId = (definition: Y.Map): boolean => { + if (String(definition.get("id")) === id) return true; + const container = definition.get("definitions"); + const nested = container instanceof Y.Map ? container.get("subgraphs") : undefined; + if (!(nested instanceof Y.Map)) return false; + for (const child of nested.values()) { + if (child instanceof Y.Map && containsId(child)) return true; + } + return false; + }; + + for (const [rootId, definition] of definitionsMap(doc)) { + if (rootId !== excludedRootId && definition instanceof Y.Map && containsId(definition)) return true; + } + return false; +} + +function isUuid(value: unknown): value is string { + return typeof value === "string" && /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test(value); +} + +function isPlainRecord(value: unknown): value is Record { + if (typeof value !== "object" || value === null || Array.isArray(value)) return false; + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; +} + +function validateSubgraphDefinition(definition: Record, path: string): void { + if (!isUuid(definition.id) || !Array.isArray(definition.nodes) || !Array.isArray(definition.links)) { + throw new OpRejectedError("malformed_op", `define_subgraph: ${path} needs a UUID id plus nodes and links arrays`); + } + for (const key of ["node_order", "link_order", "__definition_digest"]) { + if (Object.hasOwn(definition, key)) { + throw new OpRejectedError("malformed_op", `define_subgraph: ${path} contains reserved key '${key}'`); + } + } + assertUniqueNormalizedIds(definition.nodes, `${path}.nodes`, true); + assertUniqueNormalizedIds(definition.links, `${path}.links`); + validateSerializableValue(definition, path); + + const nested = definition.definitions; + if (nested === undefined) return; + if (!isPlainRecord(nested) || !Array.isArray(nested.subgraphs)) { + throw new OpRejectedError("malformed_op", `define_subgraph: ${path}.definitions.subgraphs must be an array`); + } + const nestedIds = new Set(); + nested.subgraphs.forEach((candidate, index) => { + const nestedPath = `${path}.definitions.subgraphs[${index}]`; + if (!isPlainRecord(candidate) || !isUuid(candidate.id)) { + throw new OpRejectedError("malformed_op", `define_subgraph: ${nestedPath} must be a definition with a UUID id`); + } + if (nestedIds.has(candidate.id)) { + throw new OpRejectedError("malformed_op", `define_subgraph: duplicate definition id '${candidate.id}' at ${nestedPath}`); + } + nestedIds.add(candidate.id); + validateSubgraphDefinition(candidate, nestedPath); + }); +} + +function assertUniqueNormalizedIds( + values: unknown[], + path: string, + required = false, + operation = "define_subgraph", +): void { + const ids = new Set(); + values.forEach((value, index) => { + const id = Array.isArray(value) ? value[0] : isPlainRecord(value) ? value.id : undefined; + if (id === undefined || id === null) { + if (required) throw new OpRejectedError("malformed_op", `${operation}: missing id at ${path}[${index}]`); + return; + } + const normalized = String(id); + if (ids.has(normalized)) { + throw new OpRejectedError("malformed_op", `${operation}: duplicate normalized id '${normalized}' at ${path}[${index}]`); + } + ids.add(normalized); + }); +} + +function validateDefinitionWidgets(definition: Record, catalog: WidgetCatalog): void { + if (typeof definition.id === "string" && Object.hasOwn(catalog.types, definition.id)) { + throw new OpRejectedError( + "malformed_op", + `define_subgraph: definition id '${definition.id}' shadows a catalog class`, + ); + } + (definition.nodes as unknown[]).forEach((candidate) => { + if (!isPlainRecord(candidate)) { + throw new OpRejectedError("malformed_op", "define_subgraph: every interior node must be an object"); + } + rejectUnprojectableWidgets(candidate.type, candidate.widgets_values, catalogEntry(catalog, candidate.type)); + }); + const nested = definition.definitions; + if (isPlainRecord(nested) && Array.isArray(nested.subgraphs)) { + nested.subgraphs.forEach((child) => { + if (isPlainRecord(child)) validateDefinitionWidgets(child, catalog); + }); + } +} + +function validateSerializableValue(value: unknown, path: string): void { + if (value === null || typeof value === "string" || typeof value === "boolean") return; + if (typeof value === "number" && Number.isFinite(value)) return; + if (Array.isArray(value)) { + value.forEach((child, index) => validateSerializableValue(child, `${path}[${index}]`)); + return; + } + if (isPlainRecord(value)) { + Object.entries(value).forEach(([key, child]) => validateSerializableValue(child, `${path}.${key}`)); + return; + } + throw new OpRejectedError("malformed_op", `define_subgraph: ${path} is not JSON-serializable`); +} + +// --------------------------------------------------------------------------- +// insert_workflow +// --------------------------------------------------------------------------- + +function numericId(value: unknown): number | undefined { + if (typeof value === "number" && Number.isFinite(value)) return value; + if (typeof value === "string" && /^[0-9]+$/.test(value)) return parseInt(value, 10); + return undefined; +} + +function scrubPrivateKeys(value: unknown): unknown { + if (Array.isArray(value)) return value.map(scrubPrivateKeys); + if (typeof value !== "object" || value === null) return value; + const clean: Record = {}; + for (const [key, child] of Object.entries(value)) { + if (!key.startsWith("__")) clean[key] = scrubPrivateKeys(child); + } + return clean; +} + +function definitionId(value: unknown): string | undefined { + if (typeof value === "string" && value.length > 0) return value; + if (typeof value === "number" && Number.isFinite(value)) return String(value); + return undefined; +} + +function validateRawGraphIds(nodes: unknown[], links: unknown[], path: string): void { + assertUniqueNormalizedIds(nodes, `${path}.nodes`, true, "insert_workflow"); + assertUniqueNormalizedIds(links, `${path}.links`, true, "insert_workflow"); +} + +function validateDefinitionInputs(subgraphs: unknown[], path = "workflow.definitions.subgraphs"): void { + const ids = new Set(); + subgraphs.forEach((candidate, index) => { + if (typeof candidate !== "object" || candidate === null || Array.isArray(candidate)) { + throw new OpRejectedError("malformed_op", "insert_workflow: every subgraph definition must be an object"); + } + const sg = candidate as Record; + const id = definitionId(sg["id"]); + if (id === undefined) { + throw new OpRejectedError("malformed_op", "insert_workflow: every subgraph definition requires a valid id"); + } + if (ids.has(id)) { + throw new OpRejectedError("malformed_op", `insert_workflow: duplicate definition id '${id}' at ${path}[${index}]`); + } + ids.add(id); + if (!Array.isArray(sg["nodes"]) || (sg["links"] !== undefined && !Array.isArray(sg["links"]))) { + throw new OpRejectedError("malformed_op", "insert_workflow: definition nodes and links must be arrays"); + } + const definitionPath = `${path}[${index}]`; + validateRawGraphIds(sg["nodes"], (sg["links"] as unknown[] | undefined) ?? [], definitionPath); + const nested = (sg["definitions"] as { subgraphs?: unknown } | undefined)?.subgraphs; + if (nested !== undefined && !Array.isArray(nested)) { + throw new OpRejectedError("malformed_op", "insert_workflow: nested definitions.subgraphs must be an array"); + } + if (Array.isArray(nested)) validateDefinitionInputs(nested, `${definitionPath}.definitions.subgraphs`); + }); +} + +function applyInsertWorkflow(doc: Y.Doc, op: InsertWorkflowOp, catalog?: WidgetCatalog): SuccessfulOutcome { + const workflow = scrubPrivateKeys(op.workflow) as unknown; + if (typeof workflow !== "object" || workflow === null || Array.isArray(workflow)) { + throw new OpRejectedError("malformed_op", "insert_workflow: workflow must be an object"); + } + let wf = workflow as Record; + if (!Array.isArray(wf["nodes"]) || (wf["links"] !== undefined && !Array.isArray(wf["links"]))) { + throw new OpRejectedError("malformed_op", "insert_workflow: nodes and links must be arrays"); + } + const definitions = wf["definitions"]; + if (definitions !== undefined && (typeof definitions !== "object" || definitions === null || Array.isArray(definitions))) { + throw new OpRejectedError("malformed_op", "insert_workflow: definitions must be an object"); + } + const subgraphs = (definitions as { subgraphs?: unknown } | undefined)?.subgraphs; + if (subgraphs !== undefined && !Array.isArray(subgraphs)) { + throw new OpRejectedError("malformed_op", "insert_workflow: definitions.subgraphs must be an array"); + } + if (wf["groups"] !== undefined && !Array.isArray(wf["groups"])) { + throw new OpRejectedError("malformed_op", "insert_workflow: groups must be an array"); + } + validateRawGraphIds(wf["nodes"] as unknown[], (wf["links"] as unknown[] | undefined) ?? [], "workflow"); + validateDefinitionInputs((subgraphs as unknown[] | undefined) ?? []); + wf = remapInsertedWorkflowIds(wf as unknown as import("./types.js").WorkflowJSON, op.op_id) as unknown as Record; + const remappedDefinitions = wf["definitions"] as { subgraphs?: unknown[] } | undefined; + const remappedSubgraphs = remappedDefinitions?.subgraphs ?? []; + + const nodes = nodesMap(doc); + const links = linksMap(doc); + const stamps = stampsMap(doc); + const stamp = stampKey(op); + const seenNodes = new Set(); + for (const candidate of wf["nodes"] as unknown[]) { + if (typeof candidate !== "object" || candidate === null || Array.isArray(candidate)) { + throw new OpRejectedError("invalid_node_payload", "insert_workflow: every node must be an object"); + } + const node = candidate as { id?: unknown; type?: unknown }; + if (node.id === undefined || typeof node.type !== "string" || node.type.length === 0) { + throw new OpRejectedError("invalid_node_payload", "insert_workflow: every node requires id and type"); + } + const key = String(node.id); + if (seenNodes.has(key)) { + throw new OpRejectedError("node_id_collision", `insert_workflow: node id '${key}' collides`); + } + const existing = nodes.get(key); + if (existing) { + const incumbent = stamps.get(JSON.stringify(["insert_workflow_node", key])) as StampKey | undefined; + if (incumbent === undefined) { + throw new OpRejectedError("node_id_collision", `insert_workflow: node id '${key}' collides`); + } + if (compareStampKeys(stamp, incumbent) <= 0) return "lww-dropped"; + } + seenNodes.add(key); + } + const seenLinks = new Set(); + const linkWrites: unknown[][] = []; + for (const candidate of (wf["links"] as unknown[] | undefined) ?? []) { + if (!Array.isArray(candidate) || candidate[0] === undefined || candidate[1] === undefined || candidate[3] === undefined) { + throw new OpRejectedError("malformed_op", "insert_workflow: every link must be a tuple with an id and two endpoints"); + } + const key = String(candidate[0]); + if (seenLinks.has(key)) { + throw new OpRejectedError("link_id_collision", `insert_workflow: link id '${key}' collides`); + } + if (links.has(key)) { + const incumbent = stamps.get(JSON.stringify(["insert_workflow_link", key])) as StampKey | undefined; + if (incumbent === undefined) { + throw new OpRejectedError("link_id_collision", `insert_workflow: link id '${key}' collides`); + } + if (compareStampKeys(stamp, incumbent) <= 0) return "lww-dropped"; + } + seenLinks.add(key); + if (linkHasMissingEndpoint(candidate, (id) => nodes.has(String(id)) || seenNodes.has(String(id)))) { + continue; + } + linkWrites.push(candidate); + } + + const targetKey = stampTargetKey(op); + const prior = stamps.get(targetKey) as StampKey | undefined; + if (prior != null && compareStampKeys(stamp, prior) <= 0) return "lww-dropped"; + + if (remappedSubgraphs.length > 0 && !catalog) { + throw new OpRejectedError("catalog_required", "insert_workflow: the pinned catalog is required to encode definitions"); + } + const cat = catalog ?? { types: {} }; + const defs = definitionsMap(doc); + const submittedDefinitionIds = new Set(); + const definitionWrites: Array<[string, Y.Map, string]> = []; + for (const candidate of remappedSubgraphs) { + if (typeof candidate !== "object" || candidate === null || Array.isArray(candidate)) { + throw new OpRejectedError("malformed_op", "insert_workflow: every subgraph definition must be an object"); + } + const sg = candidate as Record; + const id = definitionId(sg["id"])!; + validateSubgraphDefinition(sg, "workflow.definitions.subgraphs"); + validateDefinitionWidgets(sg, cat); + assertDefinitionIdsAvailable(doc, sg, undefined, submittedDefinitionIds); + const digest = definitionDigest(sg as unknown as SubgraphDefinition, cat); + definitionWrites.push([id, mintDefinition(sg, cat), digest]); + } + + const nodeWrites: Array<[string, unknown, Y.Map]> = []; + for (const candidate of wf["nodes"] as import("./types.js").WorkflowNode[]) { + const node = structuredClone(candidate); + const wv = node.widgets_values; + const entry = catalogEntry(catalog, node.type); + if (!catalog && Array.isArray(wv) && wv.length > 0) { + throw new OpRejectedError("catalog_required", `insert_workflow(${node.type}): positional widgets_values needs a catalog`); + } + rejectUnprojectableWidgets(node.type, wv, entry); + try { + nodeWrites.push([String(node.id), node.id, createNodeMap(node, entry?.widget_order)]); + } catch (err) { + throw new OpRejectedError("invalid_node_payload", `insert_workflow(${node.type}): ${err instanceof Error ? err.message : String(err)}`); + } + } + + const importedLinkStates = new Map>(); + for (const link of linkWrites) { + const key = String(link[0]); + try { + importedLinkStates.set(key, importedLinkState(link, wf as unknown as import("./types.js").WorkflowJSON)); + } catch (err) { + throw new OpRejectedError("malformed_op", `insert_workflow: ${err instanceof Error ? err.message : String(err)}`); + } + } + + for (const [id, definition, digest] of definitionWrites) { + mset(defs, id, definition); + setDefinitionDigest(doc, id, digest); + } + for (const link of linkWrites) { + const key = String((link as unknown[])[0]); + mset(links, key, cloneForMap(link, "insert_workflow: link")); + const state = importedLinkStates.get(key) ?? null; + if (state !== null) mset(linkStateMap(doc), key, cloneForMap(state, `insert_workflow: link state ${key}`)); + mset(stamps, JSON.stringify(["insert_workflow_link", key]), stamp); + } + for (const [key, id, nodeMap] of nodeWrites) { + clearObsoleteWidgetStamps(stamps, key); + mset(nodeMap, NODE_INCARNATION_KEY, `${op.op_id}:${key}`); + mset(nodes, key, nodeMap); + mset(stamps, JSON.stringify(["insert_workflow_node", key]), stamp); + mset(stamps, targetKey, stamp); + reconcileNodeLinkRefs(doc, id, nodeMap); + } + if (nodeWrites.length === 0) mset(stamps, targetKey, stamp); + + const meta = metaMap(doc); + if (Array.isArray(wf["groups"])) { + const currentGroups = Array.isArray(meta.get("groups")) ? meta.get("groups") as unknown[] : []; + const merged = new Map(); + for (const group of [...currentGroups, ...wf["groups"]]) merged.set(canonicalJson(group), group); + mset(meta, "groups", [...merged.entries()].sort(([a], [b]) => codePointCompare(a, b)).map(([, group]) => group)); + } + const currentNode = numericId(meta.get("last_node_id")) ?? 0; + const maxNode = Math.max(currentNode, ...nodeWrites.map(([, id]) => numericId(id) ?? currentNode)); + if (maxNode > currentNode) mset(meta, "last_node_id", maxNode); + const currentLink = numericId(meta.get("last_link_id")) ?? 0; + const maxLink = Math.max(currentLink, ...linkWrites.map((link) => numericId(link[0]) ?? currentLink)); + if (maxLink > currentLink) mset(meta, "last_link_id", maxLink); + return nodeWrites.length > 0 || linkWrites.length > 0 || definitionWrites.length > 0 || ((wf["groups"] as unknown[] | undefined)?.length ?? 0) > 0 + ? "applied" + : "no-op"; +} + // --------------------------------------------------------------------------- // add_node // --------------------------------------------------------------------------- @@ -637,21 +1231,13 @@ function applyAddNode(doc: Y.Doc, op: AddNodeOp, catalog?: WidgetCatalog): Succe // arrived. Everything else in the payload is still copied verbatim (FC-8) — // only `inputs[].link` / `outputs[].links` are re-derived. reconcileNodeLinkRefs(doc, op.node_id, nodeMap); + restoreDurableLinks(doc, op.node_id); // last_node_id is a max-register (vocabulary §8.3): write only on increase. - // DQ-15 Q2 (Christian, blocked-on-christian#10): node ids are normalized - // before this write so numeric vs string forms of the same id ("42" vs 42) - // can't skew the counter. A non-numeric string id (e.g. a UUID) leaves the - // register untouched, matching the pre-existing numeric-only contract. const meta = metaMap(doc); const cur = meta.get("last_node_id"); const curN = typeof cur === "number" ? cur : 0; - const idN = - typeof op.node_id === "number" - ? op.node_id - : typeof op.node_id === "string" && /^[0-9]+$/.test(op.node_id) - ? parseInt(op.node_id, 10) - : undefined; + const idN = numericId(op.node_id); if (idN !== undefined && idN > curN) { mset(meta, "last_node_id", idN); } @@ -954,8 +1540,15 @@ function applyPromotedHostWrite( key: StampKey, catalog?: WidgetCatalog, ): SuccessfulOutcome { - const target = resolveInteriorNode(doc, promoted.instancePath, catalog); - if (target === null) return "no-op"; // head instance concurrently deleted → no-op (delete wins) + const resolution = resolveInteriorNode( + doc, + promoted.instancePath, + catalog, + op.node_incarnation ?? LEGACY_NODE_INCARNATION, + false, + ); + if (resolution === null) return "no-op"; // no live or retained canonical route + const target = resolution.node; const storage = hostWriteStorage(target, catalog); switch (storage) { case "named": @@ -1006,12 +1599,28 @@ function applySetWidget(doc: Y.Doc, op: SetWidgetOp, catalog?: WidgetCatalog): S throw new OpRejectedError("malformed_op", "set_widget: a promoted host write carries no interior path"); } + // Interior paths have several legal spellings (definition id and instance + // id), but all of them can resolve to the same node. Resolve first + // so that node owns one register rather than letting each raw alias claim a + // separate stamp key. `writeTarget` remains the public op-only identity. + const interiorResolution = interior === null + ? null + : resolveInteriorNode( + doc, + interior.path.map(String), + catalog, + op.node_incarnation ?? LEGACY_NODE_INCARNATION, + ); + if (interior !== null && interiorResolution === null) return "no-op"; + // LWW gate next (comfy-cli `_apply_set_widget`): a lower-or-equal stamp is // dropped — a protocol-level apply that still consumes its op_id. It is no // longer literally FIRST: the op-only checks above it must precede it so // their verdict cannot depend on which stamp is in the document (A6). const stamps = stampsMap(doc); - const targetKey = stampTargetKey(op); + const targetKey = interiorResolution === null + ? stampTargetKey(op) + : JSON.stringify(["widget", interiorResolution.canonicalPath, op.node_incarnation ?? LEGACY_NODE_INCARNATION, interior!.inner_widget]); const prior = stamps.get(targetKey) as StampKey | undefined; const key = stampKey(op); if (prior != null && compareStampKeys(key, prior) <= 0) return "lww-dropped"; @@ -1021,8 +1630,7 @@ function applySetWidget(doc: Y.Doc, op: SetWidgetOp, catalog?: WidgetCatalog): S } if (interior !== null) { - const target = resolveInteriorNode(doc, interior.path.map(String), catalog); - if (target === null) return "no-op"; // head instance concurrently deleted → no-op (delete wins) + const target = interiorResolution!.node; if (nodeIncarnation(target) !== (op.node_incarnation ?? LEGACY_NODE_INCARNATION)) return "no-op"; const nodeType = String(target.get("type") ?? ""); const widget = interior.inner_widget; @@ -1087,11 +1695,75 @@ function projectedWidgetsLength(node: Y.Map, order: readonly string[]): * not forked: schema §5.3 pins that a conforming applier must reject interior * writes to shared definitions until forking is specced and fixtured. */ -function resolveInteriorNode(doc: Y.Doc, path: string[], catalog?: WidgetCatalog): Y.Map | null { +interface InteriorResolution { + node: Y.Map; + canonicalPath: string[]; +} + +function interiorRouteKey(instanceId: string, incarnation: string): string { + return JSON.stringify(["interior_route", instanceId, incarnation]); +} + +function resolveInteriorNode( + doc: Y.Doc, + path: string[], + catalog?: WidgetCatalog, + incarnation = LEGACY_NODE_INCARNATION, + allowRetainedRoute = true, +): InteriorResolution | null { const head = nodesMap(doc).get(path[0]!); - if (!head) return null; + if (!head) { + const directDefinition = resolveDefinition(doc, path[0]!); + const retainedDefinitionId = allowRetainedRoute + ? stampsMap(doc).get(interiorRouteKey(path[0]!, incarnation)) + : undefined; + const definition = directDefinition && String(directDefinition.get("id")) === path[0] + ? directDefinition + : typeof retainedDefinitionId === "string" + ? resolveDefinition(doc, retainedDefinitionId) + : null; + if (!definition) return null; + const definitionId = String(definition.get("id")); + // A retained route represents the deleted routing instance for authority + // purposes. Otherwise delete-first could evade the shared-definition + // guard that edit-first observes. + const instances = countDefinitionInstances(doc, definitionId, catalog) + + (typeof retainedDefinitionId === "string" ? 1 : 0); + if (instances > 1) { + throw new OpRejectedError( + "shared_definition_unforked", + `definition ${definitionId} is instantiated ${instances} times; interior writes to shared definitions are rejected until forking is specced (schema §5.3)`, + ); + } + const innerNodes = definition.get("nodes"); + const inner = innerNodes instanceof Y.Map ? innerNodes.get(path[1]!) : undefined; + if (!(inner instanceof Y.Map)) { + throw new OpRejectedError( + "interior_node_not_found", + `interior node ${path[1]} not found in subgraph ${path[0]}`, + ); + } + const canonicalPath = [definitionId, path[1]!]; + if (path.length === 2) return { node: inner, canonicalPath }; + return resolveInteriorDescendants(doc, inner, path.slice(2), canonicalPath, catalog); + } + const definition = resolveDefinition(doc, String(head.get("type") ?? "")); + const canonicalPath = definition === null + ? [path[0]!] + : [String(definition.get("id") ?? head.get("type"))]; + if (path.length === 1) return { node: head, canonicalPath }; + return resolveInteriorDescendants(doc, head, path.slice(1), canonicalPath, catalog); +} + +function resolveInteriorDescendants( + doc: Y.Doc, + head: Y.Map, + path: string[], + canonicalPath: string[], + catalog?: WidgetCatalog, +): InteriorResolution { let cur: Y.Map = head; - for (const seg of path.slice(1)) { + for (const seg of path) { const curType = String(cur.get("type") ?? ""); const def = resolveDefinition(doc, curType); if (!def) { @@ -1117,8 +1789,12 @@ function resolveInteriorNode(doc: Y.Doc, path: string[], catalog?: WidgetCatalog ); } cur = inner; + // Identity is the final owning definition and node, not the route used to + // reach it. This aliases [outerDef, nestedInstance, leaf] with + // [innerDef, leaf] while keeping equal leaf IDs in other definitions apart. + canonicalPath.splice(0, canonicalPath.length, defId, seg); } - return cur; + return { node: cur, canonicalPath }; } // --------------------------------------------------------------------------- @@ -1226,6 +1902,15 @@ function requireOutputSlot(src: Y.Map, op: ConnectOp): Y.Array function requireOpOnlyValid(op: ConnectOp): void { requireOutputSlotDomain(op); + if (op.path !== undefined) { + if (!Array.isArray(op.path) || op.path.length === 0) { + throw new OpRejectedError("malformed_op", "connect: path must be a non-empty array when present"); + } + if (op.grow != null) { + throw new OpRejectedError("malformed_op", "connect: interior autogrow is not supported"); + } + } + if (op.node_incarnation !== undefined && (typeof op.node_incarnation !== "string" || op.node_incarnation.length === 0)) { throw new OpRejectedError("malformed_op", "connect: node_incarnation must be a non-empty string"); } @@ -1273,11 +1958,191 @@ function requireOpOnlyValid(op: ConnectOp): void { } } +interface InteriorConnectScope { + nodes: Y.Map>; + links: Y.Map; + definition: Y.Map; +} + +/** Resolve the definition owned by the instance at an interior connect path. */ +function resolveInteriorConnectScope( + doc: Y.Doc, + op: ConnectOp, + catalog?: WidgetCatalog, +): InteriorConnectScope | null { + if (!op.path || op.path.length === 0) return null; + const path = op.path.map(String); + let host = nodesMap(doc).get(path[0]!); + if (!(host instanceof Y.Map)) { + // Connect accepts instance routes only. A missing head must have been a + // real instance in this incarnation; unlike set_widget, a definition id + // is never a direct addressing alias for this operation. + const retainedDefinitionId = stampsMap(doc).get(interiorRouteKey( + path[0]!, + op.node_incarnation ?? LEGACY_NODE_INCARNATION, + )); + if (typeof retainedDefinitionId !== "string") return null; + const retainedDefinition = resolveDefinition(doc, retainedDefinitionId); + if (!retainedDefinition) return null; + const retainedId = String(retainedDefinition.get("id") ?? retainedDefinitionId); + const retainedInstances = countDefinitionInstances(doc, retainedId, catalog) + 1; + rejectSharedInteriorDefinition(retainedId, retainedInstances); + if (path.length === 1) return interiorConnectScope(retainedDefinition, retainedId); + const retainedNodes = retainedDefinition.get("nodes"); + host = retainedNodes instanceof Y.Map ? retainedNodes.get(path[1]!) : undefined; + if (!(host instanceof Y.Map)) { + throw new OpRejectedError( + "interior_node_not_found", + `interior node ${path[1]} not found in subgraph ${retainedId}`, + ); + } + path.splice(0, 2); + } else { + path.shift(); + } + + for (const segment of path) { + const ownerType = String(host.get("type") ?? ""); + const owner = resolveDefinition(doc, ownerType); + if (!owner) { + throw new OpRejectedError( + "not_a_subgraph", + `node ${String(host.get("id"))} is not a subgraph; cannot descend to '${segment}'`, + ); + } + const ownerId = String(owner.get("id") ?? ownerType); + rejectSharedInteriorDefinition(ownerId, countDefinitionInstances(doc, ownerId, catalog)); + const innerNodes = owner.get("nodes"); + const inner = innerNodes instanceof Y.Map ? innerNodes.get(segment) : undefined; + if (!(inner instanceof Y.Map)) { + throw new OpRejectedError( + "interior_node_not_found", + `interior node ${segment} not found in subgraph ${ownerId}`, + ); + } + host = inner; + } + + const hostType = String(host.get("type") ?? ""); + const definition = resolveDefinition(doc, hostType); + if (!definition) { + throw new OpRejectedError( + "not_a_subgraph", + `node ${String(host.get("id"))} is not a subgraph; cannot connect inside it`, + ); + } + const definitionId = String(definition.get("id") ?? hostType); + const instances = countDefinitionInstances(doc, definitionId, catalog); + rejectSharedInteriorDefinition(definitionId, instances); + return interiorConnectScope(definition, definitionId); +} + +function rejectSharedInteriorDefinition(definitionId: string, instances: number): void { + if (instances <= 1) return; + throw new OpRejectedError( + "shared_definition_unforked", + `definition ${definitionId} is instantiated ${instances} times; interior writes to shared definitions are rejected until forking is specced (schema §5.3)`, + ); +} + +function interiorConnectScope(definition: Y.Map, definitionId: string): InteriorConnectScope { + const nodes = definition.get("nodes"); + const links = definition.get("links"); + if (!(nodes instanceof Y.Map) || !(links instanceof Y.Map)) { + throw new OpRejectedError("malformed_op", `subgraph definition ${definitionId} has malformed graph storage`); + } + return { nodes: nodes as Y.Map>, links, definition }; +} + +function applyInteriorConnect(doc: Y.Doc, op: ConnectOp, scope: InteriorConnectScope): SuccessfulOutcome { + const linkRefusal = arrayItemRefusal(op.link_id) ?? mapValueRefusal(op.link_id); + if (linkRefusal !== null) { + throw new OpRejectedError("malformed_op", `connect: link_id: ${linkRefusal}`); + } + + const src = scope.nodes.get(String(op.from_node)); + const dst = scope.nodes.get(String(op.to_node)); + if (!dst) return "no-op"; + const sourceOutputs = src ? requireOutputSlot(src, op) : null; + const toIdx = op.to_slot as number; + const inputs = dst.get("inputs"); + if (!(inputs instanceof Y.Array) || toIdx >= inputs.length) { + throw new OpRejectedError( + "input_slot_missing", + `connect: input slot ${String(toIdx)} not found on node ${String(op.to_node)}`, + ); + } + const input = inputs.get(toIdx); + if (!(input instanceof Y.Map)) { + throw new OpRejectedError("input_slot_missing", `connect: input slot ${toIdx} is not a slot record`); + } + + if (!claimLinkIdentity(doc, op, scope)) return "lww-dropped"; + const stamps = stampsMap(doc); + const targetKey = stampTargetKey(op); + const prior = stamps.get(targetKey) as StampKey | undefined; + const key = stampKey(op); + if (prior != null && compareStampKeys(key, prior) <= 0) return "lww-dropped"; + mset(stamps, targetKey, key); + + const previous = input.get("link"); + if (previous != null && previous !== op.link_id) removeLinkInScope(scope, previous); + if (!src || !sourceOutputs) return "no-op"; + + const linkKey = String(op.link_id); + if (!scope.links.has(linkKey)) { + mset(scope.links, linkKey, { + id: op.link_id, + origin_id: op.from_node, + origin_slot: op.from_slot, + target_id: op.to_node, + target_slot: toIdx, + type: op.link_type, + }); + } + const linkOrder = scope.definition.get("link_order"); + const orderedIds: unknown[] = Array.isArray(linkOrder) ? [...linkOrder] : []; + const definitionId = String(scope.definition.get("id") ?? ""); + const orderStampKey = (candidate: string) => + JSON.stringify(["interior_link_order", definitionId, candidate]); + const additions: Record = Object.create(null) as Record; + for (const candidate of orderedIds.map(String)) { + const addedStamp = stamps.get(orderStampKey(candidate)); + if (Array.isArray(addedStamp)) additions[candidate] = addedStamp as StampKey; + } + const wasAdded = Object.hasOwn(additions, linkKey); + const wasPresent = orderedIds.some((candidate) => String(candidate) === linkKey); + const nextOrder = addInteriorLinkOrder(orderedIds, linkKey, key, additions); + if (!wasPresent || wasAdded) { + mset(stamps, orderStampKey(linkKey), key); + } + if (nextOrder.some((candidate, index) => candidate !== orderedIds[index]) || nextOrder.length !== orderedIds.length) { + mset(scope.definition, "link_order", nextOrder); + } + mset(input, "link", op.link_id); + const output = sourceOutputs.get(op.from_slot) as Y.Map; + let outputLinks = output.get("links"); + if (!(outputLinks instanceof Y.Array)) { + outputLinks = new Y.Array(); + mset(output, "links", outputLinks); + } + if (!(outputLinks as Y.Array).toArray().includes(op.link_id)) { + apush(outputLinks as Y.Array, op.link_id); + } + return "applied"; +} + function applyConnect(doc: Y.Doc, op: ConnectOp, catalog?: WidgetCatalog): SuccessfulOutcome { // OP-ONLY validation first, before ANY document read decides the outcome // (KA-4, Amendment A6). requireOpOnlyValid(op); + if (op.path && op.path.length > 0) { + const scope = resolveInteriorConnectScope(doc, op, catalog); + if (scope === null) return "no-op"; + return applyInteriorConnect(doc, op, scope); + } + const nodes = nodesMap(doc); const dst = nodes.get(String(op.to_node)); // The destination is gone → the target slot does not exist and never will @@ -1414,21 +2279,59 @@ function applyConnect(doc: Y.Doc, op: ConnectOp, catalog?: WidgetCatalog): Succe if (!(outLinks as Y.Array).toArray().includes(op.link_id)) { apush(outLinks as Y.Array, op.link_id); } + const tuple = links.get(linkKey) as LinkTuple; + const destination = operationLinkDestination(ins.get(toIdx)!, toIdx, op); + const state: OperationLinkState = { + version: LINK_STATE_DESCRIPTOR_VERSION, + authority: { kind: "operation", stamp: stampKey(op) }, + tuple: structuredClone(tuple), + destination, + }; + mset(linkStateMap(doc), linkKey, state); return "applied"; } +/** Capture the destination that was actually installed, without reclassifying it as an import. */ +function operationLinkDestination( + slot: Y.Map, + toSlot: number, + op: ConnectOp, +): OperationLinkDestination { + const slotRecord = slot.toJSON() as Record; + if (op.grow?.promoted === true) { + return { kind: "promoted", to_slot: toSlot, name: op.grow.name, slot: slotRecord }; + } + if (op.grow != null) { + return { + kind: "autogrow", + to_slot: toSlot, + slot: slotRecord, + request: structuredClone(op.grow), + }; + } + return { kind: "concrete", to_slot: toSlot, slot: slotRecord }; +} + /** Claim the normalized complete-tuple link register (schema Amendment A18). */ -function claimLinkIdentity(doc: Y.Doc, op: ConnectOp): boolean { +function claimLinkIdentity(doc: Y.Doc, op: ConnectOp, scope?: InteriorConnectScope): boolean { const stamps = stampsMap(doc); const normalizedId = String(op.link_id); - const targetKey = JSON.stringify(["link", normalizedId]); + const targetKey = JSON.stringify(["link", ...(scope && op.path ? [op.path.map(String)] : []), normalizedId]); const prior = stamps.get(targetKey) as StampKey | undefined; const key = stampKey(op); if (prior != null && compareStampKeys(key, prior) <= 0) return false; - const links = linksMap(doc); + const links = scope?.links ?? linksMap(doc); if (links.has(normalizedId)) mdel(links, normalizedId); - scrubLinkRefs(doc, (candidate) => candidate != null && String(candidate) === normalizedId); + if (scope) { + scrubNodeLinkRefs(scope.nodes, (candidate) => candidate != null && String(candidate) === normalizedId); + } else { + scrubLinkRefs(doc, (candidate) => candidate != null && String(candidate) === normalizedId); + // This connect generation now owns the top-level normalized identity. + // Interior links belong to a separate scope and cannot retire its intent. + const linkState = linkStateMap(doc); + if (linkState.has(normalizedId)) mdel(linkState, normalizedId); + } mset(stamps, targetKey, key); return true; } @@ -1662,6 +2565,24 @@ function normalizeGrowFamily( const updated = [...link]; updated[4] = positions[rank]; mset(linksMap(doc), String(linkId), updated); + + // This is the same canonical move as the tuple/slot rewrite above, so + // keep an already-installed generation's durable description in that + // move. The currently applying generation is described by applyConnect + // after this function returns; only earlier generations exist here. + const state = linkStateMap(doc).get(String(linkId)); + if (typeof state === "object" && state !== null && !Array.isArray(state)) { + const descriptor = state as OperationLinkState; + mset(linkStateMap(doc), String(linkId), { + ...structuredClone(descriptor), + tuple: structuredClone(updated) as LinkTuple, + destination: { + ...structuredClone(descriptor.destination), + to_slot: positions[rank]!, + slot: structuredClone(desired), + }, + }); + } } }); const wantedRank = snapshots.findIndex( @@ -1760,11 +2681,30 @@ function removeLink(doc: Y.Doc, linkId: unknown): void { const key = String(linkId); if (links.has(key)) mdel(links, key); scrubLinkRefs(doc, (candidate) => candidate != null && String(candidate) === key); + const linkState = linkStateMap(doc); + if (linkState.has(key)) mdel(linkState, key); +} + +function removeLinkInScope(scope: InteriorConnectScope, linkId: unknown): void { + const key = String(linkId); + if (scope.links.has(key)) mdel(scope.links, key); + scrubNodeLinkRefs(scope.nodes, (candidate) => candidate != null && String(candidate) === key); + const linkOrder = scope.definition.get("link_order"); + if (Array.isArray(linkOrder)) { + mset(scope.definition, "link_order", removeInteriorLinkOrder(linkOrder, key)); + } } /** Scrub input/output references selected by one shared link-id predicate. */ function scrubLinkRefs(doc: Y.Doc, shouldRemove: (linkId: unknown) => boolean): void { - nodesMap(doc).forEach((node) => { + scrubNodeLinkRefs(nodesMap(doc), shouldRemove); +} + +function scrubNodeLinkRefs( + nodes: Y.Map>, + shouldRemove: (linkId: unknown) => boolean, +): void { + nodes.forEach((node) => { const ins = node.get("inputs"); if (ins instanceof Y.Array) { ins.forEach((slot: unknown) => { @@ -1834,27 +2774,110 @@ function applyDeleteNode(doc: Y.Doc, op: DeleteNodeOp): SuccessfulOutcome { const presenceWon = prior == null || compareStampKeys(stamp, prior) > 0; const nodeWasPresent = nodes.has(key); if (presenceWon) { + const node = nodes.get(key); + if (node instanceof Y.Map) { + const definition = resolveDefinition(doc, String(node.get("type") ?? "")); + if (definition) { + mset( + stamps, + interiorRouteKey(key, nodeIncarnation(node)), + String(definition.get("id")), + ); + } + } mset(stamps, targetKey, stamp); if (nodes.has(key)) mdel(nodes, key); // absent target → no-op-with-cleanup (delete wins) } const links = linksMap(doc); const removed = new Set(removedLinks); - const toDelete: string[] = []; + const removedIds = new Set([...removed].map(String)); + const toDelete: Array<{ key: string; retire: boolean }> = []; + let retiredStranded = false; links.forEach((ln: unknown, k: string) => { const tuple = ln as unknown[]; if (removed.has(tuple[0])) { - toDelete.push(k); + toDelete.push({ key: k, retire: true }); return; } if (!presenceWon) return; - if (String(tuple[1]) === key || String(tuple[3]) === key) toDelete.push(k); + if (String(tuple[1]) === key || String(tuple[3]) === key) toDelete.push({ key: k, retire: false }); }); - for (const k of toDelete) mdel(links, k); + for (const entry of toDelete) { + mdel(links, entry.key); + if (entry.retire && linkStateMap(doc).has(entry.key)) mdel(linkStateMap(doc), entry.key); + } + // An earlier endpoint deletion can already have removed the live tuple. The + // explicit target list still retires that stranded intent. + for (const [linkKey, raw] of linkStateMap(doc).entries()) { + if (typeof raw !== "object" || raw === null) continue; + const tuple = (raw as OperationLinkState | ImportedLinkState).tuple; + if (Array.isArray(tuple) && removedIds.has(String(tuple[0]))) { + mdel(linkStateMap(doc), linkKey); + retiredStranded = true; + } + } scrubDanglingLinkRefs(doc); - if (!presenceWon && toDelete.length === 0) return "lww-dropped"; - return nodeWasPresent || toDelete.length > 0 ? "applied" : "no-op"; + if (!presenceWon && toDelete.length === 0 && !retiredStranded) return "lww-dropped"; + return nodeWasPresent || toDelete.length > 0 || retiredStranded ? "applied" : "no-op"; +} + +/** + * Reinstall coherent live topology from retained first-class descriptors after + * an endpoint is re-added. Endpoint deletion strands intent but does not + * retire it; explicit disconnect/replacement/delete removed_links do. + * Descriptors are retained indefinitely, including across compaction, until + * one of those semantic retirement operations wins. + */ +function restoreDurableLinks(doc: Y.Doc, nodeId: unknown): void { + const restoredEndpoint = String(nodeId); + const nodes = nodesMap(doc); + linkStateMap(doc).forEach((raw, linkKey) => { + if (typeof raw !== "object" || raw === null) return; + const state = raw as OperationLinkState; + const tuple = state.tuple; + if (!Array.isArray(tuple) || tuple.length !== 6) return; + if (String(tuple[1]) !== restoredEndpoint && String(tuple[3]) !== restoredEndpoint) return; + const src = nodes.get(String(tuple[1])); + const dst = nodes.get(String(tuple[3])); + if (!src || !dst) return; + + const outs = src.get("outputs"); + if (!(outs instanceof Y.Array) || !(outs.get(tuple[2]) instanceof Y.Map)) return; + let ins = dst.get("inputs"); + if (!(ins instanceof Y.Array)) { + ins = new Y.Array(); + mset(dst, "inputs", ins); + } + const inputArray = ins as Y.Array; + const destination = state.destination; + if (!destination || typeof destination.to_slot !== "number") return; + while (inputArray.length <= destination.to_slot) { + const slotIndex = inputArray.length; + if (slotIndex !== destination.to_slot) return; + const slot = new Y.Map(); + for (const [key, value] of Object.entries(destination.slot)) slot.set(key, structuredClone(value)); + inputArray.insert(slotIndex, [slot]); + } + const slot = inputArray.get(destination.to_slot); + if (!(slot instanceof Y.Map)) return; + const incumbent = slot.get("link"); + if (incumbent != null && String(incumbent) !== String(tuple[0])) return; + + const links = linksMap(doc); + if (!links.has(linkKey)) mset(links, linkKey, structuredClone(tuple)); + mset(slot, "link", tuple[0]); + const outPort = outs.get(tuple[2]) as Y.Map; + let outLinks = outPort.get("links"); + if (!(outLinks instanceof Y.Array)) { + outLinks = new Y.Array(); + mset(outPort, "links", outLinks); + } + if (!(outLinks as Y.Array).toArray().some(value => String(value) === String(tuple[0]))) { + apush(outLinks as Y.Array, tuple[0]); + } + }); } /** @@ -1936,7 +2959,7 @@ function applyDisconnect(doc: Y.Doc, op: DisconnectOp): SuccessfulOutcome { const nodes = nodesMap(doc); const dst = nodes.get(String(op.to_node)); - if (!dst) return "no-op"; + if (!dst) return retireStrandedLink(doc, op) ? "applied" : "no-op"; const ins = dst.get("inputs"); if (!(ins instanceof Y.Array) || op.to_slot >= ins.length) { @@ -1951,8 +2974,9 @@ function applyDisconnect(doc: Y.Doc, op: DisconnectOp): SuccessfulOutcome { } const prev = slot.get("link"); + if (prev == null && retireStrandedLink(doc, op)) return "applied"; const stamps = stampsMap(doc); - const targetKey = stampTargetKey(op); + const targetKey = disconnectTargetKey(doc, op, prev); const prior = stamps.get(targetKey) as StampKey | undefined; const key = stampKey(op); if (prior != null && compareStampKeys(key, prior) <= 0) return "lww-dropped"; @@ -1962,6 +2986,46 @@ function applyDisconnect(doc: Y.Doc, op: DisconnectOp): SuccessfulOutcome { return prev != null ? "applied" : "no-op"; } +/** A disconnect addresses the durable destination register that created the occupied slot. */ +function disconnectTargetKey(doc: Y.Doc, op: DisconnectOp, linkId: unknown): string { + const raw = linkId == null ? undefined : linkStateMap(doc).get(String(linkId)); + if (typeof raw === "object" && raw !== null) { + const destination = (raw as OperationLinkState).destination; + if (destination?.kind === "promoted") { + return JSON.stringify(["input", String(op.to_node), "grow", destination.name]); + } + if (destination?.kind === "autogrow" && "request" in destination) { + return JSON.stringify(["input", String(op.to_node), "grow", destination.request.name.split(".", 1)[0]]); + } + } + return stampTargetKey(op); +} + +/** Retire retained intent even when its destination endpoint is currently absent. */ +function retireStrandedLink(doc: Y.Doc, op: DisconnectOp): boolean { + // Disconnect addresses a destination register; link_id is advisory and may + // mismatch the occupant under the established live-slot semantics. + let match: { key: string; state: OperationLinkState | ImportedLinkState } | undefined; + for (const [key, raw] of linkStateMap(doc).entries()) { + if (typeof raw !== "object" || raw === null) continue; + const state = raw as OperationLinkState | ImportedLinkState; + if (!Array.isArray(state.tuple) || !state.destination) continue; + if (String(state.tuple[3]) === String(op.to_node) && state.destination.to_slot === op.to_slot) { + match = { key, state }; + break; + } + } + if (match === undefined) return false; + const targetKey = disconnectTargetKey(doc, op, match.state.tuple[0]); + const stamps = stampsMap(doc); + const prior = stamps.get(targetKey) as StampKey | undefined; + const stamp = stampKey(op); + if (prior != null && compareStampKeys(stamp, prior) <= 0) return false; + mset(stamps, targetKey, stamp); + mdel(linkStateMap(doc), match.key); + return true; +} + // --------------------------------------------------------------------------- // clear // --------------------------------------------------------------------------- diff --git a/src/collab-trace.ts b/src/collab-trace.ts index 2189a995..4589c23a 100644 --- a/src/collab-trace.ts +++ b/src/collab-trace.ts @@ -352,7 +352,17 @@ function assertOpPayload(value: unknown, context: string) { const stamp = assertStamp(required(payload, "stamp", context), `${context}.stamp`); if (stamp[1] !== actor) invalid(`${context}.stamp`, "does not preserve semantic identity"); - if (op === "add_node") { + if (op === "insert_workflow") { + const workflow = asRecord(required(payload, "workflow", context), `${context}.workflow`); + asArray(required(workflow, "nodes", `${context}.workflow`), `${context}.workflow.nodes`); + if (Object.hasOwn(workflow, "links")) asArray(workflow["links"], `${context}.workflow.links`); + if (Object.hasOwn(workflow, "definitions")) { + const definitions = asRecord(workflow["definitions"], `${context}.workflow.definitions`); + if (Object.hasOwn(definitions, "subgraphs")) { + asArray(definitions["subgraphs"], `${context}.workflow.definitions.subgraphs`); + } + } + } else if (op === "add_node") { const nodeId = asNodeId(required(payload, "node_id", context), `${context}.node_id`); const classType = asString(required(payload, "class_type", context), `${context}.class_type`); assertNumberArray(required(payload, "pos", context), `${context}.pos`); @@ -414,6 +424,9 @@ function assertOpPayload(value: unknown, context: string) { assertNodeIdArray(required(payload, "removed_links", context), `${context}.removed_links`); } else if (op === "clear") { assertNodeIdArray(required(payload, "removed_nodes", context), `${context}.removed_nodes`); + } else if (op === "define_subgraph") { + asString(required(payload, "subgraph_id", context), `${context}.subgraph_id`); + asRecord(required(payload, "subgraph_definition", context), `${context}.subgraph_definition`); } else { // Exhaustiveness guard (issue #21), matching `dispatch` in `applier.ts`: // `op` is narrowed from `FROZEN_OPS`, so with every kind enumerated above diff --git a/src/doc.ts b/src/doc.ts index ca1d6eb4..214abb48 100644 --- a/src/doc.ts +++ b/src/doc.ts @@ -158,6 +158,7 @@ export const ROOT_DEFINITIONS = "definitions"; export const ROOT_META = "meta"; export const ROOT_APPLIED = "__applied"; export const ROOT_STAMPS = "__stamps"; +export const ROOT_LINK_STATE = "__link_state"; /** Root map holding one Y.Map per node, keyed by String(node id). */ export function nodesMap(doc: Y.Doc): Y.Map> { @@ -200,6 +201,11 @@ export function stampsMap(doc: Y.Doc): Y.Map { return doc.getMap(ROOT_STAMPS); } +/** First-class durable link intent, keyed by normalized link id (schema v3). */ +export function linkStateMap(doc: Y.Doc): Y.Map { + return doc.getMap(ROOT_LINK_STATE); +} + /** * Initialize the v1 layout on a fresh doc (idempotent). Creates the root maps * (including bookkeeping) and seeds meta with schema_version, the pinned @@ -215,6 +221,7 @@ export function initDoc(doc: Y.Doc, catalogVersion = ""): void { definitionsMap(doc); appliedMap(doc); stampsMap(doc); + linkStateMap(doc); const meta = metaMap(doc); if (meta.get("schema_version") === undefined) { meta.set("schema_version", SCHEMA_VERSION); @@ -804,12 +811,12 @@ export function createNodeMap(node: WorkflowNode, widgetOrder?: readonly string[ * never resolve). */ export function resolveDefinition(doc: Y.Doc, key: string): Y.Map | null { - const defs = definitionsMap(doc); - const byId = defs.get(key); + const all = allDefinitions(doc); + const byId = all.find((definition) => String(definition.get("id")) === key); if (byId) return byId; let found: Y.Map | null = null; let count = 0; - defs.forEach((dm) => { + all.forEach((dm) => { if (String(dm.get("name") ?? "") === key) { count++; found = dm; @@ -818,6 +825,20 @@ export function resolveDefinition(doc: Y.Doc, key: string): Y.Map | nul return count === 1 ? found : null; } +function allDefinitions(doc: Y.Doc): Y.Map[] { + const all: Y.Map[] = []; + const visit = (definition: Y.Map): void => { + all.push(definition); + const container = definition.get("definitions"); + const nested = container instanceof Y.Map ? container.get("subgraphs") : undefined; + if (nested instanceof Y.Map) nested.forEach((child) => { + if (child instanceof Y.Map) visit(child); + }); + }; + definitionsMap(doc).forEach(visit); + return all; +} + /** * Every node `type` that {@link resolveDefinition} maps to the definition * `defId` addresses: the definition's own map key, plus its cosmetic `name` @@ -859,7 +880,7 @@ function definitionAliases(doc: Y.Doc, defId: string, catalog?: WidgetCatalog): if (!catalog) return aliases; // no catalogue to ask: cannot verify, so not an alias if (Object.prototype.hasOwnProperty.call(catalog.types, name)) return aliases; // a node class let sameName = 0; - defs.forEach((dm) => { + allDefinitions(doc).forEach((dm) => { if (String(dm.get("name") ?? "") === name) sameName++; }); if (sameName === 1) aliases.add(name); @@ -893,6 +914,21 @@ function definitionAliases(doc: Y.Doc, defId: string, catalog?: WidgetCatalog): export function countDefinitionInstances(doc: Y.Doc, defId: string, catalog?: WidgetCatalog): number { const aliases = definitionAliases(doc, defId, catalog); let count = 0; + const visitDefinition = (definition: Y.Map): void => { + const inner = definition.get("nodes"); + if (inner instanceof Y.Map) { + inner.forEach((node: unknown) => { + if (node instanceof Y.Map && aliases.has(String(node.get("type") ?? ""))) count++; + }); + } + const container = definition.get("definitions"); + const nested = container instanceof Y.Map ? container.get("subgraphs") : undefined; + if (nested instanceof Y.Map) { + nested.forEach((child: unknown) => { + if (child instanceof Y.Map) visitDefinition(child); + }); + } + }; nodesMap(doc).forEach((node, key) => { if (!(node instanceof Y.Map)) { throw new TypeError( @@ -902,12 +938,7 @@ export function countDefinitionInstances(doc: Y.Doc, defId: string, catalog?: Wi if (aliases.has(String(node.get("type") ?? ""))) count++; }); definitionsMap(doc).forEach((dm) => { - const inner = dm.get("nodes"); - if (inner instanceof Y.Map) { - inner.forEach((node: unknown) => { - if (node instanceof Y.Map && aliases.has(String(node.get("type") ?? ""))) count++; - }); - } + visitDefinition(dm); }); return count; } diff --git a/src/index.ts b/src/index.ts index b72a653a..d87bc24c 100644 --- a/src/index.ts +++ b/src/index.ts @@ -2,7 +2,8 @@ * @comfyorg/comfy-multi-player — shared workflow-document package. * * One implementation of op→doc semantics, used identically by the browser - * and the server doc host. The op vocabulary is frozen at six kinds; the + * and the server doc host. The public vocabulary declares nine operation + * kinds: eight implemented kinds plus the deferred `reset_doc` kind. The * normative contract is comfy-cli's `docs/op-vocabulary-v1.md` and the stamp * shapes minted by `comfy_cli/workflow_ops.py` (`_new_op`), both pinned at * comfy-cli commit `7e732242d971daf0d2d30f22f997abfacd78986e` — by SHA and @@ -84,6 +85,7 @@ export { hasAppliedOp, hasNode, readGraph, + readLinkState, readMeta, readStamps, type GraphSnapshot, diff --git a/src/interior-link-order.ts b/src/interior-link-order.ts new file mode 100644 index 00000000..183be0f0 --- /dev/null +++ b/src/interior-link-order.ts @@ -0,0 +1,28 @@ +import { compareStampKeys } from "./stamps.js"; +import type { StampKey } from "./types.js"; + +export function addInteriorLinkOrder( + order: unknown[], + id: string, + stamp: StampKey, + priorAdditions: Record, +): string[] { + const ids = order.map(String); + const wasAdded = Object.hasOwn(priorAdditions, id); + if (ids.includes(id) && !wasAdded) return ids; + const additions = { ...priorAdditions, [id]: stamp }; + const withId = ids.includes(id) ? ids : [...ids, id]; + const imported = withId.filter((candidate) => !Object.hasOwn(additions, candidate)); + const added = withId.filter((candidate) => Object.hasOwn(additions, candidate)); + added.sort((a, b) => compareStampKeys(additions[a]!, additions[b]!)); + return [...imported, ...added]; +} + +/** The stored scalar order is already canonical for unchanged-schema readers. */ +export function projectInteriorLinkOrder(order: unknown[]): string[] { + return order.map(String); +} + +export function removeInteriorLinkOrder(order: unknown[], id: string): unknown[] { + return order.filter((entry) => String(entry) !== id); +} diff --git a/src/migrate.ts b/src/migrate.ts index b61d9571..d13d0bce 100644 --- a/src/migrate.ts +++ b/src/migrate.ts @@ -1,97 +1,24 @@ /** - * Schema-layout versioning (schema §10): stepwise vN → vN+1 migrations, - * composed in order; exact no-op at the current version; FAIL-CLOSED on a + * Schema-layout versioning (schema §10): exact no-op at the current version; + * older schemas are refused under the private-alpha no-compat-reader policy; + * FAIL-CLOSED on a * doc newer than this package, or one whose schema cannot be read at all — - * never a best-effort read. Host-only: followers receive the migrated doc via + * never a best-effort read. Host-only: followers receive the current-format doc via * the struct stream / a new epoch. */ import * as Y from "yjs"; -import { ROOT_META, ROOT_STAMPS, definitionsMap, nodesMap } from "./doc.js"; import { readSchemaVersion } from "./schema-version.js"; -import { compareStampKeys } from "./stamps.js"; -import { - LEGACY_NODE_INCARNATION, - NODE_INCARNATION_KEY, - SCHEMA_VERSION, - SchemaVersionError, - type StampKey, -} from "./types.js"; - -function migrateNodeMap(node: unknown): void { - if (!(node instanceof Y.Map)) return; - if (!node.has(NODE_INCARNATION_KEY)) node.set(NODE_INCARNATION_KEY, LEGACY_NODE_INCARNATION); -} - -function migrateStampKey(key: string): string | null { - let parsed: unknown; - try { - parsed = JSON.parse(key); - } catch { - return null; - } - if ( - !Array.isArray(parsed) || - parsed[0] !== "widget" || - parsed.length !== 3 || - (typeof parsed[1] !== "string" && typeof parsed[1] !== "number") || - typeof parsed[2] !== "string" - ) { - return null; - } - return JSON.stringify([parsed[0], String(parsed[1]), LEGACY_NODE_INCARNATION, parsed[2]]); -} - -function isStampKey(value: unknown): value is StampKey { - return ( - Array.isArray(value) && - value.length === 3 && - typeof value[0] === "number" && - Number.isSafeInteger(value[0]) && - value[0] >= 0 && - typeof value[1] === "string" && - typeof value[2] === "string" - ); -} - -/** Apply the v1 → v2 compatibility translation in one host-owned transaction. */ -function migrateV1ToV2(doc: Y.Doc): void { - doc.transact(() => { - nodesMap(doc).forEach(migrateNodeMap); - definitionsMap(doc).forEach((definition) => { - if (!(definition instanceof Y.Map)) return; - const nodes = definition.get("nodes"); - if (nodes instanceof Y.Map) nodes.forEach(migrateNodeMap); - }); - - const stamps = doc.getMap(ROOT_STAMPS); - for (const oldKey of [...stamps.keys()]) { - const newKey = migrateStampKey(oldKey); - if (newKey === null) continue; - const value = stamps.get(oldKey); - if (!isStampKey(value)) continue; - const prior = stamps.get(newKey); - if (prior === undefined || !isStampKey(prior) || compareStampKeys(value, prior) > 0) { - stamps.set(newKey, value); - } - stamps.delete(oldKey); - } - - doc.getMap(ROOT_META).set("schema_version", SCHEMA_VERSION); - }); -} +import { SCHEMA_VERSION, SchemaVersionError } from "./types.js"; /** - * Migrate a doc from schema `fromVersion` to `SCHEMA_VERSION`, in place. + * Validate that a doc is already at `SCHEMA_VERSION`. * - * The v1 → v2 step seeds the legacy incarnation (`"0"`) on imported nodes and - * rewrites legacy widget stamp keys into that namespace. New add operations - * carry their immutable `op_id` as the incarnation token. - * - * Validation runs before the migration step on EVERY path (KA-11: schema-version - * discipline is enforced on read). A rejected call and a current-version no-op - * leave the `encodeStateAsUpdate` byte-identical and the `doc.share` key set - * unchanged; only the explicit v1 → v2 path mutates the document. + * The current-version path is an exact no-op. Older, newer, unreadable, and + * caller/document-mismatched versions are refused fail-closed (KA-11). Every + * path leaves `encodeStateAsUpdate` byte-identical and the `doc.share` key set + * unchanged; this function provides no compatibility reader and never + * relabels an old layout as the current one. */ export function migrate(doc: Y.Doc, fromVersion: number): void { if (!Number.isInteger(fromVersion) || fromVersion < 1) { @@ -131,9 +58,10 @@ export function migrate(doc: Y.Doc, fromVersion: number): void { ); } - if (fromVersion === 1) { - migrateV1ToV2(doc); - return; + if (fromVersion < SCHEMA_VERSION) { + throw new SchemaVersionError( + `migrate: schema v${fromVersion} is unsupported by the private-alpha no-compat-reader policy; refusing to relabel it as v${SCHEMA_VERSION}`, + ); } // The current-version path is an EXACT no-op, byte-identical under diff --git a/src/mint.ts b/src/mint.ts index b5cb95e0..34338225 100644 --- a/src/mint.ts +++ b/src/mint.ts @@ -16,11 +16,21 @@ import { createNodeMap, definitionsMap, linksMap, + linkStateMap, metaMap, nodesMap, stampsMap, } from "./doc.js"; -import { SCHEMA_VERSION, type WidgetCatalog, type WorkflowJSON, type WorkflowNode } from "./types.js"; +import { + LINK_STATE_DESCRIPTOR_VERSION, + SCHEMA_VERSION, + type ImportedLinkDestination, + type ImportedLinkState, + type LinkTuple, + type WidgetCatalog, + type WorkflowJSON, + type WorkflowNode, +} from "./types.js"; /** Top-level keys that are NOT meta passthrough: structural keys get their own root maps; comfy-cli bookkeeping is never imported. */ const NON_META_KEYS = ["nodes", "links", "definitions", "_applied_ops", "_widget_stamps"] as const; @@ -47,9 +57,9 @@ interface SubgraphDef { * map through `catalog` — schema §1.2); * - `definitions.subgraphs` into the first-class `definitions` root map * (schema §5.1), with interior node/link mint order preserved in plain - * `node_order`/`link_order` registers (interior order is static in v1 — - * only `set_widget` is subgraph-scoped); non-`subgraphs` keys of the - * definitions container are kept in the internal `__definitions_extra` + * `node_order`/`link_order` registers; later interior links retain this + * imported prefix and project in deterministic stamp order; + * non-`subgraphs` keys of the definitions container are kept in the internal `__definitions_extra` * meta key and merged back at projection; * - every other top-level key into meta as opaque passthrough (schema §6); * - `schema_version` + the pinned `catalogVersion` into meta. @@ -81,6 +91,7 @@ export function mint(workflow: WorkflowJSON, catalog: WidgetCatalog, catalogVers definitionsMap(doc); appliedMap(doc); stampsMap(doc); + const linkState = linkStateMap(doc); for (const [k, v] of Object.entries(workflow)) { if (NON_META_KEYS.includes(k as (typeof NON_META_KEYS)[number])) continue; @@ -97,7 +108,12 @@ export function mint(workflow: WorkflowJSON, catalog: WidgetCatalog, catalogVers const links = linksMap(doc); for (const ln of workflow.links ?? []) { - links.set(String((ln as unknown[])[0]), cloneForMap(ln, "mint: link")); + const tuple = cloneForMap(ln, "mint: link"); + const state = importedLinkState(tuple, workflow); + const key = String((tuple as unknown[])[0]); + links.set(key, tuple); + if (state === null) linkState.delete(key); + else linkState.set(key, cloneForMap(state, `mint: link state ${key}`)); } const defsIn = workflow["definitions"]; @@ -123,6 +139,93 @@ export function mint(workflow: WorkflowJSON, catalog: WidgetCatalog, catalogVers return doc; } +function nodeId(value: unknown, context: string): string | number { + if (typeof value === "string" && value.length > 0) return value; + if (typeof value === "number" && Number.isSafeInteger(value)) return value; + throw new TypeError(`mint: ${context} must be a non-empty string or safe integer`); +} + +function recordAt(value: unknown, index: number, context: string): Record | null { + if (!Array.isArray(value) || index >= value.length) return null; + const slot = value[index]; + if (typeof slot !== "object" || slot === null || Array.isArray(slot)) { + throw new TypeError(`mint: ${context} slot ${String(index)} is not an object`); + } + return slot as Record; +} + +function retainedNode(workflow: WorkflowJSON, id: string | number): WorkflowNode | undefined { + for (let index = workflow.nodes.length - 1; index >= 0; index -= 1) { + const node = workflow.nodes[index]; + if (node !== undefined && String(node.id) === String(id)) return node; + } + return undefined; +} + +/** Build durable intent only when an imported tuple and both endpoint slot references agree. */ +export function importedLinkState(raw: unknown, workflow: WorkflowJSON): ImportedLinkState | null { + if (!Array.isArray(raw) || raw.length !== 6) { + throw new TypeError("mint: link tuple must contain exactly six fields"); + } + const [id, fromNodeId, fromSlot, toNodeId, toSlot] = raw; + const linkId = nodeId(id, "link tuple id"); + const sourceId = nodeId(fromNodeId, `link ${String(linkId)} source node id`); + const targetId = nodeId(toNodeId, `link ${String(linkId)} destination node id`); + if (!Number.isSafeInteger(fromSlot) || (fromSlot as number) < 0 || !Number.isSafeInteger(toSlot) || (toSlot as number) < 0) { + throw new TypeError(`mint: link ${String(id)} tuple slots must be non-negative integers`); + } + const source = retainedNode(workflow, sourceId); + const target = retainedNode(workflow, targetId); + // Existing workflow JSON can carry a dangling tuple while a save is in + // progress or after an older producer incompletely scrubbed a node. mint() + // has always preserved those tuples. They cannot yield a truthful durable + // destination descriptor, so preserve the link and omit only __link_state. + if (source === undefined || target === undefined) return null; + const output = recordAt(source.outputs, fromSlot as number, `link ${String(id)} source output`); + const input = recordAt(target.inputs, toSlot as number, `link ${String(id)} destination input`); + if (output === null || input === null) return null; + if (!Array.isArray(output["links"]) || !output["links"].some((ref) => + (typeof ref === "string" || typeof ref === "number") && String(ref) === String(linkId))) { + return null; + } + if ((typeof input["link"] !== "string" && typeof input["link"] !== "number") || + String(input["link"]) !== String(linkId)) { + return null; + } + + const definitions = workflow["definitions"]; + const subgraphsValue = typeof definitions === "object" && definitions !== null && !Array.isArray(definitions) + ? (definitions as Record)["subgraphs"] + : undefined; + const subgraphs = Array.isArray(subgraphsValue) + ? subgraphsValue.filter((value): value is SubgraphDef => typeof value === "object" && value !== null && !Array.isArray(value)) + : []; + const definition = subgraphs.find((candidate) => + String(candidate.id) === String(target.type) || + (typeof candidate["name"] === "string" && candidate["name"] === target.type), + ); + const declaredInputs = Array.isArray(definition?.["inputs"]) + ? definition["inputs"].filter((value): value is Record => + typeof value === "object" && value !== null && !Array.isArray(value)) + : []; + const slot = input; + const name = slot["name"]; + let destination: ImportedLinkDestination; + if (typeof name === "string" && declaredInputs.some((declared) => declared["name"] === name)) { + destination = { kind: "promoted", to_slot: toSlot as number, name, slot: structuredClone(slot) }; + } else if (String(slot["grow_id"]) === String(id)) { + destination = { kind: "autogrow", to_slot: toSlot as number, slot: structuredClone(slot) }; + } else { + destination = { kind: "concrete", to_slot: toSlot as number, slot: structuredClone(slot) }; + } + return { + version: LINK_STATE_DESCRIPTOR_VERSION, + authority: "imported", + tuple: structuredClone(raw) as LinkTuple, + destination, + }; +} + /** * The `links` map key for one interior link of a definition: the tuple's * `[0]` for the litegraph array form, `id` for the frontend's object form, and @@ -138,14 +241,19 @@ function definitionLinkKey(ln: unknown, index: number): string { return `#${String(index)}`; } -function mintDefinition(sg: SubgraphDef, catalog: WidgetCatalog): Y.Map { +export function mintDefinition(sg: SubgraphDef, catalog: WidgetCatalog): Y.Map { + for (const key of ["node_order", "link_order", "__definition_digest"]) { + if (Object.hasOwn(sg, key)) throw new TypeError(`mint: definition key '${key}' is reserved`); + } const dm = new Y.Map(); for (const [k, v] of Object.entries(sg)) { if (k === "nodes" && Array.isArray(v)) { const nm = new Y.Map>(); const order: string[] = []; for (const n of v as WorkflowNode[]) { + if (n.id === undefined || n.id === null) throw new TypeError("mint: definition node is missing id"); const key = String(n.id); + if (order.includes(key)) throw new TypeError(`mint: duplicate definition node id '${key}'`); order.push(key); nm.set(key, createNodeMap(n, widgetOrderFor(catalog, n.type))); } @@ -167,6 +275,24 @@ function mintDefinition(sg: SubgraphDef, catalog: WidgetCatalog): Y.Map }); dm.set("links", lm); dm.set("link_order", order); + } else if (k === "definitions" && typeof v === "object" && v !== null && !Array.isArray(v)) { + const container = new Y.Map(); + const { subgraphs, ...extra } = v as { subgraphs?: unknown; [key: string]: unknown }; + Object.entries(extra).forEach(([key, value]) => container.set(key, cloneForMap(value, `mint: definition.definitions.${key}`))); + if (Array.isArray(subgraphs)) { + const nested = new Y.Map>(); + const order: string[] = []; + for (const child of subgraphs as SubgraphDef[]) { + if (child.id === undefined || child.id === null) throw new TypeError("mint: nested definition is missing id"); + const key = String(child.id); + if (order.includes(key)) throw new TypeError(`mint: duplicate nested definition id '${key}'`); + order.push(key); + nested.set(key, mintDefinition(child, catalog)); + } + container.set("subgraphs", nested); + container.set("subgraph_order", order); + } + dm.set("definitions", container); } else { dm.set(k, cloneForMap(v, `mint: definition.${k}`)); } diff --git a/src/project.ts b/src/project.ts index 0a38c591..cd97cba7 100644 --- a/src/project.ts +++ b/src/project.ts @@ -19,8 +19,8 @@ * * Subgraph definitions project as `{...extra, subgraphs: [...]}` with * definitions sorted by id and each definition's interior nodes/links in - * mint order (`node_order`/`link_order` — interior order is static in v1 - * because only `set_widget` is subgraph-scoped). + * mint order (`node_order`/`link_order`); links added by semantic ops follow + * imported links in deterministic stamp order. */ import * as Y from "yjs"; @@ -33,6 +33,7 @@ import { widgetStorageOf, } from "./doc.js"; import { assertNever } from "./exhaustive.js"; +import { projectInteriorLinkOrder } from "./interior-link-order.js"; import { assertReadableSchema } from "./schema-version.js"; import { NODE_INCARNATION_KEY, type WidgetCatalog, type WorkflowJSON, type WorkflowNode } from "./types.js"; @@ -221,11 +222,11 @@ function tryProjectNode(value: unknown, catalog: WidgetCatalog): WorkflowNode | return projectNode(value, catalog); } -/** Definition Y.Map → subgraph definition JSON, interior nodes/links in mint order. */ -function projectDefinition(dm: Y.Map, catalog: WidgetCatalog): Record { +/** Definition Y.Map → subgraph JSON, preserving mint order before deterministic additions. */ +export function projectDefinition(dm: Y.Map, catalog: WidgetCatalog): Record { const out: Record = Object.create(null) as Record; dm.forEach((v, k) => { - if (k === "node_order" || k === "link_order") return; // internal order registers + if (k === "node_order" || k === "link_order" || k.startsWith("__")) return; // internal registers if (k === "nodes" && v instanceof Y.Map) { const order = (dm.get("node_order") as string[] | undefined) ?? [...v.keys()].sort(); out[k] = order @@ -233,12 +234,41 @@ function projectDefinition(dm: Y.Map, catalog: WidgetCatalog): Record tryProjectNode(v.get(id), catalog)) .filter((node): node is WorkflowNode => node !== null); } else if (k === "links" && v instanceof Y.Map) { - const order = (dm.get("link_order") as string[] | undefined) ?? [...v.keys()].sort(); + const storedOrder = dm.get("link_order"); + const order = Array.isArray(storedOrder) + ? projectInteriorLinkOrder(storedOrder) + : [...v.keys()].sort(); out[k] = order.filter((id) => v.has(id)).map((id) => structuredClone(v.get(id))); + } else if (k === "definitions" && v instanceof Y.Map) { + const nestedOut: Record = Object.create(null) as Record; + v.forEach((nestedValue, nestedKey) => { + if (nestedKey === "subgraph_order") return; + if (nestedKey === "subgraphs" && nestedValue instanceof Y.Map) { + const order = (v.get("subgraph_order") as string[] | undefined) ?? [...nestedValue.keys()].sort(); + nestedOut.subgraphs = order + .filter((id) => nestedValue.has(id)) + .map((id) => nestedValue.get(id)) + .filter((definition): definition is Y.Map => definition instanceof Y.Map) + .map((definition) => projectDefinition(definition, catalog)); + } else { + nestedOut[nestedKey] = structuredClone(nestedValue); + } + }); + out[k] = nestedOut; } else { out[k] = structuredClone(v); } }); + return scrubPrivateKeys(out) as Record; +} + +function scrubPrivateKeys(value: unknown): unknown { + if (Array.isArray(value)) return value.map(scrubPrivateKeys); + if (typeof value !== "object" || value === null) return value; + const out: Record = Object.create(null) as Record; + for (const [key, child] of Object.entries(value)) { + if (!key.startsWith("__")) out[key] = scrubPrivateKeys(child); + } return out; } diff --git a/src/read.ts b/src/read.ts index 90211cd6..0e38f868 100644 --- a/src/read.ts +++ b/src/read.ts @@ -61,11 +61,13 @@ import { OPAQUE_WIDGETS_KEY, ROOT_APPLIED, ROOT_LINKS, + ROOT_LINK_STATE, ROOT_META, ROOT_NODES, ROOT_STAMPS, } from "./doc.js"; import { assertReadableSchema } from "./schema-version.js"; +import { LINK_STATE_DESCRIPTOR_VERSION } from "./types.js"; /** * The reserved per-node key holding a whole `widgets_values` array verbatim for @@ -436,3 +438,65 @@ export function readStamps(doc: Y.Doc): Readonly> { assertSnapshotReadable(doc, "readStamps"); return snapshotRoot(doc, ROOT_STAMPS); } + +/** Durable imported and operation-owned link descriptors as deep-frozen plain data. */ +export function readLinkState(doc: Y.Doc): Readonly> { + assertSnapshotReadable(doc, "readLinkState"); + const state = snapshotRoot(doc, ROOT_LINK_STATE); + for (const [id, raw] of Object.entries(state)) { + if (typeof raw !== "object" || raw === null || Array.isArray(raw)) { + throw new TypeError(`readLinkState: link ${id} is not a descriptor`); + } + const descriptor = raw as Record; + const tuple = descriptor["tuple"]; + const destinationValue = descriptor["destination"]; + const destination = typeof destinationValue === "object" && destinationValue !== null && !Array.isArray(destinationValue) + ? destinationValue as Record + : undefined; + const validId = (value: unknown): boolean => + (typeof value === "string" && value.length > 0) || (typeof value === "number" && Number.isSafeInteger(value)); + const validSlot = (value: unknown): boolean => Number.isSafeInteger(value) && (value as number) >= 0; + const validSlotRecord = destination !== undefined && typeof destination["slot"] === "object" && + destination["slot"] !== null && !Array.isArray(destination["slot"]); + const slotRecord = validSlotRecord ? destination!["slot"] as Record : undefined; + const authority = descriptor["authority"]; + const operationAuthority = typeof authority === "object" && authority !== null && !Array.isArray(authority) + ? authority as Record + : undefined; + const operationStamp = operationAuthority?.["stamp"]; + // Match validateEnvelope/stampKey, including the empty fallback actor and + // non-empty opaque op IDs. Reading must not reject records our writer emits. + const validAuthority = authority === "imported" || + (operationAuthority?.["kind"] === "operation" && Array.isArray(operationStamp) && operationStamp.length === 3 && + Number.isSafeInteger(operationStamp[0]) && (operationStamp[0] as number) >= 0 && + typeof operationStamp[1] === "string" && + typeof operationStamp[2] === "string" && operationStamp[2].length > 0); + const requestValue = destination?.["request"]; + const request = typeof requestValue === "object" && requestValue !== null && !Array.isArray(requestValue) + ? requestValue as Record + : undefined; + const inputcountValue = request?.["inputcount"]; + const inputcount = typeof inputcountValue === "object" && inputcountValue !== null && !Array.isArray(inputcountValue) + ? inputcountValue as Record + : undefined; + const validRequest = destination?.["kind"] !== "autogrow" || authority === "imported" || + (request !== undefined && typeof request["name"] === "string" && + typeof request["type"] === "string" && + (request["widget"] === undefined || typeof request["widget"] === "string") && + (request["promoted"] === undefined || typeof request["promoted"] === "boolean") && + (request["inputcount"] === undefined || + (inputcount !== undefined && typeof inputcount["widget"] === "string" && "value" in inputcount))); + if (descriptor["version"] !== LINK_STATE_DESCRIPTOR_VERSION || !validAuthority || + !Array.isArray(tuple) || tuple.length !== 6 || String(tuple[0]) !== id || + !validId(tuple[0]) || !validId(tuple[1]) || !validSlot(tuple[2]) || + !validId(tuple[3]) || !validSlot(tuple[4]) || destination === undefined || + !validSlot(destination["to_slot"]) || destination["to_slot"] !== tuple[4] || !validSlotRecord || + (slotRecord !== undefined && String(slotRecord["link"]) !== String(tuple[0])) || + !["concrete", "promoted", "autogrow"].includes(String(destination["kind"])) || + !validRequest || + (destination["kind"] === "promoted" && typeof destination["name"] !== "string")) { + throw new TypeError(`readLinkState: link ${id} has an unsupported or malformed descriptor`); + } + } + return state; +} diff --git a/src/remap.ts b/src/remap.ts new file mode 100644 index 00000000..197025e5 --- /dev/null +++ b/src/remap.ts @@ -0,0 +1,142 @@ +import type { WorkflowJSON, WorkflowNode } from "./types.js"; +import { sha256Hex } from "./digest.js"; + +function derivedId(opId: string, scope: string, kind: string, original: unknown): string { + return `insert:${opId}:${scope}:${kind}:${encodeURIComponent(JSON.stringify(original))}`; +} + +function derivedDefinitionId(opId: string, scope: string, original: unknown): string { + const hex = sha256Hex(derivedId(opId, scope, "definition", original)); + return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-4${hex.slice(13, 16)}-8${hex.slice(17, 20)}-${hex.slice(20, 32)}`; +} + +function linkEndpoints(link: unknown): [unknown, unknown] | undefined { + if (Array.isArray(link) && link[1] !== undefined && link[3] !== undefined) return [link[1], link[3]]; + if (typeof link === "object" && link !== null) { + const record = link as { origin_id?: unknown; target_id?: unknown }; + if (record.origin_id !== undefined && record.target_id !== undefined) return [record.origin_id, record.target_id]; + } + return undefined; +} + +function normalizedId(id: unknown): string { + return typeof id === "string" ? id : String(id); +} + +function linkId(link: unknown): unknown { + if (Array.isArray(link)) return link[0]; + if (typeof link === "object" && link !== null) return (link as { id?: unknown }).id; + return undefined; +} + +export function linkHasMissingEndpoint(link: unknown, hasNode: (id: unknown) => boolean): boolean { + const endpoints = linkEndpoints(link); + return endpoints !== undefined && (!hasNode(endpoints[0]) || !hasNode(endpoints[1])); +} + +function remapGraph( + graph: Record, + opId: string, + scope: string, + definitionIds: Map, + dropDanglingLinks: boolean, +): void { + const nodes = graph["nodes"] as WorkflowNode[]; + const nodeIds = new Map(); + const linkIds = new Map(); + // Numeric and string aliases normalized by validation name the same node. + for (const node of nodes) nodeIds.set(normalizedId(node.id), derivedId(opId, scope, "node", node.id)); + const droppedLinkIds = new Set(); + const links = ((graph["links"] as unknown[] | undefined) ?? []).filter((link) => { + const dropped = dropDanglingLinks && linkHasMissingEndpoint(link, (id) => nodeIds.has(normalizedId(id))); + if (dropped) droppedLinkIds.add(normalizedId(linkId(link))); + return !dropped; + }); + for (const link of links) { + const id = linkId(link); + linkIds.set(normalizedId(id), derivedId(opId, scope, "link", id)); + } + + for (const node of nodes) { + node.id = nodeIds.get(normalizedId(node.id))!; + if (definitionIds.has(node.type)) node.type = definitionIds.get(node.type)!; + if (Array.isArray(node.inputs)) { + for (const input of node.inputs) { + if (typeof input === "object" && input !== null && "link" in input) { + const record = input as { link?: unknown }; + if (record.link === null || record.link === undefined) continue; + const id = normalizedId(record.link); + if (droppedLinkIds.has(id)) record.link = null; + else if (linkIds.has(id)) record.link = linkIds.get(id); + } + } + } + if (Array.isArray(node.outputs)) { + for (const output of node.outputs) { + if (typeof output === "object" && output !== null && Array.isArray((output as { links?: unknown }).links)) { + const record = output as { links: unknown[] }; + record.links = record.links + .filter((id) => !droppedLinkIds.has(normalizedId(id))) + .map((id) => linkIds.get(normalizedId(id)) ?? id); + } + } + } + } + graph["links"] = links.map((link) => { + if (Array.isArray(link)) { + link[0] = linkIds.get(normalizedId(link[0])) ?? link[0]; + link[1] = nodeIds.get(normalizedId(link[1])) ?? link[1]; + link[3] = nodeIds.get(normalizedId(link[3])) ?? link[3]; + } else if (typeof link === "object" && link !== null) { + const record = link as { id?: unknown; origin_id?: unknown; target_id?: unknown }; + record.id = linkIds.get(normalizedId(record.id)) ?? record.id; + record.origin_id = nodeIds.get(normalizedId(record.origin_id)) ?? record.origin_id; + record.target_id = nodeIds.get(normalizedId(record.target_id)) ?? record.target_id; + } + return link; + }); + if (Array.isArray(graph["groups"])) { + graph["groups"] = (graph["groups"] as unknown[]).map((group) => { + if (typeof group === "object" && group !== null && !Array.isArray(group)) { + const record = group as { id?: unknown }; + if (record.id !== undefined) record.id = derivedId(opId, scope, "group", record.id); + } + return group; + }); + } +} + +/** Deterministically remap every id carried by an insertion op, including nested definition graphs. */ +export function remapInsertedWorkflowIds(wf: WorkflowJSON, opId: string): WorkflowJSON { + const out = structuredClone(wf) as WorkflowJSON & Record; + out.links ??= []; + out.groups ??= []; + const subgraphs = (out.definitions?.subgraphs ?? []) as Array>; + const definitionIdsByScope = new Map>(); + const collect = (definitions: Array>, scope: string): void => { + const definitionIds = new Map(); + definitionIdsByScope.set(scope, definitionIds); + for (const definition of definitions) { + const original = String(definition["id"]); + const remapped = derivedDefinitionId(opId, scope, original); + definitionIds.set(original, remapped); + const nested = (definition["definitions"] as { subgraphs?: Array> } | undefined)?.subgraphs ?? []; + collect(nested, `${scope}/definition:${encodeURIComponent(JSON.stringify(original))}`); + } + }; + collect(subgraphs, "root"); + const rewrite = (definitions: Array>, scope: string): void => { + const definitionIds = definitionIdsByScope.get(scope)!; + for (const definition of definitions) { + const original = String(definition["id"]); + definition["id"] = definitionIds.get(original)!; + const definitionScope = `${scope}/definition:${encodeURIComponent(JSON.stringify(original))}`; + remapGraph(definition, opId, definitionScope, definitionIdsByScope.get(definitionScope)!, true); + const nested = (definition["definitions"] as { subgraphs?: Array> } | undefined)?.subgraphs ?? []; + rewrite(nested, definitionScope); + } + }; + remapGraph(out, opId, "root", definitionIdsByScope.get("root")!, true); + rewrite(subgraphs, "root"); + return out; +} diff --git a/src/schema-version.ts b/src/schema-version.ts index c7837218..1e26e60b 100644 --- a/src/schema-version.ts +++ b/src/schema-version.ts @@ -1,9 +1,9 @@ /** * The KA-11 read gate: schema-version discipline enforced ON READ. * - * KA-11's rule is "fail closed on unreadable schema and provide a `migrate()` - * path". `migrate()` is the *provided path*; it is not the gate, because - * nothing forces a caller through it. `project()` is what actually turns a + * KA-11 requires fail-closed reads. Under the private-alpha policy, `migrate()` + * validates current layouts and refuses older ones; it is not a compatibility + * reader. Nothing forces a caller through it. `project()` actually turns a * document into the graph every consumer reads, so the gate has to live there * too — a guard a low-context caller can skip is documentation, not * enforcement (issue #38). @@ -86,10 +86,9 @@ export function readSchemaVersion(doc: Y.Doc): number | undefined { * * NOT re-exported from the entrypoint, deliberately: a caller free to choose * `expected` could pass the document's own version and switch the gate off. - * It is exported from this module so the "document is OLDER than the reader" - * arm is reachable by test today — at `SCHEMA_VERSION = 1` no older version - * exists to construct, and an arm no test can turn red is dead code, which is - * the vacuous-coverage trap this repo keeps hitting. + * It is exported from this module to test explicit reader-version comparisons. + * Older layouts can also be tested through the public reader now that + * `SCHEMA_VERSION` is greater than 1; this helper grants no compatibility path. */ export function assertSchemaVersionAgainst(doc: Y.Doc, context: string, expected: number): void { const stored = readSchemaVersion(doc); @@ -105,7 +104,7 @@ export function assertSchemaVersionAgainst(doc: Y.Doc, context: string, expected } if (stored < expected) { throw new SchemaVersionError( - `${context}: doc schema v${stored} is older than this package's v${expected} — call migrate(doc, ${stored}) first, then read (fail-closed, schema §10)`, + `${context}: doc schema v${stored} is older than this package's v${expected} — re-mint from the source workflow on the host (fail-closed, schema §10)`, ); } } @@ -116,26 +115,20 @@ export function assertSchemaVersionAgainst(doc: Y.Doc, context: string, expected * * WHY AN OLDER DOCUMENT IS REFUSED RATHER THAN MIGRATED IN PLACE. `project()` * is a pure read: it takes a `Y.Doc` and returns JSON, and it is a read ANY - * replica may call — a browser follower included. Stated precisely, because - * the looser version is false today: no follower calls it. The frontend does - * not depend on this package at all (`@comfyorg/comfy-multi-player` is absent - * from its `package.json` on every branch), and ADR-004 records that the - * follower deliberately consumes only the doc-layout helpers, since it holds - * no catalog for the document it renders. That is exactly why the rule is - * written as a rule: `follower-boundary.md` says to treat an API that permits + * replica may call — a browser follower included. This is an API rule, not a + * claim about current consumers: `follower-boundary.md` treats an API permitting * unrestricted document mutation across this boundary as a blocking violation * "even if current callers behave correctly". Migrating inside a read would * make a read WRITE the shared document. The direct rule that breaks is - * KA-6 / FC-5: followers never write the shared doc, and schema §10 puts - * migration on the host precisely so followers receive the migrated document - * via the struct stream or a new epoch. A follower that self-migrated would + * KA-6 / FC-5: followers never write the shared doc. A host must re-mint an old + * layout from its source workflow under the private-alpha policy, then deliver + * the new lineage through the host's reset protocol. A follower that self-migrated would * become an independently edited replica, which is then the FC-1 raw-struct * divergence path (KA-1 says such replicas must exchange semantic ops, and an * in-place upgrade is not one — it is neither stamped nor in the op log). So - * the version transition stays where it can be audited: the caller runs - * `migrate(doc, storedVersion)` on the host, and only then reads. Projecting - * an older layout as-is is not an option either — that IS the mis-projection - * KA-11 names. + * the version transition stays host-owned. `migrate()` does not convert old + * layouts. Projecting an older layout as-is is not an option either — that IS + * the mis-projection KA-11 names. * * Consistent with `migrate()` by construction, and by construction is meant * literally — both call {@link readSchemaVersion}, so there is no second diff --git a/src/stamps.ts b/src/stamps.ts index f3064cf1..dd43bf9b 100644 --- a/src/stamps.ts +++ b/src/stamps.ts @@ -109,9 +109,18 @@ export function writeTarget(op: WireOp): unknown[] { } return ["input", String(op.to_node), "grow", String(op.grow.name).split(".", 1)[0]]; } + if (op.path && op.path.length > 0) { + return ["input", op.path.map(String), String(op.to_node), op.to_slot]; + } return ["input", String(op.to_node), op.to_slot]; case "disconnect": return ["input", String(op.to_node), op.to_slot]; + case "insert_workflow": + // One insertion is one register: the op is idempotent by `op_id` (exact + // replay is a no-op via the applied-set gate), and two distinct + // insertions never contend because the minter allocates fresh ids + // (ADR-022). Keyed by op_id so a stamp lookup is per-insertion. + return ["insert_workflow", op.op_id]; case "clear": case "reset_doc": // Whole-document ops: no scalar register to contest, so the target is @@ -119,9 +128,11 @@ export function writeTarget(op: WireOp): unknown[] { // made the arm look like a catch-all for future kinds as well; they are // now named, and the arm below is a guard rather than a fallback. return [op.op]; + case "define_subgraph": + return ["definition", op.subgraph_id]; default: // Exhaustiveness guard (issue #21): with every `WireOp` member cased - // above — the five `Op` kinds plus the deferred `reset_doc` — `op` is + // above — the six `Op` kinds plus the deferred `reset_doc` — `op` is // `never` here, so adding a kind to EITHER union fails `tsc` at this // line until it is given a write target. // diff --git a/src/types.ts b/src/types.ts index de391431..8b9cafb5 100644 --- a/src/types.ts +++ b/src/types.ts @@ -1,7 +1,7 @@ /** * Types and constants for @comfyorg/comfy-multi-player. * - * The op vocabulary is frozen at six kinds; the normative contract is + * The op vocabulary is frozen at eight kinds; the normative contract is * comfy-cli's `docs/op-vocabulary-v1.md` and the stamp shapes minted by * `comfy_cli/workflow_ops.py` (`_new_op`), pinned by SHA at comfy-cli * `7e732242d971daf0d2d30f22f997abfacd78986e` (FC-10: never by branch — the @@ -16,10 +16,42 @@ // --------------------------------------------------------------------------- /** - * Version of the Y.Doc layout. Bump requires FE sign-off + a `migrate` path. + * Version of the Y.Doc layout. Bump requires FE sign-off and an explicit old-layout disposition. * The authoritative layout + op-semantics reference is docs/multiplayer-schema.md. */ -export const SCHEMA_VERSION = 2; +export const SCHEMA_VERSION = 3; + +/** Version of an imported destination reconstruction descriptor in `__link_state`. */ +export const LINK_STATE_DESCRIPTOR_VERSION = 1; + +/** A coherent six-field LiteGraph link tuple retained verbatim for reconstruction. */ +export type LinkTuple = [id: NodeId, fromNode: NodeId, fromSlot: number, toNode: NodeId, toSlot: number, type: unknown]; + +export type ImportedLinkDestination = + | { kind: "concrete"; to_slot: number; slot: Record } + | { kind: "promoted"; to_slot: number; name: string; slot: Record } + | { kind: "autogrow"; to_slot: number; slot: Record }; + +/** Durable baseline imported by `mint`; operation lifecycle rules are intentionally separate. */ +export interface ImportedLinkState { + version: typeof LINK_STATE_DESCRIPTOR_VERSION; + authority: "imported"; + tuple: LinkTuple; + destination: ImportedLinkDestination; +} + +export type OperationLinkDestination = + | { kind: "concrete"; to_slot: number; slot: Record } + | { kind: "promoted"; to_slot: number; name: string; slot: Record } + | { kind: "autogrow"; to_slot: number; slot: Record; request: GrowSpec }; + +/** Durable intent created by an installed connect, owned by its A18 operation stamp. */ +export interface OperationLinkState { + version: typeof LINK_STATE_DESCRIPTOR_VERSION; + authority: { kind: "operation"; stamp: StampKey }; + tuple: LinkTuple; + destination: OperationLinkDestination; +} /** Internal per-node lifetime key. It never projects into workflow JSON. */ export const NODE_INCARNATION_KEY = "__incarnation"; @@ -33,7 +65,16 @@ export const LEGACY_NODE_INCARNATION = "0"; // --------------------------------------------------------------------------- /** The implemented op kinds. `apply` rejects anything else loudly. */ -export const FROZEN_OPS = ["add_node", "connect", "disconnect", "set_widget", "delete_node", "clear"] as const; +export const FROZEN_OPS = [ + "add_node", + "connect", + "disconnect", + "set_widget", + "delete_node", + "clear", + "define_subgraph", + "insert_workflow", +] as const; /** Defined by the vocabulary but deferred (§1.6): rejected until un-deferred by amendment. */ export const DEFERRED_OPS = ["reset_doc"] as const; @@ -60,10 +101,7 @@ export const DEFERRED_OPS = ["reset_doc"] as const; * belongs. `test/batch-policy.test.ts` pins the list, the README table, and * the deliberate non-enforcement together. */ -export const BATCHABLE_OPS = ["add_node", "connect", "disconnect", "set_widget", "delete_node"] as const; - -/** Every kind the vocabulary defines — implemented ({@link Op}) plus deferred ({@link DeferredOp}). */ -export type OpKind = WireOp["op"]; +export const BATCHABLE_OPS = ["add_node", "connect", "disconnect", "set_widget", "delete_node", "define_subgraph"] as const; /** A kind `applyOps` implements. */ export type FrozenOpKind = (typeof FROZEN_OPS)[number]; @@ -130,7 +168,7 @@ export interface OpBase { } // --------------------------------------------------------------------------- -// The seven declared op kinds: six implemented (`Op`) plus the deferred +// The nine declared op kinds: eight implemented (`Op`) plus the deferred // `reset_doc` (`DeferredOp`); together `WireOp`. "Frozen" now means // implemented — `FROZEN_OPS` is pinned to `Op["op"]` exactly (issue #17). // --------------------------------------------------------------------------- @@ -147,6 +185,27 @@ export interface AddNodeOp extends OpBase { node: WorkflowNode; } +/** + * Merge a workflow template (nodes, links, `definitions.subgraphs`) into an + * existing doc in one transaction (ADR-022; agent-subgraph TDD V1.5). + * + * The applier remaps every carried id deterministically from `op_id` and the + * original id. Exact replay therefore chooses the same ids, while distinct + * insert ops cannot collide or depend on document state. `links`, `groups`, + * and `definitions` are optional and default to empty. + */ +export interface InsertWorkflowOp extends OpBase { + op: "insert_workflow"; + /** The template to merge — AUTHORITATIVE, nodes inserted verbatim after id validation. */ + workflow: { + nodes: WorkflowNode[]; + links?: unknown[]; + groups?: unknown[]; + definitions?: { subgraphs?: unknown[]; [key: string]: unknown }; + [key: string]: unknown; + }; +} + /** Autogrow slot descriptor carried by a `connect` (vocabulary §1.2 / §8.4). */ export interface GrowSpec { /** Requested slot name (`images.image0` autogrow shape, or bare `image_2` for the inputcount family). */ @@ -215,6 +274,8 @@ export interface PromotedHostWrite { /** Fields every `connect` carries, whichever way its destination slot is addressed. */ interface ConnectOpBase extends OpBase { op: "connect"; + /** Non-empty subgraph-instance path for a connect inside its definition. */ + path?: [NodeId, ...NodeId[]]; /** Link identity, minted at op-mint time (int in comfy-cli). */ link_id: NodeId; from_node: NodeId; @@ -394,6 +455,12 @@ export interface ClearOp extends OpBase { removed_nodes: NodeId[]; } +export interface DefineSubgraphOp extends OpBase { + op: "define_subgraph"; + subgraph_id: string; + subgraph_definition: SubgraphDefinition; +} + /** * Replace the entire document with a workflow snapshot. * @@ -427,7 +494,9 @@ export type Op = | DisconnectOp | SetWidgetOp | DeleteNodeOp - | ClearOp; + | ClearOp + | DefineSubgraphOp + | InsertWorkflowOp; /** * A kind the vocabulary declares but this package refuses to apply @@ -478,18 +547,20 @@ export type WireOp = Op | DeferredOp; // --------------------------------------------------------------------------- type Equals = [A] extends [B] ? ([B] extends [A] ? true : false) : false; -type Assert = T; - -/** `FROZEN_OPS` names exactly the kinds `applyOps` implements. */ -type _FrozenIsExactlyTheImplementedUnion = Assert>; -/** `DEFERRED_OPS` names exactly the kinds declared-but-not-implemented. */ -type _DeferredIsExactlyTheDeferredUnion = Assert>; -/** Every declared kind is exactly once in FROZEN_OPS or DEFERRED_OPS. */ -type _OpKindsArePartitioned = Assert>; -/** No kind is both implemented and deferred. */ -type _FrozenAndDeferredAreDisjoint = Assert>; -/** Batchable kinds are a subset of the implemented kinds. */ -type _BatchableIsSubsetOfFrozen = Assert, never>>; +type Checked = T & + (Checks[number] extends true ? unknown : never); + +/** Every kind the vocabulary defines — implemented ({@link Op}) plus deferred ({@link DeferredOp}). */ +export type OpKind = Checked< + WireOp["op"], + [ + Equals, + Equals, + Equals, + Equals, + Equals, never>, + ] +>; // --------------------------------------------------------------------------- // Widget catalog (pinned object_info projection) @@ -543,11 +614,18 @@ export interface WorkflowNode { export interface WorkflowJSON { nodes: WorkflowNode[]; links: unknown[]; + definitions?: { subgraphs?: unknown[]; [key: string]: unknown }; groups?: unknown[]; extra?: Record; [key: string]: unknown; } +export interface SubgraphDefinition extends Record { + id: string; + nodes: unknown[]; + links: unknown[]; +} + // --------------------------------------------------------------------------- // Apply result // --------------------------------------------------------------------------- diff --git a/test/applier.test.ts b/test/applier.test.ts index 86362306..ea711e58 100644 --- a/test/applier.test.ts +++ b/test/applier.test.ts @@ -32,6 +32,7 @@ import { type WorkflowJSON, type WorkflowNode, } from "../src/index.js"; +import { noOpIds, rejectedOutcome } from "./apply-result-helpers.js"; import { canonicalize, loadCatalog, loadLwwVectors, loadSession, sessionFiles } from "./helpers.js"; const catalog = loadCatalog(); @@ -47,21 +48,10 @@ function envelope(actor: string, baseVersion: number) { return { op_id: testOpId(), actor, base_version: baseVersion, stamp: [baseVersion, actor] as [number, string] }; } -function rejectedOutcome(result: ReturnType) { - return result.outcomes.find( - (outcome): outcome is Extract<(typeof result.outcomes)[number], { outcome: "rejected" }> => - outcome.outcome === "rejected" && outcome.reason.code !== "batch_aborted", - ); -} - function processedOpIds(result: ReturnType): string[] { return result.outcomes.filter((outcome) => outcome.outcome !== "rejected").map((outcome) => outcome.op_id); } -function noOpIds(result: ReturnType): string[] { - return result.outcomes.filter((outcome) => outcome.outcome === "no-op").map((outcome) => outcome.op_id); -} - describe("idempotency (byte-identical re-apply)", () => { for (const file of sessionFiles()) { it(`${file}: full stream twice + every op re-applied → byte-identical state`, () => { diff --git a/test/apply-result-helpers.test.ts b/test/apply-result-helpers.test.ts new file mode 100644 index 00000000..f83c56e2 --- /dev/null +++ b/test/apply-result-helpers.test.ts @@ -0,0 +1,50 @@ +import { describe, expect, it } from "vitest"; +import type { ApplyResult } from "../src/types.js"; +import { noOpIds, rejectedOutcome } from "./apply-result-helpers.js"; + +describe("ApplyResult helpers", () => { + it("returns the substantive rejection when an aborted outcome precedes it", () => { + const result: ApplyResult = { + outcomes: [ + { + op_id: "aborted", + outcome: "rejected", + reason: { code: "batch_aborted", message: "not processed" }, + }, + { + op_id: "failed", + outcome: "rejected", + reason: { code: "unknown_widget", message: "missing widget" }, + }, + ], + ops_seen: 0, + }; + + expect(rejectedOutcome(result)).toEqual(result.outcomes[1]); + }); + + it("returns no substantive rejection for an aborted-only result", () => { + const result: ApplyResult = { + outcomes: [ + { + op_id: "aborted", + outcome: "rejected", + reason: { code: "batch_aborted", message: "not processed" }, + }, + ], + ops_seen: 0, + }; + + expect(rejectedOutcome(result)).toBeUndefined(); + }); + + it("reports successful no-op ids without treating them as rejections", () => { + const result: ApplyResult = { + outcomes: [{ op_id: "duplicate", outcome: "no-op" }], + ops_seen: 1, + }; + + expect(rejectedOutcome(result)).toBeUndefined(); + expect(noOpIds(result)).toEqual(["duplicate"]); + }); +}); diff --git a/test/apply-result-helpers.ts b/test/apply-result-helpers.ts index b0279991..9425b584 100644 --- a/test/apply-result-helpers.ts +++ b/test/apply-result-helpers.ts @@ -1,7 +1,10 @@ import type { ApplyResult } from "../src/types.js"; export function rejectedOutcome(result: ApplyResult) { - return result.outcomes.find((outcome) => outcome.outcome === "rejected"); + return result.outcomes.find( + (outcome): outcome is Extract => + outcome.outcome === "rejected" && outcome.reason.code !== "batch_aborted", + ); } export function rejectedOutcomeWithIndex(result: ApplyResult) { diff --git a/test/batch-policy.test.ts b/test/batch-policy.test.ts index 04adcc5f..b0dc9b5a 100644 --- a/test/batch-policy.test.ts +++ b/test/batch-policy.test.ts @@ -42,8 +42,8 @@ import { loadCatalog, loadSession, sessionFiles } from "./helpers.js"; const repoRoot = dirname(dirname(fileURLToPath(import.meta.url))); const batchable = new Set(BATCHABLE_OPS); -/** The two kinds the vocabulary calls standalone-only (§1.5, §1.6). */ -const STANDALONE_ONLY = ["clear", "reset_doc"]; +/** Kinds the vocabulary or ADR-022 calls standalone-only. */ +const STANDALONE_ONLY = ["clear", "insert_workflow", "reset_doc"]; const catalog: WidgetCatalog = { types: { Sampler: { widget_order: ["steps"] } } }; const base: WorkflowJSON = { diff --git a/test/bounded-writes-liveness.test.ts b/test/bounded-writes-liveness.test.ts index 1d547a56..ac5e552f 100644 --- a/test/bounded-writes-liveness.test.ts +++ b/test/bounded-writes-liveness.test.ts @@ -154,8 +154,9 @@ describe("schema §11: the bounded-writes count grows with the op's blast radius } // The exact law, so a change to the write pattern is a visible diff rather // than a still-passing inequality: one node delete, one delete per severed - // link, one scrub per peer input slot, and one node-presence stamp. - expect(costs).toEqual(degrees.map((d) => 2 * d + 3)); + // link, one durable-descriptor retirement per explicitly named link, one + // scrub per peer input slot, and one node-presence stamp. + expect(costs).toEqual(degrees.map((d) => 3 * d + 3)); }); it("exceeds any ceiling in this gate's range at a large enough degree, so the §11 gate can fire", () => { @@ -196,7 +197,7 @@ describe("schema §11: the counter measures the applier's real writes", () => { expect(running).toEqual([3, 6, 9, 12]); }); - it("charges an autogrow connect exactly eight, including link ownership and canonicalization ledger rows", () => { + it("charges an autogrow connect exactly nine, including durable link state and canonicalization ledger rows", () => { // Reaches both `apush` sites, which `deleteHubCost` never does: wiring the // link id into the source output port's list, and appending the grown input // slot itself. Itemized precisely because this is an exact-accounting test: @@ -218,7 +219,7 @@ describe("schema §11: the counter measures the applier's real writes", () => { } as unknown as ConnectOp; _resetMutationCount(doc); expect(applyOps(doc, [op], catalog).outcomes.some((o) => o.outcome === "rejected")).toBe(false); - expect(_getMutationCount(doc)).toBe(8); + expect(_getMutationCount(doc)).toBe(9); expect(stampsMap(doc).size, "autogrow records link ownership plus stamp + request (A7/A18)").toBe(3); }); @@ -226,12 +227,13 @@ describe("schema §11: the counter measures the applier's real writes", () => { // Reaches `adel`, the applier's only array REMOVE: deleting the middle of // 1 → 2 → 3 leaves node 1's output `links` array holding a dead link id, // which the dangling scrub removes in place. One node delete, two link - // deletes, one array delete, one input-slot scrub, one presence stamp, one applied set. + // deletes, two descriptor retirements, one array delete, one input-slot + // scrub, one presence stamp, one applied set. const doc = mint(chainWorkflow(), catalog); const op: DeleteNodeOp = { op: "delete_node", ...env(), node_id: 2, removed_links: [1, 2] }; _resetMutationCount(doc); expect(applyOps(doc, [op], catalog).outcomes.some((o) => o.outcome === "rejected")).toBe(false); - expect(_getMutationCount(doc)).toBe(7); + expect(_getMutationCount(doc)).toBe(9); }); it("charges a de-duplicated replay nothing, so the ceiling is not padded by idempotent retries (KA-4)", () => { diff --git a/test/check-import-graph.test.ts b/test/check-import-graph.test.ts index 8908c011..73de09e5 100644 --- a/test/check-import-graph.test.ts +++ b/test/check-import-graph.test.ts @@ -24,7 +24,7 @@ const script = join(repoRoot, "scripts", "check-import-graph.mjs"); /** Run the gate against a fixture root, with an optional floor override. */ function runAgainst(root: string, minModules?: number) { - const env: Record = { ...process.env, IMPORT_GRAPH_ROOT: root }; + const env: NodeJS.ProcessEnv = { ...process.env, IMPORT_GRAPH_ROOT: root }; if (minModules !== undefined) env.IMPORT_GRAPH_MIN_MODULES = String(minModules); return spawnSync("node", [script], { encoding: "utf8", env }); } @@ -129,6 +129,33 @@ describe("check-import-graph gate", () => { expect(run.stderr).toContain("could not parse"); }); + it.each([ + "{}", + "null", + '{"summary":null}', + '{"summary":{"totalCruised":"2","totalDependenciesCruised":1,"violations":[]}}', + '{"summary":{"totalCruised":2,"totalDependenciesCruised":-1,"violations":[]}}', + '{"summary":{"totalCruised":2,"totalDependenciesCruised":1,"violations":[{}]}}', + ])( + "is INCONCLUSIVE (exit 2) when dependency-cruiser returns malformed report %s", + (output) => { + writeCleanFixture(); + rmSync(join(root, "node_modules", ".bin"), { recursive: true, force: true }); + mkdirSync(join(root, "node_modules", ".bin"), { recursive: true }); + const fake = join(root, "node_modules", ".bin", "depcruise"); + writeFileSync(fake, `#!/bin/sh\nprintf '%s\\n' '${output}'\n`); + chmodSync(fake, 0o755); + + const run = runAgainst(root, 2); + + expect(run.status).toBe(2); + expect(run.stderr).toContain( + "import-graph check INCONCLUSIVE: dependency-cruiser report has no valid summary", + ); + expect(run.stderr).not.toContain("TypeError"); + }, + ); + // --- one mutant per rule ------------------------------------------------- it("FAILS (exit 1) naming src-no-node-builtins on a Node builtin import (FC-3)", () => { diff --git a/test/check-stateless.test.ts b/test/check-stateless.test.ts new file mode 100644 index 00000000..25e1ecc8 --- /dev/null +++ b/test/check-stateless.test.ts @@ -0,0 +1,126 @@ +import { spawnSync } from "node:child_process"; +import { copyFileSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; + +const repoRoot = dirname(dirname(fileURLToPath(import.meta.url))); +const script = join(repoRoot, "scripts", "check-stateless.mjs"); + +describe("check-stateless gate", () => { + let root: string; + let eslintEntrypoint: string; + let vitestEntrypoint: string; + + beforeEach(() => { + root = mkdtempSync(join(tmpdir(), "stateless check with spaces-")); + mkdirSync(join(root, "src")); + mkdirSync(join(root, "test")); + mkdirSync(join(root, "scripts")); + copyFileSync(script, join(root, "scripts", "check-stateless.mjs")); + mkdirSync(join(root, ".agents", "checks"), { recursive: true }); + for (const dependency of ["eslint", "@typescript-eslint/parser", "eslint-plugin-sonarjs"]) { + mkdirSync(join(root, "node_modules", dependency), { recursive: true }); + } + mkdirSync(join(root, "node_modules", "eslint", "bin")); + mkdirSync(join(root, "node_modules", "vitest")); + eslintEntrypoint = join(root, "node_modules", "eslint", "bin", "eslint.js"); + vitestEntrypoint = join(root, "node_modules", "vitest", "vitest.mjs"); + writeFileSync(join(root, "src", "index.ts"), "export const value = 1;\n"); + writeFileSync(join(root, "test", "stateless.test.ts"), "// fixture probe\n"); + writeFileSync(join(root, ".agents", "checks", "eslint.strict.config.js"), "export default [];\n"); + writeFileSync(join(root, "package.json"), JSON.stringify({ name: "stateless-fixture" })); + writeFileSync( + eslintEntrypoint, + `console.log(${JSON.stringify(JSON.stringify([{ filePath: join(root, "src", "index.ts"), messages: [] }]))});\n`, + ); + writeFileSync(vitestEntrypoint, "process.exit(0);\n"); + }); + + afterEach(() => rmSync(root, { recursive: true, force: true })); + + function run(nodeArgs: string[] = []) { + return spawnSync(process.execPath, [...nodeArgs, join(root, "scripts", "check-stateless.mjs")], { + encoding: "utf8", + }); + } + + it("runs package-owned JS entrypoints without relying on platform-specific .bin shims", () => { + const result = run(); + expect(result.status).toBe(0); + expect(result.stdout).toContain( + "No issues found (1 source files linted; stateless-fixture stateless probe passed)", + ); + }); + + it("returns exit 2 when the vitest executable is missing", () => { + rmSync(vitestEntrypoint); + const result = run(); + expect(result.status).toBe(2); + expect(result.stderr).toContain("INCONCLUSIVE — vitest is not installed; run npm ci"); + }); + + it("returns exit 2 when the eslint executable is missing", () => { + rmSync(eslintEntrypoint); + const result = run(); + expect(result.status).toBe(2); + expect(result.stderr).toContain("INCONCLUSIVE — eslint is not installed; run npm ci"); + }); + + it("returns exit 2 when the stateless probe is missing", () => { + rmSync(join(root, "test", "stateless.test.ts")); + const result = run(); + expect(result.status).toBe(2); + expect(result.stderr).toContain("INCONCLUSIVE — test/stateless.test.ts is missing"); + }); + + it("returns exit 2 when spawning the stateless probe fails", () => { + const preload = ` + import childProcess from "node:child_process"; + import { syncBuiltinESMExports } from "node:module"; + const spawn = childProcess.spawnSync; + childProcess.spawnSync = (command, args, options) => spawn( + args[0] === ${JSON.stringify(vitestEntrypoint)} + ? ${JSON.stringify(join(root, "missing-node-executable"))} : command, + args, options, + ); + syncBuiltinESMExports(); + `; + const result = run(["--import", `data:text/javascript,${encodeURIComponent(preload)}`]); + expect(result.status).toBe(2); + expect(result.stderr).toContain("INCONCLUSIVE — could not run the stateless probe: spawnSync"); + expect(result.stdout).not.toContain("No issues found"); + }); + + it("returns exit 2 when the stateless probe cannot execute", () => { + writeFileSync(vitestEntrypoint, "process.exit(2);\n"); + const result = run(); + expect(result.status).toBe(2); + expect(result.stderr).toContain("INCONCLUSIVE — stateless probe failed to execute"); + }); + + it("returns exit 2 when the stateless probe is terminated by a signal", () => { + writeFileSync(vitestEntrypoint, 'process.kill(process.pid, "SIGTERM");\n'); + const result = run(); + expect(result.status).toBe(2); + expect(result.stderr).toContain("INCONCLUSIVE — stateless probe terminated by signal SIGTERM"); + }); + + it("preserves a genuine lint finding as exit 1", () => { + writeFileSync( + eslintEntrypoint, + `console.log(${JSON.stringify(JSON.stringify([{ filePath: join(root, "src", "index.ts"), messages: [{ ruleId: "no-warning-comments", message: "finding" }] }]))});\n`, + ); + const result = run(); + expect(result.status).toBe(1); + expect(result.stderr).toContain('"message": "finding"'); + }); + + it("preserves a failing stateless probe as exit 1", () => { + writeFileSync(vitestEntrypoint, "process.exit(1);\n"); + const result = run(); + expect(result.status).toBe(1); + expect(result.stdout).not.toContain("No issues found"); + }); +}); diff --git a/test/connect-link-state.test.ts b/test/connect-link-state.test.ts new file mode 100644 index 00000000..f219477f --- /dev/null +++ b/test/connect-link-state.test.ts @@ -0,0 +1,143 @@ +import * as Y from "yjs"; +import { describe, expect, it } from "vitest"; +import { + applyOps, + mint, + readLinkState, + type ConnectOp, + type DeleteNodeOp, + type GrowSpec, + type WidgetCatalog, + type WorkflowJSON, + type WorkflowNode, +} from "../src/index.js"; + +const catalog: WidgetCatalog = { types: { Source: { widget_order: [] }, Destination: { widget_order: [] } } }; +const stamp = (n: number): [number, string] => [n, "agent:test"]; +const id = (n: number) => n.toString(16).padStart(32, "0"); + +function node(idValue: number, type: string, inputs: Record[], outputs: Record[]): WorkflowNode { + return { id: idValue, type, pos: [0, 0], size: [100, 100], flags: {}, order: 0, mode: 0, inputs, outputs, properties: {}, widgets_values: [] }; +} + +function workflow(promoted = false): WorkflowJSON { + return { + last_node_id: 2, last_link_id: 0, + nodes: [node(1, "Source", [], [{ name: "out", type: "IMAGE", links: [] }]), node(2, promoted ? "def.with.dots" : "Destination", [{ name: "fixed", type: "IMAGE", link: null }], [])], + links: [], groups: [], config: {}, extra: {}, version: 0.4, + ...(promoted ? { definitions: { subgraphs: [{ id: "def.with.dots", inputs: [{ name: "nested.dynamic.image", type: "IMAGE" }], outputs: [], nodes: [], links: [] }] } } : {}), + } as WorkflowJSON; +} + +function connect(linkId: string | number, n: number, grow?: GrowSpec): ConnectOp { + const base = { op: "connect" as const, op_id: id(n), actor: "agent:test", base_version: n, stamp: stamp(n), link_id: linkId, from_node: 1, from_slot: 0, to_node: 2, link_type: "IMAGE" }; + return grow == null ? { ...base, to_slot: 0 } : { ...base, grow }; +} + +function expected(linkId: string | number, n: number, destination: Record) { + return { version: 1, authority: { kind: "operation", stamp: [n, "agent:test", id(n)] }, tuple: [linkId, 1, 0, 2, destination.to_slot, "IMAGE"], destination }; +} + +describe("connect generation writes first-class durable link state", () => { + it.each([undefined, { name: "", type: "" }])("reads records produced by accepted fallback envelopes and grow strings: %j", (grow) => { + const doc = mint(workflow(), catalog); + const op = connect(31, 8, grow); + delete (op as Partial).stamp; + op.actor = ""; + op.op_id = "cli:connect"; + expect(applyOps(doc, [op]).outcomes[0]?.outcome).toBe("applied"); + expect(readLinkState(doc)["31"]).toEqual({ + version: 1, + authority: { kind: "operation", stamp: [8, "", "cli:connect"] }, + tuple: [31, 1, 0, 2, grow ? 1 : 0, "IMAGE"], + destination: grow + ? { kind: "autogrow", to_slot: 1, slot: { name: "", type: "", link: 31, grow_id: 31 }, request: { name: "", type: "" } } + : { kind: "concrete", to_slot: 0, slot: { name: "fixed", type: "IMAGE", link: 31 } }, + }); + }); + + it("records the actual concrete tuple and replaces a normalized identity in both orders", () => { + const low = connect(7, 1); + const high = connect("7", 2); + const want = expected("7", 2, { kind: "concrete", to_slot: 0, slot: { name: "fixed", type: "IMAGE", link: "7" } }); + for (const order of [[low, high], [high, low]]) { + const doc = mint(workflow(), catalog); + applyOps(doc, order); + expect(readLinkState(doc)).toEqual({ "7": want }); + } + }); + + it("preserves the full promoted name and classifies autogrow with its complete request", () => { + const promoted = mint(workflow(true), catalog); + const promotedGrow: GrowSpec = { name: "nested.dynamic.image", type: "IMAGE", promoted: true }; + applyOps(promoted, [connect(8, 3, promotedGrow)]); + expect(readLinkState(promoted)["8"]).toEqual(expected(8, 3, { + kind: "promoted", to_slot: 1, name: "nested.dynamic.image", + slot: { name: "nested.dynamic.image", type: "IMAGE", link: 8, grow_id: 8 }, + })); + + const autogrow = mint(workflow(), catalog); + const request: GrowSpec = { name: "images.image0", type: "IMAGE", widget: "upload" }; + applyOps(autogrow, [connect(9, 4, request)]); + expect(readLinkState(autogrow)["9"]).toEqual(expected(9, 4, { + kind: "autogrow", to_slot: 1, + slot: { name: "images.image0", type: "IMAGE", link: 9, grow_id: 9, widget: { name: "upload" } }, + request, + })); + }); + + it("keeps distinct autogrow descriptors synchronized with canonical slots in both orders", () => { + const request: GrowSpec = { name: "images.image0", type: "IMAGE", widget: "upload" }; + const first = connect(21, 1, request); + const second = connect(22, 2, request); + for (const order of [[first, second], [second, first]]) { + const doc = mint(workflow(), catalog); + applyOps(doc, order); + expect(readLinkState(doc)).toEqual({ + "21": expected(21, 1, { + kind: "autogrow", to_slot: 1, + slot: { name: "images.image0", type: "IMAGE", link: 21, grow_id: 21, widget: { name: "upload" } }, + request, + }), + "22": expected(22, 2, { + kind: "autogrow", to_slot: 2, + slot: { name: "images.image1", type: "IMAGE", link: 22, grow_id: 22, widget: { name: "upload" } }, + request, + }), + }); + } + }); + + it("is byte-identical on exact replay and rejection", () => { + const doc = mint(workflow(), catalog); + const op = connect(10, 5); + applyOps(doc, [op]); + const installed = Y.encodeStateAsUpdate(doc); + applyOps(doc, [op]); + expect(Y.encodeStateAsUpdate(doc)).toEqual(installed); + + const rejected = { ...connect(11, 6), to_slot: 99 } as ConnectOp; + const beforeReject = Y.encodeStateAsUpdate(doc); + expect(applyOps(doc, [rejected]).outcomes[0]?.outcome).toBe("rejected"); + expect(Y.encodeStateAsUpdate(doc)).toEqual(beforeReject); + expect(readLinkState(doc)["11"]).toBeUndefined(); + }); + + it("removes the prior generation when identity is claimed before a consumed source-delete no-op", () => { + const doc = mint(workflow(), catalog); + applyOps(doc, [connect(12, 1)]); + const deletion: DeleteNodeOp = { op: "delete_node", op_id: id(2), actor: "agent:test", base_version: 2, stamp: stamp(2), node_id: 1, removed_links: [] }; + applyOps(doc, [deletion]); + const result = applyOps(doc, [connect("12", 3)]); + expect(result.outcomes[0]?.outcome).toBe("no-op"); + expect(readLinkState(doc)["12"]).toBeUndefined(); + }); + + it("does not retain an older descriptor after identity advances but the destination gate loses", () => { + const doc = mint(workflow(), catalog); + applyOps(doc, [connect(13, 1), connect(14, 3)]); + const result = applyOps(doc, [connect("13", 2)]); + expect(result.outcomes[0]?.outcome).toBe("lww-dropped"); + expect(readLinkState(doc)["13"]).toBeUndefined(); + }); +}); diff --git a/test/convergence.test.ts b/test/convergence.test.ts index 1a17d4ee..557bda89 100644 --- a/test/convergence.test.ts +++ b/test/convergence.test.ts @@ -53,6 +53,8 @@ function touchedNodes(op: WireOp): string[] { return [String(op.from_node), String(op.to_node)]; case "disconnect": return [String(op.to_node)]; + case "insert_workflow": + return op.workflow.nodes.map((node) => String(node.id)); case "clear": case "delete_node": case "reset_doc": @@ -60,6 +62,8 @@ function touchedNodes(op: WireOp): string[] { // breakers and never calls this helper for them; listed explicitly so // the guard below is a guard and not a catch-all (#21). return []; + case "define_subgraph": + return [String(op.subgraph_id)]; default: return assertNever(op, "convergence.touchedNodes"); } @@ -96,6 +100,22 @@ function reorderableWindows(ops: Op[]): Op[][] { } describe("two-doc convergence through the single-applier discipline", () => { + it("keeps define_subgraph and an interior edit to that definition in separate windows", () => { + const definitionId = "12345678-1234-4123-8123-123456789abc"; + const define = { + op: "define_subgraph", op_id: "10000000000000000000000000000000", actor: "agent:test", + base_version: 1, stamp: [1, "agent:test"], subgraph_id: definitionId, + subgraph_definition: { id: definitionId, name: "One", inputs: [], outputs: [], nodes: [], links: [] }, + } as Op; + const edit = { + op: "set_widget", op_id: "20000000000000000000000000000000", actor: "agent:test", + base_version: 2, stamp: [2, "agent:test"], node_id: 10, path: [definitionId, "10"], + widget: "value", inner_widget: "value", value: 2, + } as Op; + + expect(reorderableWindows([define, edit])).toEqual([[define], [edit]]); + }); + for (const file of sessionFiles()) { const { header, ops } = loadSession(file); const windows = reorderableWindows(ops); diff --git a/test/define-subgraph.test.ts b/test/define-subgraph.test.ts new file mode 100644 index 00000000..cc6c2737 --- /dev/null +++ b/test/define-subgraph.test.ts @@ -0,0 +1,1007 @@ +import * as Y from "yjs" +import { createHash } from "node:crypto" +import { describe, expect, it } from "vitest" + +import { + applyOps, + mint, + project, + type DefineSubgraphOp, + type Op, + type SubgraphDefinition, + type WidgetCatalog, + type WorkflowJSON, +} from "../src/index.js" +import { appliedMap, definitionsMap, metaMap, stampsMap } from "../src/doc.js" +import { stampKey } from "../src/stamps.js" + +const catalog: WidgetCatalog = { + types: { + Inner: { widget_order: ["value"] }, + Other: { widget_order: [] }, + }, +} + +let sequence = 0 +const envelope = () => { + const op_id = String(++sequence).padStart(32, "0") + return { op_id, actor: "agent:test", base_version: sequence, stamp: [sequence, "agent:test"] as [number, string] } +} + +const subgraphId = "12345678-1234-4123-8123-123456789abc" +const catalogClassId = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" +const collisionCatalog: WidgetCatalog = { + types: { ...catalog.types, [catalogClassId]: { widget_order: [] } }, +} +const definition = (id = subgraphId, value = 1) => ({ + id, + name: "One", + inputs: [], + outputs: [], + nodes: [{ id: 10, type: "Inner", inputs: [], outputs: [], widgets_values: [value] }], + links: [], +}) + +const define = (id = subgraphId, value = 1): DefineSubgraphOp => ({ + op: "define_subgraph", + ...envelope(), + subgraph_id: id, + subgraph_definition: definition(id, value), +}) + +const empty = () => mint({ nodes: [], links: [] } as unknown as WorkflowJSON, catalog) +const rejectionCode = (doc: Y.Doc, op: Op) => + applyOps(doc, [op], catalog).outcomes.find((outcome) => outcome.outcome === "rejected")?.reason.code + +const winningReplacement = (incumbent: SubgraphDefinition, replacement: SubgraphDefinition) => { + const canonical = (value: unknown): string => JSON.stringify(value, (_key, child: unknown) => + child && typeof child === "object" && !Array.isArray(child) + ? Object.fromEntries(Object.entries(child).sort(([a], [b]) => a < b ? -1 : a > b ? 1 : 0)) + : child) + const digest = (value: unknown) => createHash("sha256").update(canonical(value)).digest("hex") + const incumbentDigest = digest(incumbent) + for (let nonce = 0; nonce < 1000; nonce++) { + const candidate: SubgraphDefinition = { ...replacement, name: `replacement-${nonce}` } + if (digest(candidate) > incumbentDigest) return candidate + } + throw new Error("independent test oracle could not construct a winning replacement") +} + +describe("define_subgraph schema", () => { + const forbidden = [ + ["add_node with subgraph_definition", { op: "add_node", node_id: 1, class_type: "Inner", pos: [0, 0], node: { id: 1, type: "Inner" }, subgraph_definition: definition() }], + ["add_node with definitions", { op: "add_node", node_id: 1, class_type: "Inner", pos: [0, 0], node: { id: 1, type: "Inner" }, definitions: { subgraphs: [] } }], + ["node-property op with definition payload", { op: "set_widget", node_id: 1, widget: "value", value: 2, subgraph_definition: definition() }], + ["widget op with definitions payload", { op: "set_widget", node_id: 1, widget: "value", value: 2, definitions: {} }], + ["connect with definition payload", { op: "connect", link_id: 1, from_node: 1, from_slot: 0, to_node: 2, to_slot: 0, link_type: "X", subgraph_definition: definition() }], + ["disconnect with definition payload", { op: "disconnect", link_id: 1, subgraph_definition: definition() }], + ["layout op with definition payload", { op: "add_node", node_id: 1, class_type: "Inner", pos: [0, 0], node: { id: 1, type: "Inner" }, layout: {}, subgraph_definition: definition() }], + ["remove-node with definition payload", { op: "delete_node", node_id: 1, removed_links: [], subgraph_definition: definition() }], + ["subgraph_id cannot substitute for a node target", { op: "set_widget", subgraph_id: subgraphId, widget: "value", value: 2 }], + ] as const + + it.each(forbidden)("rejects %s", (_name, fields) => { + const op = { ...fields, ...envelope() } as unknown as Op + const doc = empty() + const before = Y.encodeStateAsUpdate(doc) + expect(rejectionCode(doc, op)).toBeDefined() + expect(Y.encodeStateAsUpdate(doc)).toEqual(before) + expect(appliedMap(doc).has(op.op_id)).toBe(false) + }) + + it("aborts the batch suffix after a rejected definition operation", () => { + const doc = empty() + const rejected = { ...define(), subgraph_id: "not-a-uuid" } + const suffix = { + op: "add_node", ...envelope(), node_id: 99, class_type: "Other", pos: [0, 0], + node: { id: 99, type: "Other", inputs: [], outputs: [] }, + } as Op + + expect(applyOps(doc, [rejected, suffix], catalog).outcomes).toEqual([ + expect.objectContaining({ op_id: rejected.op_id, outcome: "rejected" }), + expect.objectContaining({ op_id: suffix.op_id, outcome: "rejected", reason: expect.objectContaining({ code: "batch_aborted" }) }), + ]) + expect(appliedMap(doc).has(rejected.op_id)).toBe(false) + expect(appliedMap(doc).has(suffix.op_id)).toBe(false) + expect(project(doc, catalog).nodes).toEqual([]) + }) + + it.each([ + ["top-level", definition(catalogClassId)], + ["nested", { + ...definition(), + definitions: { subgraphs: [definition(catalogClassId)] }, + }], + ])("atomically rejects a %s definition id that shadows a UUID-shaped catalog class", (_name, collidingDefinition) => { + const doc = empty() + const before = Y.encodeStateAsUpdate(doc) + const op = { ...define(), subgraph_definition: collidingDefinition, subgraph_id: String(collidingDefinition.id) } + + expect(applyOps(doc, [op], collisionCatalog).outcomes[0]?.outcome).toBe("rejected") + expect(Y.encodeStateAsUpdate(doc)).toEqual(before) + expect(appliedMap(doc).has(op.op_id)).toBe(false) + }) + + it("accepts a distinct UUID definition while a catalog node retains its class meaning", () => { + const doc = empty() + const addCatalogNode = { + op: "add_node", ...envelope(), node_id: 1, class_type: catalogClassId, pos: [0, 0], + node: { id: 1, type: catalogClassId, inputs: [], outputs: [] }, + } as Op + + expect(applyOps(doc, [addCatalogNode, define()], collisionCatalog).outcomes.map(({ outcome }) => outcome)).toEqual(["applied", "applied"]) + expect(project(doc, collisionCatalog).nodes[0]?.type).toBe(catalogClassId) + expect(definitionsMap(doc).has(subgraphId)).toBe(true) + expect(definitionsMap(doc).has(catalogClassId)).toBe(false) + }) +}) + +describe("define_subgraph application", () => { + it("digests accepted private metadata using the same public definition representation as fallback reads", () => { + const doc = empty() + const withPrivateMetadata = { ...definition(), __source: "private" } + const op = { ...define(), subgraph_definition: withPrivateMetadata } + + expect(applyOps(doc, [op], catalog).outcomes[0]?.outcome).toBe("applied") + const expected = createHash("sha256").update(JSON.stringify(definition(), (_key, child: unknown) => + child && typeof child === "object" && !Array.isArray(child) + ? Object.fromEntries(Object.entries(child).sort(([a], [b]) => a < b ? -1 : a > b ? 1 : 0)) + : child)).digest("hex") + expect((metaMap(doc).get("__definition_digests") as Record)[subgraphId]).toBe(expected) + }) + + it("matches the independently derived fixed definition digest and winner vector", () => { + const seed = Y.encodeStateAsUpdate(empty()) + const winner = define(subgraphId, 1) + const loser = define(subgraphId, 2) + // Derived with Python 3 json.dumps(sort_keys=True,separators=(',',':')) + // and hashlib.sha256, not the production canonicalizer or digest helper. + // value=2 hashes to 1872feea7ddb573ff68e47b7be0c5b369863cfa2262f6d23d18ae15ecadb9e62. + for (const ops of [[winner, loser], [loser, winner]]) { + const doc = new Y.Doc() + Y.applyUpdate(doc, seed) + applyOps(doc, ops, catalog) + expect((project(doc, catalog).definitions as { subgraphs: unknown[] }).subgraphs).toEqual([definition()]) + expect((metaMap(doc).get("__definition_digests") as Record)[subgraphId]).toBe( + "3a0b0a3e10f67853f62d5b56ed2d505449cccd161cca63bfcbd3187d32523eb3", + ) + } + }) + it("projects a newly defined subgraph", () => { + const doc = empty() + expect(applyOps(doc, [define()], catalog).outcomes[0]?.outcome).toBe("applied") + expect((project(doc, catalog).definitions as { subgraphs: unknown[] }).subgraphs).toEqual([definition()]) + }) + + it("is byte-idempotent on exact replay", () => { + const doc = empty() + const op = define() + applyOps(doc, [op], catalog) + const before = Y.encodeStateAsUpdate(doc) + expect(applyOps(doc, [op], catalog).outcomes[0]?.outcome).toBe("no-op") + expect(Y.encodeStateAsUpdate(doc)).toEqual(before) + }) + + it("does not revert an intervening interior edit when the definition is replayed with a new envelope", () => { + const doc = empty() + const op = define() + applyOps(doc, [op], catalog) + applyOps(doc, [ + { op: "set_widget", ...envelope(), node_id: 10, path: [subgraphId, "10"], inner_widget: "value", widget: "value", value: 2 }, + ], catalog) + const before = Y.encodeStateAsUpdate(doc) + + expect(applyOps(doc, [{ ...op, ...envelope() }], catalog).outcomes[0]?.outcome).toBe("no-op") + expect(Y.encodeStateAsUpdate(doc)).not.toEqual(before) // the new op_id is consumed + const projected = (project(doc, catalog).definitions as { subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> }).subgraphs[0]! + expect(projected.nodes[0]!.widgets_values).toEqual([2]) + }) + + it("replays an identical definition imported without a private digest and continues the batch", () => { + const imported = { + nodes: [], + links: [], + definitions: { subgraphs: [definition()] }, + } as unknown as WorkflowJSON + const doc = mint(imported, catalog) + const storedDefinition = definitionsMap(doc).get(subgraphId)! + const storedWidgets = (storedDefinition.get("nodes") as Y.Map>).get("10")!.get("widgets") + const before = project(doc, catalog) + const replay = define() + const suffix = { + op: "add_node", ...envelope(), node_id: 99, class_type: "Other", pos: [0, 0], + node: { id: 99, type: "Other", inputs: [], outputs: [] }, + } as Op + + expect(storedDefinition.get("__definition_digest")).toBeUndefined() + expect(metaMap(doc).get("__definition_digests")).toBeUndefined() + expect(applyOps(doc, [replay, suffix], catalog).outcomes).toEqual([ + { op_id: replay.op_id, outcome: "no-op" }, + { op_id: suffix.op_id, outcome: "applied" }, + ]) + expect(definitionsMap(doc).get(subgraphId)).toBe(storedDefinition) + expect((storedDefinition.get("nodes") as Y.Map>).get("10")!.get("widgets")).toBe(storedWidgets) + expect((project(doc, catalog).definitions as unknown)).toEqual(before.definitions) + expect(project(doc, catalog).nodes.map((node) => node.id)).toContain(99) + }) + + it("replays a projected-output definition imported without a private digest as an identity-preserving no-op", () => { + const importedDefinition = { + id: subgraphId, + name: "One2", + inputs: [], + outputs: [], + nodes: [{ + id: 10, + type: "Inner", + inputs: [], + outputs: [{ name: "out", type: "X", links: [9, 3] }], + widgets_values: [1], + }], + links: [], + } + const doc = mint({ + nodes: [], + links: [], + definitions: { subgraphs: [importedDefinition] }, + } as unknown as WorkflowJSON, catalog) + const storedDefinition = definitionsMap(doc).get(subgraphId)! + const storedNode = (storedDefinition.get("nodes") as Y.Map>).get("10")! + const replay = { + ...define(), + subgraph_definition: importedDefinition, + } as DefineSubgraphOp + + expect(applyOps(doc, [replay], catalog).outcomes[0]?.outcome).toBe("no-op") + expect(definitionsMap(doc).get(subgraphId)).toBe(storedDefinition) + expect((storedDefinition.get("nodes") as Y.Map>).get("10")).toBe(storedNode) + }) + + it("treats instance-ID and definition-ID widget paths as one LWW register", () => { + const seeded = mint({ + nodes: [{ id: 1, type: subgraphId, inputs: [], outputs: [] }], + links: [], + definitions: { subgraphs: [definition()] }, + } as unknown as WorkflowJSON, catalog) + const snapshot = Y.encodeStateAsUpdate(seeded) + const lower = { + op: "set_widget", op_id: "10000000000000000000000000000000", actor: "agent:low", + base_version: 10, stamp: [10, "agent:low"], node_id: 10, + path: [subgraphId, "10"], inner_widget: "value", widget: "value", value: "lower-definition-path", + } as Op + const higher = { + op: "set_widget", op_id: "20000000000000000000000000000000", actor: "agent:high", + base_version: 20, stamp: [20, "agent:high"], node_id: 10, + path: ["1", "10"], inner_widget: "value", widget: "value", value: "higher-instance-path", + } as Op + + const results = [[lower, higher], [higher, lower]].map((ops) => { + const doc = new Y.Doc() + Y.applyUpdate(doc, snapshot) + const outcomes = ops.flatMap((op) => applyOps(doc, [op], catalog).outcomes) + expect(outcomes).toHaveLength(2) + expect(outcomes.every(({ outcome }) => outcome === "applied" || outcome === "lww-dropped")).toBe(true) + const projected = (project(doc, catalog).definitions as { subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> }).subgraphs[0]! + const beforeReplay = Y.encodeStateAsUpdate(doc) + expect(ops.map((op) => applyOps(doc, [op], catalog).outcomes[0]?.outcome)).toEqual(["no-op", "no-op"]) + expect(Y.encodeStateAsUpdate(doc)).toEqual(beforeReplay) + return { value: projected.nodes[0]!.widgets_values, stamps: [...stampsMap(doc).values()] } + }) + + expect(results.map(({ value }) => value)).toEqual([ + ["higher-instance-path"], + ["higher-instance-path"], + ]) + expect(results.map(({ stamps }) => stamps)).toEqual([[stampKey(higher)], [stampKey(higher)]]) + }) + + it("retains an instance-routed canonical edit across concurrent deletion in both legal orders", () => { + const instanceId = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb" + const seeded = mint({ + nodes: [{ id: instanceId, type: subgraphId, inputs: [], outputs: [] }], + links: [], + definitions: { subgraphs: [definition()] }, + } as unknown as WorkflowJSON, catalog) + const snapshot = Y.encodeStateAsUpdate(seeded) + const concurrent = (op: Partial, actor: string, opId: string) => ({ + ...op, op_id: opId, actor, base_version: 3, stamp: [3, actor], + }) as Op + const remove = concurrent( + { op: "delete_node", node_id: instanceId, removed_links: [] }, + "human:delete", + "d0000000000000000000000000000000", + ) + const edit = concurrent( + { op: "set_widget", node_id: 10, path: [instanceId, "10"], inner_widget: "value", widget: "value", value: 7 }, + "agent:edit", + "e0000000000000000000000000000000", + ) + const retainedRouteKey = JSON.stringify(["interior_route", instanceId, "0"]) + + const results = [[remove, edit], [edit, remove]].map((ops) => { + let doc = new Y.Doc() + Y.applyUpdate(doc, snapshot) + const outcomes = [applyOps(doc, [ops[0]!], catalog).outcomes[0]?.outcome] + if (ops[0] === remove) { + const replayed = new Y.Doc() + Y.applyUpdate(replayed, Y.encodeStateAsUpdate(doc)) + doc = replayed + } + outcomes.push(applyOps(doc, [ops[1]!], catalog).outcomes[0]?.outcome) + const projection = project(doc, catalog) + const retainedValue = (projection.definitions as { + subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> + }).subgraphs[0]!.nodes[0]!.widgets_values + return { outcomes, projection, retainedValue, retainedRoute: stampsMap(doc).get(retainedRouteKey) } + }) + + expect(results.map(({ outcomes }) => outcomes)).toEqual([["applied", "applied"], ["applied", "applied"]]) + expect(results[0]!.projection).toEqual(results[1]!.projection) + expect(results.map(({ retainedValue }) => retainedValue)).toEqual([[7], [7]]) + expect(results.every(({ projection }) => projection.nodes.length === 0)).toBe(true) + expect(results.map(({ retainedRoute }) => retainedRoute)).toEqual([subgraphId, subgraphId]) + }) + + it("shares the canonical register between a retained instance route and a direct definition path", () => { + const instanceId = "cccccccc-cccc-4ccc-8ccc-cccccccccccc" + const doc = mint({ + nodes: [{ id: instanceId, type: subgraphId, inputs: [], outputs: [] }], + links: [], definitions: { subgraphs: [definition()] }, + } as unknown as WorkflowJSON, catalog) + const remove = { op: "delete_node", ...envelope(), node_id: instanceId, removed_links: [] } as Op + expect(applyOps(doc, [remove], catalog).outcomes[0]?.outcome).toBe("applied") + const routed = { + op: "set_widget", op_id: "61000000000000000000000000000000", actor: "agent:route", + base_version: 61, stamp: [61, "agent:route"], node_id: 10, + path: [instanceId, "10"], inner_widget: "value", widget: "value", value: "route", + } as Op + const direct = { + ...routed, op_id: "62000000000000000000000000000000", actor: "agent:direct", + base_version: 62, stamp: [62, "agent:direct"], path: [subgraphId, "10"], value: "direct", + } as Op + + expect(applyOps(doc, [routed, direct], catalog).outcomes.map(({ outcome }) => outcome)).toEqual(["applied", "applied"]) + const retained = (project(doc, catalog).definitions as { + subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> + }).subgraphs[0]!.nodes[0]!.widgets_values + expect(retained).toEqual(["direct"]) + }) + + it("does not let an unrelated deletion or a delete/re-add retarget a retained route", () => { + const instanceId = "dddddddd-dddd-4ddd-8ddd-dddddddddddd" + const unrelatedId = "eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee" + const doc = mint({ + nodes: [ + { id: instanceId, type: subgraphId, inputs: [], outputs: [] }, + { id: unrelatedId, type: "Other", inputs: [], outputs: [] }, + ], + links: [], definitions: { subgraphs: [definition()] }, + } as unknown as WorkflowJSON, catalog) + expect(applyOps(doc, [{ op: "delete_node", ...envelope(), node_id: unrelatedId, removed_links: [] } as Op], catalog).outcomes[0]?.outcome).toBe("applied") + const edit = { + op: "set_widget", ...envelope(), node_id: 10, path: [instanceId, "10"], + inner_widget: "value", widget: "value", value: 8, + } as Op + expect(applyOps(doc, [edit], catalog).outcomes[0]?.outcome).toBe("applied") + + expect(applyOps(doc, [{ op: "delete_node", ...envelope(), node_id: instanceId, removed_links: [] } as Op], catalog).outcomes[0]?.outcome).toBe("applied") + const readd = { + op: "add_node", ...envelope(), node_id: instanceId, node_incarnation: "replacement", + class_type: "Other", pos: [0, 0], node: { id: instanceId, type: "Other", inputs: [], outputs: [] }, + } as Op + expect(applyOps(doc, [readd], catalog).outcomes[0]?.outcome).toBe("applied") + const stale = { ...edit, ...envelope(), value: 9 } as Op + expect(applyOps(doc, [stale], catalog).outcomes[0]?.outcome).toBe("rejected") + const retained = (project(doc, catalog).definitions as { + subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> + }).subgraphs[0]!.nodes[0]!.widgets_values + expect(retained).toEqual([8]) + }) + + it("uses the final owning definition and node as nested alias identity without conflating equal leaf IDs", () => { + const innerId = "22345678-1234-4123-8123-123456789abc" + const nestedWorkflow = { + nodes: [{ id: 1, type: subgraphId, inputs: [], outputs: [] }], + links: [], + definitions: { subgraphs: [ + { + id: subgraphId, name: "Outer", inputs: [], outputs: [], links: [], + nodes: [ + { id: 10, type: "Inner", inputs: [], outputs: [], widgets_values: ["outer-leaf"] }, + { id: 20, type: innerId, inputs: [], outputs: [] }, + ], + }, + { + id: innerId, name: "Nested", inputs: [], outputs: [], links: [], + nodes: [{ id: 10, type: "Inner", inputs: [], outputs: [], widgets_values: ["nested-leaf"] }], + }, + ] }, + } as unknown as WorkflowJSON + const snapshot = Y.encodeStateAsUpdate(mint(nestedWorkflow, catalog)) + const lower = { + op: "set_widget", op_id: "30000000000000000000000000000000", actor: "agent:low", + base_version: 30, stamp: [30, "agent:low"], node_id: 10, + path: [subgraphId, "20", "10"], inner_widget: "value", widget: "value", value: "lower-outer-history", + } as Op + const higher = { + op: "set_widget", op_id: "40000000000000000000000000000000", actor: "agent:high", + base_version: 40, stamp: [40, "agent:high"], node_id: 10, + path: [innerId, "10"], inner_widget: "value", widget: "value", value: "higher-final-owner", + } as Op + + for (const ops of [[lower, higher], [higher, lower]]) { + const doc = new Y.Doc() + Y.applyUpdate(doc, snapshot) + const outcomes = ops.map((op) => applyOps(doc, [op], catalog).outcomes[0]?.outcome) + expect(outcomes.every((outcome) => outcome === "applied" || outcome === "lww-dropped")).toBe(true) + const definitions = (project(doc, catalog).definitions as { + subgraphs: Array<{ id: string; nodes: Array<{ id: unknown; widgets_values?: unknown[] }> }> + }).subgraphs + const inner = definitions.find(({ id }) => id === innerId)! + const outer = definitions.find(({ id }) => id === subgraphId)! + expect(inner.nodes.find(({ id }) => id === 10)!.widgets_values).toEqual(["higher-final-owner"]) + expect(outer.nodes.find(({ id }) => id === 10)!.widgets_values).toEqual(["outer-leaf"]) + expect([...stampsMap(doc).values()]).toEqual([stampKey(higher)]) + } + + const isolated = new Y.Doc() + Y.applyUpdate(isolated, snapshot) + const outerLeaf = { + ...lower, op_id: "50000000000000000000000000000000", base_version: 50, + stamp: [50, "agent:low"], path: [subgraphId, "10"], value: "changed-outer-leaf", + } as Op + expect(applyOps(isolated, [higher, outerLeaf], catalog).outcomes.map(({ outcome }) => outcome)).toEqual(["applied", "applied"]) + expect(stampsMap(isolated).size).toBe(2) + }) + + it("converges when a definition replay and an interior edit arrive in either legal order", () => { + const seed = empty() + const original = define() + applyOps(seed, [original], catalog) + const snapshot = Y.encodeStateAsUpdate(seed) + const replay = { ...original, ...envelope() } + const edit = { + op: "set_widget", ...envelope(), node_id: 10, path: [subgraphId, "10"], inner_widget: "value", widget: "value", value: 7, + } as Op + + const projections = [[replay, edit], [edit, replay]].map((ops) => { + const doc = new Y.Doc() + Y.applyUpdate(doc, snapshot) + for (const op of ops) applyOps(doc, [op], catalog) + return project(doc, catalog) + }) + + expect(projections[0]).toEqual(projections[1]) + expect((projections[0]!.definitions as { subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> }).subgraphs[0]!.nodes[0]!.widgets_values).toEqual([7]) + }) + + it("consumes a losing definition conflict without changing the winner", () => { + const doc = empty() + applyOps(doc, [define()], catalog) + expect(applyOps(doc, [define(subgraphId, 2)], catalog).outcomes[0]?.outcome).toBe("no-op") + expect((project(doc, catalog).definitions as { subgraphs: unknown[] }).subgraphs).toEqual([definition()]) + }) + + it("projects the same deterministic winner for conflicting definitions in either delivery order", () => { + const a = define(subgraphId, 1) + const b = define(subgraphId, 2) + const projections = [[a, b], [b, a]].map((ops) => { + const doc = empty() + for (const op of ops) applyOps(doc, [op], catalog) + return project(doc, catalog) + }) + + expect(projections[0]).toEqual(projections[1]) + }) + + it("preserves edits to the deterministic winner and consumes the same conflict in either legal order", () => { + const a = define(subgraphId, 1) + const b = define(subgraphId, 2) + const edit = { + op: "set_widget", ...envelope(), node_id: 10, path: [subgraphId, "10"], inner_widget: "value", widget: "value", value: 9, + } as Op + + const results = [[a, edit, b], [b, edit, a]].map((ops) => { + const doc = empty() + const outcomes = ops.map((op) => applyOps(doc, [op], catalog).outcomes[0]) + return { outcomes, projection: project(doc, catalog) } + }) + + expect(results[0]!.projection).toEqual(results[1]!.projection) + expect(results[0]!.outcomes.find((outcome) => outcome?.outcome === "no-op")?.op_id).toBe(b.op_id) + const projected = (results[0]!.projection.definitions as { subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> }).subgraphs[0]! + expect(projected.nodes[0]!.widgets_values).toEqual([9]) + }) + + it("preserves nested widget edits across a digest-winning replacement in either legal order", () => { + const nestedId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const incumbent = { + ...definition(), + nodes: [{ id: 20, type: nestedId, inputs: [], outputs: [], widgets_values: [] }], + definitions: { subgraphs: [definition(nestedId, 1)] }, + } + const replacement = winningReplacement(incumbent, { + ...incumbent, + definitions: { subgraphs: [definition(nestedId, 2)] }, + }) + const first = { ...define(), subgraph_definition: incumbent } + const second = { ...define(), subgraph_definition: replacement } + const edit = { + op: "set_widget", ...envelope(), node_id: 10, path: [subgraphId, "20", "10"], inner_widget: "value", widget: "value", value: 80, + } as Op + + const projections = [[first, edit, second], [second, edit, first]].map((ops) => { + const doc = empty() + for (const op of ops) applyOps(doc, [op], catalog) + return project(doc, catalog) + }) + + expect(projections[0]).toEqual(projections[1]) + const projected = (projections[0]!.definitions as { subgraphs: Array<{ definitions: { subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> } }> }).subgraphs[0]! + expect(projected.definitions.subgraphs[0]!.nodes[0]!.widgets_values).toEqual([80]) + }) + + it("restores an instance-ID-addressed widget edit across a digest-winning replacement in both legal orders", () => { + const incumbent = definition(subgraphId, 2) + const replacement = winningReplacement(incumbent, definition(subgraphId, 4)) + const incumbentOp = { ...define(), subgraph_definition: incumbent } + const replacementOp = { ...define(), subgraph_definition: replacement } + const edit = { + op: "set_widget", ...envelope(), node_id: 10, path: ["1", "10"], + inner_widget: "value", widget: "value", value: 9, + } as Op + + const projections = [[incumbentOp, edit, replacementOp], [replacementOp, edit, incumbentOp]].map((ops) => { + const doc = mint({ + nodes: [{ id: 1, type: subgraphId, inputs: [], outputs: [] }], + links: [], + } as unknown as WorkflowJSON, catalog) + for (const op of ops) applyOps(doc, [op], catalog) + return project(doc, catalog) + }) + + expect(projections[0]).toEqual(projections[1]) + const projected = (projections[0]!.definitions as { subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> }).subgraphs[0]! + expect(projected.nodes[0]!.widgets_values).toEqual([9]) + }) + + it.each([ + ["number", 7], + ["object", { forged: "digest" }], + ])("narrows a malformed replicated %s definition digest before deterministic comparison", (_kind, malformedDigest) => { + const results = [[1, 2], [2, 1]].map(([firstValue, secondValue]) => { + const doc = empty() + applyOps(doc, [define(subgraphId, firstValue)], catalog) + metaMap(doc).set("__definition_digests", { [subgraphId]: malformedDigest }) + + let outcome: string | undefined + expect(() => { + outcome = applyOps(doc, [define(subgraphId, secondValue)], catalog).outcomes[0]?.outcome + }).not.toThrow() + return { + outcome, + projection: project(doc, catalog), + digest: (metaMap(doc).get("__definition_digests") as Record)[subgraphId], + } + }) + + expect(results[0]!.projection).toEqual(results[1]!.projection) + expect((results[0]!.projection.definitions as { subgraphs: unknown[] }).subgraphs).toEqual([definition(subgraphId, 1)]) + expect(results.map(({ outcome }) => outcome)).toEqual(["no-op", "applied"]) + expect(typeof results[1]!.digest).toBe("string") + }) + + it("rejects an edit to a node that exists only in the replaced definition", () => { + const first = { + ...definition(subgraphId, 2), + nodes: [{ id: 11, type: "Inner", inputs: [], outputs: [], widgets_values: [2] }], + } + const second = { + ...definition(subgraphId, 1), + nodes: [{ id: 12, type: "Inner", inputs: [], outputs: [], widgets_values: [1] }], + } + const doc = empty() + applyOps(doc, [{ ...define(subgraphId, 2), subgraph_definition: first }], catalog) + applyOps(doc, [{ ...define(subgraphId, 1), subgraph_definition: second }], catalog) + const winnerNodes = (project(doc, catalog).definitions as { subgraphs: Array<{ nodes: Array<{ id: number }> }> }).subgraphs[0]!.nodes + const losingNodeId = winnerNodes[0]!.id === 11 ? 12 : 11 + + const outcome = applyOps(doc, [{ + op: "set_widget", ...envelope(), node_id: losingNodeId, path: [subgraphId, String(losingNodeId)], inner_widget: "value", widget: "value", value: 9, + }], catalog).outcomes[0] + + expect(outcome?.outcome).toBe("rejected") + expect(outcome?.outcome === "rejected" ? outcome.reason.code : undefined).toBe("interior_node_not_found") + }) + + it("keeps applying a batch suffix after the deterministic conflict loser", () => { + const a = define(subgraphId, 1) + const b = define(subgraphId, 2) + const suffix = { op: "add_node", ...envelope(), node_id: 99, class_type: "Other", pos: [0, 0], node: { id: 99, type: "Other", inputs: [], outputs: [] } } as Op + const projections = [[a, b], [b, a]].map((ops) => { + const doc = empty() + applyOps(doc, [ops[0]!, ops[1]!, suffix], catalog) + return project(doc, catalog) + }) + expect(projections[0]).toEqual(projections[1]) + expect(projections[0]!.nodes.some((node) => node.id === 99)).toBe(true) + }) + + it.each([ + ["missing nodes", { ...definition(), nodes: undefined }], + ["invalid definition id", definition("not-a-uuid")], + ["duplicate normalized node ids", { ...definition(), nodes: [{ id: 1, type: "Inner" }, { id: "1", type: "Inner" }] }], + ["missing interior node id", { ...definition(), nodes: [{ type: "Inner" }] }], + ["reserved structural key", { ...definition(), node_order: [] }], + ["duplicate normalized nested definition ids", { + ...definition(), + definitions: { + subgraphs: [ + definition("abcdefab-cdef-4abc-8def-abcdefabcdef"), + definition("abcdefab-cdef-4abc-8def-abcdefabcdef"), + ], + }, + }], + ])("rejects %s without creating a partial definition", (_name, malformed) => { + const doc = empty() + const before = Y.encodeStateAsUpdate(doc) + const op = { ...define(), subgraph_id: String(malformed.id), subgraph_definition: malformed } as DefineSubgraphOp + + expect(rejectionCode(doc, op)).toBe("malformed_op") + expect(Y.encodeStateAsUpdate(doc)).toEqual(before) + expect(definitionsMap(doc).size).toBe(0) + }) + + it("projects nested definitions", () => { + const nestedId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const nested = definition(nestedId, 4) + const outer = { ...definition(), definitions: { subgraphs: [nested] } } + const doc = empty() + + expect(applyOps(doc, [{ ...define(), subgraph_definition: outer }], catalog).outcomes[0]?.outcome).toBe("applied") + const projected = (project(doc, catalog).definitions as { subgraphs: Array> }).subgraphs[0]! + expect(projected.definitions).toEqual({ subgraphs: [nested] }) + }) + + it("projects only the valid nested definition beside a malformed scalar child", () => { + const nestedId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const malformedId = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb" + const nested = definition(nestedId, 4) + const outer = { ...definition(), definitions: { subgraphs: [nested] } } + const doc = empty() + applyOps(doc, [{ ...define(), subgraph_definition: outer }], catalog) + const storedOuter = definitionsMap(doc).get(subgraphId)! + const storedDefinitions = storedOuter.get("definitions") as Y.Map + const storedSubgraphs = storedDefinitions.get("subgraphs") as Y.Map + storedSubgraphs.set(malformedId, "malformed replicated child") + storedDefinitions.set("subgraph_order", [malformedId, nestedId]) + + const projected = (project(doc, catalog).definitions as { + subgraphs: Array<{ definitions: { subgraphs: unknown[] } }> + }).subgraphs[0]! + expect(projected.definitions.subgraphs).toEqual([nested]) + }) + + it("rejects fresh reuse of an identical nested definition id while exact replay stays idempotent", () => { + const nestedId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const nested = definition(nestedId, 4) + const outer = { ...definition(), definitions: { subgraphs: [nested] } } + const op = { ...define(), subgraph_definition: outer } + const doc = empty() + + expect(applyOps(doc, [op], catalog).outcomes[0]?.outcome).toBe("applied") + const afterCreate = Y.encodeStateAsUpdate(doc) + expect(applyOps(doc, [op], catalog).outcomes[0]?.outcome).toBe("no-op") + expect(Y.encodeStateAsUpdate(doc)).toEqual(afterCreate) + + const freshReuse = { ...define(nestedId, 4), subgraph_definition: nested } + expect(rejectionCode(doc, freshReuse)).toBe("definition_conflict") + expect(Y.encodeStateAsUpdate(doc)).toEqual(afterCreate) + expect((project(doc, catalog).definitions as { subgraphs: unknown[] }).subgraphs).toEqual([outer]) + }) + + it("applies an id-addressed widget edit to a nested definition", () => { + const nestedId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const outer = { ...definition(), definitions: { subgraphs: [definition(nestedId, 4)] } } + const doc = empty() + applyOps(doc, [{ ...define(), subgraph_definition: outer }], catalog) + + const outcome = applyOps(doc, [{ + op: "set_widget", ...envelope(), node_id: 10, path: [nestedId, "10"], inner_widget: "value", widget: "value", value: 9, + }], catalog).outcomes[0]?.outcome + const projected = (project(doc, catalog).definitions as { subgraphs: Array<{ definitions: { subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> } }> }).subgraphs[0]! + + expect(outcome).toBe("applied") + expect(projected.definitions.subgraphs[0]!.nodes[0]!.widgets_values).toEqual([9]) + }) + + it("rejects a new root definition whose id collides with an existing nested definition", () => { + const nestedId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const doc = empty() + applyOps(doc, [{ ...define(), subgraph_definition: { + ...definition(), definitions: { subgraphs: [definition(nestedId, 4)] }, + } }], catalog) + + const differing = define(nestedId, 9) + const beforeDiffering = Y.encodeStateAsUpdate(doc) + expect(rejectionCode(doc, differing)).toBe("definition_conflict") + expect(Y.encodeStateAsUpdate(doc)).toEqual(beforeDiffering) + expect(appliedMap(doc).has(differing.op_id)).toBe(false) + + const identical = define(nestedId, 4) + const beforeIdentical = Y.encodeStateAsUpdate(doc) + expect(rejectionCode(doc, identical)).toBe("definition_conflict") + expect(Y.encodeStateAsUpdate(doc)).toEqual(beforeIdentical) + expect(appliedMap(doc).has(identical.op_id)).toBe(false) + }) + + it("rejects a nested definition whose id collides with an existing root definition", () => { + const newRootId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const doc = empty() + applyOps(doc, [define()], catalog) + const colliding = { + ...definition(newRootId), definitions: { subgraphs: [definition(subgraphId, 9)] }, + } + + const rejected = { ...define(newRootId), subgraph_definition: colliding } + const before = Y.encodeStateAsUpdate(doc) + expect(rejectionCode(doc, rejected)).toBe("definition_conflict") + expect(Y.encodeStateAsUpdate(doc)).toEqual(before) + expect(appliedMap(doc).has(rejected.op_id)).toBe(false) + }) + + it("rejects a winning replacement whose nested id collides with another root and preserves the incumbent", () => { + const otherRootId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const incumbent = definition() + const replacement = winningReplacement(incumbent, { + ...definition(), definitions: { subgraphs: [definition(otherRootId, 9)] }, + }) + const doc = empty() + applyOps(doc, [define(), define(otherRootId, 4)], catalog) + + const rejected = { ...define(), subgraph_definition: replacement } + const before = Y.encodeStateAsUpdate(doc) + expect(rejectionCode(doc, rejected)).toBe("definition_conflict") + expect(Y.encodeStateAsUpdate(doc)).toEqual(before) + expect(appliedMap(doc).has(rejected.op_id)).toBe(false) + expect((project(doc, catalog).definitions as { subgraphs: unknown[] }).subgraphs).toEqual([ + incumbent, + definition(otherRootId, 4), + ]) + }) + + it.each(["ancestor", "sibling"])("rejects a winning replacement whose nested id collides with an %s branch", (collision) => { + const branchId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const siblingId = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" + const incumbent = definition() + const otherRootId = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb" + const otherRoot = { ...definition(otherRootId), definitions: { subgraphs: [ + { ...definition(branchId, 2), definitions: { subgraphs: [definition(siblingId, 3)] } }, + ] } } + const collidingId = collision === "ancestor" ? branchId : siblingId + const replacement = winningReplacement(incumbent, { + ...definition(), definitions: { subgraphs: [definition(collidingId, 6)] }, + }) + const doc = empty() + applyOps(doc, [ + { ...define(), subgraph_definition: incumbent }, + { ...define(otherRootId), subgraph_definition: otherRoot }, + ], catalog) + + const rejected = { ...define(), subgraph_definition: replacement } + const before = Y.encodeStateAsUpdate(doc) + expect(rejectionCode(doc, rejected)).toBe("definition_conflict") + expect(Y.encodeStateAsUpdate(doc)).toEqual(before) + expect(appliedMap(doc).has(rejected.op_id)).toBe(false) + expect((project(doc, catalog).definitions as { subgraphs: unknown[] }).subgraphs).toEqual([incumbent, otherRoot]) + }) + + it("accepts a winning replacement that reuses only ids owned by the incumbent", () => { + const nestedId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const incumbent = { ...definition(), definitions: { subgraphs: [definition(nestedId, 2)] } } + const replacement = winningReplacement(incumbent, { + ...definition(), nodes: [{ id: 11, type: "Inner", inputs: [], outputs: [], widgets_values: [7] }], + definitions: { subgraphs: [definition(nestedId, 8)] }, + }) + const doc = empty() + applyOps(doc, [{ ...define(), subgraph_definition: incumbent }], catalog) + + expect(applyOps(doc, [{ ...define(), subgraph_definition: replacement }], catalog).outcomes[0]?.outcome).toBe("applied") + expect((project(doc, catalog).definitions as { subgraphs: unknown[] }).subgraphs).toEqual([replacement]) + }) + + it("projects a rejected colliding winner identically in both op orders", () => { + const otherRootId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const incumbent = definition() + const replacement = winningReplacement(incumbent, { + ...definition(), definitions: { subgraphs: [definition(otherRootId, 9)] }, + }) + const incumbentOp = define() + const replacementOp = { ...define(), subgraph_definition: replacement } + const seed = empty() + applyOps(seed, [define(otherRootId, 4)], catalog) + const snapshot = Y.encodeStateAsUpdate(seed) + + const projections = [[incumbentOp, replacementOp], [replacementOp, incumbentOp]].map((ops) => { + const doc = new Y.Doc() + Y.applyUpdate(doc, snapshot) + for (const op of ops) applyOps(doc, [op], catalog) + return project(doc, catalog) + }) + + expect(projections[0]).toEqual(projections[1]) + const projected = (projections[0]!.definitions as { subgraphs: Array<{ id: string }> }).subgraphs + expect(projected.find(({ id }) => id === subgraphId)).toEqual(incumbent) + expect(projected.find(({ id }) => id === otherRootId)).toEqual(definition(otherRootId, 4)) + }) + + it("rejects a direct edit to a definition with multiple live instances", () => { + const doc = empty() + applyOps(doc, [define()], catalog) + applyOps(doc, [ + { op: "add_node", ...envelope(), node_id: 1, class_type: subgraphId, pos: [0, 0], node: { id: 1, type: subgraphId, inputs: [], outputs: [] } }, + { op: "add_node", ...envelope(), node_id: 2, class_type: subgraphId, pos: [0, 0], node: { id: 2, type: subgraphId, inputs: [], outputs: [] } }, + ], catalog) + expect(rejectionCode(doc, { op: "set_widget", ...envelope(), node_id: 10, path: [subgraphId, "10"], inner_widget: "value", widget: "value", value: 9 })).toBe("shared_definition_unforked") + }) + + it("does not treat a name shared by a root and nested definition as an instance alias", () => { + const nestedId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const root = { + ...definition(), name: "Shared", + definitions: { subgraphs: [{ ...definition(nestedId), name: "Shared" }] }, + } + const doc = mint({ + nodes: [ + { id: 1, type: subgraphId, inputs: [], outputs: [] }, + { id: 2, type: "Shared", inputs: [], outputs: [] }, + ], + links: [], definitions: { subgraphs: [root] }, + } as unknown as WorkflowJSON, catalog) + const edit = { + op: "set_widget", ...envelope(), node_id: 10, path: [subgraphId, "10"], + inner_widget: "value", widget: "value", value: 9, + } as Op + + expect(applyOps(doc, [edit], catalog).outcomes[0]?.outcome).toBe("applied") + }) + + it("rejects an edit when instances occur recursively three definition levels down", () => { + const middleId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const leafId = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" + const leaf = definition(leafId, 3) + const middle = { + ...definition(middleId), + nodes: [ + { id: 21, type: leafId, inputs: [], outputs: [], widgets_values: [] }, + { id: 22, type: leafId, inputs: [], outputs: [], widgets_values: [] }, + ], + definitions: { subgraphs: [leaf] }, + } + const outer = { ...definition(), definitions: { subgraphs: [middle] } } + const doc = empty() + applyOps(doc, [{ ...define(), subgraph_definition: outer }], catalog) + const before = Y.encodeStateAsUpdate(doc) + + expect(rejectionCode(doc, { + op: "set_widget", ...envelope(), node_id: 10, path: [leafId, "10"], + inner_widget: "value", widget: "value", value: 9, + })).toBe("shared_definition_unforked") + expect(Y.encodeStateAsUpdate(doc)).toEqual(before) + }) + + it("rejects an unprojectable named widget in an interior node atomically", () => { + const doc = empty() + const malformed = { ...definition(), nodes: [{ id: 10, type: "Inner", inputs: [], outputs: [], widgets_values: { missing: 1 } }] } + const before = Y.encodeStateAsUpdate(doc) + expect(rejectionCode(doc, { ...define(), subgraph_definition: malformed })).toBe("unknown_widget") + expect(Y.encodeStateAsUpdate(doc)).toEqual(before) + }) + + it.each([ + ["ancestor", () => ({ ...definition(), definitions: { subgraphs: [definition(subgraphId)] } })], + ["cross-branch", () => { + const duplicate = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const left = { ...definition("aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"), definitions: { subgraphs: [definition(duplicate)] } } + const right = { ...definition("bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb"), definitions: { subgraphs: [definition(duplicate)] } } + return { ...definition(), definitions: { subgraphs: [left, right] } } + }], + ])("rejects a duplicate definition id against an %s atomically", (_name, makeDefinition) => { + const doc = empty() + const before = Y.encodeStateAsUpdate(doc) + expect(rejectionCode(doc, { ...define(), subgraph_definition: makeDefinition() } as DefineSubgraphOp)).toBe("malformed_op") + expect(Y.encodeStateAsUpdate(doc)).toEqual(before) + }) + + it("never projects internal definition digests, including sender-supplied nested keys", () => { + const nestedId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const nested = { ...definition(nestedId), __definition_digest: "sender-value" } + const outer = { ...definition(), definitions: { subgraphs: [nested] } } + const doc = empty() + + expect(rejectionCode(doc, { ...define(), subgraph_definition: outer } as DefineSubgraphOp)).toBe("malformed_op") + applyOps(doc, [{ ...define(), subgraph_definition: { + ...definition(), definitions: { subgraphs: [definition(nestedId)] }, + } }], catalog) + expect(JSON.stringify(project(doc, catalog))).not.toContain("__definition_digest") + }) + + it("never projects arbitrary private definition keys at any depth", () => { + const nestedId = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const doc = empty() + const privateDefinition = { + ...definition(), + __other: "root-secret", + definitions: { + subgraphs: [{ ...definition(nestedId), __private_marker: { nested: "secret" } }], + }, + } + + expect(applyOps(doc, [{ ...define(), subgraph_definition: privateDefinition }], catalog).outcomes[0]?.outcome).toBe("applied") + const serialized = JSON.stringify(project(doc, catalog)) + expect(serialized).not.toContain("__other") + expect(serialized).not.toContain("__private_marker") + expect(serialized).not.toContain("secret") + }) + + it("uses the same unknown-node outcome for UUID and non-UUID interior targets", () => { + const outcomes = ["abcdefab-cdef-4abc-8def-abcdefabcdef", "missing"].map((target) => { + const doc = empty() + return applyOps(doc, [{ + op: "set_widget", ...envelope(), node_id: 10, path: [target, "10"], inner_widget: "value", widget: "value", value: 2, + }], catalog).outcomes[0] + }) + + expect(outcomes.map((outcome) => outcome?.outcome)).toEqual(["no-op", "no-op"]) + }) + + it("uses a key-order-independent definition digest", () => { + const doc = empty() + const first = definition() + const reordered = { + links: first.links, + nodes: first.nodes, + outputs: first.outputs, + inputs: first.inputs, + name: first.name, + id: first.id, + } + + expect(applyOps(doc, [{ ...define(), subgraph_definition: first }], catalog).outcomes[0]?.outcome).toBe("applied") + const digest = definitionsMap(doc).get(subgraphId)?.get("__definition_digest") + expect(applyOps(doc, [{ ...define(), subgraph_definition: reordered }], catalog).outcomes[0]?.outcome).toBe("no-op") + expect(definitionsMap(doc).get(subgraphId)?.get("__definition_digest")).toBe(digest) + }) + + it("applies define, interior edit, then instance in one batch", () => { + const doc = empty() + const ops: Op[] = [ + define(), + { op: "set_widget", ...envelope(), node_id: 10, path: [subgraphId, "10"], inner_widget: "value", widget: "value", value: 2 }, + { op: "add_node", ...envelope(), node_id: 1, class_type: subgraphId, pos: [0, 0], node: { id: 1, type: subgraphId, inputs: [], outputs: [] } }, + ] + expect(applyOps(doc, ops, catalog).outcomes.map((outcome) => outcome.outcome)).toEqual(["applied", "applied", "applied"]) + const subgraph = (project(doc, catalog).definitions as { subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> }).subgraphs[0]! + expect(subgraph.nodes[0]!.widgets_values).toEqual([2]) + }) + + it("accepts an opaque UUID-shaped node type before its definition arrives", () => { + const doc = empty() + const missing = "abcdefab-cdef-4abc-8def-abcdefabcdef" + const instance = { op: "add_node", ...envelope(), node_id: 1, class_type: missing, pos: [0, 0], node: { id: 1, type: missing, inputs: [], outputs: [] } } as Op + expect(applyOps(doc, [instance], catalog).outcomes[0]?.outcome).toBe("applied") + }) + + it("has root/interior widget-edit parity", () => { + const root = mint({ nodes: [{ id: 10, type: "Inner", inputs: [], outputs: [], widgets_values: [1] }], links: [] } as unknown as WorkflowJSON, catalog) + const nested = empty() + applyOps(nested, [define()], catalog) + const rootOp = { op: "set_widget", ...envelope(), node_id: 10, widget: "value", value: 3 } as Op + const nestedOp = { op: "set_widget", ...envelope(), node_id: 10, path: [subgraphId, "10"], inner_widget: "value", widget: "value", value: 3 } as Op + expect(applyOps(root, [rootOp], catalog).outcomes[0]?.outcome).toBe("applied") + expect(applyOps(nested, [nestedOp], catalog).outcomes[0]?.outcome).toBe("applied") + expect(project(root, catalog).nodes[0]!.widgets_values).toEqual( + (project(nested, catalog).definitions as { subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> }).subgraphs[0]!.nodes[0]!.widgets_values, + ) + }) +}) diff --git a/test/delete-node-lww-loser-removed-links.test.ts b/test/delete-node-lww-loser-removed-links.test.ts index 51e12cda..79c3c2ed 100644 --- a/test/delete-node-lww-loser-removed-links.test.ts +++ b/test/delete-node-lww-loser-removed-links.test.ts @@ -185,27 +185,10 @@ describe("delete_node LWW loser removed_links cleanup", () => { expect(Y.encodeStateAsUpdate(doc)).toEqual(bytesAfter); }); - it("does NOT converge across arrival order once a link is installed (known divergence)", () => { - // KNOWN DIVERGENCE — characterization of current behavior, not an endorsement. - // - // The link-free sibling case above asserts the two arrival orders converge. - // That assertion only holds because its document has no links: the branch at - // `src/applier.ts:1736-1737` scrubs EVERY link incident to the node, but only - // when the delete WINS the presence gate. When the delete loses it scrubs just - // the links it named. So the set of links removed depends on arrival order: - // - // [add_node@9, delete@5] -> delete loses -> only removed_links scrubbed -> L1 survives - // [delete@5, add_node@9] -> delete wins -> incident scrub takes L1 -> L1 gone - // - // Both orders end with node 10 present, so the divergence is confined to the - // link register, and it is permanent: link deletion carries no stamp and the - // winning re-add does not restore L1. This contradicts vocabulary §6 A7 - // ("removed_links is the authoritative target set") and the determinism - // invariant. Deciding the fix (drop the incident scrub, stamp link deletions, - // or restore on re-add) is an applier-semantics call, not a test change. - // - // If a fix lands, this test SHOULD fail — replace it with a convergence - // assertion at that point. + it("restores retained intent after a winning re-add in either arrival order", () => { + // `removed_links` is authoritative. A delete that wins node presence may + // strand an incident link while the endpoint is absent, but a later winning + // re-add restores its retained descriptor. A losing delete never strands it. const op = losingDelete([999]); const forward = linkedDoc(); @@ -220,12 +203,9 @@ describe("delete_node LWW loser removed_links cleanup", () => { "applied", ]); - // Same op set, same nodes, different links. - expect(project(forward, catalog).nodes.map(({ id: nodeId }) => nodeId)).toEqual( - project(reverse, catalog).nodes.map(({ id: nodeId }) => nodeId), - ); + // Same op set, same nodes, same restored topology. + expect(project(forward, catalog)).toEqual(project(reverse, catalog)); expect(project(forward, catalog).links).toHaveLength(1); - expect(project(reverse, catalog).links).toEqual([]); - expect(project(forward, catalog)).not.toEqual(project(reverse, catalog)); + expect(project(reverse, catalog).links).toEqual(project(forward, catalog).links); }); }); diff --git a/test/doc-mint-mutation-survivors.test.ts b/test/doc-mint-mutation-survivors.test.ts index dad7c523..3ef74abd 100644 --- a/test/doc-mint-mutation-survivors.test.ts +++ b/test/doc-mint-mutation-survivors.test.ts @@ -465,7 +465,7 @@ describe("schema §1 doc layout: root-map names and the reserved opaque key are expect(doc.getMap("nodes").has("1")).toBe(true); expect(doc.getMap("links").has("7")).toBe(true); expect(doc.getMap("definitions").has("d")).toBe(true); - expect(doc.getMap("meta").get("schema_version")).toBe(2); + expect(doc.getMap("meta").get("schema_version")).toBe(3); // The bookkeeping roots are named too — `__applied` is the idempotency // gate (§4) and `__stamps` the LWW register file (§3). const op = { op: "set_widget", ...env(), node_id: 1, widget: "text", value: "w" } as SetWidgetOp; diff --git a/test/durable-link-retirement.regression.test.ts b/test/durable-link-retirement.regression.test.ts new file mode 100644 index 00000000..9e0b3b41 --- /dev/null +++ b/test/durable-link-retirement.regression.test.ts @@ -0,0 +1,120 @@ +import * as Y from "yjs"; +import { describe, expect, it } from "vitest"; +import { + applyOps, + mint, + project, + readLinkState, + type AddNodeOp, + type DeleteNodeOp, + type DisconnectOp, + type Op, + type WidgetCatalog, + type WorkflowJSON, + type WorkflowNode, +} from "../src/index.js"; + +const catalog: WidgetCatalog = { + types: { Source: { widget_order: [] }, Destination: { widget_order: [] } }, +}; +const opId = (serial: number) => serial.toString(16).padStart(32, "0"); +const env = (serial: number) => ({ + op_id: opId(serial), + actor: "agent:regression", + base_version: serial, + stamp: [serial, "agent:regression"] as [number, string], +}); +const source: WorkflowNode = { + id: 1, + type: "Source", + inputs: [], + outputs: [{ name: "out", type: "IMAGE", links: [10] }], + widgets_values: [], +}; +const destination: WorkflowNode = { + id: 2, + type: "Destination", + inputs: [{ name: "images.image0", type: "IMAGE", link: 10, grow_id: 10 }], + outputs: [], + widgets_values: [], +}; + +function workflow(): WorkflowJSON { + return { nodes: [structuredClone(source), structuredClone(destination)], links: [[10, 1, 0, 2, 0, "IMAGE"]] }; +} + +function deletion(node_id: number, serial: number, removed_links: number[] = []): DeleteNodeOp { + return { op: "delete_node", ...env(serial), node_id, removed_links }; +} + +function readd(node: WorkflowNode, serial: number): AddNodeOp { + return { op: "add_node", ...env(serial), node_id: node.id, class_type: node.type, pos: [], node }; +} + +function disconnect(serial: number, link_id = 999, to_slot = 0): DisconnectOp { + return { op: "disconnect", ...env(serial), link_id, to_node: 2, to_slot }; +} + +function snapshot(doc: Y.Doc): Y.Doc { + const restored = new Y.Doc(); + Y.applyUpdate(restored, Y.encodeStateAsUpdate(doc)); + return restored; +} + +function apply(doc: Y.Doc, ops: Op[]): void { + expect(applyOps(doc, ops, catalog).outcomes).not.toContainEqual(expect.objectContaining({ outcome: "rejected" })); +} + +describe("durable link retirement regressions", () => { + it("regression: disconnect retires a source-stranded descriptor while its destination slot is null", () => { + let doc = mint(workflow(), catalog); + apply(doc, [deletion(1, 1)]); + expect(project(doc, catalog).nodes[0]?.inputs?.[0]).toMatchObject({ link: null }); + doc = snapshot(doc); + + apply(doc, [disconnect(2)]); + expect(readLinkState(doc)).toEqual({}); + apply(doc, [readd(source, 3)]); + expect(project(snapshot(doc), catalog).links).toEqual([]); + }); + + it("regression: absent-destination disconnect follows slot semantics despite a mismatched link_id", () => { + for (const order of ["delete-first", "disconnect-first"] as const) { + let doc = mint(workflow(), catalog); + if (order === "delete-first") { + apply(doc, [deletion(2, 1)]); + doc = snapshot(doc); + apply(doc, [disconnect(2)]); + } else { + apply(doc, [disconnect(2), deletion(2, 1)]); + } + expect(readLinkState(doc), order).toEqual({}); + apply(doc, [readd(destination, 3)]); + expect(project(snapshot(doc), catalog).links, order).toEqual([]); + } + }); + + it("regression: absent-destination disconnect does not retire a different destination slot", () => { + const doc = mint(workflow(), catalog); + apply(doc, [deletion(2, 1)]); + apply(doc, [disconnect(2, 10, 1)]); + expect(readLinkState(doc)["10"]).toBeDefined(); + }); + + it("regression: imported autogrow descriptors disconnect without an operation request", () => { + const doc = mint(workflow(), catalog); + expect(() => apply(doc, [disconnect(1, 10)])).not.toThrow(); + expect(project(doc, catalog).links).toEqual([]); + expect(readLinkState(doc)).toEqual({}); + }); + + it("regression: explicit removed_links retires an already stranded descriptor", () => { + let doc = mint(workflow(), catalog); + apply(doc, [deletion(1, 1)]); + doc = snapshot(doc); + apply(doc, [deletion(2, 2, [10])]); + expect(readLinkState(doc)).toEqual({}); + apply(doc, [readd(source, 3), readd(destination, 4)]); + expect(project(snapshot(doc), catalog).links).toEqual([]); + }); +}); diff --git a/test/durable-link-state.acceptance.test.ts b/test/durable-link-state.acceptance.test.ts new file mode 100644 index 00000000..d4df1248 --- /dev/null +++ b/test/durable-link-state.acceptance.test.ts @@ -0,0 +1,270 @@ +import * as Y from "yjs"; +import { describe, expect, it } from "vitest"; +import { + applyOps, + mint, + project, + readLinkState, + type AddNodeOp, + type ClearOp, + type ConnectOp, + type DeleteNodeOp, + type DisconnectOp, + type Op, + type WorkflowJSON, + type WorkflowNode, +} from "../src/index.js"; +import { loadCatalog } from "./helpers.js"; + +const catalog = loadCatalog(); +const definitionId = "f2fdebf6-dfaf-43b6-9eb2-7f70613cfdc1"; + +const source: WorkflowNode = { + id: 10, + type: "LoadImage", + inputs: [], + outputs: [{ name: "IMAGE", type: "IMAGE", links: [] }], + widgets_values: [], +}; +const concreteTarget: WorkflowNode = { + id: 20, + type: "PreviewImage", + inputs: [{ name: "images", type: "IMAGE", link: null }], + outputs: [], + widgets_values: [], +}; +const promotedInstance: WorkflowNode = { + id: 57, + type: definitionId, + inputs: [], + outputs: [{ name: "IMAGE", type: "IMAGE", links: [] }], + widgets_values: [], +}; +const workflow: WorkflowJSON = { + nodes: [source, concreteTarget, promotedInstance], + links: [], + groups: [], + extra: {}, + last_node_id: 57, + last_link_id: 0, + definitions: { + subgraphs: [{ + id: definitionId, + name: "Stored image subgraph", + inputs: [{ name: "nested.dynamic.image", type: "IMAGE" }], + outputs: [{ name: "IMAGE", type: "IMAGE" }], + nodes: [{ id: 1, type: "PreviewImage", inputs: [{ name: "images", type: "IMAGE", link: null }], outputs: [], widgets_values: [] }], + links: [], + }], + }, +}; + +const seededSnapshot = Y.encodeStateAsUpdate(mint(workflow, catalog)); +const opId = (serial: number) => serial.toString(16).padStart(32, "0"); +const envelope = (serial: number, version: number, actor = "agent:reviewer") => ({ + op_id: opId(serial), + actor, + base_version: version, + stamp: [version, actor] as [number, string], +}); + +function fork() { + const doc = new Y.Doc(); + Y.applyUpdate(doc, seededSnapshot); + return doc; +} + +function applyAccepted(ops: Op[]) { + const doc = fork(); + const result = applyOps(doc, ops, catalog); + expect(result.outcomes).toHaveLength(ops.length); + expect(result.outcomes, "valid acceptance operations must reach semantic assertions").not.toContainEqual( + expect.objectContaining({ outcome: "rejected" }), + ); + for (const op of ops) { + const before = Y.encodeStateAsUpdate(doc); + expect(applyOps(doc, [op], catalog).outcomes).toEqual([{ op_id: op.op_id, outcome: "no-op" }]); + expect(Y.encodeStateAsUpdate(doc), `duplicate ${op.op_id} must be byte-identical`).toEqual(before); + } + return project(doc, catalog); +} + +function applyToDoc(ops: Op[]) { + const doc = fork(); + const result = applyOps(doc, ops, catalog); + expect(result.outcomes).toHaveLength(ops.length); + expect(result.outcomes).not.toContainEqual(expect.objectContaining({ outcome: "rejected" })); + return doc; +} + +function permutations(values: T[]): T[][] { + if (values.length <= 1) return [values]; + return values.flatMap((value, index) => + permutations(values.filter((_, candidate) => candidate !== index)).map(rest => [value, ...rest]), + ); +} + +const promotedConnect = (linkId: 911 | 912, serial: number, version: number): ConnectOp => ({ + op: "connect", + ...envelope(serial, version), + link_id: linkId, + from_node: 10, + from_slot: 0, + to_node: 57, + link_type: "IMAGE", + grow: { name: "nested.dynamic.image", type: "IMAGE", promoted: true }, +}); +const deletePromoted: DeleteNodeOp = { + op: "delete_node", + ...envelope(0xd1, 3), + node_id: 57, + removed_links: [], +}; +const readdPromoted: AddNodeOp = { + op: "add_node", + ...envelope(0xa1, 4), + node_id: 57, + class_type: definitionId, + pos: [], + node: promotedInstance, +}; + +describe("ADR-022 bounded durable-link acceptance", () => { + it("restores concrete intent and the exact endpoint references after delete/re-add", () => { + const connect: ConnectOp = { + op: "connect", ...envelope(0xc1, 1), link_id: 100, from_node: 10, from_slot: 0, + to_node: 20, to_slot: 0, link_type: "IMAGE", + }; + const remove: DeleteNodeOp = { op: "delete_node", ...envelope(0xd2, 2), node_id: 20, removed_links: [] }; + const readd: AddNodeOp = { op: "add_node", ...envelope(0xa2, 3), node_id: 20, class_type: concreteTarget.type, pos: [], node: concreteTarget }; + const out = applyAccepted([connect, remove, readd]); + expect(out.links).toEqual([[100, 10, 0, 20, 0, "IMAGE"]]); + expect(out.nodes.find(node => node.id === 10)).toMatchObject({ outputs: [{ links: [100] }] }); + expect(out.nodes.find(node => node.id === 20)).toMatchObject({ inputs: [{ link: 100 }] }); + }); + + it("reproduces reviewer 911/912: only the winning promoted generation returns with its full-name slot", () => { + const out = applyAccepted([ + promotedConnect(911, 0x911, 1), + promotedConnect(912, 0x912, 2), + deletePromoted, + readdPromoted, + ]); + expect(out.links).toEqual([[912, 10, 0, 57, 0, "IMAGE"]]); + expect(out.nodes.find(node => node.id === 57)).toMatchObject({ + inputs: [{ name: "nested.dynamic.image", type: "IMAGE", link: 912, grow_id: 912 }], + }); + }); + + it("orders a promoted replacement against a same-stamp disconnect", () => { + const first = promotedConnect(911, 0x911, 1); + const disconnect: DisconnectOp = { + op: "disconnect", ...envelope(0x910, 2), link_id: 911, to_node: 57, to_slot: 0, + }; + const replacement = promotedConnect(912, 0x912, 2); + const orderA = applyAccepted([first, disconnect, replacement]); + const orderB = applyAccepted([first, replacement, disconnect]); + expect(orderA).toEqual(orderB); + expect(orderA.links).toEqual([[912, 10, 0, 57, 0, "IMAGE"]]); + }); + + it("preserves the pass-7 scalar reuse and mismatched-disconnect counterexample", () => { + const first: ConnectOp = { + op: "connect", ...envelope(0xa, 0), link_id: 100, from_node: 10, + from_slot: 0, to_node: 20, to_slot: 0, link_type: "IMAGE", + }; + const sever: DisconnectOp = { + op: "disconnect", ...envelope(0xb, 0), link_id: 101, to_node: 20, to_slot: 0, + }; + const winner: ConnectOp = { ...first, ...envelope(0xc, 0) }; + const arrivals = permutations([first, sever, winner]); + expect(arrivals).toHaveLength(6); + const projections = arrivals.map((arrival, index) => { + const out = applyAccepted(arrival); + expect(out.links, `permutation ${index}: ${arrival.map(op => op.op_id).join(",")}`).toEqual([ + [100, 10, 0, 20, 0, "IMAGE"], + ]); + return out; + }); + for (const [index, out] of projections.entries()) { + expect(out, `full projection permutation ${index}`).toEqual(projections[0]); + } + }); + + it("restores operation-owned autogrow metadata, reconstructed slot, and exact endpoint references", () => { + const grow = { name: "images.image0", type: "IMAGE", widget: "upload" }; + const connect: ConnectOp = { + op: "connect", ...envelope(0xe1, 1), link_id: 913, from_node: 10, from_slot: 0, + to_node: 20, link_type: "IMAGE", grow, + }; + const remove: DeleteNodeOp = { op: "delete_node", ...envelope(0xe2, 2), node_id: 20, removed_links: [] }; + const readd: AddNodeOp = { op: "add_node", ...envelope(0xe3, 3), node_id: 20, class_type: concreteTarget.type, pos: [], node: concreteTarget }; + const doc = applyToDoc([connect, remove]); + expect(readLinkState(doc)["913"]).toMatchObject({ + authority: { kind: "operation" }, tuple: [913, 10, 0, 20, 1, "IMAGE"], + destination: { + kind: "autogrow", to_slot: 1, request: grow, + slot: { name: "images.image0", type: "IMAGE", link: 913, grow_id: 913, widget: { name: "upload" } }, + }, + }); + expect(applyOps(doc, [readd], catalog).outcomes[0]?.outcome).not.toBe("rejected"); + const out = project(doc, catalog); + expect(out.links).toEqual([[913, 10, 0, 20, 1, "IMAGE"]]); + expect(out.nodes.find(node => node.id === 10)).toMatchObject({ outputs: [{ links: [913] }] }); + expect(out.nodes.find(node => node.id === 20)).toMatchObject({ + inputs: [ + { name: "images", link: null }, + { name: "images.image0", type: "IMAGE", link: 913, grow_id: 913, widget: { name: "upload" } }, + ], + }); + }); + + it("clear strands durable intent without retiring it and re-add restores the link", () => { + const connect: ConnectOp = { + op: "connect", ...envelope(0xf1, 1), link_id: 914, from_node: 10, from_slot: 0, + to_node: 20, to_slot: 0, link_type: "IMAGE", + }; + const clear: ClearOp = { op: "clear", ...envelope(0xf2, 2), removed_nodes: [20] }; + const readd: AddNodeOp = { op: "add_node", ...envelope(0xf3, 3), node_id: 20, class_type: concreteTarget.type, pos: [], node: concreteTarget }; + const doc = applyToDoc([connect, clear]); + expect(project(doc, catalog).links).toEqual([]); + expect(readLinkState(doc)["914"]).toBeDefined(); + expect(applyOps(doc, [readd], catalog).outcomes[0]?.outcome).not.toBe("rejected"); + expect(project(doc, catalog).links).toEqual([[914, 10, 0, 20, 0, "IMAGE"]]); + expect(readLinkState(doc)["914"]).toBeDefined(); + }); + + const lifecycleKinds = [ + { name: "concrete", connect: { op: "connect", ...envelope(0x201, 1), link_id: 921, from_node: 10, from_slot: 0, to_node: 20, to_slot: 0, link_type: "IMAGE" } as ConnectOp, target: concreteTarget }, + { name: "promoted", connect: promotedConnect(911, 0x202, 1), target: promotedInstance }, + { name: "autogrow", connect: { op: "connect", ...envelope(0x203, 1), link_id: 923, from_node: 10, from_slot: 0, to_node: 20, link_type: "IMAGE", grow: { name: "images.image0", type: "IMAGE" } } as ConnectOp, target: concreteTarget }, + ]; + const retirementModes = ["restore", "disconnect", "explicit removed_links"] as const; + + it.each(lifecycleKinds.flatMap(kind => retirementModes.map(mode => ({ ...kind, mode }))))( + "$name durable intent: $mode", + ({ connect, target, mode }) => { + const targetId = target.id; + const remove: DeleteNodeOp = { op: "delete_node", ...envelope(0x210, 2), node_id: targetId, removed_links: [] }; + const doc = applyToDoc([connect, remove]); + const descriptor = readLinkState(doc)[String(connect.link_id)] as { destination: { to_slot: number } }; + expect(descriptor).toBeDefined(); + if (mode === "disconnect") { + const disconnect: DisconnectOp = { op: "disconnect", ...envelope(0x211, 3), link_id: connect.link_id, to_node: targetId, to_slot: descriptor.destination.to_slot }; + expect(applyOps(doc, [disconnect], catalog).outcomes[0]?.outcome).not.toBe("rejected"); + } else if (mode === "explicit removed_links") { + const retire: DeleteNodeOp = { op: "delete_node", ...envelope(0x212, 3), node_id: targetId, removed_links: [connect.link_id] }; + expect(applyOps(doc, [retire], catalog).outcomes[0]?.outcome).not.toBe("rejected"); + } + const readd: AddNodeOp = { op: "add_node", ...envelope(0x213, 4), node_id: targetId, class_type: target.type, pos: [], node: target }; + expect(applyOps(doc, [readd], catalog).outcomes[0]?.outcome).not.toBe("rejected"); + if (mode === "restore") { + expect(project(doc, catalog).links).toEqual([[connect.link_id, 10, 0, targetId, descriptor.destination.to_slot, "IMAGE"]]); + expect(readLinkState(doc)[String(connect.link_id)]).toBeDefined(); + } else { + expect(project(doc, catalog).links).toEqual([]); + expect(readLinkState(doc)[String(connect.link_id)]).toBeUndefined(); + } + }, + ); +}); diff --git a/test/incarnation-stamps.test.ts b/test/incarnation-stamps.test.ts index 57038719..7fcaba84 100644 --- a/test/incarnation-stamps.test.ts +++ b/test/incarnation-stamps.test.ts @@ -66,6 +66,17 @@ function fork(): Y.Doc { return doc; } +// Preserve the historical migration counterexamples as refusal cases. Schema +// v3 deliberately has no compatibility reader; refusing must not rewrite even +// a malformed or colliding legacy register before reporting the error. +function assertOldSchemaRefused(doc: Y.Doc): void { + const before = Y.encodeStateAsUpdate(doc); + const roots = [...doc.share.keys()]; + expect(() => migrate(doc, 1)).toThrow(/private-alpha no-compat-reader policy/); + expect(Y.encodeStateAsUpdate(doc)).toEqual(before); + expect([...doc.share.keys()]).toEqual(roots); +} + describe("incarnation-namespaced widget stamps (DQ-11)", () => { it("converges when a life-1 write arrives before or after delete/re-add", () => { const replacement = readd(); @@ -84,7 +95,7 @@ describe("incarnation-namespaced widget stamps (DQ-11)", () => { .toBeUndefined(); }); - it("migrates v1 node lifetimes and widget stamp keys to legacy life 0", () => { + it("refuses to relabel a v1 document as the current schema", () => { const doc = mint(base(), catalog); const node = doc.getMap>("nodes").get("1")!; node.delete(NODE_INCARNATION_KEY); @@ -93,21 +104,17 @@ describe("incarnation-namespaced widget stamps (DQ-11)", () => { stamps.set(oldKey, [7, "human:a", id("old")]); doc.getMap("meta").set("schema_version", 1); - migrate(doc, 1); - - expect(doc.getMap("meta").get("schema_version")).toBe(2); - expect(node.get(NODE_INCARNATION_KEY)).toBe(LEGACY_NODE_INCARNATION); - expect(stamps.get(JSON.stringify(["widget", "1", LEGACY_NODE_INCARNATION, "text"]))).toEqual([ - 7, - "human:a", - id("old"), - ]); - expect(stamps.has(oldKey)).toBe(false); + const before = Y.encodeStateAsUpdate(doc); + expect(() => migrate(doc, 1)).toThrow(/private-alpha no-compat-reader policy/); + expect(Y.encodeStateAsUpdate(doc)).toEqual(before); + expect(doc.getMap("meta").get("schema_version")).toBe(1); + expect(node.has(NODE_INCARNATION_KEY)).toBe(false); + expect(stamps.has(oldKey)).toBe(true); }); for (const legacyOrder of ["numeric-first", "string-first"] as const) { for (const winnerForm of ["numeric", "string"] as const) { - it(`normalizes colliding numeric/string widget targets and keeps the ${winnerForm} LWW winner (${legacyOrder})`, () => { + it(`refuses legacy numeric/string collisions without choosing the ${winnerForm} LWW winner (${legacyOrder})`, () => { const doc = mint(base(), catalog); const stamps = doc.getMap("__stamps"); const numericKey = JSON.stringify(["widget", 1, "text"]); @@ -121,11 +128,11 @@ describe("incarnation-namespaced widget stamps (DQ-11)", () => { for (const [key, stamp] of entries) stamps.set(key, stamp); doc.getMap("meta").set("schema_version", 1); - migrate(doc, 1); + assertOldSchemaRefused(doc); - expect(stamps.get(migratedKey)).toEqual(winner); - expect(stamps.has(numericKey)).toBe(false); - expect(stamps.has(stringKey)).toBe(false); + expect(stamps.has(migratedKey)).toBe(false); + expect(stamps.get(numericKey)).toEqual(winnerForm === "numeric" ? winner : loser); + expect(stamps.get(stringKey)).toEqual(winnerForm === "string" ? winner : loser); }); } } @@ -138,7 +145,7 @@ describe("incarnation-namespaced widget stamps (DQ-11)", () => { ["counter (legacy wins)", [8, "human:z", id("zzz")], [9, "human:a", id("aaa")]], ["actor (legacy wins)", [8, "human:a", id("zzz")], [8, "human:z", id("aaa")]], ["op_id (legacy wins)", [8, "human:a", id("aaa")], [8, "human:a", id("zzz")]], - ] as const)(`preserves the greater stamp against a normalized incumbent at the %s tier (${insertionOrder})`, (tier, incumbent, legacy) => { + ] as const)(`refuses legacy/normalized collisions without rewriting either stamp at the %s tier (${insertionOrder})`, (_tier, incumbent, legacy) => { const doc = mint(base(), catalog); const stamps = doc.getMap("__stamps"); const oldKey = JSON.stringify(["widget", 1, "text"]); @@ -149,10 +156,10 @@ describe("incarnation-namespaced widget stamps (DQ-11)", () => { for (const [key, stamp] of entries) stamps.set(key, stamp); doc.getMap("meta").set("schema_version", 1); - migrate(doc, 1); + assertOldSchemaRefused(doc); - expect(stamps.get(migratedKey)).toEqual(tier.endsWith("(legacy wins)") ? legacy : incumbent); - expect(stamps.has(oldKey)).toBe(false); + expect(stamps.get(migratedKey)).toEqual(incumbent); + expect(stamps.get(oldKey)).toEqual(legacy); }); } @@ -169,7 +176,7 @@ describe("incarnation-namespaced widget stamps (DQ-11)", () => { stamps.set(malformedKey, value); doc.getMap("meta").set("schema_version", 1); - migrate(doc, 1); + assertOldSchemaRefused(doc); expect(stamps.get(malformedKey)).toEqual(value); expect(stamps.has(JSON.stringify(["widget", String(target[1]), LEGACY_NODE_INCARNATION, target[2]]))).toBe(false); @@ -182,13 +189,13 @@ describe("incarnation-namespaced widget stamps (DQ-11)", () => { stamps.set(oldKey, ["not-a-counter", "human:a", id("malformed")]); doc.getMap("meta").set("schema_version", 1); - migrate(doc, 1); + assertOldSchemaRefused(doc); expect(stamps.get(oldKey)).toEqual(["not-a-counter", "human:a", id("malformed")]); expect(stamps.has(JSON.stringify(["widget", "1", LEGACY_NODE_INCARNATION, "text"]))).toBe(false); }); - it("replaces a malformed colliding target with a valid legacy stamp", () => { + it("refuses a malformed colliding target without replacing it with a valid legacy stamp", () => { const doc = mint(base(), catalog); const stamps = doc.getMap("__stamps"); const oldKey = JSON.stringify(["widget", 1, "text"]); @@ -198,9 +205,9 @@ describe("incarnation-namespaced widget stamps (DQ-11)", () => { stamps.set(oldKey, valid); doc.getMap("meta").set("schema_version", 1); - migrate(doc, 1); + assertOldSchemaRefused(doc); - expect(stamps.get(migratedKey)).toEqual(valid); - expect(stamps.has(oldKey)).toBe(false); + expect(stamps.get(migratedKey)).toEqual(["not-a-counter"]); + expect(stamps.get(oldKey)).toEqual(valid); }); }); diff --git a/test/insert-workflow-link-validation.regression.test.ts b/test/insert-workflow-link-validation.regression.test.ts new file mode 100644 index 00000000..23b32826 --- /dev/null +++ b/test/insert-workflow-link-validation.regression.test.ts @@ -0,0 +1,37 @@ +import * as Y from "yjs"; +import { expect, it } from "vitest"; +import { applyOps, hasAppliedOp, mint, type Op, type WidgetCatalog } from "../src/index.js"; + +// Joint insertion/durable-state recovery must validate every descriptor before +// committing any prefix, including definitions and an earlier valid link. +it("regression: invalid later inserted link leaves the document and operation ledger untouched", () => { + const catalog: WidgetCatalog = { types: { Source: { widget_order: [] }, Sink: { widget_order: [] } } }; + const doc = mint({ nodes: [], links: [] }, catalog); + const op = { + op: "insert_workflow", + op_id: "a7".repeat(16), + actor: "agent:validation", + base_version: 5, + stamp: [5, "agent:validation"], + workflow: { + nodes: [ + { id: 1, type: "Source", outputs: [{ name: "out", type: "IMAGE", links: [10, 11] }] }, + { id: 2, type: "Sink", inputs: [{ name: "in", type: "IMAGE", link: 10 }] }, + { id: 3, type: "Sink", inputs: [{ name: "in", type: "IMAGE", link: 11 }] }, + ], + links: [[10, 1, 0, 2, 0, "IMAGE"], [11, 1, -1, 3, 0, "IMAGE"]], + definitions: { subgraphs: [{ id: "retained-template", nodes: [], links: [] }] }, + }, + } as Op; + const before = Y.encodeStateAsUpdate(doc); + for (let attempt = 0; attempt < 2; attempt++) { + const result = applyOps(doc, [op], catalog); + expect(Y.encodeStateAsUpdate(doc)).toEqual(before); + expect(result.outcomes).toEqual([ + expect.objectContaining({ + op_id: op.op_id, outcome: "rejected", reason: expect.objectContaining({ code: "malformed_op" }), + }), + ]); + expect(hasAppliedOp(doc, op.op_id)).toBe(false); + } +}); diff --git a/test/insert-workflow.test.ts b/test/insert-workflow.test.ts new file mode 100644 index 00000000..36ecb2bb --- /dev/null +++ b/test/insert-workflow.test.ts @@ -0,0 +1,955 @@ +/** + * `insert_workflow` — merge a template workflow (nodes, links, + * `definitions.subgraphs`) into an existing doc in ONE transaction. + * + * TDD V1.5 (in-app-agent program, ADR-T8): today a template with subgraphs can + * only reach a live doc through a whole-doc reseed, because every other op + * schema rejects definition-bearing fields (KA-11). This op carries the + * template as an opaque payload the applier merges under the same guards the + * seven existing kinds obey: collision-free ids (validated BEFORE any mutation, + * KA-4), definition dedupe/fork by content hash, exact-replay no-op through + * the op_id gate, and byte-identical doc on rejection. + * + * ID allocation belongs to the applier and is derived from the immutable op + * id plus each raw id, never from mutable document state. + */ +import { describe, expect, it } from "vitest"; +import * as Y from "yjs"; + +import { appliedMap } from "../src/doc.js"; +import { + OPAQUE_WIDGETS_KEY, + applyOps, + mint, + project, + type Op, + type AddNodeOp, + type DefineSubgraphOp, + type DeleteNodeOp, + type DisconnectOp, + type SubgraphDefinition, + type WidgetCatalog, + type WorkflowJSON, +} from "../src/index.js"; +import { remapInsertedWorkflowIds } from "../src/remap.js"; +import { appliedOpIds, noOpIds, rejectedOutcomeWithIndex } from "./apply-result-helpers.js"; + +const catalog: WidgetCatalog = { + types: { + Src: { widget_order: [] }, + Sink: { widget_order: ["inputcount"] }, + KSampler: { widget_order: ["seed", "steps"] }, + Inner: { widget_order: ["text"] }, + }, +}; + +function baseWorkflow(): WorkflowJSON { + return { + last_node_id: 7, + last_link_id: 7, + nodes: [ + { id: 1, type: "Src" }, + { id: 2, type: "Src" }, + { id: 3, type: "Sink", inputs: [{ name: "in", link: 7 }], widgets_values: [1] }, + { id: 4, type: "KSampler", widgets_values: [1, 2] }, + { id: 6, type: "def-1" }, + ], + links: [[7, 2, 0, 3, 0, "X"]], + definitions: { + subgraphs: [{ id: "def-1", name: "D", nodes: [{ id: 27, type: "Inner", widgets_values: ["t"] }], links: [] }], + }, + } as unknown as WorkflowJSON; +} + +/** A template with its OWN definition `def-2` and ids already disjoint from the base. */ +function template(): WorkflowJSON { + return { + nodes: [ + { id: 100, type: "Src", pos: [0, 0] }, + { id: 101, type: "def-2", inputs: [{ name: "a", link: 200 }] }, + ], + links: [[200, 100, 0, 101, 0, "X"]], + definitions: { + subgraphs: [{ id: "def-2", name: "E", nodes: [{ id: 5, type: "Inner", widgets_values: ["u"] }], links: [] }], + }, + } as unknown as WorkflowJSON; +} + +let seq = 0; +function env(): Pick { + return { + op_id: ("i" + String(seq++).padStart(4, "0")).padEnd(32, "0"), + actor: "a", + base_version: 1, + stamp: [1, "a"] as [number, string], + }; +} + +function insertOp(workflow: unknown, overrides: Partial = {}): Op { + return { ...env(), op: "insert_workflow", workflow, ...overrides } as unknown as Op; +} + +const bytes = (doc: Y.Doc): Buffer => Buffer.from(Y.encodeStateAsUpdate(doc)); + +function ids(wf: WorkflowJSON): unknown[] { + return (wf.nodes ?? []).map((n) => n.id); +} +function linkIds(wf: WorkflowJSON): unknown[] { + return (wf.links ?? []).map((l) => (l as unknown[])[0]); +} +function defIds(wf: WorkflowJSON): string[] { + const defs = (wf as { definitions?: { subgraphs?: { id: unknown }[] } }).definitions; + return (defs?.subgraphs ?? []).map((s) => String(s.id)); +} + +describe("insert_workflow: happy path", () => { + it("maps one explicit raw group id differently for two fixed operation ids", () => { + // Independently derived with Node 22.22.2, without importing the production remapper: + // node -e 'for(const op of ["c641df0c31f9440b9385ac8e01e099b2","0123456789abcdef0123456789abcdef"]) console.log(`insert:${op}:root:group:${encodeURIComponent(JSON.stringify("raw-group"))}`)' + const vectors = [ + [ + "c641df0c31f9440b9385ac8e01e099b2", + "insert:c641df0c31f9440b9385ac8e01e099b2:root:group:%22raw-group%22", + ], + [ + "0123456789abcdef0123456789abcdef", + "insert:0123456789abcdef0123456789abcdef:root:group:%22raw-group%22", + ], + ] as const; + + for (const [opId, expectedGroupId] of vectors) { + const doc = mint({ nodes: [], links: [] }, catalog); + const op = insertOp( + { nodes: [], links: [], groups: [{ id: "raw-group", title: "Pinned group" }] }, + { op_id: opId }, + ); + seq--; + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + expect(project(doc, catalog).groups).toEqual([{ id: expectedGroupId, title: "Pinned group" }]); + } + }); + + it("maps the frozen CLI asymmetric raw-ID fixture to exact operation-derived IDs", () => { + const doc = mint({ nodes: [], links: [] }, catalog); + const op = { + op: "insert_workflow", + op_id: "c641df0c31f9440b9385ac8e01e099b2", + actor: "consumer-pass30", + base_version: 48, + stamp: [48, "consumer-pass30"], + workflow: { + nodes: [{ id: 701, type: "def-asym" }], + definitions: { + subgraphs: [{ id: "def-asym", links: [], nodes: [{ id: "inner-Z9", type: "Inner" }] }], + }, + }, + } as Op; + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const workflow = project(doc, catalog); + const definition = workflow.definitions!.subgraphs![0] as { + id: string; + nodes: Array<{ id: string }>; + }; + expect(workflow.nodes![0]!.id).toBe("insert:c641df0c31f9440b9385ac8e01e099b2:root:node:701"); + expect(workflow.nodes![0]!.type).toBe("c7d70ff1-6dca-4b80-83ee-351b8510f528"); + expect(definition.id).toBe("c7d70ff1-6dca-4b80-83ee-351b8510f528"); + expect(definition.nodes[0]!.id).toBe( + "insert:c641df0c31f9440b9385ac8e01e099b2:root/definition:%22def-asym%22:node:%22inner-Z9%22", + ); + }); + + it("resolves string link endpoints to normalized numeric node ids", () => { + const doc = mint(baseWorkflow(), catalog); + const op = insertOp({ + nodes: [{ id: 1, type: "Src", title: "alias source" }, { id: 2, type: "Sink", title: "alias sink" }], + links: [[3, "1", 0, "2", 0, "alias"]], + }); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const workflow = project(doc, catalog); + const source = workflow.nodes!.find((node) => node.title === "alias source")!; + const sink = workflow.nodes!.find((node) => node.title === "alias sink")!; + const link = workflow.links!.find((candidate) => (candidate as unknown[])[5] === "alias") as unknown[]; + expect(link).toBeDefined(); + expect(link[1]).toBe(source.id); + expect(link[3]).toBe(sink.id); + }); + + it("scopes repeated nested definition ids to their sibling definition paths", () => { + const doc = mint(baseWorkflow(), catalog); + const nested = (name: string) => ({ id: "shared", name, nodes: [], links: [] }); + const op = insertOp({ + nodes: [], + links: [], + definitions: { + subgraphs: [ + { + id: "left", + name: "Left", + nodes: [{ id: 1, type: "shared" }], + links: [], + definitions: { subgraphs: [nested("Left shared")] }, + }, + { + id: "right", + name: "Right", + nodes: [{ id: 1, type: "shared" }], + links: [], + definitions: { subgraphs: [nested("Right shared")] }, + }, + ], + }, + }); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const definitions = project(doc, catalog).definitions!.subgraphs! as Array>; + const left = definitions.find((definition) => definition["name"] === "Left")!; + const right = definitions.find((definition) => definition["name"] === "Right")!; + const nestedId = (definition: Record): string => + String(((definition["definitions"] as { subgraphs: Array<{ id: unknown }> }).subgraphs[0]!).id); + const hostType = (definition: Record): string => + String(((definition["nodes"] as Array<{ type: unknown }>)[0]!).type); + + expect(nestedId(left)).not.toBe(nestedId(right)); + expect(hostType(left)).toBe(nestedId(left)); + expect(hostType(right)).toBe(nestedId(right)); + }); + + it("drops a depth-2 object link with a missing endpoint and keeps its valid sibling", () => { + const nested = { + id: "nested", + nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], + links: [ + { id: 10, origin_id: 1, origin_slot: 0, target_id: 2, target_slot: 0, type: "valid" }, + { id: 11, origin_id: 1, origin_slot: 0, target_id: 999, target_slot: 0, type: "dangling" }, + ], + }; + const outer = { id: "outer", nodes: [], links: [], definitions: { subgraphs: [nested] } }; + + const remapped = remapInsertedWorkflowIds( + { nodes: [], definitions: { subgraphs: [outer] } } as unknown as WorkflowJSON, + "object-link-op".padEnd(32, "0"), + ) as unknown as { definitions: { subgraphs: Array<{ definitions: { subgraphs: Array<{ links: Array<{ type: string }> }> } }> } }; + const links = remapped.definitions.subgraphs[0]!.definitions.subgraphs[0]!.links; + + expect(links).toHaveLength(1); + expect(links[0]!.type).toBe("valid"); + }); + + it("scrubs a dropped depth-2 link from node slot references", () => { + const nested = { + id: "nested", + nodes: [ + { id: 1, type: "Src", outputs: [{ name: "out", links: [11, 10] }] }, + { id: 2, type: "Sink", inputs: [{ name: "in", link: 11 }] }, + ], + links: [ + [10, 1, 0, 2, 0, "valid"], + [11, 999, 0, 2, 0, "dangling"], + ], + }; + const outer = { id: "outer", nodes: [], links: [], definitions: { subgraphs: [nested] } }; + + const remapped = remapInsertedWorkflowIds( + { nodes: [], definitions: { subgraphs: [outer] } } as unknown as WorkflowJSON, + "nested-slot-op".padEnd(32, "0"), + ) as unknown as { + definitions: { + subgraphs: Array<{ + definitions: { subgraphs: Array<{ nodes: Array<{ inputs?: Array<{ link: unknown }>; outputs?: Array<{ links: unknown[] }> }>; links: unknown[][] }> }; + }>; + }; + }; + const graph = remapped.definitions.subgraphs[0]!.definitions.subgraphs[0]!; + + expect(graph.nodes[0]!.outputs![0]!.links).toEqual([graph.links[0]![0]]); + expect(graph.nodes[1]!.inputs![0]!.link).toBeNull(); + }); + + it("scrubs a dropped top-level link from node slot references", () => { + const remapped = remapInsertedWorkflowIds( + { + nodes: [ + { id: 1, type: "Src", outputs: [{ name: "out", links: [11, 10] }] }, + { id: 2, type: "Sink", inputs: [{ name: "in", link: 11 }] }, + ], + links: [ + [10, 1, 0, 2, 0, "valid"], + [11, 999, 0, 2, 0, "dangling"], + ], + } as unknown as WorkflowJSON, + "top-level-slot-op".padEnd(32, "0"), + ); + + const graph = remapped as unknown as { + nodes: Array<{ inputs?: Array<{ link: unknown }>; outputs?: Array<{ links: unknown[] }> }>; + links: unknown[][]; + }; + expect(graph.nodes[0]!.outputs![0]!.links).toEqual([graph.links[0]![0]]); + expect(graph.nodes[1]!.inputs![0]!.link).toBeNull(); + }); + + it("normalizes string aliases when scrubbing dropped link slot references", () => { + const remapped = remapInsertedWorkflowIds( + { + nodes: [ + { id: 1, type: "Src", outputs: [{ name: "out", links: ["11", 10] }] }, + { id: 2, type: "Sink", inputs: [{ name: "in", link: "11" }] }, + ], + links: [ + [10, 1, 0, 2, 0, "valid"], + [11, 999, 0, 2, 0, "dangling"], + ], + } as unknown as WorkflowJSON, + "string-alias-slot-op".padEnd(32, "0"), + ); + + const graph = remapped as unknown as { + nodes: Array<{ inputs?: Array<{ link: unknown }>; outputs?: Array<{ links: unknown[] }> }>; + links: unknown[][]; + }; + expect(graph.nodes[0]!.outputs![0]!.links).toEqual([graph.links[0]![0]]); + expect(graph.nodes[1]!.inputs![0]!.link).toBeNull(); + }); + + it.each(["root", "nested"] as const)("preserves null input sentinels beside a literal 'null' link at %s", (scope) => { + const graph = { + nodes: [ + { id: 1, type: "Src" }, + { + id: 2, + type: "Sink", + title: `${scope} sentinel sink`, + inputs: [{ name: "unconnected", link: null }, { name: "connected", link: "null" }], + }, + ], + links: [["null", 1, 0, 2, 1, "literal-null-id"]], + }; + const workflow = scope === "root" + ? graph + : { nodes: [], links: [], definitions: { subgraphs: [{ id: "sentinel-def", ...graph }] } }; + const source = structuredClone(workflow); + const doc = mint({ nodes: [], links: [] }, catalog); + const op: Op = { + op: "insert_workflow", workflow, op_id: `sentinel-${scope}`, + actor: "sentinel", base_version: 1, stamp: [1, "sentinel"], + }; + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const projected = project(doc, catalog); + const projectedGraph = (scope === "root" + ? projected + : projected.definitions!.subgraphs![0]!) as { + nodes: Array<{ title?: string; inputs?: Array<{ name: string; link: unknown }> }>; + links: unknown[][]; + }; + const sink = projectedGraph.nodes!.find((node) => node.title === `${scope} sentinel sink`)!; + const remappedLinkId = (projectedGraph.links![0] as unknown[])[0]; + + expect(sink.inputs).toEqual([ + { name: "unconnected", link: null }, + { name: "connected", link: remappedLinkId }, + ]); + expect(workflow).toEqual(source); + }); + + it("drops a depth-3 array link with a missing endpoint", () => { + const deepest = { + id: "deepest", + nodes: [{ id: 1, type: "Src" }], + links: [[10, 1, 0, 999, 0, "dangling"]], + }; + const middle = { id: "middle", nodes: [], links: [], definitions: { subgraphs: [deepest] } }; + const outer = { id: "outer", nodes: [], links: [], definitions: { subgraphs: [middle] } }; + + const remapped = remapInsertedWorkflowIds( + { nodes: [], definitions: { subgraphs: [outer] } } as unknown as WorkflowJSON, + "array-link-op".padEnd(32, "0"), + ) as unknown as { definitions: { subgraphs: Array<{ definitions: { subgraphs: Array<{ definitions: { subgraphs: Array<{ links: unknown[] }> } }> } }> } }; + + expect(remapped.definitions.subgraphs[0]!.definitions.subgraphs[0]!.definitions.subgraphs[0]!.links).toEqual([]); + }); + + it("keeps valid links at every nested definition depth", () => { + const deepest = { + id: "deepest", + nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], + links: [[10, 1, 0, 2, 0, "valid"]], + }; + const outer = { id: "outer", nodes: [], links: [], definitions: { subgraphs: [deepest] } }; + + const remapped = remapInsertedWorkflowIds( + { nodes: [], definitions: { subgraphs: [outer] } } as unknown as WorkflowJSON, + "valid-link-op".padEnd(32, "0"), + ) as unknown as { definitions: { subgraphs: Array<{ definitions: { subgraphs: Array<{ links: unknown[][] }> } }> } }; + const links = remapped.definitions.subgraphs[0]!.definitions.subgraphs[0]!.links; + + expect(links).toHaveLength(1); + expect(links[0]![5]).toBe("valid"); + expect(links[0]![1]).toEqual(expect.stringContaining("valid-link-op")); + expect(links[0]![3]).toEqual(expect.stringContaining("valid-link-op")); + }); + + it("merges template nodes, links and definitions into the doc in one op", () => { + const doc = mint(baseWorkflow(), catalog); + const op = insertOp(template()); + const result = applyOps(doc, [op], catalog); + + expect(appliedOpIds(result)).toEqual([op.op_id]); + const wf = project(doc, catalog); + expect(ids(wf).slice(0, 5)).toEqual([1, 2, 3, 4, 6]); + expect(ids(wf).slice(5).every((id) => typeof id === "string" && id.includes(op.op_id))).toBe(true); + expect(linkIds(wf)[1]).toEqual(expect.stringContaining(op.op_id)); + expect(defIds(wf).find((id) => id !== "def-1")).toMatch(/^[0-9a-f-]{36}$/); + // Inserted nodes project with their widgets resolved through the catalog. + const n100 = wf.nodes!.find((n) => n.pos?.[0] === 0)!; + expect(n100.type).toBe("Src"); + expect(n100.pos).toEqual([0, 0]); + const n101 = wf.nodes!.find((n) => n.type === defIds(wf)[1])!; + expect((n101.inputs?.[0] as { link?: unknown } | undefined)?.link).toBe(linkIds(wf)[1]); + expect(wf.last_node_id).toBe(7); + expect(wf.last_link_id).toBe(7); + }); + + it("retains coherent imported link intent across snapshot and endpoint lifetimes until disconnect", () => { + const inserted = mint({ nodes: [], links: [] }, catalog); + const op = insertOp({ + nodes: [ + { id: 1, type: "Src", outputs: [{ name: "out", links: [3] }] }, + { id: 2, type: "Sink", inputs: [{ name: "in", link: 3 }] }, + ], + links: [[3, 1, 0, 2, 0, "X"]], + }); + applyOps(inserted, [op], catalog); + const doc = new Y.Doc(); + Y.applyUpdate(doc, Y.encodeStateAsUpdate(inserted)); + const initial = project(doc, catalog); + const source = initial.nodes![0]!; + const target = initial.nodes![1]!; + const link = initial.links![0] as unknown[]; + const remove = { op: "delete_node", ...env(), node_id: target.id, removed_links: [] } as DeleteNodeOp; + const readd = { + op: "add_node", ...env(), node_id: target.id, class_type: target.type, pos: [], node: target, + } as AddNodeOp; + + applyOps(doc, [remove, readd], catalog); + expect(project(doc, catalog).links).toEqual([link]); + expect(project(doc, catalog).nodes!.find(node => node.id === source.id)).toMatchObject({ outputs: [{ links: [link[0]] }] }); + + const disconnect = { + op: "disconnect", ...env(), link_id: link[0], to_node: target.id, to_slot: 0, + } as DisconnectOp; + const removeAgain = { op: "delete_node", ...env(), node_id: target.id, removed_links: [] } as DeleteNodeOp; + const readdAgain = { + op: "add_node", ...env(), node_id: target.id, class_type: target.type, pos: [], node: target, + } as AddNodeOp; + applyOps(doc, [disconnect, removeAgain, readdAgain], catalog); + expect(project(doc, catalog).links).toEqual([]); + }); + + it("treats an identical define after insertion as the same projected definition", () => { + const doc = mint({ nodes: [], links: [] }, catalog); + const op = insertOp({ + nodes: [], + definitions: { subgraphs: [{ id: "raw", name: "D", nodes: [{ id: 1, type: "Inner", widgets_values: ["x"] }], links: [] }] }, + }); + applyOps(doc, [op], catalog); + const inserted = project(doc, catalog).definitions!.subgraphs![0]! as SubgraphDefinition; + const define = { + op: "define_subgraph", ...env(), subgraph_id: inserted.id, subgraph_definition: inserted, + } as DefineSubgraphOp; + expect(applyOps(doc, [define], catalog).outcomes[0]?.outcome).toBe("no-op"); + expect(project(doc, catalog).definitions!.subgraphs).toEqual([inserted]); + }); + + it("is an exact-replay no-op through the op_id gate (no second copy)", () => { + const doc = mint(baseWorkflow(), catalog); + const op = insertOp(template()); + applyOps(doc, [op], catalog); + const before = bytes(doc); + const again = applyOps(doc, [op], catalog); + expect(noOpIds(again)).toEqual([op.op_id]); + expect(bytes(doc).equals(before)).toBe(true); + expect(ids(project(doc, catalog))).toHaveLength(7); + }); + + it("remaps a template definition even when its raw id matches live content", () => { + const doc = mint(baseWorkflow(), catalog); + const tpl = template(); + (tpl as { definitions: { subgraphs: unknown[] } }).definitions.subgraphs = [ + { id: "def-1", name: "D", nodes: [{ id: 27, type: "Inner", widgets_values: ["t"] }], links: [] }, + ]; + tpl.nodes![1] = { id: 101, type: "def-1", inputs: [{ name: "a", link: 200 }] }; + const op = insertOp(tpl); + const result = applyOps(doc, [op], catalog); + expect(appliedOpIds(result)).toEqual([op.op_id]); + const wf = project(doc, catalog); + expect(defIds(wf)).toHaveLength(2); + const insertedDefinitionId = defIds(wf).find((id) => id !== "def-1")!; + expect(insertedDefinitionId).toMatch(/^[0-9a-f-]{36}$/); + expect(wf.nodes!.find((n) => n.id !== 6 && n.type === insertedDefinitionId)).toBeDefined(); + }); + + it("preserves opaque widgets_values for uncatalogued inserted nodes", () => { + const doc = mint(baseWorkflow(), catalog); + const tpl = { nodes: [{ id: 300, type: "Note", widgets_values: ["hello"] }], links: [] } as unknown as WorkflowJSON; + applyOps(doc, [insertOp(tpl)], catalog); + const wf = project(doc, catalog); + expect(wf.nodes!.find((n) => n.type === "Note")!.widgets_values).toEqual(["hello"]); + expect(OPAQUE_WIDGETS_KEY).toBeTruthy(); + }); + + it("accepts nodes without optional links, groups, or definitions", () => { + const doc = mint(baseWorkflow(), catalog); + const op = insertOp({ nodes: [{ id: 1, type: "Src", title: "minimal" }] }); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + expect(project(doc, catalog).nodes!.find((node) => node.title === "minimal")?.id).toEqual(expect.stringContaining(op.op_id)); + }); + + it("merges groups deterministically without dropping existing groups", () => { + const seeded = { ...baseWorkflow(), groups: [{ title: "Base" }] } as WorkflowJSON; + const doc = mint(seeded, catalog); + applyOps(doc, [insertOp({ nodes: [], links: [], groups: [{ title: "Inserted" }] })], catalog); + + expect(project(doc, catalog).groups).toEqual([{ title: "Base" }, { title: "Inserted" }]); + }); + + it("scrubs private __ keys recursively from every inserted payload branch", () => { + const doc = mint(baseWorkflow(), catalog); + applyOps(doc, [insertOp({ + nodes: [{ id: 100, type: "Src", properties: { visible: true, __definition_digest: "node-secret" } }], + links: [], + groups: [{ title: "G", nested: { __private: "group-secret", visible: true } }], + definitions: { + subgraphs: [{ id: "def-2", nodes: [], links: [], metadata: { __definition_digest: "def-secret", visible: true } }], + }, + })], catalog); + + const json = JSON.stringify(project(doc, catalog)); + expect(json).not.toContain("__definition_digest"); + expect(json).not.toContain("__private"); + expect(json).toContain('"visible":true'); + }); + + it("makes colliding concurrent inserts converge in either legal arrival order", () => { + const seed = Y.encodeStateAsUpdate(mint(baseWorkflow(), catalog)); + const run = (ops: Op[]): WorkflowJSON => { + const doc = new Y.Doc(); + Y.applyUpdate(doc, seed); + for (const op of ops) applyOps(doc, [op], catalog); + return project(doc, catalog); + }; + const lower = insertOp( + { nodes: [{ id: 100, type: "Src", title: "lower" }, { id: 101, type: "Sink" }], links: [[200, 100, 0, 101, 0, "lower"]] }, + { op_id: "a".repeat(32), stamp: [2, "a"] }, + ); + const higher = insertOp( + { nodes: [{ id: 100, type: "Src", title: "higher" }, { id: 101, type: "Sink" }], links: [[200, 100, 0, 101, 0, "higher"]] }, + { op_id: "b".repeat(32), stamp: [3, "b"] }, + ); + + expect(run([lower, higher])).toEqual(run([higher, lower])); + expect(run([lower, higher]).nodes!.filter((node) => node.title === "lower" || node.title === "higher")).toHaveLength(2); + expect(run([lower, higher]).links!.filter((link) => ["lower", "higher"].includes((link as unknown[])[5] as string))).toHaveLength(2); + }); + + it("keeps valid inserted content when a carried link endpoint was concurrently deleted", () => { + const seed = Y.encodeStateAsUpdate(mint(baseWorkflow(), catalog)); + const insert = insertOp({ + nodes: [{ id: 100, type: "Src", title: "inserted source" }, { id: 101, type: "Sink", title: "inserted sink" }], + links: [[200, 100, 0, 101, 0, "valid"], [201, 100, 0, 1, 0, "missing endpoint"]], + groups: [{ title: "Inserted group" }], + definitions: { subgraphs: [{ id: "inserted-def", nodes: [], links: [] }] }, + }); + const remove = { + ...env(), op: "delete_node", node_id: 1, removed_links: [], stamp: [2, "b"], + } as unknown as Op; + const run = (ops: Op[]): WorkflowJSON => { + const doc = new Y.Doc(); + Y.applyUpdate(doc, seed); + for (const op of ops) applyOps(doc, [op], catalog); + return project(doc, catalog); + }; + + const deleteThenInsert = run([remove, insert]); + const insertThenDelete = run([insert, remove]); + expect(deleteThenInsert).toEqual(insertThenDelete); + expect(deleteThenInsert.nodes!.filter((node) => typeof node.title === "string" && node.title.startsWith("inserted"))).toHaveLength(2); + expect(deleteThenInsert.links!.filter((link) => (link as unknown[])[5] === "valid")).toHaveLength(1); + expect(deleteThenInsert.links!.some((link) => (link as unknown[])[5] === "missing endpoint")).toBe(false); + expect(deleteThenInsert.groups).toContainEqual({ title: "Inserted group" }); + expect(defIds(deleteThenInsert).some((id) => id !== "def-1" && /^[0-9a-f-]{36}$/.test(id))).toBe(true); + }); + + it("drops both boundary directions and retains only the sibling link and its exact slot references", () => { + const opId = "0123456789abcdef0123456789abcdef"; + const siblingLinkId = "insert:0123456789abcdef0123456789abcdef:root:link:903"; + const sourceNodeId = "insert:0123456789abcdef0123456789abcdef:root:node:701"; + const sinkNodeId = "insert:0123456789abcdef0123456789abcdef:root:node:702"; + const doc = mint({ nodes: [], links: [] }, catalog); + const op = insertOp( + { + nodes: [ + { id: 701, type: "Src", title: "boundary source", outputs: [{ name: "out", links: [901, 903] }] }, + { id: 702, type: "Sink", title: "boundary sink", inputs: [{ name: "in", link: 902 }, { name: "sibling", link: 903 }] }, + ], + links: [ + [901, 701, 0, 999, 0, "inserted-to-external"], + [902, 999, 0, 702, 0, "external-to-inserted"], + [903, 701, 0, 702, 1, "sibling"], + ], + }, + { op_id: opId }, + ); + seq--; + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const workflow = project(doc, catalog); + expect(workflow.links).toEqual([[siblingLinkId, sourceNodeId, 0, sinkNodeId, 1, "sibling"]]); + expect(workflow.nodes).toEqual([ + expect.objectContaining({ id: sourceNodeId, outputs: [{ name: "out", links: [siblingLinkId] }] }), + expect.objectContaining({ + id: sinkNodeId, + inputs: [{ name: "in", link: null }, { name: "sibling", link: siblingLinkId }], + }), + ]); + }); + + it("rejects a remapped definition id already present anywhere in the stored tree", () => { + const op = insertOp(template()); + const remapped = remapInsertedWorkflowIds(template(), op.op_id); + const remappedId = defIds(remapped)[0]!; + const base = baseWorkflow(); + base.definitions!.subgraphs!.push({ id: remappedId, nodes: [], links: [] }); + const doc = mint(base, catalog); + const before = bytes(doc); + + const result = applyOps(doc, [op], catalog); + + expect(rejectedOutcomeWithIndex(result)).toMatchObject({ + index: 0, + code: "definition_conflict", + }); + expect(bytes(doc).equals(before)).toBe(true); + }); + + it("stores nested definitions as addressable maps for later interior set_widget", () => { + const doc = mint(baseWorkflow(), catalog); + const nested = { id: "nested-def", nodes: [{ id: 9, type: "Inner", widgets_values: ["before"] }], links: [] }; + const outer = { + id: "outer-def", name: "Outer", nodes: [{ id: 8, type: "nested-def" }], links: [], definitions: { subgraphs: [nested] }, + }; + applyOps(doc, [insertOp({ nodes: [{ id: 100, type: "outer-def" }], links: [], definitions: { subgraphs: [outer] } })], catalog); + const projected = project(doc, catalog) as { nodes: Array<{ id: string; type: string }>; definitions: { subgraphs: Array> } }; + const projectedOuter = projected.definitions.subgraphs.find((definition) => definition["name"] === "Outer") ?? projected.definitions.subgraphs.find((definition) => definition["id"] !== "def-1")!; + const projectedNested = (projectedOuter["definitions"] as { subgraphs: Array> }).subgraphs[0]!; + const outerNode = (projectedOuter["nodes"] as Array<{ id: string }>)[0]!; + const innerNode = (projectedNested["nodes"] as Array<{ id: string }>)[0]!; + const host = projected.nodes.find((node) => node.type === projectedOuter["id"])!; + const edit: Op = { ...env(), op: "set_widget", node_id: host.id, path: [host.id, outerNode.id, innerNode.id], inner_widget: "text", widget: "text", value: "after" }; + + expect(applyOps(doc, [edit], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + const defs = (project(doc, catalog) as { definitions: { subgraphs: Array> } }).definitions.subgraphs; + const updatedOuter = defs.find((definition) => definition.id === projectedOuter["id"]) as { + definitions: { subgraphs: Array<{ nodes: Array<{ widgets_values: unknown[] }> }> }; + }; + expect(updatedOuter.definitions.subgraphs[0]!.nodes[0]!.widgets_values).toEqual(["after"]); + }); +}); + +describe("insert_workflow: rejection (KA-4 byte identity, op_id absent from applied)", () => { + const cases: { name: string; workflow: unknown; code: string; withCatalog: boolean }[] = [ + { + name: "exact duplicate raw node ids", + workflow: { nodes: [{ id: 100, type: "Src" }, { id: 100, type: "Src" }], links: [] }, + code: "malformed_op", + withCatalog: true, + }, + { + name: "normalized duplicate raw node ids", + workflow: { nodes: [{ id: 1, type: "Src" }, { id: "1", type: "Src" }], links: [] }, + code: "malformed_op", + withCatalog: true, + }, + { + name: "missing raw node id", + workflow: { nodes: [{ type: "Src" }], links: [] }, + code: "malformed_op", + withCatalog: true, + }, + { + name: "exact duplicate raw link ids", + workflow: { nodes: [], links: [[7, 1, 0, 2, 0, "X"], [7, 1, 0, 2, 0, "X"]] }, + code: "malformed_op", + withCatalog: true, + }, + { + name: "normalized duplicate raw link ids", + workflow: { nodes: [], links: [[1, 1, 0, 2, 0, "X"], ["1", 1, 0, 2, 0, "X"]] }, + code: "malformed_op", + withCatalog: true, + }, + { + name: "missing raw link id", + workflow: { nodes: [], links: [[undefined, 1, 0, 2, 0, "X"]] }, + code: "malformed_op", + withCatalog: true, + }, + { + name: "definition normalized duplicate raw node ids", + workflow: { + nodes: [], + links: [], + definitions: { subgraphs: [{ id: "outer", nodes: [{ id: 1 }, { id: "1" }], links: [] }] }, + }, + code: "malformed_op", + withCatalog: true, + }, + { + name: "definition missing raw node id", + workflow: { + nodes: [], + links: [], + definitions: { subgraphs: [{ id: "outer", nodes: [{ type: "Inner" }], links: [] }] }, + }, + code: "malformed_op", + withCatalog: true, + }, + { + name: "definition normalized duplicate raw link ids", + workflow: { + nodes: [], + links: [], + definitions: { + subgraphs: [ + { + id: "outer", + nodes: [{ id: 10, type: "Inner" }, { id: 11, type: "Inner" }], + links: [[1, 10, 0, 11, 0, "X"], ["1", 10, 0, 11, 0, "X"]], + }, + ], + }, + }, + code: "malformed_op", + withCatalog: true, + }, + { + name: "definition missing raw link id", + workflow: { + nodes: [], + links: [], + definitions: { + subgraphs: [ + { + id: "outer", + nodes: [{ id: 10, type: "Inner" }, { id: 11, type: "Inner" }], + links: [[undefined, 10, 0, 11, 0, "X"]], + }, + ], + }, + }, + code: "malformed_op", + withCatalog: true, + }, + { + name: "nested definition normalized duplicate raw node ids", + workflow: { + nodes: [], + links: [], + definitions: { + subgraphs: [ + { + id: "outer", + nodes: [], + links: [], + definitions: { subgraphs: [{ id: "inner", nodes: [{ id: 1 }, { id: "1" }], links: [] }] }, + }, + ], + }, + }, + code: "malformed_op", + withCatalog: true, + }, + { name: "missing workflow", workflow: undefined, code: "malformed_op", withCatalog: true }, + { name: "non-object workflow", workflow: "nope", code: "malformed_op", withCatalog: true }, + { name: "nodes is not an array", workflow: { nodes: {}, links: [] }, code: "malformed_op", withCatalog: true }, + { + name: "node without type", + workflow: { nodes: [{ id: 100, pos: [0, 0] }], links: [] }, + code: "invalid_node_payload", + withCatalog: true, + }, + { + name: "positional widgets with no catalog", + workflow: { nodes: [{ id: 100, type: "KSampler", widgets_values: [1, 2] }], links: [] }, + code: "catalog_required", + withCatalog: false, + }, + ]; + + for (const c of cases) { + it(`rejects: ${c.name} → ${c.code}`, () => { + const doc = mint(baseWorkflow(), catalog); + const before = bytes(doc); + const projectedBefore = project(doc, catalog); + const op = insertOp(c.workflow); + const result = applyOps(doc, [op], c.withCatalog ? catalog : undefined); + expect(rejectedOutcomeWithIndex(result)!.code).toBe(c.code); + expect(bytes(doc).equals(before)).toBe(true); + expect(project(doc, catalog)).toEqual(projectedBefore); + expect(appliedMap(doc).has(op.op_id)).toBe(false); + expect(applyOps(doc, [op], c.withCatalog ? catalog : undefined)).toEqual(result); + expect(bytes(doc).equals(before)).toBe(true); + }); + } + + it("remaps raw ids that collide with live ids without touching incumbents", () => { + const doc = mint(baseWorkflow(), catalog); + const before = bytes(doc); + const wf = { nodes: [{ id: 100, type: "Src" }, { id: 101, type: "Src" }, { id: 4, type: "Src" }], links: [] }; + expect(applyOps(doc, [insertOp(wf)], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + expect(bytes(doc).equals(before)).toBe(false); + expect(project(doc, catalog).nodes!.find((node) => node.id === 4)?.type).toBe("KSampler"); + expect(ids(project(doc, catalog))).toHaveLength(8); + }); + + const occupiedIdVectors = [ + { + kind: "node", + code: "node_id_collision", + seed: { + nodes: [{ id: "insert:c641df0c31f9440b9385ac8e01e099b2:root:node:701", type: "Src", title: "incumbent node" }], + links: [], + }, + }, + { + kind: "link", + code: "link_id_collision", + seed: { + nodes: [{ id: 1, type: "Src" }, { id: 2, type: "Sink" }], + links: [["insert:c641df0c31f9440b9385ac8e01e099b2:root:link:901", 1, 0, 2, 0, "incumbent link"]], + }, + }, + { + kind: "definition", + code: "definition_conflict", + seed: { + nodes: [], + links: [], + definitions: { + subgraphs: [{ id: "c7d70ff1-6dca-4b80-83ee-351b8510f528", name: "incumbent definition", nodes: [], links: [] }], + }, + }, + }, + ] as const; + + for (const vector of occupiedIdVectors) { + it(`refuses an independently occupied derived ${vector.kind} id byte-identically`, () => { + // Fixed values were independently produced with Node 22.22.2 node:crypto: + // node --input-type=module -e 'import{createHash}from"node:crypto";const op="c641df0c31f9440b9385ac8e01e099b2",raw="def-asym",s=`insert:${op}:root:definition:${encodeURIComponent(JSON.stringify(raw))}`,h=createHash("sha256").update(s).digest("hex");console.log(`insert:${op}:root:node:701`,`insert:${op}:root:link:901`,`${h.slice(0,8)}-${h.slice(8,12)}-4${h.slice(13,16)}-8${h.slice(17,20)}-${h.slice(20,32)}`)' + const doc = mint(vector.seed as unknown as WorkflowJSON, catalog); + const beforeBytes = bytes(doc); + const incumbent = project(doc, catalog); + const op = insertOp( + { + nodes: [{ id: 701, type: "def-asym" }, { id: 702, type: "Sink" }], + links: [[901, 701, 0, 702, 0, "candidate link"]], + definitions: { subgraphs: [{ id: "def-asym", name: "candidate definition", nodes: [], links: [] }] }, + }, + { op_id: "c641df0c31f9440b9385ac8e01e099b2" }, + ); + seq--; + + expect(rejectedOutcomeWithIndex(applyOps(doc, [op], catalog))).toMatchObject({ index: 0, code: vector.code }); + expect(bytes(doc).equals(beforeBytes)).toBe(true); + expect(project(doc, catalog)).toEqual(incumbent); + expect(appliedMap(doc).has(op.op_id)).toBe(false); + }); + } + + it("routes dangling link endpoints through the existing unknown-node no-op path", () => { + const doc = mint(baseWorkflow(), catalog); + const op = insertOp({ nodes: [], links: [[200, 100, 0, 999, 0, "X"]] }); + const result = applyOps(doc, [op], catalog); + + expect(result.outcomes[0]).toMatchObject({ outcome: "no-op" }); + expect(appliedMap(doc).has(op.op_id)).toBe(true); + }); + + it("rejects an id-less nested subgraph atomically", () => { + const doc = mint(baseWorkflow(), catalog); + const before = bytes(doc); + const op = insertOp({ + nodes: [{ id: 100, type: "Src" }], + links: [], + definitions: { + subgraphs: [{ id: "outer", nodes: [], links: [], definitions: { subgraphs: [{ nodes: [], links: [] }] } }], + }, + }); + + expect(rejectedOutcomeWithIndex(applyOps(doc, [op], catalog))!.code).toBe("malformed_op"); + expect(bytes(doc).equals(before)).toBe(true); + expect(appliedMap(doc).has(op.op_id)).toBe(false); + }); + + it("remaps a nested definition id already used in the live tree", () => { + const doc = mint(baseWorkflow(), catalog); + const op = insertOp({ + nodes: [{ id: 100, type: "outer" }], links: [], + definitions: { + subgraphs: [{ id: "outer", nodes: [], links: [], definitions: { subgraphs: [{ id: "def-1", nodes: [], links: [] }] } }], + }, + }); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + }); + + it("inserts same-raw-id definitions identically in either arrival order", () => { + const seed = Y.encodeStateAsUpdate(mint(baseWorkflow(), catalog)); + const makeDoc = (): Y.Doc => { + const doc = new Y.Doc(); + Y.applyUpdate(doc, seed); + return doc; + }; + const makeConflict = (name: string, nodeId: number, opId: string): Op => + insertOp( + { + nodes: [{ id: nodeId, type: "def-1" }], + links: [], + definitions: { subgraphs: [{ id: "def-1", name, nodes: [], links: [] }] }, + }, + { op_id: opId }, + ); + const first = makeConflict("A", 100, "a".repeat(32)); + const second = makeConflict("B", 101, "b".repeat(32)); + const docA = makeDoc(); + const docB = makeDoc(); + + expect(applyOps(docA, [first], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + expect(applyOps(docA, [second], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + expect(applyOps(docB, [second], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + expect(applyOps(docB, [first], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + expect(project(docA, catalog)).toEqual(project(docB, catalog)); + expect(defIds(project(docA, catalog))).toHaveLength(3); + }); + + it("never overwrites an occupied legacy fork-shaped id", () => { + const wf = baseWorkflow() as WorkflowJSON & { definitions: { subgraphs: Record[] } }; + wf.definitions.subgraphs.push({ id: "def-1-deadbeef", name: "Occupied", nodes: [], links: [] }); + const doc = mint(wf, catalog); + const before = bytes(doc); + const op = insertOp({ + nodes: [{ id: 100, type: "def-1" }], + links: [], + definitions: { subgraphs: [{ id: "def-1", name: "Different", nodes: [], links: [] }] }, + }); + + expect(applyOps(doc, [op], catalog).outcomes[0]).toMatchObject({ outcome: "applied" }); + expect(bytes(doc).equals(before)).toBe(false); + const definitions = (project(doc, catalog) as { definitions: { subgraphs: { id: string; name: string }[] } }).definitions.subgraphs; + expect(definitions.find((definition) => definition.id === "def-1-deadbeef")!.name).toBe("Occupied"); + }); +}); diff --git a/test/interior-connect.regression.test.ts b/test/interior-connect.regression.test.ts new file mode 100644 index 00000000..ad04dcee --- /dev/null +++ b/test/interior-connect.regression.test.ts @@ -0,0 +1,485 @@ +import { describe, expect, it } from "vitest"; +import * as Y from "yjs"; +import { + applyOps, + mint, + project, + type ConnectOp, + type DeleteNodeOp, + type WidgetCatalog, + type WorkflowJSON, +} from "../src/index.js"; +import { appliedMap } from "../src/doc.js"; + +const catalog: WidgetCatalog = { + types: { + Source: { widget_order: [] }, + Sink: { widget_order: [] }, + }, +}; + +const workflow = { + nodes: [{ id: 100, type: "definition-1", inputs: [], outputs: [] }], + links: [], + definitions: { + subgraphs: [ + { + id: "definition-1", + nodes: [ + { + id: 1, + type: "Source", + inputs: [], + outputs: [{ name: "text", type: "STRING", links: null }], + }, + { + id: 2, + type: "Sink", + inputs: [{ name: "text", type: "STRING", link: null }], + outputs: [], + }, + ], + links: [], + }, + ], + }, +} as unknown as WorkflowJSON; + +function connect(overrides: Partial = {}): ConnectOp { + return { + op: "connect", + op_id: "interiorconnect0000000000000001", + actor: "human:a", + base_version: 1, + stamp: [1, "human:a"], + path: ["100"], + link_id: 41, + from_node: 1, + from_slot: 0, + to_node: 2, + to_slot: 0, + link_type: "STRING", + ...overrides, + } as ConnectOp; +} + +function remove(node_id = 100): DeleteNodeOp { + return { + op: "delete_node", + op_id: `delete${node_id}0000000000000000000000`, + actor: "human:b", + base_version: 2, + stamp: [2, "human:b"], + node_id, + removed_links: [], + }; +} + +function definitionOf(doc: Y.Doc) { + return (project(doc, catalog).definitions as { + subgraphs: Array<{ + nodes: Array<{ + id: number; + inputs?: Array<{ link: number | null }>; + outputs?: Array<{ links: number[] | null }>; + }>; + links: Array<{ id: number }>; + }>; + }).subgraphs[0]!; +} + +describe("interior connect regression", () => { + it("regression: retained host connect projects identically in both connect/delete orders", () => { + const snapshot = Y.encodeStateAsUpdate(mint(workflow, catalog)); + const projections = [[connect(), remove()], [remove(), connect()]].map((order) => { + const doc = new Y.Doc(); + Y.applyUpdate(doc, snapshot); + const outcomes = order.flatMap((op) => applyOps(doc, [op], catalog).outcomes); + expect(outcomes.map(({ outcome }) => outcome)).toEqual(["applied", "applied"]); + expect(project(doc, catalog).nodes).toEqual([]); + const definition = definitionOf(doc); + expect(definition.links).toEqual([{ id: 41, origin_id: 1, origin_slot: 0, target_id: 2, target_slot: 0, type: "STRING" }]); + expect(definition.nodes.find(({ id }) => id === 1)?.outputs?.[0]?.links).toEqual([41]); + expect(definition.nodes.find(({ id }) => id === 2)?.inputs?.[0]?.link).toBe(41); + return definition; + }); + expect(projections[0]).toEqual(projections[1]); + }); + + it("regression: retained shared definitions reject without mutation in both orders", () => { + const shared = structuredClone(workflow) as unknown as WorkflowJSON; + shared.nodes.push({ id: 101, type: "definition-1", inputs: [], outputs: [] }); + const snapshot = Y.encodeStateAsUpdate(mint(shared, catalog)); + const projections = [[connect(), remove()], [remove(), connect()]].map((order) => { + const doc = new Y.Doc(); + Y.applyUpdate(doc, snapshot); + for (const op of order) { + const before = Y.encodeStateAsUpdate(doc); + const outcome = applyOps(doc, [op], catalog).outcomes[0]!; + if (op.op === "connect") { + expect(outcome).toMatchObject({ outcome: "rejected", reason: { code: "shared_definition_unforked" } }); + expect(Y.encodeStateAsUpdate(doc)).toEqual(before); + expect(appliedMap(doc).has(op.op_id)).toBe(false); + } else { + expect(outcome.outcome).toBe("applied"); + } + } + return definitionOf(doc); + }); + expect(projections[0]).toEqual(projections[1]); + }); + + it("regression: retained outer route supports a nested instance host path", () => { + const nested = structuredClone(workflow) as unknown as WorkflowJSON & { + definitions: { subgraphs: Array> }; + }; + const leafNodes = nested.definitions.subgraphs[0]!.nodes; + nested.definitions.subgraphs[0]!.nodes = [{ id: 10, type: "definition-2", inputs: [], outputs: [] }]; + nested.definitions.subgraphs.push({ + id: "definition-2", + nodes: leafNodes, + links: [], + }); + const doc = mint(nested, catalog); + expect(applyOps(doc, [remove()], catalog).outcomes[0]?.outcome).toBe("applied"); + const op = connect({ path: [100, "10"] }); + expect(applyOps(doc, [op], catalog).outcomes).toEqual([{ op_id: op.op_id, outcome: "applied" }]); + const definitions = (project(doc, catalog).definitions as { subgraphs: Array<{ id: string; links: unknown[] }> }).subgraphs; + expect(definitions.find(({ id }) => id === "definition-2")?.links).toEqual([ + { id: 41, origin_id: 1, origin_slot: 0, target_id: 2, target_slot: 0, type: "STRING" }, + ]); + }); + + it("regression: direct definition connect remains an unsupported no-op", () => { + const doc = mint(workflow, catalog); + expect(applyOps(doc, [remove()], catalog).outcomes[0]?.outcome).toBe("applied"); + const op = connect({ path: ["definition-1"] }); + const before = definitionOf(doc); + expect(applyOps(doc, [op], catalog).outcomes).toEqual([{ op_id: op.op_id, outcome: "no-op" }]); + expect(definitionOf(doc)).toEqual(before); + }); + + it("normalizes numeric and string instance/link aliases across both arrival orders", () => { + const low = connect({ path: [100], link_id: 41 }); + const high = connect({ op_id: "alias-high", actor: "human:b", base_version: 2, stamp: [2, "human:b"], path: ["100"], link_id: "41" }); + const projections = [[low, high], [high, low]].map((order) => { + const doc = mint(workflow, catalog); + for (const op of order) expect(applyOps(doc, [op], catalog).outcomes[0]?.outcome).not.toBe("rejected"); + return definitionOf(doc); + }); + expect(projections[0]).toEqual(projections[1]); + expect(projections[0]!.links).toHaveLength(1); + }); + + it("applies and projects a concrete link inside one subgraph definition", () => { + const doc = mint(workflow, catalog); + const op = connect(); + + expect(applyOps(doc, [op], catalog).outcomes).toEqual([ + { op_id: op.op_id, outcome: "applied" }, + ]); + + const definitions = project(doc, catalog).definitions as { + subgraphs: Array<{ + nodes: Array<{ + id: number; + inputs?: Array<{ link: number | null }>; + outputs?: Array<{ links: number[] | null }>; + }>; + links: unknown[]; + }>; + }; + const definition = definitions.subgraphs[0]!; + + expect(definition.links).toEqual([ + { + id: 41, + origin_id: 1, + origin_slot: 0, + target_id: 2, + target_slot: 0, + type: "STRING", + }, + ]); + expect(definition.nodes.find(({ id }) => id === 1)?.outputs?.[0]?.links).toEqual([41]); + expect(definition.nodes.find(({ id }) => id === 2)?.inputs?.[0]?.link).toBe(41); + }); + + it.each([ + ["empty path", { path: [] }], + [ + "interior autogrow", + { + grow: { name: "text", type: "STRING", grow_id: "grow-1" }, + to_slot: undefined, + }, + ], + ])("rejects %s before changing document bytes or consuming op_id", (_name, invalid) => { + const doc = mint(workflow, catalog); + const op = connect(invalid as unknown as Partial); + const before = Y.encodeStateAsUpdate(doc); + + const result = applyOps(doc, [op], catalog); + + expect(result.outcomes).toEqual([ + { + op_id: op.op_id, + outcome: "rejected", + reason: { code: "malformed_op", message: expect.any(String) }, + }, + ]); + expect(Y.encodeStateAsUpdate(doc)).toEqual(before); + expect(appliedMap(doc).has(op.op_id)).toBe(false); + }); + + it("refuses an interior write when two instances share the definition", () => { + const shared = structuredClone(workflow) as unknown as WorkflowJSON; + shared.nodes.push({ id: 101, type: "definition-1", inputs: [], outputs: [] }); + const doc = mint(shared, catalog); + const op = connect(); + const before = Y.encodeStateAsUpdate(doc); + + const result = applyOps(doc, [op], catalog); + + expect(result.outcomes).toEqual([ + { + op_id: op.op_id, + outcome: "rejected", + reason: { code: "shared_definition_unforked", message: expect.any(String) }, + }, + ]); + expect(Y.encodeStateAsUpdate(doc)).toEqual(before); + expect(appliedMap(doc).has(op.op_id)).toBe(false); + }); + + it("uses path-scoped input stamps and converges on the higher-stamped link", () => { + const low = connect(); + const high = connect({ + op_id: "interiorconnect0000000000000002", + stamp: [2, "human:b"], + actor: "human:b", + base_version: 2, + link_id: 42, + }); + + for (const order of [[low, high], [high, low]]) { + const doc = mint(workflow, catalog); + applyOps(doc, order, catalog); + const definition = definitionOf(doc); + expect(definition.links.map(({ id }) => id)).toEqual([42]); + expect(definition.nodes.find(({ id }) => id === 1)?.outputs?.[0]?.links).toEqual([42]); + expect(definition.nodes.find(({ id }) => id === 2)?.inputs?.[0]?.link).toBe(42); + } + }); + + it("treats an identical interior-connect retry as a byte-identical no-op", () => { + const doc = mint(workflow, catalog); + const op = connect(); + expect(applyOps(doc, [op], catalog).outcomes[0]?.outcome).toBe("applied"); + const afterFirstApply = Y.encodeStateAsUpdate(doc); + + expect(applyOps(doc, [op], catalog).outcomes).toEqual([ + { op_id: op.op_id, outcome: "no-op" }, + ]); + expect(Y.encodeStateAsUpdate(doc)).toEqual(afterFirstApply); + }); + + // Recovery PR 198 must preserve KA-4 for independent links, not just one contested input. + it("regression: independent interior connects project identically in both arrival orders", () => { + const independent = structuredClone(workflow); + const definitions = independent.definitions as { + subgraphs: Array<{ nodes: WorkflowJSON["nodes"] }>; + }; + definitions.subgraphs[0]!.nodes.push({ + id: 3, + type: "Sink", + inputs: [{ name: "text", type: "STRING", link: null }], + outputs: [], + }); + const seed = mint(independent, catalog); + const snapshot = Y.encodeStateAsUpdate(seed); + const first = connect(); + const second = connect({ + op_id: "independentconnect0000000000002", + stamp: [2, "human:b"], + actor: "human:b", + link_id: 42, + to_node: 3, + }); + const projections = [[first, second], [second, first]].map((order) => { + const doc = new Y.Doc(); + Y.applyUpdate(doc, snapshot); + expect(applyOps(doc, order, catalog).outcomes.map(({ outcome }) => outcome)).toEqual([ + "applied", + "applied", + ]); + const projection = definitionOf(doc); + expect(projection.links.map(({ id }) => id).sort()).toEqual([41, 42]); + expect(projection.nodes.find(({ id }) => id === 2)?.inputs?.[0]?.link).toBe(41); + expect(projection.nodes.find(({ id }) => id === 3)?.inputs?.[0]?.link).toBe(42); + return project(doc, catalog); + }); + expect(projections[0]).toEqual(projections[1]); + }); + + // https://github.com/Comfy-Org/comfy-multi-player/pull/198 + it("regression: a winning same-id rewrite updates addition order in all six permutations", () => { + const independent = structuredClone(workflow); + const definitions = independent.definitions as { + subgraphs: Array<{ nodes: WorkflowJSON["nodes"] }>; + }; + definitions.subgraphs[0]!.nodes.push({ + id: 3, type: "Sink", inputs: [{ name: "text", type: "STRING", link: null }], outputs: [], + }); + const snapshot = Y.encodeStateAsUpdate(mint(independent, catalog)); + const early = connect(); + const middle = connect({ op_id: "middle", stamp: [2, "human:a"], link_id: 42, to_node: 3 }); + const late = connect({ op_id: "late", stamp: [3, "human:a"] }); + const orders = [[early, middle, late], [early, late, middle], [middle, early, late], + [middle, late, early], [late, early, middle], [late, middle, early]]; + expect(orders).toHaveLength(6); + for (const order of orders) { + let doc = new Y.Doc(); + Y.applyUpdate(doc, snapshot); + for (const op of order) { + expect(applyOps(doc, [op], catalog).outcomes[0]?.outcome).not.toBe("rejected"); + // Resume on a new replica after every op: ordering metadata must persist. + const resumed = new Y.Doc(); + Y.applyUpdate(resumed, Y.encodeStateAsUpdate(doc)); + doc.destroy(); + doc = resumed; + } + expect.soft(definitionOf(doc).links.map(({ id }) => id), order.map((op) => op.op_id).join(",")) + .toEqual([42, 41]); + const storedDefinition = doc.getMap>("definitions").get("definition-1"); + const rawOrder = storedDefinition?.get("link_order"); + expect.soft(rawOrder, order.map((op) => op.op_id).join(",")).toEqual(["42", "41"]); + expect.soft((rawOrder as unknown[]).every((entry) => typeof entry === "string")).toBe(true); + } + }); + + it("preserves asymmetric imported order ahead of deterministically ordered additions", () => { + const imported = structuredClone(workflow) as unknown as WorkflowJSON & { + definitions: { subgraphs: Array<{ nodes: WorkflowJSON["nodes"]; links: Array> }> }; + }; + const definition = imported.definitions.subgraphs[0]!; + (definition.nodes[0]!.outputs as Array<{ links: number[] | null }>)[0]!.links = [90, 7]; + (definition.nodes[1]!.inputs as Array<{ link: number | null }>)[0]!.link = 90; + definition.nodes.push( + { + id: 3, + type: "Sink", + inputs: [{ name: "text", type: "STRING", link: 7 }], + outputs: [], + }, + { + id: 4, + type: "Sink", + inputs: [{ name: "text", type: "STRING", link: null }], + outputs: [], + }, + { + id: 5, + type: "Sink", + inputs: [{ name: "text", type: "STRING", link: null }], + outputs: [], + }, + ); + definition.links = [ + { id: 90, origin_id: 1, origin_slot: 0, target_id: 2, target_slot: 0, type: "STRING" }, + { id: 7, origin_id: 1, origin_slot: 0, target_id: 3, target_slot: 0, type: "STRING" }, + ]; + const snapshot = Y.encodeStateAsUpdate(mint(imported, catalog)); + const first = connect({ to_node: 4, link_id: 41 }); + const second = connect({ + op_id: "independentconnect0000000000002", + stamp: [2, "human:b"], + actor: "human:b", + base_version: 2, + to_node: 5, + link_id: 42, + }); + + for (const order of [[first, second], [second, first]]) { + const doc = new Y.Doc(); + Y.applyUpdate(doc, snapshot); + applyOps(doc, order, catalog); + expect(definitionOf(doc).links.map(({ id }) => id)).toEqual([90, 7, 41, 42]); + } + }); + + it("removes a replaced imported link without disturbing surviving imported order", () => { + const imported = structuredClone(workflow) as unknown as WorkflowJSON & { + definitions: { subgraphs: Array<{ nodes: WorkflowJSON["nodes"]; links: Array> }> }; + }; + const definition = imported.definitions.subgraphs[0]!; + (definition.nodes[0]!.outputs as Array<{ links: number[] | null }>)[0]!.links = [90, 7]; + (definition.nodes[1]!.inputs as Array<{ link: number | null }>)[0]!.link = 90; + definition.nodes.push({ + id: 3, + type: "Sink", + inputs: [{ name: "text", type: "STRING", link: 7 }], + outputs: [], + }); + definition.links = [ + { id: 90, origin_id: 1, origin_slot: 0, target_id: 2, target_slot: 0, type: "STRING" }, + { id: 7, origin_id: 1, origin_slot: 0, target_id: 3, target_slot: 0, type: "STRING" }, + ]; + const doc = mint(imported, catalog); + + expect(applyOps(doc, [connect()], catalog).outcomes[0]?.outcome).toBe("applied"); + + expect(definitionOf(doc).links.map(({ id }) => id)).toEqual([7, 41]); + expect(definitionOf(doc).nodes.find(({ id }) => id === 1)?.outputs?.[0]?.links).toEqual([7, 41]); + }); + + // https://github.com/Comfy-Org/comfy-multi-player/pull/198 + it("regression: prototype-named imported link ids retain their original order", () => { + const imported = { + ...workflow, + definitions: { + subgraphs: [{ + id: "definition-1", + nodes: [], + links: [ + { id: "constructor", origin_id: 1, origin_slot: 0, target_id: 2, target_slot: 0, type: "STRING" }, + { id: "ordinary", origin_id: 1, origin_slot: 0, target_id: 3, target_slot: 0, type: "STRING" }, + ], + }], + }, + } as unknown as WorkflowJSON; + expect(definitionOf(mint(imported, catalog)).links.map(({ id }) => id)) + .toEqual(["constructor", "ordinary"]); + }); + + it("keeps top-level and interior bookkeeping distinct when link ids collide", () => { + const withRootGraph = structuredClone(workflow) as unknown as WorkflowJSON; + withRootGraph.nodes.push( + { + id: 1, + type: "Source", + inputs: [], + outputs: [{ name: "text", type: "STRING", links: null }], + }, + { + id: 2, + type: "Sink", + inputs: [{ name: "text", type: "STRING", link: null }], + outputs: [], + }, + ); + const doc = mint(withRootGraph, catalog); + const root = connect({ op_id: "rootconnect000000000000000001" }); + delete root.path; + const interior = connect({ op_id: "interiorconnect0000000000000003", stamp: [2, "human:a"] }); + + expect(applyOps(doc, [root, interior], catalog).outcomes.map(({ outcome }) => outcome)).toEqual([ + "applied", + "applied", + ]); + + const projected = project(doc, catalog); + expect(projected.links).toEqual([[41, 1, 0, 2, 0, "STRING"]]); + expect(definitionOf(doc).links.map(({ id }) => id)).toEqual([41]); + }); +}); diff --git a/test/ka4-rejection-byte-identity.test.ts b/test/ka4-rejection-byte-identity.test.ts index 309a3bfa..aee050a3 100644 --- a/test/ka4-rejection-byte-identity.test.ts +++ b/test/ka4-rejection-byte-identity.test.ts @@ -49,6 +49,7 @@ import { type WorkflowJSON, } from "../src/index.js"; import { appliedMap } from "../src/doc.js"; +import { remapInsertedWorkflowIds } from "../src/remap.js"; const catalog: WidgetCatalog = { types: { @@ -108,6 +109,7 @@ interface Row { /** The rejected outcome code the applier must return. */ code: string; build: () => Op; + seed?: (op: Op) => WorkflowJSON; } const CASES: Row[] = [ @@ -187,6 +189,60 @@ const CASES: Row[] = [ }) as unknown as Op, }, + // ---- insert_workflow ---------------------------------------------------- + { + kind: "insert_workflow", + why: "duplicate raw node ids are ambiguous before deterministic remapping", + code: "malformed_op", + build: () => ({ op: "insert_workflow", ...env(), workflow: { nodes: [{ id: 1, type: "Src" }, { id: 1, type: "Src" }] } }) as Op, + }, + { + kind: "insert_workflow", + why: "duplicate raw link ids are ambiguous before deterministic remapping", + code: "malformed_op", + build: () => ({ op: "insert_workflow", ...env(), workflow: { nodes: [], links: [[7, 2, 0, 3, 0, "X"], [7, 2, 0, 3, 0, "X"]] } }) as Op, + }, + { + kind: "insert_workflow", + why: "remapped node id collides with the live tree", + code: "node_id_collision", + build: () => ({ op: "insert_workflow", ...env(), workflow: { nodes: [{ id: 1, type: "Src" }], links: [] } }) as Op, + seed: (op) => { + const workflow = baseWorkflow(); + const remapped = remapInsertedWorkflowIds((op as { workflow: WorkflowJSON }).workflow, op.op_id); + workflow.nodes!.push({ id: remapped.nodes![0]!.id, type: "Src" }); + return workflow; + }, + }, + { + kind: "insert_workflow", + why: "remapped link id collides with the live tree", + code: "link_id_collision", + build: () => + ({ + op: "insert_workflow", + ...env(), + workflow: { nodes: [{ id: 2, type: "Src" }, { id: 3, type: "Sink" }], links: [[7, 2, 0, 3, 0, "X"]] }, + }) as Op, + seed: (op) => { + const workflow = baseWorkflow(); + const remapped = remapInsertedWorkflowIds((op as { workflow: WorkflowJSON }).workflow, op.op_id); + const [remappedLinkId] = remapped.links![0] as [string, number, number, number, number, string]; + workflow.links!.push([remappedLinkId, 2, 0, 3, 0, "X"]); + return workflow; + }, + }, + { + kind: "insert_workflow", + why: "duplicate remapped definition ids violate define_subgraph tree uniqueness", + code: "malformed_op", + build: () => ({ + op: "insert_workflow", + ...env(), + workflow: { nodes: [], definitions: { subgraphs: [{ id: "d", nodes: [], links: [] }, { id: "d", nodes: [], links: [] }] } }, + }) as Op, + }, + // ---- set_widget ---------------------------------------------------------- { kind: "set_widget", @@ -469,10 +525,10 @@ const KNOWN_KA4_VIOLATIONS: readonly string[] = []; describe("KA-4: a rejected op leaves the doc byte-identical and does not consume its op_id", () => { it.each(CASES.map((c) => [`${c.kind}: ${c.why} → ${c.code}`, c] as const))("%s", (_name, row) => { - const doc = mint(baseWorkflow(), catalog); + const op = row.build(); + const doc = mint(row.seed?.(op) ?? baseWorkflow(), catalog); const before = bytes(doc); const beforeProjection = project(doc, catalog); - const op = row.build(); const res = applyOps(doc, [op], catalog); @@ -592,11 +648,35 @@ function sharedDefinitionCase(): { doc: Y.Doc; op: Op; catalog?: WidgetCatalog; }; } +function definitionConflictCase(): { doc: Y.Doc; op: Op; catalog?: WidgetCatalog; code: string } { + const id = "12345678-1234-4123-8123-123456789abc"; + const nested = "abcdefab-cdef-4abc-8def-abcdefabcdef"; + const doc = mint({ + nodes: [], + links: [], + definitions: { subgraphs: [{ id, name: "outer", nodes: [], links: [], definitions: { + subgraphs: [{ id: nested, name: "nested", nodes: [], links: [] }], + } }] }, + } as unknown as WorkflowJSON, catalog); + return { + doc, + op: { + op: "define_subgraph", + ...env(), + subgraph_id: nested, + subgraph_definition: { id: nested, name: "different", nodes: [], links: [] }, + } as unknown as Op, + catalog, + code: "definition_conflict", + }; +} + const FIXTURE_CASES = [ ["add_node without a catalog", catalogRequiredCase], ["promoted host write without a catalog", promotedCatalogRequiredCase], ["interior set_widget past the positional length", widgetOutOfRangeCase], ["interior set_widget into a definition two nodes instantiate", sharedDefinitionCase], + ["define_subgraph whose root id collides with a nested definition", definitionConflictCase], ] as const; describe("KA-4: the rejection codes that need their own fixture", () => { @@ -633,6 +713,9 @@ const ALL_REJECTION_CODES = [ "op_deferred", "catalog_required", "invalid_node_payload", + "node_id_collision", + "link_id_collision", + "definition_conflict", "unknown_widget", "opaque_widgets", "widget_out_of_range", @@ -642,6 +725,7 @@ const ALL_REJECTION_CODES = [ "not_a_subgraph", "interior_node_not_found", "shared_definition_unforked", + "definition_conflict", ] as const; // These require an already-consumed op_id or an intentionally deep payload; diff --git a/test/link-state-foundation.test.ts b/test/link-state-foundation.test.ts new file mode 100644 index 00000000..593e22f1 --- /dev/null +++ b/test/link-state-foundation.test.ts @@ -0,0 +1,137 @@ +import * as Y from "yjs"; +import { describe, expect, it } from "vitest"; +import { SCHEMA_VERSION, mint, readLinkState, type WorkflowJSON } from "../src/index.js"; +import { initDoc, linkStateMap } from "../src/doc.js"; +import { loadCatalog } from "./helpers.js"; + +const catalog = loadCatalog(); + +function workflow(): WorkflowJSON { + return { + nodes: [ + { id: 1, type: "LoadImage", inputs: [], outputs: [{ name: "IMAGE", type: "IMAGE", links: [10, 11, 12] }], widgets_values: [] }, + { id: 2, type: "PreviewImage", inputs: [{ name: "images", type: "IMAGE", link: 10 }], outputs: [], widgets_values: [] }, + { id: 3, type: "Stored image subgraph", inputs: [{ name: "nested.dynamic.image", type: "IMAGE", link: 11, widget: { name: "image" } }], outputs: [], widgets_values: [] }, + { id: 4, type: "PreviewImage", inputs: [{ name: "images.image7", type: "IMAGE", link: 12, grow_id: 12, widget: { name: "images" }, custom: { retained: true } }], outputs: [], widgets_values: [] }, + ], + links: [ + [10, 1, 0, 2, 0, "IMAGE"], + [11, 1, 0, 3, 0, "IMAGE"], + [12, 1, 0, 4, 0, "IMAGE"], + ], + definitions: { + subgraphs: [{ id: "definition-id", name: "Stored image subgraph", inputs: [{ name: "nested.dynamic.image", type: "IMAGE" }], nodes: [], links: [] }], + }, + }; +} + +describe("schema v3 first-class imported link state", () => { + it("stores complete concrete, full-name promoted, and grown descriptors", () => { + const doc = mint(workflow(), catalog); + expect(SCHEMA_VERSION).toBe(3); + expect(readLinkState(doc)).toEqual({ + "10": { version: 1, authority: "imported", tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "concrete", to_slot: 0, slot: { name: "images", type: "IMAGE", link: 10 } } }, + "11": { version: 1, authority: "imported", tuple: [11, 1, 0, 3, 0, "IMAGE"], destination: { kind: "promoted", to_slot: 0, name: "nested.dynamic.image", slot: { name: "nested.dynamic.image", type: "IMAGE", link: 11, widget: { name: "image" } } } }, + "12": { version: 1, authority: "imported", tuple: [12, 1, 0, 4, 0, "IMAGE"], destination: { kind: "autogrow", to_slot: 0, slot: { name: "images.image7", type: "IMAGE", link: 12, grow_id: 12, widget: { name: "images" }, custom: { retained: true } } } }, + }); + }); + + it("rejects malformed tuples but preserves supported dangling imports without inventing descriptors", () => { + const malformed = workflow(); + malformed.links[0] = [10, 1, 0, 2, 0]; + expect(() => mint(malformed, catalog)).toThrow(/link tuple.*six fields/i); + + const missing = workflow(); + missing.nodes = missing.nodes.filter((node) => node.id !== 2); + const doc = mint(missing, catalog); + expect(readLinkState(doc)["10"]).toBeUndefined(); + expect((doc.getMap("links").get("10") as unknown[])[0]).toBe(10); + }); + + it("removes a coherent descriptor when a later normalized duplicate link id is dangling", () => { + const value = workflow(); + value.links.splice(1, 0, ["10", 1, 0, 999, 0, "IMAGE"]); + + const doc = mint(value, catalog); + + expect(doc.getMap("links").get("10")).toEqual(["10", 1, 0, 999, 0, "IMAGE"]); + expect(readLinkState(doc)["10"]).toBeUndefined(); + }); + + it("builds descriptors against the last node retained for a normalized duplicate node id", () => { + const value = workflow(); + value.nodes.splice(1, 0, { id: "1", type: "LoadImage", inputs: [], outputs: [], widgets_values: [] }); + + const doc = mint(value, catalog); + + expect(doc.getMap("nodes").get("1")).toBeDefined(); + expect(readLinkState(doc)["10"]).toBeUndefined(); + }); + + it.each([ + ["invalid tuple id", (value: WorkflowJSON) => { value.links[0] = [{ bad: true }, 1, 0, 2, 0, "IMAGE"]; }, /link tuple id must be/], + ["invalid endpoint id", (value: WorkflowJSON) => { value.links[0] = [10, null, 0, 2, 0, "IMAGE"]; }, /source node id must be/], + ])("rejects %s", (_label, corrupt, error) => { + const value = workflow(); + corrupt(value); + expect(() => mint(value, catalog)).toThrow(error); + }); + + it.each([ + ["missing source slot", (value: WorkflowJSON) => { value.nodes[0]!.outputs = []; }], + ["missing destination slot", (value: WorkflowJSON) => { value.nodes[1]!.inputs = []; }], + ["source reference mismatch", (value: WorkflowJSON) => { (value.nodes[0]!.outputs![0] as { links: unknown[] }).links = [11, 12]; }], + ["destination reference mismatch", (value: WorkflowJSON) => { (value.nodes[1]!.inputs![0] as { link: unknown }).link = 99; }], + ])("preserves incomplete import with %s and omits its descriptor", (_label, makeIncomplete) => { + const value = workflow(); + makeIncomplete(value); + const doc = mint(value, catalog); + expect(doc.getMap("links").has("10")).toBe(true); + expect(readLinkState(doc)["10"]).toBeUndefined(); + }); + + it("survives snapshot round trips and its read does not create a root", () => { + const encoded = Y.encodeStateAsUpdate(mint(workflow(), catalog)); + const fork = new Y.Doc(); + Y.applyUpdate(fork, encoded); + expect(readLinkState(fork)).toEqual(readLinkState(mint(workflow(), catalog))); + expect(fork.share.has("__link_state")).toBe(true); + + const empty = new Y.Doc(); + expect(readLinkState(empty)).toEqual({}); + expect([...empty.share.keys()]).toEqual([]); + }); + + it("fails closed on an unsupported persisted descriptor", () => { + const doc = mint(workflow(), catalog); + linkStateMap(doc).set("10", { version: 2, authority: "imported", tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "concrete" } }); + expect(() => readLinkState(doc)).toThrow(/unsupported or malformed descriptor/); + linkStateMap(doc).set("10", { version: 1, authority: "imported", tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "future" } }); + expect(() => readLinkState(doc)).toThrow(/unsupported or malformed descriptor/); + }); + + it.each([ + ["map-key/tuple-id mismatch", "10", { version: 1, authority: "imported", tuple: [99, 1, 0, 2, 0, "IMAGE"], destination: { kind: "concrete", to_slot: 0, slot: {} } }], + ["descriptor version", "10", { version: 2, authority: "imported", tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "concrete", to_slot: 0, slot: {} } }], + ["destination kind", "10", { version: 1, authority: "imported", tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "future", to_slot: 0, slot: {} } }], + ["array destination", "10", { version: 1, authority: "imported", tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: [] }], + ["destination tuple slot mismatch", "10", { version: 1, authority: "imported", tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "concrete", to_slot: 1, slot: {} } }], + ["NaN operation stamp", "10", { version: 1, authority: { kind: "operation", stamp: [NaN, "agent:test", "0".repeat(32)] }, tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "concrete", to_slot: 0, slot: { link: 10 } } }], + ["negative operation stamp", "10", { version: 1, authority: { kind: "operation", stamp: [-1, "agent:test", "0".repeat(32)] }, tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "concrete", to_slot: 0, slot: { link: 10 } } }], + ["fractional operation stamp", "10", { version: 1, authority: { kind: "operation", stamp: [1.5, "agent:test", "0".repeat(32)] }, tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "concrete", to_slot: 0, slot: { link: 10 } } }], + ["non-string operation actor", "10", { version: 1, authority: { kind: "operation", stamp: [1, 12, "0".repeat(32)] }, tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "concrete", to_slot: 0, slot: { link: 10 } } }], + ["empty operation id", "10", { version: 1, authority: { kind: "operation", stamp: [1, "agent:test", ""] }, tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "concrete", to_slot: 0, slot: { link: 10 } } }], + ["empty autogrow request", "10", { version: 1, authority: { kind: "operation", stamp: [1, "agent:test", "0".repeat(32)] }, tuple: [10, 1, 0, 2, 0, "IMAGE"], destination: { kind: "autogrow", to_slot: 0, slot: { link: 10 }, request: {} } }], + ])("rejects malformed persisted %s", (_label, key, descriptor) => { + const doc = mint(workflow(), catalog); + linkStateMap(doc).set(key, descriptor); + expect(() => readLinkState(doc)).toThrow(/unsupported or malformed descriptor/); + }); + + it("initDoc creates the first-class root", () => { + const doc = new Y.Doc(); + initDoc(doc); + expect(linkStateMap(doc).size).toBe(0); + expect(doc.share.has("__link_state")).toBe(true); + }); +}); diff --git a/test/permutation/full-op-pool.permutation.test.ts b/test/permutation/full-op-pool.permutation.test.ts index ffe50d6b..32ba9eb1 100644 --- a/test/permutation/full-op-pool.permutation.test.ts +++ b/test/permutation/full-op-pool.permutation.test.ts @@ -2,9 +2,10 @@ * Full declared-op-vocabulary permutation matrix (perm-4). * * The vocabulary is read from FROZEN_OPS and DEFERRED_OPS rather than copied - * into the matrix. Pair coverage is exhaustive across kind pairs, eight - * salient document preconditions, stamp relations, arrival orders, and batch - * boundaries. Longer streams are fixed-seed fast-check samples with shrinking + * into the matrix. Pair coverage is exhaustive across kind pairs and eight + * salient document preconditions, with named representative actor/stamp + * equivalence classes, both arrival orders, and both batch boundaries. + * Longer streams are fixed-seed fast-check samples with shrinking * left enabled. * * Amendment A6 / docs/decisions/EXCEPTIONS.md and schema §2.5 item 2 are the @@ -33,8 +34,13 @@ import { import { canonicalize } from "../helpers.js"; const KINDS = [...FROZEN_OPS, ...DEFERRED_OPS]; +const EXHAUSTIVE_KINDS = [...FROZEN_OPS]; const ACTORS = ["agent:perm4:0", "agent:perm4:1", "human:perm4:0", "human:perm4:1"] as const; -const VERSION_PAIRS = [[0, 0], [0, 1], [1, 0], [1, 1], [0, 9], [9, 0], [4, 4], [4, 5]] as const; +const VERSION_PAIRS = [[0, 0], [0, 1], [1, 0], [9, 0]] as const; +const ACTOR_PAIRS = [ + ["agent:perm4:0", "agent:perm4:1"], + ["agent:perm4:0", "human:perm4:0"], +] as const; const PRECONDITIONS = [ "present-valid", "source-missing", @@ -45,8 +51,10 @@ const PRECONDITIONS = [ "interior-or-inputcount", "promoted-or-autogrow", ] as const; -const PAIR_EXECUTIONS = 150_528; -const SAMPLED_RUNS = 24_736; +// The bounded exhaustive subset remains below the normal-suite case budget; +// additional kind combinations are covered by deterministic sampling below. +const PAIR_EXECUTIONS = 16_384; +const SAMPLED_RUNS = 1_696; const SAMPLED_EXECUTIONS = SAMPLED_RUNS * 2; const TOTAL_EXECUTIONS = PAIR_EXECUTIONS + SAMPLED_EXECUTIONS; const SAMPLE_SEED = 0x4f70504; @@ -215,6 +223,21 @@ function makeOp( return { ...env, op: "delete_node", node_id: side === 0 ? 10 : 20, removed_links: [80, 100, 101] }; case "clear": return { ...env, op: "clear", removed_nodes: side === 0 ? [10, 40, 57] : [20, 40, 57] }; + case "define_subgraph": + const id = "12345678-1234-4123-8123-123456789abc"; + return { + ...env, + op: "define_subgraph", + subgraph_id: id, + subgraph_definition: { + id, + name: `side-${side}`, + nodes: [], + links: [], + }, + }; + case "insert_workflow": + return { ...env, op: "insert_workflow", workflow: { nodes: [node(140 + serial, "Aux", [], [], [value])], links: [] } }; case "reset_doc": return { ...env, op: "reset_doc", workflow: { nodes: [], links: [] } }; } @@ -422,16 +445,12 @@ function classify( throw new Error(`unexpected divergence ${stable({ precondition, mode, ops, left, right })}`); } -function actorPairs(): Array { - return ACTORS.flatMap((left) => ACTORS.filter((right) => right !== left).map((right) => [left, right] as const)); -} - function kindPairs(): Array { - return KINDS.flatMap((left) => KINDS.map((right) => [left, right] as const)); + return EXHAUSTIVE_KINDS.flatMap((left) => EXHAUSTIVE_KINDS.map((right) => [left, right] as const)); } describe("full op-pool permutation equivalence", () => { - it("exhausts every declared op-kind pair across state, stamp, order, and batch dimensions", () => { + it("covers every declared op-kind pair across representative state, stamp, actor, order, and batch dimensions", () => { let executions = 0; let serial = 1; const taxonomy: Taxonomy = { @@ -444,7 +463,7 @@ describe("full op-pool permutation equivalence", () => { for (const kinds of kindPairs()) { for (const precondition of PRECONDITIONS) { - for (const actors of actorPairs()) { + for (const actors of ACTOR_PAIRS) { for (const versions of VERSION_PAIRS) { const pair = kinds.map((kind, side) => makeOp(kind, precondition, side as 0 | 1, serial++, actors[side]!, versions[side]!)); for (const mode of ["together", "split"] as const) { @@ -466,7 +485,7 @@ describe("full op-pool permutation equivalence", () => { } expect(KINDS).toEqual([...FROZEN_OPS, ...DEFERRED_OPS]); - expect(kindPairs()).toHaveLength(KINDS.length ** 2); + expect(kindPairs()).toHaveLength(EXHAUSTIVE_KINDS.length ** 2); expect(executions).toBe(PAIR_EXECUTIONS); expect(Object.values(taxonomy).reduce((sum, count) => sum + count, 0)).toBe(PAIR_EXECUTIONS / 2); // Deferred reset_doc never consumes its op_id; every frozen kind does and @@ -529,7 +548,7 @@ describe("full op-pool permutation equivalence", () => { expect(runs).toBe(SAMPLED_RUNS); expect(executions).toBe(SAMPLED_EXECUTIONS); - expect(TOTAL_EXECUTIONS).toBe(200_000); + expect(TOTAL_EXECUTIONS).toBe(19_776); for (const [kind, count] of Object.entries(hits)) expect(count, `${kind} was not sampled`).toBeGreaterThan(0); expect(Object.values(taxonomy).reduce((sum, count) => sum + count, 0)).toBe(SAMPLED_RUNS); console.info("perm-4 sampled taxonomy", { diff --git a/test/readonly-surface.test.ts b/test/readonly-surface.test.ts index d959aaf2..1bd6a58e 100644 --- a/test/readonly-surface.test.ts +++ b/test/readonly-surface.test.ts @@ -36,6 +36,7 @@ import { project, SchemaVersionError, readGraph, + readLinkState, readMeta, readStamps, type Op, @@ -120,6 +121,7 @@ function fixtureDoc(): Y.Doc { function readSurfaceResults(doc: Y.Doc): [string, unknown][] { return [ ["readGraph(doc)", readGraph(doc)], + ["readLinkState(doc)", readLinkState(doc)], ["readMeta(doc)", readMeta(doc)], ["docCatalogPin(doc)", docCatalogPin(doc)], ["hasNode(doc, id)", hasNode(doc, KSAMPLER_ID)], @@ -183,7 +185,7 @@ describe("read-only surface — it actually reads the document", () => { it("readMeta returns schema/catalog version and the §6 passthrough keys", () => { const meta = readMeta(fixtureDoc()); - expect(meta["schema_version"]).toBe(2); + expect(meta["schema_version"]).toBe(3); expect(meta["catalog_version"]).toBe(CATALOG_SHA); expect(meta["groups"]).toEqual([{ title: "g", bounding: [0, 0, 10, 10] }]); expect(meta["extra"]).toEqual({ ds: { scale: 1, offset: [0, 0] } }); @@ -441,7 +443,7 @@ describe("read-only surface — the KA-11 read gate (#38)", () => { } const UNREADABLE: [string, () => Y.Doc][] = [ - ["newer than this package", () => docWithSchemaVersion(3)], + ["newer than this package", () => docWithSchemaVersion(4)], ["older than this package", () => docWithSchemaVersion(1)], ["not an integer version", () => docWithSchemaVersion("1")], ["a zero version", () => docWithSchemaVersion(0)], @@ -496,7 +498,7 @@ describe("read-only surface — the KA-11 read gate (#38)", () => { // The positive control: without it every assertion above passes for a // surface that refused unconditionally. const doc = fixtureDoc(); - expect(readMeta(doc)["schema_version"]).toBe(2); + expect(readMeta(doc)["schema_version"]).toBe(3); expect(Object.keys(readGraph(doc).nodes).sort()).toEqual([ String(KSAMPLER_ID), String(NOTE_ID), @@ -674,7 +676,7 @@ describe("read-only surface — the KA-11 read gate (#38)", () => { // an untypable root as content instead, so the refusal type matches. const doc = new Y.Doc(); doc.getArray("nodes").push([1]); - metaMap(doc).set("schema_version", 3); + metaMap(doc).set("schema_version", 4); expect(() => project(doc, catalog)).toThrow(SchemaVersionError); expect(() => readGraph(doc)).toThrow(SchemaVersionError); expect(() => readStamps(doc)).toThrow(SchemaVersionError); @@ -722,6 +724,7 @@ describe("read-only surface — classification", () => { "mint", "migrate", "SCHEMA_VERSION", + "LINK_STATE_DESCRIPTOR_VERSION", "OpRejectedError", "SchemaVersionError", "FROZEN_OPS", @@ -762,6 +765,7 @@ describe("read-only surface — classification", () => { ]; const READ_SURFACE: readonly string[] = [ "readGraph", + "readLinkState", "readMeta", "docCatalogPin", "hasNode", diff --git a/test/roundtrip.test.ts b/test/roundtrip.test.ts index 3d4669a9..fed49e16 100644 --- a/test/roundtrip.test.ts +++ b/test/roundtrip.test.ts @@ -5,7 +5,7 @@ * dropped passthrough keys) and loses nothing but array order. * * Also pins migrate() (schema §10): validate the doc's own schema_version on - * every read, then an EXACT no-op at v1 that materializes no root type; + * every read, then an EXACT no-op at the current version that materializes no root type; * fail-closed on anything newer, unknown, contradictory, or unreadable. */ import { describe, expect, it } from "vitest"; diff --git a/test/schema-version-on-read.test.ts b/test/schema-version-on-read.test.ts index 201b0569..2d91468e 100644 --- a/test/schema-version-on-read.test.ts +++ b/test/schema-version-on-read.test.ts @@ -38,7 +38,6 @@ import { readSchemaVersion, type WorkflowJSON, } from "../src/index.js"; -import { assertSchemaVersionAgainst } from "../src/schema-version.js"; import { canonicalize, loadCatalog, loadSession, sessionFiles } from "./helpers.js"; const catalog = loadCatalog(); @@ -205,26 +204,24 @@ describe("project() enforces schema-version on read (KA-11, #38)", () => { expect(Y.encodeStateAsUpdate(accepted)).toEqual(acceptedBytes); }); - it("refuses an OLDER document and points at migrate() rather than projecting or migrating it", () => { - // v1 is the first layout, so no document older than this reader can be - // constructed today. The rule is exercised against an explicit reader - // version instead — the same code path `project()` takes, with the one - // value that cannot yet vary held to a future value. + it("refuses an OLDER document and directs the host to re-mint from the source workflow", () => { + // Regression: https://github.com/Comfy-Org/comfy-multi-player/pull/206#discussion_r4041772404 + // Exercise the public reader with a real old layout, not a hypothetical + // future reader. Private alpha deliberately provides no compatibility path. const doc = readableDoc(); - const futureReader = SCHEMA_VERSION + 1; + metaMap(doc).set("schema_version", 1); + const before = Y.encodeStateAsUpdate(doc); + const rootsBefore = [...doc.share.keys()].sort(); - expect(() => assertSchemaVersionAgainst(doc, "project", futureReader)).toThrow(SchemaVersionError); - expect(() => assertSchemaVersionAgainst(doc, "project", futureReader)).toThrow( - new RegExp(`doc schema v${SCHEMA_VERSION} is older than this package's v${futureReader}`), + expect(() => project(doc, catalog)).toThrow(SchemaVersionError); + expect(() => project(doc, catalog)).toThrow( + new RegExp(`doc schema v1 is older than this package's v${SCHEMA_VERSION}`), ); - // The remedy is named, and it is NOT "project it anyway" and NOT "migrate - // it here": `project()` is a pure read and a migration is a host-only - // write (schema §10). - expect(() => assertSchemaVersionAgainst(doc, "project", futureReader)).toThrow( - new RegExp(`call migrate\\(doc, ${SCHEMA_VERSION}\\) first`), + expect(() => project(doc, catalog)).toThrow( + /re-mint from the source workflow on the host/, ); - // …and the read really is refused, not merely warned about. - expect(() => assertSchemaVersionAgainst(doc, "project", SCHEMA_VERSION)).not.toThrow(); + expect(Y.encodeStateAsUpdate(doc)).toEqual(before); + expect([...doc.share.keys()].sort()).toEqual(rootsBefore); }); it("the gate cannot be reached past a wrong catalog: schema is checked first", () => { diff --git a/test/schema.test.ts b/test/schema.test.ts index 5ae3ca70..a32011f5 100644 --- a/test/schema.test.ts +++ b/test/schema.test.ts @@ -9,8 +9,8 @@ import { import { OPAQUE_WIDGETS_KEY, SCHEMA_VERSION, linksMap, nodesMap } from "../src/index.js"; describe("schema", () => { - it("pins SCHEMA_VERSION at 2", () => { - expect(SCHEMA_VERSION).toBe(2); + it("pins SCHEMA_VERSION at 3", () => { + expect(SCHEMA_VERSION).toBe(3); }); it("initDoc creates the v1 layout: nodes/links/definitions/meta + bookkeeping", () => { @@ -21,6 +21,7 @@ describe("schema", () => { expect(definitionsMap(doc)).toBeInstanceOf(Y.Map); expect(doc.getMap("__applied")).toBeInstanceOf(Y.Map); expect(doc.getMap("__stamps")).toBeInstanceOf(Y.Map); + expect(doc.getMap("__link_state")).toBeInstanceOf(Y.Map); const meta = metaMap(doc); expect(meta.get("schema_version")).toBe(SCHEMA_VERSION); expect(meta.get("catalog_version")).toBe("object_info@2026-08-01"); diff --git a/test/set-widget-interior-incarnation.test.ts b/test/set-widget-interior-incarnation.test.ts index 87b12730..e60291e2 100644 --- a/test/set-widget-interior-incarnation.test.ts +++ b/test/set-widget-interior-incarnation.test.ts @@ -88,7 +88,9 @@ describe("set_widget interior incarnation guard (current-behavior characterizati { op_id: op.op_id, outcome: "applied" }, ]); expect(interiorValue(doc)).toBe("value-1"); - expect(readStamps(doc)[stampTargetKey(op)]).toEqual(stampKey(op)); + expect(readStamps(doc)).toEqual({ + [JSON.stringify(["widget", [DEFINITION_ID, String(INTERIOR_NODE_ID)], REPLACEMENT_INCARNATION, "text"])]: stampKey(op), + }); }); it("makes a mismatched-incarnation write a graph no-op without recording a stamp", () => { @@ -125,7 +127,9 @@ describe("set_widget interior incarnation guard (current-behavior characterizati { op_id: op.op_id, outcome: "applied" }, ]); expect(interiorValue(doc)).toBe("value-3"); - expect(readStamps(doc)[stampTargetKey(op)]).toEqual(stampKey(op)); + expect(readStamps(doc)).toEqual({ + [JSON.stringify(["widget", [DEFINITION_ID, String(INTERIOR_NODE_ID)], LEGACY_NODE_INCARNATION, "text"])]: stampKey(op), + }); }); it("makes an absent-incarnation write a no-op for a non-legacy target", () => { diff --git a/test/type-negatives.test.ts b/test/type-negatives.test.ts index ebd91844..6fe5256d 100644 --- a/test/type-negatives.test.ts +++ b/test/type-negatives.test.ts @@ -49,7 +49,7 @@ const negatives = join(root, "test", "types", "invalid-states.negative.ts"); * tests green. An exact count also means ADDING a state without updating this * constant fails, which is the reminder you want. */ -const EXPECTED_DIRECTIVES = 13; +const EXPECTED_DIRECTIVES = 14; const DIRECTIVE = /^[ \t]*\/\/ @ts-expect-error\b/gm; describe("invalid op states are unrepresentable (issue #17)", () => { diff --git a/test/types/invalid-states.negative.ts b/test/types/invalid-states.negative.ts index 2c70f777..132b201a 100644 --- a/test/types/invalid-states.negative.ts +++ b/test/types/invalid-states.negative.ts @@ -25,8 +25,10 @@ import type { ConnectOp, DeleteNodeOp, GrowConnectOp, + InsertWorkflowOp, InteriorSetWidgetOp, Op, + OpKind, ResetDocOp, SetWidgetOp, TopLevelSetWidgetOp, @@ -196,6 +198,20 @@ declare function applyOpsSignature(ops: Op[]): void; // @ts-expect-error #17: the applier cannot be handed an op it always refuses. applyOpsSignature([reset]); +const insertWorkflow: InsertWorkflowOp = { + op: "insert_workflow", + ...env, + workflow: { nodes: [], links: [], definitions: { subgraphs: [] } }, +}; +const insertWorkflowAsOp: Op = insertWorkflow; + +// The public vocabulary projection accepts every declared kind and no value +// outside the wire union. This is an external-consumer check through index.ts. +const okImplementedKind: OpKind = "add_node"; +const okDeferredKind: OpKind = "reset_doc"; +// @ts-expect-error #21: undeclared operation kinds are not public OpKind values. +const unknownKind: OpKind = "unknown_op"; + // --------------------------------------------------------------------------- // Positive controls — these MUST compile, or the gate above is vacuous // --------------------------------------------------------------------------- diff --git a/test/wire-layout-contract.test.ts b/test/wire-layout-contract.test.ts index b53bdb1d..81154a61 100644 --- a/test/wire-layout-contract.test.ts +++ b/test/wire-layout-contract.test.ts @@ -96,6 +96,7 @@ import { import { appliedMap, definitionsMap, + linkStateMap, metaMap, stampsMap, } from "../src/doc.js"; @@ -140,8 +141,20 @@ const env = () => { */ const richWorkflow: WorkflowJSON = { nodes: [ - { id: 1, type: "Inner", inputs: [], outputs: [], widgets_values: ["v"] }, - { id: 2, type: "UnknownClass", inputs: [], outputs: [], widgets_values: [7, 8] }, + { + id: 1, + type: "Inner", + inputs: [{ name: "in", type: "X", link: 12 }], + outputs: [{ name: "out", type: "X", links: [11] }], + widgets_values: ["v"], + }, + { + id: 2, + type: "UnknownClass", + inputs: [{ name: "in", type: "X", link: 11 }], + outputs: [{ name: "out", type: "X", links: [12] }], + widgets_values: [7, 8], + }, ], links: [ [11, 1, 0, 2, 0, "X"], @@ -375,7 +388,7 @@ describe("layer 3: the code, the golden vector and the schema documents agree (K it("uses the supported wire-layout vector format", () => { expect(golden.format_version).toBe(1); - expect(goldenRootNames.length).toBe(6); + expect(goldenRootNames.length).toBe(7); }); it("is reachable from the conformance manifest, so a second implementation finds it", () => { @@ -401,6 +414,7 @@ describe("layer 3: the code, the golden vector and the schema documents agree (K meta: metaMap(doc), applied: appliedMap(doc), stamps: stampsMap(doc), + link_state: linkStateMap(doc), }; for (const [role, wireName] of Object.entries(golden.roots)) { expect(rootNameOf(doc, byRole[role]), `role '${role}' must live under wire name '${wireName}'`).toBe(wireName);