Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 64 additions & 61 deletions examples/dochost-poc/README.md
Original file line number Diff line number Diff line change
@@ -1,70 +1,73 @@
# dochost CRDT POC harness
# Document-host package parity harness

A small **no-mock** harness that drives the **real** doc-host sidecar (from
`Comfy-Org/cloud`, `services/agent/dochost`) running the **real** applier in this package,
so you can watch a workflow document converge from ops + deltas without standing up the whole
cloud. It is a backend/protocol proof, **not** a live multiplayer demo.
This harness compares the checked-out CMP build with that exact package installed
in the real cloud document-host sidecar. It tests loopback HTTP and document
semantics, **not browser behavior or deployment readiness**.

> This example is a shareable way to exercise and reason about the V1 document-semantics path.
> It is not wired into CI.
## Run

## What is real vs. absent

| Piece | Here | Notes |
| --- | --- | --- |
| Applier / merge / projection / mint | **REAL** | this package (`@comfyorg/comfy-multi-player`) |
| doc-host sidecar server | **REAL** | `services/agent/dochost/src/server.ts` from cloud main |
| HTTP transport (mint/apply/project) | **REAL** | loopback `127.0.0.1:8095`, same as in-pod |
| Follower integration | **REAL** | a real `Y.Doc` fold of `snapshot + host deltas` (raw-struct fan-out) |
| Widget catalog / base graph / ops | **REAL fixtures + frozen op shapes** | `fixtures/catalog.json`, `fixtures/session-edit-heavy.session.jsonl` |
| Go agent, ingest relay, Redis, Postgres, Temporal, auth | **absent** | transport + persistence around the core, not doc semantics |
| Browser, `/ws` WebSocket, litegraph | **absent** | no FE doc-frame client exists yet |

The document-semantics path is 100% real; the networking + browser + persistence shell is
simply not part of this slice. Nothing is stubbed.

## What it proves (all green)

1. concurrent human `set_widget(seed)` and agent `add_node(CLIPTextEncode)+connect` both land;
2. a follower that applied **only** the host's incremental Yjs deltas converges to the host
projection (raw-struct fan-out; convergence is projection-equality, per schema §2.5);
3. redelivering a delta is a no-op (idempotency);
4. deltas integrate order-independently at the follower.

## Run it

You need a checkout of `Comfy-Org/cloud` main for the doc-host sidecar source at
`services/agent/dochost`. Then one command builds both halves, starts the sidecar, and drives it:
Use Node 22+ and a compatible checkout of `Comfy-Org/cloud` with
`services/agent/dochost`. Install this repository's development dependencies first:

```bash
DOCHOST_SRC=/path/to/cloud-main/services/agent/dochost ./examples/dochost-poc/run.sh
# -> ALL GREEN: 7 passed, 0 failed
npm ci
DOCHOST_SRC=/path/to/cloud/services/agent/dochost ./examples/dochost-poc/run.sh
```

`services/agent/dochost/package.json` depends on the published npm-registry package
`@comfyorg/comfy-multi-player@0.1.0`, so its `npm ci` pulls the published applier. This harness's
fixtures come from the local build of this repo; `0.1.0` matches this repo at the publish point,
so the projection-equality check covers that local-build/published-package boundary.

Manual equivalent, if you'd rather not use the script:

```bash
npm ci && npm run build # 1. build the applier (this repo)
( cd "$DOCHOST_SRC" && npm ci && npm run build ) # 2. build the sidecar
# Yjs must be a SINGLE instance — a nested second copy breaks Yjs instanceof checks and makes
# project() throw "could not be cloned". npm ci dedupes; if you used a file: link, also:
rm -rf "$DOCHOST_SRC/node_modules/@comfyorg/comfy-multi-player/node_modules/yjs"
PORT=8095 node "$DOCHOST_SRC/dist/server.js" & # 3. start sidecar
node examples/dochost-poc/dochost-driver.mjs # 4. drive it (fixtures read from this repo)
The launcher builds and packs this checkout, prints the artifact SHA-256, and
copies the sidecar source into a disposable directory. All sidecar installation,
build and execution happen there. It installs the tarball over the sidecar's
registry dependency, verifies every installed `dist` file and the package manifest
against the tarball, verifies import resolution and one shared Yjs dependency,
then starts the sidecar. It never installs into or deletes dependencies from
`DOCHOST_SRC`. The temporary directory and child server are cleaned up on exit.

An incompatible sidecar fails at build or comparison; the launcher does not patch
the consumer or fall back to an older package. Version equality alone is not
artifact identity. Record both repositories' exact revisions and the printed
artifact digest when saving a test result.

```diagram
Checked-out package ── pack ──► disposable sidecar ──► HTTP results
│ │ │
│ one snapshot │
│ │ │
└──── direct apply ◄─────────┘ │
└──── outcomes + projection compare ─┘
```

Env: `DOC_HOST` (default `http://127.0.0.1:8095`), `CMP_PIN` (default = repo root, for fixtures),
`PORT` (default `8095`).

## What it is NOT

Not a live multiplayer session and not something your laptop can join over the network. A true
"team connects to the graph and watches the agent edit" demo additionally needs: an FE `/ws`
doc-frame client + Follower `Y.Doc` + litegraph bridge (none on any branch yet); the Go agent +
ingest relay + Redis actually running; and a network-reachable endpoint. Those are where the
integration risk now lives.
## Checks and limits

- A human seed edit and an agent node-plus-link batch both apply, with their
actual graph effects checked independently of result parity.
- Each ordered `ApplyResult` matches the direct build: operation identity,
outcome, stable rejection code and `ops_seen`. Human-readable error prose is
deliberately excluded. Legacy-only result shapes fail.
- Projections match after each batch. A real unknown-widget refusal aborts a
valid trailing operation; folding its returned delta must leave encoded state
byte-identical and must not record the trailing operation in `__applied`.
- Followers consuming only host deltas converge; duplicate and reversed delta
delivery preserve the projection.

All direct comparisons fork from the **same server-minted snapshot** and prior
host deltas. Direct-comparison updates never enter the host or follower stream
(KA-10, KA-6, FC-1). There is one shared applier implementation (KA-3/FC-3).
Object key order is immaterial; outcome and array order remain significant.
See [the invariants](../../docs/INVARIANTS.md).

`PORT` defaults to `8095`; the launcher refuses an already healthy service on that
port. When invoking the driver alone, `DOC_HOST` selects the HTTP endpoint and
`CMP_PIN` selects the local checkout containing fixtures, `dist` and dependencies.
**A driver-only run does not verify the server's installed artifact identity.**

The real sidecar run is manual, not part of CI. Automated harness regressions use
explicit launcher fakes and a small HTTP fixture; their results are not real
cloud acceptance evidence. No browser, canvas, WebSocket, agent turn loop,
authentication or persistence is exercised. Browser-visible changes still need
separate consumer QA. This example makes no claim about which frontend clients
exist on other branches.

Glossary: CMP = the standalone `@comfyorg/comfy-multi-player` package;
sidecar = cloud's Node document host; parity = matching results across execution
paths; artifact = the package tarball; snapshot = shared initial Yjs document
bytes; QA = quality assurance; CI = automated repository checks.
131 changes: 121 additions & 10 deletions examples/dochost-poc/dochost-driver.mjs
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
/**
* No-mock CRDT POC — drives the REAL doc-host sidecar from cloud main
* (services/agent/dochost/src/server.ts). The sidecar consumes the published
* @comfyorg/comfy-multi-player@0.1.0 package; this driver exercises that applier
* through the sidecar's HTTP contract.
* (services/agent/dochost/src/server.ts). run.sh installs the checked-out packed
* package in a disposable sidecar, verifies its identity, and invokes this
* driver to compare its HTTP results with the direct local build.
*
* Nothing here is stubbed: mint, apply, and project all execute inside the real
* server over loopback HTTP; the follower is a real Y.Doc integrating only the
* host's incremental Yjs deltas (raw-struct fan-out), never the whole doc.
*
* Proves, end to end through the deployed server contract:
* Checks through the supplied sidecar's HTTP contract (not a deployment):
* 1. concurrent human set_widget || agent add_node+connect both land;
* 2. a follower that applied ONLY the host deltas converges to the host
* projection (projection-equality, per schema §2.5 — state bytes differ by
Expand All @@ -19,12 +19,16 @@
* Run: node examples/dochost-poc/dochost-driver.mjs (dochost must be on :8095)
*/
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { fileURLToPath, pathToFileURL } from "node:url";

const HOST = process.env.DOC_HOST || "http://127.0.0.1:8095";
const CMP = process.env.CMP_PIN || fileURLToPath(new URL("../../", import.meta.url)); // repo root

const catalog = JSON.parse(readFileSync(`${CMP}/fixtures/catalog.json`, "utf8"));
// Keep package-dependent imports below `const catalog =`: the launcher copies
// the source prefix to an artificial root as a portability regression probe.
const cmp = await import(pathToFileURL(`${CMP}/dist/index.js`).href);
const Y = await import(pathToFileURL(`${CMP}/node_modules/yjs/dist/yjs.mjs`).href);
// Real base workflow: the team-spike edit-heavy session's base graph.
const session = readFileSync(`${CMP}/fixtures/session-edit-heavy.session.jsonl`, "utf8")
.split("\n")
Expand All @@ -33,7 +37,6 @@ const session = readFileSync(`${CMP}/fixtures/session-edit-heavy.session.jsonl`,
const base = session[0].base_workflow;

const KSAMPLER = base.nodes.find((n) => n.type === "KSampler");
const CLIP = base.nodes.find((n) => n.type === "CLIPTextEncode");

let idc = 0;
const opId = () => (Date.now().toString(16) + (idc++).toString(16).padStart(4, "0")).padEnd(32, "0").slice(0, 32);
Expand All @@ -44,13 +47,49 @@ async function post(path, body) {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(body),
signal: AbortSignal.timeout(10_000),
});
const j = await r.json();
if (!r.ok) throw new Error(`${path} ${r.status}: ${JSON.stringify(j)}`);
return j;
}

const eq = (a, b) => JSON.stringify(a) === JSON.stringify(b);
const canonical = (value) => {
if (Array.isArray(value)) return value.map(canonical);
if (value && typeof value === "object") {
return Object.fromEntries(Object.keys(value).sort().map((key) => [key, canonical(value[key])]));
}
return value;
};
const eq = (a, b) => JSON.stringify(canonical(a)) === JSON.stringify(canonical(b));
const stableOutcomes = (result) => {
if (!result || !Array.isArray(result.outcomes) || !Number.isInteger(result.ops_seen)) {
throw new Error(`sidecar returned legacy/invalid ApplyResult: ${JSON.stringify(result)}`);
}
return {
outcomes: result.outcomes.map((item) => ({
op_id: item.op_id,
outcome: item.outcome,
...(item.outcome === "rejected" ? { code: item.reason?.code } : {}),
})),
ops_seen: result.ops_seen,
};
};

const directApply = (updates_b64, ops) => {
// KA-10: every direct comparison starts from the one server-minted bootstrap
// snapshot, then folds only prior HOST deltas. Its update is never sent to
// the host or followers.
const doc = new Y.Doc();
try {
Y.applyUpdate(doc, Buffer.from(snapshot_b64, "base64"));
for (const update of updates_b64) Y.applyUpdate(doc, Buffer.from(update, "base64"));
const apply_result = cmp.applyOps(doc, ops, catalog);
return { apply_result, projection: cmp.project(doc, catalog) };
} finally {
doc.destroy();
}
};
let pass = 0,
fail = 0;
const check = (name, cond, extra = "") => {
Expand All @@ -59,7 +98,9 @@ const check = (name, cond, extra = "") => {

console.log("== no-mock CRDT POC through the real doc-host sidecar ==");
console.log(`host=${HOST} cmp-pin=${CMP}`);
const health = await (await fetch(HOST + "/health")).json();
const healthResponse = await fetch(HOST + "/health", { signal: AbortSignal.timeout(10_000) });
if (!healthResponse.ok) throw new Error(`sidecar health returned ${healthResponse.status}`);
const health = await healthResponse.json();
console.log(`dochost /health -> ${JSON.stringify(health)}\n`);

// 1. MINT the bootstrap snapshot every replica forks from.
Expand Down Expand Up @@ -128,7 +169,10 @@ const r1 = await post("/apply", {
turn_id: "t1",
catalog,
});
check("human set_widget applied", eq(r1.apply_result.applied, [humanOp.op_id]), JSON.stringify(r1.apply_result));
const d1 = directApply([], [humanOp]);
check("human ApplyResult matches local executable package", eq(stableOutcomes(r1.apply_result), stableOutcomes(d1.apply_result)), JSON.stringify(r1.apply_result));
check("human projection matches local executable package", eq(r1.projection, d1.projection));
check("human set_widget applied", r1.apply_result.outcomes?.[0]?.op_id === humanOp.op_id && r1.apply_result.outcomes[0].outcome === "applied", JSON.stringify(r1.apply_result));

const r2 = await post("/apply", {
snapshot_b64,
Expand All @@ -138,9 +182,19 @@ const r2 = await post("/apply", {
turn_id: "t2",
catalog,
});
const d2 = directApply([r1.update_b64], [agentAdd, agentConnect]);
check(
"agent ApplyResult matches local executable package",
eq(stableOutcomes(r2.apply_result), stableOutcomes(d2.apply_result)),
JSON.stringify(r2.apply_result),
);
check("agent projection matches local executable package", eq(r2.projection, d2.projection));
check(
"agent add_node+connect applied",
eq(r2.apply_result.applied, [agentAdd.op_id, agentConnect.op_id]),
eq(r2.apply_result.outcomes?.map(({ op_id, outcome }) => ({ op_id, outcome })), [
{ op_id: agentAdd.op_id, outcome: "applied" },
{ op_id: agentConnect.op_id, outcome: "applied" },
]),
JSON.stringify(r2.apply_result),
);

Expand All @@ -149,6 +203,63 @@ const hostProj = r2.projection;
const hostK = hostProj.nodes.find((n) => String(n.id) === String(KSAMPLER.id));
check("host: human seed edit present", hostK && Number(hostK.widgets_values[0]) === 424242, JSON.stringify(hostK?.widgets_values));
check("host: agent node present", !!hostProj.nodes.find((n) => String(n.id) === String(newNodeId)));
check("host: agent link present", eq(hostProj.links.find((link) => String(link[0]) === String(newLinkId)),
[newLinkId, newNodeId, 0, KSAMPLER.id, 1, "CONDITIONING"]));

// A real refusal pins comparison of the stable reason code (never its
// intentionally human/volatile message).
const rejectedOp = { ...humanOp, op_id: opId(), stamp: [2, "human:jo"], widget: "not_a_real_widget" };
const abortedOp = { ...humanOp, op_id: opId(), stamp: [3, "human:jo"], value: 515151 };
const r3 = await post("/apply", {
snapshot_b64,
updates_b64: [r1.update_b64, r2.update_b64],
ops: [rejectedOp, abortedOp],
actor: "human:jo",
turn_id: "t3-rejection-probe",
catalog,
});
const d3 = directApply([r1.update_b64, r2.update_b64], [rejectedOp, abortedOp]);
check(
"rejected ApplyResult and stable reason code match local executable package",
eq(stableOutcomes(r3.apply_result), stableOutcomes(d3.apply_result)),
JSON.stringify(r3.apply_result),
);
check("unknown widget is rejected", r3.apply_result.outcomes?.[0]?.outcome === "rejected"
&& r3.apply_result.outcomes[0].reason?.code === "unknown_widget");
check("valid trailing operation is batch_aborted", r3.apply_result.outcomes?.[1]?.op_id === abortedOp.op_id
&& r3.apply_result.outcomes[1].outcome === "rejected"
&& r3.apply_result.outcomes[1].reason?.code === "batch_aborted");
check("rejected batch projection matches local executable package", eq(r3.projection, d3.projection));
check("rejection leaves graph unchanged", eq(r3.projection, hostProj));
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

// Reconstruct both documents from the same bootstrap and prior HOST deltas,
// then fold the returned rejection delta into only the after-document. This
// direct comparison never enters either the host or follower update stream.
const rejectionBefore = new Y.Doc();
const rejectionAfter = new Y.Doc();
try {
for (const doc of [rejectionBefore, rejectionAfter]) {
Y.applyUpdate(doc, Buffer.from(snapshot_b64, "base64"));
Y.applyUpdate(doc, Buffer.from(r1.update_b64, "base64"));
Y.applyUpdate(doc, Buffer.from(r2.update_b64, "base64"));
}
Y.applyUpdate(rejectionAfter, Buffer.from(r3.update_b64, "base64"));
check(
"rejected batch leaves encoded document state unchanged",
Buffer.from(Y.encodeStateAsUpdate(rejectionBefore)).equals(Buffer.from(Y.encodeStateAsUpdate(rejectionAfter))),
);
check(
"rejected operation is absent from __applied",
!rejectionAfter.getMap("__applied").has(rejectedOp.op_id),
);
check(
"batch-aborted trailing operation is absent from __applied",
!rejectionAfter.getMap("__applied").has(abortedOp.op_id),
);
} finally {
rejectionBefore.destroy();
rejectionAfter.destroy();
}

// 4. FOLLOWER converges from the host DELTAS only (raw-struct fan-out).
// /project folds snapshot + the two host updates and projects — this is
Expand Down
Loading
Loading