Repository navigation
test(agent): pin the empty canvas a displaced agent build leaves behind (PM-1535) - #18610
Conversation
Adds a two-session CRDT harness that boots one cloud app, binds two agent workflows through the real agent_active_tab path against a shared in-process doc host, and switches between their tabs. Two failing repros (test.fail) plus one passing control: - returning to the first session's tab shows stale values because the follower never resubscribes the active tab's workflow; - a human edit on the returned-to tab is dropped because the mint port stays bound to the other session's workflow; - control: an edit on a bound, active tab does reach its doc.
Fallow's audit gate (CI: Fallow) flagged the new onClientFrame method with a CRAP score of 63.6 (threshold 30, cyclomatic 15). Split it into a pure parseClientFrame helper plus recordOutboundDocOps/handleDocSubscribe, matching fallow's own suggested fix (extract helper functions). Behavior is unchanged; test.fail() assertions untouched.
The control ("a human edit on the bound active tab reaches its doc")
polled docOpsCount() with the default 5s expect.poll timeout while its
sibling assertions in the same file use 20s. Under the
playwright-video-new-tests job (SLOW_MO=250, --retries=0) that 5s
budget is too tight for the real mint round trip and the control
failed. Match the 20s timeout already used elsewhere in this spec.
Key was renamed to agent.entryButton in #17621, leaving OPEN_AGENT_LABEL undefined and getByRole matching every button on the page.
…ion-crdt-subscription
Pin the boot tab as the composer target so Send actually posts (with no target it opens the Switch workflow picker), take thread ids from the mocked POST ack, and rebuild the fixture around user-visible actions: startThread, bindViaActiveTab, finishTurn, newChat. Bug tests keep test.fail placed after setup so a fixture failure is still reported.
PM-1535: two agent threads on two workflow tabs, each asked to build a workflow. The build lands in its own document, but by then the other thread holds the binding, so nothing is subscribed to it and clicking back to its tab does not resubscribe. Where PM-1319 showed a stale widget value, the tab the user returns to has no nodes at all while the agent reports the workflow built. Three tests share that arrangement and assert one reported observation each: the canvas is empty on return, still empty after a refresh, and refilled by the next turn bound to that workflow. The refresh leg gets its own expected failure because it fails through a different path — after a reload the session holds no binding until the next turn ack, and the persisted doc id does not cover it (refused across a reload by design, and naming the other workflow here anyway) — so a fix to the tab-return target will not turn it green. The next-turn leg passes today and is what proves the build really is in the document and reachable. A control pins materialization on a bound, active tab, so the expected failures cannot pass vacuously. Two limits, also recorded in the spec. Thread one's turn is over before its build lands, where the report had both threads working at once; the inbound gate is stamped by workflow rather than thread, so both routes leave the follower on the same target and the tab return meets the same state. And only one workflow is ever built, which keeps the canvas unambiguous but leaves the report's second empty canvas unexplained — the control shows a bound, active tab does materialize its build, so PM-1535 is not fully covered until that second one is understood. Fixture: workflows can be seeded empty, and a whole bound turn and a reload are single calls.
🎭 Playwright: ✅ 2202 passed, 0 failed📊 Browser Reports
🌾 Fallow: ✅ PassedFallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened. Fallow gates on new findings in changed files only, so everything below was introduced or moved by this PR (6 changed files).
Pre-existing findings are baselined in Reproduce locally# the base the gate compared against
npx fallow audit --root . --quiet --format json \
--changed-since cdc18a92d227a965810b08928963124959951908
Full docs: ADR-DEVEX-LINT-0015 🧩 Custom Nodes Ecosystem Matrix: ✅ PassedExecutes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight
On a pin-bump PR, a red gate means the ecosystem moved — not that your diff broke something. That is what the bump PR is for. Outside a pin bump or a cold rebuild of the unpinned tail, a red means the diff. Note the corpus size and the executed count are different numbers, and the verdict is against the executed count: roughly 3,100 of ~5,100 packs ship no extension-shaped JS and are skipped. Reproduce locally# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
npx vitest run --config vitest.matrix.config.mts
Full docs: scripts/registry-census/README.md 📦 Bundle: 10.2 MB gzip 🟢 -264 BDetailsSummary
Category Glance App Entry Points — 3.98 kB (baseline 3.98 kB) • ⚪ 0 BMain entry bundles and manifests
Status: 1 added / 1 removed Graph Workspace — 1.47 MB (baseline 1.47 MB) • ⚪ 0 BGraph editor runtime, canvas, workflow orchestration Status: 3 unchanged Views & Navigation — 135 kB (baseline 136 kB) • 🟢 -206 BTop-level views, pages, and routed surfaces
Status: 2 added / 2 removed / 16 unchanged Panels & Settings — 648 kB (baseline 648 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens Status: 28 unchanged User & Accounts — 53.7 kB (baseline 53.9 kB) • 🟢 -175 BAuthentication, profile, and account management bundles
Status: 2 added / 2 removed / 13 unchanged Editors & Dialogs — 133 kB (baseline 133 kB) • ⚪ 0 BModals, dialogs, drawers, and in-app editors Status: 9 unchanged UI Components — 194 kB (baseline 194 kB) • ⚪ 0 BReusable component library chunks Status: 18 unchanged Data & Services — 4.17 MB (baseline 4.17 MB) • ⚪ 0 BStores, services, APIs, and repositories Status: 16 unchanged Utilities & Hooks — 580 kB (baseline 580 kB) • ⚪ 0 BHelpers, composables, and utility bundles Status: 38 unchanged Vendor & Third-Party — 17.2 MB (baseline 17.2 MB) • ⚪ 0 BExternal libraries and shared vendor chunks Status: 18 unchanged Other — 18.5 MB (baseline 18.5 MB) • 🟢 -542 BBundles that do not match a named category
Status: 2 added / 2 removed / 307 unchanged ⚡ Performance Report
Show regressions
All metrics
Historical variance (last 15 runs)
Trend (last 15 commits on main)
Raw data{
"timestamp": "2026-10-06T00:02:51.313Z",
"gitSha": "2653e2d7e1d4010ae8406fb670725f743c593919",
"branch": "glary/pm-1535-two-tab-empty-canvas",
"measurements": [
{
"name": "canvas-idle",
"durationMs": 2008.149000000003,
"styleRecalcs": 12,
"styleRecalcDurationMs": 4.225,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 311.622,
"heapDeltaBytes": 503596,
"heapUsedBytes": 69535304,
"domNodes": 0,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 11.396,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.665289256198346,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "canvas-idle",
"durationMs": 2008.7580000000003,
"styleRecalcs": 11,
"styleRecalcDurationMs": 4.621,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 330.278,
"heapDeltaBytes": 539636,
"heapUsedBytes": 68588632,
"domNodes": 0,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 11.045,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66611570247934,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "canvas-mouse-sweep",
"durationMs": 1679.0810000000533,
"styleRecalcs": 72,
"styleRecalcDurationMs": 14.844,
"layouts": 12,
"layoutDurationMs": 4.063000000000001,
"taskDurationMs": 464.459,
"heapDeltaBytes": -2664604,
"heapUsedBytes": 64731364,
"domNodes": 0,
"jsHeapTotalBytes": 1048576,
"scriptDurationMs": 94.929,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666336633663366,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "canvas-mouse-sweep",
"durationMs": 1679.8689999999965,
"styleRecalcs": 71,
"styleRecalcDurationMs": 16.666,
"layouts": 12,
"layoutDurationMs": 4.6290000000000004,
"taskDurationMs": 574.693,
"heapDeltaBytes": -2146752,
"heapUsedBytes": 66512768,
"domNodes": 0,
"jsHeapTotalBytes": 262144,
"scriptDurationMs": 119.025,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666336633663366,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1710.520000000031,
"styleRecalcs": 29,
"styleRecalcDurationMs": 9.375000000000002,
"layouts": 6,
"layoutDurationMs": 1.179,
"taskDurationMs": 253.286,
"heapDeltaBytes": 3712792,
"heapUsedBytes": 71119232,
"domNodes": 53,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 12.628,
"eventListeners": 15,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666019417475727,
"p95FrameDurationMs": 16.799999999999727
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1722.1720000000005,
"styleRecalcs": 31,
"styleRecalcDurationMs": 8.937,
"layouts": 6,
"layoutDurationMs": 1.1969999999999998,
"taskDurationMs": 222.77599999999998,
"heapDeltaBytes": 3703712,
"heapUsedBytes": 72369260,
"domNodes": 53,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 11.236,
"eventListeners": 15,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.665384615384614,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "dom-widget-clipping",
"durationMs": 459.68600000003335,
"styleRecalcs": 17,
"styleRecalcDurationMs": 4.708,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 300.158,
"heapDeltaBytes": -8117716,
"heapUsedBytes": 58665268,
"domNodes": 0,
"jsHeapTotalBytes": 524288,
"scriptDurationMs": 104.48599999999999,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999727
},
{
"name": "dom-widget-clipping",
"durationMs": 531.8269999999643,
"styleRecalcs": 15,
"styleRecalcDurationMs": 4.132000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 322.403,
"heapDeltaBytes": -6782472,
"heapUsedBytes": 60947924,
"domNodes": 0,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 111.824,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.662499999999994,
"p95FrameDurationMs": 16.700000000000273
},
{
"name": "dom-widget-order-dense-n200-w200-i120",
"durationMs": 873.3019999999954,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 473.703,
"heapDeltaBytes": 12023544,
"heapUsedBytes": 81447660,
"domNodes": 0,
"jsHeapTotalBytes": 2260992,
"scriptDurationMs": 36.977999999999994,
"eventListeners": 0,
"totalBlockingTimeMs": 266,
"frameDurationMs": 25.979411764705887,
"p95FrameDurationMs": 16.799999999999727
},
{
"name": "dom-widget-order-dense-n200-w200-i120",
"durationMs": 887.5009999999861,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 512.1759999999999,
"heapDeltaBytes": 38873268,
"heapUsedBytes": 108520348,
"domNodes": 0,
"jsHeapTotalBytes": 1998848,
"scriptDurationMs": 40.404999999999994,
"eventListeners": 0,
"totalBlockingTimeMs": 272,
"frameDurationMs": 26.76666666666666,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "dom-widget-order-sparse-n1000-w20-i120",
"durationMs": 716.5479999999889,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 287.2490000000001,
"heapDeltaBytes": 17338324,
"heapUsedBytes": 104948036,
"domNodes": 0,
"jsHeapTotalBytes": 266240,
"scriptDurationMs": 101.84800000000003,
"eventListeners": 0,
"totalBlockingTimeMs": 92,
"frameDurationMs": 20.4742857142857,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "dom-widget-order-sparse-n1000-w20-i120",
"durationMs": 719.2450000000008,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 291.21700000000004,
"heapDeltaBytes": 17223648,
"heapUsedBytes": 103375680,
"domNodes": 0,
"jsHeapTotalBytes": 266240,
"scriptDurationMs": 105.154,
"eventListeners": 0,
"totalBlockingTimeMs": 95,
"frameDurationMs": 21.079411764705878,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "extension-clean-combined-draw",
"durationMs": 5786.514000000011,
"styleRecalcs": 5,
"styleRecalcDurationMs": 1.4909999999999992,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 4735.6720000000005,
"heapDeltaBytes": -16309928,
"heapUsedBytes": 69111040,
"domNodes": -142,
"jsHeapTotalBytes": -10883072,
"scriptDurationMs": 3737.134,
"eventListeners": -78,
"totalBlockingTimeMs": 0,
"frameDurationMs": 20.220629370629368,
"p95FrameDurationMs": 33.399999999999636
},
{
"name": "extension-clean-combined-draw",
"durationMs": 6042.397999999992,
"styleRecalcs": 5,
"styleRecalcDurationMs": 1.4689999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 5082.906,
"heapDeltaBytes": 9347672,
"heapUsedBytes": 94915900,
"domNodes": -171,
"jsHeapTotalBytes": -8523776,
"scriptDurationMs": 4010.1509999999994,
"eventListeners": -78,
"totalBlockingTimeMs": 0,
"frameDurationMs": 20.71849315068493,
"p95FrameDurationMs": 33.399999999999636
},
{
"name": "extension-rgthree-prototype-draw",
"durationMs": 5912.491999999986,
"styleRecalcs": 4,
"styleRecalcDurationMs": 1.063999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 4982.099,
"heapDeltaBytes": -11746064,
"heapUsedBytes": 72921292,
"domNodes": -142,
"jsHeapTotalBytes": -11145216,
"scriptDurationMs": 3921.8579999999997,
"eventListeners": -78,
"totalBlockingTimeMs": 0,
"frameDurationMs": 20.687062937062937,
"p95FrameDurationMs": 33.399999999999636
},
{
"name": "extension-rgthree-prototype-draw",
"durationMs": 5663.513000000023,
"styleRecalcs": 3,
"styleRecalcDurationMs": 1.0759999999999987,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 4882.599,
"heapDeltaBytes": -24816,
"heapUsedBytes": 85223336,
"domNodes": -143,
"jsHeapTotalBytes": -9834496,
"scriptDurationMs": 3851.13,
"eventListeners": -82,
"totalBlockingTimeMs": 5,
"frameDurationMs": 20.90922509225092,
"p95FrameDurationMs": 33.40000000000009
},
{
"name": "large-graph-idle",
"durationMs": 2036.43500000004,
"styleRecalcs": 10,
"styleRecalcDurationMs": 4.904000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 333.693,
"heapDeltaBytes": -10594120,
"heapUsedBytes": 78052848,
"domNodes": 0,
"jsHeapTotalBytes": -225280,
"scriptDurationMs": 32.217999999999996,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.6655737704918,
"p95FrameDurationMs": 16.700000000000273
},
{
"name": "large-graph-idle",
"durationMs": 2009.4550000000027,
"styleRecalcs": 10,
"styleRecalcDurationMs": 3.3760000000000003,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 316.6700000000001,
"heapDeltaBytes": -10841832,
"heapUsedBytes": 77152348,
"domNodes": 0,
"jsHeapTotalBytes": -225280,
"scriptDurationMs": 29.044000000000008,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666942148760334,
"p95FrameDurationMs": 16.799999999999727
},
{
"name": "large-graph-pan",
"durationMs": 2117.271999999957,
"styleRecalcs": 70,
"styleRecalcDurationMs": 13.301999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1036.989,
"heapDeltaBytes": -19577232,
"heapUsedBytes": 70290856,
"domNodes": 0,
"jsHeapTotalBytes": 1085440,
"scriptDurationMs": 558.7840000000001,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-pan",
"durationMs": 2114.4819999999527,
"styleRecalcs": 70,
"styleRecalcDurationMs": 12.753,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1038.132,
"heapDeltaBytes": -20505792,
"heapUsedBytes": 69466896,
"domNodes": 0,
"jsHeapTotalBytes": 561152,
"scriptDurationMs": 556.411,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.665873015873014,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-zoom",
"durationMs": 3096.8050000000176,
"styleRecalcs": 73,
"styleRecalcDurationMs": 16.316999999999997,
"layouts": 60,
"layoutDurationMs": 15.312,
"taskDurationMs": 1166.0150000000003,
"heapDeltaBytes": 1153384,
"heapUsedBytes": 69351708,
"domNodes": 24,
"jsHeapTotalBytes": 1048576,
"scriptDurationMs": 543.4669999999999,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666129032258066,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "large-graph-zoom",
"durationMs": 3083.2149999999956,
"styleRecalcs": 74,
"styleRecalcDurationMs": 14.871,
"layouts": 60,
"layoutDurationMs": 14.886,
"taskDurationMs": 1093.12,
"heapDeltaBytes": 1625484,
"heapUsedBytes": 70225400,
"domNodes": 24,
"jsHeapTotalBytes": 786432,
"scriptDurationMs": 507.929,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.665945945945946,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "legacy-node-drag",
"durationMs": 2291.8660000000273,
"styleRecalcs": 49,
"styleRecalcDurationMs": 8.660999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1408.473,
"heapDeltaBytes": 3893756,
"heapUsedBytes": 73358036,
"domNodes": 0,
"jsHeapTotalBytes": 1048576,
"scriptDurationMs": 757.992,
"eventListeners": 180,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.78759124087591,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "legacy-node-drag",
"durationMs": 2119.0429999999765,
"styleRecalcs": 49,
"styleRecalcDurationMs": 8.015999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1209.887,
"heapDeltaBytes": 4076224,
"heapUsedBytes": 73306912,
"domNodes": 0,
"jsHeapTotalBytes": 1572864,
"scriptDurationMs": 656.263,
"eventListeners": 180,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.665354330708663,
"p95FrameDurationMs": 16.799999999999727
},
{
"name": "minimap-idle",
"durationMs": 2020.3960000000052,
"styleRecalcs": 10,
"styleRecalcDurationMs": 3.516,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 342.52900000000005,
"heapDeltaBytes": -10019272,
"heapUsedBytes": 79480800,
"domNodes": 0,
"jsHeapTotalBytes": -225280,
"scriptDurationMs": 29.352999999999998,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666115702479342,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "minimap-idle",
"durationMs": 2005.7559999999626,
"styleRecalcs": 10,
"styleRecalcDurationMs": 3.411000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 308.47799999999995,
"heapDeltaBytes": -9175224,
"heapUsedBytes": 81349012,
"domNodes": 0,
"jsHeapTotalBytes": -225280,
"scriptDurationMs": 26.604000000000003,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666115702479342,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 527.3960000000102,
"styleRecalcs": 48,
"styleRecalcDurationMs": 8.119000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 343.31100000000004,
"heapDeltaBytes": -12618752,
"heapUsedBytes": 57493076,
"domNodes": 0,
"jsHeapTotalBytes": 524288,
"scriptDurationMs": 121.35300000000001,
"eventListeners": 6,
"totalBlockingTimeMs": 0,
"frameDurationMs": 17.20322580645162,
"p95FrameDurationMs": 16.799999999999727
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 414.5510000000172,
"styleRecalcs": 49,
"styleRecalcDurationMs": 6.509999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 280.289,
"heapDeltaBytes": -12016260,
"heapUsedBytes": 57753736,
"domNodes": 0,
"jsHeapTotalBytes": 524288,
"scriptDurationMs": 98.496,
"eventListeners": 6,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.664000000000016,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-idle",
"durationMs": 2007.833000000005,
"styleRecalcs": 13,
"styleRecalcDurationMs": 4.625000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 266.178,
"heapDeltaBytes": 986284,
"heapUsedBytes": 71595380,
"domNodes": 0,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 8.929000000000002,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.665289256198346,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-idle",
"durationMs": 2010.6739999999945,
"styleRecalcs": 12,
"styleRecalcDurationMs": 4.678000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 347.66200000000003,
"heapDeltaBytes": 1018320,
"heapUsedBytes": 69705364,
"domNodes": 0,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 11.804000000000002,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.665289256198346,
"p95FrameDurationMs": 16.799999999999727
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1693.7090000000126,
"styleRecalcs": 75,
"styleRecalcDurationMs": 18.335,
"layouts": 16,
"layoutDurationMs": 5.733999999999999,
"taskDurationMs": 514.4820000000001,
"heapDeltaBytes": -6306484,
"heapUsedBytes": 64316120,
"domNodes": 0,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 87.515,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66568627450981,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1679.337999999973,
"styleRecalcs": 75,
"styleRecalcDurationMs": 18.392,
"layouts": 16,
"layoutDurationMs": 5.724,
"taskDurationMs": 529.5210000000001,
"heapDeltaBytes": -6276452,
"heapUsedBytes": 63419900,
"domNodes": 0,
"jsHeapTotalBytes": 262144,
"scriptDurationMs": 86.20599999999999,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666336633663366,
"p95FrameDurationMs": 16.799999999999727
},
{
"name": "subgraph-transition-enter",
"durationMs": 817.4930000000131,
"styleRecalcs": 18,
"styleRecalcDurationMs": 20.53400000000001,
"layouts": 5,
"layoutDurationMs": 9.146,
"taskDurationMs": 627.5619999999999,
"heapDeltaBytes": -2529288,
"heapUsedBytes": 89285752,
"domNodes": 14227,
"jsHeapTotalBytes": 1310720,
"scriptDurationMs": 21.322,
"eventListeners": 2463,
"totalBlockingTimeMs": 72,
"frameDurationMs": 18.990697674418612,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "viewport-pan-sweep",
"durationMs": 8177.024000000018,
"styleRecalcs": 251,
"styleRecalcDurationMs": 43.708,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 4074.8109999999997,
"heapDeltaBytes": 13173272,
"heapUsedBytes": 101734024,
"domNodes": 0,
"jsHeapTotalBytes": 1048576,
"scriptDurationMs": 2115.68,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.700204498977502,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "viewport-pan-sweep",
"durationMs": 8189.342000000011,
"styleRecalcs": 251,
"styleRecalcDurationMs": 43.729,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 3862.37,
"heapDeltaBytes": 7837544,
"heapUsedBytes": 96710968,
"domNodes": 0,
"jsHeapTotalBytes": 1572864,
"scriptDurationMs": 1982.036,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66598778004073,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "vue-large-graph-idle",
"durationMs": 12322.142999999982,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 11260.859,
"heapDeltaBytes": -34070240,
"heapUsedBytes": 179675980,
"domNodes": -3045,
"jsHeapTotalBytes": 6049792,
"scriptDurationMs": 175.91299999999998,
"eventListeners": -14915,
"totalBlockingTimeMs": 0,
"frameDurationMs": 20.839424703891712,
"p95FrameDurationMs": 33.400000000001455
},
{
"name": "vue-large-graph-idle",
"durationMs": 12374.196999999982,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 11530.634000000002,
"heapDeltaBytes": -30771920,
"heapUsedBytes": 182956076,
"domNodes": -3045,
"jsHeapTotalBytes": 10768384,
"scriptDurationMs": 181.797,
"eventListeners": -14915,
"totalBlockingTimeMs": 0,
"frameDurationMs": 21.16735042735043,
"p95FrameDurationMs": 33.400000000000546
},
{
"name": "vue-large-graph-pan",
"durationMs": 14431.039999999995,
"styleRecalcs": 71,
"styleRecalcDurationMs": 15.21900000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 13867.314,
"heapDeltaBytes": -35146756,
"heapUsedBytes": 177596028,
"domNodes": -8043,
"jsHeapTotalBytes": 17674240,
"scriptDurationMs": 598.2890000000001,
"eventListeners": -14915,
"totalBlockingTimeMs": 3,
"frameDurationMs": 25.958093525179855,
"p95FrameDurationMs": 50
},
{
"name": "vue-large-graph-pan",
"durationMs": 14390.579000000002,
"styleRecalcs": 69,
"styleRecalcDurationMs": 14.854000000000006,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 13781.298,
"heapDeltaBytes": -12233380,
"heapUsedBytes": 200869528,
"domNodes": -3045,
"jsHeapTotalBytes": 7700480,
"scriptDurationMs": 613.643,
"eventListeners": -14915,
"totalBlockingTimeMs": 0,
"frameDurationMs": 25.247110332749564,
"p95FrameDurationMs": 50
},
{
"name": "vue-node-resize-workload",
"durationMs": 4344.3219999999765,
"styleRecalcs": 20,
"styleRecalcDurationMs": 52.49200000000001,
"layouts": 20,
"layoutDurationMs": 23.308000000000003,
"taskDurationMs": 4197.038000000001,
"heapDeltaBytes": 805752,
"heapUsedBytes": 214570756,
"domNodes": 0,
"jsHeapTotalBytes": 1048576,
"scriptDurationMs": 70.062,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 25.738461538461536,
"p95FrameDurationMs": 50
},
{
"name": "vue-node-resize-workload",
"durationMs": 4421.692000000007,
"styleRecalcs": 20,
"styleRecalcDurationMs": 54.24599999999999,
"layouts": 20,
"layoutDurationMs": 23.592000000000002,
"taskDurationMs": 4283.363999999999,
"heapDeltaBytes": 891220,
"heapUsedBytes": 200555328,
"domNodes": 0,
"jsHeapTotalBytes": 1048576,
"scriptDurationMs": 72.382,
"eventListeners": 0,
"totalBlockingTimeMs": 0,
"frameDurationMs": 25.52890173410405,
"p95FrameDurationMs": 50
},
{
"name": "vue-zoom-out-idle",
"durationMs": 6289.348000000018,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 5959.565999999999,
"heapDeltaBytes": -41487768,
"heapUsedBytes": 172588860,
"domNodes": -3046,
"jsHeapTotalBytes": 14962688,
"scriptDurationMs": 92.33400000000003,
"eventListeners": -14915,
"totalBlockingTimeMs": 0,
"frameDurationMs": 21.211111111111112,
"p95FrameDurationMs": 33.400000000001455
},
{
"name": "vue-zoom-out-idle",
"durationMs": 6426.992999999925,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 6016.083,
"heapDeltaBytes": -29732616,
"heapUsedBytes": 181903264,
"domNodes": -3045,
"jsHeapTotalBytes": 7294976,
"scriptDurationMs": 94.663,
"eventListeners": -14915,
"totalBlockingTimeMs": 0,
"frameDurationMs": 21.515384615384622,
"p95FrameDurationMs": 33.400000000001455
},
{
"name": "workflow-execution",
"durationMs": 85.47100000004093,
"styleRecalcs": 6,
"styleRecalcDurationMs": 10.127,
"layouts": 2,
"layoutDurationMs": 0.637,
"taskDurationMs": 62.46199999999999,
"heapDeltaBytes": 3910988,
"heapUsedBytes": 69839016,
"domNodes": 97,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 6.992999999999999,
"eventListeners": 33,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "workflow-execution",
"durationMs": 85.741999999982,
"styleRecalcs": 7,
"styleRecalcDurationMs": 10.641000000000002,
"layouts": 2,
"layoutDurationMs": 1.152,
"taskDurationMs": 58.19499999999999,
"heapDeltaBytes": 3702668,
"heapUsedBytes": 69720728,
"domNodes": 114,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 5.061000000000001,
"eventListeners": 17,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.650000000000016,
"p95FrameDurationMs": 16.700000000000273
},
{
"name": "agent-composer-empty-conversation-typing",
"durationMs": 525.1749999999902,
"styleRecalcs": 78,
"styleRecalcDurationMs": 94.159,
"layouts": 76,
"layoutDurationMs": 24.613,
"taskDurationMs": 479.843,
"heapDeltaBytes": 14240816,
"heapUsedBytes": 87436944,
"domNodes": 76,
"jsHeapTotalBytes": 10747904,
"scriptDurationMs": 157.136,
"eventListeners": 876,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.667741935483864,
"p95FrameDurationMs": 16.700000000000273
},
{
"name": "agent-composer-empty-conversation-typing",
"durationMs": 457.62200000001485,
"styleRecalcs": 78,
"styleRecalcDurationMs": 85.83,
"layouts": 76,
"layoutDurationMs": 23.383999999999997,
"taskDurationMs": 429.54800000000006,
"heapDeltaBytes": 13727112,
"heapUsedBytes": 94367868,
"domNodes": 76,
"jsHeapTotalBytes": 13107200,
"scriptDurationMs": 138.28199999999998,
"eventListeners": 876,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.660714285714285,
"p95FrameDurationMs": 16.799999999999727
},
{
"name": "agent-composer-long-conversation-typing",
"durationMs": 820.096000000035,
"styleRecalcs": 78,
"styleRecalcDurationMs": 210.554,
"layouts": 76,
"layoutDurationMs": 25.356,
"taskDurationMs": 798.3509999999999,
"heapDeltaBytes": 9627372,
"heapUsedBytes": 139442208,
"domNodes": 76,
"jsHeapTotalBytes": 262144,
"scriptDurationMs": 203.046,
"eventListeners": 876,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666000000000004,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "agent-composer-long-conversation-typing",
"durationMs": 863.7979999999743,
"styleRecalcs": 78,
"styleRecalcDurationMs": 220.96699999999998,
"layouts": 76,
"layoutDurationMs": 26.188,
"taskDurationMs": 834.876,
"heapDeltaBytes": 10500048,
"heapUsedBytes": 131282976,
"domNodes": 76,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 217.70999999999998,
"eventListeners": 876,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66538461538462,
"p95FrameDurationMs": 16.799999999999272
}
]
}🔬 E2E Coverage
Top 10 files by uncovered lines
🔗 Website PreviewWebsite Preview: https://comfy-website-preview-pr-18610.vercel.app Models is built and gated by PostHog (workshop-enabled), using staging Cloud. This commit: https://website-frontend-gvh681336-comfyui.vercel.app Last updated: 2026-09-30T05:51:12Z for |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml Review profile: ASSERTIVE Plan: Team Run ID: 📒 Files selected for processing (2)
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review. Your free on-demand review promotion remains active until October 9, 2026 at 6:00 PM UTC. 📝 WalkthroughWalkthroughThis PR adds a Playwright fixture that mocks agent APIs and workflow CRDT WebSocket traffic for two sessions. Two tests check Alpha’s canvas while Bravo is displayed, after returning to Alpha, and after a later turn bound to Alpha. ChangesTwo-session workflow CRDT testing
Priority: ⬇️ Low Estimated code review effort: 3 (Moderate) | ~25 minutes Change: Other Merge Risk: ⚪ Minimal · up to This change adds browser test coverage only, so it does not alter product behavior. No merge-blocking risk is evident from the supplied context. 🚥 Pre-merge checks | ✅ 7✅ Passed checks (7 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Playwright videos for new testsDownload the recorded videos and HTML report Recorded projects: cloud Recorded test titles:
|
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #18610 +/- ##
==========================================
+ Coverage 84.48% 85.88% +1.39%
==========================================
Files 3345 2449 -896
Lines 161323 140142 -21181
Branches 43859 39152 -4707
==========================================
- Hits 136300 120366 -15934
+ Misses 23128 19399 -3729
+ Partials 1895 377 -1518
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Expected, and self-resolving — no change needed here. This PR deliberately targets In the meantime the change is not unreviewed. CI is green on
Worth noting for anyone reading the PR description: the three pre-existing widget-based tests in this spec that I reported as failing in my local sandbox pass in CI. That confirms they were environmental, as claimed — the total is 0 failed — and rules out this commit having disturbed them. The change also went through five rounds of adversarial review before opening, which is where the two scope caveats in the description came from (only one of the two reported empty canvases is reproducible; the refresh leg is a distinct defect from the tab return). Those still stand and are the most useful thing for a human reviewer to push on. |
There was a problem hiding this comment.
🔍 Cursor Review — Consolidated panel
Triggered by @christian-byrne.
Found 10 finding(s).
| Severity | Count |
|---|---|
| 🟡 Medium | 5 |
| 🟢 Low | 4 |
| ⚪ Nit | 1 |
Panel: 8/8 reviewers contributed findings.
| test.setTimeout(120_000) | ||
| await displacedBuild(twoSessionCrdt) | ||
|
|
||
| test.fail() |
There was a problem hiding this comment.
🟡 Medium — Under test.fail() every failure of this poll counts as expected, so the test stays green if Alpha renders another workflow's nodes, the page crashes, or the canvas never mounts — not just on the known empty-canvas defect (same at line 371). Assert the observed state positively before the marker, e.g. await expect(twoSessionCrdt.vueNodes.nodes).toHaveCount(0), so only the missing build is accepted as the expected failure. Raised by 4 of 8 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case, gpt-5.6-sol-max adversarial, gpt-5.6-sol-max edge-case).
| test('the displaced build survives a refresh of its tab', async ({ | ||
| twoSessionCrdt | ||
| }) => { | ||
| test.setTimeout(120_000) |
There was a problem hiding this comment.
🟡 Medium — A test that exhausts its budget is recorded as timedOut, which does not match the failed status test.fail() expects, so it reports as an unexpected failure. This test must burn the full 20s of a poll that is expected never to match, on top of displacedBuild (a full boot plus two agent turns) and reload() (a second boot with three 30s waits), all inside 120s — much tighter than the sibling test.fail() tests, so a slow CI run turns a known-bug pass into a red build. Raised by 1 of 8 reviewers (claude-opus-5-thinking-max edge-case).
| const objectInfo = this.page.waitForResponse((response) => | ||
| new URL(response.url()).pathname.endsWith('/api/object_info') | ||
| ) | ||
| await this.page.reload() |
There was a problem hiding this comment.
🟡 Medium — page.reload() takes Playwright's default waitUntil: 'load' and 30s navigation timeout, but this app's own boot budget is 45s (CLOUD_APP_BOOT_TIMEOUT in cloudAppFixture), so a slow CI boot throws here before any of the explicit waits below are reached. The sibling helper agentConversationFixture.reloadWithoutLocalWorkflow uses page.reload({ waitUntil: 'domcontentloaded' }) for exactly this reason. Raised by 1 of 8 reviewers (claude-opus-5-thinking-max edge-case).
| // Startup restores the workflow tabs after extensionManager exists, so | ||
| // the overlay is the boundary for "the tabs are back". | ||
| const loadingOverlay = this.page.getByTestId(TestIds.app.loadingOverlay) | ||
| await loadingOverlay.waitFor({ |
There was a problem hiding this comment.
🟡 Medium — The attached-then-hidden pair is not the tab-restoration barrier the comment claims, under either rendering shape: if the overlay is v-show-rendered it is attached from the first render and hidden holds whenever workspaceStore.spinner is false, so both waits can pass before restoreWorkflowTabsState() even starts; if it is conditionally rendered, a fast mocked boot can unmount it before waitFor({ state: 'attached' }) observes it, timing out after 30s on a healthy app. Wait on a deterministic tab-restored signal instead. Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k3-max edge-case).
| // workflowStore.activeWorkflow, so seeing Alpha checked is what rules | ||
| // out a click the app never processed — without which "the canvas | ||
| // filled" below could pass on an app that never heard it. | ||
| await twoSessionCrdt.topbar.getWorkflowTab(WORKFLOW_A.name).click() |
There was a problem hiding this comment.
🟡 Medium — displacedBuild() ends by clicking back to Alpha, which is itself the tab-return recovery under test, so once that path is fixed Alpha is already materialized before the refresh and later-turn scenarios begin. Those tests could then pass from restored local state even if reload or turn-bound catch-up is still broken; keep Alpha inactive and unmaterialized until each scenario performs its own recovery action. Raised by 1 of 8 reviewers (gpt-5.6-sol-max adversarial).
| */ | ||
| async reload(): Promise<void> { | ||
| const socket = this.socket | ||
| const objectInfo = this.page.waitForResponse((response) => |
There was a problem hiding this comment.
🟢 Low — The objectInfo barrier does not order what its comment claims: it is awaited at line 309, after waitForCloudApp and the overlay waits, by which point boot, subscribe and any catch-up materialization have already happened. It also carries Playwright's default 30s response timeout counted from creation while up to ~105s of waits sit before the await, so a slow boot (or any earlier await throwing) leaves a rejected promise with no handler and surfaces as an unhandled rejection at an arbitrary point in the run. Await it immediately after page.reload() or via Promise.all with the other waits. Raised by 3 of 8 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case, kimi-k3-max edge-case).
There was a problem hiding this comment.
Addressed in 6a40cf8: boot() now awaits Promise.all([objectInfoLoaded, bootAgentApp(...)]), so a rejection from the object_info waiter is always observed.
| await displacedBuild(twoSessionCrdt) | ||
|
|
||
| await twoSessionCrdt.reload() | ||
| await expect(twoSessionCrdt.topbar.getActiveTab()).toContainText( |
There was a problem hiding this comment.
🟢 Low — After the reload this checks only the tab's label, not that the restored tab still resolves to WORKFLOW_A.id; with **/api/workflows** mocked as an empty list and mockWorkflowPersistence knowing only HOME_WORKFLOW_ID, a restored "Alpha" tab may carry no cloud id at all. The test would then be red for a reason unrelated to the subscription bug it pins, and test.fail() hides that — asserting subscribeCount(WORKFLOW_A.id) would separate "bound to Bravo, never subscribed" from "tab has no cloud id to subscribe". Raised by 1 of 8 reviewers (claude-opus-5-thinking-max edge-case).
| // Alpha's build lands while Bravo is the tab on screen. Only Alpha's | ||
| // document ever receives a node, so anything a canvas shows below can | ||
| // only have come from Alpha. | ||
| twoSessionCrdt.hostEdit(alpha, BUILD_OPS) |
There was a problem hiding this comment.
🟢 Low — Alpha's build is broadcast while Bravo is the tab on screen, but nothing checks Bravo's canvas before the switch, so a regression that projects Alpha's nodes onto Bravo and clears them during the tab change still satisfies every downstream assertion. Add await expect(twoSessionCrdt.vueNodes.nodes).toHaveCount(0) before clicking Alpha's tab. Raised by 1 of 8 reviewers (gpt-5.6-sol-max edge-case).
| await twoSessionCrdt.boot() | ||
| await twoSessionCrdt.runBoundTurn(BUILD_PROMPT, alpha) | ||
| await twoSessionCrdt.newChat() | ||
| await twoSessionCrdt.runBoundTurn(BUILD_PROMPT, bravo) |
There was a problem hiding this comment.
🟢 Low — Both turns are sent with the identical BUILD_PROMPT, which weakens startThread's readiness check (panel.getByText(prompt).first() visible, whose purpose is to prove the page recorded the new turn): a transcript that newChat() has not finished clearing satisfies it via thread one's bubble. The earlier tests in this file use distinct prompts ("Open Alpha" / "Open Bravo") to avoid this. Raised by 1 of 8 reviewers (claude-opus-5-thinking-max edge-case).
|
|
||
| /** The node ids a workflow's host doc currently holds, ascending. */ | ||
| hostNodeIds(bound: AgentBoundWorkflow): string[] { | ||
| return Object.keys(bound.host.graph().nodes).sort() |
There was a problem hiding this comment.
⚪ Nit — The default Array.prototype.sort() compares as strings, so this returns lexicographic order despite the doc comment's "ascending". It matches numeric order only because today's ids are all 16 digits; the template seed this harness ships with has single-digit node ids, so the first caller that mixes widths gets '10' before '3' and a confusing toEqual diff. Pass a numeric comparator or soften the claim to "sorted". Raised by 3 of 8 reviewers (claude-opus-5-thinking-max adversarial, claude-opus-5-thinking-max edge-case, kimi-k3-max edge-case).
DrJKL
left a comment
There was a problem hiding this comment.
Verdict: request changes. The PM-1535 test cannot fail for PM-1535: its conditional test.fail stays green while the canvas is empty today and also after a fix regresses.
I reviewed d8514a0 against main (merge base 4bc1719). ESLint, type-aware oxlint, oxfmt and pnpm typecheck:browser pass locally on both files. All hosted checks pass at this head, including the four cloud Playwright shards. I could not run the cloud spec here because it needs a ComfyUI backend.
Pre-merge blockers:
- P1
agentTwoSessionCrdtSubscription.spec.ts:116:test.fail(canvas.length === 0, ...)computes the expected status from the result. With the bug present the test passes as an expected failure. Once PM-1535 is fixed it passes for real and nobody is told to remove the marker. If the fix later regresses, the condition is true again and the test is green again. I ran the three cases through Playwright 1.61.1: only an unconditional marker reportsExpected to fail, but passed. Inline comment has a three-line fix that keeps the wrong-node hard failure.
The other 17 inline comments are P2 and P3 items: dead #18088 fixture API, a second fake /ws host next to AgentFollowerHostSocket, an untyped POST ack, and smaller test and comment cleanups.
Findings without an anchorable line (2)
- P2: This branch absorbs #18088 and deletes its tests.
pr-18088atdf40cb1is an ancestor of this head (git merge-base --is-ancestorsucceeds).11c4ccfthen removes all three #18088 cases:returning to the first session tab shows the edit made while away,a human edit on the bound active tab reaches its doc, anda human edit on the returned-to tab is dropped, never reaching its doc. The fixture API those cases needed stays (hostWidgetValue,docOpsCount,outboundDocOps, the humandoc_opsjudge). If this merges first, #18088's head becomes reachable frommainwith its PM-1319 pins deleted and its fixture kept. The description says only "targetsmaindirectly and carries two focused cases" and does not mention #18088. Please decide which PR owns these paths. Either restack this on #18088 so the diff is only the PM-1535 cases, or close #18088 on purpose and delete the API that only its tests used (see the inline comment atagentTwoSessionCrdtFixture.ts:290). - P3: The description does not follow
.github/pull_request_template.md.ChangesandReview Focusare missing, and there is no separate issue line.Fixeswould be wrong here because the PR says it must not close PM-1535, so a line likeRelated: PM-1535keeps that intent.
Existing findings confirmed or contested (13)
- Cursor, unconditional
test.failaccepts any failure (r4087273624): confirmed ata4a5a85.ff1df0amade the marker conditional, so a wrong or partial canvas now fails outright. That change creates the P1 above: an empty canvas is accepted forever. - Cursor, refresh test would report
timedOutundertest.fail(r4087273629): confirmed at the time. The refresh case was deleted in11c4ccf, so nothing remains to fix. - Cursor,
page.reload()defaultload/30s against a 45s boot (r4087273637): confirmed at the time.rg page.reloadfinds nothing in the fixture at this head. - Cursor, overlay attached-then-hidden is not a tab-restore barrier (r4087273643): confirmed at the time. The waits went away with the reload helper.
- Cursor,
displacedBuildclicked back to Alpha before later scenarios (r4087273649): fixed.displacedBuild(spec:66-94) now returns without touching Alpha's tab, and the recovery test never clicks it. - Cursor,
objectInfowaiter awaited late (r4087273655): still applies at the surviving site.boot()creates the waiter at fixture:161 and awaits it only at fixture:173, afterbootAgentApp(fixture:164-170). If boot throws or runs past the waiter's timeout, nothing handles that rejection. I traced this but did not reproduce it. To settle it, makebootAgentAppfail before/api/object_infoand keep the worker alive past the timeout.await Promise.all([bootAgentApp(...), objectInfo])covers both paths.agentConversationFixture.ts:304-320has the same shape, so this is P3. - Cursor, post-reload check only reads the tab label (r4087273668): superseded. The reload case is gone.
- Cursor, Bravo's canvas unchecked before the switch (r4087273675): addressed by spec:91, but that check passes on its first sample before Alpha's frame can arrive. See the inline comment there.
- Cursor, identical prompts weaken
startThreadreadiness (r4087273687): fixed. The prompts end in "in Alpha" and "in Bravo", and neither is a substring of the other. - Cursor, default
sort()is lexicographic (r4087273696): fixed. BothhostNodeIdsandcanvasNodeIdsuseNumber(left) - Number(right). - CodeRabbit,
toEqual([])encodes the defect (r4141284912): fixed. The test assertsBUILT_NODE_IDS. The remaining gap is the P1. - CodeRabbit, move the settling helper out of the spec (r4141845123): fixed.
canvasNodeIdsWhenSettledis on the harness. - CodeRabbit, poll for the whole build, not the first node (r4141845131): fixed. The poll compares against the full list. The helper still returns a second read instead of the polled value (inline at fixture:336).
Checked and cleared
- Fallow reports
typescript-native-bridgeas an unused devDependency. The PR does not touchpackage.json, so this is not introduced here. - The spec says current-main golden-path coverage already proves a bound, active tab materializes agent-added nodes. That holds:
agentRemoteApplyReachesCanvas.spec.ts:18-44andcmpDocumentAcceptance.spec.ts:23-41cover it. - The KNOWN BUG mechanism matches product code. A tab click changes the active workflow but not
boundWorkflowId, and on deactivation the follower callsbridge.unsubscribe()(useAgentCrdtFollower.ts:690-790,useAgentSession.ts:941-976). - The recovery test earns its place. It fails if a later bound turn does not subscribe Alpha and replay the catch-up, and it is the only thing showing that the expected failure in the first test is not vacuous.
- Isolation holds. The fixture builds a fresh harness per test, and all maps, the socket, and
threadCounterare instance state. - The
objectInfowait is not redundant.bootAgentAppwaits for app readiness, not for node definitions. waitForSubscribecannot double-resolve, and a timed-out waiter removes itself. The live-array iteration at fixture:532 would skip a second waiter on the same workflow, but no current caller creates one.
Coverage
Depth standard: 727 added lines in two test files, no risk domain beyond async E2E timing. Five lanes ran as separate reviewers: correctness, maintainability, idiomaticity, tests, and documentation. Lenses run: blast-radius, regression-risk-reviewer (no pre-image lines, both files are new), idempotence, shared-state, boundary-discipline, fix-root-causes, ponytail, no-comments, minimize-reader-load, subtract-before-you-add, laziness-protocol, migrate-callers, typescript-best-practices, type-system-discipline, removing-typescript-suppressions, designing-test-suites, reviewing-playwright-specs (helper inventory and rulebook), writing-playwright-tests, hardening-flaky-e2e-tests, test-mock-quality-analysis, consolidating-test-setup, prove-it-works, simplifying-technical-documents, technical-writing. No lane was skipped. Fallow ran diff-scoped against the merge base. Not run: the cloud spec itself (no backend in my environment), and the unhandled-rejection experiment in item 6 above.
| // KNOWN BUG (PM-1535): the follower's subscribe target is the session's | ||
| // boundWorkflowId, still Bravo after thread two bound it, so returning to | ||
| // Alpha unsubscribes rather than resubscribing and Alpha's build is never | ||
| // projected. Only the empty canvas is expected: nodes that arrive but do | ||
| // not match fail this outright, and once the follower resubscribes the | ||
| // active tab's workflow the marker goes inert and can be deleted. | ||
| test.fail( | ||
| canvas.length === 0, | ||
| 'PM-1535: returning to Alpha does not resubscribe its document' | ||
| ) | ||
| expect(canvas).toEqual(BUILT_NODE_IDS) |
There was a problem hiding this comment.
P1: This marker takes its expected status from the result, so the test can never go red for PM-1535.
- Today the canvas is
[], the condition is true, and the failingtoEqualcounts as a pass. - After a fix the condition is false and the test passes. Playwright only prints
Expected to fail, but passedwhen the marker is armed, so nothing tells the fixer to delete it. - If the fix later regresses, the canvas is
[]again, the marker re-arms, and CI stays green.
I checked all three with Playwright 1.61.1 (workerProcessEntry.js:982-994 sets expectedStatus = 'failed' only when the condition is true). Taken together, the test accepts [] or BUILT_NODE_IDS and rejects only a wrong non-empty canvas. You can keep that rejection and still get a signal when the fix lands. Hard-assert the two allowed outcomes before an unconditional marker:
| // KNOWN BUG (PM-1535): the follower's subscribe target is the session's | |
| // boundWorkflowId, still Bravo after thread two bound it, so returning to | |
| // Alpha unsubscribes rather than resubscribing and Alpha's build is never | |
| // projected. Only the empty canvas is expected: nodes that arrive but do | |
| // not match fail this outright, and once the follower resubscribes the | |
| // active tab's workflow the marker goes inert and can be deleted. | |
| test.fail( | |
| canvas.length === 0, | |
| 'PM-1535: returning to Alpha does not resubscribe its document' | |
| ) | |
| expect(canvas).toEqual(BUILT_NODE_IDS) | |
| expect([[], BUILT_NODE_IDS]).toContainEqual(canvas) | |
| test.fail( | |
| true, | |
| 'PM-1535: returning to Alpha does not resubscribe its document' | |
| ) | |
| expect(canvas).toEqual(BUILT_NODE_IDS) |
An empty canvas stays an expected failure. A wrong or partial canvas fails at the first assertion, before the marker is armed. A fixed canvas reports an unexpected pass, which makes the fix PR delete the marker, and from then on an empty canvas fails normally. This also replaces the KNOWN BUG paragraph, which describes the marker going inert.
There was a problem hiding this comment.
You're right, and the third bullet is the one that convinced me: a regression after the fix would re-arm the marker and stay green. A regression test that cannot report a regression is worse than useless. Taken your shape verbatim in 9069969.
I verified all three transitions against a local backend by stubbing the canvas reader, rather than reasoning from the Playwright source:
| Canvas after the tab return | Result |
|---|---|
[] (today) |
2 passed — expected failure |
['99999999999999'] |
1 failed at toContainEqual, before the marker arms |
BUILT_NODE_IDS (fix simulated) |
1 failed — "Expected to fail, but passed." |
That third row is exactly what my version could not produce, and it is what makes the fix PR delete the marker. Once deleted, expect(canvas).toEqual(BUILT_NODE_IDS) is a plain assertion and a regression fails normally — the property my version lost.
Rewrote the KNOWN BUG paragraph accordingly: it no longer claims the marker "goes inert", it says the unexpected pass is the signal to delete those three lines and leave the assertion behind as the regression guard.
Worth noting this is the second time the marker shape has been wrong in this PR, in opposite directions: CodeRabbit flagged toEqual([]) encoding the defect, I over-corrected into a self-satisfying condition, and this is the shape that actually holds in all four states. Thanks for working it through with the worker source — I would not have got there from the docs.
| twoSessionCrdt.hostEdit(alpha, BUILD_OPS) | ||
| expect(twoSessionCrdt.hostNodeIds(alpha)).toEqual(BUILT_NODE_IDS) | ||
| expect(twoSessionCrdt.hostNodeIds(bravo)).toEqual([]) | ||
| await expect(twoSessionCrdt.vueNodes.nodes).toHaveCount(0) |
There was a problem hiding this comment.
P3: This does not prove Bravo stayed empty. hostEdit (fixture:286-288) only calls socket.send() and returns. The canvas is already empty before Alpha's doc_update reaches the page, so toHaveCount(0) passes on its first sample whatever the follower later does with that frame. That is the same objection you raised to CodeRabbit's toHaveCount(0) suggestion. agentConversationFixture.ts:1172-1227 already has a delivery receipt (sendHostFrame(frame, true) waits for the app's doc_update event with the matching workflow_id and seq). Awaiting that before this line would turn it into a real check. Otherwise I would drop the line and the matching sentence in the description.
There was a problem hiding this comment.
Dropped the line and the matching sentence in 9069969. You're right, and it is the same objection I raised against the identical suggestion from CodeRabbit — I should have applied it to the line already sitting in my own arrange.
I went with removal rather than wiring up the delivery receipt deliberately. Making it real means adding a receipt path to this harness, and the sibling thread on duplication is pushing the other way; I would rather this harness shrink than grow a second copy of agentConversationFixture's receipt machinery for one assertion. If the leak-onto-the-visible-tab case is worth pinning, it deserves its own test with the real receipt, not a line in a shared arrange.
| expect(twoSessionCrdt.hostNodeIds(alpha)).toEqual(BUILT_NODE_IDS) | ||
| expect(twoSessionCrdt.hostNodeIds(bravo)).toEqual([]) |
There was a problem hiding this comment.
P3: These two lines check the fake, not the app. Bravo's host was seeded with emptySeed() and nothing ever writes to it, so line 90 cannot fail. Line 89 is there to catch the recording drifting away from BUILT_NODE_IDS, but the recovery test already fails in that case: its canvas would show the drifted ids and its toEqual(BUILT_NODE_IDS) at spec:138 would fail. I would delete both. That also removes the last use of hostNodeIds.
There was a problem hiding this comment.
Both deleted in 9069969, and hostNodeIds with them.
Your reasoning on the drift guard holds: if the recording drifts, the recovery test renders the drifted ids and its toEqual(BUILT_NODE_IDS) fails, so the assertion in the arrange was buying nothing the suite did not already have. And hostNodeIds(bravo) could not fail — nothing writes to Bravo's host — so it was asserting the fake's own initialisation.
| // test; rethrowing here would pre-empt the caller's decision. | ||
| void neverMatched | ||
| } | ||
| return this.canvasNodeIds() |
There was a problem hiding this comment.
P3: This returns a second read, not the value the poll saw. If the poll matched and the canvas changed before this canvasNodeIds() call, the caller classifies the later value. For example, a build that renders and is then cleared comes back as [] and is silently marked as the expected failure. Capture the value inside the poll callback, for example let observed: string[] = [] with observed = await this.canvasNodeIds(); return observed, and return observed after the try.
There was a problem hiding this comment.
Agreed, real bug — fixed in 9069969. The value the poll matched and the value returned were two different reads, so a build that rendered and was then cleared came back [] and got misfiled as the expected failure, which is precisely the case the helper exists to distinguish.
Now captures inside the callback:
let observed: string[] = []
try {
await expect
.poll(async () => {
observed = await this.canvasNodeIds()
return observed
}, { timeout })
.toEqual(expected)
} catch {
// Never matched. That is an outcome under test, so report what the poll
// actually saw instead of re-reading a canvas that may have moved on.
}
return observedThis also composes with the marker change in the sibling thread: a cleared canvas now surfaces as whatever was last seen and gets judged by toContainEqual, instead of silently becoming the "empty" branch.
| try { | ||
| await expect | ||
| .poll(() => this.canvasNodeIds(), { timeout }) | ||
| .toEqual([...expected]) |
There was a problem hiding this comment.
Nit (P3): The spread is unnecessary. toEqual(expected: unknown) (Playwright test.d.ts:8317) accepts the readonly array as-is, so this copies the array on every poll for no reason.
| .toEqual([...expected]) | |
| .toEqual(expected) |
There was a problem hiding this comment.
Addressed in 9069969: the poll now compares with .toEqual(expected) and no longer copies the array.
| await page.route('**/api/agent/threads', (route) => | ||
| route.fulfill(jsonRoute({ threads: [] })) | ||
| ) | ||
| await page.route('**/api/agent/threads/*/messages', (route) => { |
There was a problem hiding this comment.
P3: Fallow flags this as clone dup:6f87acd9. The /api/agent/threads and threads/*/messages routes also exist in agentConversationFixture.ts:672-677 and load3dAgentFixture.ts:276-281, and the three copies have already drifted in their POST and history handling. agentPanelFixture owns the other agent API mocks, so a shared helper there, with a callback for the POST ack, would keep the next contract change to one place.
There was a problem hiding this comment.
Same answer as the AgentFollowerHostSocket thread, and I'd group them: the shared helper belongs in agentPanelFixture, which already owns the other agent API mocks, and it changes behaviour for agentConversationFixture and load3dAgentFixture as well as this one. You note the three copies have already drifted in their POST and history handling, which means the consolidation is not a pure move — someone has to decide which POST semantics are canonical and re-verify the two existing consumers against that choice. That is a decision and a test run I don't think should ride along inside a bug-pinning PR.
The mocks here did get a little less divergent in 9069969: the POST ack is now built as a typed AgentTurnAccepted and parsed with zAgentTurnAccepted on the way back, so this copy is at least pinned to the generated contract rather than a hand-written shape.
Happy to file the consolidation as a follow-up covering both clone sites, or to fold it in here if you'd prefer it not land twice.
| ) | ||
| await bootAgentApp(this.page, true, { | ||
| settings: { | ||
| 'Comfy.VueNodes.Enabled': true, |
There was a problem hiding this comment.
P3: The suite is tagged @vue-nodes, and mockAgentBoot already sets Comfy.VueNodes.Enabled when that tag is present (agentPanelFixture.ts:77-79). .agents/checks/playwright-e2e.md item 11 says "Never manually set Comfy.VueNodes.Enabled". This line gives the renderer a second owner.
| 'Comfy.VueNodes.Enabled': true, |
There was a problem hiding this comment.
Removed in 9069969. Confirmed the tag path covers it: mockAgentBoot sets Comfy.VueNodes.Enabled when cloudAppFixture.info().tags.includes('@vue-nodes') (agentPanelFixture.ts:70-72), and this describe carries @vue-nodes. boot() now passes only Comfy.Graph.CanvasInfo: false.
Verified rather than assumed — the suite still passes with the line gone, so the tag really is supplying it.
| return alpha | ||
| } | ||
|
|
||
| test('returning to the displaced build shows it on the canvas', async ({ |
There was a problem hiding this comment.
P3: Each test runs boot, an Alpha turn, New chat, a Bravo turn, the host edit, a tab switch or rebind, and a 20s settle, with no test.step. When one times out, the report points at a fixture line instead of the phase the user was in. Wrapping the phases inside displacedBuild and the two acts in test.step (bind Alpha, bind Bravo, Alpha's build lands, return to Alpha, rebind Alpha) would make CI failures readable without the source.
There was a problem hiding this comment.
Added in 9069969. displacedBuild is now three steps — "thread one binds Alpha", "thread two binds Bravo, displacing Alpha", "thread one's build lands while Bravo is on screen" — and each test wraps its own act, "user clicks back to Alpha" and "a later turn rebinds Alpha".
The step names double as the narrative the prose docblock used to carry, which is part of how that block got shorter.
|
|
||
| test.describe( | ||
| 'Agent build displaced by a second thread', | ||
| { tag: ['@cloud', '@agent', '@vue-nodes'] }, |
There was a problem hiding this comment.
Nit (P3): Both tests take well over 10s, and the first always spends its full 20s settle budget today. The README's organizational @slow tag fits here. 120_000 and 20_000 are also each repeated. test.describe.configure({ timeout: 120_000 }) and one named settle constant would remove the duplicates.
There was a problem hiding this comment.
All three done in 9069969: @slow added to the describe tags, test.describe.configure({ timeout: 120_000 }) replaces the two per-test setTimeout calls, and the repeated 20_000 is now a named SETTLE_TIMEOUT.
| // The cloud id the boot tab ("Unsaved Workflow") receives when the composer | ||
| // pins it as the first target. The composer refuses to send without a target | ||
| // (useAgentDraftSubmission), so every thread starts here before the agent's | ||
| // `agent_active_tab` moves it onto Alpha or Bravo. Distinct from both. |
There was a problem hiding this comment.
P3: "every thread starts here" is only true for the first thread. selectHomeTarget() runs once in boot(), newChat() does not run it again, and useAgentDraftSubmission only needs some target, not this id. The second thread in displacedBuild starts after newChat() from whatever target the composer kept. "The first thread starts here" would be accurate.
There was a problem hiding this comment.
Correct — fixed in 9069969. selectHomeTarget() runs once inside boot(), newChat() does not re-pin, and the second thread starts from whatever target the composer kept. The comment now reads:
The cloud id the boot tab ("Unsaved Workflow") receives when
boot()pins it once as the composer's first target; the composer refuses to send without one (useAgentDraftSubmission). Later threads reuse whatever target the composer kept. Distinct from every workflow a test registers.
Marker (P1): deriving the expected status from the result meant the test could never go red for PM-1535 — a fix produced a silent pass with nothing telling the fixer to remove the marker, and a later regression re-armed it and stayed green. Hard-assert the two allowed outcomes, then arm the marker unconditionally: a wrong canvas fails before the marker, an empty one is the expected failure, and the fix reports an unexpected pass, which is the signal to delete it. Settled reader returned a second read rather than the value the poll saw, so a build that rendered and was then cleared came back empty and was misfiled as the expected failure. Capture inside the callback. Dead harness code: hostWidgetValue, hostNodeIds, docOpsCount, outboundDocOps and the doc_ops judge had no caller here. Instrumenting the socket shows these flows send only doc_subscribe, doc_unsubscribe and feature_flags — no doc_ops at all — so the judge and its casts go too. waitForSubscribe's waiter registry collapses to expect.poll. Also: parse the turn ack with zAgentTurnAccepted instead of hand-typing response.json(); reuse the existing emptySeed and require the seed; drop the manual Comfy.VueNodes.Enabled that the @vue-nodes tag already owns; add test.step phases and @slow; and delete or correct the comments that described #18088's widget-edit coverage, which this branch no longer has.
This comment has been minimized.
This comment has been minimized.
DrJKL
left a comment
There was a problem hiding this comment.
Verdict: comment. 9069969 fixes the P1 marker and 16 of my 18 inline findings, and I accept the other two as follow-ups. Before I approve, #18088's ownership needs a decision and the description needs updating.
The new marker shape behaves as intended. An empty canvas is an expected failure. A wrong or partial canvas fails at toContainEqual before the marker is armed. A fixed canvas reports Expected to fail, but passed. Your stubbed run of all three matches what I traced in the code. The harness is also much smaller: the doc_ops leg, the waiter registry, and the #18088-only readers are gone. Four new inline comments cover small things in the new code: one P3 where a canvas reader that keeps throwing gets classified as the expected failure, and three nits.
Checks at this head: ESLint, type-aware oxlint, and oxfmt pass on both files, pnpm typecheck:browser passes, and playwright test --list --project=cloud lists both tests. The hosted cloud Playwright shards pass. The one failing hosted job, unit test-shards (4), fails in src/platform/workspace/billing/sdk/billingSdkTelemetryOwnership.test.ts. This PR does not touch that file, and main updated its expectation in #19648, so the failure is unrelated here.
Still open (3)
- P2, #18088 ownership (round-1 body item 1, no response yet).
pr-18088is still an ancestor of this head, #18088 is still open, and this branch still deletes its three PM-1319 cases in11c4ccf. If this merges first, #18088's head is reachable frommainwith its tests gone. That is fine if it is deliberate. Please either close #18088 with a note that this PR supersedes it and that the PM-1319 pins are dropped (or will come back separately), or restack this PR on it. I am not asking for more code, only for the decision to be recorded on one of the two PRs. - P3, description (round-1 body item 2, plus new staleness). The template sections and a
Related: PM-1535line are still missing. The body now also describes code that no longer exists. It says the first test "is expected to fail only when the settled canvas is empty" and that "a PM-1535 fix passes without inverting the oracle", but the fix now produces an unexpected pass that the fix PR must clear. It says "the arrangement also proves Alpha's host holds the three-node build, Bravo's host is empty, and Bravo's visible canvas remains empty", but those assertions were removed. The Verification section still citesd8514a0. - P3,
objectInfoawaited afterbootAgentApp(round-1 existing item 6, Cursor r4087273655). This is unchanged at fixture:142-151.agentConversationFixture.ts:304-320has the same shape, so I am fine leaving it.await Promise.all([bootAgentApp(...), objectInfo])is the change if you want it.
Deferred to follow-up, accepted (2)
- P2, second fake
/wshost next toAgentFollowerHostSocket(r4147830138). Your reply (r4148010359) makes a fair point: nine consumers depend on that class's shape, and dropping thedoc_opsleg took this copy down to subscribe/catch-up plus the workflow map. A follow-up is fine. Please file it and link it here, so the second copy has an owner once it lands. - P3, third copy of the agent thread route mocks (r4147830179). I agree that picking the canonical POST semantics is its own decision (r4148012440). The same follow-up can cover it.
Addressed at 9069969 (16)
- P1, conditional
test.fail(r4147830055): now atoContainEqualguard followed bytest.fail(true, ...). Author replied. - P2, POST ack typed by hand (r4147830119): parsed with
zAgentTurnAccepted, and the mock is typedAgentTurnAccepted. Author replied. - P2, #18088-only fixture API (r4147830132): readers, recorder, judge and both casts deleted. The socket log shows no
doc_opsin these flows. Author replied. - P3, Bravo
toHaveCount(0)(r4147830084): line removed. Author replied. - P3, host-state assertions (r4147830095): removed along with
hostNodeIds. Author replied. - P3, second canvas read (r4147830103): the poll callback now captures
observed. Author replied. - P3, waiter registry (r4147830148): now
expect.pollwith a rethrow. You were right thatmessagecannot be a callback. Author replied. - P3, duplicate
emptySeedand unused default (r4147830155): imports the shared constant andseedis required. Author replied. - P3, narration comments (r4147830161): the docblock is down to six lines. I accept the three comments you kept and your reasons for them. Author replied.
- P3, docblocks describing #18088 coverage (r4147830167): both rewritten. Author replied.
- P3, manual
Comfy.VueNodes.Enabled(r4147830189): removed. Author replied. - P3, no
test.step(r4147830197): five steps added. Author replied. - P3, "every thread starts here" (r4147830209): comment corrected. Author replied.
- Nit, spread of
expected(r4147830110): removed. My reply: r4148260074. - Nit,
catch (neverMatched)(r4147830114): nowcatch {}. My reply: r4148260479. - Nit,
@slowand repeated budgets (r4147830203): tag added,describe.configure, andSETTLE_TIMEOUT. Author replied.
Checked and cleared
- Unit
test-shards (4)failure: not introduced here (see above). - The
test.stepwrapping a synchronoushostEditcompiles and runs;test.stepaccepts a sync body. - Sharing the
emptySeedconstant between twoHostDocs is safe. The constructor passes the seed tomintand keeps only the resulting doc (agentConversationHostDoc.ts:81-87).
| let observed: string[] = [] | ||
| try { | ||
| await expect | ||
| .poll( | ||
| async () => { | ||
| observed = await this.canvasNodeIds() | ||
| return observed | ||
| }, | ||
| { timeout } | ||
| ) | ||
| .toEqual(expected) | ||
| } catch { | ||
| // Never matched. That is an outcome under test, so report what the poll | ||
| // actually saw instead of re-reading a canvas that may have moved on. | ||
| } | ||
| return observed |
There was a problem hiding this comment.
P3: catch {} catches more than the timeout. If canvasNodeIds() throws on every attempt (the page closed, evaluateAll failed), expect.poll retries until the timeout and then throws. observed keeps its initial [], the spec's toContainEqual accepts it, and the test reports an expected failure. I ran that shape with a reader that always throws, and Playwright reported 1 passed. That lets a harness failure pass as PM-1535, which is what the guard at spec:93 was added to prevent. Only treat the timeout as an outcome if at least one read succeeded. The docblock above already says what happens on timeout, so the comment in the catch can go too:
| let observed: string[] = [] | |
| try { | |
| await expect | |
| .poll( | |
| async () => { | |
| observed = await this.canvasNodeIds() | |
| return observed | |
| }, | |
| { timeout } | |
| ) | |
| .toEqual(expected) | |
| } catch { | |
| // Never matched. That is an outcome under test, so report what the poll | |
| // actually saw instead of re-reading a canvas that may have moved on. | |
| } | |
| return observed | |
| let observed: string[] | undefined | |
| try { | |
| await expect | |
| .poll( | |
| async () => { | |
| observed = await this.canvasNodeIds() | |
| return observed | |
| }, | |
| { timeout } | |
| ) | |
| .toEqual(expected) | |
| } catch (error) { | |
| if (observed === undefined) throw error | |
| } | |
| return observed ?? [] |
I applied this locally: ESLint, type-aware oxlint and typecheck:browser pass. TypeScript does not track the assignment inside the closure, which is why the return needs ?? [].
There was a problem hiding this comment.
Addressed in c61b350: observed starts as undefined and the catch rethrows unless at least one read succeeded, so a reader that always throws now fails the test instead of passing as the expected failure.
| // KNOWN BUG (PM-1535): the follower's subscribe target is the session's | ||
| // boundWorkflowId, still Bravo after thread two bound it, so returning | ||
| // to Alpha unsubscribes rather than resubscribing. The marker is | ||
| // unconditional on purpose: when the fix lands this reports an | ||
| // unexpected pass, which is the signal to delete these three lines and | ||
| // leave the assertion guarding against regression. |
There was a problem hiding this comment.
Nit (P3): "delete these three lines" is ambiguous. The test.fail call is four lines as formatted, and after the fix the toContainEqual guard at 93 is redundant too, because toEqual(BUILT_NODE_IDS) alone rejects both an empty and a wrong canvas. Naming them saves the fixer from guessing: "delete the test.fail call and the toContainEqual guard, and keep the final assertion".
There was a problem hiding this comment.
Addressed in c61b350: the KNOWN BUG comment now says to delete the test.fail call and the toContainEqual guard and keep the final assertion.
| const WORKFLOW_B = { id: '22222222-2222-4222-8222-222222222222', name: 'Bravo' } | ||
|
|
||
| const BUILD_PROMPT = 'Build a basic text to image workflow' | ||
| // How long the canvas gets to show a build before a test reads it. |
There was a problem hiding this comment.
Nit (P3): The constant name already says this.
| // How long the canvas gets to show a build before a test reads it. |
There was a problem hiding this comment.
Addressed in c61b350: the comment above SETTLE_TIMEOUT is gone.
| readonly panel: Locator | ||
| readonly agentPanel: AgentPanel | ||
| readonly vueNodes: VueNodeHelpers | ||
| readonly topbar: Topbar |
There was a problem hiding this comment.
Nit (P3): The spec only calls topbar, addWorkflow, boot, runBoundTurn, newChat, hostEdit, canvasNodeIds and canvasNodeIdsWhenSettled. panel, agentPanel and vueNodes here, and startThread, bindViaActiveTab, finishTurn and subscribeCount below, are public with no consumer outside the class. docs/guidance/engineering.md rule 5 asks to keep exported surface minimal, and making them private shows the next test author which calls the harness actually supports.
There was a problem hiding this comment.
Leaving these members unchanged in this repair-only carrier; there is no behavioral defect here. #19663 now owns tightening the two-session harness API together with restoring the PM-1319 consumers, so the supported surface can be chosen against actual callers rather than guessed here.
There was a problem hiding this comment.
Addressed in 6a40cf8: panel, agentPanel, vueNodes, startThread, bindViaActiveTab, finishTurn and subscribeCount are now private, so the spec-facing surface is topbar, addEmptyWorkflow, boot, runBoundTurn, newChat, hostEdit and the two canvas readers. #19663 can widen it again for the restored consumers.
DrJKL
left a comment
There was a problem hiding this comment.
Verdict: comment. c61b350 fixes the three round-2 code findings, and the test code is now in the shape I would approve. What still keeps me from approving is the #18088 decision from round 1, which has had no response.
d7e0938 merges main and does not touch the two PR files. c61b350 applies the three round-2 suggestions as written, and I replied in each thread. I have no new findings on it.
Checks at this head: ESLint, type-aware oxlint, and oxfmt pass on both files, pnpm typecheck:browser passes, and playwright test --list --project=cloud lists both tests. Against the new merge base 10f7698, the diff is still the same two files (+591). Hosted checks were still running when I reviewed, so I have no CI result for this head.
Still open (4)
- P2, #18088 ownership (round 1 body item 1, round 2 open item 1, no response). #18088 is still open,
pr-18088is still an ancestor of this head, and11c4ccfstill deletes its three PM-1319 cases. I am not asking for code. Please record the decision on one of the two PRs: either close #18088 as superseded (saying where the PM-1319 pins go), or restack this PR on it. I will approve once that is settled. - P3, description (round 1 body item 2, round 2 open item 2). The body is unchanged. It still says the first test "is expected to fail only when the settled canvas is empty" and that the arrangement proves the host and Bravo states, and the Verification section still cites
d8514a0. The template sections and aRelated: PM-1535line are still missing. - P3,
objectInfoawaited afterbootAgentApp(Cursor r4087273655). Unchanged at fixture:142-151. As I said last round, I am fine leaving it. - Nit, public harness members with no outside caller (r4148278962). Unchanged and unanswered. It does not block. A one-line "leaving as is" in the thread is enough if you disagree.
Deferred to follow-up, accepted (2)
- P2, second fake
/wshost next toAgentFollowerHostSocket(r4147830138). - P3, third copy of the agent thread route mocks (r4147830179).
I still accept both as follow-ups. No issue is linked yet. Please link one here before merge.
Addressed at c61b350 (3)
- P3,
catch {}swallowing reader errors (r4148278909): now rethrows unless a read succeeded. My reply: r4148800159. - Nit, ambiguous "delete these three lines" (r4148278936): the comment now names the
test.failcall and thetoContainEqualguard. My reply: r4148800503. - Nit, comment restating
SETTLE_TIMEOUT(r4148278951): removed. My reply: r4148800844.
The 16 round-1 findings recorded as addressed at 9069969 in my round-2 review are unchanged at this head.
|
Addressed the exact-head review without changing code or the PM-1535 oracle:
Fresh focused black-box run at unchanged head c61b350: 2 passed. No source, assertions, skips, marker shape, or branch history changed. |
DrJKL
left a comment
There was a problem hiding this comment.
Verdict: ready to approve. At c61b350 the last blocking question is settled: #18088 is closed as superseded, #19663 owns the PM-1319 pins and both accepted follow-ups, and the description now follows the template.
The head has not changed since my round 3 review. christian-byrne's comment closes out the remaining items without touching code. #18088 was closed on purpose, and its closing note says the PM-1319 cases are not landed here. #19663 covers restoring those three pins, consolidating the second /ws host with AgentFollowerHostSocket, sharing the Agent route mocks, and narrowing the harness API. The rewritten description has Summary, Changes, Review Focus, a Verification section citing c61b350, and Related: PM-1535.
One nit, not blocking: Review Focus says "toContainEqual(BUILT_NODE_IDS) must reject wrong or partial canvases". The code is expect([[], BUILT_NODE_IDS]).toContainEqual(canvas) (spec:92), which allows exactly the empty canvas or the build. Worth correcting the next time you edit the description.
Review record
Round 1 (round 1, at d8514a0, request changes):
- P1, conditional
test.failcould never go red for PM-1535 (r4147830055): addressed in9069969with atoContainEqualguard and an unconditional marker. Author reply r4147986766. - P2, POST ack typed by hand (r4147830119): addressed in
9069969withzAgentTurnAcceptedand a typed mock. Author reply r4147997023. - P2, #18088-only fixture API and its two casts (r4147830132): addressed in
9069969by deletion. Author reply r4147990702. - P2, second fake
/wshost (r4147830138): deferred to #19663 by agreement. Author reply r4148010359. - P2, branch absorbs #18088 and deletes its tests (body): resolved by closing #18088 as superseded and moving the pins to #19663. Author comment.
- P3, Bravo
toHaveCount(0)did not discriminate (r4147830084): addressed in9069969by removing the line. Author reply r4147994216. - P3, host-state assertions tested the fake (r4147830095): addressed in
9069969. Author reply r4147991955. - P3, settled helper returned a second read (r4147830103): addressed in
9069969. Author reply r4147988459. - P3, hand-rolled subscribe waiter (r4147830148): addressed in
9069969withexpect.poll. Author reply r4147995797. - P3, duplicate
emptySeedand unused seed default (r4147830155): addressed in9069969. Author reply r4147998730. - P3, narration comments (r4147830161): addressed in
9069969. I accepted the three kept comments. Author reply r4148002263. - P3, docblocks claimed #18088 coverage (r4147830167): addressed in
9069969. Author reply r4148000199. - P3, third copy of the route mocks (r4147830179): deferred to #19663 by agreement. Author reply r4148012440.
- P3, manual
Comfy.VueNodes.Enabled(r4147830189): addressed in9069969. Author reply r4148003660. - P3, no
test.step(r4147830197): addressed in9069969. Author reply r4148004998. - P3, "every thread starts here" (r4147830209): addressed in
9069969. Author reply r4148007710. - P3, PR description off-template with no issue line (body): addressed in the rewritten description. Author comment.
- Nit, spread of
expected(r4147830110): addressed in9069969. My reply r4148260074. - Nit,
catch (neverMatched)(r4147830114): addressed in9069969. My reply r4148260479. - Nit,
@slowand repeated budgets (r4147830203): addressed in9069969. Author reply r4148006295. - Earlier Cursor and CodeRabbit findings, 13 items: 12 were fixed or superseded by
9069969and earlier commits, as listed in round 1. The one I carried, P3objectInfoawaited afterbootAgentApp(r4087273655), I withdrew in round 2 becauseagentConversationFixture.ts:304-320uses the same pattern.
Round 2 (round 2, at 9069969, comment):
- P3,
catch {}let a reader that always throws pass as PM-1535 (r4148278909): addressed inc61b350. My reply r4148800159. - P3, stale description (body): addressed in the rewritten description. Author comment.
- Nit, ambiguous "delete these three lines" (r4148278936): addressed in
c61b350. My reply r4148800503. - Nit, comment restating
SETTLE_TIMEOUT(r4148278951): addressed inc61b350. My reply r4148800844. - Nit, public harness members with no outside caller (r4148278962): the author kept them and moved API tightening to #19663. Reply r4148885791. I accept that.
Round 3 (round 3, at c61b350, comment): no new findings. It carried the #18088 decision, the description, and the follow-up link, all resolved as above.
Checks at c61b350: ESLint, type-aware oxlint, and oxfmt pass on both files, pnpm typecheck:browser passes, and playwright test --list --project=cloud lists both tests (run locally in round 3, and the head is unchanged since). Hosted checks: 100 success, 25 skipped, 0 failed. I did not run the cloud spec itself because there is no backend in my environment. The author reports 2/2 passing at this head, and the mutation where the reader always throws now fails as intended.
Replace explanatory comments with assertion and marker messages, parse doc_subscribe with a schema instead of hand-rolled casts, take the widget catalog from the build recording, make harness internals private, await object_info alongside boot so its rejection is always observed, and drop the unused cancel route.
DrJKL
left a comment
There was a problem hiding this comment.
Verdict: approved at 6a40cf8. I pushed 6a40cf8 myself at the maintainer's request, so no independent reviewer has seen that commit.
6a40cf8 is a test-only cleanup I made after re-reviewing for redundant comments, simplicity, and test design. Neither oracle changed. It does the following:
- Removes every remaining source comment in both files. In a test file a comment should become a name, an assertion message, or nothing. The guard now carries
'only an empty canvas is the known PM-1535 failure'as its assertion message, and thetest.faildescription says what to delete once the fix lands. The PM-1535 mechanism and scope already live in the PR description. - Parses
doc_subscribewith a small zod schema instead of the hand-rolledparseClientFrame, which removes twoascasts and theClientDocFrametype. - Replaces
addWorkflow(id, name, seed)withaddEmptyWorkflow(id, name, catalog). The catalog now comes from the build recording the spec already loads, soTEMPLATE_CASE, the borrowed recording, and thenamesmap are gone.waitForSubscribetakes the bound workflow and gets its name from there. - Makes
panel,agentPanel,vueNodes,startThread,bindViaActiveTab,finishTurnandsubscribeCountprivate (round-2 nit r4148278962, reply r4190150255). - Awaits
object_infotogether with boot throughPromise.all, so a rejection from that waiter is always handled (Cursor r4087273655, reply r4190150563). - Drops the
/messages/*/cancelroute, which nothing in these flows calls.
Net change: +45/-187. The fixture went from 466 to 341 lines.
Review record
Round 1 (review, request changes at d8514a0): 18 inline findings and 2 body findings. The P1 conditional test.fail and 15 more inline items were addressed in 9069969, with author replies on each thread, except the two nits where I replied: r4148260074, r4148260479. The second /ws host (r4147830138) and the route-mock clone (r4147830179) moved to #19663 by agreement. The #18088 ownership and description items were resolved when christian-byrne closed #18088 as superseded, filed #19663, and rewrote the description (comment). Of the earlier Cursor and CodeRabbit findings, 12 were fixed or superseded. The thirteenth, the objectInfo late await, is now fixed in 6a40cf8.
Round 2 (review, comment at 9069969): the reader-error swallow, the marker wording, and the SETTLE_TIMEOUT comment were fixed in c61b350 (my replies r4148800159, r4148800503, r4148800844). The public-members nit was deferred by the author and is now fixed in 6a40cf8.
Round 3 (review): no new findings. Round 4 (review): ready to approve, with one nit about how the description quoted the guard. That is fixed in the description.
Round 5 (this review, at the maintainer's request): the comment, simplicity, and test-design pass described above, pushed as 6a40cf8. The description now quotes the guard correctly and records this commit.
Checks at 6a40cf8: oxfmt, type-aware oxlint, ESLint, and pnpm typecheck:browser pass locally and in the pre-commit hook, and the pre-push knip/fallow audit passed. Hosted CI: every test job passed (101 success, 22 skipped, 0 failed at review time; only report-merge jobs were still running). That includes all four playwright-tests-cloud-sharded jobs, and playwright-video-new-tests (cloud) ran this spec with 2 passed, which means the displaced-build test failed as expected and the recovery test passed. I did not run the cloud spec locally because there is no backend in my environment.
|
@christian-byrne Successfully backported to #20378 |
|
@christian-byrne Successfully backported to #20379 |
…nd (PM-1535) (#18610) ## Summary Adds a black-box E2E pin for [PM-1535](https://linear.app/comfyorg/issue/PM-1535): after two Agent threads bind two workflow tabs, Alpha's build reaches Alpha's document while Bravo remains active, but returning to Alpha leaves the selected canvas empty. The first case keeps an unconditional known-failure marker. An empty canvas is the expected PM-1535 failure, a wrong or partial canvas fails before the marker, and a fixed canvas becomes an unexpected pass that tells the fixer to remove the marker and containment guard. The second case is the positive control: a later turn rebinding Alpha materializes the exact build already held by Alpha's document. This carrier supersedes #18088 but intentionally does not carry its three PM-1319 cases. Those pins and the shared-fixture consolidation are tracked separately in #19663. ## Changes - **What**: Adds `agentTwoSessionCrdtFixture.ts` and two focused cloud Playwright cases for displaced-build return and recovery. - **Dependencies**: None. ## Review Focus - The unconditional `test.fail` must remain until PM-1535 is fixed. - `expect([[], BUILT_NODE_IDS]).toContainEqual(canvas)` must reject wrong or partial canvases before the known-failure marker. - The exact-id positive control must continue proving that the stored build is recoverable after rebinding. - Scope is one of the two empty canvases in the report. This carrier must not close PM-1535 by itself. ## Verification ```bash PLAYWRIGHT_TEST_URL=http://127.0.0.1:6254 \ PLAYWRIGHT_SETUP_API_URL=http://127.0.0.1:6290 \ DEV_SERVER_COMFYUI_URL=http://127.0.0.1:6290 \ DISTRIBUTION=cloud NODE_OPTIONS=--no-experimental-webstorage \ pnpm exec playwright test \ browser_tests/tests/agent/agentTwoSessionCrdtSubscription.spec.ts \ --project=cloud --workers=1 --retries=0 ``` Head `c61b3507f136c3c687833024af59a0fb8432aad7`: 2/2 focused tests passed against a local backend. Mutation proof at that head makes a canvas reader that always throws fail at that exact error instead of passing as PM-1535. `6a40cf8` trims the harness (schema-parsed `doc_subscribe`, private internals, catalog taken from the build recording, comments moved into assertion and marker messages) without changing either oracle; ESLint, type-aware oxlint, oxfmt, `pnpm typecheck:browser`, and the hosted cloud Playwright shards cover it. Related: PM-1535 --------- Co-authored-by: Alexander Brown <drjkl@comfy.org>
Summary
Adds a black-box E2E pin for PM-1535: after two Agent threads bind two workflow tabs, Alpha's build reaches Alpha's document while Bravo remains active, but returning to Alpha leaves the selected canvas empty.
The first case keeps an unconditional known-failure marker. An empty canvas is the expected PM-1535 failure, a wrong or partial canvas fails before the marker, and a fixed canvas becomes an unexpected pass that tells the fixer to remove the marker and containment guard. The second case is the positive control: a later turn rebinding Alpha materializes the exact build already held by Alpha's document.
This carrier supersedes #18088 but intentionally does not carry its three PM-1319 cases. Those pins and the shared-fixture consolidation are tracked separately in #19663.
Changes
agentTwoSessionCrdtFixture.tsand two focused cloud Playwright cases for displaced-build return and recovery.Review Focus
test.failmust remain until PM-1535 is fixed.expect([[], BUILT_NODE_IDS]).toContainEqual(canvas)must reject wrong or partial canvases before the known-failure marker.Verification
Head
c61b3507f136c3c687833024af59a0fb8432aad7: 2/2 focused tests passed against a local backend. Mutation proof at that head makes a canvas reader that always throws fail at that exact error instead of passing as PM-1535.6a40cf8trims the harness (schema-parseddoc_subscribe, private internals, catalog taken from the build recording, comments moved into assertion and marker messages) without changing either oracle; ESLint, type-aware oxlint, oxfmt,pnpm typecheck:browser, and the hosted cloud Playwright shards cover it.Related: PM-1535