Skip to content

test: keep a human-added node across a tab switch while its op is in flight - #20051

Merged
christian-byrne merged 2 commits into
mainfrom
claude/test-human-add-survives-tab-switch
Oct 4, 2026
Merged

christian-byrne merged 2 commits into
mainfrom
claude/test-human-add-survives-tab-switch

Conversation

@claude

@claude claude Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Christian Byrne · Slack thread

Before: No unit test ran the real follower, bridge, op sender and doc-op minter together with a human add_node still in flight across a tab deactivation and reactivation; the retention guarantee for that sequence was covered only by Playwright.

After: useAgentCrdtFollower.humanAddTabSwitch.test.ts adds that node (catalogued and virtual) and queues a second human edit behind the in-flight add. The tab deactivates, the host advances with an agent edit, and the tab reactivates against a catch-up frame that carries that real delta but lacks the human node. It asserts the agent edit lands, the human node survives in the live graph, the node data store and serialize(), only one doc_ops frame is on the wire until the first ack, and the held second batch is sent after it. The virtual arm asserts the positional widgets_values of the minted add_node (the catalogued arm asserts the name-keyed shape).

Test-only change; no production code is touched.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NQVcfKhCKjLstqo3mJS7fc


Generated by Claude Code

@claude
claude Bot requested a review from a team October 4, 2026 00:07
@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: d837febf-785a-407a-9b9b-8ddab72d7853

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 10/04/2026, 01:13:39 AM UTC

Links

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since d348bddc595545719937913514cb386c0970a721

Full docs: ADR-DEVEX-LINT-0015

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Oct 4, 2026
@codecov

codecov Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main   #20051      +/-   ##
==========================================
+ Coverage   84.96%   85.69%   +0.72%     
==========================================
  Files        3450     2423    -1027     
  Lines      165892   138513   -27379     
  Branches    43333    37515    -5818     
==========================================
- Hits       140957   118695   -22262     
+ Misses      22965    19442    -3523     
+ Partials     1970      376    -1594     
Flag Coverage Δ
billing-web-unit ?
unit 82.54% <ø> (+0.01%) ⬆️
website-unit ?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1034 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@christian-byrne christian-byrne left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at head 945fcd4c2ac449d64202f198dc91eb205987058a. Test-only, CI green, no production code touched — so the only thing that matters is what this pin would falsify. I ran it, then mutated production under it. It does not falsify the sequence in its title, and what it does falsify is already pinned elsewhere with stronger assertions.

1. The frame the test is named after carries no changes

deliverCatchUp(host, 1) on return re-sends Y.encodeStateAsUpdate(host) for a host that has not moved since the first catch-up. Yjs integrates nothing, so DocChangeCollector reports an empty delta. Instrumenting AgentCrdtProjection to dump the collected delta per pass, for keeps a catalogued node …:

applyFrame     seq=1 catchUp=true nodes=[["1","add"]]   # initial catch-up
applyCollected                    nodes=[]              # reactivation
applyFrame     seq=1 catchUp=true nodes=[]              # "a snapshot without it arrives on return"

Confirmed by deleting both post-return deliverCatchUp calls: 3/3 still pass. The "snapshot that lacks it" never asks the applier to do anything, so expect(graph.getNodeById(added.id)).toBe(added) cannot distinguish retention from inertness. For the frame to be load-bearing the host has to actually advance before the return catch-up (apply an unrelated op), and the delta should be asserted non-empty.

2. "while its op is in flight" is not pinned either

Replacing the same-workflow hold in useAgentCrdtFollower.deactivateTarget with a fall-through to rebindProjection(null) / retarget(null) — i.e. deleting the suspend-and-hold behaviour outright — leaves 3/3 passing. The reason: the batch is transmitted on enqueue (the test asserts addFrames is length 1 before deactivation), so it is awaiting an ack, not queued. The hold protects batches queued behind an in-flight one, which this test never creates. agentDeleteTabSwitchRace.test.ts gets this right by enqueuing an unrelated set_widget first.

That same mutation is caught by 6 existing tests in useAgentCrdtFollower.test.ts → tab suspension of the human write leg, at this same composable level.

3. The third case asserts an unreachable configuration

keeps a node that was never minted … runs with withMinter: false, so attachDocOpMinter is never called and expect(frames(sent, 'doc_ops')).toHaveLength(0) is true by construction — nothing can mint. AgentPanelRoot.vue attaches the minter unconditionally and gates it through isEnabled / isDocBound; there is no product state where the follower is mounted and subscribed but the minter is absent. If the intent is "the host never heard about this node", model it by gating (isEnabled: () => false, or an op the host refuses) rather than by omitting the minter. As written the test name also reads as a policy claim — that such a node is retained and never sent — which is the opposite of the ruling that an edit made while the connection is down must eventually be delivered.

4. Overlap with current main

Forcing takeApplyMode to always return 'replace' (so removeAbsent clobbers every doc-absent live node) is the only mutation I found that this file catches — and it is already caught by, with stronger assertions:

  • agentCrdtProjection.localEdits.test.ts → after a tab return > keeps a node the user added whose add_node has not reached the doc — same scenario, and asserts live nodes plus nodeDataStore records plus serialize(), where this PR asserts only getNodeById.
  • agentCrdtProjection.integration.test.ts → keeps a local node whose add has not reached the doc through a later incremental frame.
  • agentCrdtProjection.localEdits.test.ts:238 already parameterises catalogued vs. frontend-only for a human add.

The TestVirtual arm here asserts only ops.map(op => op.op), never the payload. The one thing isVirtualNode changes in wireNodeSnapshot is positional vs. name-keyed widgets_values, and inverting that check is already caught by docOpMinter.test.ts → keeps add_node widgets_values positional for a frontend-only node, and the applier takes it. So the second arm costs a case without adding an assertion.

Also worth checking against open #17531, which covers tab open/switch/close/remount subscription lifetime in the same file.

What is genuinely new, and worth keeping

Driving useAgentCrdtFollower + attachDocOpMinter + opSender + real wire frames together is a level no existing test covers, and expect(frames(sent, 'doc_unsubscribe')).toHaveLength(1) / doc_subscribe length 2 across the switch is real. The composition is the contribution; the retention assertion is not. Suggested shape:

  1. Advance the host before the return catch-up so the frame carries a delta, and assert it is non-empty.
  2. Enqueue a second batch behind the in-flight one so the suspend/hold is actually exercised (mutation in §2 should then go red).
  3. Either drop case 3 or drive "never minted" through the gates rather than by detaching the minter.
  4. Assert nodeDataStore records and serialize() alongside getNodeById, matching the sibling test.
  5. For the virtual arm, assert the positional widgets_values shape, or drop the arm.

Mutations run: remove the return catch-up frames (pass); force 'replace' apply mode (fail, already covered); delete the inactive-tab op hold (pass); invert the isVirtualNode widget-shape branch (fail, already covered); disable the own-echo skip (pass). All against origin/main at 6d15eed0647700aee434c5c78f8a9aa4e8e555c9, npx vitest run src/workbench/extensions/agent/crdt/.

@claude

claude Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor Author

Reworked at e323ed8, point by point:

  1. The host now advances (an agent set_widget on the source node) before the return catch-up, and the test asserts the live steps value moves 20 to 55, so the frame is applied and non-empty.
  2. A second human edit (a source-node rename) is queued behind the in-flight add_node: one doc_ops frame on the wire, none sent while inactive, and the held batch goes out after reactivation and the first ack.
  3. The never-minted case and the withMinter parameter are gone.
  4. Live nodes, nodeDataStore records and serialize() are asserted together via a local nodeIds(graph) that mirrors the one in agentCrdtProjection.localEdits.test.ts.
  5. The virtual arm now asserts the minted positional widgets_values (['x']); the catalogued arm asserts the name-keyed { steps: 20 }.

Mutations, each run against this file only and reverted afterwards (both cases red every time):

  • Drop the return catch-up frame: red (expected 20 to be 55).
  • Delete the same-workflow hold in deactivateTarget: red (the queued batch is never sent: 1 doc_ops frame, expected 2).
  • Force takeApplyMode to 'replace': red (the human node is removed).

#17531 only touches useAgentCrdtFollower.test.ts, not this file. It pins subscribe/unsubscribe call counts on a mocked bridge, while this test counts wire doc_subscribe/doc_unsubscribe frames, so I see no conflict.


Generated by Claude Code

@christian-byrne
christian-byrne added this pull request to the merge queue Oct 4, 2026
Merged via the queue into main with commit 07b78a1 Oct 4, 2026
94 checks passed
@christian-byrne
christian-byrne deleted the claude/test-human-add-survives-tab-switch branch October 4, 2026 01:57
Penguinjanator pushed a commit to Penguinjanator/ComfyUI_frontend that referenced this pull request Oct 4, 2026
## Summary

Add black-box coverage for the visible behavior proposed by Comfy-Org#20051: a
human-added node queued behind an in-flight edit remains on the canvas
across a workflow-tab switch and is eventually delivered to the host.

## Changes

- **What**: Adapts the existing `agentHumanAddTabSwitch` spec instead of
adding another follower spec. The case uses the real canvas, follower,
minter, sender, held host, tab UI, and visible node assertions.

## Review Focus

The first widget edit is held by the fake host, so the added Note is
demonstrably queued (only one `doc_ops` frame). Removing
`holdOpsForInactiveTab` makes the case fail because the queued node
batch never resumes; restoring it makes the full 7-case spec pass.

Tests: `PLAYWRIGHT_LOCAL=1 PLAYWRIGHT_TEST_URL=http://127.0.0.1:6206
PLAYWRIGHT_SETUP_API_URL=http://127.0.0.1:8188 DISTRIBUTION=cloud pnpm
exec playwright test
browser_tests/tests/agent/agentHumanAddTabSwitch.spec.ts --project=cloud
--retries=0`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants