Skip to content

test(agent): separate absent subscription sequence fixture - #18486

Merged
DrJKL merged 5 commits into
mainfrom
test/doc-frame-seq-helper
Sep 26, 2026
Merged

DrJKL merged 5 commits into
mainfrom
test/doc-frame-seq-helper

Conversation

@christian-byrne

@christian-byrne christian-byrne commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Keep invalid-sequence parser cases type-safe and make absent doc_subscribed.seq coverage verify key omission explicitly.

Changes

  • What: Requires sequencedFrame callers to supply a sequence, gives the absent-sequence case its own fixture, and asserts the parsed frame has no seq property. sequencedFrame now emits seq directly, so unsequencedSubscribedFrame is the only omission path and the two fixtures cannot own the same state.

Review Focus

This closes the two residual test-quality findings from #16969 (comment) and #16969 (comment).

Corrected — the third finding is not covered, contrary to what this description said before. The consumer-level concern from #16969 (comment) is still open. Merged PR #17611 pins a different ordering: a seq-less (or seq-0) ok ack followed by doc_update seq 1, which is the healthy path. The hazard in that finding is the overtaking order — a seq-less ack sets ackSeq = null, so catchUpPending stays false (layoutFollowerBridge.ts onDocSubscribed), isCatchUp can therefore never become true, and once a live frame has advanced lastSeq the catch-up at the omitted baseline is dropped by rejectStaleUpdate as stale. Nothing tests that.

Narrowing the claim rather than adding the test here, because the missing case belongs in followerSubscription.test.ts against the bridge and, on the reading above, would land as a failing pin rather than a passing one — which does not belong in a fixture-split PR. Raised and confirmed by @DrJKL twice on this PR; recorded as a follow-up so it is not lost with this description.

Local verification: focused Vitest (28 passed), oxlint and oxfmt --check on the changed file. Hosted CI remains the authority for the full suite.

Screenshots (if applicable)

Not applicable; test-only change.

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

🎭 Playwright: ✅ 2097 passed, 0 failed · 4 flaky

📊 Browser Reports
  • chromium: View Report (✅ 2076 / ❌ 0 / ⚠️ 4 / ⏭️ 13)
  • chromium-2x: View Report (✅ 2 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • chromium-0.5x: View Report (✅ 1 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • mobile-chrome: View Report (✅ 18 / ❌ 0 / ⚠️ 0 / ⏭️ 0)

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 09/26/2026, 05:38:50 AM UTC

Links

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since d90da45a1bf9932820bae42f893bb8332493c022

Full docs: ADR-DEVEX-LINT-0015

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: 3a7b523e-8b54-4a50-8c28-1224716ae5a4

📥 Commits

Reviewing files that changed from the base of the PR and between 781e7a4 and da67048.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: b0e4dd0d-567f-45d2-bfac-4980da784e73

📥 Commits

Reviewing files that changed from the base of the PR and between 24f1463 and 781e7a4.

📒 Files selected for processing (1)
  • src/workbench/extensions/agent/crdt/docFrameClient.numeric.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 4 remain after this review. Your free on-demand review promotion remains active until October 9, 2026 at 6:00 PM UTC.


📝 Walkthrough

Walkthrough

The CRDT numeric tests now use separate fixtures for sequenced and unsequenced frames. The unsequenced subscription test verifies that the parsed frame omits seq.

Changes

CRDT frame test validation

Layer / File(s) Summary
Frame fixtures and parsing assertion
src/workbench/extensions/agent/crdt/docFrameClient.numeric.test.ts
sequencedFrame now requires and includes seq. A separate fixture creates unsequenced doc_subscribed frames. The test asserts that parsed unsequenced data omits seq.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~3 minutes

Change: Other

Suggested reviewers: anupreet-comfy

Merge Risk: ⚪ Minimal · up to 781e7

This test-only change explicitly checks the absent-sequence case. No merge-blocking risk from the described change is evident; normal checks remain applicable.

🚥 Pre-merge checks | ✅ 7
✅ Passed checks (7 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
End-To-End Regression Coverage For Fixes ✅ Passed No bug-fix language appears in the PR title or any commit subject. The custom check therefore does not apply, regardless of the changed src/ test file and the absence of a browser_tests/ change.
Website End-To-End Regression Coverage ✅ Passed The pull request changes only src/workbench/extensions/agent/crdt/docFrameClient.numeric.test.ts. It does not change files under apps/website/src/ or apps/website/public/, so this check does not…
Adr Compliance For Entity/Litegraph Changes ✅ Passed The changed-file list contains only src/workbench/extensions/agent/crdt/docFrameClient.numeric.test.ts. It does not modify src/lib/litegraph/, src/ecs/, or files related to graph entities. The c…
Title check ✅ Passed The title clearly and concisely describes the main change: separating the fixture for an absent subscription sequence.
Description check ✅ Passed The description includes the required summary, changes, review focus, verification results, and screenshots status. It also clearly documents the remaining consumer-level follow-up.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Sep 23, 2026
@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Sep 23, 2026
@codecov

codecov Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main   #18486      +/-   ##
==========================================
+ Coverage   84.01%   84.93%   +0.91%     
==========================================
  Files        3157     2308     -849     
  Lines      154223   132868   -21355     
  Branches    39672    35627    -4045     
==========================================
- Hits       129574   112847   -16727     
+ Misses      23050    19709    -3341     
+ Partials     1599      312    -1287     
Flag Coverage Δ
billing-web-unit ?
unit 81.08% <ø> (-0.01%) ⬇️
website-unit ?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 857 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@DrJKL DrJKL added agent Comfy Agent product, integration, tests or product documentation agent-crdt Agent document synchronization, graph projection or CRDT operations agent-test-only Agent tests, fixtures and test support only; no production changes agent-test-unit Touches Agent-related unit/component tests or their support labels Sep 24, 2026
@DrJKL
DrJKL self-requested a review September 25, 2026 03:59

@DrJKL DrJKL left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: follow-up needed on fixture separation and the claimed consumer coverage

No P0 or P1 blockers.

Existing finding contested
  • [P2] I could not confirm the Review Focus claim that PR #17611 covered the seq-less acknowledgement race. That PR sends an acknowledgement without seq, then a live update at sequence 1. The cited race is different: sequence 1 overtakes the acknowledgement, then the catch-up at the omitted baseline sequence 0 arrives. With ackSeq === null, catchUpPending stays false, so that catch-up can be rejected as stale against lastSeq === 1. Please either add that exact consumer test or narrow the description to the behavior PR #17611 actually covers.
Checked and cleared
  • [P2] The invalid-present-sequence table still permits an own seq: undefined result, but that assertion is unchanged on the base branch. A focused mutation confirmed the pre-existing gap, so it is not attributed to this PR.
  • [P3] The prior absent-sequence omission finding is fixed. A seq: undefined result fails the new not.toHaveProperty('seq') assertion.
  • Fallow reported 15 inherited dev-dependency findings and no introduced dead-code, complexity, duplication, or styling findings. Each inherited item is outside this test-only diff: @pinia/testing, @tailwindcss/vite, jsonc-parser, markdown-table, mixpanel-browser, monocart-coverage-reports, openai, picocolors, pretty-bytes, unplugin-icons, unplugin-vue-components, uuid, yaml, zip-dir, and zod-to-json-schema.
Coverage
  • Depth: standard.
  • Lanes: correctness, maintainability, idiomaticity, tests, and documentation. No changed path was left untraced.
  • Standard lenses covered blast radius and regression history; boundary, idempotency, shared-state, and root-cause checks; deletion and reader-load checks; TypeScript and type-system rules; unit-test design, mock boundaries, setup ownership, and executable proof; and technical-writing checks. Playwright-specific lenses were not applicable because this PR does not touch browser_tests/.
  • Verification: the focused Vitest file passed 28 tests; scoped formatting and ESLint passed; pnpm typecheck passed. Fallow's review gate passed, though its saved baselines reported path-matching warnings.
  • Existing feedback: CodeRabbit reported no actionable comments. I independently checked the three PR #16969 findings linked in the description.

@christian-byrne christian-byrne added the cursor-review Trigger multi-model Cursor agent review on this PR label Sep 26, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @christian-byrne.

✅ No high-signal findings.

Panel: 8/8 reviewers contributed findings.

@christian-byrne christian-byrne left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the split against the behaviour it is meant to preserve, since a fixture split that quietly weakens an assertion is the trap here.

It does not weaken anything — it is strictly stronger, and for a reason worth stating. On main the helper's ...(seq !== undefined && { seq }) already omitted the key entirely when called with no argument, so the old test was exercising the absent-seq case. What it could not do is prove it: toEqual ignores undefined-valued properties, so toEqual({ type, data: { workflowId, ok } }) would have passed just as happily against data: { workflowId, ok, seq: undefined }. The added expect(frame?.data).not.toHaveProperty('seq') is the first assertion here that distinguishes omitted from present but undefined.

That is also what makes the split worth doing rather than cosmetic: the old test's meaning was a property of the helper's conditional spread, not of the test. Anything that made sequencedFrame always include seq would have turned it into a silent duplicate of the omits an invalid doc_subscribed seq case above it, with nothing failing. Making seq required removes the way to write that mistake. Worth a line in the PR body — as written it reads as tidying, and it is closing a real hole in strictness.

Verified the two fixtures produce identical frames: { v: 1, workflow_id: 'wf-1', ok: true } either way (the old path got ok from the type === 'doc_subscribed' spread), so the toEqual half asserts exactly what it did before.

No overlap with #18700 (0 of 1 files), and docFrameClient.numeric.test.ts is a wire-parsing test rather than a projection-shaped one, so it survives the refactor unchanged.

@DrJKL DrJKL left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: the coverage claim remains unresolved; two non-blocking code comments are still open

Still open
  • [P2] Unresolved: the PR description still says PR #17611 covers the seq-less acknowledgement race from PR #16969. The follow-up review does not address this point. PR #17611 tests an acknowledgement without seq followed by sequence 1, not the cited ordering where sequence 1 arrives before the acknowledgement and the baseline catch-up at sequence 0 follows it. Please add that exact consumer test or narrow the description.
  • [P3] Confirmed: sequencedFrame still omits the key for explicit undefined. The author's follow-up independently calls the conditional spread vestigial and recommends replacing it with direct seq; the unchanged head does not yet make that edit.
  • [P3] Nit, unresolved: the new fixture declaration still uses an arrow where the repository guidance prefers a function declaration.
Feedback dispositions
  • Confirmed: the PR is strictly stronger than the base test, and not.toHaveProperty('seq') closes the old absent-key assertion gap. My prior review already marked that original finding fixed; it does not conflict with the remaining vestigial fixture mode.
  • Not introduced here: the suggested symmetric split for docOpsResultFrame concerns unchanged base code, and that test already checks property absence directly.
Verification
  • Head remains 24f14634d4943126d5e32ddd3daa876b021d74fa; no code changed since the prior review.
  • I reused the prior focused Vitest, lint/format, typecheck, mutation, and Fallow evidence rather than rerunning unchanged checks.
  • No new finding was introduced by the follow-up review.

`seq` became a required parameter of `sequencedFrame`, so the
`seq !== undefined` branch can no longer be taken: every caller passes a
defined value. Leaving it in implies a caller that omits it, which is
exactly the implicit-omission coupling this PR removes.

Addresses review feedback on PR #18486 (review 5324690344).
Repository guidance prefers a function declaration where a helper
captures no state and needs no expression semantics.

Addresses DrJKL's P3 nit on PR #18486 (discussion r4109490223).

@DrJKL DrJKL left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: ready to approve

All prior findings are resolved on 781e7a4bb1f2cf14f34815d8536a693cc3bcffe9:

  • [P3] Fixed: sequencedFrame now emits seq directly, leaving unsequencedSubscribedFrame as the only omission path.
  • [P3] Nit, fixed: unsequencedSubscribedFrame is now a function declaration.
  • [P2] Fixed: the PR description no longer says PR #17611 covers the overtaking race. It distinguishes that PR's healthy ordering and records the still-missing consumer regression case as follow-up work.

I reviewed the effective one-file diff against the refreshed base after the merge from main and found no new issues. The focused Vitest file passed all 28 tests; scoped formatting and ESLint passed; and pnpm typecheck passed.

@DrJKL
DrJKL added this pull request to the merge queue Sep 26, 2026
Merged via the queue into main with commit cd4c22e Sep 26, 2026
78 checks passed
@DrJKL
DrJKL deleted the test/doc-frame-seq-helper branch September 26, 2026 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Comfy Agent product, integration, tests or product documentation agent-crdt Agent document synchronization, graph projection or CRDT operations agent-test-only Agent tests, fixtures and test support only; no production changes agent-test-unit Touches Agent-related unit/component tests or their support cursor-review Trigger multi-model Cursor agent review on this PR risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants