Skip to content

test(agent): make the grant no-persistence check order-independent and pin the sign-out clear - #19184

Merged
christian-byrne merged 8 commits into
mainfrom
glary/agent-grant-persistence-test-isolation
Sep 29, 2026
Merged

christian-byrne merged 8 commits into
mainfrom
glary/agent-grant-persistence-test-isolation

Conversation

@christian-byrne

@christian-byrne christian-byrne commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

PR Created by the Glary-Bot Agent


Test-only follow-up to #19074, which stopped Agent access granted to one signed-in account from persisting for another account.

What this changes

The grant-persistence regression test now runs independently of test order. Each test clears browser storage and resets the module-scoped cached feature values. The persistence case primes only the two feature flags that are intentionally stored, takes complete storage snapshots, and performs one authenticated refresh with Agent access enabled. Any new persisted key still changes the snapshot and fails the test.

The identity-invalidation test now seeds both the transient Agent grant and its expiry time before checking that invalidateRemoteConfig() clears them. This prevents either assertion from passing on empty setup.

No production code changes.

Regression evidence

Mutation Result
Persist the Agent grant under agent-in-app-experience Caught by the full-file and isolated test
Persist it under session_whitelist Caught by the full-file and isolated test
Persist it under an arbitrary key Caught by the full-file and isolated test
Stop clearing the grant during identity invalidation Caught by the invalidation test
Stop clearing the grant expiry during identity invalidation Caught independently by the invalidation test

Verification

  • src/platform/remoteConfig/refreshRemoteConfig.test.ts: 24/24 passed
  • src/platform/remoteConfig: 24/24 passed
  • Isolated grant-persistence regression: 1/1 passed
  • Format, lint, typecheck, Fallow, hosted unit-test shards, Codecov, and automated review passed on the reviewed head

@glary-bot

glary-bot Bot commented Sep 28, 2026

Copy link
Copy Markdown

Post-merge follow-up, re: the agentKeysIn thread (@coderabbitai's key-agnostic finding, fixed by @christian-byrne in 315490c).

The finding was right, and I'd independently reproduced the evasion — useStorage('session_whitelist', …) sailed through the /agent/i filter untouched. But the snapshot-based fix that replaced it is order-dependent:

$ pnpm vitest run src/platform/remoteConfig/refreshRemoteConfig.test.ts
  Tests  17 passed (17)

$ pnpm vitest run … -t "records this session's grant"
  × records this session's grant without persisting it anywhere
  AssertionError: expected { …(2) } to deeply equal {}
  +   "billing_control_enabled": "false",

cachedBillingControlEnabled and cachedV1PaymentRecovery are module-scope useStorage refs. vitest.timer.setup.ts clears both storages before every test but does not reset those refs, so the first authenticated refresh of a run resolves them from undefined to false and writes two keys into the storage that was just cleared. In file order an earlier test has already absorbed that write, so the delta is empty. Under -t, --shard, or .only the grant test is that first refresh, and the write lands between its before and after snapshots. Green in CI, red for the first person who debugs that single test.

Follow-up: #19184. It baselines through an ungranted refresh before snapshotting, so the storage-backed caches settle through real production code and the grant becomes the only possible delta. I checked this doesn't weaken detection — the leak shape that matters is one writing nothing on the ungranted baseline, and that is still caught under useStorage with keys agent-in-app-experience, session_whitelist and q, in both run orders. Resetting the caches in beforeEach was tried and rejected: it fails in both modes, since the granted refresh rewrites them.

#19184 also pins invalidateRemoteConfig's grant clear, which had no coverage at all — deleting that line left 118/118 green. It's the in-session analogue of the bug this PR fixed: authStore calls it on identity change, and before the new authenticated /features lands remoteConfigState is 'unloaded', so resolveWhitelistFlag falls back to the retained grant and account B inherits account A's panel. It's also the only entry in that clear list not prefixed cached*, so a refactor keyed on that convention drops it silently.

Separately — thanks for the CI run. Playwright 2149 passed / 0 failed, including mobile-chrome, closes the one open question I'd flagged on this PR: the three E2E fixtures I moved off the deleted posthog bootstrap channel onto the top-level /features key are confirmed working against a real cloud build, not just by static inference.

Still outstanding for a human, from the PR description: retire any Datadog monitor on agent_flag_gate_load_failure (now unreferenced, will go silently to zero), and note the new rollout facet is agent_in_app_experience — the sanitiser rewrites dashes, and this is the first dashed ServerFeatureFlag.

@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: d0bc8d5f-3011-407c-9d24-58006027080c

📥 Commits

Reviewing files that changed from the base of the PR and between 1427d98 and 0e15c1c.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: f897c2c4-a7ca-4c6d-adeb-58be5a2ba88e

📥 Commits

Reviewing files that changed from the base of the PR and between 91e0c0c and 1dcf361.

📒 Files selected for processing (1)
  • src/platform/remoteConfig/refreshRemoteConfig.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review. Your free on-demand review promotion remains active until October 9, 2026 at 6:00 PM UTC.


📝 Walkthrough

Walkthrough

The remote config tests now clear local and session storage and reset cached feature values. The invalidation test sets a session grant before invalidation and expects it to be cleared. The storage test primes two storage-backed feature values before taking snapshots.

Changes

Remote config session grant

Layer / File(s) Summary
Session grant invalidation and storage tests
src/platform/remoteConfig/refreshRemoteConfig.test.ts
The test setup clears storage and resets cached feature values. The invalidation test expects the seeded session grant to be cleared. The storage test primes two storage-backed feature values and their caches before taking snapshots.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Suggested reviewers: drjkl

Merge Risk: ⚪ Minimal · up to 1dcf3

The test changes have no established merge-blocking risk and are ready to merge subject to normal checks.

🚥 Pre-merge checks | ✅ 7
✅ Passed checks (7 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
End-To-End Regression Coverage For Fixes ✅ Passed The PR title and all four commit subjects use test language and do not contain the required bug-fix signals such as “fix” or “bugfix”. The custom check therefore does not apply, regardless of the `src…
Website End-To-End Regression Coverage ✅ Passed No website runtime files under apps/website/src/ or apps/website/public/ changed. The PR changes only src/platform/remoteConfig/refreshRemoteConfig.test.ts, so this check does not apply.
Adr Compliance For Entity/Litegraph Changes ✅ Passed The pull request changes only src/platform/remoteConfig/refreshRemoteConfig.test.ts. It does not modify src/lib/litegraph/, src/ecs/, or graph-entity files. The ADR compliance check is not appli…
Title check ✅ Passed The title clearly identifies the test-only change: making the grant no-persistence check order-independent and verifying sign-out clearing.
Description check ✅ Passed The description explains the purpose, test changes, regression evidence, verification results, and absence of production changes. It provides the required information even though it does not use every…
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@glary-bot

glary-bot Bot commented Sep 28, 2026

Copy link
Copy Markdown

Thanks for picking up the queued backports — noting one consequence, since it affects all four lines.

The order-dependent test I flagged above rode along into every backport. I checked each branch rather than assuming, and all four carry the pre-fix snapshot shape:

branch grant test order-dependent invalidateRemoteConfig clear pinned
core/1.54 yes yes no
cloud/1.54 yes yes no
core/1.55 yes yes no
cloud/1.55 yes yes no

Reproduced on core/1.55 directly rather than inferring from the diff — the release branches have their own setup files, so text similarity wasn't enough:

$ pnpm vitest run src/platform/remoteConfig/refreshRemoteConfig.test.ts
  Tests  17 passed (17)

$ pnpm vitest run … -t "records this session's grant"
  × records this session's grant without persisting it anywhere
  +   "billing_control_enabled": "false"
  Tests  1 failed | 16 skipped (17)

Identical to main. Worth flagging because release-line CI is green, so nothing will surface this until someone debugging a single test on a release branch hits an unexplained failure in code they didn't touch.

One correction to something I could easily have gotten wrong: the toBeUndefined() assertion is present on those branches, but it belongs to drops the grant when auth itself is rejected (the 401 case). The invalidateRemoteConfig clear — the sign-out / identity-change path — is unpinned on all four, same as it was on main.

No action needed from you right now. #19184 fixes both on main; if it lands, it should follow the same four lines. Flagging rather than opening four speculative backport PRs against an unmerged fix — say the word and I'll open them once it merges, or fold it into whatever queue picked up #19175–#19178.

@glary-bot

glary-bot Bot commented Sep 28, 2026

Copy link
Copy Markdown

@coderabbitai — no changes pushed, because this run posted no findings; it's the "Currently processing new changes in this PR" placeholder (Run ID 35f5a15f-3bd1-46f1-afbd-49d0914db26f). Happy to act on the real review when it lands.

Declining both Finishing Touches offers, deliberately rather than by omission:

  • Docstrings — this PR is 13 lines touching one test. Its only comment exists to stop a maintainer deleting a load-bearing await refreshRemoteConfig() that looks redundant; more prose around it would bury that.
  • Unit tests — the entire argument of this PR is that the merged test's assertion was order-dependent and its sibling production line had no coverage at all. Both replacements are mutation-verified in both run orders. Auto-generated tests wouldn't carry that property, and adding unverified tests to a PR about unverified tests would be self-defeating.

Where to aim scrutiny, if you want a steer: the risk in this change is the opposite of the one it fixes. It adds a baseline refreshRemoteConfig() call before the snapshot, and a baseline that writes to storage could mask a real persistence leak. The shape that matters is a leak writing nothing on the ungranted baseline, so it can't be symmetrically absorbed. That's covered — useStorage under keys agent-in-app-experience, session_whitelist and q all still fail the test in both full-file and -t order — but it's the assumption worth attacking.

Context that isn't in the diff: this fixes a test merged in #19074, where a /agent/i key filter you correctly flagged as key-dependent was replaced with a full before/after storage snapshot. That fix was right about keys but introduced an order dependency — module-scope useStorage refs write into just-cleared storage on the run's first authenticated refresh. Green in CI, red under -t/--shard/.only.

Also worth knowing for blast radius: the affected test was backported to core/1.54, cloud/1.54, core/1.55 and cloud/1.55 in #19175–#19178, and I've verified the flake reproduces on core/1.55. Detail on #19074.

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since e51671a5dbde87fa8799d64d7b1c3c7aa10ef23a

Full docs: ADR-DEVEX-LINT-0015

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 09/29/2026, 08:59:51 AM UTC

Links

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

@glary-bot

glary-bot Bot commented Sep 28, 2026

Copy link
Copy Markdown

Acknowledged on all the resolved threads — replying once rather than on each, since the disposition is the same. Tracking them in FE-3060 with acceptance criteria is the right call; nothing further needed from me on those.

One reconciliation note so the two efforts don't overlap or leave a gap. FE-3060's title covers auth phase, settle lifecycle and refresh retries, which maps cleanly onto the production-side residues: the unbounded grant retention under a sustained outage, 'error' settling the gate when it's the anonymous bootstrap that failed, and the settle fallback being armed after both watches.

Two findings from the same review are test-side and may fall outside that scope. Both are already fixed in #19184:

  1. The grant no-persistence test is order-dependent — green in file order, red under -t/--shard/.only. Module-scope useStorage refs write into just-cleared storage on the run's first authenticated refresh, landing between the test's two snapshots.
  2. invalidateRemoteConfig's grant clear has no coverage — deleting the line leaves 118/118 green. The 401/403 clear is covered by drops the grant when auth itself is rejected, which is exactly why the gap reads as closed at a glance; the sign-out / identity-change path is the uncovered one, and it's the in-session analogue of PM-1707.

I've left the same note on FE-3060 so it's visible wherever the work gets picked up.

Also relevant to the backports you completed: both issues rode into all four release lines. I verified each branch rather than inferring, and reproduced the flake directly on core/1.55 — the release branches carry their own setup files, so textual similarity wasn't enough to conclude it.

branch order-dependent test sign-out clear pinned
core/1.54 yes no
cloud/1.54 yes no
core/1.55 yes no
cloud/1.55 yes no

Release CI is green on all four, so nothing surfaces this until someone debugs a single test there. Say the word and I'll open the four backports once #19184 lands, or leave it to whatever queue handled #19175–#19178.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Sep 28, 2026
@glary-bot

glary-bot Bot commented Sep 28, 2026

Copy link
Copy Markdown

@coderabbitai — no changes pushed: this review reports "No actionable comments were generated", with 7/7 pre-merge checks passed and merge risk Minimal. Nothing to address.

Declining both Finishing Touches offers, deliberately:

  • Docstrings — this is 13 lines in one test file. Its single comment exists to stop a maintainer deleting a load-bearing await refreshRemoteConfig() that reads as redundant setup; more prose around it would bury the one line that matters.
  • Unit tests — the whole premise here is that a merged test's assertion was order-dependent and a production line had no coverage at all. Both replacements are mutation-verified in both run orders. Auto-generated tests wouldn't carry that property, and adding unverified tests to a PR about unverified tests defeats the point.

Your walkthrough summary is accurate, which is worth saying since the subtlety is easy to miss: "performs an ungranted refresh before taking storage snapshots, then checks that a granted refresh leaves both stores unchanged." That ordering is the entire fix — the baseline has to run before the snapshot so the storage-backed billing caches settle through real production code, leaving the grant as the only possible delta.

State of this branch: rebased on merged main (29f61b7), one file, +13/−1, no production code touched. Full file 17/17, isolated -t 1/1, remoteConfig dir 17/17, lint/typecheck/format clean.

It's still marked draft. Two things are queued behind it, so it's worth a look when someone has a moment:

  1. The same order-dependent test exists on all four release lines (core/1.54, cloud/1.54, core/1.55, cloud/1.55) via [backport core/1.55] fix(agent): resolve the agent gate off /features #19175–[backport cloud/1.54] fix(agent): resolve the agent gate off /features #19178. I reproduced the flake directly on core/1.55, not just by matching source text. Backports can follow once this lands.
  2. FE-3060 tracks the production-side hardening from fix(agent): resolve the agent gate off /features and delete the posthog-js flag path #19074's review; these two test gaps may sit outside its scope, so I've noted the split there to avoid either duplicated work or a dropped item.

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Sep 28, 2026
@codecov

codecov Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main   #19184      +/-   ##
==========================================
+ Coverage   84.15%   84.65%   +0.50%     
==========================================
  Files        3226     2337     -889     
  Lines      155844   134037   -21807     
  Branches    42175    34548    -7627     
==========================================
- Hits       131146   113469   -17677     
+ Misses      22970    20214    -2756     
+ Partials     1728      354    -1374     
Flag Coverage Δ
billing-web-unit ?
unit 81.38% <ø> (ø)
website-unit ?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 971 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@glary-bot

glary-bot Bot commented Sep 28, 2026

Copy link
Copy Markdown

Re: "Implemented in #19219" — I checked that branch against the two test-side findings, running them on fix/fe-3060-agent-gate-lifecycle rather than reading the diff. Neither is covered, and it collides with #19184. Flagging before it merges, since it's slated for all four backports.

1. The order-dependent test survives. It still snapshots storage before the first authenticated refresh:

$ pnpm vitest run src/platform/remoteConfig/refreshRemoteConfig.test.ts
  Tests  19 passed (19)

$ pnpm vitest run … -t "records this session's grant"
  × records this session's grant without persisting it anywhere
  +   "billing_control_enabled": "false",
  +   "v1_payment_recovery": "false",
  Tests  1 failed | 18 skipped (19)

Marginally worse than on main — two leaked keys instead of one, since cachedV1PaymentRecovery also settles on that first refresh.

2. The sign-out grant clear is still unpinned. Deleting sessionAgentGrant.value = undefined from invalidateRemoteConfig on the #19219 branch leaves 121/121 green. The 401/403 clear is covered by drops the grant when auth itself is rejected — which is exactly why this reads as covered at a glance — but the identity-change path isn't. That's the in-session analogue of PM-1707: account B inheriting account A's panel in the window before the new authenticated /features lands.

To be clear, this isn't a criticism of #19219's scope. Bounding transient grants to 15 minutes directly closes the unbounded-retention residue I flagged, and verifying /features actually carried credentials is a better fix than anything I'd proposed for the auth-phase problem. These two are simply orthogonal: one is a test-isolation bug, the other a coverage hole.

3. They conflict. git merge-tree reports 5 conflict markers between #19184 and #19219 — both rewrite the same test.

Suggestion: fold #19184 into #19219 and close #19184. It's 13 lines in a file you're already editing, and it means the fix rides the same four backports instead of needing a second round. The substance: run an ungranted refreshRemoteConfig() before taking the snapshots, so the storage-backed caches settle through real production code and the grant becomes the only possible delta — verified to still catch a persisted grant under keys agent-in-app-experience, session_whitelist and q, in both run orders. Plus two lines on the existing invalidation test.

Alternatively I'll rebase #19184 onto your branch and keep them separate. Your call — I won't push to your branch either way.

@glary-bot

glary-bot Bot commented Sep 28, 2026

Copy link
Copy Markdown

Correction and an update — #19219 has moved since my last comment (ef71a98a49 → 2a17c079b4). I cited that commit by hash, so re-verifying rather than letting a stale claim stand.

Good news first: the settle-lifecycle residues I raised are genuinely closed. Confirmed in the new setupFlagGate:

  • authenticatedRemoteConfigState replaces the undifferentiated remoteConfigState, so an anonymous bootstrap failure no longer settles the gate as "decided" — that was my sharpest residual finding and it's properly fixed
  • the signed-out fallback is now scoped (if (isSignedOut()) settle()) rather than firing unconditionally
  • onScopeDispose(() => clearTimeout(settleTimer)) cleans the timer up
  • remoteConfigRevision in the watch source retries consent on every completed refresh — which also addresses the re-offer channel I'd flagged as deliberately dropped

Plus tests for each, including does not settle a signed-in gate while authenticated config is slow and returns the gate to unsettled when authenticated config reloads. That's a better design than what I left behind; consider those items closed from my side.

The two test-side findings still stand on the new head, re-run just now:

$ pnpm vitest run src/platform/remoteConfig/refreshRemoteConfig.test.ts
  Tests  19 passed (19)

$ pnpm vitest run … -t "records this session's grant"
  × records this session's grant without persisting it anywhere
  +   "billing_control_enabled": "false",
  +   "v1_payment_recovery": "false",
  Tests  1 failed | 18 skipped (19)

And the sign-out clear is still unpinned — deleting sessionAgentGrant.value = undefined from invalidateRemoteConfig on 2a17c079b4 leaves 121/121 green. Worth stressing this one isn't hypothetical: returns the gate to unsettled when authenticated config reloads shows readiness reset is now deliberate behaviour, which makes the grant-clear on that same invalidation path more load-bearing than before, not less.

Neither is a criticism of #19219's scope — both are orthogonal to what it set out to fix. The offer stands: fold #19184's 13 lines in (it's a file you're already editing, and it rides the same four backports), or I rebase #19184 onto your branch. Still haven't touched it.

…sistence-test-isolation

# Conflicts:
#	src/platform/remoteConfig/refreshRemoteConfig.test.ts
@christian-byrne
christian-byrne marked this pull request as ready for review September 28, 2026 23:00
@christian-byrne
christian-byrne requested a review from a team September 28, 2026 23:00
@christian-byrne christian-byrne added the cursor-review Trigger multi-model Cursor agent review on this PR label Sep 28, 2026
@christian-byrne

Copy link
Copy Markdown
Contributor Author

Refreshed onto current main at 91e0c0c98 and preserved the two mutation-proven test guarantees: the grant no-persistence check is order-independent, and identity invalidation clears the session grant.

Local exact-head evidence: 24/24 focused tests pass, the previously order-dependent test passes alone, and format/lint/typecheck pass (lint reports only pre-existing warnings outside this one-file diff). Marked ready and triggered current-head review.

@christian-byrne christian-byrne added agent Comfy Agent product, integration, tests or product documentation agent-backport Agent changes targeting a stable cloud/core release branch agent-test-unit Touches Agent-related unit/component tests or their support cloud/1.54 Backport PRs for cloud 1.54 cloud/1.55 Backport PRs for cloud 1.55 core/1.54 Backport PRs for core 1.54 core/1.55 Backport PRs for core 1.55 needs-backport Fix/change that needs to be cherry-picked to the current feature freeze branch labels Sep 28, 2026

@christian-byrne christian-byrne left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head self-review at 91e0c0c98: the diff is limited to one test file. The baseline refresh now occurs before storage snapshots, so the grant is the only possible delta; the invalidation case seeds a grant and asserts that identity invalidation clears it. No production behavior, API, performance, security boundary, docs, or browser-visible flow changes. Full-file and isolated-order tests pass, and the existing mutation evidence remains the causal coverage proof.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Sep 28, 2026
@glary-bot

glary-bot Bot commented Sep 28, 2026

Copy link
Copy Markdown

Verified your rebase at 91e0c0c98a independently, and all three guarantees hold:

guarantee check result
order-independence full file / isolated -t 24/24 · 1/1 — fixed
invalidation clears the grant remove clearSessionAgentGrant() from invalidateRemoteConfig caught (expected true to be undefined)
grant never persisted swap ref → useStorage under an arbitrary key caught (1 failed)

Your summary is accurate. Nothing further from me.


A correction I owe you, because it's mine and it's load-bearing.

Earlier in this thread I reported that deleting the sign-out grant clear "leaves 121/121 green" and "126/126 green" — i.e. that the invalidation path was unpinned. At least one of those was a false negative, and I can't fully stand behind either.

My mutation script located the line by counting, then removed it with a str.replace() on an exact-indentation pattern — and I asserted only on the count, never that the edit actually landed. When #19219 refactored the inline sessionAgentGrant.value = undefined into clearSessionAgentGrant(), my pattern stopped matching. The script reported success, changed nothing, and the suite passed for the trivial reason that production code was untouched. A green result from a mutation that never mutated reads exactly like "this isn't covered."

I caught it here only because the numbers stopped making sense: the test visibly seeds sessionAgentGrant.value = true and asserts toBeUndefined(), so it had to fail under a real deletion. Re-running with a line-index removal and a printed before/after call-site count (5 → 4) turned it red immediately.

The fix on my side is a harness that verifies the file changed before trusting the run — which is the same lesson this PR is about, one level up: an assertion that can't fail is worse than no assertion, whether it's in a test or in the tool you're using to check the test.

Sorry for the noise on #19074 and #19219. The order-independence and no-persistence findings were separately reproduced and stand; the "invalidation unpinned" claim should be struck.

My local copy is still at the pre-rebase 5c0b3111 and I have not pushed — 91e0c0c98a is yours and I'm leaving it alone.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @christian-byrne.

Round 2 — ledger: 3 prior finding(s) across 1 round(s) (0 never answered).

Found 10 finding(s).

Severity Count
🟠 High 3
🟡 Medium 7

Panel: 8/8 reviewers contributed findings.


The finding(s) below could not be anchored to a line the reviewed diff carries, so they are reported here instead of inline:

.github/workflows/workshop-workflow-sweep.yaml:24 — 🟠 High — queue: max is not a recognized key under concurrency — GitHub Actions accepts only group and cancel-in-progress there, and an unknown key fails workflow validation ("Unexpected value 'queue'"), so neither the six-hourly free sweep nor the daily paid run would ever execute. The failure surfaces as a repo-level invalid-workflow error rather than a failed run, so nothing here would ever go red to tell you.

Raised by 3 of 8 reviewers (claude-opus-5-thinking-max edge-case, kimi-k3-max adversarial, kimi-k3-max edge-case).

src/platform/workspace/stores/workspaceAuthStore.ts:211 — 🟠 High — switchSessionWorkspace sets currentWorkspace.value = null before awaiting the confirming /workspaces/current call and leaves it null when that call is refused or returns a mismatched id. Because requestScope() omits X-Comfy-Workspace-ID for a null workspace, and an absent header means the personal workspace server-side, any request racing the switch — or any request after a failed switch or after dropDeniedWorkspace — silently reads and writes in the personal workspace instead of erroring. Keep the previous scope until the new one is confirmed, and represent "unresolved" distinctly from "personal" so those requests fail closed.

Raised by 3 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k3-max adversarial, kimi-k3-max edge-case).

src/platform/auth/session/cloudWebSessionStore.ts:180 — 🟠 High — requestScope() awaits the session-readiness promise with no timeout and no caller signal, and api.fetchApi does not create its 60-second controller until after getWebSessionSend() has awaited this, so a boot GET /auth/session that never settles hangs every API request and the workspace auth gate indefinitely. The token rail bounds the equivalent wait (waitForAuthInitialization, 10s cap); give ready a comparable deadline so a stalled probe degrades to an error rather than silently freezing all traffic.

Raised by 2 of 8 reviewers (gpt-5.6-sol-max adversarial, claude-opus-5-thinking-max adversarial).

src/platform/auth/session/cloudWebSessionStore.ts:197 — 🟡 Medium — When readWebSession reports that the cookie now belongs to a different account, rereadFor discards the result and returns undefined, leaving the store signed_in as the previous user with that user's CSRF token and workspace. Every unsafe request then repeats the stale-token 403-and-reread cycle until the next heartbeat (indefinitely while the tab is hidden), while safe requests already run under the new cookie with the old workspace metadata; propagate the identity change into identity instead of dropping it on the floor.

Raised by 1 of 8 reviewers (gpt-5.6-sol-max adversarial).

src/stores/authStore.ts:244 — 🟡 Medium — On the web-session rail getAuthHeader() short-circuits to getUserAuthHeader(), returning only the account-scoped Firebase bearer with no workspace header and no CSRF token, so any caller that authenticates its own request with this instead of going through fetchApi/fetchOnWebSession loses team scoping and resolves against the personal workspace. getWorkspaceAuthHeader has no session branch at all, so the axios workspaceApi calls (members, invites, billing mutations) still mint Firebase/workspace tokens — worth confirming which callers remain on that path, since the session and the local Firebase user can name different accounts.

Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, gpt-5.6-sol-max adversarial).

apps/billing-web/src/session/unifiedBillingSession.ts:158 — 🟡 Medium — The identity subscriber clears workspace.value on every state push, not just on an account change, so a routine session republish for the same user (refresh, cross-tab sync) blanks the resolved workspace for a full /workspaces/current round trip: livePhase goes undefined, billedScope blanks the UI, and scopeSource.getScope() returns undefined for anything issued in that window — the same headerless window the rebind clear on line 139 opens. Gate this clear on the user id actually changing (as resolveWorkspace() already does for the binding), and confirm createCredentialedBillingTransport blocks an unresolved scope rather than sending the request with no workspace header.

Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k3-max adversarial).

apps/billing-web/src/session/unifiedBillingSession.ts:126 — 🟡 Medium — fetchWorkspace accepts whatever /workspaces/current returns without checking it matches the requested binding, so if X-Comfy-Workspace-ID is dropped by a proxy or ignored by the backend, the personal workspace is stored as the resolved billing scope and checkout charges it while the entry link still names the team. workspaceAuthStore.switchSessionWorkspace, added in this same PR, guards exactly this case (current.data.id !== workspaceId throws workspace-switch-mismatch); apply the same comparison here and treat a mismatch as a resolution failure.

Raised by 1 of 8 reviewers (claude-opus-5-thinking-max adversarial).

apps/billing-web/src/views/CheckoutView.vue:471 — 🟡 Medium — The REACTIVATION_CONFIRMATION_REQUIRED branch re-quotes with no promotion code, and the applied code lives only on the preview it is about to replace, so a customer who applied a valid promo and then hit server-side reactivation confirmation silently loses the discount: the new preview is undiscounted and buildSubscribeRequest reads promotion_code off that preview, so the subscribe carries none. Pass the code through the re-quote (preview.value?.promotion_code) or retain it in component state.

Raised by 1 of 8 reviewers (claude-opus-5-thinking-max edge-case).

apps/billing-web/src/views/CheckoutView.vue:232 — 🟡 Medium — teamCheckoutPlan falls back to monthlyUsd = 0 / monthlyCredits = 0 whenever the stop is missing from plans.value?.team_credit_stops, and team plans are pricedByQuote: false, so the confirm renders "$0 USD / mo" and "0 credits" for as long as usePlans() is still loading (the only load gate is quoting, which watches the preview, not the plans) and forever if the link names a stop the catalog no longer lists — with Pay enabled throughout, so a customer can submit a charge against a page showing $0. Gate the confirm on the plans read, or price the stop from the quote rather than failing open to zero.

Raised by 2 of 8 reviewers (claude-opus-5-thinking-max edge-case, kimi-k3-max adversarial).

apps/website/src/components/workshop/WorkflowPlayground.vue:303 — 🟡 Medium — WorkflowCreditsGuard renders its default slot only when the gate resolves to 'run', and workflowCreditsGate returns 'run' only while busy or funded, so wrapping all of WorkflowRunControls means a zero or refused balance in a non-busy terminal phase (e.g. interrupted) removes Reconnect and Dismiss entirely — precisely the state whose own footer copy warns the run "may still be running on Cloud and using credits", leaving the user no way to reconnect or clear it. Gate only the start affordance, not the whole control block.

Raised by 1 of 8 reviewers (claude-opus-5-thinking-max edge-case).

@christian-byrne

Copy link
Copy Markdown
Contributor Author

The exact-head Cursor panel completed, but its ten findings are outside this PR. origin/main...1427d9834 contains only src/platform/remoteConfig/refreshRemoteConfig.test.ts; none of the workflow, workspace-auth, billing, or website paths named by the panel are changed by #19184 (the order-independent Agent grant regression tests). No unrelated production changes belong in this PR. The Cursor gate and consolidated panel passed; one model job was cancelled after the panel received contributions from all eight reviewers.

@glary-bot

glary-bot Bot commented Sep 29, 2026

Copy link
Copy Markdown

Independently confirmed the scope call — dismissing ten automated findings is worth a second pair of eyes rather than taking on trust.

git diff origin/main...1427d9834a --name-only returns exactly one path:

src/platform/remoteConfig/refreshRemoteConfig.test.ts

And zero matches across every path family the panel named:

family files touched
workflow 0
workspace-auth 0
workspace 0
billing 0
website 0

So the ten findings are genuinely outside this diff, and none of them could be introduced by it. Your read is right, and pulling unrelated production changes in to satisfy a panel that mis-scoped its input would be the wrong fix.

One item that postdates your comment: main has moved again, 95932b424c → be3e1a1308. Still fine — zero commits have touched this PR's one file since the base, and a real git merge --no-commit against the newest main is clean (exit 0, no conflicted paths). That's now the third main advance this branch has outlived without drift, which is what you'd expect from a one-file test-only diff.

Nothing outstanding from me. Head 1427d9834a carries the full mutation matrix I verified — order-independence in both run orders, plus independent falsifiers for grant persistence, the invalidation clear, the deadline clear, and the priming.

@DrJKL DrJKL removed their assignment Sep 29, 2026
@glary-bot

glary-bot Bot commented Sep 29, 2026

Copy link
Copy Markdown

Thanks @DrJKL. One thing worth flagging before merge: the head moved after your review. You reviewed 1427d9834a; it's now 0e15c1caf4, a merge of main into the branch. GitHub may or may not surface your approval as stale, so I re-verified rather than assume it carries over.

main brought in a large agent refactor on the way (#18700 deletes the reconcile layer; agentPanel.ts ±90, agentPanelRoot ±183, the whole CRDT tree reshaped). None of it touched the files these tests pin — git diff 1427d9834a..0e15c1caf4 -- src/platform/remoteConfig/ is empty — but "probably fine" isn't the standard this PR has been held to, so I re-ran the whole matrix on the merged head:

check result
full file / isolated -t / whole dir 24/24 · 1/1 · 24/24
M1 persist grant via useStorage, arbitrary key caught
M2 remove clearSessionAgentGrant() from invalidation caught
M3 remove only the deadline clear caught (×3)
M4 remove the four priming lines caught in both orders

Each edit confirmed applied before its run (M2 5 → 4, M3 1 → 0, M4 4 → 0). Identical to 1427d9834a — nothing drifted.

Branch contribution is still exactly one file against current main:

src/platform/remoteConfig/refreshRemoteConfig.test.ts

So the scope conclusion from the Cursor-panel thread holds unchanged too.

Nothing outstanding from me. If the approval did go stale on the merge commit, this is the evidence for re-approving 0e15c1caf4 without re-reviewing from scratch.

@github-actions github-actions Bot added risk:medium PR risk grade (advisory shadow check; grader-owned) and removed risk:high PR risk grade (advisory shadow check; grader-owned) labels Sep 29, 2026
@christian-byrne
christian-byrne added this pull request to the merge queue Sep 29, 2026
Merged via the queue into main with commit 37ce436 Sep 29, 2026
89 checks passed
@christian-byrne
christian-byrne deleted the glary/agent-grant-persistence-test-isolation branch September 29, 2026 09:16
@comfy-pr-bot

Copy link
Copy Markdown
Member

@christian-byrne Successfully backported to #19395

@comfy-pr-bot

Copy link
Copy Markdown
Member

@christian-byrne Successfully backported to #19396

@comfy-pr-bot

Copy link
Copy Markdown
Member

@christian-byrne Successfully backported to #19397

@comfy-pr-bot

Copy link
Copy Markdown
Member

@christian-byrne Successfully backported to #19398

@comfy-pr-bot comfy-pr-bot removed the needs-backport Fix/change that needs to be cherry-picked to the current feature freeze branch label Sep 29, 2026
comfy-pr-bot added a commit that referenced this pull request Sep 29, 2026
… order-independent and pin the sign-out clear (#19395)

Backport of #19184 to `core/1.54`

Automatically created by backport workflow.

Co-authored-by: Christian Byrne <cbyrne@comfy.org>
Co-authored-by: Glary-Bot <glary-bot@users.noreply.github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
comfy-pr-bot added a commit that referenced this pull request Sep 29, 2026
… order-independent and pin the sign-out clear (#19397)

Backport of #19184 to `core/1.55`

Automatically created by backport workflow.

Co-authored-by: Christian Byrne <cbyrne@comfy.org>
Co-authored-by: Glary-Bot <glary-bot@users.noreply.github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
comfy-pr-bot added a commit that referenced this pull request Sep 29, 2026
…k order-independent and pin the sign-out clear (#19396)

Backport of #19184 to `cloud/1.54`

Automatically created by backport workflow.

Co-authored-by: Christian Byrne <cbyrne@comfy.org>
Co-authored-by: Glary-Bot <glary-bot@users.noreply.github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
comfy-pr-bot added a commit that referenced this pull request Sep 29, 2026
…k order-independent and pin the sign-out clear (#19398)

Backport of #19184 to `cloud/1.55`

Automatically created by backport workflow.

Co-authored-by: Christian Byrne <cbyrne@comfy.org>
Co-authored-by: Glary-Bot <glary-bot@users.noreply.github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
@comfy-pr-bot comfy-pr-bot added the released:cloud PR has been released to cloud label Oct 1, 2026
j0n4t pushed a commit to j0n4t/ComfyUI_frontend that referenced this pull request Nov 4, 2026
…y-Org#20742)

*PR Created by the Glary-Bot Agent*

---

## Summary

Adds a regression guard for FE-3332 and FE-2812 (Pylon Comfy-Org#19184). From Sep
9 to Sep 30, users' Imported tab on Cloud showed every base-template
input. The cause: Comfy-Org#15381 moved the Assets panel to `useAssetsQuery`
without `include_public=false`. Cloud ingest defaults that parameter to
`true`, so the shared template-input catalog came back with the user's
own uploads. Comfy-Org#18633 restored the parameter (backported as Comfy-Org#19542,
shipped in `cloud/v1.55.15`), but no test pinned it.

The new `@cloud` spec
`browser_tests/tests/sidebar/assets-imported-public.spec.ts` stubs
`/api/assets` to match the server default: it returns public assets
unless the request sends `include_public=false`. It then asserts:
- the Imported tab shows only the user's own input;
- the Generated tab shows no other user's assets;
- every panel request (`tags_any=input` and `tags_any=output`) sends
`include_public=false`.

Test only, no production code changes. Companion PR in
`Comfy-Org/cloud`: Comfy-Org/cloud#12841. It
pins the same contract at the ingest handler and in the daily
staging/prod e2e.

## Verification

- Ran `pnpm build:cloud`, served `dist/` from local ComfyUI (`--cpu
--multi-user` plus devtools, same as CI), then `pnpm exec playwright
test --project=cloud
browser_tests/tests/sidebar/assets-imported-public.spec.ts`: **2 passed
(~12s)**.
- Negative control: I removed `include_public: false` from `BASE_PARAMS`
in `useAssetsQuery.ts` and rebuilt. **Both tests fail.**
`template_sample_a` and `published_render` cards render,
`toHaveCount(0)` gets 1, and the parameter assertion fails. Then I
restored the line.
- oxfmt, oxlint (type-aware), `typecheck:browser` and the server-facts
gate passed via pre-commit.
- Screenshots come from the same spec: with the fix, the Imported tab
shows only `my_upload`. With `include_public` removed, extra
template-input cards appear.

## Screenshots

![Imported tab with include_public=false: only the user's own upload is
listed](https://pub-1fd11710d4c8405b948c9edc4287a3f2.r2.dev/sessions/b243707d77b5c39371e8e47cf5ecef811fed0c9acb9a4f02b6719c2b3a3e7bd0/pr-images/1791668381531-c02c699e-5a0d-46aa-a0c0-30b5c9cbbe57.png)

![Negative control with include_public removed: public template-input
cards appear in the Imported
tab](https://pub-1fd11710d4c8405b948c9edc4287a3f2.r2.dev/sessions/b243707d77b5c39371e8e47cf5ecef811fed0c9acb9a4f02b6719c2b3a3e7bd0/pr-images/1791668381915-e5d8d6d4-f67f-4804-8dbc-514bd08bbfc1.png)

---------

Co-authored-by: Glary-Bot <glary-bot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Comfy Agent product, integration, tests or product documentation agent-backport Agent changes targeting a stable cloud/core release branch agent-test-unit Touches Agent-related unit/component tests or their support cloud/1.54 Backport PRs for cloud 1.54 cloud/1.55 Backport PRs for cloud 1.55 core/1.54 Backport PRs for core 1.54 core/1.55 Backport PRs for core 1.55 cursor-review Trigger multi-model Cursor agent review on this PR greenlight released:cloud PR has been released to cloud risk:medium PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants