Skip to content

[backport core/1.55] fix(agent): resolve the agent gate off /features - #19175

Merged
christian-byrne merged 1 commit into
core/1.55from
backport-19074-to-core-1.55
Sep 28, 2026
Merged

christian-byrne merged 1 commit into
core/1.55from
backport-19074-to-core-1.55

Conversation

@christian-byrne

Copy link
Copy Markdown
Contributor

Backport of #19074 to core/1.55.\n\nManual fallback after the automatic backport run remained queued; conflicts were resolved by carrying the reviewed agent-panel test and removing the superseded dependency-failure test alongside the production implementation.

@christian-byrne christian-byrne added backport Backporting a PR onto a release candidate agent-backport Agent changes targeting a stable cloud/core release branch greenlight labels Sep 28, 2026
@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Ignore keyword(s) in the title.

⛔ Ignored keywords (2)
  • [release]
  • [backport

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: dd0b7f85-a189-4dcc-a563-f9071c048bf3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

🌾 Fallow: ❌ Failed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

Fallow reports a failing verdict, but this renderer found no findings it
knows how to describe. Open the run log and the Audit new findings
step — and please extend scripts/cicd/fallow-findings.ts so the next
person does not have to.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since e81c461e1e2b7deb7ff54c78260f4d35361e1d2a

Full docs: ADR-DEVEX-LINT-0015

🎭 Playwright: ✅ 2125 passed, 0 failed

📊 Browser Reports
  • chromium: View Report (✅ 2104 / ❌ 0 / ⚠️ 0 / ⏭️ 13)
  • chromium-2x: View Report (✅ 2 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • chromium-0.5x: View Report (✅ 1 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • mobile-chrome: View Report (✅ 18 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • New-test walkthrough (chromium, recorded video): View Report

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 09/28/2026, 04:13:33 AM UTC

Links

📦 Bundle Size

⏳ Size data collection in progress…

⚡ Performance

⚠️ Performance tests failed. Check the CI workflow logs.

🔬 E2E Coverage

⚠️ Coverage collection failed. Check the CI workflow logs.

🧩 Custom Nodes Ecosystem Matrix: ✅ Passed

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

Stage Result What a failure here means
pin-status success Informational. It never fails the build; it warns when the corpus snapshot is over 30 days old.
corpus success Infrastructure: the pack mirror could not be fetched, validated or cached. Not your diff.
shards success Harness integrity, not pack breakage. A shard only checks that every built spec wrote its row; vitest's own exit code is deliberately ignored because pack code leaks unhandled rejections.
verdicts success The population verdict per renderer. This is the stage that judges packs. Exit 1 is FAIL; exit 2 is withheld — rows were missing, so nothing was measured, which is not the same as your change breaking packs.
detection-proofs success The instrument self-test: the matrix must FAIL on a deliberately poisoned corpus. A failure here means the gate cannot be trusted this run, in either direction.

On a pin-bump PR, a red gate means the ecosystem moved — not that your diff broke something. That is what the bump PR is for. Outside a pin bump or a cold rebuild of the unpinned tail, a red means the diff.

Note the corpus size and the executed count are different numbers, and the verdict is against the executed count: roughly 3,100 of ~5,100 packs ship no extension-shaped JS and are skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Sep 28, 2026
@datadog-official

This comment has been minimized.

…og-js flag path (#19074)

*PR Created by the Glary-Bot Agent*

---

Fixes the root cause behind
[PM-1707](https://linear.app/comfyorg/issue/PM-1707/test-agentcomfyorg-shows-the-agent-on-my-phone-but-not-on-desktop-in)
/
[PM-1706](https://linear.app/comfyorg/issue/PM-1706/fix-agent-does-not-appear-on-desktop-browsers-for-a-user-while-it)
— an allowlisted user saw the agent on one browser but not another.
Supersedes #18842, which only characterized the old behaviour.

`agent-in-app-experience` was already published on `/features` (it's in
`FrontendFlags` in the cloud repo, evaluated server-side per
authenticated user) and the app already fetched that payload into
`window.__CONFIG__` at bootstrap. The agent gate ignored it and re-asked
PostHog over a second, independently-failing client path — the only flag
in the codebase still doing so. Billing was migrated to this rail in

Because posthog-js keys flags by its own `distinct_id` and caches them
per-origin in `localStorage` while sharing identity cross-subdomain via
cookie, two browsers on the same account could disagree. That's the
divergence users saw.

`agent-in-app-experience` is now a `ServerFeatureFlag` resolved through
`useFeatureFlags` off `remoteConfig`, via a new
`resolveWhitelistFlag()`:

- `false` off the Cloud build, and `false` before the first
authenticated answer (the anonymous bootstrap payload cannot answer a
per-user allowlist)
- after an authenticated answer, a *failed* refresh falls back to that
answer, held **in memory for the session only** — never storage. A
persisted grant is precisely what made two browsers disagree.
- revocation propagates on the next successful poll; 401/403 and
sign-out clear it

`setupFlagGate` loses the dynamic imports, the `onFeatureFlags`
subscription and the delivery plumbing, becoming two reactive watches.
Deleted: `postHogFlagSource.ts`, `useAgentFeatureGate.ts` (dead code —
nothing but its own test imported it, and it lacked the `errorsLoading`
guard, so the two copies behaved differently), and their tests.

`posthog-js` stays — telemetry still uses it. This removes only the flag
path.

Every guard is mutation-tested; each of these turns the suite red:

| Mutation | |
|---|---|
| anonymous-window guard removed (fail-open before auth) | caught |
| `isCloud` guard removed | caught |
| fail-open before the first authenticated answer | caught |
| grant fallback removed (a blip unmounts the panel) | caught |
| grant survives sign-out / 401 | caught |
| grant never recorded on a successful load | caught |
| **grant persisted via `useStorage`** — the PM-1707 shape, under either
the flag key *or* a renamed key | caught |
| gate `enabled` unwired from the flag | caught |
| gate settles on the anonymous config | caught |
| signed-out settle fallback removed | caught |

`pnpm typecheck`, `lint`, `knip`, `format:check` clean; 12,129 unit
tests pass. The one broad-run failure (`agentSubgraphDefinitions`)
reproduces identically on `main`.

Two bugs were caught during review and fixed here rather than shipped: a
transient `/features` failure would have unmounted the panel mid-session
for a granted user (`DockedAgentPanel` is `v-if`, so composer draft and
in-flight turn UI are destroyed), and the E2E fixtures were still
feeding the flag through the deleted posthog channel, which would have
regressed the whole agent E2E suite to a dead panel.

- **E2E fixtures changed.** All three now put the key at the top level
of the `/features` body. Specs can't be executed in my sandbox; the
static chain is verified end to end (`cloudBootMocks` serves that body
to both fetches, and `WorkspaceAuthGate` throws unless state is
`authenticated`, so any spec that renders has reached the state the
resolver needs). **One CI run of `--project=cloud` on a single agent
spec would settle it conclusively.**
- **Retired telemetry slug.** `agent_flag_gate_load_failure` no longer
exists — the guarded dynamic import it reported on is gone. Any monitor
keyed on it will go silently to zero.
- **New Datadog facet.** The flag is added to
`startFeatureFlagTelemetry`. It's the first dashed `ServerFeatureFlag`,
and the sanitiser rewrites dashes — the RUM facet is
`agent_in_app_experience`, not the hyphenated name.
- **Deliberately dropped:** an in-session re-offer retry that existed
only because `posthog.onFeatureFlags` re-delivered unchanged values. The
real retry triggers (`offerHeld && screenIsClear`, identity change) are
unaffected and still covered.
- **Known residue:** the grant-retention window is unbounded, so a
sustained `/features` outage leaves a revoked user with an inert panel
until reload. Low impact — the backend enforces the flag independently
at four choke points, so every call 404s regardless.

---------

Co-authored-by: Glary-Bot <glary-bot@users.noreply.github.com>
@christian-byrne
christian-byrne force-pushed the backport-19074-to-core-1.55 branch from 1836b77 to 7bdd97a Compare September 28, 2026 04:01
@christian-byrne
christian-byrne merged commit 45b2d07 into core/1.55 Sep 28, 2026
37 of 38 checks passed
@christian-byrne
christian-byrne deleted the backport-19074-to-core-1.55 branch September 28, 2026 04:03
@codecov

codecov Bot commented Sep 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@             Coverage Diff             @@
##           core/1.55   #19175    +/-   ##
===========================================
  Coverage      84.65%   84.65%            
===========================================
  Files           2302     2301     -1     
  Lines         131881   132015   +134     
  Branches       35615    35652    +37     
===========================================
+ Hits          111649   111763   +114     
- Misses         19913    19933    +20     
  Partials         319      319            
Flag Coverage Δ
unit 80.58% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/composables/__mocks__/useFeatureFlags.ts 87.50% <ø> (ø)
src/composables/useFeatureFlags.ts 99.35% <100.00%> (+0.05%) ⬆️
src/extensions/core/agentPanel.ts 98.87% <100.00%> (ø)
src/extensions/core/index.ts 6.66% <ø> (ø)
...rc/platform/remoteConfig/__mocks__/remoteConfig.ts 86.95% <ø> (ø)
src/platform/remoteConfig/refreshRemoteConfig.ts 97.29% <100.00%> (+0.11%) ⬆️
src/platform/remoteConfig/remoteConfig.ts 100.00% <100.00%> (ø)

... and 15 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

lover520f pushed a commit to lover520f/ComfyUI_frontend-2026 that referenced this pull request Nov 15, 2026
Comfy-Org#23404)

> **Do not merge yet.** Ready for review, not for merge. Merge only
after the Cloud release that contains Comfy-Org/cloud#19173 and
Comfy-Org/cloud#19175 is running on staging and production. On Cloud
`main` today the ingest image is pinned to `v0.546.15` for both
`comfy-cloud-stg-v2` and `comfy-cloud-prod-v2`, and that tag contains
neither PR; only `comfy-cloud-test-v2` (`9d3282f`) has them. The staging
frontend follows this repo's `main`, so merging earlier would make the
billing capabilities and auto top-up reads fail on staging, where After
M4 QA is running. Checked from the repo's overlay values, not from the
live cluster.

## Summary

The bot sync Comfy-Org#23331 is red. Its regenerated types make three response
fields required, and the fixtures that build those responses no longer
typecheck. We need the sync for `GET /api/billing/ops/{id}/outcome`
(`BillingOpOutcomeResponse`, Comfy-Org/cloud#19213), which the
billing-web follow-up for FRONT-3855 reads.

This PR takes the bot's generated files unchanged (`update-ingest-types`
at d4a238c, cloud@9d3282f6) on top of `main` and fixes the fallout.

Supersedes Comfy-Org#23331

## Dependency graph

```mermaid
graph LR
  A["Comfy-Org/cloud#19213<br/>billing op outcome route, merged"] --> B["This PR<br/>ingest-types sync"]
  B --> C["billing-web follow-up for FRONT-3855<br/>reads the outcome route, not opened yet"]
  D["Comfy-Org/cloud#19173 and Comfy-Org#19175<br/>new required fields, merged, deploy first"] -.-> B
  style B fill:#2563eb,color:#fff
```

## What the sync brings

- New route: `GET /api/billing/ops/{id}/outcome` with
`BillingOpOutcomeResponse` (`status` is `pending`, `succeeded`, `failed`
or `reconciliation_needed`) and `zGetBillingOpOutcomeResponse`.
- Removed route: `GET /api/v1/catalog/docs` (`GetCatalogDocs*`). Nothing
in the repo imports it.
- New required fields: `BillingCapabilities.can_set_budgets` and
`can_set_hard_caps` (cloud#19173), `auto_top_up_conflict` on
`AutoReloadSettings`, `BudgetList` and `BudgetWriteResult`
(cloud#19175), and `CccPageRow.on_desktop`. Nothing in the repo builds
or reads `BudgetList`, `BudgetWriteResult` or `CccPageRow`.
- Enum change: `denied_reasons` gains the keys `can_set_budgets` and
`can_set_hard_caps`, and with them one new reason,
`plan_not_enterprise`.
- New error types: `GetCatalogAgentInstructionsError` and
`GetCatalogOpenApiError` (400, 401 and 403 added).
- Doc-only changes on `BillingOpStatusResponse` and `SubscribeRequest`.

## Fallout fixed

| Files | What was missing | Value and reason |
| --- | --- | --- |
| 21 `BillingCapabilities` literals in 20 fixture, mock and test files
under `src/platform/{auth,billing}`, `packages/account-core`,
`packages/account-ui`, `apps/billing-web` and `browser_tests` |
`can_set_budgets`, `can_set_hard_caps` | `false` for both. Ingest serves
`false` when billing-api omits the field, and no code reads either one.
|
| 12 `AutoReloadSettings` literals under `src/platform/billing`,
`packages/account-core`, `packages/account-ui` and `browser_tests` |
`auto_top_up_conflict` | `false`, which the spec reads as "show no
warning". The schema is `z.boolean()`, not nullable. |
| `billingCapabilities.contract.test.ts` | the two capability keys in
the pinned type | Added, so the pinned shape equals the generated one. |

Some of these fixtures are untyped wire bodies that the Zod readers
parse, so they fail at run time and not at typecheck. With `main`'s
fixtures, 79 tests fail in four account-core files (`capabilities`,
`topup`, `scopedReader`, `autoTopUp`), and the billing-web browser specs
fail because the order summary never renders. Comfy-Org#23331 did not show this:
its unit job was skipped behind the typecheck failure, and its
`billing-web-e2e` job was cancelled at the 20-minute limit.

Production code has two changes:

- `packages/account-core/src/core/billing/capabilityDenials.ts`:
`CapabilityDenialReason` and `KNOWN_REASONS` gain `plan_not_enterprise`.
The contract test requires the client union to equal the server union.
- `apps/billing-web/src/checkout/endingScreen.ts`: `REFUSAL_COPY` maps
`plan_not_enterprise` to `unknown`, the generic copy. The record is
exhaustive over the union, so it needs the key.

## No user-visible change

No code reads the new fields or calls the new route. Cloud sends
`plan_not_enterprise` only under `denied_reasons.can_set_hard_caps`,
which nothing reads. Before this PR the reason decoded as `unspecified`,
which billing web also shows with the generic copy. There is nothing to
screenshot.

One condition applies. The generated Zod makes the three fields
required, and the SDK readers parse with it (`capabilities:
zBillingCapabilities` in `capabilities.ts`,
`zGetBillingAutoReloadResponse` in `autoTopUp.ts`). An ingest that does
not send the fields fails those reads, and billing-web checkout does not
render. Cloud main includes cloud#19173 and cloud#19175, both merged on
2026-11-13. The latest Cloud release tag, v0.546.15, predates both.
Deploy ingest with those two changes before a frontend that contains
this sync reaches users. Comfy-Org#22913 carried the same condition for its two
capability fields.

## Feature gate

Types and test data only. No behavior changes behind or outside any
flag.

## Verification

- Pre-commit hook: oxfmt, oxlint, eslint, `lint:server-facts`,
`typecheck:app`, `typecheck:browser`, `typecheck:billing-web`,
`typecheck:account-core` and `typecheck:account-ui` pass. The pre-push
hook (`deps:audit`, `knip`, `check:generated`) passes.
- Vitest: account-core `src/core/billing` 1373/1373, account-ui
`src/billing` 406/406, billing-web 1655/1655, app `src/platform/billing`
with the web-session and billing-tab tests 4713/4713.
- Playwright, local: billing-web `checkout.spec.ts` and
`full-page-topup.spec.ts` 31/31. Without the two capability fields in
the e2e fixture, `full-page-topup.spec.ts` fails.
- The generated files equal the bot branch: `git diff
origin/update-ingest-types HEAD -- packages/ingest-types` is empty.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-backport Agent changes targeting a stable cloud/core release branch backport Backporting a PR onto a release candidate greenlight risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants