Skip to content

test(agent): run the mid-turn tab switch on a tab that was never saved - #21663

Merged
skishore23 merged 1 commit into
skishore23/agent-tabswitch-midturn-reprofrom
oploop-883a/unsaved-tab-target
Oct 22, 2026
Merged

skishore23 merged 1 commit into
skishore23/agent-tabswitch-midturn-reprofrom
oploop-883a/unsaved-tab-target

Conversation

@christian-byrne

Copy link
Copy Markdown
Contributor

Summary

Stacked on #21657: makes its spec run on a tab that was actually never saved.

The conversation harness picks the seed tab in the target picker before the first turn, and picking a temporary tab saves it (saveWorkflowAs). So #21657's "unsaved tab" was workflows/Unsaved Workflow.json, isTemporary: false, and the turn posted a workflow_id instead of current_tab_unbound. The minted-workflow binding from the Slack report was never exercised.

This adds a workflowTarget harness option (default unchanged) and sets 'unsaved-draft' in the spec. The seed tab stays a draft the chat follows, the first turn mints the workflow, and the ack binds it.

Verification

Local, --project=cloud, ComfyUI e2e container + DISTRIBUTION=cloud dev server:

Frontend Mutation #21657 as-is this branch
head none pass ×3 pass ×3
head unsaved tab never adopts its minted workflow (onWorkflowAdopted refuses sent.id === undefined) pass (blind) fail: a third, agent-created tab appears (expected 2 tabs, got 3)
head tab-return resubscribe gets no catch-up fail fail
prod pin ba2c2c2269 none pass pass
  • Control: agentTabSwitchCatchUp.spec.ts 4/4 pass on the default target.
  • oxfmt, type-aware oxlint, typecheck:browser, pre-push audits pass.

Still a regression guard, not a red repro: the unsaved-draft variant is also green on the prod pin, so the prod symptom is in something the fake host does not model. The mutation's failure (edits going to a new agent tab) matches the agent's own explanation in the #agent-sorry transcripts ("the build went to another tab"), which points at the minted binding missing in prod rather than at the follower.

The conversation harness picks its seed tab in the target picker before the
first turn, and picking a temporary tab saves it. The spec named for an
unsaved tab therefore sent from a saved workflow with a cloud id and never
took the minted-workflow path the report is about: with the unsaved tab's
minted adoption disabled, both cases still passed.

Add a workflowTarget option that leaves the seed tab an unsaved draft the
chat follows, and use it here. The same mutation now fails both cases (the
edits land in a third, agent-created tab), and dropping the tab-return
catch-up still fails them.
@christian-byrne
christian-byrne requested review from a team as code owners October 22, 2026 05:32
@coderabbitai

coderabbitai Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Team
  • Run ID: b566dc2d-af6d-42e9-a25b-561dbf49dc84

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

🎭 Playwright: ✅ 3051 passed, 0 failed · 2 flaky

📊 Browser Reports
  • chromium: View Report (✅ 2311 / ❌ 0 / ⚠️ 1 / ⏭️ 0)
  • chromium-2x: View Report (✅ 2 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • chromium-0.5x: View Report (✅ 1 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • cloud: View Report (✅ 709 / ❌ 0 / ⚠️ 1 / ⏭️ 0)
  • desktop: View Report (✅ 6 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • mobile-chrome: View Report (✅ 19 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • mobile-safari: View Report (✅ 3 / ❌ 0 / ⚠️ 0 / ⏭️ 0)

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

No verdict is available. Open the workflow run to confirm the check was intentionally skipped, or fix the collection failure and re-run it.

Findings and check details

The matrix runs on same-repository version-bump PRs with relevant changes, main-branch pushes, and manual dispatches. This candidate does not qualify, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

Findings and check details

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since ef08a01a5c025096140b18be3cbdaa0249896fbc

Full docs: ADR-DEVEX-LINT-0015

🏛️ Architecture: ✅ Passed

Findings and check details

Warnings (139)

  • billing path matches no source files: src/components/dialog/content/credit/**. Remove or update it in docs/architecture/domains/records/billing.domain.json.
  • workspace path matches no source files: src/platform/workspace/fixtures/**. Remove or update it in docs/architecture/domains/records/workspace.domain.json.
  • src/components/graph/selectionToolbox/LinkSelectionActions.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/searchbox/v2/filterChip.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/Popover.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/card/Card.stories.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/card/Card.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/card/CardFooter.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/card/CardHeader.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/ui/card/card.variants.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/widgetHistory/WidgetHistoryButtons.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/components/widgetHistory/WidgetHistoryButtons.vue is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/billing/useSubscriptionPaywall.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/billing/useSubscriptionPaywall.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/graph/menuOption.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/useWidgetValueHistory.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/useWidgetValueHistory.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/widgetHistoryScope.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/composables/widgetHistoryScope.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/litegraphServiceAccess.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/nodeDef/ComfyNodeDefImpl.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/widgets/dynamicGroupDiagnostics.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/graph/widgets/dynamicGroupDiagnostics.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/core/selection/selectionIntent.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/agentPanel.historyIdentity.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/cameraInfo.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/cloudBadges.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/dynamicPrompts.serializeValue.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/load3dHost.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/load3dHost.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/load3dInstances.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/reportLoadModelOutcome.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/extensions/core/load3d/reportLoadModelOutcome.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/i18n.literalMessages.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/ContextMenu.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/LiteGraphGlobal.cloneObject.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/canvas/canvasClassAccess.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/canvas/linkHideTransition.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/canvas/linkHideTransition.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/canvas/linkSelection.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/canvas/linkSelection.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/canvas/linkSnapGeometry.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/lib/litegraph/src/canvas/linkSnapGeometry.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/platform/auth/authStoreAccess.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/platform/auth/currentUserAccess.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/platform/auth/session/sessionCookieAccess.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/platform/auth/session/sessionRevokedError.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/platform/auth/session/webSessionFetch.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/platform/auth/sso/ssoRequiredContext.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • src/platform/missingModel/releaseModelOptions.test.ts is not claimed by any domain. Add it to a module in docs/architecture/domains/records.
  • …and 89 more
Reproduce locally
pnpm architecture:check

Full docs: Frontend domain catalog

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Oct 22, 2026
@skishore23
skishore23 merged commit 5afce6c into skishore23/agent-tabswitch-midturn-repro Oct 22, 2026
103 checks passed
@skishore23
skishore23 deleted the oploop-883a/unsaved-tab-target branch October 22, 2026 05:43
@codecov

codecov Bot commented Oct 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@                             Coverage Diff                             @@
##           skishore23/agent-tabswitch-midturn-repro   #21663     +/-   ##
===========================================================================
  Coverage                                     87.50%   87.50%             
===========================================================================
  Files                                          2728     2728             
  Lines                                        150951   150951             
  Branches                                      39312    41442   +2130     
===========================================================================
  Hits                                         132089   132089             
  Misses                                        18229    18229             
  Partials                                        633      633             
Flag Coverage Δ
unit 85.88% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

smhc pushed a commit to smhc/ComfyUI_frontend that referenced this pull request Oct 22, 2026
…ck (Comfy-Org#21657)

## Summary

Adds browser coverage for a frequent `#agent-sorry` report: **"agent
said it created a workflow but tab is empty."** Repro: start a chat on a
new **unsaved** tab, switch to another tab while the agent is still
editing, then switch back. The tab is blank even though the agent says
the workflow is built.

`agentTabSwitchCatchUp.spec.ts` only switches after the turn has
finished, and only on a saved tab. This spec switches **mid-turn** on an
unsaved tab, so the remaining edits can only reach the canvas through
the catch-up sent when the tab resubscribes:

- **switch after the first edit:** the follower unsubscribes, the host
applies the remaining edits, and on return the catch-up must bring them
all in.
- **switch before any edit:** every edit lands while the tab is in the
background.

## Results

The spec runs with `workflowTarget: 'unsaved-draft'` (Comfy-Org#21663, folded
in). The seed tab stays a draft that was never saved, so the first turn
sends `current_tab_unbound`, the backend creates the workflow, and the
ack binds it to the tab. Without that option the harness's target picker
saves the tab first, and the creation path never runs.


| Frontend | after first edit | before any edit |
|---|---|---|
| `main` (3566677) | pass ×3 | pass ×3 |
| `main` + `unsaved-draft` (Comfy-Org#21663) | pass | pass |
| prod pin `ba2c2c2269` (cloud/1.57) | pass ×2 | pass ×2 |

Against a well-behaved host, the frontend binds the unsaved tab,
unsubscribes it in the background, resubscribes on return and applies
the catch-up. **This is a regression guard, not a red repro.**

The prod symptom is still real, so the gap must be in something the fake
host does not model. Candidates:
- the real lazy mint plus `doc_reset` on an unsaved tab's first turn;
- a missed or refused catch-up on resubscribe. Prod logged 60 subscribe
refusals in 24h: `retry` 33, `forbidden` 25, stale schema 2;
- run, `new_tab`, or timing behaviour that a recorded turn skips.

A headless run of the same flow against testcloud with the real agent is
the next step.

## Test plan

- [x] `pnpm exec playwright test agentUnsavedTabSwitchMidTurn
--project=cloud`, run locally against the e2e ComfyUI container
(`scripts/start-comfyui-e2e.sh`) and `DISTRIBUTION=cloud pnpm dev`
- [x] Control: `agentTabSwitchCatchUp.spec.ts`, 4/4 pass
- [x] Pre-commit hooks: oxfmt, type-aware oxlint, `typecheck:browser`

---------

Co-authored-by: Christian Byrne <cbyrne@comfy.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants