Repository navigation
test(agent): run the mid-turn tab switch on a tab that was never saved - #21663
skishore23 merged 1 commit into
Conversation
The conversation harness picks its seed tab in the target picker before the first turn, and picking a temporary tab saves it. The spec named for an unsaved tab therefore sent from a saved workflow with a cloud id and never took the minted-workflow path the report is about: with the unsaved tab's minted adoption disabled, both cases still passed. Add a workflowTarget option that leaves the seed tab an unsaved draft the chat follows, and use it here. The same mutation now fails both cases (the edits land in a third, agent-created tab), and dropping the tab-return catch-up still fails them.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
🎭 Playwright: ✅ 3051 passed, 0 failed · 2 flaky📊 Browser Reports
🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measureExecutes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight
Findings and check detailsThe matrix runs on same-repository version-bump PRs with relevant changes, main-branch pushes, and manual dispatches. This candidate does not qualify, so every expensive stage was skipped. Reproduce locally# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
npx vitest run --config vitest.matrix.config.mts
Full docs: scripts/registry-census/README.md 🌾 Fallow: ✅ PassedFallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened. Findings and check detailsNo new findings in the changed files. Reproduce locally# the base the gate compared against
npx fallow audit --root . --quiet --format json \
--changed-since ef08a01a5c025096140b18be3cbdaa0249896fbc
Full docs: ADR-DEVEX-LINT-0015 🏛️ Architecture: ✅ PassedFindings and check detailsWarnings (139)
Reproduce locallypnpm architecture:checkFull docs: Frontend domain catalog |
5afce6c
into
skishore23/agent-tabswitch-midturn-repro
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## skishore23/agent-tabswitch-midturn-repro #21663 +/- ##
===========================================================================
Coverage 87.50% 87.50%
===========================================================================
Files 2728 2728
Lines 150951 150951
Branches 39312 41442 +2130
===========================================================================
Hits 132089 132089
Misses 18229 18229
Partials 633 633
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
…ck (Comfy-Org#21657) ## Summary Adds browser coverage for a frequent `#agent-sorry` report: **"agent said it created a workflow but tab is empty."** Repro: start a chat on a new **unsaved** tab, switch to another tab while the agent is still editing, then switch back. The tab is blank even though the agent says the workflow is built. `agentTabSwitchCatchUp.spec.ts` only switches after the turn has finished, and only on a saved tab. This spec switches **mid-turn** on an unsaved tab, so the remaining edits can only reach the canvas through the catch-up sent when the tab resubscribes: - **switch after the first edit:** the follower unsubscribes, the host applies the remaining edits, and on return the catch-up must bring them all in. - **switch before any edit:** every edit lands while the tab is in the background. ## Results The spec runs with `workflowTarget: 'unsaved-draft'` (Comfy-Org#21663, folded in). The seed tab stays a draft that was never saved, so the first turn sends `current_tab_unbound`, the backend creates the workflow, and the ack binds it to the tab. Without that option the harness's target picker saves the tab first, and the creation path never runs. | Frontend | after first edit | before any edit | |---|---|---| | `main` (3566677) | pass ×3 | pass ×3 | | `main` + `unsaved-draft` (Comfy-Org#21663) | pass | pass | | prod pin `ba2c2c2269` (cloud/1.57) | pass ×2 | pass ×2 | Against a well-behaved host, the frontend binds the unsaved tab, unsubscribes it in the background, resubscribes on return and applies the catch-up. **This is a regression guard, not a red repro.** The prod symptom is still real, so the gap must be in something the fake host does not model. Candidates: - the real lazy mint plus `doc_reset` on an unsaved tab's first turn; - a missed or refused catch-up on resubscribe. Prod logged 60 subscribe refusals in 24h: `retry` 33, `forbidden` 25, stale schema 2; - run, `new_tab`, or timing behaviour that a recorded turn skips. A headless run of the same flow against testcloud with the real agent is the next step. ## Test plan - [x] `pnpm exec playwright test agentUnsavedTabSwitchMidTurn --project=cloud`, run locally against the e2e ComfyUI container (`scripts/start-comfyui-e2e.sh`) and `DISTRIBUTION=cloud pnpm dev` - [x] Control: `agentTabSwitchCatchUp.spec.ts`, 4/4 pass - [x] Pre-commit hooks: oxfmt, type-aware oxlint, `typecheck:browser` --------- Co-authored-by: Christian Byrne <cbyrne@comfy.org>
Summary
Stacked on #21657: makes its spec run on a tab that was actually never saved.
The conversation harness picks the seed tab in the target picker before the first turn, and picking a temporary tab saves it (
saveWorkflowAs). So #21657's "unsaved tab" wasworkflows/Unsaved Workflow.json,isTemporary: false, and the turn posted aworkflow_idinstead ofcurrent_tab_unbound. The minted-workflow binding from the Slack report was never exercised.This adds a
workflowTargetharness option (default unchanged) and sets'unsaved-draft'in the spec. The seed tab stays a draft the chat follows, the first turn mints the workflow, and the ack binds it.Verification
Local,
--project=cloud, ComfyUI e2e container +DISTRIBUTION=clouddev server:onWorkflowAdoptedrefusessent.id === undefined)ba2c2c2269agentTabSwitchCatchUp.spec.ts4/4 pass on the default target.typecheck:browser, pre-push audits pass.Still a regression guard, not a red repro: the unsaved-draft variant is also green on the prod pin, so the prod symptom is in something the fake host does not model. The mutation's failure (edits going to a new agent tab) matches the agent's own explanation in the
#agent-sorrytranscripts ("the build went to another tab"), which points at the minted binding missing in prod rather than at the follower.