Repository navigation
feat(agent): In-App Agent side panel (V1) - #13892
CodeJuggernaut wants to merge 320 commits into
Conversation
The module map caches a failed module resolution, so re-importing the same specifier rejects from cache without refetching; the retry could never succeed. Stale-chunk failures are already handled app-wide by the vite:preloadError listener in App.vue.
The 2MB cap guarded a server limit that does not exist yet and silently skipped uploads for large graphs, which are normal on a video/image platform. The server owns its own limits.
The backend now seeds the thread's draft from the graph the client
sends each turn. Align to its shape and rules:
- The turn POST carries draft: { content: <serialize() save format>,
version: <last seen draft version | null> } (was the inert
workflow/graph/last_seen_version shape).
- Snapshots gate on canvas-non-empty, not workflow_id, so templates,
unsaved tabs, and new tabs seed correctly; skipping an unchanged
canvas stays as the server-blessed optimization.
- A 409 {error, version} means the agent's draft advanced: adopt the
returned version and retry the send once.
- autosaveAppliedDraft only reset the change tracker and isModified, but the tab dot renders whenever a workflow is not persisted, so agent-applied drafts always showed unsaved and lived only in memory - save through workflowService instead: temporary tabs use a silent saveWorkflowAs with a collision-safe filename (no prompt or overwrite dialog), persisted tabs re-save silently - rebind workflow_id to the tab current path in finally so service renames (collision bump, app extension mismatch) and failed saves never strand the binding on a dead path - known limit: host saveWorkflow drift handling can rarely toast or confirm on persisted tabs; reusing the canonical save path there is intentional - tests: failure isolation keeps the applied draft and surfaces no error, an app-mode collision bumps the filename and rebinds, a save that fails after rename still rebinds; binding tests now stub a valid empty threads page
Per the per-file consolidation audit (temp/plans/pr13472-file-consolidation-audit.md): safetyTypes -> ConflictDialog (SFC type export, ActiveTabStrip pattern), agentTranscript + useDraftCanvasApply -> AgentPanelRoot, ThinkingStatus -> AgentMessage, SuggestedPromptChip -> EmptyState, SelectionTagChip -> Composer, SessionBar -> AgentPanel. All moves byte-identical; net -9 files. agentTranscript's test is replaced by a stronger end-to-end copy test asserting the exact clipboard markdown. useDraftCanvasApply's tests were retired as redundant rather than merged: patch-to-canvas firing is covered end-to-end by the draft binding suite, rejection semantics are owned by agentDraftStore.test.ts, and the stop handle was unused API removed by the fold.
…JECT_TOKEN Deployed builds keep reading the token from the backend-injected __CONFIG__ (precedence unchanged); the env fallback only enables event verification from local dev servers, which otherwise have no key and silently queue forever.
- loadThread and newChat fired a server-side cancel for the in-flight
turn; the backend recorded it and the reply was lost ("Stopped
streaming..."). Panel unmount did the same. All implicit cancels are
removed; the explicit Stop button is the only cancel left
- switching away parks the live turn (message + event transport) in a
per-thread registry; events keep accumulating silently and cannot
bleed into the displayed chat; switching back re-attaches the turn
live with no lost text, or renders the finished reply from history
- a done landing during the return fetch keeps the parked entry so the
full reply still renders; overlapping or double-clicked history loads
are generation-gated; socket death and panel close settle parked
turns; remount refreshes a surviving thread from history
- draft patches from a backgrounded thread no longer drive the
displayed canvas (thread guard; version heartbeat unchanged)
- transport ignores events after settle; chat history date buckets
follow a reactive clock across midnight; onboarding coach gains
dialog semantics and Escape to dismiss; dead approval-flow fixtures
removed
Move the cascading rules (keyframes, .agent-shimmer-text, scoped reset) from globally imported agentTheme.css into agentPanel.css, loaded only by the lazy AgentPanelRoot chunk; agentTheme.css keeps the @theme blocks.
Handle agent_active_tab (activate the bound tab or mint, open, and autosave a named temporary tab; concurrent focus events serialize latest-wins), send the open_tabs and current_tab snapshot with each message, persist workflow-tab bindings, and resolve every open saved tab to its Cloud Ingest workflow id (paginated name index refreshed per send, binding fallback, never guesses on ambiguity). Reverse resolution activates existing tabs on switch_tab. Bound tabs send their bound id, unbound tabs adopt via the draft upload - no speculative ids. Draft uploads that 5xx degrade to plain chat and re-arm. Friendly labels for the tab and memory tools.
Pulls in the i18n crash fix (#13631): production bundles threw SyntaxError: Invalid linked format compiling nodeDefs.ByteDanceSeedAudio locale strings with bare @, aborting registerNodes on any backend that serves the node. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nt-side-panel-fe-implementation
…el-fe-implementation' into nathaniel/fe-1187-agent-side-panel-fe-implementation
Extract the docked panel chrome (width container, resize handle, async panel root card) from the splitter overlay and GraphCanvas into DockedAgentPanel, and host it in both GraphCanvas and LinearView. Gate the graph-mode host on !linearMode so exactly one live panel root exists across mode switches; the splitter overlay is agent-agnostic again. FE-1298
Distinguishes the feedback affordance from agent chat next to the Ask Comfy Agent entry, on both the integrated tab bar and the legacy topbar button. FE-1283
Swap the back arrow to a left chevron and rename the tooltip key to backToPreviousChat per the dev-ready design. FE-1291
Persist the open state in localStorage and stop the PostHog flag gate from force-closing on sync, so a slow or transient flag read cannot wipe the stored preference. A rehydrated panel emits an opened event with source restored only once it actually docks, so flag-off users with stale state never produce phantom opens; the now-unused flag_disabled close source is dropped. FE-1284
The litegraph canvas showed through the docked panel padding and rounded corners; fill the wrapper with the menu background and add a left border at the canvas boundary. FE-1285
Take the horizontal edge from the graph splitter panel so toasts no longer sit behind the docked agent panel, while the vertical offset keeps reading the canvas container so nothing shifts for flag-off users. Re-anchor on panel open and resize and on every toast shown, and skip hidden zero-size anchors so app mode keeps the last good position instead of pushing toasts off screen. FE-1287
Add a once-per-browser dismissible banner above the composer telling users the agent modifies the graph but only they can run it, and replace the caption under the composer with the AI-mistakes disclaimer, both with the design's exact copy. No auto-mode state is introduced; running from chat remains a separate open decision. FE-1307
The empty-state chip row spanned the full panel width and overflowed the 640px content column on a maximized panel; constrain it to the same column as the composer. FE-1288
Inserting a suggestion now focuses the textarea and flashes the accent border for a second, restarting on repeat clicks. The highlight carries its own focus-within variant because the insert focuses the field, which would otherwise let the muted focus border out-specify the accent. FE-1327
The textarea was pinned at its minimum height with the max bound dead because nothing auto-sized it; field-sizing-content lets it grow with input and scroll internally past max-h-48. Firefox falls back to the previous fixed-height behavior. FE-1329
The active-workflow label rendered inside the header row between the badge and the actions; give it its own bordered strip under the header, visible in both chat and history views. FE-1286
A dropdown chip above the composer lists every open tab and activates the picked one through the same service path the agent's switch_tab uses, giving the panel a visible workflow context at the input. FE-1290
A generic per-tab activity store, written by the agent extension and read by the topbar, drives three new tab states: a spinner while the agent edits a tab, a skeleton placeholder while it creates one, and a dot on tabs it modified in the background, cleared on activation. An always-alive tracker registered at extension setup keeps seen and prune semantics working while the panel is closed. FE-1289
Unknown tool names render sentence-cased with spaces instead of raw underscores and drop the mono face; a completed tool's glyph goes subtle instead of green; the expanded list hangs off an indented left rail under the group icon. FE-1312
Active thumb votes drop their accent and danger hues for the neutral foreground. The copy button becomes a dropdown offering rendered plain text or the raw markdown source, which replaces the raw-markdown toggle and its pre branch entirely. FE-1292
A top sentinel mirrors the existing bottom IntersectionObserver wiring and drives Tailwind mask utilities on the scroll container, so the transcript softly fades at whichever edges have more content. FE-1293
## Summary - add a dedicated Chat History control so the current title can single-click into inline rename - keep the chat-options chevron beside the rendered title while allowing the title to use available width before truncating - separate the History back icon from the non-interactive `Chat History` heading - preserve the existing menu rename path and keyboard behavior ## Validation - `AgentPanel.test.ts`: 4/4 - `ChatHistoryScreen.test.ts`: 10/10 - changed `AgentPanelRoot` behavior and stale back-control assertions verified - pre-commit oxfmt, oxlint, ESLint, Stylelint, typecheck, and knip passed - manually reviewed through the local Cloud dev proxy targeting `agent.comfy.org` ## Notes - stacked on #13892 and intentionally targets `nathaniel/agent-panel-v1` - rename persistence remains the existing frontend-local overlay pending BE-3130 Linear: [FE-1702](https://linear.app/comfyorg/issue/FE-1702/fe-align-chat-rename-interaction-with-design)
## Summary - allow the last explicitly stopped Agent prompt to be edited and resent - restore its text into the focused composer while preserving the original transcript entry - keep earlier prompts and normally completed turns non-editable ## Validation - 67 targeted unit and component tests passed - `pnpm typecheck`, `pnpm typecheck:browser`, `pnpm format:check`, `pnpm lint`, and `pnpm knip` passed - browser coverage was added and statically validated; local execution remains blocked by the browser-test backend configuration - manually reviewed through the local dev proxy targeting `agent.comfy.org` ## Notes - stacked on #13892 and intentionally targets `nathaniel/agent-panel-v1` - resubmission uses the existing append-only send flow; it does not mutate history or replay attachments and node selection Linear: [FE-1705](https://linear.app/comfyorg/issue/FE-1705/fe-allow-editing-a-previously-sent-agent-prompt) · [PM-214](https://linear.app/comfyorg/issue/PM-214/allow-editing-a-previously-sent-prompt-to-the-agent) ---------
|
The Full context for agent readersSource: todo.md row Verified state (FE main
This is a status update, not a new finding — the original review's concern is confirmed still live rather than resolved by subsequent slice landings. |
|
QA: four CRDT MVP flows pass at exact head Flag forced on locally; development evidence only. Full report, screenshots, and backend IDs. Full context for agent readersTested https://fe-pr-13892.testenvs.comfy.org; HTTP 200 and
PostHog did not enroll the QA account, so browser-local |
# Conflicts: # .agents/checks/follower-boundary.md # .coderabbit.yaml # browser_tests/fixtures/ws.ts # browser_tests/tests/agent/agentPanel.spec.ts # browser_tests/tests/agent/agentPanelAppMode.spec.ts # browser_tests/tests/agent/agentPanelMocks.ts # docs/adr/README.md # package.json # pnpm-lock.yaml # pnpm-workspace.yaml # src/components/graph/GraphCanvas.vue # src/components/graph/NodeSelectionModeBanner.test.ts # src/components/load3d/modelThumbnail.test.ts # src/components/load3d/modelThumbnail.ts # src/components/rightSidePanel/errors/useErrorGroups.test.ts # src/components/rightSidePanel/errors/useErrorGroups.ts # src/components/sidebar/tabs/queue/MediaLightbox.test.ts # src/components/sidebar/tabs/queue/MediaLightbox.vue # src/components/toast/GlobalToast.test.ts # src/components/topbar/WorkflowTab.test.ts # src/components/topbar/WorkflowTabs.test.ts # src/components/topbar/WorkflowTabs.vue # src/composables/canvas/useFocusNode.test.ts # src/composables/canvas/useFocusNode.ts # src/composables/canvas/visibleCanvasViewport.test.ts # src/composables/graph/useSelectionOperations.deleteGuard.test.ts # src/composables/graph/useSelectionOperations.ts # src/composables/useCoreCommands.ts # src/composables/useTooltipConfig.ts # src/extensions/core/agentPanel.test.ts # src/extensions/core/agentPanel.ts # src/lib/litegraph/src/DragAndScale.animateToBounds.test.ts # src/lib/litegraph/src/LGraphCanvas.selectOnly.test.ts # src/locales/en/main.json # src/platform/assets/components/MediaAssetCard.test.ts # src/platform/assets/schemas/mediaAssetSchema.ts # src/platform/errorCatalog/errorMessageResolver.test.ts # src/platform/errorCatalog/promptErrorResolver.ts # src/platform/settings/composables/useLitegraphSettings.test.ts # src/platform/settings/composables/useLitegraphSettings.ts # src/platform/telemetry/TelemetryRegistry.ts # src/platform/telemetry/providers/cloud/PostHogTelemetryProvider.ts # src/platform/telemetry/types.ts # src/platform/workflow/core/services/workflowService.test.ts # src/platform/workflow/core/services/workflowService.ts # src/renderer/extensions/vueNodes/components/NodeWidgets.vue # src/renderer/extensions/vueNodes/composables/useNodePointerInteractions.ts # src/scripts/app.test.ts # src/scripts/app.ts # src/stores/agentNodeSelectionStore.test.ts # src/stores/subgraphNavigationStore.navigateToHash.test.ts # src/stores/subgraphNavigationStore.test.ts # src/stores/subgraphNavigationStore.ts # src/stores/workflowTabActivityStore.ts # src/types/comfy.ts # src/utils/eventUtils.ts # src/views/LinearView.vue # src/workbench/extensions/agent/AgentPanelRoot.test.ts # src/workbench/extensions/agent/AgentPanelRoot.vue # src/workbench/extensions/agent/README.md # src/workbench/extensions/agent/agentPanel.css # src/workbench/extensions/agent/agentTheme.css # src/workbench/extensions/agent/components/agent/AgentPanel.test.ts # src/workbench/extensions/agent/components/agent/AgentPanel.vue # src/workbench/extensions/agent/components/agent/ChatHistoryScreen.test.ts # src/workbench/extensions/agent/components/agent/ChatHistoryScreen.vue # src/workbench/extensions/agent/components/agent/Composer.test.ts # src/workbench/extensions/agent/components/agent/ConversationView.test.ts # src/workbench/extensions/agent/components/agent/ConversationView.vue # src/workbench/extensions/agent/components/agent/DockedAgentPanel.test.ts # src/workbench/extensions/agent/components/agent/DockedAgentPanel.vue # src/workbench/extensions/agent/components/agent/EmptyState.test.ts # src/workbench/extensions/agent/components/agent/OnboardingCoach.test.ts # src/workbench/extensions/agent/components/agent/OnboardingCoach.vue # src/workbench/extensions/agent/components/agent/PanelHeader.test.ts # src/workbench/extensions/agent/components/agent/PanelHeader.vue # src/workbench/extensions/agent/components/agent/RunNoticeBanner.test.ts # src/workbench/extensions/agent/components/agent/composer/RunModePopover.vue # src/workbench/extensions/agent/components/agent/composer/WorkflowSelectorChip.test.ts # src/workbench/extensions/agent/components/agent/message/AgentMessage.test.ts # src/workbench/extensions/agent/components/agent/message/AgentMessage.vue # src/workbench/extensions/agent/components/agent/message/CodeBlock.vue # src/workbench/extensions/agent/components/agent/message/MarkdownStream.vue # src/workbench/extensions/agent/components/agent/message/MessageFeedback.test.ts # src/workbench/extensions/agent/components/agent/message/ReplyAssetGroup.test.ts # src/workbench/extensions/agent/components/agent/message/TabLinkCard.test.ts # src/workbench/extensions/agent/components/agent/message/TabLinkCard.vue # src/workbench/extensions/agent/components/agent/message/ToolCallGroup.test.ts # src/workbench/extensions/agent/components/agent/message/UserMessage.test.ts # src/workbench/extensions/agent/components/agent/message/UserMessage.vue # src/workbench/extensions/agent/composables/agent/useAgentSession.test.ts # src/workbench/extensions/agent/composables/agent/useAgentSession.ts # src/workbench/extensions/agent/composables/agent/useAttachment.test.ts # src/workbench/extensions/agent/composables/agent/useCanvasSelection.test.ts # src/workbench/extensions/agent/composables/agent/useCanvasSelection.ts # src/workbench/extensions/agent/crdt/CrdtDevPanel.vue # src/workbench/extensions/agent/crdt/RUN-CLOUD-E2E.md # src/workbench/extensions/agent/crdt/devPanelLog.ts # src/workbench/extensions/agent/crdt/docFrameClient.test.ts # src/workbench/extensions/agent/crdt/docFrameClient.ts # src/workbench/extensions/agent/crdt/followerGate.test.ts # src/workbench/extensions/agent/crdt/followerGate.ts # src/workbench/extensions/agent/crdt/followerSubscription.test.ts # src/workbench/extensions/agent/crdt/layoutFollowerBridge.ts # src/workbench/extensions/agent/crdt/schemaGuard.ts # src/workbench/extensions/agent/crdt/useAgentCrdtFollower.ts # src/workbench/extensions/agent/schemas/__fixtures__/agent/rest-responses.jsonl # src/workbench/extensions/agent/schemas/__fixtures__/agent/ws-turn-edit.jsonl # src/workbench/extensions/agent/schemas/agentApiSchema.test.ts # src/workbench/extensions/agent/schemas/agentApiSchema.ts # src/workbench/extensions/agent/services/agent/agentEventSource.test.ts # src/workbench/extensions/agent/services/agent/agentEventSource.ts # src/workbench/extensions/agent/services/agent/agentEventTransport.test.ts # src/workbench/extensions/agent/services/agent/agentEventTransport.ts # src/workbench/extensions/agent/services/agent/agentMessageParts.ts # src/workbench/extensions/agent/services/agent/agentRestClient.test.ts # src/workbench/extensions/agent/services/agent/agentRestClient.ts # src/workbench/extensions/agent/services/agent/workflowTabActivityTracker.test.ts # src/workbench/extensions/agent/services/agent/workflowTabActivityTracker.ts # src/workbench/extensions/agent/stores/agent/agentChatHistoryStore.ts # src/workbench/extensions/agent/stores/agent/agentConversationEntries.test.ts # src/workbench/extensions/agent/stores/agent/agentConversationStore.test.ts # src/workbench/extensions/agent/stores/agent/agentConversationStore.ts # src/workbench/extensions/agent/stores/agent/agentPanelStore.test.ts # src/workbench/extensions/agent/stores/agent/agentPanelStore.ts # src/workbench/extensions/agent/stores/agent/agentRunModeStore.test.ts # src/workbench/extensions/agent/stores/agent/agentRunModeStore.ts # src/workbench/extensions/agent/stores/agent/agentWorkflowTabBindingStore.test.ts # src/workbench/extensions/agent/utils/replyAssets.test.ts # src/workbench/extensions/agent/utils/replyAssets.ts
Preserve the reviewed PM-985 workflow-loading and display-title fixes. Cache source reads in the translation checker so the merge checks do not repeatedly reread the same files. Add a regression test for unchanged key reporting and one read per source file.
Preserve native workflow-tab scrolling, Agent creation indicators and forced widget disabling across the main update. Keep the real-store test harness and align its telemetry mocks with the current store and lint rule.
Keep existing widget types, labels, options and visibility when applying remote values. Restore positional Markdown values using the live widget order, without dropping editor-only controls. Restore named values before newly materialized nodes configure. Cover widget identity, Markdown text, remote mutation provenance and stale-widget removal. Validation: 446 tests passed across 17 focused suites; typecheck and cloud build passed. Independent review passed. Human QA remains pending. Linear issue: PM-985
Keep virtual-node values positional when adding them to a shared workflow. Verify note creation and text projection through the real shared applier. The host rejects name-keyed values for frontend-only nodes, causing new notes to disappear when switching back to the workflow. Validation: 131 tests passed across seven affected suites; typecheck and cloud build passed. Independent senior QA passed this two-file delta. The final test syntax correction passed scoped type-aware lint and 14 tests. Later note-text edits remain blocked by the host's opaque_widgets contract. PM-985 human QA and closeout remain pending. Linear issue: PM-985
Playwright videos for new testsDownload the recorded videos and HTML report Recorded projects: cloud Recorded test titles:
|
|
Superseded by the main-based agent slices; the V1 panel branch is retired. |
Summary
Ship the in-app Comfy Cloud Agent side panel (V1, ALPHA): a docked, workflow-aware chat that streams the agent's reasoning and tool calls and applies its edits straight to the canvas.
Changes
src/workbench/extensions/agent, registered as a core extension and docked on the right of the graph and app-mode views; it stays open across page refresh.@-mention node tagging (selection to chips to{node_ids}), auto-growing input up to a max height, a suggested-prompt flash, a run-notice banner, and the AI disclaimer caption.workflow_idto open tabs; each message carries the active tab's workflow and adopts the agent's on ack. The selector moved out of the header to above the composer and now has its full anatomy: type-to-filter search over open tabs, per-row status (spinner while the agent edits a tab, blue dot for unseen agent changes, unsaved dot, active check), an unsaved dot and clear-X on the chip, and a "Choose a workflow" detach state that sends turns with no workflow context.workflowTabActivityStore, decoupled from panel lifecycle.node.widgets,node.serialize, orgraph._version; the panel is additive and flag/ALPHA gated.shiki(via the workspace catalog) for chat code-block highlighting.Review Focus
comfy-agent-v1contract (WS draft/tool-call/thinking events, REST threads/drafts). It is not yet on production; live behavior was validated against thepr-4432preview env. Ships behind the ALPHA gate until the backend lands.activeWorkflowTurnContext,takeWorkflowSnapshot, andopenTabsSnapshot.current_taball go undefined, and the editing spinner is not re-armed on the previously bound tab; re-attach happens on row pick, new chat, or history load. SeeAgentPanelRoot.vue.main. The rendered diff is coherent (110 files, agent panel + host touches), but it needs a rebase before merge and CI runs on the stale base; watch for conflicts inlocales/en/main.json, telemetry, and the topbar.src/workbench/extensions/agent) rather than unrelated churn.Linear: FE-1187, FE-1283 through FE-1293, FE-1298, FE-1307, FE-1312, FE-1327, FE-1329, PM-121.
Screenshots
Live-verified in Chrome against the
pr-4432preview backend: docked panel and app-mode render, the draft-to-canvas apply loop (KSampler edit reflected on canvas), the workflow selector (search filter, per-row spinner/dot/check, chip clear-X to the "Choose a workflow" placeholder), and the thinking-narration lifecycle (deltas stream into the shimmer, then clear when a tool call lands). Full agent unit suite green (345 tests); typecheck, lint, format, and knip clean.