Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
0185761
test(agent): add conversation replay fixture for agent panel e2e
christian-byrne Sep 2, 2026
73220d5
fix(agent): drop unused export flagged by knip
Sep 2, 2026
82842a1
fix(agent): mint the agent tab from blankGraph, not the default template
christian-byrne Sep 2, 2026
9d898c5
test(agent): replay recorded agent conversations turn by turn
CodeJuggernaut Sep 3, 2026
6458833
test(agent): replay a cancelled turn through the panel's stop button
CodeJuggernaut Sep 3, 2026
12ebbf2
test(agent): let the replay PR stand alone
CodeJuggernaut Sep 3, 2026
7e49097
test(agent): twelve conversations recorded on the local agent
CodeJuggernaut Sep 3, 2026
6eb5c3b
test(agent): record a cancelled turn whose first edit had landed
CodeJuggernaut Sep 3, 2026
9471647
docs(browser-tests): point the capture step at the fixtures directory…
CodeJuggernaut Sep 3, 2026
c9c13b5
Merge branch 'restack/a' into restack/b
CodeJuggernaut Sep 3, 2026
7c6335d
test(agent): move the in-process doc host out of the replay harness
CodeJuggernaut Sep 3, 2026
9489aec
Merge branch 'restack/a' into restack/b
CodeJuggernaut Sep 3, 2026
e8ab9c3
test(agent): stop asserting the CRDT status chip the panel no longer …
CodeJuggernaut Sep 3, 2026
177ce09
Merge remote-tracking branch 'origin/main' into restack/a
CodeJuggernaut Sep 3, 2026
0c90e91
Merge branch 'restack/a' into restack/b
CodeJuggernaut Sep 3, 2026
fc1fb59
docs(browser-tests): playbook for running the agent replays
CodeJuggernaut Sep 3, 2026
8a63a45
Merge branch 'restack/a' into restack/b
CodeJuggernaut Sep 3, 2026
cd0876c
docs(browser-tests): replay playbook is commands only
CodeJuggernaut Sep 3, 2026
73a9e75
Merge branch 'restack/a' into restack/b
CodeJuggernaut Sep 3, 2026
56f3738
test(agent): client op batches fail loud and are typed as wire ops
CodeJuggernaut Sep 3, 2026
79f54a5
Merge branch 'restack/a' into restack/b
CodeJuggernaut Sep 3, 2026
8a75a4a
test(agent): assert deleted nodes leave the canvas after replay
CodeJuggernaut Sep 3, 2026
1cc3d4e
Merge branch 'nathaniel/be-11470-recorded-replay-fixtures' into pr3/m…
CodeJuggernaut Sep 3, 2026
203f13c
test(agent): run conversation replays in a 2560x1440 viewport and rec…
CodeJuggernaut Sep 3, 2026
5b36d0f
Merge branch 'nathaniel/be-11470-recorded-replay-fixtures' into pr3/m…
CodeJuggernaut Sep 3, 2026
8e7cdcc
test(agent): expect the registered display name for untitled replayed…
CodeJuggernaut Sep 3, 2026
4db5e84
test(agent): replay against real node definitions and compare connect…
CodeJuggernaut Sep 3, 2026
05e52f6
test(agent): compare replayed graphs without node titles and drop the…
CodeJuggernaut Sep 4, 2026
dda02dd
test(agent): share the cloud boot routes between the fixture helpers
CodeJuggernaut Sep 4, 2026
2d83d18
test(agent): fold the agent panel mocks onto the shared cloud boot ro…
CodeJuggernaut Sep 4, 2026
38068ae
test(agent): type recorded events and doc frames from the production …
CodeJuggernaut Sep 4, 2026
d05a3c3
test(agent): type the replay fixtures from the production schemas
CodeJuggernaut Sep 4, 2026
b94cb46
test(agent): assert the replay against the panel and the document
CodeJuggernaut Sep 4, 2026
ef51de4
test(agent): assert connects on the origin slot only and drop the too…
CodeJuggernaut Sep 4, 2026
f17699e
Merge branch 'main' into the replay harness
CodeJuggernaut Sep 4, 2026
f7a1363
test(agent): drop a redundant string conversion and single-argument e…
CodeJuggernaut Sep 4, 2026
41e7a20
style(agent): drop the comments that restate the blank tab change
CodeJuggernaut Sep 4, 2026
c08c4a8
fix(agent): never reuse an open workflow for the agent's new tab
CodeJuggernaut Sep 5, 2026
634c462
test(agent): give the workflow store fake the no-reuse constructor
CodeJuggernaut Sep 5, 2026
780ccb5
test(agent): assert every replayed turn as it lands and pin the no-re…
CodeJuggernaut Sep 5, 2026
47b3772
test(agent): validate every host frame with the production parser
CodeJuggernaut Sep 5, 2026
03801ab
test(agent): probe rendered text by words, not by the recording's markup
CodeJuggernaut Sep 5, 2026
42dd7f8
Merge branch 'main' into the replay harness
CodeJuggernaut Sep 8, 2026
4e3fd3d
test(agent): judge the replay by what the user sees
CodeJuggernaut Sep 8, 2026
2e6fa49
fix(agent): open the blank agent tab at the template view and never s…
CodeJuggernaut Sep 8, 2026
552f927
test(agent): expect the title the follower actually gives an untitled…
CodeJuggernaut Sep 8, 2026
e68bd8a
test(agent): accept either spelling of an untitled node's identity fo…
CodeJuggernaut Sep 8, 2026
8f8a43a
test(agent): fold a recorded clear into the expected graph
CodeJuggernaut Sep 8, 2026
dc88dfd
test(agent): judge each replayed turn in full against the production …
CodeJuggernaut Sep 9, 2026
d4a902c
test(agent): accept either spelling only for a catch-up node's title
CodeJuggernaut Sep 9, 2026
b71f737
test(agent): run the applier gate where a recording enters a replay
CodeJuggernaut Sep 9, 2026
7f2d14f
test(agent): rebase recorded asset URLs the way the panel does
CodeJuggernaut Sep 9, 2026
e56bfd5
Merge branch 'main' into the replay harness
CodeJuggernaut Sep 9, 2026
da9f456
Merge branch 'main' into the replay harness
CodeJuggernaut Sep 9, 2026
90fec63
test(agent): judge replays by explicit visible expectations and paint…
CodeJuggernaut Sep 9, 2026
7f13c41
test(agent): render the composer label without the app's i18n build
CodeJuggernaut Sep 10, 2026
c9da51a
test(agent): keep the loader's intermediate types internal
CodeJuggernaut Sep 10, 2026
7c6b5c7
test(agent): judge painted wires by the app's own render loop
CodeJuggernaut Sep 10, 2026
05913a4
fix(agent): repaint the canvas after every reconciled remote frame
CodeJuggernaut Sep 10, 2026
7361951
test(agent): keep recorded ops structural and judge the wire by a scr…
CodeJuggernaut Sep 10, 2026
01adbe8
test(agent): ship the painted-wire baseline and close the failed-open…
CodeJuggernaut Sep 10, 2026
fb30675
test(agent): boot the replay without the canvas FPS overlay
CodeJuggernaut Sep 10, 2026
8299c72
[automated] Update test expectations
invalid-email-address Sep 10, 2026
3189f62
Merge remote-tracking branch 'origin/main' into christian/16764-rebas…
christian-byrne Sep 10, 2026
c3775ac
test(agent): read the replay's tool rows from the folded work summary
CodeJuggernaut Sep 10, 2026
6a1c539
test(agent): match Worked-for group label after #17339
christian-byrne Sep 10, 2026
c755c2c
Merge #17391: main and the Worked label into the replay branch
CodeJuggernaut Sep 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions browser_tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -657,6 +657,50 @@ test.afterEach(async ({ comfyPage }) => {
`debugShowCanvasOverlay()`, `debugGetCanvasDataURL()` are for local debugging
only. Never commit them.

## Replay coverage for agent bug fixes

### Playbook

```bash
DISTRIBUTION=cloud DEV_SERVER_COMFYUI_URL=http://127.0.0.1:8188 pnpm dev
```

```bash
PLAYWRIGHT_LOCAL=1 PLAYWRIGHT_TEST_URL=http://localhost:5173 DISTRIBUTION=cloud pnpm exec playwright test agentConversation --project=cloud
```

Watch one: add `--headed -g <case id>`. Recorded gaps: `AGENT_REPLAY_TIMING=recorded`.

When a fix changes how the agent's turns affect the app (graph edits,
CRDT frames, panel state), add a conversation replay case alongside the
fix so the bug stays fixed:

1. **Capture the conversation.** Reproduce the bug's turn against the
cloud agent run locally in its non-standalone mode, with Postgres,
Redis and the doc host beside it. In that mode the agent writes the
per-op audit rows (`agent_tool_calls` parent and child rows) to
Postgres, which is what the exporter reads; the doc host is a separate
required service and writes none of them. The same agent in
standalone mode (SQLite, no doc host) never writes them, so it can
only yield text-only or tool-error turns. Record it with the recorder
(`scripts/agentConversationRecord.ts`, landing in #16782; until it merges,
record from that PR's branch), which writes the conversation JSON into
`conversations/` marked `response_side: 'recorded'`. Never write
`graph_ops` by hand and never relabel a synthesized response as recorded.
2. **Add the replay case.** Drive the fixture through the conversation
replay fixture (`agentConversationFixture`), asserting the
canvas-observable outcome the bug corrupted (graph end-state or
panel state - not mock call counts, not model text). Write what a
user sees for each turn (complete assistant text, tool rows in order)
into `agentConversationExpectations.ts`; the replay asserts those
literals and never predicts production rendering.
3. **Prove it bites.** Before merging, run the case once against the
fix's parent commit (red) and once at the fix (green); paste both
run lines in the PR description. A replay case that never went red
against the bug does not count as regression coverage.

Name the case after the behavior it protects (`agent-<behavior-slug>.json`, with the fix PR cited in the fixture's `source.note`). The exporter writes the provenance the replay keeps: `source.capture` (thread and message ids) and `source.note` (row ids and the raw capture hash), which is enough to re-export the recording.

## Test Data & Typed API Mocks

Mock data in `fixtures/data/` exports **typed** objects that conform to
Expand Down
Loading
Loading