Skip to content

test(e2e): running an e2e test with E2E_VIDEO=1 saves a video a reviewer can watch - #2125

Merged
vqt123 merged 2 commits into
mainfrom
vinh/desk-317-e2e-video
Nov 16, 2026
Merged

vqt123 merged 2 commits into
mainfrom
vinh/desk-317-e2e-video

Conversation

@vqt123

@vqt123 vqt123 commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

When a developer runs an e2e test with E2E_VIDEO=1, Desktop now saves a video of the app during that test. A reviewer can watch the change happen in the PR, instead of reading the test.

  • e2e test: an automated test that starts the real Desktop app and clicks through it the way a person would.

Why

A Desktop PR that changes what the app does should show a reviewer the change happening, as a video of the e2e test that covers it. Today the e2e harness records nothing, even though playwright.config.ts sets video: 'retain-on-failure': the app is started through electron.launch, which gets no recording option, so that setting has no effect. To get a video, a developer edits two harness files by hand and undoes the edits before committing. Even then, clicks are too fast to see: the Home page's Auto start test ticks and unticks the box within 132 ms, between two frames.

What a person sees

Before: running an e2e test saves no video, even with the setting that should keep one. Developers who wanted a video edited two test files by hand and undid the edits before committing. Clicks also happen so fast that a video misses them: the Home page's Auto start test ticks and unticks the box within 132 ms, between two frames.

After: with E2E_VIDEO=1, the test saves .webm videos at 1280x800 under test-results/videos/. Each click, clickNth, clickByText and fill in e2e/support/cdpPages.ts pauses about 1 second before and after, so each change stays on screen long enough to see. The dropdown helpers (clickUntilVisible, selectOption) and clicks made in the page's own script are not slowed down. Without E2E_VIDEO, tests run exactly as before: no video and no pauses. CI never sets it, so CI is unchanged.

How to try it

E2E_VIDEO=1 pnpm exec playwright test e2e/home.test.ts --project=macos -g "Auto start"

Playwright writes one .webm per window view. The page view is the largest file; it shows the Comfy Cloud card's Auto start box ticked for about 1.5 s, then unticked.

Video

Video: the Auto start test recorded with E2E_VIDEO=1 on this branch.

desk-317-auto-start.webm

Tests

e2e/support/videoRecording.test.ts (Vitest, node project) checks:

  • recording is off when E2E_VIDEO is unset, empty or 0, and on for 1, true or yes;
  • when on, the video options are 1280x800 and point into test-results/; when off there are none;
  • when off, a click or fill runs at once with no timer; when on, it waits 1000 ms before and after (checked with fake timers, so the test does not wait in real time).

The test fails on main because the module it tests does not exist there.

Recorded on this branch (head 73bdf06, macOS): the command above passed in 4.1 s and wrote a 1280x800 page video of 5.6 s. The box shows ticked from 1.2 s to 2.9 s, then unticked to the end. Without E2E_VIDEO the same test passed in 52 ms and wrote no .webm.

Not run: the e2e recording in CI (CI never sets E2E_VIDEO).

Details for reviewers

  • e2e/support/videoRecording.ts (new): reads E2E_VIDEO in one place (videoRecordingEnabled), builds the recordVideo option (recordVideoOptions), and wraps an action in the pauses (pacedForVideo; when off it returns the action's own promise).
  • e2e/support/electronHarness.ts: launchLauncherApp passes recordVideo to electron.launch only when recording is on. playwright.config.ts keeps video: 'retain-on-failure' and is not touched.
  • e2e/support/cdpPages.ts: clickElement (behind click, clickNth, clickByText) and fill go through pacedForVideo. fill's old body moved unchanged into a private fillNow.
  • vitest.config.ts: the new test runs in the node project, since it reads process.env and node:path (the browser project's clickAim.test.ts list would not run it).
  • README.md: how to record, at the end of "Run tests".

Change breakdown

Total changed lines: 133 (130 added, 3 deleted).

Category Files Added Deleted Share
Product code (app) 0 0 0 0%
e2e harness code 3: e2e/support/videoRecording.ts, e2e/support/electronHarness.ts, e2e/support/cdpPages.ts 53 3 42.1%
Tests 1: e2e/support/videoRecording.test.ts 68 0 51.1%
Documentation 1: README.md 8 0 6.0%
Configuration 1: vitest.config.ts 1 0 0.8%

Ref DESK-317

@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The E2E harness now supports optional video recording. CDP click and fill helpers add pacing when recording is enabled. Tests cover configuration and pacing behavior, and the README documents how to record videos.

Changes: E2E video recording

Layer / File(s) Summary
Recording options and pacing
e2e/support/videoRecording.ts, e2e/support/videoRecording.test.ts, vitest.config.ts
The new helpers enable recording based on E2E_VIDEO, provide video options, and pace async actions. Tests cover environment values, options, and pacing.
Electron and CDP integration
e2e/support/electronHarness.ts, e2e/support/cdpPages.ts, README.md
The Electron harness passes recording options when enabled. CDP click and fill helpers use the pacing helper. The README documents setup and which interactions are paced.

Sequence Diagram(s)

sequenceDiagram
  participant E2ETest
  participant ElectronHarness
  participant VideoRecording
  participant Electron
  participant CDPPages
  E2ETest->>ElectronHarness: launch Electron
  ElectronHarness->>VideoRecording: request recording options
  VideoRecording-->>ElectronHarness: return options when enabled
  ElectronHarness->>Electron: launch with optional recordVideo
  E2ETest->>CDPPages: invoke click or fill
  CDPPages->>VideoRecording: pace action
  VideoRecording-->>CDPPages: resolve after action and pacing
Loading

Suggested reviewers: kosinkadink, benceruleanlu

Merge Risk: Minimal · up to 73bdf

This adds optional video recording and pacing for end-to-end tests, enabled only when E2E_VIDEO is set. Default test behavior is unchanged, and no merge-blocking risk was found.

Pre-merge checks: Passed 2

Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
Simplify code

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 2803ad6a-077b-4658-95bb-cb5b26eea2a5
Commits

Reviewing files that changed from the base of the PR and between 688be47 and 73bdf06.

Files selected for processing: 6

  • README.md
  • e2e/support/cdpPages.ts
  • e2e/support/electronHarness.ts
  • e2e/support/videoRecording.test.ts
  • e2e/support/videoRecording.ts
  • vitest.config.ts

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

@vqt123
vqt123 force-pushed the vinh/desk-317-e2e-video branch from 0f23c88 to 73bdf06 Compare November 15, 2026 14:23
@vqt123
vqt123 marked this pull request as ready for review November 15, 2026 14:36
@vqt123
vqt123 requested review from a team as code owners November 15, 2026 14:37
@vqt123
vqt123 requested a review from benceruleanlu November 15, 2026 14:37
@comfy-greenlight-bot

comfy-greenlight-bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Swarmhost agentic review

The detailed evaluation is available to employees in the internal Slack review thread.

Evaluation budget remaining for this pull request: 98 automatic and 99 manual.

Updated by Swarmhost's agentic review process.

@comfy-greenlight-bot comfy-greenlight-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pull request at head commit 73bdf06 has been reviewed and approved by Swarmhost's agentic review process.

The detailed evaluation is available to employees in the internal Slack review thread.

@vqt123

vqt123 commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

No activity TTL, 8 hours, merging on greenlight approval.

@vqt123
vqt123 added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit 2cc3b16 Nov 16, 2026
29 checks passed
@vqt123
vqt123 deleted the vinh/desk-317-e2e-video branch November 16, 2026 01:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants