Repository navigation
test(e2e): running an e2e test with E2E_VIDEO=1 saves a video a reviewer can watch - #2125
Conversation
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
0f23c88 to
73bdf06
Compare
Swarmhost agentic reviewThe detailed evaluation is available to employees in the internal Slack review thread. Evaluation budget remaining for this pull request: 98 automatic and 99 manual. Updated by Swarmhost's agentic review process. |
comfy-greenlight-bot
left a comment
There was a problem hiding this comment.
This pull request at head commit 73bdf06 has been reviewed and approved by Swarmhost's agentic review process.
The detailed evaluation is available to employees in the internal Slack review thread.
|
No activity TTL, 8 hours, merging on greenlight approval. |
What this changes
When a developer runs an e2e test with
E2E_VIDEO=1, Desktop now saves a video of the app during that test. A reviewer can watch the change happen in the PR, instead of reading the test.Why
A Desktop PR that changes what the app does should show a reviewer the change happening, as a video of the e2e test that covers it. Today the e2e harness records nothing, even though
playwright.config.tssetsvideo: 'retain-on-failure': the app is started throughelectron.launch, which gets no recording option, so that setting has no effect. To get a video, a developer edits two harness files by hand and undoes the edits before committing. Even then, clicks are too fast to see: the Home page's Auto start test ticks and unticks the box within 132 ms, between two frames.What a person sees
Before: running an e2e test saves no video, even with the setting that should keep one. Developers who wanted a video edited two test files by hand and undid the edits before committing. Clicks also happen so fast that a video misses them: the Home page's Auto start test ticks and unticks the box within 132 ms, between two frames.
After: with
E2E_VIDEO=1, the test saves.webmvideos at 1280x800 undertest-results/videos/. Eachclick,clickNth,clickByTextandfilline2e/support/cdpPages.tspauses about 1 second before and after, so each change stays on screen long enough to see. The dropdown helpers (clickUntilVisible,selectOption) and clicks made in the page's own script are not slowed down. WithoutE2E_VIDEO, tests run exactly as before: no video and no pauses. CI never sets it, so CI is unchanged.How to try it
Playwright writes one
.webmper window view. The page view is the largest file; it shows the Comfy Cloud card's Auto start box ticked for about 1.5 s, then unticked.Video
Video: the Auto start test recorded with
E2E_VIDEO=1on this branch.desk-317-auto-start.webm
Tests
e2e/support/videoRecording.test.ts(Vitest, node project) checks:E2E_VIDEOis unset, empty or0, and on for1,trueoryes;test-results/; when off there are none;The test fails on main because the module it tests does not exist there.
Recorded on this branch (head 73bdf06, macOS): the command above passed in 4.1 s and wrote a 1280x800 page video of 5.6 s. The box shows ticked from 1.2 s to 2.9 s, then unticked to the end. Without
E2E_VIDEOthe same test passed in 52 ms and wrote no.webm.Not run: the e2e recording in CI (CI never sets
E2E_VIDEO).Details for reviewers
e2e/support/videoRecording.ts(new): readsE2E_VIDEOin one place (videoRecordingEnabled), builds therecordVideooption (recordVideoOptions), and wraps an action in the pauses (pacedForVideo; when off it returns the action's own promise).e2e/support/electronHarness.ts:launchLauncherApppassesrecordVideotoelectron.launchonly when recording is on.playwright.config.tskeepsvideo: 'retain-on-failure'and is not touched.e2e/support/cdpPages.ts:clickElement(behindclick,clickNth,clickByText) andfillgo throughpacedForVideo.fill's old body moved unchanged into a privatefillNow.vitest.config.ts: the new test runs in the node project, since it readsprocess.envandnode:path(the browser project'sclickAim.test.tslist would not run it).README.md: how to record, at the end of "Run tests".Change breakdown
Total changed lines: 133 (130 added, 3 deleted).
e2e/support/videoRecording.ts,e2e/support/electronHarness.ts,e2e/support/cdpPages.tse2e/support/videoRecording.test.tsREADME.mdvitest.config.tsRef DESK-317