Skip to content

test(e2e): run the fake-ComfyUI e2e specs on macOS too - #2134

Merged
vqt123 merged 3 commits into
mainfrom
vinh/e2e-fake-comfy-macos
Nov 16, 2026
Merged

vqt123 merged 3 commits into
mainfrom
vinh/e2e-fake-comfy-macos

Conversation

@vqt123

@vqt123 vqt123 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

TL;DR: the end-to-end tests that start a fake ComfyUI now run on macOS as well as Linux.

Why These tests refused to run anywhere but Linux, so the Mac CI job skipped them and nobody could run them on a Mac. The fake ComfyUI is a shell script that stands in for venv/bin/python3. macOS runs a shell script there the same way Linux does. Only Windows needs python.exe, so only Windows is refused now.

Before / After

  • Before: writeFakeComfyInstall threw on macOS, and 12 spec files were tagged @linux only. After: those 12 files are tagged @linux @macos, and the Mac CI job runs them.

  • Before: the apps-local "Open" step passed once the ComfyUI window existed, and ended before the window drew anything. After: it waits until the window shows the ComfyUI page, and holds it for a second when E2E_VIDEO=1. The fake page's text is readable, so the video shows the window.

  • Before: assets-startup-failure failed once in the macOS merge-queue run. Its second test clicked the first test's leftover error badge while the new launch had swapped it for the progress spinner. After: the test marks the old badge before each launch and waits for a new one.

e2e/process-ownership.test.ts stays Linux-only. On a Mac it fails at launchInstall with "the install never booted a new ComfyUI stub". The cause is not found yet.

Seen in production by: not a product change.

Run: E2E_VIDEO=1 pnpm exec playwright test e2e/apps-local.test.ts --project=macos on an Apple silicon Mac, 5 of 5 passed. The other 11 newly tagged files were run on the same Mac before the apps-local change and passed. e2e/assets-startup-failure.test.ts --project=macos --repeat-each=8 --retries=0: 16 of 16 passed after the fix. Not run: Windows, unchanged.

Rule bent: none.

Ref DESK-305

The fake install refused every platform but Linux. Its python3 stub is a shell script, which macOS runs as well, so only Windows is refused now. Twelve spec files that use it are tagged @MacOS. process-ownership stays Linux-only.

The apps-local Open step waits for the ComfyUI window to show the stub page, and the stub page has readable text, so its video shows the window.
@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Selected end-to-end tests now run on Linux and macOS. The fake ComfyUI installer permits macOS, asset-startup tests distinguish earlier error badges, and the apps-local window check waits for visible page content.

Changes: End-to-end coverage and page readiness

Layer / File(s) Summary
Fake install platform support
e2e/support/fakeComfyInstall.ts
The fake ComfyUI installer now rejects only Windows. macOS proceeds through the existing shell-stub setup.
macOS test coverage
e2e/assets-startup-failure.test.ts, e2e/beta-activation-notice*.test.ts, e2e/core-beta-args-pill-*.test.ts, e2e/performance-test-guardrail.test.ts, e2e/launch-own-window.test.ts, e2e/whats-new-over-canvas.test.ts
Selected tests now include the macOS tag alongside Linux. Asset-startup tests mark existing error badges and select only a badge without that marker.
Apps-local page readiness
e2e/apps-local.test.ts, src/main/apps/__fixtures__/fakeComfyApi.ts
The apps-local tests now include the macOS tag. The ComfyUI window check waits for its body to become visible. The fake API serves a styled page with an updated label.

Possibly related PRs

Merge Risk: Minimal · up to fa02b

This only extends end-to-end test coverage to macOS and makes the page-readiness checks more reliable. It does not change the product. The risk of merging is minimal.

Pre-merge checks: Passed 2

Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
Simplify code

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 60f0a732-4a3b-45f7-be83-32380928bf89
Commits

Reviewing files that changed from the base of the PR and between e043363 and fa02b95.

Files selected for processing: 2

  • e2e/core-beta-args-pill-commit.test.ts
  • e2e/launch-own-window.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

@comfy-greenlight-bot

comfy-greenlight-bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Swarmhost agentic review

The detailed evaluation is available to employees in the internal Slack review thread.

Evaluation budget remaining for this pull request: 96 automatic and 99 manual.

Updated by Swarmhost's agentic review process.

@comfy-greenlight-bot comfy-greenlight-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pull request at head commit 4936f67 has been reviewed and approved by Swarmhost's agentic review process.

The detailed evaluation is available to employees in the internal Slack review thread.

@vqt123

vqt123 commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

No activity TTL, 8 hours, merging on greenlight approval.

@vqt123
vqt123 added this pull request to the merge queue Nov 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 16, 2026
The second test found the first test's error badge still on the tile, and the launch check passed before Desktop had registered the new launch. The click then landed while the new launch had swapped the badge for the progress spinner, and found no badge. The old badge is marked before each launch, and the test waits for a new one.
@swarmhost-app
swarmhost-app Bot dismissed comfy-greenlight-bot’s stale review November 16, 2026 12:39

Swarmhost approval covered 4936f67; the pull request advanced to e043363.

@vqt123
vqt123 force-pushed the vinh/e2e-fake-comfy-macos branch from e4ce125 to e043363 Compare November 16, 2026 16:07

@comfy-greenlight-bot comfy-greenlight-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pull request at head commit fa02b95 has been reviewed and approved by Swarmhost's agentic review process.

The detailed evaluation is available to employees in the internal Slack review thread.

@vqt123

vqt123 commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

No activity TTL, 8 hours, merging on greenlight approval.

@vqt123
vqt123 added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit 7527125 Nov 16, 2026
35 of 38 checks passed
@vqt123
vqt123 deleted the vinh/e2e-fake-comfy-macos branch November 16, 2026 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants