Skip to content

WIP: Add E2E UI smoke test to run in BrowserStack#47

Draft
rwood-moz wants to merge 1 commit into
mainfrom
testing/browserstack-e2e
Draft

WIP: Add E2E UI smoke test to run in BrowserStack#47
rwood-moz wants to merge 1 commit into
mainfrom
testing/browserstack-e2e

Conversation

@rwood-moz

@rwood-moz rwood-moz commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

What changed?

Add a separate set of E2E tests that can run against stormbox stage or prod in BrowserStack.

Why?

The existing E2E tests in /tests/e2e (which run in CI against PRs) require a local stormbox dev stack which they make direct DB/cache/JMAP calls etc. and are not built to run in BrowserStack against a deployed stormbox instance; so adding a separate set of E2E tests specifically to be able to run in BrowserStack.

Details

Added a standalone BrowserStack E2E test package under tests/browserstack for UI-only testing against deployed Stormbox stage/prod. This includes its own package.json, package-lock.json, tsconfig.json, and pinned dependencies for Playwright 1.59.1, BrowserStack SDK 1.57.2, and dotenv 16.6.1.

Implemented desktop and mobile ui-smoke coverage that signs in through the Stormbox UI using TB Pro credentials from .env.browserstack, then verifies the basic authenticated Stormbox UI appears. Desktop uses a dedicated tests/desktop/auth.desktop.ts setup file to save auth state, while mobile signs in per test.

Added shared constants, auth helpers, and a Stormbox page object for the new suite. Updated the BrowserStack Playwright config to load .env.browserstack and match auth.desktop.ts.

Added tests/browserstack/README.md documenting that these tests target deployed stage/prod only, not the local stack, with setup and run instructions for local Playwright and BrowserStack runs. Added tests/browserstack/.env.browserstack to .gitignore so credentials stay local.

Limitations and Notes

This patch will add a UI smoke test that will run in BrowserStack on Firefox, Chromium, and Safari desktop, as well as on Chrome on a real Google Pixel 10 device. More tests will be added to this suite in future PRs.

Worked with Codex AI to get the initial framework and test in place, then further refined the code and the test myself.

Applicable Issues

Fixes #17.

QA Log

Ran this new set of E2E tests:

  • Firefox desktop on my local machine
  • Playwright's simulated google pixel 10 viewport on my local machine
  • Firefox desktop on OSX on BrowserStack [todo]
  • Chromium desktop on Win 11 on BrowserStack [todo]
  • Safari desktop on OSX on BrowserStack [todo]
  • Android chrome on a Google Pixel 10 device on BrowserStack [todo] * note: test skips as expected because BrowserStack doesn't support Android Chrome 148+ yet and that is needed for Stormbox (sharedworkers) *
  • iOS Safari on an iPhone 17 device on BrowserStack [todo]
  • iOS Safari on an iPad 13 device on BrowserStack [todo]

@rwood-moz rwood-moz linked an issue Jun 24, 2026 that may be closed by this pull request
@rwood-moz rwood-moz force-pushed the testing/browserstack-e2e branch from 35a79df to d51e5dc Compare June 24, 2026 18:40
@rwood-moz rwood-moz changed the title WIP: Add E2E tests to run in BrowserStack WIP: Add E2E UI smoke test to run in BrowserStack Jun 25, 2026
@rwood-moz rwood-moz force-pushed the testing/browserstack-e2e branch 4 times, most recently from 93bed7b to 66632f2 Compare June 26, 2026 15:21
@rwood-moz rwood-moz force-pushed the testing/browserstack-e2e branch from 66632f2 to 0df1c7c Compare June 26, 2026 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Develop E2E tests to run in BrowserStack

1 participant