Repository navigation
Conversation
PR template validation failedPlease fix the following issues by editing your PR description:
See CONTRIBUTING.md for the full contribution policy. |
This was referenced Oct 27, 2026
The dev-bypass, dev-reset and captured-email endpoints, the console email provider and localhost registry URLs are gated on import.meta.env.DEV, so an end-to-end suite that signs in through them can only run against `astro dev`. The new integration option sets a build-time constant that the shared isDevOnlyEnabled() gate reads alongside import.meta.env.DEV. A build without the option gets `false`, so nothing at runtime can switch the endpoints on, and the build-time registry check accepts localhost only when the option is set. `astro build` warns while it is on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EMDASH_E2E_BUILD=production now runs `astro build` on the Node fixture and serves dist/server/entry.mjs instead of running `astro dev`. The fixture sets dangerouslyEnableTestHooks so the suite can still sign in and reset the site, and EMDASH_SITE_URL because production setup and image.remotePatterns need it. Also fixes what a built fixture exposes: manual i18n routing so the fixture's [locale] pages win over Astro's fallback routes, sharp for the image endpoint, the registry fixture installed at server start, an https image URL for the CSP, and an ignore rule for the migrations journal that builds write into the fixture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ostenbom
force-pushed
the
e2e/production-build-flag
branch
from
October 27, 2026 20:49
dc5dae7 to
4660103
Compare
9 of 22 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Runs the Node Playwright suite against a real production build (
astro build) instead ofastro dev. It adds a build-timedangerouslyEnableTestHooksoption to theemdash()integration, so the suite can still sign in and reset the site. It is one of four PRs comparing ways to speed up the E2E sweep (see Related PRs). This one tests production behaviour and gets the largest speedup.EMDASH_E2E_BUILD=productionswitchese2e/global-setup.tsfromastro devtoastro buildplusnode dist/server/entry.mjs. Runs without the variable still useastro dev. CI sets it for the Node shards only; the Cloudflare step and the 8-shard layout are unchanged.Why each part is needed
dangerouslyEnableTestHooksintegration option, read through a newisDevOnlyEnabled()gate/_emdash/api/setup/dev-bypassand/_emdash/api/auth/dev-bypass, resets the site throughdev-reset, reads invites through/_emdash/api/dev/emails, and installs from a localhost mock marketplace. All of these were gated onimport.meta.env.DEV, which isfalsein a production build. The gate now also reads a build-time constant (__EMDASH_TEST_HOOKS__, a Vitedefine) that the option sets.import.meta.env.DEVat those call sites (dev-bypass x2, dev-reset, dev emails, console email provider, registry localhost checks inregistry/config.ts,registry/artifact-fetch.ts,api/handlers/registry.tsand the registry artifact route)import.meta.env.DEVchecks are untouched.astro buildwarns while it is onregistryin Node at build time, before Vite runs, so it reads the option directly.e2e/fixture/astro.config.mjssetsdangerouslyEnableTestHooks: trueglobal-setup.ts: build, servedist/server/entry.mjs, skip the warm-upsastro devcompiles on first request and reloads after optimizing dependencies. The SQLite path is baked into the build, so the data folder is fixed ($TMPDIR/emdash-pw-production) and cleared each run.EMDASH_SITE_URLset for the build and serverimage.remotePatternsis derived from it at build time.e2e/fixture:i18n.routing: "manual"plus a pass-throughsrc/middleware.ts[locale]pages, so/fr/posts/<slug>showed the English post.e2e/fixture:sharpdependencyMissingSharp;astro devfound Astro's optional copy.outer-middleware.tsinstalls the registry fixture at startupastro.config.mjsinstalls it, and a built server never runs the config.editor-image-panel.spec.tsuses anhttps://image URLhttps:images. Playwright intercepts the URL either way..gitignore:e2e/fixture*/.emdash/migrations.jsonThree tests that fail only when the whole suite runs against one server are fixed in #10. The CI shards don't need that fix; a full local run does.
Safety
The option is fixed at build time. In a build without it, the fixture's compiled
isDevOnlyEnabled()is literallyreturn false;. Its handlers are still in the bundle but can't be reached, and no environment variable or request can turn them on. That is stricter than the earlier prototype on #4, which used a runtimeEMDASH_INSECURE_TEST_BYPASSenvironment variable that any production deployment could have set. The dev routes are still injected in every build, as onmain, and return 403. Skipping their injection unless the option is set would be a separate change.How much it helps
Bottom line: after syncing with upstream, a real production build cuts the Node E2E time by about 60% with the 8-shard layout unchanged (#7: slowest shard 7.1 → 2.8 min). A dev-output build saves about a quarter (#6: 7.1 → 5.5 min) and fails on Endform at concurrency 64 on every attempt (#8). The production build on Endform (#9) runs the whole suite in one job in under 6 minutes, install and build included.
Local, native Playwright, 1 worker, full Node suite (378 tests), same Mac, run back to back:
astro devastro devmainastro build --devOutputastro build+dangerouslyEnableTestHooksastro build+ runtimeEMDASH_INSECURE_TEST_BYPASS(prototype)* The same flaky test both times:
media-library.spec.ts"moves one local media item into a visible folder by dragging", which fails about 1 run in 3 when the root folder is crowded. It isn't specific to production builds.CI, GitHub Actions
ubuntu-latest, after syncing with upstreammain(c1efce5cc):astro devmainastro build --devOutputastro build+dangerouslyEnableTestHooksastro build --devOutputon Endformastro build+dangerouslyEnableTestHookson Endformastro build+ runtime switch on EndformFor the shard rows, wall time is the slowest shard's "Run Node E2E" step and runner time is that step summed over all 8 shards. Install, build and the Cloudflare step are left out because they don't change between methods. emdash-cms#4317 now builds the fixtures before that step, which is why every shard number is lower than before the sync. Before the sync, against the older baseline (8.0 min, 42.6 shard-min), the shard results were the same picture: #6 −26% and #7 −51%, and on Endform #9 took 4m55s with no retries. For the Endform rows, wall time is the whole job, install and build included.
***
visual-editing.spec.ts"flushes unsaved edits when navigating away", the known Endform keepalive issue, andmedia-library.spec.ts"matches the compact responsive folder layout".** The same dev-output build passes natively (344 of 344), and its client bundles are within 3% of the production build's (4.9 MB vs 4.8 MB of gzipped JS). The extra time is most likely server-side rendering in development mode, which the one shared server can't keep up with at 64 tests at once. That is the same gap as in the native timings (10.3 vs 6.0 min). Not profiled further.
Runs after the sync: #6 CI, #7 CI, #8 Endform E2E, #9 Endform E2E. Before the sync: #6, #7, #8, #9. One or two runs each, so treat small differences as noise.
Other checks: Smoke Isolated Install passes after the sync, since upstream now installs the newest Node 22 for it. Browser Tests (the admin Vitest browser suite, which none of these PRs touch) still fails intermittently on these runners. It fails with "Failed to fetch dynamically imported module" / "Cannot connect to the iframe" on a different file each attempt, and once locally with a timing assertion (
PortableTextEditor.test.tsx, toolbar fade) that passes 3 of 3 times on its own. After the sync it passed first time on #6 and #10, and on the third attempt on #7.Related PRs
astro build --devOutputastro builddangerouslyEnableTestHooksoptionastro build --devOutputastro buildastro buildwith a runtime env switchType of change
Your connection to this change
Checklist
pnpm typecheckpassespnpm lintpassespnpm testpasses (or targeted tests for my change): core unit tests (6100 passed), full Node E2E suite against the production build (see table); core integration tests not run locallypnpm formathas been runNot done: the option isn't in the configuration reference docs yet.
AI assistance disclosure
Screenshots / test output
Not applicable. Timings are in the table above.
🤖 Generated with Claude Code