feat(www): automate edge bundle-size benchmark with segmented toggle - #1744
feat(www): automate edge bundle-size benchmark with segmented toggle#1744yamcodes wants to merge 1 commit into
Conversation
- Add scripts/benchmark-bundle-size.ts: esbuild (platform:neutral, target:es2022) measures raw minified uncompressed bytes — the true V8 isolate parse cost. Full edge payload mode bundles adapter+validator. Adapter-only mode externalizes peer deps (arktype/@ark/* for core, zod for t3) to isolate wrapper footprint. Hardcoded fallbacks for competitors not in workspace (varlock, @t3-oss/env-core). - Add apps/www/lib/benchmark/benchmark.json: committed baseline artifact with real measured sizes (full: standard+valibot=23.3kB, core+arktype=156.0kB, varlock=28.4kB, t3+zod=325.0kB; adapter: standard=10.0kB, core=6.3kB, t3=14.2kB). - Hook into prebuild: tsx ../../scripts/benchmark-bundle-size.ts regenerates the artifact on every production build so numbers stay fresh. - Rewrite RuntimeBloatShowcase (use client): URL-backed segmented toggle (?view=adapter via window.history.replaceState), bar widths computed from real byte counts, metric subtitle 'Minified, uncompressed JS evaluated during V8 isolate cold starts', benchmark receipts footer linking to the script. - Add Aurora CSS: heading-row flex, toggle pill + aria-pressed state, subtitle ink-2, footer hairline with receipt link styles. - Rewrite tests (6 tests, all passing): heading/subtitle, toggle buttons, default full view data, adapter view switch via userEvent.click, npmx link hrefs, receipts link URL. beforeEach resets window.location to prevent replaceState leaks across tests.
|
github-actions
Bot
added
www
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
This chart's whole selling point is that the numbers are now a verifiable, build-time fact, but two gaps undercut that: the esbuild receipt is not true for 3 of the 7 bars (they're hardcoded bundlephobia constants that can never resolve in this workspace), and nothing guards the committed artifact against the silent drift that the new prebuild regeneration introduces.
Reviewed changes
- Benchmark script (
scripts/benchmark-bundle-size.ts) — esbuild-bundles 7 cases (fullvsadapter) intobenchmark.json, with hardcoded fallbacks forvarlockand@t3-oss/env-core(not workspace packages). - Committed artifact (
apps/www/lib/benchmark/benchmark.json) — shipped so the site builds offline/CI without running the script first. - UI (
RuntimeBloatShowcase) — now a client component with a segmented full/adapter toggle, URL-backed state (?view=adapter) viareplaceState, and a benchmark-receipt footer. - Tests — rewrote the spec into 6 cases, including a click-through view switch;
beforeEachresetswindow.locationto isolatereplaceState. - Build hook (
apps/www/package.json) —prebuildnow runs the benchmark on everynext build. - CSS (
aurora.css) — toggle pill, heading row, subtitle, and footer rules on existing Aurora tokens.
⚠️ The esbuild receipt overstates provenance for 3 of 7 bars
The footer (esbuild · platform: neutral · target: es2022) and the metric subtitle present every bar as the output of this script's esbuild run. That is not true: varlock (29,082 B → "28.4 kB") and both @t3-oss/env-core rows (332,800 B → "325.0 kB", 14,541 B → "14.2 kB") can never resolve in this workspace (confirmed: neither package exists in the lockfile or workspace), so those numbers are permanently the hardcoded fallbackBytes constants on the catch path. On a chart whose credibility is verifiability, that's a misattribution readers could check. Mark each row's data source in the JSON (e.g. "source": "esbuild" | "bundlephobia") and reflect non-esbuild rows in the receipt so the claim is honest, or wire the competitors in as real measurements.
⚠️ No drift gate between the committed artifact and prebuild regeneration
prebuild regenerates and overwrites the committed benchmark.json at every production build, but the tests pin the committed values ("23.3 kB", "156.0 kB", "325.0 kB", "28.4 kB"). Any bump to @arkenv/core, @arkenv/standard, valibot, or esbuild shifts the deployed bars while the repo and the tests keep the old numbers — with no CI check comparing them, staleness is invisible until someone manually re-runs the script and commits. Since the artifact is now a footgun in two directions (build overwrites it, tests depend on it), a CI step that runs the script and fails on git diff --exit-code apps/www/lib/benchmark/benchmark.json would turn drift into a test failure instead of silent marketing rot.
ℹ️ Nitpicks
- Biome:
benchmark.jsonlacks a trailing newline; the script uses 2-space indentation (repo uses tabs), omits thenode:protocol on thefs/path/urlimports, and usesinterfacewheretypeis required — autofix will sweep most of these, butcatch (e: any)(noExplicitAny) and thediv role="group"(a11y/useSemanticElements) are not auto-fixable, sopnpm checkstays red until those two are hand-fixed. kbduplicatesbytes / 1024in the JSON; a single source of truth (compute at render, or dropbytes) would prevent a hand-edited artifact from disagreeing with its own bar widths.- The
aria-pressedsegmented control is fine as-is; a radiogroup/tablist would also be defensible, but not required.
DeepSeek Flash (free via Pullfrog for OSS) ｜ 𝕏
| external: t.external ?? [], | ||
| }); | ||
| bytes = res.outputFiles[0].contents.length; | ||
| } catch (e: any) { |
There was a problem hiding this comment.
A non-fallback failure silently drops the row, overwrites the committed benchmark.json with a truncated file, prints a misleading success line, and exits 0. For the two ArkEnv cases there is no fallbackBytes, so a broken/missing packages/*/dist at build time ships the chart missing its own bars right when the artifact claims to be generated. Fail the build for any non-fallback case (or leave the existing artifact untouched on partial failure).
| } catch (e: any) { | |
| } catch (e: unknown) { | |
| if (t.fallbackBytes) { | |
| bytes = t.fallbackBytes; | |
| console.warn( | |
| `Using fallback for ${t.name}: ${e instanceof Error ? e.message : e}`, | |
| ); | |
| } else { | |
| throw new Error( | |
| `Benchmark case ${t.name} failed (no fallback available): ${ | |
| e instanceof Error ? e.message : e | |
| }`, | |
| ); | |
| } | |
| } |
| const adapterBtn = screen.getByRole("button", { | ||
| name: /Adapter engine only/i, | ||
| }); | ||
| await userEvent.click(adapterBtn); |
There was a problem hiding this comment.
URL-backed state is the headline feature (?view=adapter deep links), but no test asserts it: after this click, window.location.search is never checked, and the mount-from-?view=adapter path — our only SSR/crawler consideration — is untested. The beforeEach reset only exists because replaceState leaks across tests, so a regression in the URL sync would go unnoticed. Cheap additions: assert window.location.search === "?view=adapter" after the click (and back to "" when switching back to Full), and one case that renders with ?view=adapter preset and expects the adapter rows.
1 participant
What
Replaces the static, hardcoded bar chart in the "Optimized for the edge" homepage section with a live, reproducible benchmark driven by a committed artifact.
Why
The previous chart showed made-up percentages (
8%,28%) and stale numbers (1.5 kB, 7.4 kB) with no way to verify them. Edge-conscious developers are rightly skeptical of static marketing numbers — this PR turns the chart into a verifiable, build-time fact.How
Benchmark script (
scripts/benchmark-bundle-size.ts)Runs at build time via
prebuildinapps/www/package.json:This measures minified, uncompressed bytes — the exact payload V8 has to read and compile during an isolate cold start. Not gzipped (that's a transfer metric), not install size (irrelevant post-bundle).
Two modes:
Competitors (
varlock,@t3-oss/env-core) are not in the workspace — the script falls back to hardcoded constants for them and logs esbuild resolution warnings (exits 0).Baseline artifact (
apps/www/lib/benchmark/benchmark.json)Committed so the site builds offline / in CI without running the script first.
ArkType's heavy weight in the full payload actually strengthens the story: ArkType is 149 kB of compile-time validator — still less than half of the Zod+t3-env ecosystem (325 kB).
UI (
RuntimeBloatShowcase)"use client"—useState+useEffecttogglewindow.history.replaceState(?view=adapter) — zero layout shift, SSR always renders thefulldefault for crawlersbytes / maxBytes × 100%)+ Validatorsuffix for the npmx link hrefesbuild · platform: neutral · target: es2022 · View benchmark script ↗CSS (
aurora.css)New Aurora token-based rules:
home-aurora__telemetry-heading-rowflex, toggle pill +aria-pressedactive state (accent bottom border), subtitle ink-2, footer hairline.Tests
6 tests — all passing (208 total across the suite):
userEvent.click(10.0 kB, 6.3 kB, 14.2 kB; no+ ArkType/+ Zod)beforeEachresetswindow.locationto preventreplaceStateleaks across tests