[Next.js/dylib] Full production App Route compatibility tracker

Goal

Run a real production Next.js 16 App Route as an app-only Perry dylib, with runtime and stdlib supplied once by separate shared provider images, and pass the same black-box verifier as Node without compatibility fallbacks.

This is the correctness gate required before the Perry-vs-Node 1/10/100 deployment RSS, CPU, and startup comparison is meaningful. The current benchmark is promising but lower-bound-only because it calls userland directly and fabricates the final response.

Fixed reproduction — do not design another app

#8034 contains a complete copy/paste fixture generator pinned to Next.js 16.3.0 / React 19.2.4, plus a 21-request verifier. The app already exercises:

production next build --webpack output;
the generated routeModule.handle / AppRouteRouteModule.handle path;
a re-exported imported GET/POST handler;
method, pathname, query, headers, and POST body;
Next's AsyncLocalStorage-backed headers() before and after async continuations;
a request-time dynamic import;
a two-chunk streamed NextResponse;
status, custom response header, and Set-Cookie; and
20 distinct concurrent request contexts.

Every child issue uses that app. Smaller compiler/runtime fixtures are welcome for localization, but they cannot replace #8034 as acceptance evidence.

Work items

[Next.js/dylib] Add a pinned production App Route parity fixture and CI gate #8034 — commit the pinned production fixture and add the app-only-dylib CI gate
[Next.js/dylib] Preserve NextRequest and nextUrl.searchParams across App Route imports #8036 — preserve NextRequest and nextUrl.searchParams across the production import/re-export boundary
[Next.js/dylib] Preserve returned NextResponse headers, cookies, and stream across module boundaries #8038 — preserve returned NextResponse status, headers, cookies, and stream
[Next.js/dylib] Preserve request/work AsyncLocalStorage through concurrent App Routes #8037 — preserve Next request/work AsyncLocalStorage under concurrent App Routes (scoped cut of [parity] node:async_hooks — 113 failing node-suite tests (2026-07-22 baseline) #6764)
[Next.js/dylib] Make production webpack App Route lazy loading deadlock-free #8039 — make cold/concurrent production webpack route and dynamic-module loading deadlock-free
[GC/dylib] Install array-growth forwarding for low-address macOS arena allocations #8035 — install array-growth forwarding for valid low-address macOS arena allocations

Existing groundwork to retain

Next.js 16 standalone app-router bring-up — walls 1–5 (codegen/runtime/HIR fixes) #5438: production .next/server/** discovery and path-based lazy module initialization
fix(url): route URLSearchParams subclass instances to their native backing #6738: native backing for URLSearchParams subclasses
fix(gc): root every argument of a cross-module direct call (#7240, by @jdalton) #7252: GC rooting for cross-module call arguments
Web Streams API: implement ReadableStream + blob.stream() / response.body #237 and Web Fetch: Response.body is a numeric handle, not a ReadableStream | null #1650: Web Streams / Response.body groundwork
[parity] node:async_hooks — 113 failing node-suite tests (2026-07-22 baseline) #6764: broad node:async_hooks parity inventory

These reduce the work but do not replace the composed production test. If current main already fixes a child behavior, close that child with a #8034 regression proving it rather than with a synthetic-only test.

Suggested execution order

Land [Next.js/dylib] Add a pinned production App Route parity fixture and CI gate #8034's Node oracle and a Perry test marked expected-fail, recording the exact current Perry SHA and first failing boundary.
Fix [GC/dylib] Install array-growth forwarding for low-address macOS arena allocations #8035 independently so GC diagnostics and stale growth references cannot contaminate later debugging.
Make the request enter and survive the real route boundary ([Next.js/dylib] Preserve NextRequest and nextUrl.searchParams across App Route imports #8036).
Make cold production modules initialize exactly once ([Next.js/dylib] Make production webpack App Route lazy loading deadlock-free #8039).
Make Next's request/work context survive concurrency and continuations ([Next.js/dylib] Preserve request/work AsyncLocalStorage through concurrent App Routes #8037).
Carry and drain the real returned response ([Next.js/dylib] Preserve returned NextResponse headers, cookies, and stream across module boundaries #8038).
Remove the expected-fail marker and every compatibility branch; run the full stability matrix below.

Items 2–6 may proceed in parallel when their focused fixtures isolate them, but #8034 is the shared final oracle.

Definition of done

Runtime and stdlib are separate provider files loaded once per process.
The target app dylib excludes embedded runtime/stdlib implementations and resolves the documented provider ABI.
Providers load first; the app loads with eager relocation so first-request binding latency is not hidden.
The production webpack build enters the actual generated App Route module and private handle pipeline.
[Next.js/dylib] Add a pinned production App Route parity fixture and CI gate #8034's verifier passes on Node and Perry with identical observable status, headers/cookie, and JSON.
Ten consecutive cold Perry process starts pass; each process passes two verifier runs.
A 100-iteration run of the 20-way concurrent request batch has zero context leaks, hangs, truncations, or wrong values.
Normal GC and forced/verified GC runs pass.
Startup/request stderr contains no skipped-forwarding, unsettled-await, unimplemented, or compatibility-fallback diagnostic.
A source/test audit finds no direct userland GET invocation, ignored Response, hard-coded output, lazy-module eager-init workaround, or legacy fallback in the Perry path.

Explicitly out of scope

Turbopack; this first gate pins webpack.
The final performance verdict and thresholds.
Eager relocation optimization for 100 unique app images.
Completing every unrelated async_hooks provider failure in [parity] node:async_hooks — 113 failing node-suite tests (2026-07-22 baseline) #6764.

Once this tracker is green, run the separate performance matrix for 1, 10, and 100 copies of the same real Next deployment, followed by 100 distinct builds, against one Node process per app and a defensible shared-Node topology.

Baseline

The lower-bound observation was recorded with Perry 0.5.1503 at 564c56308d221a51b50308d9165578fbb176e877. Reproduce #8034 on current main before implementing and record that SHA in the test output; do not assume the old baseline identifies the first current failure.

Production App Route replay on exact merged main b0f13fb5 advanced to 91/91 emitted objects, then exposed dedicated final-link blocker #8064 (___perry_wrap_perry_unknown_func duplicated by four split production modules). This is after all #8057 failures cleared. #8034 remains open; no generated route execution or moving/verifier acceptance has occurred yet.

fix(hir): a nested class's own name beats an enclosing binding in `new`

fix(cjs): resolve computed relative requires against the module's own…

test(cjs): pin the computed-relative-require join

fix(codegen): record path->init addresses before the eager init loop

test(cjs): the presence probe follows the joined specifier

fix(codegen): bundle `super.m(…)` into the callee's trailing array slot

fix(hir): a nested class's own name beats an enclosing binding in `new`

Perry executed the production App Route — 21/21, twice, handler not bypassed

First successful production route execution against the #8034 fixture.

PERRY_NEXT_APP_ROUTE_READY http://127.0.0.1:31837
PASS: 21 production App Route requests
PASS: 21 production App Route requests

bypassed routeModule.handle fired 0 times across both verifier runs. That is the load-bearing part: the requests went through Next's real AppRouteRouteModule.handle pipeline, not a direct GET call. (Guard signal is in the host log, not verify.mjs's exit code — the assertion throws inside a .then() that is caught after the response is sent, so a gate reading only the exit status cannot detect a bypass.)

Compiled as an app-only dylib (104 modules) against separate runtime/stdlib provider images, PERRY_RS4GC=0.

What it took — five defects, each hiding the next

#	defect	where
1	duplicate `__perry_wrap_perry_unknown_func` across split modules	fixed by #8078
2	RS4GC statepoint-wrapped the issue-#74 inline-asm loop barrier → invalid IR → SIGBUS in ISel	fixed on `main` via #8082's sibling work
3	computed relative `require("./chunks/" + …)` never matched the absolute-keyed path registry, and path→init addresses were recorded after the eager init loop that needs them	#8146
4	the fixture's provider workspace declared no `panic` key, so it built `panic = "unwind"` and rustc planted RFC-2945 abort guards in every `extern "C"` fn — any JS throw killed the process	fixture `provider/Cargo.toml`; contract gap closed by #8147
5	a nested class's own name lost to an enclosing same-named binding in `new <Ident>`	#8153
6	`arguments` in a class method held only `max(0, argc − declaredParams)` entries, so OpenTelemetry's `if (arguments.length < 2) return;` guard fired on every valid call and `tracer.trace()` never invoked its callback	#8162

Defects 3 and 4 each produced no visible change when fixed alone — the route still died at the same line — which is worth knowing for anyone bisecting this stack.

Remaining: the forced moving-GC arm

[6/7] still fails, but only under forced/verified evacuation. Normal cold start passes both verifier runs; the forced arm dies at startup:

gc evacuation verification failed: stale forwarded pointer in runtime mutable
root scanner: slot=0x0 old=0x2c8c76505f0 forwarded_to=0x2c8c9ba3160
    at crates/perry-runtime/src/gc/verify.rs:59

[gc-schedule] FAILURE (signal 6) under seed=8036
[gc-schedule]   safepoints=68 scheduled_collections=67
[gc-schedule]   REPRODUCER: re-run with PERRY_GC_SCHEDULE_SEED=8036

PERRY_GC_VERIFY_EVACUATION catching an unrooted runtime-side cache — the class documented in CLAUDE.md as invisible to the static checker. Deterministic and seeded, so it should be tractable. Being worked separately; it is a GC rooting defect, not a Next.js one.

Not yet claimed

This is one normal cold start passing twice. The tracker's definition of done also wants ten cold processes, the 100-iteration 20-way concurrent batch, and the forced-GC arm — none of which this run establishes. PERRY_RS4GC=1 on the full app is also still unproven end to end.

fix(gc): prune FUNCTION_CLASS_IDS keys whose closure died (#8040)

fix(cjs): resolve computed relative requires against the module's own…

test(cjs): pin the computed-relative-require join

fix(gc): give the six unpruned rekeyed tables a death story (#8190-#8195

fix(gc): refuse a forwarding walk out of, or into, a non-object, and …

Definition-of-done audit on `main` `3c95020f8` (2026-08-16) — the 100-iteration batch is RED

Ran on the bench mini with the untouched tests/release/packages/next-app-route/fixture.sh (armed perry-host.js dropped in from tests/fixtures/next-app-route/ so the bypass guard is live), then 100 verifier passes against one warm process, five times.

DoD bullet	status	evidence
separate providers loaded once; app dylib excludes runtime/stdlib; providers first, app `RTLD_NOW`	✅	fixture steps 3–5, `nm -u … js_gc_init` assert, `provider-host.c`
enters real `AppRouteRouteModule.handle`	✅	`bypassed routeModule.handle`: 0 fires in 520 batches (10×2 + 5×100)
verifier identical Node ↔ Perry	✅	oracle PASS; every passing batch byte-equal
10 cold starts × 2 verifier runs	✅	`PASS next-app-route`, 10/10 processes
100-iteration 20-way batch, zero hangs/truncations/wrong values	❌	98/100, 100/100, 99/100, 97/100, 98/100 — 8 of 500 batches lost one response (empty body, `TypeError: value is not a function` in the host); each failure lands right after a default-mode copying minor. Details + reproducer: #8163 (comment of today). No hangs, no wrong IDs/headers/cookies.
normal GC ✅ / forced+verified GC ❌		forced arm = #8163, and the row above shows the same defect reaching the default configuration
no fallback diagnostics	✅	fixture grep + host logs
source/test audit (no direct `GET`, no fabrication)	✅	armed guard, 0 fires

Also observed, not in the DoD list but relevant to "retained requests": over 100 batches (2100 requests) [gc-step] post_in_use climbs 5.4 → 39.7 MB and RSS 160 → 510–620 MB with no plateau (5 runs).

Two housekeeping items surfaced by the same run:

The other copy of the fixture, tests/test_next_app_route_dylib.sh / next-app-route.yml, aborts on its very first request for a fixture-layout reason (Rust host + -flat_namespace ⇒ mixed allocators) — filed as [Next.js/dylib] tests/test_next_app_route_dylib.sh aborts on the first request: Rust provider-host + flat_namespace bind the stdlib image's __rust_dealloc to the System shim (mimalloc buffer freed by libsystem) #8205. Its first nightly run (2026-08-17 03:17 UTC) will be red for that, not for the route.
Stale notes in the 08-15 comment: PERRY_RS4GC=1 end-to-end and 10 cold starts are now both established (the release fixture uses the default native-statepoint lowering; fix(next): pass production App Route dylib gate #8082 kept statepoint roots in app dylibs).

So this tracker stays open on exactly one defect, #8163, which now blocks two DoD bullets instead of one. #8037's "100 repetitions" bullet fails on the same defect (its ALS fields were never wrong in any of the 500 batches — the failures are lost responses, not context leaks).

fix(codegen): fill a class method's `arguments` from every passed arg…

test(next): arm the routeModule.handle guard and make verifier passes…

Status: the last #8040 blocker is fixed pending review, but a NEW main regression (#8228) now blocks the fixture itself

#8163's residual holder is named and fixed — js_headers_for_each's hoisted closure pointer, a native Rust frame slot (not a side table), stale after any copying minor inside the loop. Full evidence chain and matched A/B in #8163 (comment); fix in draft PR #8220 (also closes #8217). Measured: 8 from-space faults → 0 under protection; 3 failed batches / 300 passes → 0; 1 failure / 6,334 collections → 0. The consumer was Next's pipeToNodeResponse header copy (c.headers.forEach(...)), which is why the failure was always an empty body and never a wrong id/header/cookie.

That accounts for both DoD bullets #8163 was blocking (forced-GC arm, and the 100-iteration batch).

The new blocker: #8228

Current main cannot compile this fixture at all. The in-process LLVM backend fails 5 of the 104 modules — insertelement <2 x i64> (the module-init header-image compose from bf8fd868e, #8204) has no case in perry-codegen/src/dialect/mod.rs, which bails bad binary op. app-route.runtime.prod.js is among the failures, so PERRY_ALLOW_PARTIAL_CODEGEN=1 is not a workaround. Range narrowed to one commit; filed as #8228. Every A/B above was therefore run on 3b01efba4, with the transfer argument verified rather than assumed: crates/perry-stdlib/src/fetch/headers.rs is blob-identical (a3120f1600…) at 3b01efba4, b8d32ab19 and current main 2e7a12818.

Second, separate finding worth a decision by whoever owns the provider model: an app dylib compiled before bf8fd868e does not run against providers built after it (TypeError: Cannot set properties of null or undefined (setting 'id') during Next module init; a clean-main control with no patch fails identically). The fixture rebuilds both together, so it would never notice compiler/runtime skew. If the shared-provider model is meant to tolerate skew, that needs a gate; if it is not, that should be stated.

Remaining checklist for closing this tracker

fix(gc): root the Headers/FormData iteration frame slots (#8163, #8217) #8220 reviewed and merged (the residual fix).
[codegen] In-process LLVM backend fails the 5 biggest Next App Route modules on current main (silent per-module failure; regression in 3c95020f8..07c8040bf) #8228 fixed, so the fixture builds on main again — otherwise nothing here can be re-validated at HEAD.
Re-run the acceptance instrument on merged main: the warm-process soak (test(next): add the warm-process soak arm #8163's residual needs #8215, PERRY_NEXT_ROUTE_WARM_PASSES) at N≥300, plus the 10-cold-start arm with the bypass guard armed (test(next): arm the routeModule.handle guard and make verifier passes per cold start a knob (default 10) in the release App Route fixture #8210, PERRY_NEXT_ROUTE_VERIFIERS_PER_START). These measure different regimes and neither substitutes for the other: fresh processes stay in the small-heap regime (~2 collections per 10-pass start), only a warm process reaches the accelerating-cadence one.
Decide whether the retention in [Next.js/dylib] Warm App Route server retains ~1,100 async box cells per request: 113 KB/req live Rust heap, minor-GC pause grows 59 ms -> 0.5 s #8213 (~1,100 box cells + 4 immortal PROXIES entries per request; RSS 160 → ~600 MB per 100 batches) blocks this tracker's "zero retained requests" reading, or is tracked separately. My view: separate — it is a throughput/footprint defect, not a correctness one, and it has its own issue and owner.
[Next.js/dylib] tests/test_next_app_route_dylib.sh aborts on the first request: Rust provider-host + flat_namespace bind the stdlib image's __rust_dealloc to the System shim (mimalloc buffer freed by libsystem) #8205 (the other fixture copy's gate aborts on request 1 — Rust host + -flat_namespace mixing allocators) via draft PR fix(test): C host + two-level namespace for the Next App Route dylib gate (#8205) #8209, if the tests/fixtures/next-app-route gate is meant to be part of the acceptance story.

One more finding for the follow-up pile

The bug class is systemic and unaudited: a scan for "raw heap pointer bound, then used after another allocating call" across perry-stdlib + perry-ext-* reports ~150 candidate sites, and no current instrument covers them (gc_runtime_root_holders.py audits declarations; gc_root_dominance_check.py reads IR from compiled JS; raw_handle_debt.py is scoped to crates/perry-runtime/src). A ratchet over the stdlib/ext crates is the obvious next gate.

test(next): arm the routeModule.handle guard and make verifier passes…

Matched post-#8204 acceptance run: both blocked DoD bullets PASS — via the text backend only

Run on clean origin/main 695daf302b (v0.5.1512), one fresh worktree; compiler, runtime, stdlib, provider images and app dylib all built from that single tree, so no artifact crosses bf8fd868e and the object layout is the 48 B one the merged code actually runs on. This closes the gap my 2026-08-16 audit and #8220's qualification left open — the earlier A/B was measured on a pre-#8204 layout. Bench mini (M1, 8 GB, quiet), Node 26.5.1, LLVM 22.1.8. PERRY_GC_DIAG=1 exported for all arms so copying-minor counters exist in normal-mode logs too (verified print-only first: gc_diag_enabled() gates only eprintln!s and the scanner profiler).

arm	config	result	copying minors	Σ copied_objects
cold-start, normal (5 × 10)	default GC	PASS 50/50	2/proc, 10 total	370,673
cold-start, forced (5 × 10)	`FORCE_EVACUATE=1 VERIFY_EVACUATION=1 HEAP_LIMIT=8 SEED=8036`	PASS 50/50	1,376–1,378/proc, 6,884	2,275,677
warm soak (#8215), 1 process × 300	default GC	PASS 300/300	14,057	5,825,889
dylib gate (#8209), 10 × 10	default GC, guard armed	PASS 100/100	2/proc, 20 total	744,548

Across all 500 verifier passes (10,500 requests): 0 TypeError: value is not a function, 0 E180, 0 generated handler bypassed routeModule.handle. The zero on the guard is non-vacuous — the string is present in both perry-host.js copies on main (verified: 1 occurrence each). Forced-arm liveness fired on all five processes (evacuation live — 1375 copying minor(s), 457412 objects copied), so that arm demonstrably moved objects rather than merely not throwing. Fixture verdicts: PASS next-app-route (100 batches, with forced-evacuation arm) and PASS: 100 production App Route verifier repetitions through app-only dylib providers.

Definition of done

(a) 100-iteration 20-way concurrent batch, zero hangs/truncations/wrong values — satisfied twice independently (release fixture 10×10, dylib gate 10×10), plus 300 further passes against one warm process. The stricter reading my audit applied — 100 passes against a single warm process, where 8 of 500 batches previously lost a response — is satisfied at triple that length.
(b) normal AND forced/verified GC — both pass, with the forced arm's liveness asserted rather than assumed.

So #8220's fix reproduces on the layout the merged code actually runs on.

The most informative number: minors per 100 passes vs the native path

	minors / 100 warm passes
native path, historical baseline (pre-#8204)	21–22
this run, warm soak passes 1–120	20
this run, cold-start normal procs (15 processes, both fixture copies)	20
this run, warm soak passes 131–300	8,400 (pinned at 84.0/pass = 4 per request)
this run, overall	4,686

In the small/moderate-heap regime the text-path schedule is indistinguishable from the native baseline (20 vs 21–22, reproduced by 15 fresh processes across two fixture copies) — so PERRY_LLVM_INPROCESS=off is a usable GC instrument there, not merely a way to get a build. At pass ~130 (live post_in_use ≈ 47 MiB) the process undergoes a sharp phase transition to exactly 84 minors/pass and stays pinned there:

passes   1-120: 24 minors   (0.2/pass)
passes 121-130: 25          (2.5/pass)   <- transition
passes 131-140: 559         (55.9/pass)
passes 141-300: 840 per 10  (84.0/pass, flat to the end)

The soak therefore did not under-exercise the collector — it delivered ~213× the collections a native-rate 300-pass run would have (14,057 vs ~66) and stayed clean throughout. Whether that saturation is a text-path artefact or a property of main's collector at ~50 MiB live is not settled here. Heap growth is consistent with #8213 and unabated: post_in_use 5.2 → 130.3 MB, RSS peak ~882 MB.

What this does NOT establish

The native in-process codegen path — the default, and what users actually get — is unmeasured, because main cannot compile the fixture with it (#8228). Arm 0 reproduced that failure deterministically at [5/7] on the same five modules. Everything above ran under PERRY_LLVM_INPROCESS=off, which is not a transparent swap: 16 modules (release fixture) and 62 (dylib gate) dropped to -O0, and the app dylib is 203 MB against the native path's ~87 MB. Parts of the soaked binary are unoptimised. No timing or RSS figure here may be compared against a native one.

PR #8241 (ready for review) is the unblock, and a native-path re-run of exactly these arms is the required follow-up. This evidence does not substitute for it.

Two corrections found along the way

[codegen] In-process LLVM backend fails the 5 biggest Next App Route modules on current main (silent per-module failure; regression in 3c95020f8..07c8040bf) #8228's aggravating detail Support custom menu bar items #1 no longer holds at 695daf302b: per-module Error compiling module … text is emitted. What misleads is the progress counter — a module whose unit failed still prints LLVM unit N/M finished up to M.
tests/test_next_app_route_dylib.sh is silently cwd-sensitive, and its failure names the wrong culprit — filed separately.

Remaining before this tracker can close

fix(codegen): teach the in-process IR reader the vector instructions codegen emits (#8228) #8241 merged, then re-run these four arms on the native path.
[Next.js/dylib] Warm App Route server retains ~1,100 async box cells per request: 113 KB/req live Rust heap, minor-GC pause grows 59 ms -> 0.5 s #8213 (retention: post_in_use 5.2 → 130.3 MB, RSS ~882 MB over 300 passes) — decide in or out of scope. My view: out — a footprint/throughput defect with its own issue, not a correctness one.

Evidence preserved on the bench mini: ~/perry-bench.noindex/logs-final/, ~/perry-bench.noindex/build-final/perry-{normal,forced}-*.log, perry-warm-soak.log (33 MB), logs-final/dylib/host-{1..10}.log, worktree wt-8040-final.

Native-codegen-path acceptance: both blocked DoD bullets PASS on the default backend — the caveat from my previous comment is now removed

The previous comment's stated limit was that everything ran under PERRY_LLVM_INPROCESS=off, so the native path — the default, what users actually get — was unmeasured. #8241 (f70abf752) landed and closed #8228, so this run measures it.

Tree: one clean worktree at 183d30c53a (v0.5.1512); 310c95ec4 (#8220), f70abf752 (#8241) and bf8fd868e (#8204) all verified ancestors. Compiler, runtime, stdlib, provider images and app dylib all built from that single tree — no artifact crosses bf8fd868e. Bench mini (M1, 8 GB), Node 26.5.1, clang/opt 22.1.8. PERRY_GC_DIAG=1 on every arm (verified print-only first). PERRY_LLVM_INPROCESS never set for the release fixture — the literal default.

arm	result	copying minors	Σ copied	TypeError / E180 / bypass / `[gc-forwarding]`
cold-start normal (5 × 10)	PASS 50/50	2/proc, 10	371,710	0 / 0 / 0 / 0
cold-start forced (5 × 10)	PASS 50/50	1,375–1,381/proc, 6,886	2,301,112	0 / 0 / 0 / 0
dylib gate, run 1 (10 × 10)	PASS 100/100	(capture raced)	—	bypass 0
dylib gate, run 2 (10 × 10)	PASS 100/100	2/proc, 20	746,745	0 / 0 / 0 / 0

310 verifier passes = 6,510 requests on the native path, zero failures. Fixture verdicts verbatim: PASS next-app-route (100 batches, with forced-evacuation arm) and PASS: 100 production App Route verifier repetitions through app-only dylib providers (twice, exit 0).

Forced-arm liveness fired on 5/5 processes — it moved objects rather than merely not throwing:

next-app-route-forced-1: evacuation live — 1376 copying minor(s), 456116 objects copied
next-app-route-forced-9: evacuation live — 1374 copying minor(s), 465620 objects copied

The bypass zero is non-vacuous: generated handler bypassed routeModule.handle is present exactly once in each perry-host.js actually compiled. [gc-forwarding] (#8257's stale-forwarding refusal, which prints only on a refusal) fired zero times across all 20 host logs.

The native backend is asserted live, not assumed. freeze/LLVM pipeline started — emitted only by native_emit::compile_module_units_native — appears 5× per native compile log (chunks/2.js 3 units, chunks/430.js 6, jsonwebtoken 2, app-page.runtime.prod.js 10, app-route.runtime.prod.js 4) and 0× in the text-path logs. Those are exactly the five modules #8228 could not compile.

The cross-transport question, answered

	minors / 10-pass process	minors / 100 passes	Σ copied / 100 passes
release fixture, native	2	20	371,710
release fixture, text (`695daf302b`)	2	20	370,673
dylib gate, native	2	20	746,745
dylib gate, text	2	20	744,548

Identical cadence, Σ copied within 0.3%, across 20 fresh processes on both fixture copies — and identical minor placement (first at pass 3, second at pass 5, eligible=true fallback=none). So in the small-heap regime PERRY_LLVM_INPROCESS=off is a neutral GC instrument, and the previous comment's text-path evidence transfers rather than merely being caveated.

Not settled here: whether the 84-minors/pass saturation at ~47 MiB live is a text-path artefact. A 10-pass cold process never leaves the small-heap regime (post_in_use 5.21 → 5.62 MB), so only the 300-pass warm soak can answer it — that arm is owned by the parallel session working #8163.

Optimisation drops and size (native vs text — sizes only, never timing/RSS)

	sub-`-O3`	at `-O0`	app dylib
release fixture, native	28	1	87,475,088 B (87.5 MB)
release fixture, text	31	16	203 MB
dylib gate, native	59	59	not captured

Native is not merely smaller — it drops 1 module to -O0 where the text path dropped 16, so far less of the measured binary is unoptimised.

Native-path timing/RSS, for the record and not comparable to any text-path figure: release fixture 27m43s end to end; dylib gate 17m09s / 15m14s; 10 verifier passes (210 requests) in 1.60 s wall on one cold process; sampled peak RSS ~287 MB (200 ms sampling, treat as a floor); normal post_in_use 5.21 → 5.62 MB, forced plateau 9.41 MB under HEAP_LIMIT=8.

Verdict

Both blocked bullets are satisfied on the native codegen path. (a) The 100-iteration 20-way concurrent batch passes three times independently (release fixture 10×10, dylib gate 10×10 twice). (b) Normal and forced/verified GC pass, with liveness asserted on 5/5 forced processes. The app compiles on the default backend (104/104 modules, no bad binary op), so nothing here depends on the =off workaround.

A gate defect found in the process — filed separately

tests/test_next_app_route_dylib.sh:176 pins PERRY_LLVM_INPROCESS="${PERRY_LLVM_INPROCESS:-0}". That defaults the gate to the text backend, and ${VAR:-0} cannot express "unset", so the gate as written is structurally incapable of exercising the compiler default. The nightly next-app-route workflow therefore does not test the backend users get. It was a reasonable pin while #8228 was open; now that #8241 has landed it should be removed. (My arm 3 was run with PERRY_LLVM_INPROCESS=native — a superset of the default's usage — precisely because the gate cannot express the default. Arms 1 and 2 used the release fixture, which does not pin it.)

Evidence preserved on the bench mini: ~/perry-bench.noindex/logs-native/ (all arm logs incl. dylib2/host-{1..10}.log), ~/perry-bench.noindex/build-native/, worktree wt-8040-native; the text-path baseline is intact at ~/perry-bench.noindex/logs-final/.

test(next): stop the App Route dylib gate pinning the text backend (#…

Two corrections to the numbers in my two previous comments, before they get cited in a close

1. "minors per 100 passes" is a statement about where you stopped, not about the collector. Both of my tables label a figure as minors / 100 passes. That is accurate for what was measured but invites over-generalisation, because the cadence is not linear — it is nearly flat and then near-vertical. From the in-progress native warm soak on the parallel session's host (same tree 183d30c53, native backend, no flag):

pass   3: 1 minor        pass 100: 0 minors
pass   5: 1 minor        pass 120: 1 minor
pass  20: 0 minors       pass 140: 4 minors
pass  60: 0 minors       pass 160: 84 minors

Cumulative 1,419 minors at pass 160, 2,239 at pass 170 — 820 collections in ten passes. The same binary yields ~0.2 minors/pass over the first 100 passes, ~13/pass averaged over 170, and ~82/pass locally at the end. So:

my cold-start figure should be read as "small-heap regime, first 10 passes", not as a baseline;
the historical "~21–22 per 100 warm passes" is likewise a pre-knee figure;
my earlier text-path soak's "8,400 per 100 passes" for passes 131–300 is a post-knee figure, and averaging the two ends into "4,686 overall" is the least meaningful of the three.

Independently, this confirms the ~84 minors/pass saturation exactly — measured 82/pass between passes 160 and 170 on a different host, a different backend, and a different fixture invocation than the run where I first saw it. The knee is real and reproducible; it is not a text-path artefact. That was the open question my native arms were structurally unable to answer.

2. Attribution. My #8163 comment 5309776328 and #8040 comment 5309774248 report a 300/300 text-path warm soak. That run is mine. The parallel session working #8163 attempted a text-path soak, hit ENOSPC (105/105 modules failing as clang -c failed (stderr empty)), and deliberately stood down rather than re-running it — so there is exactly one text-path soak in this record, not two, and no independent text-path replication should be inferred. Their contribution is the native warm soak now in progress, which is the arm nothing else covers.

Neither correction changes any verdict: the native-path arms in comment 5311221631 stand as measured, and the cross-transport cadence equivalence (2 minors per 10-pass process, Σcopied within 0.3%, identical minor placement) was measured on my own two runs and does not depend on anyone else's.

Closing: every work item and every definition-of-done bullet is satisfied on the shipped configuration

Perry runs a real production Next.js 16 App Route as an app-only dylib against separately loaded runtime and stdlib providers, through Next's own AppRouteRouteModule.handle, passing the same black-box verifier as Node, with no compatibility fallbacks — on the default codegen backend and default GC.

Work items

item	status
#8034 pinned production fixture + dylib gate	closed (#8082, #8161)
#8036 preserve `NextRequest` / `nextUrl.searchParams` across the import boundary	closed
#8038 preserve returned `NextResponse` status, headers, cookies, stream	closed (#8045, #8089)
#8037 request/work `AsyncLocalStorage` under concurrency	closed (#8042)
#8039 deadlock-free cold/concurrent lazy loading	closed
#8035 array-growth forwarding for low-address arena allocations	closed

Plus the defects this tracker uncovered on the way: #8064/#8078, #8121/#8128, #8131, #8146, #8153, #8162, #8163/#8217/#8220, #8168, #8174/#8196, #8205/#8209, #8218/#8219, #8228/#8241, #8257.

Definition of done

bullet	evidence
runtime + stdlib are separate provider files loaded once	fixture steps 3–4; provider ABI hash recorded per run
app dylib excludes runtime/stdlib, resolves the provider ABI	`nm -u` asserts `js_gc_init` undefined; ABI symbol diff empty
providers first, app eagerly relocated	`provider-host` loads runtime/stdlib `RTLD_GLOBAL`, app `RTLD_NOW\|RTLD_LOCAL`
enters the real generated `routeModule.handle`	bypass guard armed in both hosts, 0 fires in every run
verifier passes identically on Node and Perry	Node oracle green each run; per-request id/header/cookie/JSON/checksum asserted
10 cold starts × 2 verifier runs	exceeded — 10 × 10, twice, on the native backend
100-iteration 20-way concurrent batch, zero leaks/hangs/truncations/wrong values	satisfied three times on the native backend (release fixture 10×10, dylib gate 10×10 ×2) — 6,510 requests, zero failures
normal AND forced/verified GC	both pass; forced arm's evacuation-liveness assert fired 5/5 with 1,374–1,381 copying minors and ~456–466 k objects relocated per process
no skipped-forwarding / unsettled-await / unimplemented / fallback diagnostics	grepped per cold start; zero, plus zero `[gc-forwarding]` refusals
no direct userland `GET`, ignored `Response`, hard-coded output or legacy fallback	bypass guard is the mechanical proof, and it is non-vacuous (string present in both `perry-host.js` copies)

Beyond the DoD, because the last defect (#8163) was per-collection rather than per-request and the pass count turned out not to be the meaningful quantity: a 300-pass warm-process soak on the native backend, 13,089 copying minors, zero residual signatures (comment 5314589892), plus 10,995 collections of the same clean signature on a second host. ~24,000 collections of combined exposure; the most conservative pre-fix rate anyone measured predicts ~4 failures across that, and zero were observed.

Anti-vacuity throughout: the native backend is asserted live (freeze/LLVM pipeline started = 5, the five modules #8228 rejected, 0 on the text path), the collector is asserted live (copying-minor counts and Σcopied_objects reported per arm), and the guard's zero is asserted non-vacuous rather than assumed.

What is deliberately not closed here, and why

#8213 — warm-process retention, and it is a real defect with user-visible failures. Over 300 passes, post_in_use grows 5.2 → 123.4 MB with no plateau, peak RSS reaches 894 MB, the saturated collector reclaims a median 0.043% per cycle at a pacing step pinned to its 1 GiB ceiling, and throughput degrades ~300× (0.16 s/pass → 53 s/pass). On one host a stochastic collection storm (644 minors in a single pass against a steady 84) starved the event loop until 8 of that pass's 21 concurrent requests failed with UND_ERR_HEADERS_TIMEOUT.

I argued earlier in this thread that this was "footprint, not correctness". That was wrong on the facts and I withdrew it — a warm server that fails requests outright is not a footprint issue. It is nonetheless a distinct defect from anything this tracker set out to fix: it is root-caused (async box cells per request), it has its own issue and its own owner, and none of it is evidence against #8220 — every residual discriminator is zero across ~24,000 collections. The two-host comparison further localises it: 150 consecutive passes at a flat 84 minors/pass produced zero client-visible failures, so the storm is the hazard, not the saturation.

Closing this tracker records what it set out to establish. #8213 carries the remaining work, and it should be read as a live defect rather than a footnote.

Also open and unrelated to the DoD: #8249 (retained-on-failure LLVM scratch is unbounded), #8258 (the dylib gate is silently cwd-sensitive), #8286/#8300 (the warm-soak arm reports a client timeout as an #8163 residual).

The performance matrix this tracker was the prerequisite for — 1/10/100 copies of the same real deployment, then 100 distinct builds, against one Node process per app and a defensible shared-Node topology — is now unblocked, with the caveat that #8213 will dominate any long-running measurement until it is addressed.

fix(gc): admit array-growth forwarding stubs to the budgeted-cycle cl…

[Next.js/dylib] Full production App Route compatibility tracker #8040

Description

Goal

Fixed reproduction — do not design another app

Work items

Existing groundwork to retain

Suggested execution order

Definition of done

Explicitly out of scope

Baseline

Activity

proggeramlug commented on Aug 14, 2026

proggeramlug commented on Aug 15, 2026

Perry executed the production App Route — 21/21, twice, handler not bypassed

What it took — five defects, each hiding the next

Remaining: the forced moving-GC arm

Not yet claimed

13 remaining items

proggeramlug commented on Aug 16, 2026

Definition-of-done audit on main 3c95020f8 (2026-08-16) — the 100-iteration batch is RED

proggeramlug commented on Aug 16, 2026

Status: the last #8040 blocker is fixed pending review, but a NEW main regression (#8228) now blocks the fixture itself

The new blocker: #8228

Remaining checklist for closing this tracker

One more finding for the follow-up pile

proggeramlug commented on Aug 16, 2026

Matched post-#8204 acceptance run: both blocked DoD bullets PASS — via the text backend only

Definition of done

The most informative number: minors per 100 passes vs the native path

What this does NOT establish

Two corrections found along the way

Remaining before this tracker can close

proggeramlug commented on Aug 17, 2026

Native-codegen-path acceptance: both blocked DoD bullets PASS on the default backend — the caveat from my previous comment is now removed

The cross-transport question, answered

Optimisation drops and size (native vs text — sizes only, never timing/RSS)

Verdict

A gate defect found in the process — filed separately

proggeramlug commented on Aug 17, 2026

Two corrections to the numbers in my two previous comments, before they get cited in a close

proggeramlug commented on Aug 17, 2026

Closing: every work item and every definition-of-done bullet is satisfied on the shipped configuration

Work items

Definition of done

What is deliberately not closed here, and why

Next

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Definition-of-done audit on `main` `3c95020f8` (2026-08-16) — the 100-iteration batch is RED