Frontend: reuse scratch token storage across template substitutions

[glennmichael123]

Parent: #496. Related correctness work: #911, #912.

Evidence

The existing frozen representative_frontend_templates_tagged_substitutions_{1024,2048,4096} runner on bff189a shows linear instruction growth, but allocation replay at width 4096 reports 8,205 requests and 1,411,152 cumulative allocated bytes per complete parse. The diagnostic used 10 jobs and 3 samples, unchanged workload source, real zig-gc 0285ef8 and zig-regex c60344d; it is not a publication-quality throughput comparison.

Both template paths construct a fresh Parser token array for every substitution using the AST arena. Once a substitution is parsed, its Token structs are no longer needed: AST nodes retain source/decoded payload slices, not addresses of Token structs. Keeping every token buffer until arena destruction adds one allocation per short substitution and retains capacity proportional to the sum of expression tokens.

Root-cause change

Reuse a scratch-owned token list across sibling substitutions of each template. Keep decoded text and AST allocations in the original arena; give nested templates distinct lists; preserve full lex/parse/cover validation and source diagnostics. Release scratch on success and every error path. No grammar-specific fast path, source cache, skipped parse work, or precompiled input.

Gates

• Lifetime tests after buffer overwrite: decoded identifiers/strings, function source, nested templates, and error tokens.
• Scratch allocation reuse and spill/growth/OOM cleanup witnesses.
• Template/context parser and forced tree-walker/VM tests; exact-parent relevant test262 accounting.
• Clean exact-parent alternating evidence at the existing 1K/2K/4K widths and non-substitution controls. Publish only metrics accepted by the owned evidence tools, retaining raw artifacts and all guard failures.

[glennmichael123]

Implementation pushed in 261981c. Each template now owns one reusable scratch token list, with separate nested buffers and arena-owned AST/decoded/source payloads. Four allocation reuse/lifetime/OOM tests, both template-context tests (15 runtime scenarios in tree-walker/required VM), and all 63 parser tests pass with zero leaked. Seven relevant test262 subtrees: 1,131 -> 1,131 passes, zero failures. JSC context diagnostics retain 12/14 exact matches, with two pre-existing wording-only differences. This issue remains open: publication requires versioned direct-binary provenance support tracked in #914, followed by clean alternating exact-parent measurement. No throughput claim is being made from the preliminary diagnostic.

[glennmichael123]

Sep 14 exact-parent progress: attribution-v4 direct provenance successfully collected all three frozen substitution widths (1K/2K/4K) and the plain-template control, seven pairs each. Exact requests per parse: 2061→1038, 4109→2062, 8205→4110; cumulative allocated bytes: 354384→125232, 706640→248112, 1411152→493872. Plain-control allocation requests/bytes are unchanged. These are diagnostic-host allocation observations, not throughput wins; the candidate timing medians so far are slightly slower.

Not complete: escaped/tagged controls remain uncollected. First attempt failed the unchanged 60% measured CPU occupancy gate (31.9% at 4K); that whole row was recollected. Second attempt completed 4K/plain, then rejected an escaped-control invocation when an unrelated zig build test:android appeared. Both complete attempt logs and all completed raw artifacts are retained locally. No guard was weakened and no failed sample was promoted. Pausing measurement retries while other machine jobs run; focused correctness work continues through #917. Public report and closure wait for the remaining controls.