[midagedev]

A Metal process's first prompt call costs what a later one does. Part of #593.

The first prompt call of a process was 85–101 ms slower than the same call warm (M4 Pro, 873 ids). generate_metal measures one cold call per process, so this was in every prompt rate, and in the ~100–118 ms fixed part of a call on the contributor's M3 and M5 Ultra.

Decomposition, predicted before the run, then measured (M4 Pro, 873 ids, timers only, two runs):

term	predicted	measured
(a) rows arena alloc, 15 buffers	0.1–0.3 ms	0.28 / 0.44 ms
(b) host zero of 465 MB	20–60 ms	29.1 / 44.4 ms
(c) residency commit	4–10 ms	16.0 / 15.2 ms
(d) prefill pipelines	0.2–3 ms	0.34 / 1.15 ms
(f1) checkpoint slot 0, first touch	+3–12 ms	+8.5 / +8.9 ms
(f2) the pass itself, cold − warm	+10–60 ms	+31.4 / +29.9 ms
sum		85.6 / 100.0 ms (measured cold − warm 85.4 / 99.5)

(c) was the wrong term: about twice the M1 Pro figure the prediction used.

• (a), (c), (d) move to Session::open_as: the rows arena is made there and wired by the open's one residency commit, and the prefill pipelines are built first, so a refused build fails the open before the load. Every session now holds the arena (464,863,776 bytes on this model, cap 2048 rows; a smaller ctx caps it).
• (b) is gone. Buffers come from newBufferWithLength, which Metal clears to zero, and no kernel of a pass reads a cell an earlier op of that pass did not write. The arena was zeroed only at its open before, so no pass sees other bits.
• MarkCopies { AtOpen, OnFirstMark }: the server opens the mark copies (metal: message marks kept inside one prompt call (prefill_marked) #792, 147,456,000 bytes) with its session; other sessions keep them lazy.
• generate_metal prints open_load_s, open_rows_ms, open_residency_ms, open_pipelines_ms and always rows_arena_bytes.
• Left open: (f1) and (f2), about 36 ms of cold − warm after this change.

Gates: r_resident now holds that the first pass allocates nothing on the device and adds nothing to the residency set (the open's set check covers the arena). FAIL-first, the arena opened lazily again: "FAIL (the first pass moved the device's bytes by 14532608 and the residency set from 125 buffers of 2769174528 bytes to 140 of 2783707136…)". q_short, q_state and q_mark_cut used "the rows arena is open" as proof a pass ran; they now read StepStats::passes, the same fact counted directly.

Gates (M1 Pro, fixture), each rc 0: cargo fmt --all --check, check-unsafe, check-comments, just check-recipes, just mac-metal-lint, just lab-metal (114 mutants, each red on its clause), just lab-metal-e2e (11 mutants, each red on its clause).

M4 Pro 48 GB, Qwen3.6-35B-A3B MLX 4-bit, development-loop numbers (not public). generate_metal --prompt "$(cat p.txt)" -n 8 --ctx 8192, base and after interleaved, arm order rotated, two runs each:

	base	after
873 ids, prompt tok/s	928.9 / 977.9	1019.4 / 1036.7
873 ids, cold − warm (ms)	127.0 / 79.6	43.2 / 28.9
7985 ids, prompt tok/s	1000.5 / 999.2	1007.9 / 1008.5
open (s)	45.1–46.1	45.0–46.4
decode control, tok/s	101.2 / 99.0	102.0 / 100.7

The open's added work is inside its noise (±0.5 s). After: rows 0.08–0.21 ms, residency 2500–2713 ms (the 20 GB of weights with it), pipelines 3.0–3.6 ms.

Evidence: base = #792's branch head before its review fix (which touches only the ring copy near the last u32 position); after = this change before a use cleanup and cargo fmt. macOS 27.0.1, Apple M4 Pro 48 GB; model file Qwen3.6-35B-A3B-mlx4-00001-of-00001.gguf; release builds, one process per run. Load (vm.loadavg, 1 min) 2.8–5.1.

The code and gates were written with Claude Code rounds and reviewed by the maintainer.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Q5zTV27LbaVZFTjnkdmTjJ

[github-actions]

Verdict: CLEAN: No must-fix defect remains after tracing allocation failures, pass bounds and checkpoint validity.

Non-blocking

⚠ Cover eager mark copies: crates/metal/src/arch/qwen3moe/session.rs:739
All end-to-end gate constructors use open or open_with; r_resident:61 uses the latter and runs an unmarked pass at line 80. Deleting the eager branch leaves those gates green.
Extend r_resident to exercise MarkCopies::AtOpen with a marked pass and assert unchanged device bytes and residency entries. This protects the server's allocation guarantee while its owning gate is being changed.

Checked

Duplicates: 4 new items and the renamed fresh checked; no duplicated owner found. Dryer reported 0 pairs.
Complexity: open_as (18) moves the existing open orchestration and adds timing and eager-copy selection; existing owners handle allocation and residency. r_resident (18→19) adds an equality check, harmless even growth.
Silent failure: 3 optional-copy branches followed; unavailable copies are refused by checkpoint_copy. Allocation and pipeline errors propagate.
Tests (static mutation review): r_resident protects crates/metal/src/arch/qwen3moe/session.rs:738; restoring lazy arena allocation makes its growth assertion red, as the body reports. q_short, q_state and q_mark_cut protect crates/metal/src/arch/qwen3moe/target.rs:207; forcing the opposite prompt path triggers a refusal or the pass-count assertions. q_layers changes only a comment.
Gates: no numeric pins moved; the changed first-pass allocation invariant has FAIL-first evidence.
Performance: prefill path; 15 buffer allocations and the arena's residency registration move to open. Host arena zeroing is removed; kernel launches and reduction order are unchanged.
Escape catalog: C1–C4 do not match.

Notes

Metal gates require Apple Silicon/macOS and were not run in this Linux environment. Attach exact measured base/after commit hashes and gate command output; the body currently supplies branch descriptions and summarized return codes.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}