[CocoaBeans]

Summary

An exact-prefix SSD session restore (session_restore_mode: "ssd_clone") ran its suffix prefill and its whole decode on the paged decode layout. Prefill was about 6× slower and decode about 1.55× slower than a RAM restore at the same context.

• Cause: SessionBank.restore() reaches the SSD tier through cold_fallback() → _restore_cold(). _restore_cold built its cache with runtime.make_cache() and dropped the cache_factory the caller passed. generation passes the contiguous target-prefill layout (_session_restore_cache_factory → _make_target_prefill_cache) and repages after the suffix prefill. make_cache() installs the tail-owned decode layout instead, which is vLLM-Metal paged attention under turbo, and compiled verify then falls back on every call with context_above_threshold.
• Fix: forward cache_factory and mtp_cache_factory into _restore_cold and build through them, mirroring the RAM clone lane, including restore_meta_state=cache_factory is None. With no factory, behavior is unchanged.
• Scope: only exact-prefix SSD hits were affected. The RAM lanes and the near-prefix lane (restore_entry_prefix_cache, reported as ssd_block_prefix_boundary_clone) already build through the factory. generation passes only cache_factory today. Forwarding mtp_cache_factory keeps the cold lane consistent with the RAM lane.
• Repro: benchmarks/repro_ssd_clone_restore_layout.py triggers the bug on demand and compares the installed MTPLX against this checkout. benchmarks/results/ssd-clone-restore-layout-m5pro-64gb.json holds the run below.
• Related: Write an idle conversation to SSD before the admission releases it #618 writes idle conversations to SSD so their next turn restores from disk instead of re-reading. That restore is an exact-prefix SSD hit, so without this fix it runs paged at the speeds below. The two PRs touch separate hunks of session_bank.py.

Verification

• New regression test: tests/test_cache_bank.py::test_session_bank_ssd_exact_restore_builds_caches_with_caller_factories fails on v2.12.2 with assert [] is ['prefill-layout-trunk'] and passes here.
• Session-bank and cold-tier suites: all 43 test files touching the session bank, cold tier, cache_factory or ssd_clone: 1,198 passed, 2 skipped, 1 xfailed, 0 failed.
• CI test set: test_no_mlx_imports, test_public_cli, test_runtime_kpis, test_server_openai, test_server_responses, test_endpoint_probe_hygiene, test_openai_bridge, test_thermal, test_cache_state and test_aria2_downloader, plus test_cache_bank: 1,085 passed, 4 skipped, 0 failed.
• CI build and hygiene steps: python -m compileall -q mtplx tests, python -m build, twine check dist/* (both artifacts passed), scripts/fresh_venv_smoke.sh and scripts/hygiene_scan.sh all pass.
• Live A/B: results below. MTP acceptance by depth on the fixed ssd_clone turns, 0.80/0.60/0.41 and 0.78/0.59/0.43, stays near the RAM controls, which ranged 0.82–0.87, 0.57–0.71 and 0.37–0.50. The model still copies from the restored context, so there is no sign of a corrupted restore. This is indirect evidence. A temperature-0 token-equality check against a RAM restore would be the strict version.

Benchmark Evidence

Hardware	MacBook Pro, Apple M5 Pro (Mac17,8), 64 GB unified memory, macOS 26.4.1 (25E253)
Model	Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed at 1d5087d
Quantization	4-bit dynamic, group 32. Embeddings, lm_head, GDN out_proj and the last 8 MLP blocks are 8-bit.
Sampler	temperature 1.0, top-p 0.95, top-k 20, thinking off, reasoning effort medium
Tokens	first turn 56,012; each later turn about 6,035 new; generation capped at 256, then 768
Profile	turbo, auto-selected; MTP depth 3, adaptive expected_value, ar_batch, batching preset agent, context 131,072
Fan mode	smart, which boosts only while a request generates. Thermal pressure was nominal on every turn.
Date	2026-10-24
Commit	baseline: installed 2.12.2 (9882703f); fixed: v2.12.2 with this PR's session_bank.py change

Same prompts and serve flags on both arms. Each arm used a private, initially empty SSD bank. The server restarted before A2 and A4, so those turns had to restore from SSD.

Turn	Path	Context	Prefill tok/s, 2.12.2 → fix	Decode tok/s, 2.12.2 → fix	TTFT, 2.12.2 → fix	Paged blocks, 2.12.2 → fix	Compiled verify, 2.12.2 → fix
A1	cold	56,044	387.7 → 393.3	21.7 → 22.9	145.4 s → 142.8 s	0 → 0	98/98 → 92/92
A2	ssd_clone	62,347	50.1 → 303.2	15.0 → 23.7	124.7 s → 23.4 s	3,953 → 0	0/276 → 231/278
A3	RAM clone	69,163	282.7 → 287.8	23.2 → 23.8	21.5 s → 21.2 s	0 → 0	241/250 → 242/257
A4	ssd_clone	75,978	41.6 → 277.6	14.3 → 22.3	150.3 s → 24.8 s	4,805 → 0	0/164 → 203/271
A5	RAM clone	82.5k / 82.8k	266.3 → 265.8	21.2 → 23.0	22.9 s → 22.9 s	0 → 0	47/47 → 119/119

The SSD read itself took 0.85–1.82 s on both arms.

Without fan control. Production logs on the same Mac show the bug under default fans. In about 50 hours of Hermes Agent use before the fix, every one of the 8 ssd_clone restores ran paged, with 5,071–7,510 blocks. Of the 587 other restores at 60k+ tokens in that window, 583 ran dense. The other four were 171k–177k-token prompts above the 131,072-token dense-decode ceiling, where paged is expected. Five of the eight ran with fan mode default:

Time	Fans	Context	New tokens	Prefill tok/s	Decode vs same-context peers	TTFT
10-21 19:43	default	97,911	39	18.7	0.56×	10.4 s
10-22 10:17	default	64,611	6,416	45.5	0.51×	143.8 s
10-23 10:45	default	92,296	44	23.0	0.51×	6.1 s
10-23 10:56	default	103,530	32	16.6	0.48×	6.6 s
10-23 14:06	default	103,635	14,114	26.8	0.51×	533.9 s
10-23 19:45	smart	67,063	5,448	45.8	0.67×	123.0 s
10-23 20:07	smart	68,101	6,486	45.9	0.63×	145.8 s
10-23 21:55	smart	92,156	63	26.0	0.47×	6.1 s

To reproduce on your own machine, stop any running MTPLX server first, then run the repro with the Python environment that has the MTPLX release installed:

<env>/bin/python benchmarks/repro_ssd_clone_restore_layout.py --dry-run
<env>/bin/python benchmarks/repro_ssd_clone_restore_layout.py --out benchmarks/results/ssd-clone-restore-layout-<machine>.json

The repro defaults to fan mode default. It takes about 25 minutes for both arms on an M5 Pro.