mudler · lu-zero · Aug 27, 2026 · Aug 26, 2026
diff --git a/.agents/benchmark-record.md b/.agents/benchmark-record.md
@@ -28752,6 +28752,62 @@ the repository: `docs/bench-evidence/tower-skip-rss-qwen3vl-thor-20260824.log`
 `docs/bench-evidence/tower-skip-rss-qwen3vl-thor-20260824.legs.log` (five
 `/usr/bin/time -v` records, four server logs, the cmake configure).
 
+## TT P150 refresh at post-W2c main: the host-hybrid opt-out OUTPERFORMS the host-free eager DEFAULT 1.24x — the #1604 flip premise inverted (#2003); Mistral-7B eager warm reproduced at 9.8 tok/s (2026-08-26, `bench/tenstorrent-p150-refresh`, P150 `thalia`)
+
+Base `21fe11cf1` (origin/main, post-BACKEND-TENSTORRENT-QWEN35 W2a–W2c,
+#1715). TT Release build against
+`/home/lu_zero/Sources/tt/tt-metal/build_Release/lib64/cmake` +
+`.../share/cmake` — **the #1604-era `-DCMAKE_PREFIX_PATH=.../build_Release`
+recipe is STALE**: after the tt-metal rebase (`a3d33028975`, still carrying our
+`copy_default_tilized` patches) the exported configs moved to install layout
+and the stale top-level `tt-metalium-config.cmake` shadows `find_package`.
+One `$HOME/gpu.lock` hold across every leg, `tt-smi -r` inside it first; an
+idle leftover `vllm-server` (Qwen3.5-0.8B b32 on :8123) held that lock for
+3h08m and was stopped before any leg. Evidence:
+[`../docs/bench-evidence/tt-p150-refresh-20260826.log`](../docs/bench-evidence/tt-p150-refresh-20260826.log)
+(verbatim per-leg stderr). Workload. Workload: greedy, batch 1,
+prompt = "Write a short story about a robot learning to paint.".
+
+**L1 Qwen3-0.6B host-free A/B (`--repeat 5`, in-process run 1 discarded;
+order-alternated pairs ×3):** default host-free eager **median 10.822 tok/s**
+(n=12, 10.51–11.03) vs `VT_TT_HOST_FREE_DECODE=0` host-hybrid
+**median 13.369** (n=12; 13.09–13.55 across 11 of 12 legs, one 11.49 outlier).
+**The opt-out wins 1.24×** — the #1604 default-rate result read the other way.
+The DEFAULT arm is unchanged against its 2026-08-21 figures (10.94–11.06),
+within what no-clock-attribution can resolve, so the movement since then is a
+~2.5× improvement of the OPT-OUT arm with mechanism UNATTRIBUTED;
+[#2003](https://github.com/mudler/vllm.cpp/issues/2003) owns it and its next
+traceable step is a per-op delta of the host-hybrid path from `b86e3705f` to
+here. No clock window was sampled for ANY figure in this entry
+(`tools/bench/gpu_clock_state.py` is NVIDIA-only); ordering was alternated to
+cancel drift, but every number below is clock-unattributed and quotable only
+as such.
+
+**L2 Mistral-7B-v0.3 default arm (`--repeat 6`, whole-process JIT leg
+discarded per process, ~128 s each cold proc): warm median 9.817 tok/s**
+(n=15, 9.46–9.94), coherent greedy prose throughout. This REPLACES the
+2026-08-12 single-run 4.26 tok/s anecdote as the row's standing figure: that
+run had a different prompt and no lock or repetition discipline, so the ~2.3×
+difference is a reproduction-class upgrade rather than an attributable delta.
+No vLLM ratio exists or can: vLLM has no Tenstorrent backend.
+Checkpoint pin: `mistralai/Mistral-7B-v0.3` snapshot
+`caa1feb0e54d415e2df31207e5f4e273e33509b1`, 3 shards totalling
+14,496,080,928 bytes (bf16 arm; correctness standing on this line is the
+refreshed golden gate green at `c31cad9c1`). Qwen checkpoint =
+the documented `docs/USAGE.md` pin `c1899de289a04d12100db370d81485cdf75e47ca`.
+
+**L3 `kGdnDecode` op microbench (op-level ONLY — production-unreached until
+the #1715 wiring row): composed 1.148 ms/step vs
+`VT_TT_GDN_DECODE=chunked` 3.356** (B=8, GQA 2:8, Dk=Dv=128, 50 steps) —
+composed stays the right default at **2.92×**, agreeing with the W2 decision
+measured 2026-08-23 (1.139 vs 3.164). Steady-state traffic h2d=0 d2h=0 on BOTH
+arms (state shadow residency holds under load). Not published to the speed-gap
+rows: not an end-to-end number.
+
+Not retested here: the captured opt-in arm (27.1 tok/s single-request on the
+old tree) — multi-request capture still hangs
+([#1625](https://github.com/mudler/vllm.cpp/issues/1625)), TT async readback
+remains [#1627](https://github.com/mudler/vllm.cpp/issues/1627).
 ## TT P150 #2003 RE-ADJUDICATED CLOCK-ATTRIBUTED: the inversion stands at governor parity — every busy sample of both arms at the 1350 MHz cap; tt_clock_state lands as the TT sibling of gpu_clock_state (#2005) (2026-08-26, `bench/tt-clock-state`, P150 `thalia`)
 
 Same binary (`21fe11cf1` bench build), workload, order-alternation, and lock

diff --git a/.agents/issue-index.md b/.agents/issue-index.md
@@ -797,3 +797,4 @@ rather than merged. `scripts/check-agent-record.py` gates both.
 | [#2042](https://github.com/mudler/vllm.cpp/issues/2042) | `SPEC-DFLASH2` | **`--enable-prefix-caching` with a DFlash2 draft kills EngineCore on the first request that takes a cache hit, at concurrency 1, and that makes the SGLang-compat `lpm` scheduler unreachable.** Measured on `3d895a202`, `sm_121a`, dgx:gpu0 under an `rc` lease, DFlash2 k=8, 1024 in / 512 out: the same binary serves 8/8 with `--no-enable-prefix-caching` and reads `ok=0 failed=8` with it on, throwing `propose_drafts_block: context position discontinuity` from inside the EngineCore step, after which every later request returns `[request submitted to a stopped AsyncLLM]`. **The invariant is the DETECTOR, not the defect**, and the three facts that settle it are: the scheduler admits a cache-hit request with `num_computed_tokens` already equal to the cached prefix (`sched/scheduler.cpp`, the waiting-admission `get_computed_blocks` arm) and the worker turns that straight into absolute positions (`prepare_inputs.cpp`, `positions[t] = num_computed_tokens_cpu[r] + query_pos[t]`); the target is served from cache and never produces the aux hidden states the draft projects, so the private store genuinely holds ZERO context rows while the target has committed N, which the second `VT_CHECK` (`L == DeviceKVNumCtx`) confirms rather than contradicts; and **upstream never reaches that state because it keeps no private store at all** — its DFlash draft writes the context K/V into the engine's own paged KV cache through `attn.impl.do_kv_cache_update(...)` (`vllm/model_executor/models/qwen3_dflash.py:601-619` at pin `5559679229`) on a slot mapping built from the TARGET's block table (`vllm/v1/spec_decode/dflash.py:145-153`), so a prefix hit hands it the draft context for free. FIXED by mirroring upstream's OTHER answer, the one for a proposer that cannot serve a request: an EMPTY draft and the target running alone (`vllm/v1/spec_decode/ngram_proposer.py:150-159`, `suffix_decoding.py:55-62`, both `continue` and neither raises), which is [#1919](https://github.com/mudler/vllm.cpp/issues/1919)'s `disabled` fallback reached from a second place. **STACKED ON [#2010](https://github.com/mudler/vllm.cpp/pull/2010) ([#2008](https://github.com/mudler/vllm.cpp/issues/2008)) AND CANNOT LAND FIRST, for correctness rather than tidiness:** the classification keys on #2010's `first_sight` predicate ("this runner has never held context for this request"), and under the pre-#2010 row-indexed state that question could not be asked, because a request the batch had MOVED presented identically to a never-seen one — so the same fallback would have swallowed #2008's crash and turned it into a silent acceptance loss. Measured, not argued: mutation M3 drops the freshness gate and reddens exactly that assertion. #2010 does NOT fix this — on its head the engine still throws the discontinuity on the second request, seven times in one run. **What it buys and costs is stated rather than implied, and it is not a free win:** prefix caching's TTFT half is kept because the target still skips the cached prefill, and a hit request stops speculating for its life, so on a shared-system-prompt workload prefix caching and DFlash2 become mutually exclusive in effect and output throughput can fall; what is unambiguously fixed is that the configuration is currently a CRASH. The repair that removes the trade is the paged context store owed under [dflash2-ctx-store-capacity.md](specs/dflash2-ctx-store-capacity.md) and tracked by #1919; a cheaper partial that keeps speculation over a TRUNCATED draft context anchored at the cache boundary is recorded under `## Owed` and deliberately not taken, because it moves draft acceptance and acceptance cannot be measured without a device. Gated by `tests/vllm/v1/spec_decode/test_dflash2_prefix_cache.cpp` (5 cases, 69 assertions, CPU, through the production `AsyncLLM` front): red-before 3/5 cases fail with the engine dead, green-after 5/5, with G3 and G5 green on both sides as controls. Wave spec [dflash2-prefix-cache.md](specs/dflash2-prefix-cache.md) | bug |
 | [#2067](https://github.com/mudler/vllm.cpp/issues/2067) | `MODEL-MM-glm5-next-glm5-next-for-conditional-generation` | **The `glm5_next` converter writes a file nothing in this tree can open: register the architecture, and give it its `general.architecture` dispatch row (O9).** W1 of [#1998](https://github.com/mudler/vllm.cpp/issues/1998); spec [`specs/glm5-next-flash.md`](specs/glm5-next-flash.md) §W1. W7a ([#2011](https://github.com/mudler/vllm.cpp/issues/2011)) authored `scripts/convert-glm5-next-gguf.py`, which emits `general.architecture = glm5next`; that key had no row in `kGgufArchArms` and `Glm5NextForConditionalGeneration` was registered by no translation unit, so both entry points refused the model by name as unrecognized and every downstream wave (W3, W5, W6, W7b) had nothing to load. **One parser, two sources.** `Glm5NextHfConfigFromGguf` reads the converter's metadata and synthesizes an HF-shaped `text_config`/`vision_config` under the *same key spellings* `config.json` uses, so a GGUF descends through the SAME `ParseGlm5NextParams` a `config.json` does — one validation surface, not two that can drift. `ParseGlm5NextParams` mirrors `Glm5NextTextConfig.__post_init__` and all five `validate_architecture` rejections at transformers **v5.16.1** (`eb4d9e2a64`, the first release carrying `glm5_next`; `v5.16.0` is 404): the `full_attention` -> `deepseek_sparse_attention` layer-kind rewrite (so `Glm5NextLayerKind` has no `kFullAttention` enumerator at all and the checkpoint's spelling is unrepresentable rather than merely unused); the `linear_attn_config` -> `linear_{head_dim,num_heads,conv_kernel_dim,lower_bound}` remap together with its `safe_gate`-defaults-True rule, and the deliberate IGNORING of that dict's `kda_layers`/`full_attn_layers` index lists, which the reference never reads; the `mlp_layer_types` default `[dense]*min(3,L) + [sparse]*(L-3)`; the `indexer_types` freq/offset schedule; and the forced `head_dim = qk_rope_head_dim`, `qk_head_dim = qk_rope_head_dim + qk_nope_head_dim` overrides. **The two validators are exact complements, and that is the structural finding.** Upstream RAISES when `qk_rope_head_dim > 0` ("Expecting NoPE for the DSA attention layers"); our `MlaBlockDims::Validate` RAISES when it is not `> 0` (`mla_attention.cpp:90-93`). No value satisfies both. W1 mirrors upstream and accepts `0`; the relaxation is W3's and is recorded as **O11**, pinned by a test so W3 cannot land the geometry without moving the pin. The HF->GGUF tensor name map is enumerated structurally per layer KIND, and the config builder uses it for one reachable, shard-safe check: a `blk.N` that carries KDA tensors while the metadata declares that layer `deepseek_sparse_attention` (or the converse) is refused, because absence proves nothing on a sharded file but a CONTRADICTION is a wrong model loading quietly. **Scope honesty.** This makes the architecture RESOLVE and its config PARSE and VALIDATE. It does NOT make the model load and it does NOT make it forward: the loader, the forward and the KV-cache spec each refuse by name, naming the missing primitive and the wave that owes it (**O10**). No token, no speed, no artifact — O1 holds unchanged, and no oracle can execute this model on any device this fleet reaches. vLLM implements `glm5_next` at NO revision, so no pin was advanced and none is owed; the sole admissible reference is transformers, and **W0's lane pin for `v5.16.1` is still unwritten** — this wave cites the revision it read without recording a pin, which stays W0's deliverable | feature |
 | [#2070](https://github.com/mudler/vllm.cpp/issues/2070) | `MODEL-MM-glm5-next-glm5-next-for-conditional-generation` | **The shared config reader synthesizes `layer_types` from `linear_attn_config.kda_layers` as ONE-INDEXED, and GLM-5.3-Flash's list is ZERO-INDEXED.** Found while implementing W1 of [#1998](https://github.com/mudler/vllm.cpp/issues/1998) ([#2067](https://github.com/mudler/vllm.cpp/issues/2067)) by a test that erased `layer_types` to check the port reproduces upstream's default schedule; it produced a schedule off by one and every assertion about which layer is KDA failed. `src/vllm/transformers_utils/hf_config.cpp` synthesizes `cfg.layer_types` from `text_config.linear_attn_config.kda_layers` when `layer_types` is absent, resolving it as `is_kda[one_indexed - 1] = true` and dropping any entry below 1 — correct for Kimi-Linear, whose upstream defines `is_kda_layer(l) := (l+1) in kda_layers`. `zai-org/GLM-5.3-Flash`'s list is ZERO-indexed, and the checkpoint settles it two ways: it contains `0`, which a one-indexed list of 45 layers cannot, and its maximum is `44` on `num_hidden_layers: 45`. Read through the one-indexed rule the `0` is dropped and everything else shifts down, so layer 2 comes out `full_attention` where the checkpoint calls it `linear_attention` — a wrong attention kind on a third of the stack, chosen silently. **Worse than an ordinary off-by-one:** the transformers reference IGNORES `kda_layers` entirely for `glm5_next`. `Glm5NextTextConfig.__post_init__` reads only `head_dim`, `num_heads`, `short_conv_kernel_size` and `gate_lower_bound` out of that dict and derives the schedule from the top-level `layer_types` or from `idx % 4 != 3`, so the shared reader would be deriving a load-bearing schedule from a list upstream never consults, under another family's indexing convention. **Not live on `main` today**, and that is the only reason this is not a shipped defect: no `glm5_next` reached `ParseHfConfig` at all until #2067 registered it, and every published `glm5_next` config carries an explicit `layer_types`, which the synthesis is guarded behind (`cfg.layer_types.empty()`). It is a trap set for the first wave to hand this model a config without one — which is what a converter, a hand-written test fixture, or a text-only variant produces. **REPAIRED IN FLOW by #2067:** `ParseGlm5NextParams` resolves `layer_types` from its own `text_config` and from upstream's `idx % 4 != 3` default, never from `cfg.layer_types`, so this model's schedule cannot be decided by a heuristic written for another family; it additionally cross-checks `kda_layers` / `full_attn_layers` against the resolved schedule AS ZERO-INDEXED and refuses on a disagreement rather than picking a winner. The shared reader's Kimi-Linear branch is left exactly as it is — it is correct for the family it was written for, and narrowing it is a change to Kimi-Linear's behaviour this row has no gate for. Pinned by `test_glm5_next_scaffold.cpp`'s `kda_layers is ZERO-indexed, and the schedule ignores it` case; mutation M5, taking `layer_types` from the shared reader again, reds it | bug |
+| [#2003](https://github.com/mudler/vllm.cpp/issues/2003) | `BACKEND-TENSTORRENT-HOST-FREE-FORWARD` | **The #1604 flip premise inverted at post-W2c `21fe11cf1`: `VT_TT_HOST_FREE_DECODE=0` (host-hybrid) outperforms the shipped eager DEFAULT 1.24x on the P150** — Qwen3-0.6B b1 greedy, order-alternated pairs ×3, in-process run 1 discarded, one `$HOME/gpu.lock` hold, `tt-smi -r` first: default median 10.822 tok/s (n=12, 10.51–11.03) vs opt-out median 13.369 (n=12; ≥13.09 on 11 of 12). The default arm is UNCHANGED against its 2026-08-21 figures (10.94–11.06 at `b86e3705f`), so what moved is a ~2.5x improvement of the opt-out arm whose mechanism is unattributed; the next traceable step is a per-op delta of the host-hybrid path from `b86e3705f` to `21fe11cf1`. Stated rather than implied: NO clock window was sampled (`tools/bench/gpu_clock_state.py` is NVIDIA-only), so every figure including the record entry that cites this issue is clock-unattributed and quotable only as such; one model shape, one board (Blackhole P150, aarch64 host, tt-metal `a3d33028975`); the captured opt-in arm was NOT retested (#1625 still blocks multi-request capture, #1627 still open). The shipped default now serves the slower of the two eager arms, which any gate using the default as denominator inherits | perf |
diff --git a/.agents/specs/tenstorrent-host-free-forward.md b/.agents/specs/tenstorrent-host-free-forward.md
@@ -326,6 +326,15 @@ investigation row but MUST be addressed by the item-5 port:
 
 ## Owed
 
+- **The default-polarity question reopened by
+  [#2003](https://github.com/mudler/vllm.cpp/issues/2003).** At post-W2c
+  `21fe11cf1` the host-hybrid opt-out outperforms the shipped eager default
+  1.24x on the P150 (Qwen3-0.6B b1, order-alternated pairs ×3, clock
+  unattributed — see `.agents/benchmark-record.md`, 2026-08-26 entry); the
+  default arm is unchanged against its #1604 figures and the opt-out improved
+  ~2.5x unattributed. Owed: a per-op delta of the host-hybrid path from
+  `b86e3705f` to current main, then a polarity decision that carries a clock
+  window per arm.
 - **No case pins `HostFreeDecodeEnabled()`'s no-caching contract on the RAC
   path ([#1688](https://github.com/mudler/vllm.cpp/issues/1688)).** The R5
   fresh review found `ReshapeAndCacheKernel` still latching the flag in a